WebStat analyser/test, branch webstat-0.7WebStat web site access analyserhttp://git.randomdan.homeip.net/repo/webstat/atom?h=webstat-0.72026-07-06T18:44:52ZFix compilation of test-ingestor with gcc-162026-07-06T18:44:52ZDan Goodliffedan@randomdan.homeip.net2026-07-06T18:44:52Zurn:sha1:1d035cfa7b4b477c11267f15b2e6820466c004ebUpdate parsed type of duration field2026-07-05T16:12:53ZDan Goodliffedan@randomdan.homeip.net2026-07-05T16:12:53Zurn:sha1:cb47664b464331b88dad62796729fe08f07f30e4
uint32_t doesn't cut it, changed to uint64_t.
Include extra details when storing unparsable and uninsertable lines.2026-07-04T18:36:30ZDan Goodliffedan@randomdan.homeip.net2026-07-04T11:11:38Zurn:sha1:61e106fe6a15f76b73125682e91d5c17ee875be2
Now includes the hostnameId, timestamp and optionally error message.
Updates idx_entities_retryinsert to account for details not being NULL.
Also fixes the issue where the entities inserted and rollback would
still be recorded in the entity cache if the access log insert failed.
This would have lead to PK violations later on.
Replace use of CAUSEPARSEFAIL in sample strings2026-07-02T21:45:59ZDan Goodliffedan@randomdan.homeip.net2026-07-02T21:45:59Zurn:sha1:409e41d1d97b180233be778a2d23ba0b4328b5c2
Use of that as a http_verb doesn't actually cause parse failure (whether
it should or not is another matter)
So it's replaced with CAUSEINSERTFAIL in one case (it does cause this)
and in the parse failure test the whole line is ripped out.
Fixes to parseEscapedString2026-07-02T14:46:25ZDan Goodliffedan.goodliffe@octal.co.uk2026-07-02T14:46:25Zurn:sha1:fbb5038105e26e07d34184544ea4c70892acef92
Fix the warnings picked up by the latest clang-tidy.
Add test cases lifted from the bad_lines log.
Add missing support for \b in escaping.
Fix entity function sometimes returning null/wrong id2026-06-28T17:47:14ZDan Goodliffedan@randomdan.homeip.net2026-06-28T17:47:14Zurn:sha1:ad2a2ff05af8df1ba552d906e377afa345a449a7
Includes tests over the errant behaviour.
Didn't match on type and control flow could return multiple rows.
Add job to retry insertion of log lines which had previously failed2026-05-18T19:43:51ZDan Goodliffedan@randomdan.homeip.net2026-05-18T19:43:51Zurn:sha1:e1a6654bd5e284842ffbc3b93bd390f3bad7a187
Entities are reparsed and reinserted, removed on success.
Failure to parse updates the entity type to UnparsableLine.
Failure to insert again updates the detail with the reason.
Add ThreadSafeT helper2026-05-17T15:18:08ZDan Goodliffedan@randomdan.homeip.net2026-05-17T13:32:11Zurn:sha1:7b1aeee4565fe0a2eed4a4fa8695b2a5fb671e06
Wraps a templated value and a templated mutex (defaults to shared_mutex)
and provides safe access, locked with either a shared_lock (const value)
or lock_guard (non-const value).
Applies this to existingEntities.
Fix checking of background job completeness2026-05-13T14:21:06ZDan Goodliffedan.goodliffe@octal.co.uk2026-05-13T14:21:06Zurn:sha1:3e7a1ce999b2223b321573de19cef840021f77f5
OK, this is on me. future::valid does not tell you if the thread has
completed and the result is ready. It tells you if there is some state
you can get() maybe later. Here we replace those checks with a 0s wait
and a test to see if the status is ready.
The exit steps are updated to reflect this. Calling reset will invoke
the future's destructor, which blocks until the thread is joined as
needed.
This should hopefully address the issue where the main thread would
still block if it attempted to runJobAsNeeded for a job that was still
running.
Start curl operations from any thread2026-05-02T22:13:08ZDan Goodliffedan@randomdan.homeip.net2026-05-02T22:13:08Zurn:sha1:af422b45c7fb406dae35dc3f811e50d92e854f58
Ingest is now background only, so don't limit where they're started
from. Adds some unfortunate locking around the curl maps.