WebStat analyser/test/test-ingest.cpp, branch webstat-0.5WebStat web site access analyserhttp://git.randomdan.homeip.net/repo/webstat/atom?h=webstat-0.52026-05-02T17:58:21ZIngest log lines in a background thread2026-05-02T17:58:21ZDan Goodliffedan@randomdan.homeip.net2026-05-02T17:58:21Zurn:sha1:4675ab65ea5e807e0d457845a0ca84edcf1262c9
This prevents halting reading input during data insertion.
Rename jobIngestParkedLines to jobReadParkedLines2026-05-01T16:13:29ZDan Goodliffedan.goodliffe@octal.co.uk2026-05-01T16:13:29Zurn:sha1:5c0206f48dc7f90009629d0a74bdc1dd6b4f67ea
Actual ingest is performed by the main process, jobReadParkedLines just
reads the park file and adds it to the queue.
Append unparked lines to queue in finalise function2026-05-01T16:01:56ZDan Goodliffedan.goodliffe@octal.co.uk2026-05-01T15:58:01Zurn:sha1:0e768a00fab74818533458b5b40c84556b2137e0
Fixes issue where queuedLines would be accessed from background thread.
Return a callable from jobs2026-05-01T14:31:02ZDan Goodliffedan.goodliffe@octal.co.uk2026-05-01T14:31:02Zurn:sha1:ed3598ac6f77f62b0c4fe32d4dab05784e5fa51d
Allows safely running finalisation code in the main thread if required.
Replace use of crc32 for entity id2026-04-15T11:05:20ZDan Goodliffedan.goodliffe@octal.co.uk2026-04-15T11:03:21Zurn:sha1:5b2166496e5f3ff2c4276e0b5b28f109c70673d5
Entity value is MD5 hashed same as DB unique key, but the id itself is
now taken from the DB primary key which is sequence generated.
Return path of parked lines log file from parkQueuedLogLines2026-04-10T19:22:35ZDan Goodliffedan@randomdan.homeip.net2026-04-10T19:22:35Zurn:sha1:72bfa9dd305258789b0d2e80f8af13962e5aac42
Or the last errno on failure.
Parse escaping in query strings2026-03-27T11:43:18ZDan Goodliffedan.goodliffe@octal.co.uk2026-03-27T11:43:18Zurn:sha1:1e551e618a63c869fde6a4b327566b38696a5f45Revise stats and add signal handlers to log them and reset them2026-03-25T18:44:15ZDan Goodliffedan@randomdan.homeip.net2026-03-25T18:44:15Zurn:sha1:426a6a3d9309e3f7cf5fc89d6a687c7895785cc3
Also logs them on main loop exit.
Add logging :-o2026-03-20T23:48:32ZDan Goodliffedan@randomdan.homeip.net2026-03-20T23:48:32Zurn:sha1:05c47ab65e73b16887b7c7a1eb31acf6d364ef41
Adds virtual log function, real implementation writes to syslog.
Test implementation writes to BOOST_TEST_MESSAGE, perf implementation
discards.
Replaces existing prints to stderr and adds logs to all key points.
Insert log entries in batches2026-03-20T02:23:08ZDan Goodliffedan@randomdan.homeip.net2026-03-20T02:17:04Zurn:sha1:0f5a0a8e2d43774288d4d6ea747278ca6e085a2a
Store log lines in memory until threshold is reach or idle occurs, then
insert all the lines in a single transaction. Save points handle the
case of insertion errors. On success the queue is cleared.
Parked lines also saved in bulk, only necessary if queued lines could
not be inserted on shutdown, else the queue simply grows until ability
to insert is restored. Importing parked lines just adds them to the
queue and the normal process then follows.