WebStat analyser/src/ingestor.hpp, branch webstat-0.3.1WebStat web site access analyserhttps://git.randomdan.homeip.net/repo/webstat/atom?h=webstat-0.3.12026-03-25T18:44:15ZRevise stats and add signal handlers to log them and reset them2026-03-25T18:44:15ZDan Goodliffedan@randomdan.homeip.net2026-03-25T18:44:15Zurn:sha1:426a6a3d9309e3f7cf5fc89d6a687c7895785cc3
Also logs them on main loop exit.
Employ temporary/short files to handle errors reading/writing park logs2026-03-22T23:41:56ZDan Goodliffedan@randomdan.homeip.net2026-03-22T23:41:56Zurn:sha1:6a475f58c0fcb4398e48618d90b0ca05750c3ca8Add missing -Wshadow2026-03-22T18:03:14ZDan Goodliffedan@randomdan.homeip.net2026-03-22T17:02:40Zurn:sha1:d92816a5f55cea9b67b7aab306eb1109af6dcbbeAdd logging :-o2026-03-20T23:48:32ZDan Goodliffedan@randomdan.homeip.net2026-03-20T23:48:32Zurn:sha1:05c47ab65e73b16887b7c7a1eb31acf6d364ef41
Adds virtual log function, real implementation writes to syslog.
Test implementation writes to BOOST_TEST_MESSAGE, perf implementation
discards.
Replaces existing prints to stderr and adds logs to all key points.
Insert log entries in batches2026-03-20T02:23:08ZDan Goodliffedan@randomdan.homeip.net2026-03-20T02:17:04Zurn:sha1:0f5a0a8e2d43774288d4d6ea747278ca6e085a2a
Store log lines in memory until threshold is reach or idle occurs, then
insert all the lines in a single transaction. Save points handle the
case of insertion errors. On success the queue is cleared.
Parked lines also saved in bulk, only necessary if queued lines could
not be inserted on shutdown, else the queue simply grows until ability
to insert is restored. Importing parked lines just adds them to the
queue and the normal process then follows.
Gracefully handle SIGTERM2026-03-19T12:33:48ZDan Goodliffedan.goodliffe@octal.co.uk2026-03-19T12:33:48Zurn:sha1:8c6fecd356003309f8eebec30374344272ca6072
Apache sends SIGTERM to the logger process to it shutdown. Honestly I
thought it would just close stdin and I should have checked.
Use std::future over std::thread for background jobs2026-03-17T17:33:14ZDan Goodliffedan.goodliffe@octal.co.uk2026-03-17T17:33:14Zurn:sha1:ae0ee6bbfb4dcd1f112876d0e7e5b3bcabdfb002
Easier checking if a job has completed [successfully] and reseting state
for the next time.
Don't start new curl operations outside the main thread2026-03-17T15:24:46ZDan Goodliffedan.goodliffe@octal.co.uk2026-03-17T15:24:46Zurn:sha1:fc96582867acd2bcaef29feaae5c1193969d9a27
Neither the curl handle, not the operation map is thread safe. This
isn't ideal, but it does solve the problem in a safe manor.
Execute jobs even when processing incoming logs2026-03-17T11:48:34ZDan Goodliffedan.goodliffe@octal.co.uk2026-03-17T11:48:34Zurn:sha1:d2ecb7f49c3f3b60e3d1f297fd033071a02cfa9a
Jobs run on background threads now, so we can happily run them even when
we're busy.
Run jobs on a background thread2026-03-17T11:11:14ZDan Goodliffedan.goodliffe@octal.co.uk2026-03-17T11:11:14Zurn:sha1:0e3c2e8dd462a7d56f8b61e15c8cf10681898a1f