WebStat analyser/src/ingestor.hpp, branch webstat-0.5 WebStat web site access analyser http://git.randomdan.homeip.net/repo/webstat/atom?h=webstat-0.5 2026-05-07T14:41:19Z Handle completed curl operations in a job 2026-05-07T14:41:19Z Dan Goodliffe dan.goodliffe@octal.co.uk 2026-05-07T14:41:19Z urn:sha1:157614349f5f4c2387e0a970a4ca3ddaeaa23446 Removes the need to block the main thread from reading stdin while performing post curl operation actions, such as updating user agent details. Add support for conditional job execution 2026-05-07T14:40:11Z Dan Goodliffe dan.goodliffe@octal.co.uk 2026-05-07T14:40:11Z urn:sha1:cbb2035fd33ba84fea56c7a7223c563b925e8649 Performs a check before launching a job thread, rather than just having it exit immediately. Switch to std::map for existingEntities cache 2026-05-05T09:00:45Z Dan Goodliffe dan.goodliffe@octal.co.uk 2026-05-05T09:00:45Z urn:sha1:b7346649926b4357b27aa81238eb34cebe39671b Insertion cost of flat_map too high when map grows large. Start curl operations from any thread 2026-05-02T22:13:08Z Dan Goodliffe dan@randomdan.homeip.net 2026-05-02T22:13:08Z urn:sha1:af422b45c7fb406dae35dc3f811e50d92e854f58 Ingest is now background only, so don't limit where they're started from. Adds some unfortunate locking around the curl maps. Ingest log lines in a background thread 2026-05-02T17:58:21Z Dan Goodliffe dan@randomdan.homeip.net 2026-05-02T17:58:21Z urn:sha1:4675ab65ea5e807e0d457845a0ca84edcf1262c9 This prevents halting reading input during data insertion. Rename jobIngestParkedLines to jobReadParkedLines 2026-05-01T16:13:29Z Dan Goodliffe dan.goodliffe@octal.co.uk 2026-05-01T16:13:29Z urn:sha1:5c0206f48dc7f90009629d0a74bdc1dd6b4f67ea Actual ingest is performed by the main process, jobReadParkedLines just reads the park file and adds it to the queue. Append unparked lines to queue in finalise function 2026-05-01T16:01:56Z Dan Goodliffe dan.goodliffe@octal.co.uk 2026-05-01T15:58:01Z urn:sha1:0e768a00fab74818533458b5b40c84556b2137e0 Fixes issue where queuedLines would be accessed from background thread. Return a callable from jobs 2026-05-01T14:31:02Z Dan Goodliffe dan.goodliffe@octal.co.uk 2026-05-01T14:31:02Z urn:sha1:ed3598ac6f77f62b0c4fe32d4dab05784e5fa51d Allows safely running finalisation code in the main thread if required. Explicitly wait for and finalize any running jobs on exit 2026-05-01T10:12:39Z Dan Goodliffe dan.goodliffe@octal.co.uk 2026-05-01T10:12:39Z urn:sha1:917487a6de80bf5e81eebcdbbf48bcf9763262fa Limit the number lines stored at once 2026-05-01T09:32:48Z Dan Goodliffe dan.goodliffe@octal.co.uk 2026-05-01T09:32:48Z urn:sha1:1c929e46d8e7b14a74f80a21b5f30c9abbee410f Limits the number lines inserted per transactions, and the number of transactions before returning to reading input. Prevents long running transactions in the case when queued lines has grown in size.