WebStat analyser/src, branch mainWebStat web site access analyserhttp://git.randomdan.homeip.net/repo/webstat/atom?h=main2026-06-08T22:36:01ZSelect uninsertable is for update2026-06-08T22:36:01ZDan Goodliffedan@randomdan.homeip.net2026-06-08T22:36:01Zurn:sha1:856527b65e3c4807cf2bfb9d0e495d9712d56a90
This prevents other instances acting on the same rows in parallel,
potentially duplicating the rows and effort.
Remove the purge time limit2026-06-08T11:31:07ZDan Goodliffedan.goodliffe@octal.co.uk2026-06-08T11:31:07Zurn:sha1:ee9517e452376bc4963aa3a43ca90ebb9d710b85
This is a background job, has been for a while, so we can sanely allow
it to run for as long as is required.
Don't order rows when purging2026-06-08T10:27:49ZDan Goodliffedan.goodliffe@octal.co.uk2026-06-08T10:27:49Zurn:sha1:aa8f64a3e140e74a1f8d0074b606dd1f8560ad0a
The need to purge rows in request_time order is not necessary (all rows
will be deleted at some point in the process anyway) and is otherwise
extremely expensive. Removing the order by clause is a roughly 300x
speed up.
Switch to std::fstream and std::print for parkLogLines2026-05-20T16:04:30ZDan Goodliffedan.goodliffe@octal.co.uk2026-05-20T16:03:23Zurn:sha1:bca5ac479de5721e354b70d627fb9654bd94bdeeChange bytesToHexRange into a transform view variable2026-05-20T11:22:27ZDan Goodliffedan.goodliffe@octal.co.uk2026-05-20T11:22:27Zurn:sha1:161f32ebaea5988fe97ad03ad669eca2222e2a54
Simply create the transform constexpr and then apply it to any
collection with the restriction of passing it to a function.
Add index for uninsertable lines with no failure detail2026-05-18T23:53:27ZDan Goodliffedan@randomdan.homeip.net2026-05-18T23:53:27Zurn:sha1:cabd894779c2d2e453c8ed7771fdbd17c881bc2c
This index would be empty under normal conditions.
Run the retry uninsertable process batched2026-05-18T23:36:50ZDan Goodliffedan@randomdan.homeip.net2026-05-18T23:36:50Zurn:sha1:b298237a297cb70640c85c016b702a35ecafd9b9
Standard sized batch in a transaction, ordered by entity id.
Includes early exit if terminated.
Add job to retry insertion of log lines which had previously failed2026-05-18T19:43:51ZDan Goodliffedan@randomdan.homeip.net2026-05-18T19:43:51Zurn:sha1:e1a6654bd5e284842ffbc3b93bd390f3bad7a187
Entities are reparsed and reinserted, removed on success.
Failure to parse updates the entity type to UnparsableLine.
Failure to insert again updates the detail with the reason.
Extract ENTITY_IDS helper2026-05-17T16:45:53ZDan Goodliffedan@randomdan.homeip.net2026-05-17T16:45:53Zurn:sha1:e8a0fb7b9c61d5603a86b52e0f2144a91fd1d84e
Transform view for getting just the hash and record of a stored entity.
Add ThreadSafeT helper2026-05-17T15:18:08ZDan Goodliffedan@randomdan.homeip.net2026-05-17T13:32:11Zurn:sha1:7b1aeee4565fe0a2eed4a4fa8695b2a5fb671e06
Wraps a templated value and a templated mutex (defaults to shared_mutex)
and provides safe access, locked with either a shared_lock (const value)
or lock_guard (non-const value).
Applies this to existingEntities.