summaryrefslogtreecommitdiff
path: root/src/sql
Commit message (Collapse)AuthorAge
* Include extra details when storing unparsable and uninsertable lines.Dan Goodliffe8 days
| | | | | | | | Now includes the hostnameId, timestamp and optionally error message. Updates idx_entities_retryinsert to account for details not being NULL. Also fixes the issue where the entities inserted and rollback would still be recorded in the entity cache if the access log insert failed. This would have lead to PK violations later on.
* Fix uniqueness of entitiesDan Goodliffe13 days
| | | | | Should be unique for value and type, they're not just strings, they can be different things with the same value... Albeit rarely.
* Select uninsertable is for updatewebstat-0.6Dan Goodliffe2026-06-08
| | | | | This prevents other instances acting on the same rows in parallel, potentially duplicating the rows and effort.
* Don't order rows when purgingDan Goodliffe2026-06-08
| | | | | | | The need to purge rows in request_time order is not necessary (all rows will be deleted at some point in the process anyway) and is otherwise extremely expensive. Removing the order by clause is a roughly 300x speed up.
* Run the retry uninsertable process batchedDan Goodliffe2026-05-19
| | | | | Standard sized batch in a transaction, ordered by entity id. Includes early exit if terminated.
* Add job to retry insertion of log lines which had previously failedDan Goodliffe2026-05-18
| | | | | | Entities are reparsed and reinserted, removed on success. Failure to parse updates the entity type to UnparsableLine. Failure to insert again updates the detail with the reason.
* Only call entity insert handler if detail is nullDan Goodliffe2026-05-05
| | | | | | | Improves handling of entity inserts where the entity already exists and already has detail; does not call the onInsert handler. This avoids repeatedly fetching UA detail every time the UA is first seen by a process.
* Add BRIN index to access_log.request_time and improve purgeDan Goodliffe2026-04-23
| | | | Purge is now fully request_time based and not hacked around id ranges.
* Replace use of crc32 for entity idDan Goodliffe2026-04-15
| | | | | Entity value is MD5 hashed same as DB unique key, but the id itself is now taken from the DB primary key which is sequence generated.
* Process new field, content-type, in input streamDan Goodliffe2026-01-18
|
* pg_format schema.sql and sql/*.sqlDan Goodliffe2026-01-17
| | | | No changes.
* Add job for puring old access log entries from the databaseDan Goodliffe2025-12-20
|
* Create and perform UA lookup curl op when new user agent is encounteredDan Goodliffe2025-09-13
|
* Store utsname details in the host's detail fieldDan Goodliffe2025-09-02
|
* Tag entities with their typewebstat-0.1.1Dan Goodliffe2025-09-01
|
* Store log lines in full using entity idsDan Goodliffe2025-08-25
|
* Add helpers and SQL for storing an entityDan Goodliffe2025-08-25