summaryrefslogtreecommitdiff
path: root/src/schema.sql
Commit message (Collapse)AuthorAge
* Insert uninsirtable lines using the hostnameId recorded in the bad_lines tableDan Goodliffe7 days
| | | | | | | Includes the same fix from the previous commit which stops caching entities until the inserts are complete. Updates storeLogLine to take an insert command with the hostnameId already bound.
* Include extra details when storing unparsable and uninsertable lines.Dan Goodliffe8 days
| | | | | | | | Now includes the hostnameId, timestamp and optionally error message. Updates idx_entities_retryinsert to account for details not being NULL. Also fixes the issue where the entities inserted and rollback would still be recorded in the entity cache if the access log insert failed. This would have lead to PK violations later on.
* Partition the entities tableDan Goodliffe12 days
| | | | | | Splits up the entities by type, updates the access_log foreign keys to reference the specific table, removes now redundant indices which existed only for processing removal of bad lines.
* Fix uniqueness of entitiesDan Goodliffe13 days
| | | | | Should be unique for value and type, they're not just strings, they can be different things with the same value... Albeit rarely.
* Fix entity function sometimes returning null/wrong idDan Goodliffe14 days
| | | | | Includes tests over the errant behaviour. Didn't match on type and control flow could return multiple rows.
* Increase size field to 64bit and fix scan typesDan Goodliffe2026-06-27
| | | | | 2^31 not big enough for large file downloads, bumped to 64bit. Replace woolly types with specific uintXX_t in scn call.
* Remove the PK id column from access_logDan Goodliffe2026-06-27
| | | | | Not actually required, should we need to reference a specific row, we can use the ctid column.
* Add index for uninsertable lines with no failure detailDan Goodliffe2026-05-19
| | | | This index would be empty under normal conditions.
* Be specific about values from entities tableDan Goodliffe2026-05-10
| | | | Fixes compatibility with PostgreSQL 18 which fails due to ambiguity.
* Only call entity insert handler if detail is nullDan Goodliffe2026-05-05
| | | | | | | Improves handling of entity inserts where the entity already exists and already has detail; does not call the onInsert handler. This avoids repeatedly fetching UA detail every time the UA is first seen by a process.
* Add indexes on all entity references in access_logDan Goodliffe2026-04-26
|
* Add BRIN index to access_log.request_time and improve purgeDan Goodliffe2026-04-23
| | | | Purge is now fully request_time based and not hacked around id ranges.
* Swap int for integer in schemaDan Goodliffe2026-04-18
| | | | Plays better with apgdiff
* Replace use of crc32 for entity idDan Goodliffe2026-04-15
| | | | | Entity value is MD5 hashed same as DB unique key, but the id itself is now taken from the DB primary key which is sequence generated.
* Replace unique constraint on entity value with index on hashDan Goodliffe2026-03-18
| | | | | | | UNIQUE CONSTRAINT is limited to 2704 bytes, which prevents inserting large values. Here we swap to a unique index on the MD5 hash of the value. This should more than suffice given we already map to a 32bit for the id and the index size is much much smaller.
* Fix typo in access_log_view definitionDan Goodliffe2026-03-18
| | | | Replaces accidentally duplicated user_agent for correct content_type.
* Process new field, content-type, in input streamDan Goodliffe2026-01-18
|
* Attempt to save uninsertable log lines to the entities tableDan Goodliffe2026-01-17
| | | | | | If that fails, we still park them as before, such as when the DB is unavailable. Those which are saved as entities require investigation why they couldn't be saved, much like UnparsableLines.
* pg_format schema.sql and sql/*.sqlDan Goodliffe2026-01-17
| | | | No changes.
* Add access_log_viewDan Goodliffe2025-10-15
| | | | | A pre-joined with entities view showing all the original data along with ids; ideal for human readable stuff.
* Add PROPFIND to http_verb listDan Goodliffe2025-10-09
|
* Switch to PostgreSQL's oid type for entity idsDan Goodliffe2025-09-30
| | | | | oid is an "unsigned 4 byte integer", which matches our crc32 approach perfectly, and is half the storage cost of bigint.
* Write unparsable lines to the entity tableDan Goodliffe2025-09-23
| | | | Diagnostics and the ability to ingest later.
* Add entity detail columnDan Goodliffe2025-09-02
|
* Tag entities with their typewebstat-0.1.1Dan Goodliffe2025-09-01
|
* Use SQL standard identity columnDan Goodliffe2025-08-25
|
* Add DB schema and setup a mock of it in testingDan Goodliffe2025-08-25
generated by cgit v1.3 (git 2.53.0) at 2026-07-12 07:04:28 +0000