summaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAge
* Attempt to save uninsertable log lines to the entities tableDan Goodliffe2026-01-17
| | | | | | If that fails, we still park them as before, such as when the DB is unavailable. Those which are saved as entities require investigation why they couldn't be saved, much like UnparsableLines.
* pg_format schema.sql and sql/*.sqlDan Goodliffe2026-01-17
| | | | No changes.
* Add job for puring old access log entries from the databaseDan Goodliffe2025-12-20
|
* Add a few no lint commentsDan Goodliffe2025-12-20
|
* Add support for configuring frequency of parked line jobDan Goodliffe2025-12-20
|
* Add utility for parsing an ISO like durationDan Goodliffe2025-12-20
|
* Replace that awful magic number heavy mapping functionDan Goodliffe2025-10-16
| | | | | Now a tuple of mapping functors and we pass each value through its corresponding converter.
* Refactor handling of new entity insertDan Goodliffe2025-10-15
| | | | | Replaces weird select with one thing with a function pointer stored in the type definition array.
* Update comments on custom_log formatDan Goodliffe2025-10-15
|
* Add access_log_viewDan Goodliffe2025-10-15
| | | | | A pre-joined with entities view showing all the original data along with ids; ideal for human readable stuff.
* Allows handle curl things if there are anywebstat-0.2.2Dan Goodliffe2025-10-10
|
* Fix premature remembering of saved entity idswebstat-0.2.1Dan Goodliffe2025-10-09
| | | | | | | Don't persist entity ids saved to the DB until the transaction is committed. Prevents the issue where a later DB operation fails, the transaction is rolled back, but we still think the entity has been saved.
* Fix up QuotedString/CLFString parsingDan Goodliffe2025-10-09
| | | | | | | | Refactors CLFString in terms of QuotedString, but with the optional of being null (nullopt) Moves the whole decode function into QuotedString's parser, fixing support for escaping of " which would otherwise prematurely end the string in the middle.
* Add PROPFIND to http_verb listDan Goodliffe2025-10-09
|
* Add parked line import jobwebstat-0.2Dan Goodliffe2025-10-06
| | | | Periodically, on idle, scan for and import previously parked lines.
* Add point to execute scheduled jobs when idleDan Goodliffe2025-10-02
|
* Switch to PostgreSQL's oid type for entity idsDan Goodliffe2025-09-30
| | | | | oid is an "unsigned 4 byte integer", which matches our crc32 approach perfectly, and is half the storage cost of bigint.
* Write log lines to files on errorDan Goodliffe2025-09-30
| | | | | We call this parking, later we can reattempt ingestion after whatever caused the failure has been fixed.
* Create settings structureDan Goodliffe2025-09-24
| | | | | | | Holds all the settings and their defaults for use in program_options and tests. Disables missing-field-initializers in tests because its over sensitive to structures with defaults where you only provide some values specifically.
* Write unparsable lines to the entity tableDan Goodliffe2025-09-23
| | | | Diagnostics and the ability to ingest later.
* Make DB pool protected for access from unit testsDan Goodliffe2025-09-23
|
* Gracefully handle fatal exceptions and display messageDan Goodliffe2025-09-19
|
* Create and perform UA lookup curl op when new user agent is encounteredDan Goodliffe2025-09-13
|
* Perform background curl operations when not processing log inputDan Goodliffe2025-09-13
|
* Always pass API URL to curlGetUserAgentDetailDan Goodliffe2025-09-13
|
* Simplify storeEntities with bindManyDan Goodliffe2025-09-10
|
* Use curl_multi_poll in main ingestLog loopDan Goodliffe2025-09-10
| | | | Preparation step for having background curl operations.
* Change getUserAgentDetail to return a packaged CURL objectDan Goodliffe2025-09-05
| | | | | This will be usable with curl_multi_* instead of being a single synchronous request.
* Replace DeleteWith lambda with a normal typeDan Goodliffe2025-09-05
| | | | Lambdas have internal linkage.
* Add function to grab UA details from useragentstring.comDan Goodliffe2025-09-04
|
* Add libcurlDan Goodliffe2025-09-04
|
* Move DeleteWith helper in src/utilDan Goodliffe2025-09-04
|
* Store utsname details in the host's detail fieldDan Goodliffe2025-09-02
|
* Add bindMany helperDan Goodliffe2025-09-02
|
* Add entity detail columnDan Goodliffe2025-09-02
|
* Tag entities with their typewebstat-0.1.1Dan Goodliffe2025-09-01
|
* Reuse same insert entity commandDan Goodliffe2025-09-01
|
* Support configuring the DB poolDan Goodliffe2025-09-01
|
* Use a DB connection pool rather than a single connectionDan Goodliffe2025-09-01
|
* Add basic boost::program_options supportDan Goodliffe2025-09-01
|
* Use SQL standard identity columnDan Goodliffe2025-08-25
|
* Insert new entities and log entry in a transactionDan Goodliffe2025-08-25
| | | | If no new entities are required, no transaction is created.
* Merge storeEntities into a single thingDan Goodliffe2025-08-25
| | | | Add visitSum for tuples and makes storeEntities into a single lambda.
* Track existing entities, don't attempt reinsertDan Goodliffe2025-08-25
| | | | | This is an ever growing set, but in theory should be finite, and is only 4bytes per entity.
* Return count of entities passed to the databaseDan Goodliffe2025-08-25
|
* Store log lines in full using entity idsDan Goodliffe2025-08-25
|
* Store the entities for each log lineDan Goodliffe2025-08-25
|
* Provide hostname to IngestorDan Goodliffe2025-08-25
| | | | Store its Entity immediately for later use.
* Add helpers and SQL for storing an entityDan Goodliffe2025-08-25
|
* Pass a DB connection to IngestorDan Goodliffe2025-08-25
|
generated by cgit v1.2.3 (git 2.46.0) at 2026-02-04 06:30:27 +0000