| Commit message (Collapse) | Author | Age | |
|---|---|---|---|
| * | Process new field, content-type, in input streamHEADmain | Dan Goodliffe | 2026-01-18 |
| | | |||
| * | Attempt to save uninsertable log lines to the entities table | Dan Goodliffe | 2026-01-17 |
| | | | | | | | If that fails, we still park them as before, such as when the DB is unavailable. Those which are saved as entities require investigation why they couldn't be saved, much like UnparsableLines. | ||
| * | Add job for puring old access log entries from the database | Dan Goodliffe | 2025-12-20 |
| | | |||
| * | Add utility for parsing an ISO like duration | Dan Goodliffe | 2025-12-20 |
| | | |||
| * | Rename test utilities to avoid name conflict. | Dan Goodliffe | 2025-12-20 |
| | | |||
| * | Fix premature remembering of saved entity idswebstat-0.2.1 | Dan Goodliffe | 2025-10-09 |
| | | | | | | | | Don't persist entity ids saved to the DB until the transaction is committed. Prevents the issue where a later DB operation fails, the transaction is rolled back, but we still think the entity has been saved. | ||
| * | Fix up QuotedString/CLFString parsing | Dan Goodliffe | 2025-10-09 |
| | | | | | | | | | Refactors CLFString in terms of QuotedString, but with the optional of being null (nullopt) Moves the whole decode function into QuotedString's parser, fixing support for escaping of " which would otherwise prematurely end the string in the middle. | ||
| * | Add parked line import jobwebstat-0.2 | Dan Goodliffe | 2025-10-06 |
| | | | | | Periodically, on idle, scan for and import previously parked lines. | ||
| * | Switch to PostgreSQL's oid type for entity ids | Dan Goodliffe | 2025-09-30 |
| | | | | | | oid is an "unsigned 4 byte integer", which matches our crc32 approach perfectly, and is half the storage cost of bigint. | ||
| * | Write log lines to files on error | Dan Goodliffe | 2025-09-30 |
| | | | | | | We call this parking, later we can reattempt ingestion after whatever caused the failure has been fixed. | ||
| * | Create settings structure | Dan Goodliffe | 2025-09-24 |
| | | | | | | | | Holds all the settings and their defaults for use in program_options and tests. Disables missing-field-initializers in tests because its over sensitive to structures with defaults where you only provide some values specifically. | ||
| * | Write unparsable lines to the entity table | Dan Goodliffe | 2025-09-23 |
| | | | | | Diagnostics and the ability to ingest later. | ||
| * | Create and perform UA lookup curl op when new user agent is encountered | Dan Goodliffe | 2025-09-13 |
| | | |||
| * | Always pass API URL to curlGetUserAgentDetail | Dan Goodliffe | 2025-09-13 |
| | | |||
| * | Add test over mocked user agent fetch | Dan Goodliffe | 2025-09-09 |
| | | | | | Uses a file:// URL to curl fetch a local fixture. | ||
| * | Add test over Ingestor::ingestLog | Dan Goodliffe | 2025-09-08 |
| | | |||
| * | Use a TestIngestor as a fixture in tests | Dan Goodliffe | 2025-09-08 |
| | | |||
| * | Move fixture log file generator to test util | Dan Goodliffe | 2025-09-08 |
| | | |||
| * | Change getUserAgentDetail to return a packaged CURL object | Dan Goodliffe | 2025-09-05 |
| | | | | | | This will be usable with curl_multi_* instead of being a single synchronous request. | ||
| * | Add function to grab UA details from useragentstring.com | Dan Goodliffe | 2025-09-04 |
| | | |||
| * | Move DeleteWith helper in src/util | Dan Goodliffe | 2025-09-04 |
| | | |||
| * | Store utsname details in the host's detail field | Dan Goodliffe | 2025-09-02 |
| | | |||
| * | Use a DB connection pool rather than a single connection | Dan Goodliffe | 2025-09-01 |
| | | |||
| * | Add MockDBPool | Dan Goodliffe | 2025-09-01 |
| | | | | | Feels like this should be provided by libdbpp, but it isn't. | ||
| * | Add an ingestor performance test | Dan Goodliffe | 2025-08-25 |
| | | | | | Generates a random long file of random data and then ingests it. | ||
| * | Move common test code to its own library | Dan Goodliffe | 2025-08-25 |
| | | |||
| * | Store log lines in full using entity ids | Dan Goodliffe | 2025-08-25 |
| | | |||
| * | Add DB schema and setup a mock of it in testing | Dan Goodliffe | 2025-08-25 |
| | | |||
| * | Pull fixture log lines into shared global vars | Dan Goodliffe | 2025-08-25 |
| | | |||
| * | Add helpers for hashing values extracted from log input | Dan Goodliffe | 2025-08-25 |
| | | | | | | zlib's crc32 used here, the interface is a bit C and as a result a bit casty, but it'll work. | ||
| * | Initial commit; basic Apache log parsing | Dan Goodliffe | 2025-08-25 |
