Commit message (Collapse) | Author | Age | |
---|---|---|---|
* | Fix premature remembering of saved entity idswebstat-0.2.1 | Dan Goodliffe | 5 days |
| | | | | | | | Don't persist entity ids saved to the DB until the transaction is committed. Prevents the issue where a later DB operation fails, the transaction is rolled back, but we still think the entity has been saved. | ||
* | Fix up QuotedString/CLFString parsing | Dan Goodliffe | 5 days |
| | | | | | | | | Refactors CLFString in terms of QuotedString, but with the optional of being null (nullopt) Moves the whole decode function into QuotedString's parser, fixing support for escaping of " which would otherwise prematurely end the string in the middle. | ||
* | Add parked line import jobwebstat-0.2 | Dan Goodliffe | 8 days |
| | | | | Periodically, on idle, scan for and import previously parked lines. | ||
* | Switch to PostgreSQL's oid type for entity ids | Dan Goodliffe | 2025-09-30 |
| | | | | | oid is an "unsigned 4 byte integer", which matches our crc32 approach perfectly, and is half the storage cost of bigint. | ||
* | Write log lines to files on error | Dan Goodliffe | 2025-09-30 |
| | | | | | We call this parking, later we can reattempt ingestion after whatever caused the failure has been fixed. | ||
* | Create settings structure | Dan Goodliffe | 2025-09-24 |
| | | | | | | | Holds all the settings and their defaults for use in program_options and tests. Disables missing-field-initializers in tests because its over sensitive to structures with defaults where you only provide some values specifically. | ||
* | Write unparsable lines to the entity table | Dan Goodliffe | 2025-09-23 |
| | | | | Diagnostics and the ability to ingest later. | ||
* | Create and perform UA lookup curl op when new user agent is encountered | Dan Goodliffe | 2025-09-13 |
| | |||
* | Always pass API URL to curlGetUserAgentDetail | Dan Goodliffe | 2025-09-13 |
| | |||
* | Add test over mocked user agent fetch | Dan Goodliffe | 2025-09-09 |
| | | | | Uses a file:// URL to curl fetch a local fixture. | ||
* | Add test over Ingestor::ingestLog | Dan Goodliffe | 2025-09-08 |
| | |||
* | Use a TestIngestor as a fixture in tests | Dan Goodliffe | 2025-09-08 |
| | |||
* | Change getUserAgentDetail to return a packaged CURL object | Dan Goodliffe | 2025-09-05 |
| | | | | | This will be usable with curl_multi_* instead of being a single synchronous request. | ||
* | Add function to grab UA details from useragentstring.com | Dan Goodliffe | 2025-09-04 |
| | |||
* | Store utsname details in the host's detail field | Dan Goodliffe | 2025-09-02 |
| | |||
* | Use a DB connection pool rather than a single connection | Dan Goodliffe | 2025-09-01 |
| | |||
* | Move common test code to its own library | Dan Goodliffe | 2025-08-25 |
| | |||
* | Store log lines in full using entity ids | Dan Goodliffe | 2025-08-25 |
| | |||
* | Add DB schema and setup a mock of it in testing | Dan Goodliffe | 2025-08-25 |
| | |||
* | Pull fixture log lines into shared global vars | Dan Goodliffe | 2025-08-25 |
| | |||
* | Add helpers for hashing values extracted from log input | Dan Goodliffe | 2025-08-25 |
| | | | | | zlib's crc32 used here, the interface is a bit C and as a result a bit casty, but it'll work. | ||
* | Initial commit; basic Apache log parsing | Dan Goodliffe | 2025-08-25 |