parsefields
Tool for parse JSON-like logs for collecting unique fields. Main purpose to collect JSON-data with typical events and fields it is useful when you want to create mapping schema for database and you want to reduce the risks of forgotten fields. By default separator between to nested structs is "->", but you can change it with environment.
API consist:
- POST /v1/json/
- POST /v1/mjson/
- GET /v1/fileds/
- GET /v1/events/
- GET /v1/events/:logname/:eventid
- DELETE /v1/events/:logname/:eventid
- DELETE /v1/fields/:field
P.S. additionally info about all new events/fields will be show in stdout.
Deploy
docker build . -t parsefield
docker run -d -p 8000:8000 parsefield
or
docker-compose -p 8000:8000 -d up
Usage
Push new log for parse
Single message per request
curl -X POST -d '{"process_name": "calc.exe", "process_path":"C:\\windows\\system32"}' 127.0.0.1:8000/v1/json/
Multiple message per request
curl -X POST -d '[{"process_name": "calc.exe", "process_path":"C:\\windows\\system32"},{"process_image": "calc.exe", "process_path":"C:\\windows\\system32"},{"pid":"1"}]' 127.0.0.1:8000/v1/mjson/
All unique fields
curl 127.0.0.1:8000/v1/fields/
All unique events
curl 127.0.0.1:8000/v1/events/
Show body of event
curl 127.0.0.1:8000/v1/events/Sysmon/999
Delete events, fields
curl -X DELETE 127.0.0.1:8000/v1/events/Sysmon/999 - delete events with logname Sysmon and eventId 999
curl -X DELETE 127.0.0.1:8000/v1/fields/key - delete field with name key
