xyr [WIP]
xyr
is a very lightweight, simple and powerful data ETL platform that helps you to query available data sources usingSQL
.
Supported Drivers
-
local+json
: for extracting, transforming and loading json documents from local filesystem intoxyr
. -
local+csv
: for extracting, transforming and loading csv documents from local filesystem intoxyr
. -
s3+json
: for extracting, transforming and loading json documents fromS3
. -
s3+csv
: for extracting, transforming and loading csv documents fromS3
. -
postgresql
: for extracting, transforming and loading postgres results. -
clickhouse
: for extracting, transforming and loading clickhouse results. -
redis
: for extracting, transforming and loading redis datastructures.
Use Cases
- Simple Presto Alternative.
- Simple AWS Athena Alternative.
- Convert your json documents into a SQL db.
- Query your CSV files easily and join them with other data.
How it works?
internaly
xyr
utilizesSQLite
as an embeded sql datastore (it may be changed in future and we can add multiple data stores), when you define a table inXYRCONFIG
file then run$ xyr import
you will be able to import all defined tables as well querying them via$ xyr exec "SELECT * FROM TABLE_NAME_HERE"
.
Plan
- Building the initial core.
- Add the basic
import
command for importing the tables intoxyr
. - Add the
exec
command to execute SQL query. - Expose another API beside the
CLI
to enable external Apps to queryxyr
.- JSON Endpoint?
- Mysql Protocol?
- Redis Protocol?
- Improving the code base (iteration 1).