Durable time-series database that's API-compatible with Prometheus.

The tests in internal/cluster run very slowly when race detection is enabled, especially since 831cd72 added matrix tests for varying cluster sizes and replication factors. The tests now around a minute to run.

Consider disabling race detection when running just these tests, which will reduce the test run time significantly.

Build on the work done in #54 to partition and replicate ingested data samples to peers in the cluster using the consistent hashing algorithm chosen in #27 and the partition key schema chosen in #12.

When a node in the cluster has corrupted or missing data, it must be able to receive a known good copy of data from another node.

The aim of this issue is to determine what the interface for sending/receiving data for the purpose of repair should look like.

The primary requirement for such an interface is that it must be efficient; the CPU and memory cost of serialising and deserialising the data as well as the size of data sent over the wire must be kept to a minimum.

Command-line flags are self-documenting via the flag.Usage() function, so use those instead of environment variables for configuration.

I may also allow the flag values to be set using environment variables.

Probably worth evaluating flag libraries other than the one from Go's standard library as it could do this for me.

The Prometheus 'remote read' API should be supported to improve operability with Prometheus, allowing AthensDB to be used a storage backend for Prometheus.

Use the MkDocs static site generator to add stub documentation.

Uses the Material theme for MkDocs because:

• it has a clean design
• the search functionality is very slick
• it has a responsive layout; works on phones/tablets
• it supports box-out notes and warnings
• it supports Markdown

https://github.com/squidfunk/mkdocs-material

To see a documentation in a development environment, use:

make servedocs

Include building the documentation in strict mode as part of the test make target to ensure that we have not broken the docs when making changes. Add the generated site directory to .gitignore.

Add Docker as a dependency for Travis tests so that we can use Docker to test the documentation builds correctly.

Pin the version of the squidfunk/mkdocs-material Docker image to ensure the build is reproducible.

Options: https://github.com/prometheus/prombench/blob/master/apps/load-generator/main.py https://github.com/prometheus/tsdb/blob/master/testdata/20k.series

Add basic acceptance tests for the Prometheus v1 API, i.e. for simple queries. As opposed to the API unit tests already in master, these tests should test the server as a 'blackbox'; they should start the server and query the API endpoints as a normal client would do.

Potential options:

• use the webdriver protocol (e.g. Selenium, PhantomJS), which will work for acceptance-testing the API and also for testing any future web UI (e.g. for viewing the status of the cluster)
• use Go's net/http library

Vendor the current project dependencies so that we can have reproducible builds and add a Makefile so that future dependencies can be vendored easily.

• Rename fanoutQuerier to remoteQuerier, which is more descriptive of its role

• Only pass a URL to remoteQuerier to improve encapsulation

• Rename Read() to remoteRead() and make it a method of remoteQuerier since most of its arguments are fields belonging to remoteQuerier d4e2a2e

• Remove duplicated error handling

Update to the latest version of the Prometheus libraries and make necessary changes to satisfy API changes:

• Use NewMergeSeriesSet() instead of DeduplicateSeriesSet(), which was removed in prometheus/prometheus@bb724f1.

• Explicitly specify the options for the PromQL engine to ensure that PromQL metrics are registered following the change in prometheus/prometheus@f8fccc7. The options used here are copied from the defaults.

Prometheus vendors its own dependencies but also provides library packages, meaning that we have to manage its dependencies as transient dependencies. Using the dep tool, we cannot manage transient dependencies using regular constraints so I had to pin the versions in Gopkg.toml using overrides.

See: https://github.com/golang/dep/blob/master/docs/FAQ.md#how-do-i-constrain-a-transitive-dependencys-version golang/dep#999 (comment) golang/dep#302

The current partition key schema, in addition to the lack of any centralised index, requires that queries must be run against all nodes in the cluster.

The current schema can be represented as:

<salt>:<bucket_end_time_as_YYYYMMDD>:<metric_name>:[<label_name>,<label_name>...]

Since the label names are often not known at query time, and PromQL allows querying without the metric name, the current schema means that all nodes must be queried in order to ensure that all matches time-series are retrieved.

The partition key schema should be improved to limit the number of nodes that must be queried. There is an inherent tension between limiting the number of nodes that need to be queried and balancing ingestion across as many nodes in the cluster as possible.

Using Jumphash, nodes are represented by buckets.

In order to minimise the amount of data that moves when nodes are added or removed, nodes must remember which bucket they belong to and use that same bucket when they reconnect to the cluster.

Found during the implementation of #149.

This limitation is mentioned in the abstract for Jumphash:

Its main limitation is that the buckets must be numbered sequentially, which makes it more suitable for data storage applications than for distributed web caching.

...the consequences of which were not clear to me when deciding on #27.

The API endpoint to delete series is disabled currently: https://github.com/mattbostock/timbala/blob/8a2f093838778f5e84d1355970ed339eea918c17/internal/api/v1/api.go#L346-L347

Determine whether deleting series is something Timbala should support; doing so has implications for consistency in a distributed system.