Modular Kubernetes operator to manage the lifecycle of databases

This PR introduces a big refactor change for the state. Before, K8s api server was the state where Ensemble would store internal information like for example Nodes in a cluster. However, it was a bit limiting in terms of how complex the state could be. Now, the Operator uses a boltdb file to store the state and it is the main interface that the Operator uses to reconcile. Then, the K8s provider updates that state with changes in the CRDs.

Before, the operator would need both the deployment id and the instance id to get the reference of the instance. Then, there was always the need to keep both values together available at the same time on the different async ops (i.e. Provider). Now, it only needs the id to get the reference of the instance.

The K8s provider was updating the instance every time there was a Pod event update but we only need to do it for state changes that might create a reconciliation. This PR filters the watch events and only updates when the state changes either to Running or Stopped.

This PR does two things:

• Closes #72 and only uses one state to represent that the instance is not running anymore.
• Uses a Pod watcher instead of an Event watcher to track pod changes.

This PR introduces the control plane interface for the Operator. Both Providers like Kubernetes/Docker and backends (databases) use the interface to listen for instance updates and act accordingly. Thus, we can safely remove the synchronous Create/Delete Resource methods in the Provider which were not that flexible.

This PR replaces the current hardcoded readiness probe with a generic readiness probe using hooks. This is the first iteration of the hooks controller. Later on we will expand the concept to include not just post-hooks like readiness but also pre-hooks before removing a node.

This PR does two things:

• Moves some of the logic to schedule clusters to his own method Scheduler.
• Uses the new scheduler to test the backends without a running docker instance. Closes #40.

Right now Ensemble uses boltdb as a backend. It is nice because it is embedded and does not require any external service to run it. However, it is hard to build complex database layouts on top without reinventing indexes every time.

This issue proposes to replace boltdb with sqlite3 which is also embedded and it gives us more flexibility since it uses SQL. I think the only problem is that it requires cgo to run.

With the new scheduler we can generate Plans for the specs without executing them. We can use that feature to include a plan Grpc and cli endpoint to retrieve information and annotations about the execution plan.

There are some use cases that require generic pre/post hook support:

• Redis "join" command to create the cluster.
• Elasticsearch drain the nodes during scale down.
• Clickhouse replica distributed tables during scale up.

Even the generic concept of Readiness (after the start) could be modeled as a post hook. A hook should be a one-off operation. Long running jobs like monitoring are not applied here.

In #45 we included support for a default value in the schema. Right now, the default value is used only for the default params, we should apply the default value to all the schema config items (resources, storage, node and database config).

There are two types of CRD specs that belong to a deployment: cluster and resource. We should be able to return for a specific deployment his specs alongside information of their state (completed, pending, canceled...).

This change requires the following changes:

• State: We need to add an index to keep track an index of spec -> deployment.
• Grpc: Add a Grpc endpoint.
• Cli: Add the command (name TBD).

Once we know the liveness of the nodes in the cluster we need to create a load balancer entrypoint to access the cluster. We might need to use envoy since I do not know if we can change with an API the status of the nodes in the builtin Service K8s stanza.