Go(lang) client library for accessing information of an Apache Mesos cluster.

megos

GoDoc Build Status Go Report Card Coverage Status

Go(lang) client library for accessing an Apache Mesos cluster.

Features

  • Determine the Mesos leader
  • Get the current state of every mesos node (master or slave)
  • Retrieve stdout and stderr of tasks
  • Covered with unit tests

Installation

It is go gettable

$ go get github.com/andygrunwald/megos

(optional) to run unit / example tests:

$ cd $GOPATH/src/github.com/andygrunwald/megos
$ go test -v ./...

API

Please have a look at the GoDoc documentation for a detailed API description.

Examples / use cases

A (small) list of usecases how this library can be used:

  • Determine the leader of a Mesos cluster
  • Get a list of all completed Mesos tasks
  • Get the stdout and stderr of a failed mesos task
  • Get the statistics and push it to a different backend

Further more a few examples how the API can be used and the code looks like.

Determine the leader node

node1, _ := url.Parse("http://192.168.1.120:5050/")
node2, _ := url.Parse("http://192.168.1.122:5050/")

mesos := megos.NewClient([]*url.URL{node1, node2}, nil)
leader, err := mesos.DetermineLeader()
if err != nil {
	panic(err)
}

fmt.Println(leader)
// Output:
// [email protected]:5050

Get the version of Mesos

node1, _ := url.Parse("http://192.168.1.120:5050/")
node2, _ := url.Parse("http://192.168.1.122:5050/")

mesos := megos.NewClient([]*url.URL{node1, node2}, nil)
state, err := mesos.GetStateFromCluster()
if err != nil {
	panic(err)
}

fmt.Printf("Mesos v%s", state.Version)
// Output:
// Mesos v0.26.0

Get stdout and stderr of a task

Get stdout and stderr from a task of the chronos framework. Error checks are dropped for simplicity.

node1, _ := url.Parse("http://192.168.1.120:5050/")
node2, _ := url.Parse("http://192.168.1.122:5050/")
mesos := megos.NewClient([]*url.URL{node1, node2}, nil)

frameworkPrefix := "chronos"
taskID := "ct:1444578480000:0:example-chronos-task:"

mesos.DetermineLeader()
state, _ := mesos.GetStateFromLeader()

framework, _ := mesos.GetFrameworkByPrefix(state.Frameworks, frameworkPrefix)
task, _ := mesos.GetTaskByID(framework.CompletedTasks, taskID)

slave, _ := mesos.GetSlaveByID(state.Slaves, task.SlaveID)

pid, _ := mesos.ParsePidInformation(slave.PID)
slaveState, _ := mesos.GetStateFromPid(pid)

framework, _ = mesos.GetFrameworkByPrefix(slaveState.CompletedFrameworks, frameworkPrefix)
executor, _ := mesos.GetExecutorByID(framework.CompletedExecutors, taskID)

stdOut, _ := mesos.GetStdOutOfTask(pid, executor.Directory)
stdErr, _ := mesos.GetStdErrOfTask(pid, executor.Directory)

fmt.Println(string(stdOut))
fmt.Println("================")
fmt.Println(string(stdErr))
// Output:
// Registered executor on 192.168.1.123
// Starting task ct:1444578480000:0:example-chronos-task:
// sh -c 'MY COMMAND'
// Forked command at 10629
// ...
// ================
// I1011 17:48:00.390614 10602 exec.cpp:132] Version: 0.22.1
// I1011 17:48:00.532158 10618 exec.cpp:206] Executor registered on slave 20150603-103119-2046951690-5050-24382-S1

Version compatibility

This library was tested with Apache Mesos in version 0.26.0. In theory this should work with versions >= v0.25.x.

In version 0.25.x they renamed various API endpoints (like state.json to /state). See Upgrading Mesos - Upgrading from 0.24.x to 0.25.x for details. This is the reason why we support no lower versions of Mesos.

Other/Similar projects

  • boldfield/go-mesos: A client for discovering information about a Mesos exposed via HTTP API
  • antonlindstrom/mesos_stats: Statistics definition for Mesos /monitor/statistics.json
  • Clever/marathon-stats: A simple container which queries marathon and mesos for stats about their current state, and logs these stats to stderr
  • bolcom/mesos_metrics: Go definitions for the Mesos {master}:5050/metrics/snapshot and {slave}:5051/metrics/snapshot endpoints

Contribution

  • You have a question?
  • Don`t know if a feature is supported?
  • Want to implement a new feature, but don`t know how?
  • You like the library and use it for your implementation / use case?
  • You found a bug or incompatibility?
  • Something is not working as expected?

Feel free to open a new issue. I will be happy to answer them and try to help you. It might be useful to add as much information as possible into the issue like Mesos version, example URL, (parts) of your code and the expected and current behaviour.

License

This project is released under the terms of the MIT license.

Owner
Andy Grunwald
I solve problems and put things into production.
Andy Grunwald
Comments
  • Incompatible types during json unmarshalling

    Incompatible types during json unmarshalling

    I'm wondering why couple of fields on the struct (like activated_slaves https://github.com/andygrunwald/megos/blob/master/types.go#L10) are declared as integers while in mesos they are declared as doubles (https://github.com/apache/mesos/blob/837a13f3b8912f3bc847afecff7bfd06536414aa/src/master/master.hpp#L1214)

    Using go 1.5.1 querying cluster state panic'ed every single time due to unmarshalling error. I've patched megos locally and it runs flawlessly on mesos 0.25.0 and go 1.5.1.

    From go 1.5 changelog:

    The encoding/json package now returns an UnmarshalTypeError if a JSON value is not appropriate for the target variable or component to which it is being unmarshaled.

    I can send you over some nice PR :)

  • Get stdout and stderr of a task example is broken

    Get stdout and stderr of a task example is broken

    executor, _ := mesos.GetExecutorByID(framework.CompletedExecutors, taskID)
    
    framework.CompletedExecutors undefined (type *megos.Framework has no field or method CompletedExecutors)
    
  • Allow the injection of an http client

    Allow the injection of an http client

    This is an API breaking change that allows the injection of an HTTP client. This allows API consumers to handle special cases like TLS configurations. It's also just good practice.

    Up to you if you want to change the existing APIs for it. We needed it, so I thought I'd make the PR.

  • json: cannot unmarshal number into Go value of type string

    json: cannot unmarshal number into Go value of type string

    Mesos slave attributes are not necessary strings:

          {
            "rack": 106,
            "type": "storage"
          }
    

    This causes megos to fail on state deserialization:

    json: cannot unmarshal number into Go value of type string
    

    https://github.com/mesosphere/marathon/issues/1711

  • Change http.Client to HTTPClient interface

    Change http.Client to HTTPClient interface

    Allow use of any compatible HTTP client.

    Examples:

    • https://github.com/hashicorp/go-retryablehttp
    • https://github.com/sethgrid/pester

    Minor: change Http to HTTP for linter.

  • ReservedResources is a hash of names to resouces, updating the struct to match.

    ReservedResources is a hash of names to resouces, updating the struct to match.

    Example of this is:

    reserved_resources: {
        my_role: {
            disk: 256592,
            mem: 115712,
            gpus: 0,
            cpus: 38,
            ports: "[10001-32767]"
        }
    }
    

    I'm unsure if unreserved_resources looks the same.

  • Replace float32 with float64

    Replace float32 with float64

    float32 is very approximate (https://play.golang.org/p/S-Y0_tuyOd):

    package main
    
    import (
        "log"
    )
    
    func main() {
        // proper float64
        f64 := float64(1463070016.93235)
        // make it float32
        f32 := float32(1463070016.93235)
    
        log.Printf("float64 : %.5f", f64)
        log.Printf("float32 : %.5f", f32)
        log.Printf("diff    : %.5f", f64 - float64(f32))
    }
    
    2009/11/10 23:00:00 float64 : 1463070016.93235
    2009/11/10 23:00:00 float32 : 1463070080.00000
    2009/11/10 23:00:00 diff    : -63.06765
    

    This is not really useful for timestamps :)

May 25, 2021
NotionGo is a Go client library for accessing the Notion API v1.

NotionGo (WIP) NotionGo is a Go client library for accessing the Notion API v1. Installation NotionGo is compatible with modern Go releases in module

May 22, 2021
go-ftx go-ftx is a Go client library for accessing the FTX API

go-ftx go-ftx is a Go client library for accessing the FTX API

Nov 14, 2022
Go(lang) client library for Cachet (open source status page system).

cachet Go(lang) client library for Cachet (open source status page system). Features Full API support Components Incidents Metrics Subscribers Various

Sep 27, 2022
Go library for accessing the Codeship API v2

Codeship API v2 Client for Go Codeship API v2 client for Go. Documentation https://godoc.org/github.com/codeship/codeship-go Usage go get -u github.co

Sep 27, 2022
Go library for accessing the GitHub API

go-github go-github is a Go client library for accessing the GitHub API v3. Currently, go-github requires Go version 1.9 or greater. go-github tracks

Dec 30, 2022
Go library for accessing the MyAnimeList API: http://myanimelist.net/modules.php?go=api

go-myanimelist go-myanimelist is a Go client library for accessing the MyAnimeList API. Project Status The MyAnimeList API has been stable for years a

Sep 28, 2022
Go library for accessing trending repositories and developers at Github.
Go library for accessing trending repositories and developers at Github.

go-trending A package to retrieve trending repositories and developers from Github written in golang. This package were inspired by rochefort/git-tren

Dec 21, 2022
Go library for accessing the Keycloak API

keycloak keycloak is a Go client library for accessing the Keycloak API. Installation go get github.com/zemirco/keycloak Usage package main import (

Dec 1, 2022
Go library for accessing the BlaBlaCar API

go-blablacar is a Go client library for accessing the BlaBlaCar API.

Nov 27, 2022
🤖🚀📦 A Discord Bot for accessing the cdnjs platform
🤖🚀📦 A Discord Bot for accessing the cdnjs platform

A bridge between https://cdnjs.com/api and Discord Big shoutout to Br1ght0ne for helping me with writing helpers.go/SplitIn

Aug 17, 2022
efsu is for accessing AWS EFS from your machine without a VPN

efsu: VPN-less access to AWS EFS efsu is for accessing AWS EFS from your machine without a VPN. It achieves this by deploying a Lambda function and sh

Mar 11, 2022
A GoLang wrapper for Politics & War's API. Forego the hassle of accessing the API directly!

A GoLang wrapper for Politics & War's API. Forego the hassle of accessing the API directly!

Mar 5, 2022
Support for Usagi API developed by @DanielaGC. The library is written in Go Lang

?? Usagi API - Wrapper This library is simple and easy to use and was developed entirely for Usagi API written in Go Lang. ??‍?? Example package main

Oct 18, 2021
A Go client implementing a client-side distributed consumer group client for Amazon Kinesis.
A Go client implementing a client-side distributed consumer group client for Amazon Kinesis.

Kinesumer is a Go client implementing a client-side distributed consumer group client for Amazon Kinesis.

Jan 5, 2023
Clusterpedia-client - clusterpedia-client supports the use of native client-go mode to call the clusterpedia API

clusterpedia-client supports the use of native client-go mode to call the cluste

Jan 7, 2022
Client-go - Clusterpedia-client supports the use of native client-go mode to call the clusterpedia API

clusterpedia-client supports the use of native client-go mode to call the cluste

Dec 5, 2022
Aoe4-client - Client library for aoe4 leaderboards etc

AOE4 Client Overview This is a go client used to query AOE4 data from either the

Jan 18, 2022
A simple Go utility to display track information from, and send commands to, spotifyd from Tiling Window Managers like Sway and i3
A simple Go utility to display track information from, and send commands to, spotifyd from Tiling Window Managers like Sway and i3

Untitled Spotifyd Controller A simple Go utility to display track information from, and send commands to, spotifyd from Tiling Window Managers like Sw

Mar 8, 2022