Make IoT a lot more fun with data.

Eywa

What is Eywa?

"Eywa is the guiding force and deity of Pandora and the Na'vi. All living things on Pandora connect to Eywa." -- Avatar Wiki

Project Eywa is essentially a connection manager that keeps track of connected devices. But more than just connecting devices, it is also capable of designing flexible APIs, sending control commands to them, collecting data from them, real-time monitoring and debugging, in the end, providing query interfaces that can be used for data visualization.

Eywa lets the teams of embedded system developers forget about reinventing the backend services and provides a commonly used protocol, websocket, to make real-time communication easily achievable.

Why is it useful?

We are a group of people who are interested in Home Automations and Smart Devices. Often time, these projects involve connecting devices into cloud, tracking the usage of different functionalities, collecting the data and also controlling them. After worked on several similar projects, we found there is no reason to reinvent the wheel each time for different applications. So we came up with Project Eywa to help small teams like us reduce their development circles.

What features does it have?

Here is a growing list of features we want to support:

  • Connection Manager
  • Device Control
  • Command Line Tools
  • Connection Attach Mode
  • Admin Panel
  • Basic Authentication
  • SSL protection
  • Data Indexing
  • Data Streaming
  • Data Export
  • Data Retention
  • Data Visualization
  • Query Interface
  • Clustering
  • Custom Web hooks
  • Custom Monitors
  • M2M (machine to machine) communication
  • HTTP Long-Polling
  • Websocket
  • MQTT integration
  • Dockerized image

Our Admin Panel and Data Visualization dashboard will be releasing soon.

Please let us know if you want more features by creating issues. Pull requests are also very much welcome!

Performance

How performant is Eywa? Well, we did a simple benchmark and the benchmark script is also available in the repo under benchmark directory.

The latest benchmark shows, on a machine of 12 CPUs + 32GB mem from Digital Ocean. A single Eywa node can keep track of more than 1.5 million devices, with a lot of potential. CPU is merely used, but memory is the limiting factor.

For more details please check out wiki on Performance.

How to use?

You can get started with our wiki.

Community / Contributing

Eywa maintains a forum goeywa, where you should feel free to ask questions, request features, or to announce projects that are built with Eywa. You should also see updates and road maps on Eywa in this forum.

Contributions to Eywa are very much welcomed. Fork us if you would like to.

Comments
  • TestApiToDevice fails

    TestApiToDevice fails

    Failures:

    • /Users/alexsun/.gvm/pkgsets/go1.7.3/global/src/github.com/eywa/api_tests/api_device_test.go Line 115: Expected: 'response message' Actual: '{"error":"websocket connection response timed out for 8s"}' (Should be equal)

    For 7 requests made FAILED [1/7] [send message to device] - Expected Status 200, but got 502: "502 Bad Gateway"

  • models::auth_token_test fails

    models::auth_token_test fails

    Failures:

    • /home/ec2-user/.gvm/pkgsets/go1.7.3/global/src/github.com/eywa/models/auth_token_test.go Line 34: Expected: true Actual: false

    4 total assertions

    It fails on AWS but it succeeds locally on my laptop

  • api_tests::TestHttpConnection fails

    api_tests::TestHttpConnection fails

    --- FAIL: TestHttpConnection (6.59s) FAIL FAIL github.com/eywa/api_tests 6.597s Tests are done... Test results are in auto_test.log. iterations=1

    In auto_test.log: Failures:

    • /home/ec2-user/.gvm/pkgsets/go1.7.3/global/src/github.com/eywa/api_tests/device_http_test.go Line 61: Expected: '1' Actual: '0' (Should be equal)

    2 total assertions

  • api test causes panic

    api test causes panic

    $ ./auto_test.sh -vif -T ApiToDevice -n 1 Warn: EYWA_HOME is not set. Using current directory as EYWA_HOME. Running test setup... Starting test server... Test server started... Starting Tests... panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x75aafa]

    goroutine 41 [running]: panic(0xa41ec0, 0xc420012090) /home/ec2-user/.gvm/gos/go1.7.3/src/runtime/panic.go:500 +0x1a1 github.com/eywa/message_handlers.glob..func1.1(0xff61a0, 0xc4201295f0, 0xff5640, 0xc42023da90, 0xfed100, 0xc4202146f0) /home/ec2-user/.gvm/pkgsets/go1.7.3/global/src/github.com/eywa/message_handlers/indexer.go:41 +0xa5a created by github.com/eywa/connections.(*WebsocketConnection).start /home/ec2-user/.gvm/pkgsets/go1.7.3/global/src/github.com/eywa/connections/websocket_connection.go:371 +0x10c .--- FAIL: TestApiToDevice (0.25s) FAIL github.com/eywa/api_tests 0.258s

  • race condition on registering device

    race condition on registering device

    When running test "./auto_test.sh -vif -T ApiToDevice -n 1"

    • /Users/alexsun/.gvm/pkgsets/go1.7.3/global/src/github.com/eywa/api_tests/api_device_test.go Line 108: Expected: 'response message' Actual: '{"error":"device is not online"}' (Should be equal)

    After adding some debug info as below, I see that server isn't aware of websocket disconnection as soon as the client close the websocket abnormally, i.e. without sending "disconnect" message and there is a delay until server figures it out by getting a read socket error. So server won't unregister the connection in time. As a result, if the same device issue the connection again, the server will establish a new websocket under the legacy device ID. Now, the problem appears. Since the connection is maintained by a btree with key of device ID, at the moment, there will be two connections under one device ID, i.e. the new established one and the legacy one. Then a few seconds later, the legacy connection gets read socket error and will unregister the devcie ID. Therefore, the second test case gets error message of "device is not online".

    Log of same device ID. INFO 2016-12-12T03:41:28.200581 [ip-172-31-34-137/SWuOTApNyG-000002] Started GET /channels/GDE3N17PA5gXAlVj/devices/abc/ws from 127.0.0.1:60728 INFO 2016-12-12T03:41:28.201330 websocket connection started id=abc DEBUG 2016-12-12T03:41:28.202699 [ip-172-31-34-137/SWuOTApNyG-000002] Returning 200 in 2.097894ms, with response: INFO 2016-12-12T03:41:28.211787 [ip-172-31-34-137/SWuOTApNyG-000003] Started POST /api/channels/GDE3N17PA5gXAlVj/devices/abc/send? from 127.0.0.1:50634 DEBUG 2016-12-12T03:41:28.211966 [ip-172-31-34-137/SWuOTApNyG-000003] Returning 200 in 169.646µs, with response: INFO 2016-12-12T03:41:28.212528 [ip-172-31-34-137/SWuOTApNyG-000004] Started GET /channels/GDE3N17PA5gXAlVj/devices/abc/ws from 127.0.0.1:60730 INFO 2016-12-12T03:41:28.212609 Existing connection found, id=abc INFO 2016-12-12T03:41:28.212626 websocket connection started id=abc DEBUG 2016-12-12T03:41:28.212631 [ip-172-31-34-137/SWuOTApNyG-000004] Returning 200 in 97.153µs, with response: INFO 2016-12-12T03:41:28.213065 WebsocketError: error reading message from websocket connection, websocket: close 1006 (abnormal closure): unexpected EOF INFO 2016-12-12T03:41:28.213068 rListen websocket error INFO 2016-12-12T03:41:28.214366 [ip-172-31-34-137/SWuOTApNyG-000005] Started POST /api/channels/GDE3N17PA5gXAlVj/devices/abc/send? from 127.0.0.1:50634 DEBUG 2016-12-12T03:41:28.214444 [ip-172-31-34-137/SWuOTApNyG-000005] Returning 200 in 70.891µs, with response: INFO 2016-12-12T03:41:28.216459 WebsocketError: error reading message from websocket connection, websocket: close 1006 (abnormal closure): unexpected EOF INFO 2016-12-12T03:41:28.216461 rListen websocket error INFO 2016-12-12T03:41:28.216476 Unregister connection, id=abc

    Log of different device ID. INFO 2016-12-12T03:48:46.870992 [ip-172-31-34-137/jJHQz1Q01K-000002] Started GET /channels/GDE3N17PA5gXAlVj/devices/abc/ws from 127.0.0.1:60758 INFO 2016-12-12T03:48:46.871722 websocket connection started id=abc DEBUG 2016-12-12T03:48:46.873198 [ip-172-31-34-137/jJHQz1Q01K-000002] Returning 200 in 2.184106ms, with response: INFO 2016-12-12T03:48:46.883102 [ip-172-31-34-137/jJHQz1Q01K-000003] Started POST /api/channels/GDE3N17PA5gXAlVj/devices/abc/send? from 127.0.0.1:50664 DEBUG 2016-12-12T03:48:46.885952 [ip-172-31-34-137/jJHQz1Q01K-000003] Returning 200 in 2.838171ms, with response: INFO 2016-12-12T03:48:46.886547 [ip-172-31-34-137/jJHQz1Q01K-000004] Started GET /channels/GDE3N17PA5gXAlVj/devices/efg/ws from 127.0.0.1:60760 INFO 2016-12-12T03:48:46.886647 websocket connection started id=efg DEBUG 2016-12-12T03:48:46.886872 [ip-172-31-34-137/jJHQz1Q01K-000004] Returning 200 in 318.189µs, with response: INFO 2016-12-12T03:48:46.886905 WebsocketError: error reading message from websocket connection, websocket: close 1006 (abnormal closure): unexpected EOF INFO 2016-12-12T03:48:46.886906 rListen websocket error INFO 2016-12-12T03:48:46.886916 Unregister connection, id=abc INFO 2016-12-12T03:48:46.888396 [ip-172-31-34-137/jJHQz1Q01K-000005] Started POST /api/channels/GDE3N17PA5gXAlVj/devices/efg/send? from 127.0.0.1:50664 DEBUG 2016-12-12T03:48:46.888487 [ip-172-31-34-137/jJHQz1Q01K-000005] Returning 200 in 84.126µs, with response: INFO 2016-12-12T03:48:46.890039 WebsocketError: error reading message from websocket connection, websocket: close 1006 (abnormal closure): unexpected EOF INFO 2016-12-12T03:48:46.890041 rListen websocket error

  • Eywa tool new

    Eywa tool new

    Reorganized eywa_tool.rb. The eywa_tool_new.rb provide subcommands. Currently, it supports "channel" and "connection" two subcommands. Also it supports profile and channel template so that user doesn't have to manually input stuffs every time.

An Open-Source Platform for Quantified Self & IoT
An Open-Source Platform for Quantified Self & IoT

Heedy Note: Heedy is currently in alpha. You can try it out by downloading it from the releases page, but there is no guarantee that future versions w

Jan 1, 2023
Suite of libraries for IoT devices (written in Go), experimental for x/exp/io

Go libraries/drivers for IoT devices This repo contains a suite of libraries for IoT devices/sensors/actuators. The suite is meant to be as dependency

Sep 26, 2022
Golang framework for robotics, drones, and the Internet of Things (IoT)
Golang framework for robotics, drones, and the Internet of Things (IoT)

Gobot (https://gobot.io/) is a framework using the Go programming language (https://golang.org/) for robotics, physical computing, and the Internet of

Dec 29, 2022
A Go client for Google IoT Core

IoT A simple framework for implementing a Google IoT device. This package makes use of the context package to handle request cancelation, timeouts, an

Sep 26, 2022
Industrial IoT Messaging and Device Management Platform
Industrial IoT Messaging and Device Management Platform

Mainflux Mainflux is modern, scalable, secure, open-source, and patent-free IoT cloud platform written in Go. It accepts user and thing (sensor, actua

Dec 31, 2022
Gobot - Golang framework for robotics, drones, and the Internet of Things (IoT)
Gobot - Golang framework for robotics, drones, and the Internet of Things (IoT)

Gobot (https://gobot.io/) is a framework using the Go programming language (https://golang.org/) for robotics, physical computing, and the Internet of Things.

Jan 8, 2023
An embeddable lightweight Go/Golang MQTT broker(server) for IoT.
An embeddable lightweight Go/Golang MQTT broker(server) for IoT.

Snple MQTT 简体中文 Note: The API of this library is still unstable and has not been sufficiently tested, please do not use it in production environments.

Sep 12, 2022
🐼 IoT worm written in pure golang.
🐼 IoT worm written in pure golang.

GoriaNet Most powerfull cross compiler (27arch). Kill process by port and check for duplicate instance. Killing process by port. Cross compiler. Infor

Oct 17, 2022
Next-generation IoT open source platform.
Next-generation IoT open source platform.

tKeel Next-generation IoT open source platform High performance, High security and easy to use tKeel is a strong and reusable IoT platform that helps

Dec 28, 2022
Whichip: discover (IoT) device's IP in local network
Whichip: discover (IoT) device's IP in local network

whichip: discover (IoT) device's IP in local network Install On (IoT) Device wget -O install.sh

Dec 8, 2021
A opinionated multi-tenant hyperscale Internet of Things platform to connect IoT devices fast and securely with minimal TCO

infinimesh IoT Platform infinimesh is a opinionated multi-tenant hyperscale Internet of Things platform to connect IoT devices fast and securely with

Feb 14, 2022
Exploring and comparing different IOT messaging protocols / transports.

IOT Messaging Protocols Blynk https://blynk.io/ A fully integrated suite of IoT software Device provisioning Sensor data visualization Remote control

Jan 2, 2022
IoT platform with things/user management and visualization, in Go with Docker using microservices

BARIOT IoT platform to Manage Users and their Things and visualize their data. Microservices services architecture build with Go and docker (compose).

Jun 22, 2022
A project outputs Bluetooth Low Energy (BLE) sensors data in InfluxDB line protocol formatA project outputs Bluetooth Low Energy (BLE) sensors data in InfluxDB line protocol format

Intro This project outputs Bluetooth Low Energy (BLE) sensors data in InfluxDB line protocol format. It integrates nicely with the Telegraf execd inpu

Apr 15, 2022
mqtt-rewriter is a tool that can forward data from a topic to another topic.

mqtt-rewriter Background mqtt-rewriter is a tool that can forward data from a topic to another topic. Install Todo... Usage Currently only supports tw

Feb 18, 2022
Golang DNSTAP sensor use to collect passive dns data from a recursive name server

dnstap-sensor DNSTAP-SENSOR is a Golang program that is used to collect passive dns data from a recursive name server and submit it to Deteque's DNSTA

Nov 21, 2022
IoT Manager: use IoT platforms with Mender

Mender: Azure IoT Manager: use Azure IoT with Mender General Mender is an open source over-the-air (OTA) software updater for embedded Linux devices.

Jan 10, 2022
Run your MapReduce workloads as a single binary on a single machine with multiple CPUs and high memory. Pricing of a lot of small machines vs heavy machines is the same on most cloud providers.

gomap Run your MapReduce workloads as a single binary on a single machine with multiple CPUs and high memory. Pricing of a lot of small machines vs he

Sep 16, 2022
Brokerage Tax Lot Processor (River)

Tax Lot Processor Information An important part of a brokerage product is keeping track of tax lots. A tax lot is created when a purchase is made. Whe

Feb 4, 2022
simpleChatInGo - This is a simple chat that i made for fun asnd learn more about websocket
simpleChatInGo - This is a simple chat that i made for fun asnd learn more about websocket

simpleChatInGo This is a simple chat that i made for fun asnd learn more about websocket deploy For deploy this you only need to run the command : $ d

Sep 21, 2022