distributed monitoring system

Last update: Dec 24, 2022

Comments: 3

OWL

OWL 是由国内领先的第三方数据智能服务商 TalkingData 开源的一款企业级分布式监控告警系统，目前由 Tech Operation Team 持续开发更新维护。

OWL 后台组件全部使用 Go 语言开发，Go 语言是 Google 开发的一种静态强类型、编译型、并发型，并具有垃圾回收功能的编程语言，它的并发机制可以充分利用多核，同平台一次编译可以到处运行，运维成本极低，更多的信息可以参考官方文档。前端页面使用 iView 开发，iView 同样是由 TalkingData 开源的一套基于 Vue.js 的 UI 组件库，主要服务于 PC 界面的中后台产品。

Features

Go语言开发，部署维护简单
分布式，支持多机房
多维的数据模型，类opentsdb
支持多种报警算法，支持多条件组合、时间范围、报警模板等
灵活的插件机制，支持任意语言编写，支持传参，自动同步到客户端
丰富的报警渠道，邮件、企业微信、短信、电话以及自定义脚本
原始数据永久存储，支持发送到 opentsdb、kairosdb、kafka
自带 web 管理界面以及强大的自定义图表功能能

Architecture

Components

agent：安装在每台被监控机器上，用于采集监控数据

netcollect：通过 SNMP V2 采集网络设备的接口数据

repeater：接收 agent 发送过来的监控数据，并写入后端存储

cfc：维护客户端需要执行的插件列表，主机名、ip地址更新以及采集到的指标列表

controller：从数据库加载告警策略，生成任务发送给 inspector，并且根据执行结果进行告警

inspector：从 controller 获取监控任务，根据 tsdb 中的数据进行计算，并将结果返回 controller

api：对外提供 http rest api接口，web 页面就是通过它来获取数据

MySQL：所有配置信息的持久化存储，包含主机信息，告警策略，主机组，人员等

TSDB：时序数据库(time seires database)，用于存储采集到的监控数据

frontend：web 管理页面，可以方便的进行系统管理维护工作

Demo

http://54.223.127.87/

普通用户：demo/demo
管理员：admin/111111
注: demo 环境数据库每隔 1 个小时会自动恢复

rpm包地址

https://pan.baidu.com/s/1UTYOOB8YE8nng0guXOXkmg#list/path=%2Fowl

前端源码地址

https://github.com/TalkingData/owl-frontend

QQ Group

492850035

Owner

https://github.com/TalkingData/owl

Similar Resources

A flexible process data collection, metrics, monitoring, instrumentation, and tracing client library for Go

Package monkit is a flexible code instrumenting and data collection library. See documentation at https://godoc.org/gopkg.in/spacemonkeygo/monkit.v3 S

Dec 14, 2022

A GNU/Linux monitoring and profiling tool focused on single processes.

Uroboros is a GNU/Linux monitoring tool focused on single processes. While utilities like top, ps and htop provide great overall details, they often l

Dec 26, 2022

Open source framework for processing, monitoring, and alerting on time series data

Kapacitor Open source framework for processing, monitoring, and alerting on time series data Installation Kapacitor has two binaries: kapacitor – a CL

Dec 26, 2022

Ping monitoring engine used in https://ping.gg

Disclaimer: If you are new to Go this is not a good place to learn best practices, the code is not very idiomatic and there's probably a few bad ideas

Dec 22, 2022

Simple and extensible monitoring agent / library for Kubernetes: https://gravitational.com/blog/monitoring_kubernetes_satellite/

Satellite Satellite is an agent written in Go for collecting health information in a kubernetes cluster. It is both a library and an application. As a

Nov 10, 2022

Gowl is a process management and process monitoring tool at once. An infinite worker pool gives you the ability to control the pool and processes and monitor their status.

Nov 10, 2022

mtail - extract internal monitoring data from application logs for collection into a timeseries database

mtail - extract internal monitoring data from application logs for collection into a timeseries database mtail is a tool for extracting metrics from a

Dec 29, 2022

SigNoz helps developers monitor their applications & troubleshoot problems, an open-source alternative to DataDog, NewRelic, etc. 🔥 🖥. 👉 Open source Application Performance Monitoring (APM) & Observability tool

Monitor your applications and troubleshoot problems in your deployed applications, an open-source alternative to DataDog, New Relic, etc. Documentatio

Sep 24, 2021

Detecctor is a ⚡ fast, fully customizable 💗 monitoring platform. It uses Telegram as a notification 📥 service

Detecctor is a ⚡ fast, fully customizable 💗 monitoring platform. It uses Telegram as a notification 📥 service. The main components are a TCP server, MongoDB and multiple clients.

Nov 16, 2021

Comments

client 增加开启http服务，接受post的metric的功能

client增加http接收metrics功能。在配置配置文端口号不为0开启。例如：关闭此功能：metric_bind=127.0.0.1:0。开启此功能：metric_bind=127.0.0.1:10057

python的post数据的例子：

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# @Date    : 2018/10/21
# @Author  :  ()
# @Version : $

import requests
import time

data = {
    "metric": "my.system.cpu.softirq",
    "data_type": "GAUGE",
    "value": 355.53,
    "cycle": 60,
    "tags": {
        "cpu": "cpu-total66"
    }
}

for i in range(0, 1000):
    data["value"] = i
    re = requests.post("http://127.0.0.1:10057/", json=data)
    print (data)
    print (re.content)
    time.sleep(60)

自定义服务的功能

owl的k8sagent输入到owl中的数据为： { "metric": "k8s.restart.count", "data_type": "counter", "value": 0, "tags": { "container":"fronted-prod-v1", "namespace":"web-ns", "port": "3306", pod="fronted-prod-v1-7d7fd69755-62dlh" } }

我希望可以根据其中的任意的标签定义一个服务，比如namespace和container。那么在TSDB中的所有的包含这个标签的metrcs都属于这个服务的监控项目。报警：可以根据自定义的tsdb查询语句报警，比如k8s.restart.count携带定义服务时候的标签，能够对查询到的多个结果报警
Time series queue support persistent storage

In current situation，when the repeater receivers the time series data sent by the owl client，it will be stored in go channel。if repeater process crashes at this time，it will lost all time series data。therefore，we want to use a persistent queue to ensure that data is not lost.

Nightingale - A Distributed and High-Performance Monitoring System. Prometheus enterprise edition

Introduction ?? A Distributed and High-Performance Monitoring System. Prometheus

Jan 7, 2022

Monitoring-go - A simple monitoring tool to sites of MOVA

Monitoring GO A simple monitoring tool to sites of MOVA How to use Clone Repo gi

Feb 14, 2022

The Prometheus monitoring system and time series database.

Prometheus Visit prometheus.io for the full documentation, examples and guides. Prometheus, a Cloud Native Computing Foundation project, is a systems

Dec 31, 2022

rtop is an interactive, remote system monitoring tool based on SSH

rtop rtop is a remote system monitor. It connects over SSH to a remote system and displays vital system metrics (CPU, disk, memory, network). No speci

Dec 30, 2022

A system and resource monitoring tool written in Golang!

Grofer A clean and modern system and resource monitor written purely in golang using termui and gopsutil! Currently compatible with Linux only. Curren

Jan 8, 2023

An open-source and enterprise-level monitoring system.

Falcon+ Documentations Usage Open-Falcon API Prerequisite Git >= 1.7.5 Go >= 1.6 Getting Started Docker Please refer to ./docker/README.md. Build from

Jan 1, 2023

checkah is an agentless SSH system monitoring and alerting tool.

CHECKAH checkah is an agentless SSH system monitoring and alerting tool. Features: agentless check over SSH (password, keyfile, agent) config file bas

Oct 14, 2022

Cloudprober is a monitoring software that makes it super-easy to monitor availability and performance of various components of your system.

Cloudprober is a monitoring software that makes it super-easy to monitor availability and performance of various components of your system. Cloudprobe

Dec 30, 2022

Distributed-Log-Service - Distributed Log Service With Golang

Distributed Log Service This project is essentially a result of my attempt to un

Jun 1, 2022

Distributed Commit Log from Travis Jeffery's Distributed Services book

proglog A distributed commit log. This repository follows the book "Distributed Services with Go" by Travis Jeffrey. The official repository for this

May 23, 2022