An open-source and enterprise-level monitoring system.

Falcon+

Open-Falcon

Build Status codecov GoDoc Code Issues Go Report Card License Backers on Open Collective Sponsors on Open Collective

Documentations

Prerequisite

  • Git >= 1.7.5
  • Go >= 1.6

Getting Started

Docker

Please refer to ./docker/README.md.

Build from source

before start, please make sure you prepared this:

yum install -y redis
yum install -y mysql-server

NOTE: be sure to check redis and mysql-server have successfully started.

And then

# Please make sure that you have set `$GOPATH` and `$GOROOT` correctly.
# If you have not golang in your host, please follow [https://golang.org/doc/install] to install golang.

mkdir -p $GOPATH/src/github.com/open-falcon
cd $GOPATH/src/github.com/open-falcon
git clone https://github.com/open-falcon/falcon-plus.git

And do not forget to init the database first (if you have not loaded the database schema before)

cd $GOPATH/src/github.com/open-falcon/falcon-plus/scripts/mysql/db_schema/
mysql -h 127.0.0.1 -u root -p < 1_uic-db-schema.sql
mysql -h 127.0.0.1 -u root -p < 2_portal-db-schema.sql
mysql -h 127.0.0.1 -u root -p < 3_dashboard-db-schema.sql
mysql -h 127.0.0.1 -u root -p < 4_graph-db-schema.sql
mysql -h 127.0.0.1 -u root -p < 5_alarms-db-schema.sql

NOTE: if you are upgrading from v0.1 to v0.2.0(or above),then. More upgrading instruction

mysql -h 127.0.0.1 -u root -p < 5_alarms-db-schema.sql

Compilation

cd $GOPATH/src/github.com/open-falcon/falcon-plus/

# make all modules
make all

# make specified module
make agent

# pack all modules
make pack
  • after make pack you will got open-falcon-vx.x.x.tar.gz
  • if you want to edit configure file for each module, you can edit config/xxx.json before you do make pack

Unpack and Decompose

export WorkDir="$HOME/open-falcon"
mkdir -p $WorkDir
tar -xzvf open-falcon-vx.x.x.tar.gz -C $WorkDir
cd $WorkDir

Start all modules in single host

cd $WorkDir
./open-falcon start

# check modules status
./open-falcon check

Run More Open-Falcon Commands

for example:

# ./open-falcon [start|stop|restart|check|monitor|reload] module
./open-falcon start agent

./open-falcon check
        falcon-graph         UP           53007
          falcon-hbs         UP           53014
        falcon-judge         UP           53020
     falcon-transfer         UP           53026
       falcon-nodata         UP           53032
   falcon-aggregator         UP           53038
        falcon-agent         UP           53044
      falcon-gateway         UP           53050
          falcon-api         UP           53056
        falcon-alarm         UP           53063
  • For debugging , You can check $WorkDir/$moduleName/logs/xxx.log

Install Frontend Dashboard

NOTE: if you want to use grafana as the dashboard, please check this.

Package Management

We use govendor to manage the golang packages. Please install govendor before compilation.

go get -u github.com/kardianos/govendor

Most depended packages are saved under ./vendor dir. If you want to add or update a package, just run govendor fetch xxxx@commitID or govendor fetch [email protected], then you will find the package have been placed in ./vendor correctly.

Package Release

make clean all pack

API Standard

Q&A

  • Any issue or question is welcome, Please feel free to open github issues :)
  • FAQ

Contributors

This project exists thanks to all the people who contribute. [Contribute].

Owner
Open-Falcon
An open-source and enterprise-level monitoring system which is designed for modern distributed systems.
Open-Falcon
Comments
  • hostgroup新增 aggregator 后跳转页面404

    hostgroup新增 aggregator 后跳转页面404

    新增的hostgroup,然后在新增aggregator后跳转到的页面是http://domain/group/2/cluster,提示404
    集群监控项的具体内容:
    分子:$(agent.alive)
    分母:$#
    endpoint:all
    metric:agent.alive.percent
    tag:job=all
    周期:60

  • make api时报错

    make api时报错

    使用make all 编译到api 模块时,报以下错误

    github.com/falcon-plus/modules/api

    modules/api/main.go:73: cannot use routes (type *"github.com/falcon-plus/vendor/github.com/gin-gonic/gin".Engine) as type *"github.com/open-falcon/falcon-plus/vendor/github.com/gin-gonic/gin".Engine in argument to controller.StartGin make: *** [api] 错误 2

    尝试使用 make clean all 进行编译报错依旧

  • 用grafana的singlestat来展示部分监控数据出现问题

    用grafana的singlestat来展示部分监控数据出现问题

    grafana配了0.2的api url, datasource

    在用grafana的singlestat面板时,数据展示有问题。 singlestat

    查看api调用的日志,发现里面查数据库用的是正则,

    log

    例如'mem.memused'这个指标,用正则查的话,就会把mem.memused.percent也查出来 query

    grafana singlestat面板报错如下: r1 r2

  • transfer的日志文件无限增长直到用完磁盘空间

    transfer的日志文件无限增长直到用完磁盘空间

    在运行了几天时间之后,transfer.log文件的大小到达了28G。该文件最后几行打印如下: 2018/03/27 20:50:39 rpc.go:35: listener.Accept occur error: accept tcp [::]:8433: accept4: too many open files 2018/03/27 20:50:39 rpc.go:35: listener.Accept occur error: accept tcp [::]:8433: accept4: too many open files 2018/03/27 20:50:39 rpc.go:35: listener.Accept occur error: accept tcp [::]:8433: accept4: too many open files 2018/03/27 20:50:39 rpc.go:35: listener.Accept occur error: accept tcp [::]:8433: accept4: too many open files 2018/03/27 20:50:39 rpc.go:35: listener.Accept occur error: accept tcp [::]:8433: accept4: too many open files 2018/03/27 20:50:39 rpc.go:35: listener.Accept occur error: accept tcp [::]:8433: accept4: too many open files 2018/03/27 20:50:39 rpc.go:35: listener.Accept occur error: accept tcp [::]:8433: accept4: too many open files 2018/03/27 20:50:39 rpc.go:35: listener.Accept occur error: accept tcp [::]:8433: accept4: too many open files 2018/03/27 20:50:39 rpc.go:35: listener.Accept occur error: accept tcp [::]:8433: accept4: too many open files 问题的原因应该是 rpc.go 监听连接循环里面的打印造成的,希望能修复。

  • 告警邮件不生效

    告警邮件不生效

    curl http://8.15.4.85:4000/sender/mail -d "[email protected]&subject=告警测试&content=告警测试" x509: certificate is valid for 域名, not 邮件服务器域名 配置文件: /root/project/src/github.com/open-falcon/mail-provider/cfg.json

    {
        "debug": true,
        "http": {
            "listen": "0.0.0.0:4000",
            "token": ""
        },
        "smtp": {
            "addr": "邮件服务器域名:25",
            "username": "user",
            "password": "passwd",
            "from": "[email protected]"
        }
    }
    

    配置文件: /home/work/open-falcon/alarm/config/cfg.json image

  • judge报错后自动重启

    judge报错后自动重启

    • 组件: judge v0.2.0

    • 日志:

    2018/08/20 22:00:15 judge.go:47: [ERROR] parse func pdiff(#2) fail: strconv.Atoi: parsing "2\xef\xbc": invalid syntax. strategy id: 4023
    2018/08/20 22:00:15 judge.go:47: [ERROR] parse func pdiff(#2) fail: strconv.Atoi: parsing "2\xef\xbc": invalid syntax. strategy id: 4023
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x742c56]
    
    goroutine 86615156 [running]:
    github.com/open-falcon/falcon-plus/modules/judge/store.(*SafeLinkedList).HistoryData(0xc42d7c4e00, 0xa, 0x801600, 0xc487cd7701, 0xc5d9b22db0, 0xc5e7c70688)
            /data/service/open-falcon/workspace/src/github.com/open-falcon/falcon-plus/modules/judge/store/linkedlist.go:81 +0x426
    github.com/open-falcon/falcon-plus/modules/judge/store.AvgFunction.Compute(0x0, 0x0, 0xa, 0xc4b604f12c, 0x2, 0x4055400000000000, 0xc42d7c4e00, 0x411568, 0x70, 0xc5d9b22db0, ...)
            /data/service/open-falcon/workspace/src/github.com/open-falcon/falcon-plus/modules/judge/store/func.go:150 +0x3f
    github.com/open-falcon/falcon-plus/modules/judge/store.(*AvgFunction).Compute(0xc5d9b22db0, 0xc42d7c4e00, 0xc4b604f12c, 0x2, 0x4055400000000000, 0x9f3240, 0xc5d9b22db0)
            <autogenerated>:12 +0x66
    github.com/open-falcon/falcon-plus/modules/judge/store.judgeItemWithStrategy(0xc42d7c4e00, 0xb39, 0xc5a0d4b860, 0x12, 0xc55e705620, 0xc4b604f120, 0x8, 0xc4b604f12c, 0x2, 0x4055400000000000, ...)
            /data/service/open-falcon/workspace/src/github.com/open-falcon/falcon-plus/modules/judge/store/judge.go:51 +0x205
    github.com/open-falcon/falcon-plus/modules/judge/store.CheckStrategy(0xc42d7c4e00, 0xc4645b5ef0, 0x5b7ac970)
            /data/service/open-falcon/workspace/src/github.com/open-falcon/falcon-plus/modules/judge/store/judge.go:40 +0x3cc
    github.com/open-falcon/falcon-plus/modules/judge/store.Judge(0xc42d7c4e00, 0xc4645b5ef0, 0x5b7ac970)
            /data/service/open-falcon/workspace/src/github.com/open-falcon/falcon-plus/modules/judge/store/judge.go:12 +0x3f
    github.com/open-falcon/falcon-plus/modules/judge/store.(*JudgeItemMap).PushFrontAndMaintain(0xc42014e7e0, 0xc471bb2580, 0x20, 0xc4645b5ef0, 0xb, 0x5b7ac970)
            /data/service/open-falcon/workspace/src/github.com/open-falcon/falcon-plus/modules/judge/store/history.go:79 +0xa8
    github.com/open-falcon/falcon-plus/modules/judge/rpc.(*Judge).Send(0xc4201425e8, 0xc52123a500, 0x99, 0x99, 0xc461dc7710, 0x0, 0x0)
            /data/service/open-falcon/workspace/src/github.com/open-falcon/falcon-plus/modules/judge/rpc/receiver.go:22 +0x10f
    reflect.Value.call(0xc420150180, 0xc42014a058, 0x13, 0x8362bd, 0x4, 0xc5e7c70f20, 0x3, 0x3, 0xc4e96a4600, 0x100000000, ...)
            /data/service/go1.8.3/src/reflect/value.go:434 +0x91f
    reflect.Value.Call(0xc420150180, 0xc42014a058, 0x13, 0xc420559720, 0x3, 0x3, 0xa47498, 0xc420559768, 0xa47498)
            /data/service/go1.8.3/src/reflect/value.go:302 +0xa4
    net/rpc.(*service).call(0xc4201463c0, 0xc420069840, 0xc42c455398, 0xc42014c380, 0xc42b5f9f80, 0x79d540, 0xc56a104f20, 0x197, 0x7bf840, 0xc461dc7710, ...)
            /data/service/go1.8.3/src/net/rpc/server.go:387 +0x144
    created by net/rpc.(*Server).ServeCodec
            /data/service/go1.8.3/src/net/rpc/server.go:481 +0x404
    2018/08/20 22:00:17 cfg.go:90: read config file: /data/service/of_service/judge.2/config/cfg.json successfully
    2018/08/20 22:00:17 http.go:60: http listening 0.0.0.0:7081
    2018/08/20 22:00:17 rpc.go:24: rpc listening 0.0.0.0:7080
    2018/08/20 22:00:18 judge.go:47: [ERROR] parse func pdiff(#2) fail: strconv.Atoi: parsing "2\xef\xbc": invalid syntax. strategy id: 2397
    
  • 急求falcon部分常见繁琐问题处理办法!!!

    急求falcon部分常见繁琐问题处理办法!!!

    各位大大好:

    1、目前alarm删除case的方式一次性十行还是不够效率,我想在mysql数据库中的alarm_case 直接删除数据,是否安全?副作用?

    2、服务器目前几千台,部分hostname变化或人为修改,导致太多垃圾endpoint,是否有自动清理或者手动清理endpoint无效值的方法。

  • Add arm64 support

    Add arm64 support

    Hi,

    I am checking on adding arm64 jobs in travis.

    I was trying to build the docker image for arm64 but it was failing with below error

    standard_init_linux.go:211: exec user process caused "exec format error"
    

    This package uses openfalcon/makegcc-golang:1.10-alpine as base image for building docker images which are not available for arm64. I am unable to find this image and its Dockerfile to add support for arm64.

    Travis Link for arm64 : https://travis-ci.com/github/odidev/falcon-plus/jobs/382607644

    I want to know whether you are planning on adding arm64 support and can I get any suggestions on adding arm64 support to Dockerfile and uploading it to docker hub?

  • /api/v1/plugin 信息错误

    /api/v1/plugin 信息错误

    rra=requests.post(url+'/api/v1/plugin',headers=falcon_header,data={'hostgroup_id':40, 'dir_path':'dgb'})
    
    rra.content: 
    b'{"error":"Key: \'APICreatePluginInput.GrpId\' Error:Field validation for \'GrpId\' failed on the \'required\' tag\\nKey: \'APICreatePluginInput.DirPaht\' Error:Field validation for \'DirPaht\' failed on the \'required\' tag"}\n'
    
    rra=requests.post(url+'/api/v1/plugin',headers=falcon_header,data={'GrpId':40, 'DirPaht':'dgb'})
    
    rra.content:
    b'{"id":37,"grp_id":40,"dir":"dgb","create_user":"xieguangtai"}\n'
    

    你们的api网站上可不是这么说的 http://api.open-falcon.com/#/plugin_create

    Request
    {
      "hostgroup_id": 343,
      "dir_path": "testpath"
    }
    
    Response
    Status: 200
    {
      "id": 1501,
      "grp_id": 343,
      "dir": "testpath",
      "create_user": "root"
    }
    
  • 编译模块graph时报错 -std=gnu99/c99

    编译模块graph时报错 -std=gnu99/c99

    MacOs 11.1 运行make graph,出现下边的结果 if [ graph = "gateway" ]; then
    go build -ldflags "-X main.BinaryName=gateway -X main.GitCommit=git rev-parse --short HEAD -X main.Version=0.3.x"
    -o bin/gateway/falcon-gateway ./modules/transfer ;
    else
    go build -ldflags "-X main.BinaryName=graph -X main.GitCommit=git rev-parse --short HEAD -X main.Version=0.3.x"
    -o bin/graph/falcon-graph ./modules/graph ;
    fi

    github.com/open-falcon/falcon-plus/vendor/github.com/open-falcon/rrdlite

    rrd_create.c:161:15: error: implicit declaration of function 'strdup' is invalid in C99 [-Werror,-Wimplicit-function-declaration] rrd_create.c:161:15: note: did you mean 'strcmp'? /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/string.h:77:6: note: 'strcmp' declared here rrd_create.c:161:13: warning: incompatible integer to pointer conversion assigning to 'char *' from 'int' [-Wint-conversion] rrd_create.c:162:12: error: implicit declaration of function 'strtok_r' is invalid in C99 [-Werror,-Wimplicit-function-declaration] rrd_create.c:162:12: note: did you mean 'strtok'? /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/string.h:90:7: note: 'strtok' declared here rrd_create.c:162:10: warning: incompatible integer to pointer conversion assigning to 'char *' from 'int' [-Wint-conversion] rrd_create.c:409:11: warning: incompatible integer to pointer conversion assigning to 'char *' from 'int' [-Wint-conversion] make: *** [graph] Error 2

  • graph缩容

    graph缩容

    graph缩容是否可以这样配置transfer :将下掉的机器配置成双打 下掉前 "cluster": { "graph-00": "要下掉的机器1:6070,存在机器1:6070", "graph-01": "要下掉的机器2:6070,存在机器2:6070", "graph-03": "存在机器1:6070", "graph-04": "存在机器2:6070", } 下掉后 "cluster": { "graph-00": "存在机器1:6070", "graph-01": "存在机器2:6070", "graph-03": "存在机器1:6070", "graph-04": "存在机器2:6070", }

  • Integrate pull request preview environments

    Integrate pull request preview environments

    I would like to support Falcon by implementing Uffizzi preview environments. Disclaimer: I work on Uffizzi.

    Uffizzi is a Open Source full stack previews engine and our platform is available completely free for Falcon (and all open source projects). This will provide maintainers with preview environments of every PR in the cloud, which enables faster iterations and reduces time to merge. You can see the open source repos which are currently using Uffizzi over here

    Uffizzi is purpose-built for the task of previewing PRs and it integrates with your workflow to deploy preview environments in the background without any manual steps for maintainers or contributors.

    We can go ahead and create an Initial PoC for you right away if you think there is value in this proposal.

    TODO:

    • [ ] Intial PoC

    cc @waveywaves

  • 进程监控获取数据异常

    进程监控获取数据异常

    proc.num/name=StandaloneSessionClusterEntrypoint

    监控模版是这样设置的,name名称是 java jps命令提出出来的 覆盖/proc/$pid/cmdline 但是获取的值一直是0,而且关掉进程后, 数值还在获取 这是什么原因呢 我用了 nginx 简单的进程 也是这个问题

  • 请教大佬Centos8部署falcon-agent启动后报错。无法启动采集agent。日志报错内容如下,网上看有人遇到同样问题,说不支持centos8

    请教大佬Centos8部署falcon-agent启动后报错。无法启动采集agent。日志报错内容如下,网上看有人遇到同样问题,说不支持centos8

    2022/02/26 17:01:37 cfg.go:142: read config file: cfg.json successfully 2022/02/26 17:01:37 http.go:88: listening :1988 panic: runtime error: index out of range

    goroutine 15 [running]: github.com/open-falcon/falcon-plus/vendor/github.com/toolkits/nux.SocketStatSummary(0xc000030000, 0x50, 0x48) /home/work/gopath/src/github.com/open-falcon/falcon-plus/vendor/github.com/toolkits/nux/ss_s.go:49 +0x766 github.com/open-falcon/falcon-plus/modules/agent/funcs.SocketStatSummaryMetrics(0x0, 0x0, 0x0) /home/work/gopath/src/github.com/open-falcon/falcon-plus/modules/agent/funcs/sockstat.go:24 +0x37 github.com/open-falcon/falcon-plus/modules/agent/cron.collect(0x3c, 0xc0000fe1c0, 0x2, 0x2) /home/work/gopath/src/github.com/open-falcon/falcon-plus/modules/agent/cron/collector.go:63 +0x14d created by github.com/open-falcon/falcon-plus/modules/agent/cron.Collect /home/work/gopath/src/github.com/open-falcon/falcon-plus/modules/agent/cron/collector.go:44 +0xbf

Nightingale - A Distributed and High-Performance Monitoring System. Prometheus enterprise edition
Nightingale - A Distributed and High-Performance Monitoring System. Prometheus enterprise edition

Introduction ?? A Distributed and High-Performance Monitoring System. Prometheus

Jan 7, 2022
Open source framework for processing, monitoring, and alerting on time series data

Kapacitor Open source framework for processing, monitoring, and alerting on time series data Installation Kapacitor has two binaries: kapacitor – a CL

Dec 26, 2022
Open Source Software monitoring platform tools.

ByteOpen Open Source Software monitoring platform tools. Usage Clone the repo to your own go src path cd ~/go/src git clone https://code.byted.org/inf

Nov 21, 2021
Monitoring-go - A simple monitoring tool to sites of MOVA

Monitoring GO A simple monitoring tool to sites of MOVA How to use Clone Repo gi

Feb 14, 2022
Distributed simple and robust release management and monitoring system.
Distributed simple and robust release management and monitoring system.

Agente Distributed simple and robust release management and monitoring system. **This project on going work. Road map Core system First worker agent M

Nov 17, 2022
An Open Source video surveillance management system for people making this world a safer place.
An Open Source video surveillance management system for people making this world a safer place.

Kerberos Open Source Docker Hub | Documentation | Website Kerberos Open source (v3) is a cutting edge video surveillance management system made availa

Dec 30, 2022
The Prometheus monitoring system and time series database.

Prometheus Visit prometheus.io for the full documentation, examples and guides. Prometheus, a Cloud Native Computing Foundation project, is a systems

Dec 31, 2022
A system and resource monitoring tool written in Golang!
A system and resource monitoring tool written in Golang!

Grofer A clean and modern system and resource monitor written purely in golang using termui and gopsutil! Currently compatible with Linux only. Curren

Jan 8, 2023
checkah is an agentless SSH system monitoring and alerting tool.

CHECKAH checkah is an agentless SSH system monitoring and alerting tool. Features: agentless check over SSH (password, keyfile, agent) config file bas

Oct 14, 2022
Cloudprober is a monitoring software that makes it super-easy to monitor availability and performance of various components of your system.

Cloudprober is a monitoring software that makes it super-easy to monitor availability and performance of various components of your system. Cloudprobe

Dec 30, 2022
rtop is an interactive, remote system monitoring tool based on SSH

rtop rtop is a remote system monitor. It connects over SSH to a remote system and displays vital system metrics (CPU, disk, memory, network). No speci

Dec 30, 2022
distributed monitoring system
distributed monitoring system

OWL OWL 是由国内领先的第三方数据智能服务商 TalkingData 开源的一款企业级分布式监控告警系统,目前由 Tech Operation Team 持续开发更新维护。 OWL 后台组件全部使用 Go 语言开发,Go 语言是 Google 开发的一种静态强类型、编译型、并发型,并具有垃圾回

Dec 24, 2022
Gowl is a process management and process monitoring tool at once. An infinite worker pool gives you the ability to control the pool and processes and monitor their status.
Gowl is a process management and process monitoring tool at once. An infinite worker pool gives you the ability to control the pool and processes and monitor their status.

Gowl is a process management and process monitoring tool at once. An infinite worker pool gives you the ability to control the pool and processes and monitor their status.

Nov 10, 2022
Open Source Supreme Monitor Based on GoLang

Open Source Supreme Monitor Based on GoLang A module built for personal use but ended up being worthy to have it open sourced.

Nov 4, 2022
An open source Pusher server implementation compatible with Pusher client libraries written in GO

Try browsing the code on Sourcegraph! IPÊ An open source Pusher server implementation compatible with Pusher client libraries written in Go. Why I wro

Jan 3, 2023
Cloudinsight Agent is a system tool that monitors system processes and services, and sends information back to your Cloudinsight account.

Cloudinsight Agent 中文版 README Cloudinsight Agent is written in Go for collecting metrics from the system it's running on, or from other services, and

Nov 3, 2022
A flexible process data collection, metrics, monitoring, instrumentation, and tracing client library for Go
A flexible process data collection, metrics, monitoring, instrumentation, and tracing client library for Go

Package monkit is a flexible code instrumenting and data collection library. See documentation at https://godoc.org/gopkg.in/spacemonkeygo/monkit.v3 S

Dec 14, 2022
A GNU/Linux monitoring and profiling tool focused on single processes.
A GNU/Linux monitoring and profiling tool focused on single processes.

Uroboros is a GNU/Linux monitoring tool focused on single processes. While utilities like top, ps and htop provide great overall details, they often l

Dec 26, 2022
Simple and extensible monitoring agent / library for Kubernetes: https://gravitational.com/blog/monitoring_kubernetes_satellite/

Satellite Satellite is an agent written in Go for collecting health information in a kubernetes cluster. It is both a library and an application. As a

Nov 10, 2022