Crocodile Distributed Task Scheduling System
English | 中文
Introduction
A distributed task scheduling system based on Golang that supports http requests, run golang、python、python3、shell、bat
Screenshot
Click Me
+----------+
+-------------+ ||--------||
||-----------|| || ||
|| || || Worker ||
|| 调度中心 || || ||
|| || RPC调用(gRPC) ||---------||
||-----------|| +-----------------------> ||-------- ||
||-----------|| || ||
|| || <-----------------------+ || Worker ||
+---> || 调度中心 || 任务实时日志 || ||
任 | || || ||--------||
务 | ||-----------|| <------+ ||--------||
完 | +------------+ | || ||
成 | ^ | | || Worker ||
持 | 实|任 | |数 || ||
久 | 时|务 |获 |据 +----------+
日 | 志|状 |取 |存
志 | 日|态 |锁 |储
| v v v
| +-------+-+ +----+---+
| | | | |
+----- | Redis | | MySQL |
| | | |
+---------+ +--------+
Features
- Add, modify, delete, clone and run tasks on the web node
- View real-time logs of running tasks and tasks in real time, and can terminate running tasks
- Multiple task types:
- Performhttprequest tasks
- Runshell,python,golang、python3、nodejscodes (of course other languages can also be supported, please provide if necessary) - Father and child tasks:
When the parent task or child task is set, theparent task->main task->subtaskwill be run first. If any task fails, the entire process will be interrupted immediately. You can also set the parent task or subtaskserialrun - Scheduling Algorithm:
Supports four scheduling algorithms: random, rotation, worker weight, minimum worker number to call Worker to run tasks, - Custom alarm strategy:
Can be set to alert multiple users when the tasksuccess,failure, orrun is completed
Set the return code or return content of the task to compare whether the actual return code or return content of the task is the same to determine whether the task runs successfully. - Host group:
A task can only be bound to any host group. The running of the task will select a task in this host group to run the task through the task's routing strategy - Host:
A host group can be bound to multiple hosts, the host is the node that actually runs the task, and the dispatch center automatically discovers after registration - security strategy
Certificate encryption encrypts communication data Access token - Task log management, clean up logs
- Alarm notification support platform
- DingDing
- Slack Channel
- Telegram Bot
- WebHook URL
- Detailed task audit function
All operations on data changes to users will be recorded - Permission control
- Administrator
Have all operating rights - general user
New tasks and host groups can be created, but only tasks or host groups that have been created can be operated, but audit records and all users cannot be viewed - Guest
Only viewing authority, no operation modification authority, but can not view audit records, all users
- Administrator
Supported platforms
- Linux
- Mac
- Windows
Quick Start
git clone https://github.com/labulaka521/crocodile
cd crocodile
docker-compose up -d
Then open urlhttp://yourip:8080,you need to input a admin user and password to install it, than you will come in this system
Running
- Click to download,uncompress it
- install
redis、mysql,then change config filecore.toml - If you need to enable certificate authentication between scheduling nodes, please generate a certificate,
Run the command to generate cert certificatecrocodile cert
Then two files cert.pem and key.pem will be generated locally in the current directory. After saving these two files, fill in the path of the file in the value configuration file. Each node needs these two files
-
Run as a scheduler center
Multiple dispatch centers can be started to prevent single points of failure from causing dispatch to hang up./crocodile server -c core.toml
-
Run as a Worker (host) node
/crocodile client -c core.toml -
View version compilation information
./crocodile version
Development
- Frontend
- Install
yarn - Go to the web directory, first download the dependency
yarn, and then run the front-endyarn run devseparately - Package to go, execute
make frontrnd,
- Install
- Backend
- Run
make runsas a schdeule center - Run
make runcas worker node
- Run
The sql directory is also packaged in
go-bindata. During installation, the sql creation statement will be read from the file generated bygo-bindata. If the sql is modified, re-executemake bin-data
Doc
Donate
If this project could help you, I hope you could donate me by alipay
License
Crocodile is under the MIT license. See the LICENSE file for details.
你好，我按github 上的操作，主机列表还是空的，要怎样才能添加主机
shell 脚本异常 应该是通过exit xx 来返回错误码 吧？
