Документация на русском: https://github.com/eltaline/ctrl/blob/master/README-RUS.md
cTRL is a server written in Go language that uses a modified version of the NutsDB database, as a backend for a continuous queue of tasks and saving the result of executing commands from given tasks in command interpreters like /bin/bash on servers where this service will be used. Using cTRL, you can receive tasks via the HTTP protocol with commands for executing them on the server and limit the number of simultaneously executed tasks.
Current stable version: 1.1.10
Features
- Multi threading
- Supports HTTPS and Auth/IP authorization
- Supported HTTP methods: GET, POST
- Parallel execution of tasks in realtime
- Receiving tasks in the queue for deferred parallel execution of commands
- Limiting the maximum number of threads per virtual host
- Supports continuous task execution queue
- Automatic simultaneous execution of only one identical task
- Supported interpreters: /bin/bash, /bin/sh
- Supported formats: JSON
Requirements
- Operating Systems: Linux, BSD, Solaris, OSX
- Architectures: amd64, arm64, ppc64 and mips64, with only amd64 tested
Real application
We use this server to perform delayed tasks for parallel video conversion using ffmpeg with limiting processes on several video servers with Nvidia GPU, but the server can execute any commands through the shell interpreters. This is a first release.
Documentation
Installation
Install packages or binaries
systemctl enable ctrl && systemctl start ctrl
Configuring and using cTRL server
Default credentials: admin:swordfish
Password generation: echo -n "mypassword" | sha512sum
In most cases it is enough to use the default configuration file. A full description of all product parameters is available here: Options
This guide uses UUIDs. But the client can set task identifiers in any format.
General methods
Run a task or task list in realtime with pending execution
curl -X POST -H "Auth: login:pass" -H "Content-Type: application/json" -d @task.json http://localhost/run
Queuing a task or task list
curl -X POST -H "Auth: login:pass" -H "Content-Type: application/json" -d @task.json http://localhost/task
Getting a task from the received queue
curl -H "Auth: login:pass" "http://localhost/show?key=777a0d24-289e-4615-a439-0bd4efab6103&type=mytype&queue=received"
Getting all tasks from the received queue
curl -H "Auth: login:pass" "http://localhost/show?queue=received"
Getting a task from the working queue
curl -H "Auth: login:pass" "http://localhost/show?key=777a0d24-289e-4615-a439-0bd4efab6103&type=mytype&queue=working"
Getting all tasks from the working queue
curl -H "Auth: login:pass" "http://localhost/show?queue=working"
Getting a task from the list of completed tasks
curl -H "Auth: login:pass" "http://localhost/show?key=777a0d24-289e-4615-a439-0bd4efab6103&type=mytype&queue=completed"
Getting all tasks from the list of completed tasks
curl -H "Auth: login:pass" "http://localhost/show?queue=completed"
Deleting a task from the received queue
curl -H "Auth: login:pass" "http://localhost/del?key=777a0d24-289e-4615-a439-0bd4efab6103&type=mytype&queue=received"
Deleting all tasks from the received queue
curl -H "Auth: login:pass" "http://localhost/del?queue=received"
Deleting a task from the working queue
curl -H "Auth: login:pass" "http://localhost/del?key=777a0d24-289e-4615-a439-0bd4efab6103&type=mytype&queue=working"
Deleting all tasks from the working queue
curl -H "Auth: login:pass" "http://localhost/del?queue=working"
Deleting a task from the list of completed tasks
curl -H "Auth: login:pass" "http://localhost/del?key=777a0d24-289e-4615-a439-0bd4efab6103&type=mytype&queue=completed"
Deleting all tasks from the list of completed tasks
curl -H "Auth: login:pass" "http://localhost/del?queue=completed"
Input format
Description of fields
- key - an arbitrary unique identifier (colon is not allowed ":")
- type - arbitrary type of task (colon is not allowed ":")
- path - path to change the directory before running the command
- lock - arbitrary lock label
- command - command
- threads - the number of threads for a particular type of task
- timeout - timeout
- ttltime - task lifetime in completed queue
- interval - the interval between starting tasks
- repeaterr - enumeration of errors that require the repeated execution of a task
- repeatcnt - the number of task reruns in case of any error from parameter
repeaterr
- interr - enumeration of errors that require kill running task
- intcnt - the number of task reruns in case of any error from parameter
interr
- lookout - enable or disable reaction on errors from stdout
- replace - overwrite the same task with the same key and the same type in the received queue
Fields threads/ttltime/interval/repeaterr/repeatcnt/interr/intcnt/lookout/replace
are relevant only for tasks queued
Examples of setting tasks
Example for one task through /run:
[
{"key":"777a0d24-289e-4615-a439-0bd4efab6103","type":"mytype","path":"/","lock":"mylock1","command":"echo \"hello\" && logger \"hello\" && sleep 5","timeout":15}
]
Example for a task list through /run:
[
{"key":"777a0d24-289e-4615-a439-0bd4efab6103","type":"mytype","path":"/","lock":"mylock1","command":"echo \"hello\" && logger \"hello\" && sleep 5","timeout":15},
{"key":"4964deca-46ff-413f-8a92-e5baefd328e7","type":"mytype","path":"/","lock":"mylock2","command":"echo \"great\" && logger \"great\" && sleep 30","timeout":15},
{"key":"3fdf744d-36f1-499d-bd39-90a004ee39f6","type":"mytype","path":"/","lock":"mylock3","command":"echo \"world\" && logger \"world\" && sleep 15","timeout":15}
]
Example for one task through /task:
[
{"key":"777a0d24-289e-4615-a439-0bd4efab6103","type":"mytype","path":"/","lock":"mylock1","command":"echo \"hello\" && logger \"hello\" && sleep 5","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false}
]
Example for a task list through /task:
[
{"key":"777a0d24-289e-4615-a439-0bd4efab6103","type":"mytype","path":"/","lock":"mylock1","command":"echo \"hello\" && logger \"hello\" && sleep 5","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false},
{"key":"4964deca-46ff-413f-8a92-e5baefd328e7","type":"mytype","path":"/","lock":"mylock2","command":"echo \"great\" && logger \"great\" && sleep 30","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false},
{"key":"3fdf744d-36f1-499d-bd39-90a004ee39f6","type":"mytype","path":"/","lock":"mylock3","command":"echo \"world\" && logger \"world\" && sleep 15","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false}
]
Output format
Description of fields
- key - an arbitrary unique identifier
- time - task receive time
- type - arbitrary type of task
- path - path to change the directory before running the command
- lock - arbitrary lock label
- command - command
- threads - the number of threads for a particular type of task
- timeout - timeout
- ttltime - task lifetime in completed queue
- interval - the interval between starting tasks
- repeaterr - enumeration of errors that require the repeated execution of a task
- repeatcnt - the number of task reruns in case of any error from parameter
repeaterr
- interr - enumeration of errors that require kill running task
- intcnt - the number of task reruns in case of any error from parameter
interr
- lookout - enable or disable reaction on errors from stdout
- replace - overwrite the same task with the same key and the same type in the received queue
- stdcode - not used at the moment
- stdout - standard output
- errcode - error code
- stderr - standard error output
- runtime - runtime in milliseconds
- delcode - error code when deleting a task
- delerr - error message when deleting a task
Output example
Output of completed tasks:
[
{"key":"777a0d24-289e-4615-a439-0bd4efab6103","time":1589737139,"type":"mytype","path":"/","lock":"mylock1","command":"echo \"hello\" && logger \"hello\" && sleep 5","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false,"stdcode":0,"stdout":"hello\n","errcode":0,"stderr":"","runtime":10010.669069},
{"key":"4964deca-46ff-413f-8a92-e5baefd328e7","time":1589737139,"type":"mytype","path":"/","lock":"mylock2","command":"echo \"great\" && logger \"great\" && sleep 30","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false,"stdcode":0,"stdout":"great\n","errcode":124,"stderr":"signal: killed","runtime":15006.034832},
{"key":"3fdf744d-36f1-499d-bd39-90a004ee39f6","time":1589737139,"type":"mytype","path":"/","lock":"mylock3","command":"echo \"world\" && logger \"world\" && sleep 15","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false,"stdcode":0,"stdout":"world\n","errcode":0,"stderr":"","runtime":15019.839685}
]
Output of deleted tasks:
[
{"key":"777a0d24-289e-4615-a439-0bd4efab6103","time":1589737139,"type":"mytype","path":"/","lock":"mylock1","command":"echo \"hello\" && logger \"hello\" && sleep 5","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false,"stdcode":0,"stdout":"hello\n","errcode":0,"stderr":"","runtime":10010.669069,"delcode":0,"delerr":""},
{"key":"4964deca-46ff-413f-8a92-e5baefd328e7","time":1589737139,"type":"mytype","path":"/","lock":"mylock2","command":"echo \"great\" && logger \"great\" && sleep 30","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false,"stdcode":0,"stdout":"great\n","errcode":124,"stderr":"signal: killed","runtime":15006.034832,"delcode":0,"delerr":""},
{"key":"3fdf744d-36f1-499d-bd39-90a004ee39f6","time":1589737139,"type":"mytype","path":"/","lock":"mylock3","command":"echo \"world\" && logger \"world\" && sleep 15","threads":4,"timeout":15,"ttltime":3600,"interval":1,"repeaterr":["CUDA_ERROR_OUT_OF_MEMORY","OtherError"],"repeatcnt":3,"interr":["Generic error in an external library","OtherError"],"intcnt":1,"lookout":false,"replace":false,"stdcode":0,"stdout":"world\n","errcode":0,"stderr":"","runtime":15019.839685,"delcode":0,"delerr":""}
]
Notes and Q&A
- In key and type, colon is not allowed ":"
- The limitation of simultaneously running tasks from queue to each virtual host is regulated by the
vthreads
parameter in the server configuration file or can be overriden through POST method for each type of task - The limitation of simultaneously running tasks in realtime to each virtual host is regulated by the
rthreads
parameter in the server configuration file - The key field, if this identifier is the same for two or more different tasks, in this case, when outputting information from the queue, you will receive information on this identifier for several tasks at once, this can be useful for grouping tasks, but they will be from the waiting queue run randomly
- The type and lock fields, if they are assigned to two or more different tasks, are absolutely identical, in which case the server will perform these tasks from the wait queue in an arbitrary order, but only in turn
- The type and lock fields set in real time do not matter, but they are required, all tasks will be performed in parallel mode whenever possible.
- To sequentially execute a list of specific commands related to each other through a waiting queue, install these commands in one task, separated by && or write a shell script
- Tasks performed in real time are executed parallel.
Error Codes
errcode
- 0 (no error)
- 1 (any error)
- 124 (timeout)
- 137 (out of memory)
- 255 (failed to start command)
delcode
- 0 (no error)
- 1 (any error)
Todo
- Any suggestions
Parameters
A full description of all product parameters is available here: Options
HTTP Core
Uses Iris as server http core
Guarantees
No warranty is provided for this software. Please test first
Contacts
- Company website: Eltaline
Copyright © 2020 Andrey Kuvshinov. Contacts: <[email protected]>
Copyright © 2020 Eltaline OU. Contacts: <[email protected]>
All rights reserved.