A reverse engineered github actions compatible self-hosted runner using nektos/act to execute your workflow steps

github-act-runner

CI awesome-runners

A reverse engineered github actions compatible self-hosted runner using nektos/act to execute your workflow steps. Unlike the official actions/runner, this works on more systems like freebsd.

Usage

Dependencies

Actions Type Host JobContainer (only Linux, Windows, macOS or Openbsd)
(composite) run steps bash or explicit shell in your PATH (prior running the runner) Docker (*1), bash or explicit shell in your PATH (inside your container image)
nodejs actions node (*2) in your PATH (prior running the runner) Docker (*1), node (*2) in your PATH (inside your container image)
docker actions Not available Docker (*1)
service container Not available Not available
composite actions with uses v0.1.0 v0.1.0
composite actions with if v0.1.0 v0.1.0
composite actions with continue-on-error v0.1.0 v0.1.0

Docker Daemon via DOCKER_HOST

(*1) Reachable docker daemon use DOCKER_HOST to specify a remote host.

NodeJS via PATH

(*2) For best compatibility with existing nodejs actions, please add nodejs in version 12 to your PATH, newer nodejs versions might lead to workflow failures.

Usage for github releases

Follow the instruction of https://github.com/ChristopherHX/github-act-runner/releases/latest.

Usage for debian repository

Add debian repository

/etc/apt/sources.list.d/github-act-runner.list file:

deb http://gagis.hopto.org/repo/chrishx/deb all main

Import repository public key

curl -sS http://gagis.hopto.org/repo/chrishx/pubkey.gpg | sudo tee -a /etc/apt/trusted.gpg.d/chrishx-github-act-runner.asc

Install the runner

sudo apt update
sudo apt install github-act-runner

Add new runner

github-act-runner new --url 
   
     --name 
    
      --labels 
     
       --token 
      
     
    
   

where

  • - github repository (e.g. https://github.com/user/repo), organization (e.g. https://github.com/organization) or enterprise URL
  • - choose a name for your runner
  • - comma-separated list of labels, e.g. label1,label2. Optional.

The new runner will be registered and started as background service.

See help:

github-act-runner --help

For more info about managing runners.

Usage from source

You need at least go 1.16 to use this runner from source. Some targets fail to build with go 1.17.

Getting Source

git clone https://github.com/ChristopherHX/github-act-runner.git --recursive

Update Source

git pull
git submodule update

Configure

go run . configure --url 
   
     --name 
    
      -l label1,label2 --token 
     

     
    
   

E.g. https://github.com/ChristopherHX/github-act-runner for this repo

E.g. Test

You find the token in
Repository /settings/actions/runners/new
Organization /organizations/ /settings/actions/runners/new
Enterprise In action runner settings of your enterprise

E.g. AWWWWWWWWWWWWWAWWWWWWAWWWWWWW

Labels

Replace label1,label2 with a custom list of runner labels.

Run

go run . run

Known Limitations

  • This runner ignores pre and post steps of javascript actions
  • actions/cache is incompatible and won't be able to save your cache
  • You won't be able to run steps after a failure without using continue-on-error: true
  • The expression interpreter of this runner doesn't always behave like you would expect and you might see errors or other expressions which shouldn't work are working here ( based on javascript, not on actions/runner )
  • Most issues of https://github.com/nektos/act/issues applies to this runner as well

How does it work?

This runner implements the same protocol as the actions/runner in a different way, as such it can be used as a self-hosted runner exactly like the official one. To get this working, I initially built an actions service replacement ChristopherHX/runner.server for the official actions/runner. My own actions service allowed me to implement the base protocol for this runner and debug how the protocol is serializeing and parsing json messages, while still being incompatible with github. After testing against github, the first thing happend was loosing the ability to run any github action workflows on my test repository. My invalid attempts to register a custom runner caused unrecoverable Internal Server Errors on githubs side, I decided to delete this test repository. After some work everything worked and finally it is safe to register this runner against github. To execute steps this runner translates the github actions job request to be compatible with a modified version of nektos/act ( ChristopherHX/act ), which adds a local task runner without the need for docker and increased platform support, also the log output of act gets redirected to github for live logs and storing log files.

Does this runner work without github?

Yes, you can use this runner together with ChristopherHX/runner.server locally on your PC without depending on compatibility with github. Also CI tests for this runner are using ChristopherHX/runner.server, this avoids requiring a PAT for github to run tests and enshures that you are always able to run it locally without github.

Comments
  • Trouble creating a Daemonized Version on FreeBSD

    Trouble creating a Daemonized Version on FreeBSD

    Hey there! First of many thanks for this tool. We need to have our builds run on FreeBSD.

    However when we try to build the runner via go build and try to create a service / daemonized version of it, we run into this error. Do you have any suggestions on how to build a self contained executable other than go build? We have already tried many different avenues but our Go knowledge is limited and we figured you might be the best source of information.

    Cheers

    he worker panicked with message: runtime error: invalid memory address or nil pointer dereference
    goroutine 70 [running]:
    runtime/debug.Stack()
            /usr/local/go/src/runtime/debug/stack.go:24 +0x9f
    main.runJob.func1.6(0xc000523580)
            /usr/local/github_runner/main.go:1107 +0x95
    panic({0xb17740, 0x10a1420})
            /usr/local/go/src/runtime/panic.go:1052 +0x2fe
    github.com/nektos/act/pkg/runner.(*StepContext).setupEnv(0xc00007e6e0, {0xcbf888, 0xc000235650})
            /usr/local/github_runner/act/pkg/runner/step_context.go:182 +0x80
    github.com/nektos/act/pkg/runner.(*RunContext).newStepExecutor.func1({0xcbf888, 0xc000235650})
            /usr/local/github_runner/act/pkg/runner/run_context.go:417 +0x2aa
    github.com/nektos/act/pkg/common.Executor.Then.func1({0xcbf888, 0xc000235650})
            /usr/local/github_runner/act/pkg/common/executor.go:145 +0x2b8
    github.com/nektos/act/pkg/common.Executor.Then.func1({0xcbf888, 0xc000235650})
            /usr/local/github_runner/act/pkg/common/executor.go:133 +0x46
    github.com/nektos/act/pkg/common.Executor.Then.func1({0xcbf888, 0xc000235650})
            /usr/local/github_runner/act/pkg/common/executor.go:133 +0x46
    github.com/nektos/act/pkg/common.Executor.Then.func1({0xcbf888, 0xc000235650})
            /usr/local/github_runner/act/pkg/common/executor.go:133 +0x46
    github.com/nektos/act/pkg/common.Executor.Finally.func1({0xcbf888, 0xc000235650})
            /usr/local/github_runner/act/pkg/common/executor.go:185 +0x93
    github.com/nektos/act/pkg/common.Executor.Finally.func1({0xcbf888, 0xc000235650})
            /usr/local/github_runner/act/pkg/common/executor.go:185 +0x93
    github.com/nektos/act/pkg/common.Executor.If.func1({0xcbf888, 0xc000235650})
            /usr/local/github_runner/act/pkg/common/executor.go:153 +0x6a
    main.runJob.func1(0xc000230304, 0xc00019b4d0, 0xc000532680, 0xc000532670, {0x1e, {0xc0005589f0, 0x17}, {0xc000558a50, 0x18}, {0xc000310000, ...}}, ...)
            /usr/local/github_runner/main.go:1654 +0x441c
    created by main.runJob
            /usr/local/github_runner/main.go:937 +0x16c```
    
    
  • latest versions of the runner doesn't run commands as from terminal

    latest versions of the runner doesn't run commands as from terminal

    Some time ago the runner was running step commands as if from terminal, this produced a nice colored output, for example see here: https://github.com/cppfw/utki/runs/3079205208?check_suite_focus=true#step:9:59

    But latest version of the runner doesn't do that anymore: https://github.com/cppfw/utki/runs/3106285086?check_suite_focus=true#step:9:60 I.e. test -t 1 doesn't return 0. Was this changed intentionally? Are there problems with running commands as from terminal?

  • Create convenient script for controlling runners and their systemd services

    Create convenient script for controlling runners and their systemd services

    Fix #17

    So, here is the implementation of systemd services control script.

    The runner executable itself is installed to /usr/bin/github-act-runner/runner, so that it is not supposed to be run manually by user. User is supposed to use the /usr/bin/github-act-runner.sh script for managing runners.

    When new runner is added:

    • it's configuration is saved in ~/.config/github-act-runner/runners/<runner-id>/ directory.
    • a systemd service is created and started for that runner
    • a file-changes-watcher service (*.restarter.service and *.path units) is added on /usr/bin/github-act-runner/runner binary. So that, when the binary is changed (e.g. due to upgrade) the corresponding runner service is restarted to use the updated binary. In case the runner package is uninstalled the service files and configs remain, but the service is moved to stopped state. See new restart.sh script.

    Runner id is composed from runner name given by user and github user/repo or org_name, e.g. my_new_runner.user.repo/my_new_runner.org_name.

    Runners can be created as root user as well as non-root user. Because systemctl starting from Debian buster supports --user option and creating and running services locally for each user, as well as ordinary system-wide services.

    This pull request is in Draft state because it needs #18 to be fixed. Otherwise, in case configuration fails it still continues to creating systemd service files and the created runner remains in unusable state. Though it can be removed by rm command of the script.

    When #18 is fixed the PR can be merged, unless no other review problems found.

  • Runner receivec SIGINT spontaniously

    Runner receivec SIGINT spontaniously

    I'm currently running 2 runners on 2 odroid xu4 boards, version 0.0.5.

    One of the runners works fine so far.

    The other one, for some reason receives SIGINT from somewhere. After starting, it may pick up the job, work for couple of minutes and then gets SIGINT:

    github@xu4r1:~$ github-act-runner log xu4r1
    running as user 'github'
    Jul 20 13:43:51 xu4r1 systemd[7135]: Started github-act-runner '/xu4r1'.
    Jul 20 13:43:53 xu4r1 runner[7304]: Listening for Jobs
    Jul 20 13:43:58 xu4r1 runner[7304]: Running Job 'linux - igagis/raspbian:buster'
    Jul 20 13:43:58 xu4r1 runner[7304]: Failed to upload timeline with Status 404
    Jul 20 13:44:16 xu4r1 systemd[7135]: Stopping github-act-runner '/xu4r1'...
    Jul 20 13:44:16 xu4r1 runner[7304]: CTRL+C received, stopping accepting new jobs
    Jul 20 13:44:16 xu4r1 runner[7304]: Canceled stopping
    Jul 20 13:44:20 xu4r1 runner[7304]: Finished Job 'linux - igagis/raspbian:buster' with result: Canceled
    Jul 20 13:44:20 xu4r1 systemd[7135]: github-act-runner.xu4r1.service: Succeeded.
    Jul 20 13:44:20 xu4r1 systemd[7135]: Stopped github-act-runner '/xu4r1'.
    

    I will do more observations.

  • Add deployment to debian repositories

    Add deployment to debian repositories

    So far it only packs the binary into the debian package.

    Next thing would be to think about how to automatically install the runner as systemd service and provide configuration scripts.

    In order the deployment to my debian repo server work you will need to add MYCI_REPO_SSH_KEY secret to your github repo with the ssh private key. If you tell me your email, I'd send you the private key you need to put there. Or check secrets of my forked repo, github tells me you should have access.

    Instructions regarding the repo added to the README.

  • Support of github secrets

    Support of github secrets

    Does the runner support github secrets? I'm getting some weird error when trying to use my secrets.MYCI_REPO_SSH_KEY secret, see run: https://github.com/cppfw/utki/runs/3072012329?check_suite_focus=true

  • No cleanup if the runner terminates

    No cleanup if the runner terminates

    If the runner exits in the middle of running a job, GitHub doesn't notice and then refuses to schedule more jobs on the same runner until the timeout (6 hours, by default). I don't know if there's anything in the protocol that lets a runner say 'I've just restarted, the last job that you think I'm doing failed'.

    Note: This may be my fault because nothing in the runner's filesystem is persisted between invocations. If there's something that I need to preserve, let me know (this is not always possible, but I can do it on the graceful shutdown paths).

  • Node dependency not documented

    Node dependency not documented

    This is a fantastic project, thank you! I have successfully managed to use it to run GitHub actions on a self-hosted FreeBSD runner. My first attempt failed with a message under a pile of JSON output telling me that node was not found in the path. It would probably help other folks if the readme mentioned that node needed to be installed.

  • remove .sh suffix from manager script name

    remove .sh suffix from manager script name

    as discussed here https://github.com/ChristopherHX/github-actions-act-runner/pull/21#discussion_r671754664

    it is desirable to remove the .sh sufix from the github-act-runner.sh script. The problem here is that there is a github-act-runner directory along with that in the /usr/bin directory. So, there can be no file with the same name in the same dir.

    So, then we also need to rename the /usr/bin/github-act-runner/ directory to something else. What would be the good name for it? Maybe /usr/bin/github-act-runner-bin/?

  • unable to build using arm32v7/debian:bullseye image

    unable to build using arm32v7/debian:bullseye image

    I'm trying to make builds with arm32v7/debain:bullseye and I get some error on the stage of setting up the container:

    https://github.com/cppfw/myci/runs/3339759944?check_suite_focus=true

    Any idea what could this be?

    All works fine with arm32v7/debain:buster.

    Same issue with i386/debain:bullseye.

  • Add optional timeout for waiting for jobs in ephemeral mode?

    Add optional timeout for waiting for jobs in ephemeral mode?

    Could we get a way to bound the amount of time spent waiting for a job, at https://github.com/ChristopherHX/github-act-runner/blob/e596b4f663c83ca1e37e0b27555ea8b069fb1f8f/main.go#L879, if we're --ephemeral?

    In particular, the concerning scenarios are of this form: if someone creates a PR which kicks off a request for ephemeral runners, and then cancels the workflow before the ephemeral runner is actually ready, the runner will subsequently come up and get stuck, here, because no jobs remain in GitHub's queue.

    This is tangentially related to #59, in that the runner doesn't know what job it was created for and so doesn't know that the job has already been cancelled. Moreover, while we do get a cancellation push message (a workflow_job message indicating "completed" but with a null runner), we can't teardown the environment of the runner associated by job ID, because it might have picked up a different job with the same labels in the same repository. It's all kind of sad. :(

    Anyway, if there's existing support for this and I've merely overlooked it, I'm sorry for the noise.

  • actions/cache compatibility

    actions/cache compatibility

    Hi

    I am using this project to run on riscv64, which isn't supported by the official runner, and it works great :smile:.

    The README says actions/cache is incompatible, why is this? I could not find about it on any other issues.

    And how is https://github.com/actions/toolkit/tree/main/packages/cache supposed to be used directly?

  • Optionally bind ephemeral runner to particular GitHub job?

    Optionally bind ephemeral runner to particular GitHub job?

    Hello and thanks for github-act-runner. We're using it in a home-brew implementation of https://docs.github.com/en/actions/hosting-your-own-runners/autoscaling-with-self-hosted-runners . Our system dispatches work requests from GitHub to a pool of consumers that queue for such requests and, upon receipt, spawn an ephemeral github-act-runner. Curiously, if GitHub pushes two work requests our way (with two distinct workflow_job.id values), there seems to be no way for us to instruct the two resulting ephemeral runners to pick up the workflow_job that triggered their creation. This is fine at present, because all our workers are uniform, but it seems like the kind of thing that might occasionally result in strange logs: worker N constructs an ephemeral runner in response to job request X from repo R but ends up running job Y (which is, at least, also guaranteed to also be from repo R), while worker M constructs an ephemeral runner for Y but ends up running X.

    Are you aware of any way that we could plumb the workflow_job.id value down to the ephemeral runner and thence to GitHub when it asks to pick up a job on the repository? I'm sorry if this should be obvious and I have simply overlooked something in the documentation.

Go-github-actions - `go-github-actions` is a package for developing GitHub Actions

go-github-actions go-github-actions is a package for developing GitHub Actions.

Feb 6, 2022
A simple Go app and GitHub workflow that shows how to use GitHub Actions to test, build and deploy a Go app to Docker Hub

go-pipeline-demo A repository containing a simple Go app and GitHub workflow that shows how to use GitHub Actions to test, build and deploy a Go app t

Nov 17, 2021
github-actions-merger is github actions that merges pull request with commit message including pull request labels.

github-actions-merger github-actions-merger is github actions that merges pull request with commit message including pull request labels. Usage Write

Dec 7, 2022
Nov 2, 2022
My over-engineered homepage project to get an idea of the Kubernetes

My over-engineered homepage project to get an idea of the Kubernetes. It shows traffic data from all my GitHub repositories. I added RabbitMQ, Redis and MongoDB so I got to play around with multiple k8s resources

Jan 15, 2022
Automatic sync from IMDb to Trakt (watchlist, lists, ratings and history) using GitHub actions

imdb-trakt-sync GoLang app that can sync IMDb and Trakt user data - watchlist, ratings and lists. For its data needs, the app is communicating with th

Jan 2, 2023
Bitrise step to parse a JaCoCo generated report and output the code coverage percentages to be used by other steps.

JaCoCo Report Parser This step parses a JaCoCo generated XML report in the jacoco_report_path and outputs the coverage percentages in a String format

Dec 6, 2021
Kube-step-podautoscaler - Controller to scale workloads based on steps
Kube-step-podautoscaler - Controller to scale workloads based on steps

Refer controller/*controller.go for implementation details and explanation for a better understanding.

Sep 5, 2022
Export GitHub Action Workflow data as traces via OTLP

Github Action to OTLP NOTE: This is still work in progress This action outputs Github Action workflows and jobs details to OTLP via gRPC. Inputs endpo

Nov 1, 2022
Git with a cup of tea, painless self-hosted git service
Git with a cup of tea, painless self-hosted git service

Gitea - Git with a cup of tea View the chinese version of this document Purpose The goal of this project is to make the easiest, fastest, and most pai

Jan 2, 2023
LinkPage is a FOSS self-hosted alternative to link listing websites such as LinkTree and Campsite.bio
LinkPage is a FOSS self-hosted alternative to link listing websites such as LinkTree and Campsite.bio

LinkPage LinkPage is a FOSS self-hosted alternative to link listing websites such as LinkTree and Campsite.bio Features Self hostable and open source

Dec 22, 2022
Kubernetes operator for the Azure DevOps self-hosted pipe-line agent.

Kubernetes operator for the Azure DevOps self-hosted pipe-line agent. The operator adds an extra layer of configuration on top of the default images like: proxy settings, pool settings and auth keys.

Sep 1, 2022
Manages nodes in hybrid k8s self-hosted cluster

node-manager Manages nodes in hybrid k8s self-hosted cluster Supported providers Contabo Hetzner Robot (dedicated) Supported commands Heal - reboots a

Dec 23, 2021
Self-hosted uptime monitor

minute Self-hosted uptime monitor. Usage $ go build $ ./minute sites.txt Configuration Configuration is done through the sites.txt file. <SMTP server

Oct 14, 2022
toghsh translates github actions workflows to shell scripts

togsh - extracts shell commands from github actions workflows toghsh is a helper tool to translate github action workflows into equivalent shell scrip

Nov 16, 2021
CI/CD with Jenkins, GitHub Actions, Travis CI etc

CI/CD with Jenkins, GitHub Actions, Travis CI etc This is a simple repository built using Golang with the main purpose to try out different CI/CD tool

Nov 11, 2021
Gophercon lab "Get Go-ing with GitHub Actions"

GitHub Action Using Go This is a starting point for a GitHub Action based in Go. This repo provides all the structure needed to build a robust GitHub

Dec 8, 2021
Example goreleaser + github actions config with keyless signing and SBOM generation

supply-chain-example GoReleaser + Go Mod proxying + Cosign keyless signing + Syft SBOM generation example. How it works GoReleaser manages the entire

Nov 15, 2022
Jan 4, 2022