Simple webhook delivery system powered by Golang and PostgreSQL

postmand

Build Status Go Report Card go.dev reference Docker Image

Simple webhook delivery system powered by Golang and PostgreSQL.

Features

  • Simple rest api with only three endpoints (webhooks/deliveries/delivery-attempts).
  • Select the status codes that are considered valid for a delivery.
  • Control the maximum amount of delivery attempts and delay between these attempts (min and max backoff).
  • Locks control of worker deliveries using PostgreSQL SELECT FOR UPDATE SKIP LOCKED.
  • Sending the X-Hub-Signature header if the webhook is configured with a secret token.
  • Simplicity, it does the minimum necessary, it will not have authentication/permission scheme among other things, the idea is to use it internally in the cloud and not leave exposed.

Quickstart

Let's start with the basic concepts, we have three main entities that we must know to start:

  • Webhook: The configuration of the webhook.
  • Delivery: The content sent to a webhook.
  • Delivery Attempt: An attempt to deliver the content to the webhook.

Run the server

To run the server it is necessary to have a database available from postgresql, in this example we will consider that we have a database called postmand running in localhost with user and password equal to user.

Docker

docker run --rm --env POSTMAND_DATABASE_URL='postgres://user:[email protected]:5432/postmand?sslmode=disable' allisson/postmand migrate # create database schema
docker run -p 8000:8000 -p 8001:8001 --env POSTMAND_DATABASE_URL='postgres://user:[email protected]:5432/postmand?sslmode=disable' allisson/postmand server # run the server

Local

git clone https://github.com/allisson/postmand
cd postmand
cp local.env .env # and edit .env
make run-migrate # create database schema
make run-server # run the server

Run the worker

The worker is responsible to delivery content to the webhooks.

Docker

docker run --env POSTMAND_DATABASE_URL='postgres://user:[email protected]:5432/postmand?sslmode=disable' allisson/postmand worker

Local

make run-worker
go run cmd/postmand/main.go worker

Create a new webhook

The fields delivery_attempt_timeout/retry_min_backoff/retry_max_backoff are in seconds.

curl --location --request POST 'http://localhost:8000/v1/webhooks' \
--header 'Content-Type: application/json' \
--data-raw '{
    "name": "Httpbin Post",
    "url": "https://httpbin.org/post",
    "content_type": "application/json",
    "valid_status_codes": [
        200,
        201
    ],
    "secret_token": "my-secret-token",
    "active": true,
    "max_delivery_attempts": 5,
    "delivery_attempt_timeout": 1,
    "retry_min_backoff": 10,
    "retry_max_backoff": 60
}'
{
  "id":"a6e9a525-ac5a-488c-b118-bd7327ce6d8d",
  "name":"Httpbin Post",
  "url":"https://httpbin.org/post",
  "content_type":"application/json",
  "valid_status_codes":[
    200,
    201
  ],
  "secret_token":"my-secret-token",
  "active":true,
  "max_delivery_attempts":5,
  "delivery_attempt_timeout":1,
  "retry_min_backoff":10,
  "retry_max_backoff":60,
  "created_at":"2021-03-08T20:41:25.433671Z",
  "updated_at":"2021-03-08T20:41:25.433671Z"
}

Create a new delivery

curl --location --request POST 'http://localhost:8000/v1/deliveries' \
--header 'Content-Type: application/json' \
--data-raw '{
    "webhook_id": "a6e9a525-ac5a-488c-b118-bd7327ce6d8d",
    "payload": "{\"success\": true}"
}'
{
  "id":"bc76122c-e56b-45c7-8dc3-b80a861191d5",
  "webhook_id":"a6e9a525-ac5a-488c-b118-bd7327ce6d8d",
  "payload":"{\"success\": true}",
  "scheduled_at":"2021-03-08T20:43:49.986771Z",
  "delivery_attempts":0,
  "status":"pending",
  "created_at":"2021-03-08T20:43:49.986771Z",
  "updated_at":"2021-03-08T20:43:49.986771Z"
}

Get deliveries

curl --location --request GET 'http://localhost:8000/v1/deliveries?webhook_id=a6e9a525-ac5a-488c-b118-bd7327ce6d8d'
{
  "deliveries":[
    {
      "id":"bc76122c-e56b-45c7-8dc3-b80a861191d5",
      "webhook_id":"a6e9a525-ac5a-488c-b118-bd7327ce6d8d",
      "payload":"{\"success\": true}",
      "scheduled_at":"2021-03-08T20:43:49.986771Z",
      "delivery_attempts":1,
      "status":"succeeded",
      "created_at":"2021-03-08T20:43:49.986771Z",
      "updated_at":"2021-03-08T20:46:51.674623Z"
    }
  ],
  "limit":50,
  "offset":0
}

Get delivery

curl --location --request GET 'http://localhost:8000/v1/deliveries/bc76122c-e56b-45c7-8dc3-b80a861191d5'
{
  "id":"bc76122c-e56b-45c7-8dc3-b80a861191d5",
  "webhook_id":"a6e9a525-ac5a-488c-b118-bd7327ce6d8d",
  "payload":"{\"success\": true}",
  "scheduled_at":"2021-03-08T20:43:49.986771Z",
  "delivery_attempts":1,
  "status":"succeeded",
  "created_at":"2021-03-08T20:43:49.986771Z",
  "updated_at":"2021-03-08T20:46:51.674623Z"
}

Get delivery attempts

curl --location --request GET 'http://localhost:8000/v1/delivery-attempts?delivery_id=bc76122c-e56b-45c7-8dc3-b80a861191d5'
{
  "delivery_attempts":[
    {
      "id":"d72719d6-5a79-4df7-a2c2-2029ab0e1848",
      "webhook_id":"a6e9a525-ac5a-488c-b118-bd7327ce6d8d",
      "delivery_id":"bc76122c-e56b-45c7-8dc3-b80a861191d5",
      "raw_request":"POST /post HTTP/1.1\r\nHost: httpbin.org\r\nContent-Type: application/json\r\nX-Hub-Signature: 3fc5d4b8ff4efb404be24faf543667d29902d6a1306bd0c1ef2084497300cee9\r\n\r\n{\"success\": true}",
      "raw_response":"HTTP/2.0 200 OK\r\nContent-Length: 538\r\nAccess-Control-Allow-Credentials: true\r\nAccess-Control-Allow-Origin: *\r\nContent-Type: application/json\r\nDate: Mon, 08 Mar 2021 20:46:51 GMT\r\nServer: gunicorn/19.9.0\r\n\r\n{\n  \"args\": {}, \n  \"data\": \"{\\\"success\\\": true}\", \n  \"files\": {}, \n  \"form\": {}, \n  \"headers\": {\n    \"Accept-Encoding\": \"gzip\", \n    \"Content-Length\": \"17\", \n    \"Content-Type\": \"application/json\", \n    \"Host\": \"httpbin.org\", \n    \"User-Agent\": \"Go-http-client/2.0\", \n    \"X-Amzn-Trace-Id\": \"Root=1-60468d3b-36d312777a03ec3e1c564e3b\", \n    \"X-Hub-Signature\": \"3fc5d4b8ff4efb404be24faf543667d29902d6a1306bd0c1ef2084497300cee9\"\n  }, \n  \"json\": {\n    \"success\": true\n  }, \n  \"origin\": \"191.35.122.74\", \n  \"url\": \"https://httpbin.org/post\"\n}\n",
      "response_status_code":200,
      "execution_duration":547,
      "success":true,
      "error":"",
      "created_at":"2021-03-08T20:46:51.680846Z"
    }
  ],
  "limit":50,
  "offset":0
}

Get delivery attempt

curl --location --request GET 'http://localhost:8000/v1/delivery-attempts/d72719d6-5a79-4df7-a2c2-2029ab0e1848'
{
  "id":"d72719d6-5a79-4df7-a2c2-2029ab0e1848",
  "webhook_id":"a6e9a525-ac5a-488c-b118-bd7327ce6d8d",
  "delivery_id":"bc76122c-e56b-45c7-8dc3-b80a861191d5",
  "raw_request":"POST /post HTTP/1.1\r\nHost: httpbin.org\r\nContent-Type: application/json\r\nX-Hub-Signature: 3fc5d4b8ff4efb404be24faf543667d29902d6a1306bd0c1ef2084497300cee9\r\n\r\n{\"success\": true}",
  "raw_response":"HTTP/2.0 200 OK\r\nContent-Length: 538\r\nAccess-Control-Allow-Credentials: true\r\nAccess-Control-Allow-Origin: *\r\nContent-Type: application/json\r\nDate: Mon, 08 Mar 2021 20:46:51 GMT\r\nServer: gunicorn/19.9.0\r\n\r\n{\n  \"args\": {}, \n  \"data\": \"{\\\"success\\\": true}\", \n  \"files\": {}, \n  \"form\": {}, \n  \"headers\": {\n    \"Accept-Encoding\": \"gzip\", \n    \"Content-Length\": \"17\", \n    \"Content-Type\": \"application/json\", \n    \"Host\": \"httpbin.org\", \n    \"User-Agent\": \"Go-http-client/2.0\", \n    \"X-Amzn-Trace-Id\": \"Root=1-60468d3b-36d312777a03ec3e1c564e3b\", \n    \"X-Hub-Signature\": \"3fc5d4b8ff4efb404be24faf543667d29902d6a1306bd0c1ef2084497300cee9\"\n  }, \n  \"json\": {\n    \"success\": true\n  }, \n  \"origin\": \"191.35.122.74\", \n  \"url\": \"https://httpbin.org/post\"\n}\n",
  "response_status_code":200,
  "execution_duration":547,
  "success":true,
  "error":"",
  "created_at":"2021-03-08T20:46:51.680846Z"
}

Swagger docs

The swagger spec is available at http://localhost:8000/swagger/index.html.

Health check

The health check server is running on port defined by envvar POSTMAND_HEALTH_CHECK_HTTP_PORT (defaults to 8001).

curl --location --request GET 'http://localhost:8001/healthz'
{
  "success":true
}

Environment variables

All environment variables is defined on file local.env.

How to build docker image

docker build -f Dockerfile -t postmand .
Owner
Comments
  • chore(main): release 1.5.0

    chore(main): release 1.5.0

  • chore(deps): bump github.com/swaggo/http-swagger from 1.1.2 to 1.2.6

    chore(deps): bump github.com/swaggo/http-swagger from 1.1.2 to 1.2.6

    Bumps github.com/swaggo/http-swagger from 1.1.2 to 1.2.6.

    Release notes

    Sourced from github.com/swaggo/http-swagger's releases.

    v1.2.6

    Changelog

    b7d83e8 fix: security improvement (#62)

    v1.2.5

    Changelog

    b045c47 Add support for swagger-ui persist-authorization 2ca6b4f README: fix indentation ffedc6b TestUIConfigOptions: fix formatting 25e73d2 chore: update .gitignore b4cc081 chore: update go.mod 777c441 feat: add support for swagger-ui persist-authorization 95f49ee fix TestUIConfigOptions 8c01e61 remove redundant code, fix formatting

    v1.2.1

    Changelog

    6981e22 chore: drop go1.14 support 4799ad4 fix: swag library dependency

    v1.2.0

    Changelog

    5c56dbb Merge pull request #55 from swaggo/mimetype-headers bd37c4b add content-type response headers 84c2fb3 chore: drop go1.13 support

    Commits
    • b7d83e8 fix: security improvement (#62)
    • 25e73d2 chore: update .gitignore
    • b4cc081 chore: update go.mod
    • 777c441 feat: add support for swagger-ui persist-authorization
    • 2ca6b4f README: fix indentation
    • ffedc6b TestUIConfigOptions: fix formatting
    • 8c01e61 remove redundant code, fix formatting
    • 95f49ee fix TestUIConfigOptions
    • b045c47 Add support for swagger-ui persist-authorization
    • 4799ad4 fix: swag library dependency
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

  • chore: release 1.4.0

    chore: release 1.4.0

  • chore: release 1.3.0

    chore: release 1.3.0

  • chore: release 1.2.0

    chore: release 1.2.0

    :robot: I have created a release *beep* *boop*

    1.2.0 (2021-04-24)

    Features

    • http: add secure headers using github.com/unrolled/secure (#28) (89a990b)

    This PR was generated with Release Please. See documentation.

  • chore: release 1.1.0

    chore: release 1.1.0

  • chore: release 1.0.0

    chore: release 1.0.0

    :robot: I have created a release *beep* *boop*

    1.0.0 (2021-03-09)

    Features


    This PR was generated with Release Please. See documentation.

  • Move HTTP error to IOTA const

    Move HTTP error to IOTA const

    What is the purpose of this pull request?

    The goal of this pull request is to improve the HTTP handler error codes by applying iota constants instead of hardcoded values.

    References:

:tophat: Small self-contained pure-Go web server with Lua, Markdown, HTTP/2, QUIC, Redis and PostgreSQL support
:tophat: Small self-contained pure-Go web server with Lua, Markdown, HTTP/2, QUIC, Redis and PostgreSQL support

Web server with built-in support for QUIC, HTTP/2, Lua, Markdown, Pongo2, HyperApp, Amber, Sass(SCSS), GCSS, JSX, BoltDB (built-in, stores the databas

Jan 1, 2023
Stream database events from PostgreSQL to Kafka

PSQL-Streamer This service receives the database events from PostgreSQL using logical replication protocol and feeds them to sinks based on the config

Dec 20, 2022
CasaOS - A simple, easy-to-use, elegant open-source home server system.
CasaOS - A simple, easy-to-use, elegant open-source home server system.

CasaOS - A simple, easy-to-use, elegant open-source home server system. CasaOS is an open-source home server system based on the Docker ecosystem and

Jan 8, 2023
A feature flag solution, with only a YAML file in the backend (S3, GitHub, HTTP, local file ...), no server to install, just add a file in a central system and refer to it. 🎛️
A feature flag solution, with only a YAML file in the backend (S3, GitHub, HTTP, local file ...), no server to install, just add a file in a central system and refer to it. 🎛️

??️ go-feature-flag A feature flag solution, with YAML file in the backend (S3, GitHub, HTTP, local file ...). No server to install, just add a file i

Dec 29, 2022
Distributed reliable key-value store for the most critical data of a distributed system

etcd Note: The master branch may be in an unstable or even broken state during development. Please use releases instead of the master branch in order

Dec 30, 2022
Async management of servers, containers, workstations...basically anything that runs an operating system.

steward What is it ? Command And Control anything asynchronously. Send shell commands to control your servers by passing a message that will have guar

Dec 14, 2022
A simple http-web server logging incoming requests to stdout with simple http-interface.
A simple http-web server logging incoming requests to stdout with simple http-interface.

http-cli-echo-logger A simple http-web server logging incoming requests to stdout with simple http-interface. Run locally go run ./cmd/main.go Default

Jul 18, 2022
A very simple Golang server handling basic GET and POST requests

GOLANG SERVER INTRO As a true Blockchain enthusiast, I had to learn Solidity and Golang to participate to several projects. This repository consists o

Nov 17, 2021
Simple Golang Product API Server

Simple Golang Product API Server Layanan API untuk menambah, merubah informasi, mengambil data dan menghapus produk. Implementasi kode terinspirasi ol

Nov 20, 2022
Simple HTTP server written in golang

Simple HTTP server written in golang Simple webserver in golang, to demonstrate basic functionalities like e.g. sending back some request header info,

Aug 31, 2022
Go-simplehttp - Simple HTTP server written in golang

Simple HTTP server written in golang Simple webserver in golang, to demonstrate

Jan 1, 2022
Oogway is a simple web server with dynamic content generation and extendability in mind supporting a Git based workflow.

Oogway Oogway is a simple web server with dynamic content generation and extendability in mind supporting a Git based workflow. It's somewhere in betw

Nov 9, 2022
Formrecevr is a simple and lightweight from receiver backend primarily designed for (but not limited to) static websites.

Formrecevr Formrecevr (pronunced "Form receiver") is a simple and lightweight from receiver backend primarily designed for (but not limited to) static

Apr 17, 2022
A simple SHOUTcast server.

DudelDu DudelDu is a simple audio/video streaming server using the SHOUTcast protocol. Features Supports various streaming clients: VLC, ServeStream,

Nov 20, 2022
Heart 💜A high performance Lua web server with a simple, powerful API
Heart 💜A high performance Lua web server with a simple, powerful API

Heart ?? A high performance Lua web server with a simple, powerful API. See the full documentation here. Overview Heart combines Go's fasthttp with Lu

Aug 31, 2022
KissLists is a very simple shared lists server
KissLists is a very simple shared lists server

KissLists is a very simple shared lists server. with mobile optimised design basic theme support websockets messages sqlite database but no

Nov 30, 2022
A simple HTTP Server to share files over WiFi via Qr Code
A simple HTTP Server to share files over WiFi via Qr Code

go-fileserver A simple HTTP server to share files over WiFi via QRCode Installation You can download compressed version from

Oct 8, 2022
a simple http server as replacement of python -m http.server

ser a simple http server as replacement of python -m http.server

Dec 5, 2022
Fileserver: A simple static fileserver

Fileserver A simple static fileserver. Serves requests to local files in a directory of your choosing so that you can preview files in your browser. B

May 26, 2022