A simple web service for storing text log files

logpaste

CircleCI Docker Pulls License

A minimalist web service for uploading and sharing log files.

Run locally

go run main.go

Run in local Docker container

The Docker container automatically replicates to an AWS S3 bucket

AWS_ACCESS_KEY_ID=YOUR-ACCESS-ID
AWS_SECRET_ACCESS_KEY=YOUR-SECRET-ACCESS-KEY
AWS_REGION=YOUR-REGION
DB_REPLICA_URL=s3://your-bucket-name/db

docker run \
  -e "AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}" \
  -e "AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}" \
  -e "AWS_REGION=${AWS_REGION}" \
  -e "DB_REPLICA_URL=${DB_REPLICA_URL}" \
  -e "CREATE_NEW_DB='true'" `# change to false after first run` \
  --name logpaste \
  mtlynch/logpaste

Run with custom site settings

SITE_TITLE="My Cool Log Pasting Service"
SITE_SUBTITLE="Upload all your logs for FooBar here"
SITE_SHOW_DOCUMENTATION="false" # Hide usage information from homepage

AWS_ACCESS_KEY_ID=YOUR-ACCESS-ID
AWS_SECRET_ACCESS_KEY=YOUR-SECRET-ACCESS-KEY
AWS_REGION=YOUR-REGION
DB_REPLICA_URL=s3://your-bucket-name/db

docker run \
  -e "AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}" \
  -e "AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}" \
  -e "AWS_REGION=${AWS_REGION}" \
  -e "DB_REPLICA_URL=${DB_REPLICA_URL}" \
  -e "SITE_TITLE=${SITE_TITLE}" \
  -e "SITE_SUBTITLE=${SITE_SUBTITLE}" \
  -e "SITE_SHOW_DOCUMENTATION=${SITE_SHOW_DOCUMENTATION}" \
  -e "CREATE_NEW_DB='true'" `# change to false after first run` \
  --name logpaste \
  mtlynch/logpaste
Comments
  • APIkey error on non-aws bucket

    APIkey error on non-aws bucket

    Trying to use filebase as bucket results in this error, using it with other programs I get no such error(s3fs)

    monitor error: cannot lookup bucket region: InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
    

    My litestream.yml

    access-key-id: ${LITESTREAM_ACCESS_KEY_ID}
    secret-access-key: ${LITESTREAM_SECRET_ACCESS_KEY}
    dbs:
      - path: ${DB_PATH}
        replicas:
          - url: s3://mypaste-new.filebase.com
            sync-interval: ${DB_SYNC_INTERVAL}
    
  • Add litestream health check

    Add litestream health check

    If logpaste expects litestream to be replicating data, and it's not, it should fail hard so that users stop uploading logs that will never be replicated.

    Maybe the Go app can have a periodic check similar to what's mentioned in this Kubernetes example and panic if litestream is no longer replicating.

  • Feature request: Dark mode.

    Feature request: Dark mode.

    Hello!

    I recently discovered logpaste, and it's a pleasure to use so far. Easy to set up, self host, and use! However, the text isn't always the easiest to read because of the white background on the home page. May can I ask, would it be possible to add dark mode, which can be enabled using a docker environment value?

  • Limit maximum request size in pastePut

    Limit maximum request size in pastePut

    Without this, clients can upload arbitrarily-large pastes. Such pastes will be caught later by validatePaste, but in the interim, the entire paste is held in memory, so a malicious client could trivially pipe /dev/urandom into curl and eventually cause your server to OOM.

  • Do not assume S3

    Do not assume S3

    I successfully deployed logpaste to Google Cloud on Google Cloud Run and Google Cloud Storage

    I had the change the code in the following way:

    Then, I built with Cloud Build gcloud builds submit . --tag gcr.io/<PROJECT>/logpaste:latest

    And deployed to Cloud Run: gcloud beta run deploy logpaste --image gcr.io/<PROJECT>/logpaste:latest --set-env-vars DB_REPLICA_URL=gcs://<BUCKET>/db --execution-environment gen2 --no-cpu-throttling

    I can try to send a PR: we can enable litestream if DB_REPLICA_URL is set, but I'm not sure about the best way handle the translation of envvars to litestream.yml

  • Add option to force HTTPS

    Add option to force HTTPS

    One small thing to mention: When using curl to upload text, the output URL defaults to http. Most webservers should already redirect from port 80 to port 443, but I think it would be nice to add an option to enable https everywhere instead of the http output.

    Originally posted by @KoxSosen in https://github.com/mtlynch/logpaste/issues/121#issuecomment-963351527

  • Switch to native HTTP muxer

    Switch to native HTTP muxer

    We're using the third-party gorilla mux package, but I don't think we really need it. We can probably get away with the standard library implementation.

  • Update fly.io.md

    Update fly.io.md

    Thanks for making this great doc -- I just went through it and wanted to suggest some edits here:

    1/ You don't need docker installed if you are just deploying following these instructions -- I say this because I deployed WITHOUT having docker installed. 2/ When I ran fly init "${APP_NAME}" --nowrite - the fly cli said Error We need your payment information to continue! Add a credit card or buy credit: https://fly.io/organizations/personal

  • Restore fails on Google Cloud Run

    Restore fails on Google Cloud Run

    Works fine in a local container, but fails for some reason on Google Cloud Run

    + litestream restore -v /app/data/store.db 
    2021/02/22 15:11:08 /app/data/store.db(s3): starting restore: generation 68ce1ab8c28d0111, index 00000000-00000000 
    2021/02/22 15:11:08 /app/data/store.db(s3): restoring snapshot 68ce1ab8c28d0111/00000000 to /app/data/store.db.tmp 
    cannot restore wal: disk I/O error: invalid argument 
    
  • Customize link ID

    Customize link ID

    Hello, Thank you for making this.

    I was curious if you would be able to add a customizable link length? Right now it's hard coded for 8 characters I believe, but would appreciate being able to set a custom length via a docker-compose variable

  • Merge PUT handler into POST

    Merge PUT handler into POST

    The PUT handler doesn't match the semantics of the traditional PUT verb.

    I chose PUT because I wanted a way for the client to signal to the server that it wanted back the URL instead of a JSON object.

    Revisiting the decision, I should have just used an Accept header in the request to let the client specify whether they want plaintext (default) or JSON. So we should combine the handlers for PUT and POST into a single handler that chooses output based on the content type and accept headers.

    See similar work in PicoShare: https://github.com/mtlynch/picoshare/pull/318/files

Distributed-Log-Service - Distributed Log Service With Golang
Distributed-Log-Service - Distributed Log Service With Golang

Distributed Log Service This project is essentially a result of my attempt to un

Jun 1, 2022
An golang log lib, supports tracking and level, wrap by standard log lib

Logex An golang log lib, supports tracing and level, wrap by standard log lib How To Get shell go get gopkg.in/logex.v1 source code import "gopkg.in/

Nov 27, 2022
Nginx-Log-Analyzer is a lightweight (simplistic) log analyzer for Nginx.
Nginx-Log-Analyzer is a lightweight (simplistic) log analyzer for Nginx.

Nginx-Log-Analyzer is a lightweight (simplistic) log analyzer, used to analyze Nginx access logs for myself.

Nov 29, 2022
Log-analyzer - Log analyzer with golang

Log Analyzer what do we have here? Objective Installation and Running Applicatio

Jan 27, 2022
BRUS - Parses your web server (e.g. nginx) log files and checks with GreyNoise how much noise your website is exposed to.

BRUS bbbbbb rrrrrr u u sssss b b r r u u s bbbbbb rrrrrr u u sssss b b r r u u s bbbbbb r r

May 29, 2022
Simple log parser written in Golang

Simple log parser written in Golang

Oct 31, 2021
Log-generator - A simple CLI tool that generates near real logs for testing

Log-generator - A simple CLI tool that generates near real logs for testing

Jan 22, 2022
A simple to use log system, minimalist but with features for debugging and differentiation of messages
A simple to use log system, minimalist but with features for debugging and differentiation of messages

A simple to use log system, minimalist but with features for debugging and differentiation of messages

Sep 26, 2022
Simple text-line analog clock

anaclock anaclock prints a simple analog clock as a line of text. Demo $ anaclock 23 . : .| 00 anaclock is easy to use in CLI prompts or anywhere

Dec 31, 2021
a golang log lib supports level and multi handlers

go-log a golang log lib supports level and multi handlers Use import "github.com/siddontang/go-log/log" //log with different level log.Info("hello wo

Dec 29, 2022
Structured log interface

Structured log interface Package log provides the separation of the logging interface from its implementation and decouples the logger backend from yo

Sep 26, 2022
lumberjack is a log rolling package for Go

lumberjack Lumberjack is a Go package for writing logs to rolling files. Package lumberjack provides a rolling logger. Note that this is v2.0 of lumbe

Jan 1, 2023
CoLog is a prefix-based leveled execution log for Go
CoLog is a prefix-based leveled execution log for Go

What's CoLog? CoLog is a prefix-based leveled execution log for Go. It's heavily inspired by Logrus and aims to offer similar features by parsing the

Dec 14, 2022
OpenTelemetry log collection library

opentelemetry-log-collection Status This project was originally developed by observIQ under the name Stanza. It has been contributed to the OpenTeleme

Sep 15, 2022
exo: a process manager & log viewer for dev
 exo: a process manager & log viewer for dev

exo: a process manager & log viewer for dev exo- prefix – external; from outside. Features Procfile compatible process manager.

Dec 28, 2022
Write log entries, get X-Ray traces.

logtoxray Write to logs, get X-Ray traces. No distributed tracing instrumenation library required. ?? ?? ?? THIS PROJECT IS A WORK-IN-PROGRESS PROTOTY

Apr 24, 2022
Binalyze logger is an easily customizable wrapper for logrus with log rotation

logger logger is an easily customizable wrapper for logrus with log rotation Usage There is only one function to initialize logger. logger.Init() When

Oct 2, 2022
Log-structured virtual disk in Ceph
Log-structured virtual disk in Ceph

lsd_ceph Log-structured virtual disk in Ceph 1. Vision and Goals of the Project Implement the basic librbd API to work with the research block device

Dec 13, 2021
Multi-level logger based on go std log

mlog the mlog is multi-level logger based on go std log. It is: Simple Easy to use NOTHING ELSE package main import ( log "github.com/ccpaging/lo

May 18, 2022