Stream data over http using curl.

PIPE TO ME

Stream data over http using curl. Hosted at pipeto.me

  • No javascript
  • Go standard library only

NAME
    pipeto.me: streaming data over http

SYNOPSIS
    Randomly generated pipe address:    https://pipeto.me/

EXAMPLES
    Input example:

    (chrome/firefox): https://pipeto.me/
    (terminal)$ curl -T- https://pipeto.me/
                hello world

    Pipe example:

    (terminal1)$ curl https://pipeto.me/
    (terminal2)$ echo hello world | curl -T- https://pipeto.me/

    Chat example(curl>=7.68):

    (terminal1)$ curl -T. https://pipeto.me/
    (terminal2)$ curl -T. https://pipeto.me/
                 hello world

    File transfer example:

    (terminal1)$ curl https://pipeto.me/ > output.txt
    (terminal2)$ cat input.txt | curl -T- https://pipeto.me/

    Watch log example:

    (chrome/firefox): https://pipeto.me/
    (terminal)$ tail -f logfile | curl -T- https://pipeto.me/

DESCRIPTION
    Data is not buffered or stored in any way.
    Data is not retrievable after it has been delivered.

    Maximum upload size: 64 MB
    Not allowed: anything illegal, malicious, inappropriate, etc.

    This is a personal project and makes no guarantees on:
    reliability, performance, privacy, etc.

    Default Mode:

    If data is sent to the pipe when no receivers are listening, 
    it will be dropped and is not retrievable.

    Fail Mode: 

    $ curl -T- https://pipeto.me/?mode=fail
    In this mode, a send request will fail if no receivers are listening.

    Block Mode:

    $ curl -T- --expect100-timeout 86400 https://pipeto.me/?mode=block
    In this mode, a send request will wait to send data until a receiver connects.

SEE ALSO
    Demo: https://raw.githubusercontent.com/jpschroeder/pipe-to-me/master/demo.gif
    Source: https://github.com/jpschroeder/pipe-to-me

Installation

With go installed:

go get -u github.com/jpschroeder/pipe-to-me

Usage

pipe-to-me -h
  -baseurl string
        the base url of the service
         (default "http://localhost:8080/")
  -httpaddr string
        the address/port to listen on for http
        use :<port> to listen on all addresses
         (default "localhost:8080")

Building

In order to build the project, just use:

go build

Deploying

You can build the project under linux (or Windows Subsystem for Linux) and just copy the executable to your server.

You can then run the program directly or use systemd to install it as a service and keep it running.

Customize the scripts/pipe-to-me.service file in the repo for your server and copy it to /lib/systemd/system/pipe-to-me.service (ubuntu)

Start the app with: systemctl start pipe-to-me
Enable it on boot with: systemctl enable pipe-to-me
Check it's status with: systemctl status pipe-to-me
See standard output/error with: journalctl -f -u pipe-to-me

NGINX

You can host the application using go directly, or you can listen on a local port and use nginx to proxy connections to the app.

Make sure that nginx is installed with: apt-get install nginx

Customize scripts/pipe-to-me.nginx.conf and copy it to /etc/nginx/sites-available/pipe-to-me.nginx.conf

Remove the default website configuration: rm /etc/nginx/sites-enabled/default

Enable the go proxy: ln -s /etc/nginx/sites-available/pipe-to-me.nginx.conf /etc/nginx/sites-enabled/pipe-to-me.nginx.conf

Restart nginx to pick up the changes: systemctl restart nginx

NGINX HTTPS

If running as a stand-alone go application, you can use the built-in https support. When running behind a proxy, you should enable https in nginx and forward to the localhost http address.

Install the letsencrypt client with:

add-apt-repository ppa:certbot/certbot
apt-get install python-certbot-nginx

Generate and install a certificate with: certbot --nginx -d pipeto.me

The certificate should auto-renew when necessary.

Docker

A prepared Docker image is available here.

docker run --name pipe-to-me -p 80:8080 jpschroeder/pipe-to-me -baseurl http://localhost/

You can build the image yourself by running docker-compose up --build.

Similar Resources

An enhanced HTTP client for Go

An enhanced HTTP client for Go

Heimdall Description Installation Usage Making a simple GET request Creating a hystrix-like circuit breaker Creating a hystrix-like circuit breaker wi

Jan 9, 2023

Enriches the standard go http client with retry functionality.

httpRetry Enriches the standard go http client with retry functionality using a wrapper around the Roundtripper interface. The advantage of this libra

Dec 10, 2022

Go (golang) http calls with retries and backoff

pester pester wraps Go's standard lib http client to provide several options to increase resiliency in your request. If you experience poor network co

Dec 28, 2022

http client for golang

http client for golang

Request HTTP client for golang, Inspired by Javascript-axios Python-request. If you have experience about axios or requests, you will love it. No 3rd

Dec 18, 2022

Simple HTTP and REST client library for Go

Resty Simple HTTP and REST client library for Go (inspired by Ruby rest-client) Features section describes in detail about Resty capabilities Resty Co

Jan 1, 2023

A nicer interface for golang stdlib HTTP client

rq A nicer interface for golang stdlib HTTP client Documents rq: here client: here jar: here Why? Because golang HTTP client is a pain in the a... Fea

Dec 12, 2022

A Go HTTP client library for creating and sending API requests

A Go HTTP client library for creating and sending API requests

Sling Sling is a Go HTTP client library for creating and sending API requests. Slings store HTTP Request properties to simplify sending requests and d

Jan 7, 2023

HTTP Load Testing And Benchmarking Tool

GBench HTTP Load Testing And Benchmarking Tool inspired by Apache Benchmark and Siege. Requirements You need Golang installed and ready on your system

Jan 2, 2020

HTTP/HTTPS load testing and benchmarking tool

Introduction I wrote that code because: (the obvious reason::I love to write code in Go) We are working so hard to optimize our servers - why shouldn'

Dec 5, 2022
Comments
  • Module path should be

    Module path should be "github.com/jpschroeder/pipe-to-me", not "pipe-to-me"

    Background

    Module path is inconsistent with go import path. GO111MODULE=on, run go get -u github.com/jpschroeder/pipe-to-me:

    go: downloading github.com/jpschroeder/pipe-to-me v0.0.0-20191212154931-8f031675506a
    go: github.com/jpschroeder/pipe-to-me upgrade => v0.0.0-20191212154931-8f031675506a
    go get: github.com/jpschroeder/[email protected]: parsing go.mod:
            module declares its path as: pipe-to-me
                    but was required as: github.com/jpschroeder/pipe-to-me
    

    Solution

    Fix the module path:

    Rename the module path to "github.com/jpschroeder/pipe-to-me". https://github.com/jpschroeder/pipe-to-me/blob/master/go.mod#L1

    module github.com/jpschroeder/pipe-to-me
    go 1.12 
    
  • Adding Docker build instructions

    Adding Docker build instructions

    These instructions build a very small Docker image (5Mb). The image is available at adamgoose/pipe-to-me:latest. I can automate this build and I can continue to manage it if you make me a collaborator on this project, or you can easily set up automated builds at https://hub.docker.com.

  • Crazy idea: stream video in real-time (say up to 5 seconds delay)

    Crazy idea: stream video in real-time (say up to 5 seconds delay)

    I wonder whether pipe-to-me could be (mis)used to stream video in real-time to multiple web clients. One way (i.e. only curl->pipe-to-me->web_browser) would be fine, no need for bidirectional piping.

    Thoughts?

    The idea is to have absolutely minimal clients using "plain old web technologies" while leveraging as much of web browser built-in functionality as possible (in this case especially the ability to play videos without any JS and hopefully without full prefetch as I would like to stream "indefinite" things like a conference).

    Latency should be say up to ~5 seconds to make it usable I think :wink:.

Http-conection - A simple example of how to establish a HTTP connection using Golang

A simple example of how to establish a HTTP connection using Golang

Feb 1, 2022
Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http
Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http

fasthttp Fast HTTP implementation for Go. Currently fasthttp is successfully used by VertaMedia in a production serving up to 200K rps from more than

Jan 2, 2023
Speak HTTP like a local. (the simple, intuitive HTTP console, golang version)

http-gonsole This is the Go port of the http-console. Speak HTTP like a local Talking to an HTTP server with curl can be fun, but most of the time it'

Jul 14, 2021
Http client call for golang http api calls

httpclient-call-go This library is used to make http calls to different API services Install Package go get

Oct 7, 2022
fhttp is a fork of net/http that provides an array of features pertaining to the fingerprint of the golang http client.

fhttp The f stands for flex. fhttp is a fork of net/http that provides an array of features pertaining to the fingerprint of the golang http client. T

Jan 1, 2023
NATS HTTP Round Tripper - This is a Golang http.RoundTripper that uses NATS as a transport.

This is a Golang http.RoundTripper that uses NATS as a transport. Included is a http.RoundTripper for clients, a server that uses normal HTTP Handlers and any existing http handler mux and a Caddy Server transport.

Dec 6, 2022
Simple HTTP package that wraps net/http

Simple HTTP package that wraps net/http

Jan 17, 2022
Httpx - a fast and multi-purpose HTTP toolkit allow to run multiple probers using retryablehttp library
Httpx - a fast and multi-purpose HTTP toolkit allow to run multiple probers using retryablehttp library

httpx is a fast and multi-purpose HTTP toolkit allow to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads.

Feb 3, 2022
Full-featured, plugin-driven, extensible HTTP client toolkit for Go

gentleman Full-featured, plugin-driven, middleware-oriented toolkit to easily create rich, versatile and composable HTTP clients in Go. gentleman embr

Dec 23, 2022
An enhanced http client for Golang
An enhanced http client for Golang

go-http-client An enhanced http client for Golang Documentation on go.dev ?? This package provides you a http client package for your http requests. Y

Dec 23, 2022