Portal is a quick and easy command-line file transfer utility from any computer to another πŸ–₯️ 🌌 πŸ’»

portal

Installation

Homebrew

brew tap ZinoKader/portal
brew install portal

Manual

Either get the latest release and install it manually, or run

curl -s https://raw.githubusercontent.com/ZinoKader/portal/master/scripts/install.sh | bash

if permission denied for moving the files to /../bin, replace " | bash" with " | sudo bash"
(the script is in the repo, so you can check it out before you blindly trust in it!)

The application

portal is a fast and secure file transfer utility for sending files from one computer to any other computer. All communication beyond the initial client handshake is encrypted. If the sender and receiver can reach each other directly, the file transfer involves no servers. Otherwise the file transfer goes through a relay server which facilitates the connection, but sees none of the data.

Sending files and folders

The file transfer starts by invoking the command from the sender side:

portal send <file1> <file2> <folder1> <folder2> ...

The application will output a temporary password on the format 1-inertia-elliptical-celestial. The sender will communicate this password to the receiver over some secure channel.

Receiving files and folders

The receiver would then issue the command:

portal receive 1-intertia-elliptical-celestial

The two clients will connect to each other and transfer the file(s)/folder(s).

Extra: hosting your own rendezvous/relay server

To make connection establishment possible, portal makes use of a rendezvous server. By default, a rendezvous server hosted at Digital Ocean is preconfigured, so you do not need to do anything. If you would like to host one on your own, build the server and start it with:

# specify port with -p or --port
portal-rendezvous --port 80

Demo

demo

Features

portal provides:

  • End-to-end encryption using PAKE2 to negotiate a shared session-key
  • Direct transfer of files if possible (e.g. sender and receiver are in the same local network)
  • Fallback to a TURN-server (rendezvous-relay) for file transfer if the sender and receiver are behind NATs in different network
  • Parallel gzip compression of files for faster and more efficient transfer

Technical details

The connection between the sender and the server is negotiated using a intermediary server called portal-rendezvous. The portal-rendezvous server is used to negotiate a secure encrypted channel while never seeing the contents of files nor the temporary password.

The communication works as follows:

  • sender application connects to rendezvous-server
  • rendezvous-server allocates an id to the sender and sends over websocket to the sender
  • sender outputs the password to the terminal, hashes the password and sends it to the rendezvous-server
  • receiver hashes the password (which has been communicated over some secure channel) and the sends it to the rendezvous-server
  • When both the sender and the receiver has sent the hashed password to the rendezvous-server the cryptographic exchange starts, during which the rendezvous-server relays messages from the sender to the receiver or vice versa
  • Once the cryptographic exchange is done, every message sent by the sender and receiver is encrypted, and the rendezvous-server cannot decrypt them
  • Now two things can happen:
    • Either the sender and receiver are behind the same NAT, in which case the file transfer will be directly between the sender and receiver. In this case, the connection to the rendezvous-server will be closed
    • If they are not behind the same NAT, the transfer will fallback to go through the rendezvous-server which will continue to relay encrypted messages until the file transfer is completed

Motivation, team

This was initially a project made for the course IK2218 Protocols and Principles of the Internet. We often used similar tools in our day-to-day and wanted to make our own!

Project members:

  • Zino Kader
  • Arvid Gotthard
  • Anton Sederlin

Software used

Comments
  • transfer not utlising full bandwidth capacity

    transfer not utlising full bandwidth capacity

    • via webrtc but is it's network bandwidth utilisation is below 30-50% where it could be 90-95%. ( a. netrwork capacity is 1gbps, b. 100mbps. So if sending is throttled to my receiving capacity it should be around b's capacity. But i'm barely getting near 30-40mbps)

    Is parallel connections / multi-threading possible?

    edit: Whatever croc is doing seems can use my full badwidth.

  • Automate test running

    Automate test running

    • Runs on pushes to all branches (for now, we have some Github Actions minutes to waste!)
    • Runs on pull requests to develop and master

    Also adds a build badge, which we later can extend with a code coverage badge and a test status badge. The latter badges are both quite finicky to get to work with GitHub, currently.

  • Error compressing files.

    Error compressing files.

    Hello, i'm trying to send a file using the command portal --verbose send E01.mp4, however it fails almost instantly with the only output being "Error compressing files." What would be a good way to diagnose what's causing the issue? as even verbose mode doesn't say anything in particular

  • F/wasm

    F/wasm

    This PR ports the portal module to JS using WASM

    I have not yet managed to import the wasm file using a bundler into a js project, but it works very well in a raw HTML file. We would likely want to create a typescript npm module for this package. The question is if we want this in this repo, or a separate one?

    The JS interface is not very JS-like, so there is potential for improvement

  • cant send file

    cant send file

    hi i was just using portal when suddenly Error initializing UI unable to set console to raw mode: The parameter is incorrect. this error came i cant send any files.

    any help would be appreciated. thanks

  • Refactored `Sender` and `Receiver` to use functional option pattern

    Refactored `Sender` and `Receiver` to use functional option pattern

    Change that is meant to clean up the API a bit by using functional option pattern for specifying the options.

    For instance, if you want to create a Sender with the UI option you would do

    sender := New(programOpts, WithUi(uiCh))
    

    Opposed to

    sender := WithUI(New(programOpts), uiCh)
    

    By doing this you can more ergonomically add more options to the constructor like this,

    sender := New(programOpts, WithServer(serverOpts), WithPayload(buf, int64(buf.Len())))
    

    And adding more options in the future would not change the API.

  • Fails to compress npm_modules folders

    Fails to compress npm_modules folders

    Generally fails to compress any npm_modules folders that I have tried to send. Most likely this is due to irregular folder or file names in some common module.

    Adding more info soon.

  • bugfix: impose no message size limit, use same version of PAKE in sender and receiver

    bugfix: impose no message size limit, use same version of PAKE in sender and receiver

    (Dev -> Master)

    • Remove message size limitations. This broke portal for larger files than 32KB since the latest release which uses the nhooyr/websocket library.

    • Sender and receiver had different versions of schollz/pake. The old version in the receiver was broken in go 1.19. (golang/go#50974)

    • Upgrade other packages.

  • F/portal module

    F/portal module

    This PR does the following:

    • Introduces an Init function to internal/sender package so that it can be called from other packages than the main package
    • Refactors the rendezvous address to a string rather than TCPAddr
    • Implements a separate package from which the portal Send and Receive functions programmatically
    • Implements an E2E test using testcontainers, and adds a Dockerfile
    • Adds a Makefile so pipeline steps can be easily reproduced locally
    • Updates the pipeline and adds new steps
  • refactor rendezvous sender client

    refactor rendezvous sender client

    • Moved ready channel into the struct and had WithPayload signal to it.
    • Returned websocket connection instead of sending it over a channel.

    These changes do not change any behavior

  • Seperate cli flags into global flags and subcommand flags

    Seperate cli flags into global flags and subcommand flags

    When looking into solving #4 I saw that all flags are shared across all subcommands and that some flags had dual purposes.

    For instance the -p flag doubles for specifying the port where to expect the rendezvous server to be running when supplied with the send and receive commands, and when supplied with the serve command it specifies on which port to run the rendezvous server.

    I think it is probably wise to separate flags on a per-subcommand basis, as it does not make sense to have a --address flag for the serve command for instance. We could probably still have some global flags like verbose etc that are available to all subcommands.

  • Unmanaged output

    Unmanaged output

  • Can not install with `go install`

    Can not install with `go install`

    I wanted to install on an armhf (32-bit arm) host, and attempted to install via go install because the releases only include arm64 binaries.

    Error output:

    go install -v github.com/ZinoKader/portal/cmd/[email protected]
    go: downloading github.com/ZinoKader/portal v1.0.3
    go: github.com/ZinoKader/portal/cmd/[email protected]: github.com/ZinoKader/[email protected]: parsing go.mod:
    	module declares its path as: www.github.com/ZinoKader/portal
    	        but was required as: github.com/ZinoKader/portal
    
  • Friends list for

    Friends list for "password-less" transfers

    While the password is already pretty short, it requires communication of the password over some other medium for every individual transfer session to establish a connection. An optional feature could be to let users decide on an email/password combination which friends can add to their friends list. This way, when your friend starts a transfer session, you can simply choose their email/name from the CLI and start receiving.

    An idea for how to still keep somewhat secure: We would not want to store the credentials on the server (only in primary memory and hashed), just like with the password. To get around this, the transfer session would send the encrypted credentials to the server only when starting a transfer (and these are stored in memory until somebody connects). The recipient would then send the same encrypted credentials to the server, and if there is a match your friend is online and will start the transfer.

  • Add interactive file-picker for send command

    Add interactive file-picker for send command

    By invoking portal send with no arguments the TUI could open an interactive file picker to specify which files to send.

    Look here and here for insporation

  • Switch default port to something other than `80`

    Switch default port to something other than `80`

    When I try to run the rendezvous server locally I get this error,

    image

    as port 80 is a privileged port and therefore a non-root process cannot use it.

    Do not know if it is my setup or whatever but we should probably change it?

A command line tool to prompt for a value to be included in another command line.

readval is a command line tool which is designed for one specific purposeβ€”to prompt for a value to be included in another command line. readval prints

Dec 22, 2021
A command line tool that builds and (re)starts your web application everytime you save a Go or template fileA command line tool that builds and (re)starts your web application everytime you save a Go or template file

# Fresh Fresh is a command line tool that builds and (re)starts your web application everytime you save a Go or template file. If the web framework yo

Nov 22, 2021
CLI filters the contents of the csv file according to the filters from the another file.

filtercsv CLI filters the contents of the csv file according to the filters from the another file. Made to process big files by a lots of filters. By

Dec 2, 2021
This command line converts .webarchive file to resources embed .html file

webarchive-to-singlefile This command line converts Safari's .webarchive file to complete .html. Notice Only tested on MacOS. Google Chrome required.

Dec 30, 2022
An open-source GitLab command line tool bringing GitLab's cool features to your command line
An open-source GitLab command line tool bringing GitLab's cool features to your command line

GLab is an open source GitLab CLI tool bringing GitLab to your terminal next to where you are already working with git and your code without switching

Dec 30, 2022
Trzsz-go - A simple file transfer tools, similar to lrzsz ( rz / sz ), and compatible with tmux

Trzsz-go - A simple file transfer tools, similar to lrzsz ( rz / sz ), and compatible with tmux

Dec 31, 2022
A client for managing authzed or any API-compatible system from your command line.

zed A client for managing authzed or any API-compatible system from your command line. Installation zed is currently packaged by as a head-only Homebr

Dec 31, 2022
Hosty is a command-line utility that allows for fast inspection and editing of /etc/hosts-like files

Hosty Description Hosty is a command-line utility that allows for fast inspection and editing of /etc/hosts-like files. It is written in golang and us

Sep 3, 2021
πŸ”„ A command-line utility to export Protocol Buffers (proto) files to YAML, and JSON

proto2yaml ?? A command-line utility to export Protocol Buffers (proto) files to YAML, and JSON. Currently supported exports are for: Packages Service

Nov 10, 2022
A command line utility and library for generating professional looking invoices in Go.
A command line utility and library for generating professional looking invoices in Go.

ginvoicer A command line utility and library for generating professional looking invoices in Go. This is a very rough draft and there could still be b

Dec 15, 2022
A command line utility for labeling GitHub issues and pull requests

A command line utility for labeling GitHub issues and pull requests

Jan 8, 2023
Gopassutil - Command line utility to hash and verify with passlib for go

gopassutil command line utility to hash and verify with passlib for go Usage Gen

Feb 1, 2022
πŸ“· Command-line utility to download all photos from Instagram
πŸ“· Command-line utility to download all photos from Instagram

Instagram Downloader This is a simple command-line tool, written in Go, to download all images from an Instagram account. Getting Started Install inst

Sep 9, 2022
A command line utility for generating language-specific project structure.
A command line utility for generating language-specific project structure.

hydra hydra is a command line utility for generating language-specific project structures. ⏬ ✨ Features Build project templates with just one command

Oct 8, 2021
A small utility command line application that can recursively download Notion pages

notionbackup A small utility command line application that can recursively download Notion pages. I needed something scriptable that could periodicall

Dec 5, 2022
πŸ“ˆ A command-line utility to interact with TradingView

tvctl ?? A command-line utility to interact with TradingView. This utility aims to improve usability of tedius and repetative tasks, such as watchlist

Oct 18, 2022
ls Xtended : A command line utility which lets you navigate through terminal like a pro 😎.
ls Xtended : A command line utility which lets you navigate through terminal like a pro 😎.

Navigate through terminal like a pro ?? ?? Demo β€’ βš—οΈ Installation β€’ ?? Contribution β€’ ❗ Known Issues ❓ Why? It's a pain to cd and ls multiple times to

Dec 14, 2022
Command-line utility to grab Github gists from your own account.
Command-line utility to grab Github gists from your own account.

gistfetch Command-line utility to grab Github gists from your own account. How do I use this? Add an API token with permissions to read Gists Fetch th

Dec 14, 2021
Go cmd utility that prints its command line arguments using strings.Join

Results This is an exercise of the book The Go Programming Language, by Alan A. A. Donovan and Brian Kernighan. Comparison between different versions

Dec 18, 2021