A decentralized P2P networking stack written in Go.

noise

GoDoc Discord MIT licensed Build Status Go Report Card Coverage Status

noise is an opinionated, easy-to-use P2P network stack for decentralized applications, and cryptographic protocols written in Go.

noise is made to be minimal, robust, developer-friendly, performant, secure, and cross-platform across multitudes of devices by making use of a small amount of well-tested, production-grade dependencies.

Features

  • Listen for incoming peers, query peers, and ping peers.
  • Request for/respond to messages, fire-and-forget messages, and optionally automatically serialize/deserialize messages across peers.
  • Optionally cancel/timeout pinging peers, sending messages to peers, receiving messages from peers, or requesting messages from peers via context support.
  • Fine-grained control over a node and peers lifecycle and goroutines and resources (synchronously/asynchronously/gracefully start listening for new peers, stop listening for new peers, send messages to a peer, disconnect an existing peer, wait for a peer to be ready, wait for a peer to have disconnected).
  • Limit resource consumption by pooling connections and specifying the max number of inbound/outbound connections allowed at any given time.
  • Reclaim resources exhaustively by timing out idle peers with a configurable timeout.
  • Establish a shared secret by performing an Elliptic-Curve Diffie-Hellman Handshake over Curve25519.
  • Establish an encrypted session amongst a pair of peers via authenticated-encryption-with-associated-data (AEAD). Built-in support for AES 256-bit Galois Counter Mode (GCM).
  • Peer-to-peer routing, discovery, identities, and handshake protocol via Kademlia overlay network protocol.

Defaults

  • No logs are printed by default. Set a logger via noise.WithNodeLogger(*zap.Logger).
  • A random Ed25519 key pair is generated for a new node.
  • Peers attempt to be dialed at most three times.
  • A total of 128 outbound connections are allowed at any time.
  • A total of 128 inbound connections are allowed at any time.
  • Peers may send in a single message, at most, 4MB worth of data.
  • Connections timeout after 10 seconds if no reads/writes occur.

Dependencies

Setup

noise was intended to be used in Go projects that utilize Go modules. You may incorporate noise into your project as a library dependency by executing the following:

% go get -u github.com/perlin-network/noise

Example

package main

import (
    "context"
    "fmt"
    "github.com/perlin-network/noise"
)

func check(err error) {
    if err != nil {
        panic(err)
    }
}

// This example demonstrates how to send/handle RPC requests across peers, how to listen for incoming
// peers, how to check if a message received is a request or not, how to reply to a RPC request, and
// how to cleanup node instances after you are done using them.
func main() { 
    // Let there be nodes Alice and Bob.

    alice, err := noise.NewNode()
    check(err)

    bob, err := noise.NewNode()
    check(err)

    // Gracefully release resources for Alice and Bob at the end of the example.

    defer alice.Close()
    defer bob.Close()

    // When Bob gets a message from Alice, print it out and respond to Alice with 'Hi Alice!'

    bob.Handle(func(ctx noise.HandlerContext) error {
        if !ctx.IsRequest() {
            return nil
        }

        fmt.Printf("Got a message from Alice: '%s'\n", string(ctx.Data()))

        return ctx.Send([]byte("Hi Alice!"))
    })

    // Have Alice and Bob start listening for new peers.

    check(alice.Listen())
    check(bob.Listen())

    // Have Alice send Bob a request with the message 'Hi Bob!'

    res, err := alice.Request(context.TODO(), bob.Addr(), []byte("Hi Bob!"))
    check(err)

    // Print out the response Bob got from Alice.

    fmt.Printf("Got a message from Bob: '%s'\n", string(res))

    // Output:
    // Got a message from Alice: 'Hi Bob!'
    // Got a message from Bob: 'Hi Alice!'
}

For documentation and more examples, refer to noise's godoc here.

Benchmarks

Benchmarks measure CPU time and allocations of a single node sending messages, requests, and responses to/from itself over 8 logical cores on a loopback adapter.

Take these benchmark numbers with a grain of salt.

% cat /proc/cpuinfo | grep 'model name' | uniq
model name : Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz

% go test -bench=. -benchtime=30s -benchmem
goos: linux
goarch: amd64
pkg: github.com/perlin-network/noise
BenchmarkRPC-8           4074007              9967 ns/op             272 B/op          7 allocs/op
BenchmarkSend-8         31161464              1051 ns/op              13 B/op          2 allocs/op
PASS
ok      github.com/perlin-network/noise 84.481s

Versioning

noise is currently in its initial development phase and therefore does not promise that subsequent releases will not comprise of breaking changes. Be aware of this should you choose to utilize Noise for projects that are in production.

Releases are marked with a version number formatted as MAJOR.MINOR.PATCH. Major breaking changes involve a bump in MAJOR, minor backward-compatible changes involve a bump in MINOR, and patches and bug fixes involve a bump in PATCH starting from v2.0.0.

Therefore, noise mostly respects semantic versioning.

The rationale behind this is due to improper tagging of prior releases (v0.1.0, v1.0.0, v1.1.0, and v1.1.1), which has caused for the improper caching of module information on proxy.golang.org and sum.golang.org.

As a result, noise's initial development phase starts from v1.1.2. Until Noise's API is stable, subsequent releases will only comprise of bumps in MINOR and PATCH.

License

noise, and all of its source code is released under the MIT License.

Owner
Comments
  • Error importing packages from project

    Error importing packages from project

    Cloning this repo, calling go mod vendor and then go run examples/chat/main.go works fine.

    The problems begin when I try to use noise from my own projects. I did the usual go mod init etc with a simple test source:

    package main
    
    import (
    	"github.com/perlin-network/noise/crypto/ed25519"
    	"github.com/perlin-network/noise/log"
    )
    
    func main() {
    	keys := ed25519.RandomKeyPair()
    	println(keys)
    	// Print out loaded public/private keys.
    	log.Info().
    		Str("private_key", keys.PrivateKeyHex()).
    		Msg("")
    	log.Info().
    		Str("public_key", keys.PublicKeyHex()).
    		Msg("")
    }
    

    Calling this with go run main.go results in the following error:

    go: finding github.com/perlin-network/noise/log latest
    go: finding github.com/perlin-network/noise/crypto/ed25519 latest
    go: finding github.com/perlin-network/noise/crypto latest
    main.go:5:2: unknown import path "github.com/perlin-network/noise/log": cannot find module providing package github.com/perlin-network/noise/log
    

    If I remove the logging function and only use the crypto stuff it works. If I try the same with the chat example from the noise repo but copied to my project I get the same error and also an error for the opcodes package.

    Since I am rather new to go modules, I am probably doing something really stupid. It's just extra confusing that the crypto package works..

    My go version go version go1.11.4 linux/amd64

  • Fix the proxy example

    Fix the proxy example

    • seems PONG adds new connected peers to the network, so the topology added randomly
    • think fix keeps the network peer connections unaltered after bootstrap
  • Migrate to KCP backend.

    Migrate to KCP backend.

    Porting from gRPC to KCP.

    TODO: ~~1. Strange bug that upon peer discovery, the node will Dial() itself. Look into network/discovery/*.go~~ ~~2. Have the bootstrapping of peers happen only after the server is ready to listen for peers. In other words, block until server is ready in main.go.~~ ~~3. Figure out how to determine whether or not a peer has properly disconnected exactly once* to remove from the Kademlia peer routing table.~~ ~~4. Rebase into master and merge.~~

  • Fix a data race by re-adding the ID if its address is different

    Fix a data race by re-adding the ID if its address is different

    Currently, there's a data race on skademlia.ID when the address of the ID is updated in case the its address is different.

    Since skademlia.ID should be immutable, the solution should be to remove the ID and add it again.

    This PR implements the solution.

  • Better strategy for selecting peers in case of failed connection

    Better strategy for selecting peers in case of failed connection

    Currently there are same peers selected and tried to dial even in case they failed recently. Which results to huge delay (especially in case of default 3s connection timeout).

    There should be better strategy which should consider latest successful/failed connection to select only live (or potentially live) peers.

  • Make noise.Info struct to be thread safe

    Make noise.Info struct to be thread safe

    Currently, there's a race condition since the Info is modified concurrently.

    This PR fixes the race condition by making Info to be thread safe by using mutex. Because of the mutex, we have to make its methods to have pointer receivers.

  • Which version of noise should we use?

    Which version of noise should we use?

    We would like to use noise in our p2p application, and we noticed that there is a v2 branch. Could you tell us whether we should use the master branch, the latest release (v1.1), or the v2 branch?

  • Crash when incoming message has nil field

    Crash when incoming message has nil field "Sender"

    I've been having random crashes with a peer (sometimes connected to other peers, and sometimes on it's own) with the error:

    2018/09/19 16:45:50
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0xa3815d]
    
    goroutine 49 [running]:
    github.com/gladiusio/gladius-node/vendor/github.com/perlin-network/noise/network.(*Network).Accept.func2(0xc4203aba60, 0xc420312d00, 0xc42006c380, 0xc42000f220, 0xc4203ad6b0, 0xc4203ad690, 0xc4203df890)
            /ext-go/1/src/github.com/gladiusio/gladius-node/vendor/github.com/perlin-network/noise/network/network.go:402 +0xed
    created by github.com/gladiusio/gladius-node/vendor/github.com/perlin-network/noise/network.(*Network).Accept
            /ext-go/1/src/github.com/gladiusio/gladius-node/vendor/github.com/perlin-network/noise/network/network.go:375 +0x1d4
    

    The random nature of this, and the fact it happens when there are no peers connected makes me think it's this section of our code:

    func (state *StatePlugin) Startup(net *network.Network) {
    	// TODO: Find out why this crashes the network on ocassion
    	 go func() {
    		time.Sleep(60 * time.Second)
    	 	net.BroadcastRandomly(&messages.SyncRequest{}, 1)
    	 }()
    }
    

    It looks like this if statement is the problem at network.go:402, dereferencing msg.Sender causes a panic.

    // Peer sent message with a completely different ID. Disconnect.
    if !client.ID.Equals(peer.ID(*msg.Sender)) {
    	glog.Errorf("message signed by peer %s but client is %s", peer.ID(*msg.Sender), client.ID.Address)
    	return
    }
    

    I also get an error about the peer should not dial itself even if I haven't explicitly called p.net.Bootstrap(addressList...) so I believe that the above code is broadcasting before the network has been bootstrapped, and that the BroadcastRandomly method seems to be able to dial itself. This is running on a VPS and listening on it's public IP with the code:

    // Use KCP instead of TCP, and config bind address and bind port
    builder.SetAddress(network.FormatAddress(
    	"kcp",
    	viper.GetString("P2P.BindAddress"),
    	uint16(viper.GetInt("P2P.BindPort")))
    
    // This is our network state object
    s := state.New()
    s.RegisterNodeSingleFields("ip_address", "content_port", "heartbeat")
    s.RegisterNodeListFields("disk_content")
    
    s.RegisterPoolListFields("required_content")
    
    // Register peer discovery plugin.
    // TODO: Setup an authorized DHT plugin
    builder.AddPlugin(new(discovery.Plugin))
    
    // Add the exponential backoff plugin
    builder.AddPlugin(new(backoff.Plugin))
    
    // Create our state sync plugin
    statePlugin := new(StatePlugin)
    statePlugin.peerState = s
    builder.AddPlugin(statePlugin)
    
    net, err := builder.Build()
    if err != nil {
    	log.Fatal(err)
    	return nil
    }
    
    go net.Listen()
    

    I haven't tested with TCP instead of KCP though.

  • Setup Issues

    Setup Issues

    I am running into a vgo mod -sync error, that tells me it cannot find:

    go: import "github.com/gogo/protobuf/protoc-gen-gogofaster/gocode/src/github.com/golang/mock/mockgen/tests/vendor_dep" -> import "a": cannot find module providing package a

    I had an issue with protobufs 3.6 not working and versioned down to 3.5.1, that fixed the protobuf issue so maybe there are other versioning issues?

    running ubuntu 18.04

  • aead: fixing import error

    aead: fixing import error

    When go building a basic example using aead go was unable to find the module because it was hidden under a subdirectory.

    Typical error message:

    % go build test.go
    go: finding github.com/perlin-network/noise/handshake/ecdh latest
    go: finding github.com/perlin-network/noise/skademlia latest
    go: finding github.com/perlin-network/noise/identity/ed25519 latest
    go: finding github.com/perlin-network/noise/rpc latest
    go: finding github.com/perlin-network/noise/cipher/aead latest
    go: finding github.com/perlin-network/noise/protocol latest
    go: finding github.com/perlin-network/noise/handshake latest
    go: finding github.com/perlin-network/noise/identity latest
    go: finding github.com/perlin-network/noise/cipher latest
    build command-line-arguments: cannot load github.com/perlin-network/noise/cipher/aead: cannot find module providing package github.com/perlin-network/noise/cipher/aead
    
  • Question about service discovery

    Question about service discovery

    Hi,

    Sorry to bother you with this question. Actually I am not sure if your library can be used for my scenario. My scenario is quite simple, I want to detect all computer running my service on a local network. A bit like UPnP/AV that are able to discover media servers on the network.

    So I've tried your examples (e.g. the chat) but in my scenario I just want to list the peers (their IP addresses) at the moment, I don't want to connect to them. So I got rid of everything related to messaging and Dial from the code. But then skademlia.FindNode(...) returns an empty array. In your example, it looks like you know in advance that a peer would run on localhost so you can connect to it, you don't discover it.

  • How well does it scale?

    How well does it scale?

    Hello,

    I'm on the hunt for the best possible P2P library for Golang and have come across go-libp2p but it seems to have a number of issues with scaling and stability.

    Can you please tell me how well Noise scales and if it is reasonably stable? Thanks in advance

  • Is there a gossip protocol for p2p network fast broadcast the message?

    Is there a gossip protocol for p2p network fast broadcast the message?

    I'm trying to build a p2p network with noise, when one peer transmit a message to many other peers, i just connect to everyone of them and send data. Is there a gossip protocol to make it faster?

  • OnPeerEvicted happens immediately after connection, disregarding timeout

    OnPeerEvicted happens immediately after connection, disregarding timeout

    If I run the chat example multiple times on the same machine, the chat clients are able to find one another.

    If I run the chat client on two different machines on my network, the client connects and then immediately fails out. I tried to change the timeout from 3 seconds to much larger, but the evicted event is immediate and not related to the timeout.

    From the machine trying to connect:

    Your ID is :46097(e60bd758). Type '/discover' to attempt to discover new peers, or '/peers' to list out all peers you are connected to.
    Learned about a new peer :38469(ea91a207).
    Forgotten a peer :38469(ea91a207).
    Did not discover any peers.```
    

    The machine receiving the connection retains memory of the new peer however:

    Learned about a new peer :46097(e60bd758).
    /peers
    You know 1 peer(s): [:46097(e60bd758)]
    
  • Use Readers/Writers instead of bytes

    Use Readers/Writers instead of bytes

    It would be great to have Readers/Writers instead of []byte when (de-)serializing data. For bigger files, like you mentioned BitTorrent (https://medium.com/perlin-network/noise-an-opinionated-p2p-networking-stack-for-decentralized-protocols-in-go-bfc6fecf157d), it would be good to not blow up the heap, but stream the data directly (e.g. using io.Copy). Maybe we could end up with something like

    type SerializableStream interface {
    	Marshal() io.ReadWriter
    }
    
  • Indirect routing

    Indirect routing

    test

    The example above describes a scenario with 3 peers. One acts as a provider and two clients would like to get information from that provider. Due to network restrictions only one of the two clients is able to communicate with the provider, but both clients are able to connect to each other.

    After estabilishing the network, kademlia.New().Discover() shows the correct routes.

    Provider knows Allowed_Client
    Allowed_Client knows Provider and Blocked_Client
    Blocked_Client knows Allowed_Client
    

    In this scenario it would be great to have a way that SendMessage would use id node.ID instead of addr string. E.g.: SendMessage(ctx context.Context, id ID, msg Serializable) error In this way Blocked_Client could send a message to Provider without knowing that it has to pass Allowed_Client.

    At the moment this has to be done somehow manually. To me this is somehow strange, as the routing is already implemented and correct.

  • Use of internal package not allowed

    Use of internal package not allowed

    When trying to run your example given in the README of this repo I came across this error.

    package command-line-arguments
    	imports github.com/perlin-network/noise
    	imports github.com/oasislabs/ed25519
    	../../go/src/github.com/oasislabs/ed25519/batch_verify.go:37:2: use of internal package github.com/oasisprotocol/ed25519/internal/curve25519 not allowed
    package command-line-arguments
    	imports github.com/perlin-network/noise
    	imports github.com/oasislabs/ed25519
    	../../go/src/github.com/oasislabs/ed25519/batch_verify.go:38:2: use of internal package github.com/oasisprotocol/ed25519/internal/ge25519 not allowed
    package command-line-arguments
    	imports github.com/perlin-network/noise
    	imports github.com/oasislabs/ed25519
    	../../go/src/github.com/oasislabs/ed25519/batch_verify.go:39:2: use of internal package github.com/oasisprotocol/ed25519/internal/modm not allowed
    

    I am using

    go version go1.15.8 linux/amd64
    

    and I installed noise via

    % go get -u github.com/perlin-network/noise
    

    Any help would be greatly appreciated! Also let me know if I can provide anymore useful information.

Xlibp2p: an opinionated, easy-to-use P2P network stack for decentralized applications written in Go

xlibp2p xlibp2p is an opinionated, easy-to-use P2P network stack for decentraliz

Feb 21, 2022
Native macOS networking for QEMU using vmnet.framework and socket networking.

qemu-vmnet Native macOS networking for QEMU using vmnet.framework and socket networking. Getting started TODO -netdev socket,id=net0,udp=:1234,localad

May 8, 2022
Steve - A peer-to-peer (p2p) decentralized network

Steve Steve is a peer-to-peer (p2p) decentralized network that enables people to

Feb 5, 2022
🚀 gnet is a high-performance, lightweight, non-blocking, event-driven networking framework written in pure Go./ gnet 是一个高性能、轻量级、非阻塞的事件驱动 Go 网络框架。
🚀 gnet is a high-performance, lightweight, non-blocking, event-driven networking framework written in pure Go./ gnet 是一个高性能、轻量级、非阻塞的事件驱动 Go 网络框架。

English | ???? 中文 ?? Introduction gnet is an event-driven networking framework that is fast and lightweight. It makes direct epoll and kqueue syscalls

May 9, 2022
High-performance, non-blocking, event-driven, easy-to-use networking framework written in Go, support tls/http1.x/websocket.

High-performance, non-blocking, event-driven, easy-to-use networking framework written in Go, support tls/http1.x/websocket.

May 10, 2022
GO2P is a P2P framework, designed with flexibility and simplicity in mind
GO2P is a P2P framework, designed with flexibility and simplicity in mind

go2p golang p2p framework By v-braun - viktor-braun.de. Description GO2P is a P2P framework, designed with flexibility and simplicity in mind. You can

Mar 23, 2022
A major platform Remote Access Terminal Tool based by Blockchain/P2P.
A major platform Remote Access Terminal Tool based by Blockchain/P2P.

NGLite A major platform Remote Access Terminal Tool based by Blockchain/P2P. No public IP address required.More anonymity Example Detection Warning!!!

May 12, 2022
fetch papers from p2p network

sci-hub P2P A project aims to fetch paper from the BitTorrent network. According to this Reddit post, currently, all `sci-hub's papers are available i

Apr 29, 2022
fetch and serve papers in p2p network

sci-hub P2P A project aims to fetch paper from the BitTorrent network. This is not a cli client of sci-hub website. English Introduction 中文简介 Contribu

Apr 29, 2022
A minimal IPFS replacement for P2P IPLD apps

IPFS-Nucleus IPFS-Nucleus is a minimal block daemon for IPLD based services. You could call it an IPLDaemon. It implements the following http api call

Apr 6, 2022
Educational project to build a p2p network

Template go Repository tl;dr This is a template go repository with actions already set up to create compiled releases What does this Template provide?

Dec 2, 2021
Openp2p - an open source, free, and lightweight P2P sharing network
Openp2p - an open source, free, and lightweight P2P sharing network

It is an open source, free, and lightweight P2P sharing network. As long as any device joins in, you can access them anywhere

May 7, 2022
P2P Forwarder - a tool for farwarding tcp/udp ports. Made using libp2p.
P2P Forwarder - a tool for farwarding tcp/udp ports. Made using libp2p.

P2P Forwarder A tool for farwarding ports. Made using libp2p. How it works A: opens desired ports ports inside P2P Forwarder A: shares it's id from P2

Jan 14, 2022
Apr 7, 2022
High performance async-io(proactor) networking for Golang。golangのための高性能非同期io(proactor)ネットワーキング
High performance async-io(proactor) networking for Golang。golangのための高性能非同期io(proactor)ネットワーキング

gaio Introduction 中文介绍 For a typical golang network program, you would first conn := lis.Accept() to get a connection and go func(net.Conn) to start a

May 6, 2022
Fast event-loop networking for Go
Fast event-loop networking for Go

evio is an event loop networking framework that is fast and small. It makes direct epoll and kqueue syscalls rather than using the standard Go net pac

May 9, 2022
Packiffer is a lightweight cross-platform networking toolkit that let you sniff/analyze/inject/filter packets.
Packiffer is a lightweight cross-platform networking toolkit that let you sniff/analyze/inject/filter packets.

Packiffer is a lightweight cross-platform networking toolkit that let you sniff/analyze/inject/filter packets.

Apr 13, 2022
Netpoll is a high-performance non-blocking I/O networking framework, which focused on RPC scenarios, developed by ByteDance.
Netpoll is a high-performance non-blocking I/O networking framework, which focused on RPC scenarios, developed by ByteDance.

Netpoll is a high-performance non-blocking I/O networking framework, which focused on RPC scenarios, developed by ByteDance. RPC is usually heavy on processing logic and therefore cannot handle I/O serially. But Go's standard library net designed blocking I/O API, so that the RPC framework can only follow the One Conn One Goroutine design.

May 15, 2022
Fork of Go stdlib's net/http that works with alternative TLS libraries like refraction-networking/utls.

github.com/ooni/oohttp This repository contains a fork of Go's standard library net/http package including patches to allow using this HTTP code with

Apr 30, 2022