dht is used by anacrolix/torrent, and is intended for use as a library in other projects both torrent related and otherwise

dht

CircleCI GoDoc Join the chat at https://gitter.im/anacrolix/torrent

Installation

Install the library package with go get github.com/anacrolix/dht, or the provided cmds with go get github.com/anacrolix/dht/cmd/....

Commands

Here I'll describe what some of the provided commands in ./cmd do.

Note that the godo command which is invoked in the following examples builds and executes a Go import path, like go run. It's easier to use this convention than to spell out the install/invoke cycle for every single example.

dht-ping

Pings DHT nodes with the given network addresses.

$ godo ./cmd/dht-ping router.bittorrent.com:6881 router.utorrent.com:6881
2015/04/01 17:21:23 main.go:33: dht server on [::]:60058
32f54e697351ff4aec29cdbaabf2fbe3467cc267 (router.bittorrent.com:6881): 648.218621ms
ebff36697351ff4aec29cdbaabf2fbe3467cc267 (router.utorrent.com:6881): 873.864706ms
2/2 responses (100.000000%)
Owner
Matt Joiner
I'm a functional programming and concurrency enthusiast that got sucked into Go. Please send help.
Matt Joiner
Comments
  • A lot of connections

    A lot of connections

    With bootstrap and when connecting to the torrent, there are about 1000 connections. Connections are not replaced, but added, if after bootstrap connect to the torrent, there will be about ~2000 connections, etc. My ISP blocked me access because it thinks it's a virus attack

    Over time, connections are disconnected

  • krpc.CompactIPv4NodeAddrs.MarshalBinary() does not produce compact IPv4 form

    krpc.CompactIPv4NodeAddrs.MarshalBinary() does not produce compact IPv4 form

    krpc.CompactIPv4NodeAddrs (or NodeAddr) does not have the essential To4() piece in its marshaller functions, unlike pp.CompactIp which does. Without this, the use sites need to carry the conversion explicitly.

  • get_peers is performed but no announce

    get_peers is performed but no announce

    if I add a log.Printf to annouce.go/beginQuery I get: 20-03-21 21:50:33 announce.go:290: doing dht announce get_peers to [address]

    and not a single doing "dht announce announce_peer"

  • bep44: mutable items

    bep44: mutable items

    Hi,

    i was trying to figure out how much of this bep was available in your package. http://bittorrent.org/beps/bep_0044.html

    I found the immutable items part was there, if i dont mistake.

    About mutable items, there are few more operation to do when a message is sent / receive. I was wondering how that would operate with your package. inside or outside of it ? if there was some specific ways to implement those extra functions with the lib ?

    I was looking for a Put method at Server.go, but i could not find it. Is it missing as i believe i understand ?

    thanks for help!

  • use dht for own storage

    use dht for own storage

    From @vtolstov on December 5, 2016 13:41

    I need a dht package for own storage system, and as i see godoc.org says that this package have more stars then other. Does it possible to use this package for own program that does not using torrents?

    Copied from original issue: anacrolix/torrent#134

  • invalid memory address or nil pointer dereference

    invalid memory address or nil pointer dereference

    ^[[0mpanic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x10de6c7]
    /vendor/github.com/anacrolix/dht/v2/bep44/store.go:54 +0x27
    
  • panic: sync: negative WaitGroup counter

    panic: sync: negative WaitGroup counter

    Running r55080c5 I'm regularly getting the following panic. Any ideas, or should I work on a reproduction?

    panic: sync: negative WaitGroup counter goroutine 17787 [running]: sync.(*WaitGroup).Add(0x13984c10, 0xffffffff) /usr/local/go/src/sync/waitgroup.go:73 +0x150 sync.(*WaitGroup).Done(0x13984c10) /usr/local/go/src/sync/waitgroup.go:98 +0x24 localhost/xxx/vendor/github.com/anacrolix/dht.(*Server).Bootstrap.func1.1(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) localhost/xxx/vendor/github.com/anacrolix/dht/server.go:662 +0x13c localhost/xxx/vendor/github.com/anacrolix/dht.(*Server).findNode.func1(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) localhost/xxx/vendor/github.com/anacrolix/dht/server.go:630 +0x80 created by localhost/xxx/vendor/github.com/anacrolix/dht.(*Server).query.func4 localhost/xxx/vendor/github.com/anacrolix/dht/server.go:540 +0x88 `

  • BEP 0044 implementation.

    BEP 0044 implementation.

    BEP 0044 implementation: Storing arbitrary data in the DHT.

    • Added two new methods to Server struct; Put and Get.
    • Item storage is pluggable, an in-memory implementation is provided by default.
    • Added some integration and unit tests to check that everything is working properly.

    It is my first PR to this repository, do not hesitate to provide all the needed feedback.

    Closes #26

    Signed-off-by: Antonio Navarro Perez [email protected]

  • Server::findNode not exposed?

    Server::findNode not exposed?

    Hi. First of all, thank you for the great work you have done on this project.

    I just discovered this repo a few days ago and would like to continue using it for a project. For that I would need to send find_node queries to target addresses. Is it possible that the findNode function in server.go is not exposed unlike Ping and Announce? If so, how can one use the function after having bootstrapped? I just started using go a few days ago, therefore, this may be an oversight on my part.

    Thank you, Charmaine

  • do not start a goroutine on import

    do not start a goroutine on import

    I'm trying to use this as part of my project, and the BT/DHT function is not always needed, it's only some part of my application.

    So the dht/v2 as a library start a goroutine in the background running forever is very weird.

    I think this goroutine should be started when the BT client is created, not the library is imported, library itself shoule bt side-effect free.

  • Does DHT work standalone?

    Does DHT work standalone?

    Hey, I've been trying to get this library to work standalone without bootstrapping from the global servers but I seem to not be getting the expected behaviour.

    Here's a gist which sets up a server standalone https://gist.github.com/asim/99d9978f76ec46083b6dcf4a65b58cb5 and then the equivalent can be run with ports changed and using the first address in BootstrapNodes list.

    When I run this locally or across the public internet I can see udp packets flowing back and forth between the servers but nothing is ever announced and I can't actually tell if anything's working.

    If I use the global servers then it all seems to work fine. Any ideas why this won't bootstrap as a standalone dht server?

  • Documentation about peer/traversal stats

    Documentation about peer/traversal stats

    Hello! Thank you for maintaining this project and publishing it as FLOSS. I've been experimenting with using it as part of the ooni project to measure mainline DHT censorship.

    However, it is not clear to me precisely what the exposed stats actually mean. For example, running the provided CLI tool returns the following:

    
    $ ./dht get-peers
    {
      "Peers": [],
      "DistinctPeerIps": 0,
      "TraversalStats": {
        "NumAddrsTried": 31,
        "NumResponses": 0
      },
      "ServerStats": {
        "GoodNodes": 2,
        "Nodes": 3,
        "OutstandingTransactions": 0,
        "SuccessfulOutboundAnnouncePeerQueries": 0,
        "BadNodes": 0,
        "OutboundQueriesAttempted": 31
      }
    }
    

    In this example, we can see NumResponses is zero, but in server stats GoodNodes is 2. This does not match my current mental model.

    In a test case i wrote, where i start a local dht.Server without any bootstrap nodes, then try to connect from it from another dht.Server (with only the first one as bootstrap nodes), i get the following results after an Announce: in announce.TraversalStats(): 1 peer tried, 0 responded. And announce.Peers is empty. Yet i can see using tcpdump that the first peer did respond to the query.

    Trying with the default set of bootstrap nodes (13 addresses), i get 19 NumResponses (NumAddrsTried 46) and announce.Peers contains 22 entries.

    It's not clear to me what NumResponses and the entries in announce.Peers are. Is the first one the number of traversed peers queried about the specific infohash, and the second the number of found peers who announce that infohash? If so, is NumResponses only counting replies from peers announcing that infohash?

    To get more comprehensive results, maybe i should call get_peers recursively myself? In the meantime, i feel like it's worth explaining more about what the exposed stats/data actually means.

    Thanks for your attention

  • WIP: Part one of I2P support

    WIP: Part one of I2P support

    This PR is part 1 of 2 changes required to add I2P support in order to make code reviews manageable. It mostly touches the krpc package with only minor changes outside of it to make sure things aren't broken.

    Since the last PR, I was able to simplify the change so no Go generics are needed. Instead, a krpc package level function ( SetNetworkType() ) controls how to treat the bencoded data (IP or I2P destination).

    You had a question in the previous PR:

    Have you looked at how this stuff works with rport and port from the spec? Perhaps it's worth following up with the I2P developers and check the spec is still up to date.

    Yes, that will be part of me next PR so that the PRs aren't massive and easier to review. I'm also in touch with the I2P devs, and the spec on the I2P website is indeed up to date.

    This change has a small breaking change for the torrent library where it explicitly references krpc.CompactIPv4NodeAddrs, but that should be a quick matter of replacing the struct name there.

Distributed lock manager. Warning: very hard to use it properly. Not because it's broken, but because distributed systems are hard. If in doubt, do not use this.

What Dlock is a distributed lock manager [1]. It is designed after flock utility but for multiple machines. When client disconnects, all his locks are

Dec 24, 2019
Easy to use Raft library to make your app distributed, highly available and fault-tolerant
Easy to use Raft library to make your app distributed, highly available and fault-tolerant

An easy to use customizable library to make your Go application Distributed, Highly available, Fault Tolerant etc... using Hashicorp's Raft library wh

Nov 16, 2022
a dynamic configuration framework used in distributed system
a dynamic configuration framework used in distributed system

go-archaius This is a light weight configuration management framework which helps to manage configurations in distributed system The main objective of

Dec 9, 2022
*DEPRECATED* Please use https://gopkg.in/redsync.v1 (https://github.com/go-redsync/redsync)

Redsync.go This package is being replaced with https://gopkg.in/redsync.v1. I will continue to maintain this package for a while so that its users do

Nov 20, 2022
Golang client library for adding support for interacting and monitoring Celery workers, tasks and events.

Celeriac Golang client library for adding support for interacting and monitoring Celery workers and tasks. It provides functionality to place tasks on

Oct 28, 2022
Dec 27, 2022
A feature complete and high performance multi-group Raft library in Go.
A feature complete and high performance multi-group Raft library in Go.

Dragonboat - A Multi-Group Raft library in Go / 中文版 News 2021-01-20 Dragonboat v3.3 has been released, please check CHANGELOG for all changes. 2020-03

Dec 30, 2022
Simple, fast and scalable golang rpc library for high load

gorpc Simple, fast and scalable golang RPC library for high load and microservices. Gorpc provides the following features useful for highly loaded pro

Dec 19, 2022
🌧 BitTorrent client and library in Go
🌧 BitTorrent client and library in Go

rain BitTorrent client and library in Go. Running in production at put.io. Features Core protocol Fast extension Magnet links Multiple trackers UDP tr

Dec 28, 2022
A Go library for master-less peer-to-peer autodiscovery and RPC between HTTP services

sleuth sleuth is a Go library that provides master-less peer-to-peer autodiscovery and RPC between HTTP services that reside on the same network. It w

Dec 28, 2022
A distributed systems library for Kubernetes deployments built on top of spindle and Cloud Spanner.

hedge A library built on top of spindle and Cloud Spanner that provides rudimentary distributed computing facilities to Kubernetes deployments. Featur

Nov 9, 2022
A distributed locking library built on top of Cloud Spanner and TrueTime.

A distributed locking library built on top of Cloud Spanner and TrueTime.

Sep 13, 2022
Compute cluster (HPC) job submission library for Go (#golang) based on the open DRMAA standard.

go-drmaa This is a job submission library for Go (#golang) which is compatible to the DRMAA standard. The Go library is a wrapper around the DRMAA C l

Nov 17, 2022
Dynatomic is a library for using dynamodb as an atomic counter

Dynatomic Dynatomic is a library for using dynamodb as an atomic counter Dynatomic Motivation Usage Development Contributing Motivation The dynatomic

Sep 26, 2022
Library for enabling asynchronous health checks in your service

go-health A library that enables async dependency health checking for services running on an orchestrated container platform such as kubernetes or mes

Jan 4, 2023
A standard library for microservices.

Go kit Go kit is a programming toolkit for building microservices (or elegant monoliths) in Go. We solve common problems in distributed systems and ap

Jan 2, 2023
A library that implements the outboxer pattern in go

Outboxer Outboxer is a go library that implements the outbox pattern. Getting Started Outboxer was designed to simplify the tough work of orchestratin

Dec 16, 2022