A Go language binding for encodeing and decoding data in the bencode format that is used by the BitTorrent peer-to-peer file sharing protocol.

bencode-go

A Go language binding for encoding and decoding data in the bencode format that is used by the BitTorrent peer-to-peer file sharing protocol.

Quick Start

Get the package

go get -u github.com/jackpal/bencode-go

Import the package

import bencode "github.com/jackpal/bencode-go"

Unmarshal a bencode stream into an object

data := myAwesomeObject{}
err := bencode.Unmarshal(reader, &data)

Decode a bencode stream

data, err := bencode.Decode(reader)

Encode an object into a bencode stream

err := bencode.Marshal(writer, data)

Complete documentation

http://godoc.org/github.com/jackpal/bencode-go

License

This project is licensed under the Go Authors standard license. (See the LICENSE file for details.)

Comments
  • don't encode []byte as array of numbers

    don't encode []byte as array of numbers

    instead, encode them as, well, bytes.

    Example:

        bencode.Marshal(os.Stdout, []byte{'1', '2', '3'})
    

    results in

        li49ei50ei51ee
    

    instead of just

        3:123
    

    Why not just use string? ~~Because in Go, they, by definition, should only contain valid Unicode. bencode "strings" however allow binary data.~~

    Edit: okay I just realized this is wrong. Anyway, []byte should be handled the same way!

  • Added support for tag options and field to ignore when Marshalling a struct

    Added support for tag options and field to ignore when Marshalling a struct

    You can now use the following syntax in a tag struct :

    type myStruct struct {
    // Ignored during Marshalling
    Field string `bencode:"-"` 
    // Ignored during Marshalling if empty otherwise named  "name"
    Field2 string `bencode:"name,omitempty`
    // Ignored during Marshalling if empty otherwise named with default field.Name in this case "Field3"
    Field3 string `bencode:",omitempty`
    }
    

    In this version, the old tag style is no longer supported.

  • Reduce allocations in bencode

    Reduce allocations in bencode

    Note the B/op column.

    Before: $ go test -benchmem -bench=. PASS BenchmarkDecodeAll 5000 483495 ns/op 62057 B/op 186 allocs/op BenchmarkUnmarshalAll 5000 788224 ns/op 66498 B/op 291 allocs/op

    After $ go test -benchmem -bench=. PASS BenchmarkDecodeAll 10000 277428 ns/op 7681 B/op 160 allocs/op BenchmarkUnmarshalAll 5000 528511 ns/op 12674 B/op 274 allocs/op

    I tried to send you this patch list before, but code.google.com doesn't have a nice "pull request" flow..

  • Only wrap parse reader in bufio.Reader when necessary

    Only wrap parse reader in bufio.Reader when necessary

    This change updates the parse function to only wrap the provided reader in a bufio.Reader when it doesn't fulfill the bencode.Reader interface. This allows parsing of a stream of concatenated bencoded messages.

  • Fixes

    Fixes

    Found by 'go vet'.

    After these fixes are applied, some errors remains, not sure how to handle these:

    $ go vet
    bencode_test.go:231: struct field tag "a" not compatible with reflect.StructTag.Get: bad syntax for struct tag pair
    bencode_test.go:237: struct field tag "t" not compatible with reflect.StructTag.Get: bad syntax for struct tag pair
    bencode_test.go:238: struct field tag "y" not compatible with reflect.StructTag.Get: bad syntax for struct tag pair
    bencode_test.go:239: struct field tag "q" not compatible with reflect.StructTag.Get: bad syntax for struct tag pair
    bencode_test.go:240: struct field tag "a" not compatible with reflect.StructTag.Get: bad syntax for struct tag pair
    exit status 1
    
  • Make parsing more forgiving of non-standard bencoded integers.

    Make parsing more forgiving of non-standard bencoded integers.

    Since the original bencode standard doesn't allow floats, some software have unfortunately decided to bencode floats as integers. This is a stupid idea, because they could've just used a string. But since .torrent files with this feature already exist, we should try to handle them gracefully. This commit allows the parsing of floats & scientific notation, and adds testing for them, casting them to int64 under the following rules: -All floats are rounded to the adjacent integer with the lowest absolute value (7.5 is rounded to 7, while -7.5 is rounded to -7). -Floats that exceed the positive or negative bounds of int64 representation are set as -9223372036854775808.

  • Skip encoding unexported fields.

    Skip encoding unexported fields.

    Unlike the json encoder, this bencode implementation tries to encode unexported struct fields. Would it make sense to change the behavior to ignore such fields? I can work on PR.

  • Unmarshal(): Reader is modified but access to bufio.Reader is lost

    Unmarshal(): Reader is modified but access to bufio.Reader is lost

    I noticed that bencode.Unmarshal() creates a bufio.Reader from the passed-in reader. After bencode.Unmarshal() returns, the passed-in reader is left modified with the new index that the bufio.reader used to fill its buffer. This leaves the caller of bencode.Unmarshal() with a reader that is likely to be at the incorrect index from where the bufio.Reader left off.

    Some solutions I can think of off the top of my head:

    • Force bencode.Unmarshal() to take in a bufio.Reader not just any io.reader
    • Return the bufio.Reader that was created from the passed-in reader (from this point on, you must use only the buffered reader to do all the reads from the underlying reader)
  • Accessing raw bytes of a bencoded value

    Accessing raw bytes of a bencoded value

    From BitTorrent's BEP 3, we need to calculate a hash of the info value (itself a bencoded dict) from a .torrent file:

    Note that this is a substring of the metainfo file. The info-hash must be the hash of the encoded form as found in the .torrent file, which is identical to bdecoding the metainfo file, extracting the info dictionary and encoding it if and only if the bdecoder fully validated the input (e.g. key ordering, absence of leading zeros). Conversely that means clients must either reject invalid metainfo files or extract the substring directly. They must not perform a decode-encode roundtrip on invalid data.

    Because info has optional fields like private, it's impossible to predictably unmarshal/marshal it to calculate the hash. It would be nice if there were a way to access the raw bytes of a bencoded value.

  • Out of memory when Decode is passed certain invalid data

    Out of memory when Decode is passed certain invalid data

    When the data contains a large string length as in the example below, decodeString tries to allocate all of this memory upfront so the program crashes.

    package main
    
    import (
    	"bytes"
    	"fmt"
    
    	"github.com/jackpal/bencode-go"
    )
    
    func main() {
    	data := []byte("99999999999:")
    	buf := bytes.NewBuffer(data)
    	res, err := bencode.Decode(buf)
    	fmt.Println(res, err)
    }
    
Powerful and versatile MIME sniffing package using pre-compiled glob patterns, magic number signatures, XML document namespaces, and tree magic for mounted volumes, generated from the XDG shared-mime-info database.

mimemagic Powerful and versatile MIME sniffing package using pre-compiled glob patterns, magic number signatures, xml document namespaces, and tree ma

Nov 3, 2022
Protobuf3 with Interface support - Designed for blockchains (deterministic, upgradeable, fast, and compact)

Amino Spec (and impl for Go) This software implements Go bindings for the Amino encoding protocol. Amino is an object encoding specification. It is a

Dec 31, 2022
peer-to-peer file sharing

what i want is a tool to use to send files my many virtual machines. I want to do this myself, and i want to make it work as expected. So maybe a daem

Jun 13, 2022
Zero Trust Network Communication Sentinel provides peer-to-peer, multi-protocol, automatic networking, cross-CDN and other features for network communication.
Zero Trust Network Communication Sentinel provides peer-to-peer, multi-protocol, automatic networking, cross-CDN and other features for network communication.

Thank you for your interest in ZASentinel ZASentinel helps organizations improve information security by providing a better and simpler way to protect

Nov 1, 2022
Peer-to-peer hypermedia protocol
Peer-to-peer hypermedia protocol

IPFS powers the Distributed Web A peer-to-peer hypermedia protocol to make the web faster, safer, and more open. TL;DR Get help and talk about ideas i

Jan 5, 2023
bencode is a golang package for bencoding and bdecoding data from and from to equivalents.

Bencode bencode is a golang package for bencoding and bdecoding data from and from to equivalents. Bencode (pronounced like Ben-code) is the encoding

Jan 8, 2022
A project outputs Bluetooth Low Energy (BLE) sensors data in InfluxDB line protocol formatA project outputs Bluetooth Low Energy (BLE) sensors data in InfluxDB line protocol format

Intro This project outputs Bluetooth Low Energy (BLE) sensors data in InfluxDB line protocol format. It integrates nicely with the Telegraf execd inpu

Apr 15, 2022
📦 Command line peer-to-peer data transfer tool based on libp2p.

pcp - Peer Copy Command line peer-to-peer data transfer tool based on libp2p. Table of Contents Motivation Project Status How does it work? Usage Inst

Jan 5, 2023
pcp - 📦 Command line peer-to-peer data transfer tool based on libp2p.
pcp - 📦 Command line peer-to-peer data transfer tool based on libp2p.

pcp - Command line peer-to-peer data transfer tool based on libp2p.

Jan 5, 2023
Berty is a secure peer-to-peer messaging app that works with or without internet access, cellular data or trust in the network
Berty is a secure peer-to-peer messaging app that works with or without internet access, cellular data or trust in the network

?? Berty is a secure peer-to-peer messaging app that works with or without internet access, cellular data or trust in the network Introduction Berty i

Dec 29, 2022
Dependency-Free Bencode Editor

rbEdit A statically compiled and dependency-free Bencode editor in Go, useful for command line use and scripts. Quick Start # Compile for linux arch:

Dec 28, 2022
A Go library for master-less peer-to-peer autodiscovery and RPC between HTTP services

sleuth sleuth is a Go library that provides master-less peer-to-peer autodiscovery and RPC between HTTP services that reside on the same network. It w

Dec 28, 2022
A utility library to make use of the X Go Binding easier. (Implements EWMH and ICCCM specs, key binding support, etc.)

xgbutil is a utility library designed to work with the X Go Binding. This project's main goal is to make various X related tasks easier. For example,

Dec 10, 2022
DeepValueNetwork is a peer-to-peer database network managed and hosted by its community.

DeepValueNetwork To understand what DeepValueNetwork will be, I suggest you read this document. In progress This software is currently being developed

Dec 10, 2022
A utility library to make use of the X Go Binding easier. (Implements EWMH and ICCCM specs, key binding support, etc.)

xgbutil is a utility library designed to work with the X Go Binding. This project's main goal is to make various X related tasks easier. For example,

Dec 10, 2022
Group peer to peer video calls for everyone written in Go and TypeScript

Peer Calls v4 WebRTC peer to peer calls for everyone. See it live in action at peercalls.com. The server has been completely rewriten in Go and all th

Dec 30, 2022
Simple Shamir's Secret Sharing (s4) - A go package giving a easy to use interface for the shamir's secret sharing algorithm

Simple Shamir's Secret Sharing (s4) With Simple Shamir's Secret Sharing (s4) I want to provide you an easy to use interface for this beautiful little

Jan 2, 2023
False-sharing-demo - Demo for performance effects of CPU cache false-sharing

Example of CPU cache false-sharing in Go. A simple example where 2 integer varia

Aug 28, 2022
Prometheus Common Data Exporter can parse JSON, XML, yaml or other format data from various sources (such as HTTP response message, local file, TCP response message and UDP response message) into Prometheus metric data.
Prometheus Common Data Exporter can parse JSON, XML, yaml or other format data from various sources (such as HTTP response message, local file, TCP response message and UDP response message) into Prometheus metric data.

Prometheus Common Data Exporter Prometheus Common Data Exporter 用于将多种来源(如http响应报文、本地文件、TCP响应报文、UDP响应报文)的Json、xml、yaml或其它格式的数据,解析为Prometheus metric数据。

May 18, 2022
An imageboard, but images are stored in a peer-to-peer network
An imageboard, but images are stored in a peer-to-peer network

Interplanetary File Dumpster An imageboard, but images are stored in a peer-to-peer network Features: Easy file sharing without registration and SMS.

Sep 30, 2022