Implements the XDR standard as specified in RFC 4506 in pure Go (Golang)

go-xdr

[Build Status] (https://travis-ci.org/davecgh/go-xdr) [![Coverage Status] (https://coveralls.io/repos/davecgh/go-xdr/badge.png?branch=master)] (https://coveralls.io/r/davecgh/go-xdr?branch=master)

Go-xdr implements the data representation portion of the External Data Representation (XDR) standard protocol as specified in RFC 4506 (obsoletes RFC 1832 and RFC 1014) in Pure Go (Golang). A comprehensive suite of tests are provided to ensure proper functionality. It is licensed under the liberal ISC license, so it may be used in open source or commercial projects.

NOTE: Version 1 of this package is still available via the github.com/davecgh/go-xdr/xdr import path to avoid breaking existing clients. However, it is highly recommended that all old clients upgrade to version 2 and all new clients use version 2. In addition to some speed optimizations, version 2 has been been updated to work with standard the io.Reader and io.Writer interfaces instead of raw byte slices. This allows it to be much more flexible and work directly with files, network connections, etc.

Documentation

[GoDoc] (http://godoc.org/github.com/davecgh/go-xdr/xdr2)

Full go doc style documentation for the project can be viewed online without installing this package by using the excellent GoDoc site here: http://godoc.org/github.com/davecgh/go-xdr/xdr2

You can also view the documentation locally once the package is installed with the godoc tool by running godoc -http=":6060" and pointing your browser to http://localhost:6060/pkg/github.com/davecgh/go-xdr/xdr2/

Installation

$ go get github.com/davecgh/go-xdr/xdr2

Sample Decode Program

package main

import (
	"bytes"
    "fmt"

    "github.com/davecgh/go-xdr/xdr2"
)

func main() {
	// Hypothetical image header format.
	type ImageHeader struct {
		Signature   [3]byte
		Version     uint32
		IsGrayscale bool
		NumSections uint32
	}

	// XDR encoded data described by the above structure.  Typically this would
	// be read from a file or across the network, but use a manual byte array
	// here as an example.
	encodedData := []byte{
		0xAB, 0xCD, 0xEF, 0x00, // Signature
		0x00, 0x00, 0x00, 0x02, // Version
		0x00, 0x00, 0x00, 0x01, // IsGrayscale
		0x00, 0x00, 0x00, 0x0A, // NumSections
	}

	// Declare a variable to provide Unmarshal with a concrete type and instance
	// to decode into.
	var h ImageHeader
	bytesRead, err := xdr.Unmarshal(bytes.NewReader(encodedData), &h)
	if err != nil {
		fmt.Println(err)
		return
	}
  
	fmt.Println("bytes read:", bytesRead)
	fmt.Printf("h: %+v", h)
}

The struct instance, h, will then contain the following values:

h.Signature = [3]byte{0xAB, 0xCD, 0xEF}
h.Version = 2
h.IsGrayscale = true
h.NumSections = 10

Sample Encode Program

package main

import (
	"bytes"
    "fmt"

    "github.com/davecgh/go-xdr/xdr2"
)

func main() {
	// Hypothetical image header format.
	type ImageHeader struct {
		Signature   [3]byte
		Version     uint32
		IsGrayscale bool
		NumSections uint32
	}

	// Sample image header data.
	h := ImageHeader{[3]byte{0xAB, 0xCD, 0xEF}, 2, true, 10}

	// Use marshal to automatically determine the appropriate underlying XDR
	// types and encode.
	var w bytes.Buffer
	bytesWritten, err := xdr.Marshal(&w, &h)
	if err != nil {
		fmt.Println(err)
		return
	}

	encodedData := w.Bytes()
	fmt.Println("bytes written:", bytesWritten)
	fmt.Println("encoded data:", encodedData)
}

The result, encodedData, will then contain the following XDR encoded byte sequence:

0xAB, 0xCD, 0xEF, 0x00,
0x00, 0x00, 0x00, 0x02,
0x00, 0x00, 0x00, 0x01,
0x00, 0x00, 0x00, 0x0A,

License

Go-xdr is licensed under the liberal ISC License.

Comments
  • Accepting contributions?

    Accepting contributions?

    Hi @davecgh,

    Are you still maintaining this package and would you be open to accepting a contribution that adds an xdr3 package, that is a copy of xdr2 but adds optionals, unions, and change how pointers work to support using optionals?

    We've been using a modified version of this package here: https://github.com/stellar/go-xdr and if you'd be up for us bringing our changes upstream we'd be up for preparing a PR.

    This is approximately what the diff would look like, but we'd also need to merge downstream some of the changes in your master branch before we opened, so there would be some differences to keep xdr3 as a copy of xdr2 at the point in time the PR is opened: https://github.com/davecgh/go-xdr/compare/master...stellar:master

    No worries if you're no longer maintaining go-xdr, and if that is the case we'll move forward with developing our fork of go-xdr at https://github.com/stellar/go-xdr.

    Cheers, Leigh

    cc @bartekn @ire-and-curses @marcopeereboom @rasky

  • Support for time.Time

    Support for time.Time

    I added XDR to my go_serialization_benchmarks and it is currently the fastest.

    Unfortunately, it does not correctly serialize time.Time. This is not surprising as it isn't supported, but it would be great to support it if possible.

    It could perhaps use a scheme similar to JSON, marshaling to RFC3339.

  • unexpected EOF while decoding 260 bytes

    unexpected EOF while decoding 260 bytes

    Maybe you could help me solving my issue.

    I've posted all necessary information in issue https://github.com/prashanthpai/sunrpc/issues/3 because I thought it could be an issue with his library. Now we figured out that it is more an issue with variable length opaque data on the decoding side ;)

    If you want me to paste all information from the other issue in this issue, I'll do that.

    Thanks :)

  • Support for optional data (4.19) of the XDR spec

    Support for optional data (4.19) of the XDR spec

    I tried decoding optional-data 4.19 and came across this comment in the code

    https://github.com/davecgh/go-xdr/blob/master/xdr2/decode.go#L572

    // RFC Sections 4.15 though 4.19 only apply to the data specification language
    // which is not implemented by this package
    

    I don't understand the "only apply to the data specification language" bit. The API I'm using (NFSv3, to be specific) is encoding a linked-list with the above optional-data format and other than decoding each element and then checking for the "follows" 4B for zero before terminating (which is what I'm currently doing here), is there a way to decode this data?

    If the loop-and-check method is the only way, are you averse to me pushing a PR to add support for this? I'm thinking it would require a StructTag to descend down the right path for a slice. Let me know and I'll be happy to get started on that.

    Thanks!

  • Please add max size to decoder.

    Please add max size to decoder.

    When reading XDR directly of the wire a remote malicious person could craft packets that could use up all of machines memory because there are no limits. It would be helpful if we could set a limit on how large a read can be. For example:

    dec := NewDecoderLimited(myReader, 1024*1024) // limit to 1MB
    _, err := dec.Decode(&myXDR)
    if err == xdr.ErrTooBig {
            myReader.Close()
            return err
    }
    
  • Add a max decoder limit in order to prevent crashes.

    Add a max decoder limit in order to prevent crashes.

    Currently, it is very simple to crash go-xdr by fuzzing a wire protocol. This PR adds a max reader size so that we don't try to allocate giants amount of memory by validating the incoming size first.

    Test coverage is 100%

  • Add support for discriminated unions and optional data

    Add support for discriminated unions and optional data

    This PR adds support for:

    • Discriminated unions (RFC 4506, 4.15)
    • Optional data (RFC 4506, 4.19)

    In both cases, the code relies on struct tags to convey the required metadata information. Please see the individual commits for more information.

Oct 8, 2022
A commandline tool to resolve URI Templates expressions as specified in RFC 6570.

URI Are you tired to build, concat, replace URL(s) (via shell scripts sed/awk/tr) from your awesome commandline pipeline? Well! here is the missing pi

Jun 9, 2021
A go implementation of the STUN client (RFC 3489 and RFC 5389)

go-stun go-stun is a STUN (RFC 3489, 5389) client implementation in golang (a.k.a. UDP hole punching). RFC 3489: STUN - Simple Traversal of User Datag

Jan 5, 2023
CBOR RFC 7049 (Go/Golang) - safe & fast with standard API + toarray & keyasint, CBOR tags, float64/32/16, fuzz tested.
CBOR RFC 7049 (Go/Golang) - safe & fast with standard API + toarray & keyasint, CBOR tags, float64/32/16, fuzz tested.

CBOR library in Go fxamacker/cbor is a CBOR encoder & decoder in Go. It has a standard API, CBOR tags, options for duplicate map keys, float64→32→16,

Jan 6, 2023
Package arp implements the ARP protocol, as described in RFC 826. MIT Licensed.

arp Package arp implements the ARP protocol, as described in RFC 826. MIT Licensed. Portions of this code are taken from the Go standard library. The

Dec 20, 2022
Package dhcp6 implements a DHCPv6 server, as described in RFC 3315. MIT Licensed.

dhcp6 Package dhcp6 implements a DHCPv6 server, as described in IETF RFC 3315. MIT Licensed. At this time, the API is not stable, and may change over

Sep 27, 2022
Package json implements encoding and decoding of JSON as defined in RFC 7159

Package json implements encoding and decoding of JSON as defined in RFC 7159. The mapping between JSON and Go values is described in the documentation for the Marshal and Unmarshal functions

Jun 26, 2022
Reload a specified go program automatically by monitoring a directory.

gowatcher A bash script to automatically reload a go program when .go or .html files change in the monitored directory. It uses inotify. Installation

Jul 7, 2020
:runner:runs go generate recursively on a specified path or environment variable and can filter by regex

Package generate Package generate runs go generate recursively on a specified path or environment variable like $GOPATH and can filter by regex Why wo

Sep 27, 2022
A concurrent map with entries expiring after a specified interval.

go-ttlmap Go TTL Map is a concurent map with entries expiring after a specified interval. This package requires go1.14 or newer. Overview This impleme

Oct 13, 2021
daemon for removing torrents from deluge after a specified timeframe

deluge-remove-after -- daemon for removing torrents from deluge after a specified timeframe Table of Contents Why Installation Docker Build from sourc

Jul 10, 2022
Attach services to specified networks automatically

Docker swarm network attacher Description docker-swarm-network-attacher aims to solve the problem of sharing a network between unrelated services. Wit

Nov 11, 2021
Server that forwards the tx to the specified smart contract

tx-forwarder Server that forwards the tx to the specified smart contract. Usage In production use ./tx-forwarder, in development use go run main.go in

Dec 6, 2021
A password manager as a CLI, where you can use a master password to retrieve a specified password and store it in your clipboard
A password manager as a CLI, where you can use a master password to retrieve a specified password and store it in your clipboard

Password manager Description CLI to store and retrieve passwords. The retrieved password will be stored on your clipboard! Usage 1.Start with Go go ru

Dec 16, 2021
A kubernetes operator sample generated by kubebuilder , which run cmd in pod on specified time

init kubebuilder init --domain github.com --repo github.com/tonyshanc/sample-operator-v2 kubebuilder create api --group sample --version v1 --kind At

Jan 25, 2022
Simple docker container to publish a fixed message to a specified queue. Created to be used with k8s CRON scheduling.

RabbitMQ Publish CRON Simple docker container to publish a fixed message to a specified rabbitmq exchange. Created to be used as part of a Kubernetes

Dec 20, 2021
Launch parallel processes (shuttles) with conditional parameter(s) specified in text file(s)

~# shuttle Launch some shuttles here and there. Usage Usage example: # Launch as many sqlmaps as the lines in targets.txt # with 4 maximum istances at

Jan 8, 2022
DeepCopy a portable app that allows you to copy all forms of specified file types from your entire file system of the computer

DeepCopy a portable app that allows you to copy all forms of specified file types from your entire file system of the computer

Dec 20, 2021
Print specified values from desktop files to stdout.
Print specified values from desktop files to stdout.

dprint Print specified values from desktop files to stdout. Look, it’s hard to describe okay? Here’s a picture of me using it with dmenu. My launcher

Dec 22, 2021
betterbattery prints the battery percentage, status, and can run a command if the percentage fell below a specified value since it was last ran.

betterbattery betterbattery prints the battery percentage, status, and can run a command if the percentage fell below a specified value since it was l

Dec 22, 2021