An always-on framework that performs end-to-end functional network testing for reachability, latency, and packet loss

Arachne GoDoc Build Status Go Report Card

Arachne is a packet loss detection system and an underperforming path detection system. It provides fast and easy active end-to-end functional testing of all the components in Data Center and Cloud infrastructures. Arachne is able to detect intra-DC, inter-DC, DC-to-Cloud, and DC-to-External-Services issues by generating minimal traffic:

  • Reachability
  • Round-trip and 1-way latency
  • Silent packet drops and black holes
  • Jitter (average of the deviation from the network mean latency)
  • PMTU or Firewall issues too related possibly to network config changes (accidental or not)
  • Whether network-level SLAs are met

Usage

There are two ways to use the Arachne package.

As a standalone program

Run Arachne as a standalone program (it's Debian packaged already too).

As a library in your own program

Import this package and call Arachne from your program/service with

    arachne.Run(config, arachne.ReceiverOnlyMode(false))

where the option provided above is among the few optional ones.

Below is the list of all the CLI options available, when Arachne is used as a standalone program. The default options should be good enough for most users.

$ arachne --help

____________________________________________________________/\\\______________________________________
 ___________________________________________________________\/\\\______________________________________
  ___________________________________________________________\/\\\______________________________________
   __/\\\\\\\\\_____/\\/\\\\\\\___/\\\\\\\\\________/\\\\\\\\_\/\\\__________/\\/\\\\\\_______/\\\\\\\\__
    _\////////\\\___\/\\\/////\\\_\////////\\\_____/\\\//////__\/\\\\\\\\\\__\/\\\////\\\____/\\\/////\\\_
     ___/\\\\\\\\\\__\/\\\___\///____/\\\\\\\\\\___/\\\_________\/\\\/////\\\_\/\\\__\//\\\__/\\\\\\\\\\\__
      __/\\\/////\\\__\/\\\__________/\\\/////\\\__\//\\\________\/\\\___\/\\\_\/\\\___\/\\\_\//\\///////___
       _\//\\\\\\\\/\\_\/\\\_________\//\\\\\\\\/\\__\///\\\\\\\\_\/\\\___\/\\\_\/\\\___\/\\\__\//\\\\\\\\\\_
        __\////////\//__\///___________\////////\//_____\////////__\///____\///__\///____\///____\//////////__


Usage: arachne [--foreground] [-c=] [--receiver_only] [--sender_only]

Arachne is a packet loss detection system and an underperforming path detection
system for Data Center and Cloud infrastructures.

Options:
  -v, --version                                     Show the version and exit
  --foreground=false                                Force foreground mode
  -c, --config_file="/usr/local/etc/arachne/arachne.yaml"     Config file path
  (default: /usr/local/etc/arachne/arachne.yaml)
  --receiver_only=false                             Force TCP receiver-only mode
  --sender_only=false                               Force TCP sender-only mode

Note on required privileges to run

Arachne is granted access to raw sockets without the need to run with sudo or as root user, by being granted CAP_NET_RAW capability (see: capabilities).

Note on BPF filtering

When receiving packets, Arachne attempts to apply a BPF filter to the raw socket so that processing of packets occurs on a much smaller set (ones destined specifically for Arachne agent testing). This is currently supported only on Linux and thus performance will be worse on BSD-based systems where a larger number of packets must be inspected.


Released under the MIT License.

Owner
Uber Open Source
Open Source Software at Uber
Uber Open Source
Comments
  • QoS DSCP value not matching one of supported classes

    QoS DSCP value not matching one of supported classes

    {"level":"error","ts":1496825431.2770894,"msg":"QoS DSCP value not matching one of supported classes","PID":1124,"hostname":"hostname1","service_name":"arachne","DSCP_value":20,"supported_classes":"[0 4 40 56 72 88 104 120 160 192 224]" }
    

    my centos 7 hosts encounter above error, any way to make it work?

  • Docs improvement

    Docs improvement

    Can't figure how to run this If I simply add all hosts to internal section of config.json and run, it should work, right?

    Getting empty (Timed Out) results so far

  • Make bins errors with dependency on go.uber.org/zap

    Make bins errors with dependency on go.uber.org/zap

    I'm unable to build the arachne binary because of dependencies on zap.

    $ make bins
    builddir build
    go build -o build/arachned github.com/uber/arachne/arachned/
    bootstrap.go:28:2: code in directory /home/mchrobak/.gvm/pkgsets/go1.7.5/global/src/github.com/uber-go/zap expects import "go.uber.org/zap"
    make: *** [bins] Error 1
    
  • New Socket created for each outbound packet

    New Socket created for each outbound packet

    In tcp/tcp.go, the send function creates a new raw socket each time it gets called.

    This causes unnecessary overhead. Socket's should be created and passed along to the send code.

    Note: The DSCP value is currently being set on each socket file descriptor, rather than in a manually created IP header, so this will have to occur on the shared socket at the start of each batch cycle.

  • Config refactor

    Config refactor

    Before opening your pull request, please make sure that you have:

    • [x] signed Uber's Contributor License Agreement;
    • [x] added tests to cover your changes;
    • [x] run the test suite locally (make test); and finally,
    • [x] run the linters locally (make lint).

    Thanks for your contribution!

  • Update dependencies

    Update dependencies

    Before opening your pull request, please make sure that you have:

    • [x] signed Uber's Contributor License Agreement;
    • [ ] added tests to cover your changes;
    • [x] run the test suite locally (make test); and finally,
    • [x] run the linters locally (make lint).

    Thanks for your contribution!

  • Change the path from /etc to /usr/local/etc for config

    Change the path from /etc to /usr/local/etc for config

    Before opening your pull request, please make sure that you have:

    • [ ] signed Uber's Contributor License Agreement;
    • [ ] added tests to cover your changes;
    • [ ] run the test suite locally (make test); and finally,
    • [ ] run the linters locally (make lint).

    Thanks for your contribution!

  • Add location info for targets being echoed

    Add location info for targets being echoed

    Mostly for metrics purposes, it would be very helpful if the targets specified to be echoed have an additional location field.

    Because differents users/orgs perceive/treat location differently, what would suffice is for this location field to be a single string field.

  • Add shared send socket and pcap functionality

    Add shared send socket and pcap functionality

    This commit adds the underlying code that will be used in the future to improve the overall sending and receiving of echo Messages. These coming future changes will address the following:

    1. Using a single, shared, file descriptor (raw socket) for the sending of Echo requests. Currently, Arachne creates a new raw socket for each outbound packet, which is not ideal.

    2. Using pcap for receiving incoming echo requests from other agents. This will allow the agents to benefit from BPF filters and thus not have to process all the incoming TCP packets to the system that Arachne is running on (which currently causes a significant overhead).

  • CleanUpRefresh attempts to close nil channel

    CleanUpRefresh attempts to close nil channel

    If ResolveDNS is off in the configuration, the killC.DNSRefresh channel will not get created (bootstrap.go line 122). This causes a panic when util.go/CleanUpRefresh attempts to close the nil channel.

  • Add support for zap v1.x APIs

    Add support for zap v1.x APIs

    Before opening your pull request, please make sure that you have:

    • [x] signed Uber's Contributor License Agreement;
    • [x] added tests to cover your changes;
    • [x] run the test suite locally (make test); and finally,
    • [x] run the linters locally (make lint).

    Thanks for your contribution!

  • I can not see all the metrics when I am running the tool

    I can not see all the metrics when I am running the tool

    Hi,

    Can you please let me know how can I get the following info when I am running the tool?

    Silent packet drops and black holes Jitter (average of the deviation from the network mean latency) PMTU or Firewall issues too related possibly to network config changes (accidental or not)

    right now the only thing that I get is:

    Reachability Round-trip and 1-way latency

    Thanks

  • timed_out: yes  while traversing a firewall

    timed_out: yes while traversing a firewall

    Hello, We are using Arachne to test our network health. It works perfectly fine in our lab environment which has hosts in different vlans . However, on deploying it on a fire-walled environment I can see timed_out: yes on all the separated hosts. The tcpdump capture does not show any issues and the behavior is same as on a lab environment. The firewall seems to block repeated new connections on the same ports and filter out ACK packets. Is there any recommended method to deploy Arachne in an environment where the hosts are firewall separated. Please let me know. Thank you.

  • Introduce traceroute-like based device detection of underperforming network paths

    Introduce traceroute-like based device detection of underperforming network paths

    Trigger manually a traceroute-like troubleshooting mode for a specific 5-tuple hash (only modifying TTL) so that all network devices along a path are identified.

  • many timeout: yes spot out

    many timeout: yes spot out

    within our cloud deployment, for internal, with nearly 10+ hosts inside the array .

    I can see many "timed_out":"yes" lines from the log output, and from statsd there is did no metrics between these 2 hosts. the communication seems working well under tcpdump view.

    any other reason can makeing this timeout happen? will this caused by host_name instead of ip inside internal array?

πŸš€πŸŒ Orbital is a simple end-to-end testing framework for Go

Orbital is a test framework which enables a developer to write end to end tests just like one would writing unit tests. We do this by effectively copying the testing.T API and registering tests to be run periodically on a configured schedule.

Nov 18, 2022
End-to-end HTTP and REST API testing for Go.

httpexpect Concise, declarative, and easy to use end-to-end HTTP and REST API testing for Go (golang). Basically, httpexpect is a set of chainable bui

Jan 5, 2023
Expressive end-to-end HTTP API testing made easy in Go

baloo Expressive and versatile end-to-end HTTP API testing made easy in Go (golang), built on top of gentleman HTTP client toolkit. Take a look to the

Dec 13, 2022
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.

Fortio Fortio (Φορτίο) started as, and is, Istio's load testing tool and now graduated to be its own project. Fortio is also used by, among others, Me

Jan 2, 2023
A simple and expressive HTTP server mocking library for end-to-end tests in Go.

mockhttp A simple and expressive HTTP server mocking library for end-to-end tests in Go. Installation go get -d github.com/americanas-go/mockhttp Exa

Dec 19, 2021
Rr-e2e-tests - Roadrunner end-to-end tests repository
Rr-e2e-tests - Roadrunner end-to-end tests repository

RoadRunner end-to-end plugins tests License: The MIT License (MIT). Please see L

Dec 15, 2022
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

Dec 12, 2022
A yaml data-driven testing format together with golang testing library

Specimen Yaml-based data-driven testing Specimen is a yaml data format for data-driven testing. This enforces separation between feature being tested

Nov 24, 2022
Testing framework for Go. Allows writing self-documenting tests/specifications, and executes them concurrently and safely isolated. [UNMAINTAINED]

GoSpec GoSpec is a BDD-style testing framework for the Go programming language. It allows writing self-documenting tests/specs, and executes them in p

Nov 28, 2022
Minimal and Beautiful Go testing framework
Minimal and Beautiful Go testing framework

Goblin A Mocha like BDD testing framework written in Go that requires no additional dependencies. Requires no extensive documentation nor complicated

Dec 25, 2022
Full-featured test framework for Go! Assertions, mocking, input testing, output capturing, and much more! πŸ•
Full-featured test framework for Go! Assertions, mocking, input testing, output capturing, and much more! πŸ•

testza ?? Testza is like pizza for Go - you could life without it, but why should you? Get The Module | Documentation | Contributing | Code of Conduct

Dec 10, 2022
Golang HTTP client testing framework

flute Golang HTTP client testing framework Presentation https://speakerdeck.com/szksh/flute-golang-http-client-testing-framework Overview flute is the

Sep 27, 2022
API testing framework inspired by frisby-js
API testing framework inspired by frisby-js

frisby REST API testing framework inspired by frisby-js, written in Go Proposals I'm starting to work on frisby again with the following ideas: Read s

Sep 27, 2022
BDD Testing Framework for Go
BDD Testing Framework for Go

Jump to the docs | δΈ­ζ–‡ζ–‡ζ‘£ to learn more. To start rolling your Ginkgo tests now keep reading! If you have a question, comment, bug report, feature reque

Jan 1, 2023
Framework of performance testing

Framework of performance testing fperf is a powerful and flexible framework which allows you to develop your own benchmark tools so much easy. You cre

Nov 30, 2022
Professional lightweight testing mini-framework for Go.
Professional lightweight testing mini-framework for Go.

is Professional lightweight testing mini-framework for Go. Easy to write and read Beautifully simple API with everything you need: is.Equal, is.True,

Dec 28, 2022
espresso - a framework for testing BigQuery queries

espresso - a framework for testing BigQuery queries Goals Componentization: compose complex queries from smaller, reusable components Test driven deve

Dec 7, 2022
Testy is a Go test running framework designed for Gametime's API testing needs.

template_library import "github.com/gametimesf/template_library" Overview Index Overview Package template_library is a template repository for buildin

Jun 21, 2022
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs

go-testdeep Extremely flexible golang deep comparison, extends the go testing package. Latest news Synopsis Description Installation Functions Availab

Dec 22, 2022