The Akita CLI for watching network traffic, automatically generating API specs, and diffing API specs.

Catch breaking changes faster

Akita builds models of your APIs to help you:

  • Catch breaking changes on every pull request, including added/removed endpoints, added/removed fields, modified types, modified data types
  • Check compliance with intended behavior
  • Auto-generate up-to-date API specs

In addition to recording traffic, Akita provides:

  • Path generalization for endpoints
  • Type and data format inference (docs)
  • Integrations with CI (docs) and source control (GitHub; GitLab)
  • Integrations with web frameworks to watch integration tests (docs)

See the full Akita docs here. Watch the first 5 minutes of this video for a demo.

Sign up for our private beta here.

About this repo

This is the open-source repository for our CLI, containing the code for:

  • apidump for listening to API traffic and generating HAR files
  • apispec for generating API specs from HAR files
  • apidiff for diffing API specs

The CLI is intended for use with the Akita SaaS tool. This repository does not contain our path generalization, type and data format, or spec generation implementations.

How to build

  1. Install Go 1.15 or above.
  2. go build .
Owner
Akita Software
Catch breaking changes faster! Akita watches API traffic to build models (and API specs). Try us out in just 5 minutes, no code changes or proxies necessary.
Akita Software
Comments
  • Catch panics during packet reassembly.

    Catch panics during packet reassembly.

    This covers the known bug... but, it's not clear whether the reassembly layer can successfully recover afterwards, or if additional cleanup is needed.

  • Handle HTTP and 'other' body types

    Handle HTTP and 'other' body types

    Depends on: https://github.com/akitasoftware/akita-ir/pull/6 and https://github.com/akitasoftware/akita-libs/pull/96

    I'm wondering if this will cause us to spend a lot of time examining things like png or jpg images returned from an endpoint.

    One mitigation is to tune MaxBufferedBody way, way down, it's currently at 5MB.

    I didn't feel comfortable making the body a "None" in a "Oneof None String" field. This will at least give us some information about whether small response bodies are identical. But I'd be happy to come up with a different way of signaling "we didn't expect to understand the body format, so we didn't try."

  • Detect when packet capture fails from architecture mismatch

    Detect when packet capture fails from architecture mismatch

    When running the Akita agent in a Docker container built for a different architecture than the host (e.g. built for amd64 but running on arm64), the agent fails to read any network interfaces with error messages of the form SIOCETHTOOL(ETHTOOL_GET_TS_INFO) ioctl failed: Function not implemented.

    This PR extends the agent to detect these errors and print a nicer error message for the user that indicates the underlying problem. It also adds the target architecture to the akita --version output, e.g. akita version 0.21.18 (09c7847085262f10ea9e2cd4bae280383970e5c1-dirty, arm64).

    When running in Docker, the CLI output will look like this:

    > docker run -e AKITA_API_KEY_ID -e AKITA_API_KEY_SECRET akitasoftware/cli:latest apidump --project akibox
    
    WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64) and no specific platform was requested
    [INFO] Akita Agent 0.21.18-rc4
    [WARNING] Skipping interface eth0 for collecting packets because of error: failed to read packets from interface eth0: eth0: SIOCETHTOOL(ETHTOOL_GET_TS_INFO) ioctl failed: Function not implemented
    [WARNING] Skipping interface lo for collecting packets because of error: failed to read packets from interface lo: lo: SIOCETHTOOL(ETHTOOL_GET_TS_INFO) ioctl failed: Function not implemented
    [WARNING] The agent received "Function not implemented" when trying to read from your network interfaces. This often indicates that the Akita agent was built for a different architecture than your host architecture. This Akita agent binary was built for arm64.
    [ERROR] No network interfaces could be used: Unable to read network interfaces. If your host architecture is not arm64, try using `docker pull --platform $YOUR_ARCHITECTURE akitasoftware/cli:latest` to pull an Akita agent built for your architecture.
    

    When not in Docker, the last error message will instead read:

    [ERROR] No network interfaces could be used: Unable to read network interfaces.  If your host architecture is not arm64, try downloading an Akita agent built for your architecture from https://github.com/akitasoftware/akita-cli/releases.
    

    Testing

    I tested this by building a CLI docker image on arm64 and running it on an amd64 macOS laptop.

  • "apidump --out" creates empty folder / no har file?

    I'm trying to use akita apidump with --out to a local folder. The out folder gets created but is empty. I've tried both on MacOS and Linux, both built from source and with the official binary (mentioned in the docs) and got the same result. Am I doing anything wrong?

    Results received: Empty out folder, no .har file.

    Expected: .har file gets created in the out dir.

    Example:

    Before: out dir doesn't exist:

    $ ls -a
    .  .. 
    

    running as user root:

    $ akita apidump -c 'curl -I -L https://api.github.com/users/torvalds/events/public' -u root --out ./dumpdir
    [INFO] Running learn mode on interfaces lo, ens18, cni-podman0
    [WARNING] --filter flag is not set, this means that:
    [WARNING]   - all network traffic is treated as your API traffic
    [WARNING]   - outbound witness collection is disabled
    [INFO] Running subcommand...
    
    
    ======= _AKITA_SUBCOMMAND_ =======
    ======= _AKITA_SUBCOMMAND_ =======
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
      0  299k    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
    HTTP/2 200
    server: GitHub.com
    <...>
    
    ======= _AKITA_SUBCOMMAND_ =======
    ======= _AKITA_SUBCOMMAND_ =======
    [INFO] Subcommand finished successfully, stopping trace collection...
    [INFO] Success! πŸŽ‰
    

    After: out dir got created but is empty:

    $ ls -a
    .  ..  dumpdir
    ls -a dumpdir/
    .  ..
    
  • Report on count of TLS handshake messages.

    Report on count of TLS handshake messages.

    Depends on https://github.com/akitasoftware/akita-libs/pull/159

    Example output:

    mark@ubuntu:~/akita-cli$ sudo -E bin/akita apidump --project mgg-test 
    [INFO] Akita Agent 0.0.0
    [INFO] Created new trace on Akita Cloud: akita://mgg-test:trace:lace-pig-d35eeeab
    [INFO] Running learn mode on interfaces lo, ens33, docker0, br-a17c933efd5a
    [WARNING] --filter flag is not set, this means that all network traffic is treated as your API traffic
    [INFO] Send SIGINT (Ctrl-C) to stop...
    [INFO] Printing packet capture statistics after 60 seconds of capture.
    [INFO] ==================================================
    [INFO] Packets per interface:
    [INFO]                             TCP       HTTP      TLS      
    [INFO]       interface       dir packets   req  resp hello   unk
    [INFO]              lo   MATCHED       0     0     0     0   244
    [INFO]           ens33   MATCHED   29955    32    30   457 16791
    [INFO]         docker0   MATCHED       0     0     0     0     0
    [INFO] br-a17c933efd5a   MATCHED       0     0     0     0     0
    [INFO] ==================================================
    
  • Add Segment telemetry to CLI.

    Add Segment telemetry to CLI.

    Moved GetClientID() to telemetry package, and Domain to the rest package, to avoid accessing the "internal" package, which is forbidden by the compiler.

    Attempts to look up /v1/user for user identity, with several fallback options.

    Has an opt-out mechanism, and rate limit for frequent errors.

    Depends on https://github.com/akitasoftware/akita-libs/pull/154

    The Segment key must be compiled in to official builds, or provided as an environment variable.

  • Convert 'learn-sessions create' to use 'apispec'

    Convert 'learn-sessions create' to use 'apispec'

    This PR converts the legacy command akita learn-sessions create to call apispec under the hood. In the short term, this adds support to learn-sessions create for detecting CI environments. Longer term, it helps ensure the legacy command remains up to date.

  • FYI: Instalation through Homebrew is failing with ARM64

    FYI: Instalation through Homebrew is failing with ARM64

    Hi folks! I've tried to install Akita CLI though Homebrew no a Macbook with M1 processor, and I got this error:

    brew tap akitasoftware/akita && brew install akita-cli
    ==> Installing akita-cli from akitasoftware/akita
    ==> Downloading https://releases.akita.software/cli/0.17.7/akita_0.17.7_darwin_arm64.zip
    Already downloaded: /Users/gkbonetti/Library/Caches/Homebrew/downloads/c63d51f2bf2c8c82a40fd47b05666ac5c1cfd24c64b9cf642f3d4d8b30a237ea--akita_0.17.7_darwin_arm64.zip
    Error: SHA256 mismatch
    Expected: 256652350362e692a5e874abedf9a2c933303176e4e4f04668f2c0b62b3976e8
      Actual: 7df38396aa89250408186d80b2e01f3d7aef1063d678b93e2ffd44f8a25a84b9
        File: /Users/gkbonetti/Library/Caches/Homebrew/downloads/c63d51f2bf2c8c82a40fd47b05666ac5c1cfd24c64b9cf642f3d4d8b30a237ea--akita_0.17.7_darwin_arm64.zip
    To retry an incomplete download, remove the file above.
    

    I've tried deleting the downloaded file and trying again a few times but the result was always the same.

    Since I couldn't install it through Homebrew, I've installed it from source (following the Readme) and it seems to be working.

  • Add --allow-host and --allow-path flags

    Add --allow-host and --allow-path flags

    Only add filters that are enabled to the stack of collectors. Add a pre-filter packet count so that we can compare and warn if all packets we eliminated due to filters.

    [INFO] Captured 8 HTTP requests before allow and exclude rules, but all were filtered.
    [ERROR] No inbound HTTP calls captured! πŸ›‘
    
  • Register a modified ECS task definition.

    Register a modified ECS task definition.

    Check whether the task definition already contains an Akita container, or the tag we will add. The code to create an AWS secret is disabled for now, because using the secret requires changes to the assigned role. Recognize ECS-specific environment variables and tag the trace.

    Depends on https://github.com/akitasoftware/akita-libs/pull/177 for the new tags.

    Verified that the new task definition works by upgrading the service in the ECS console.

  • Count oversized witnesses and report to back end

    Count oversized witnesses and report to back end

    Add and maintain a count of oversized witnesses in client_telemetry.PacketCounts. This is not reported to the user, but is reported to the back end as part of packet telemetry.

A simple network analyzer that capture http network traffic
A simple network analyzer that capture http network traffic

httpcap A simple network analyzer that captures http network traffic. support Windows/MacOS/Linux/OpenWrt(x64) https only capture clienthello colorful

Oct 25, 2022
Apache Traffic Control is an Open Source implementation of a Content Delivery Network

Apache Traffic Control Apache Traffic Control is an Open Source implementation of a Content Delivery Network. Documentation Intro CDN Basics Traffic C

Jan 6, 2023
Schema-free, document-oriented streaming database that optimized for monitoring network traffic in real-time

Basenine Schema-free, document-oriented streaming database that optimized for monitoring network traffic in real-time. Featured Aspects Has the fastes

Nov 2, 2022
🐢 A modern alternative network traffic sniffer.
🐢  A modern alternative network traffic sniffer.

sniffer A modern alternative network traffic sniffer inspired by bandwhich(Rust) and nethogs(C++). sniffer.mov Introduction 中文介绍 sniffer is designed f

Dec 29, 2022
Capdns is a network capture utility designed specifically for DNS traffic. This utility is based on tcpdump.
Capdns is a network capture utility designed specifically for DNS traffic. This utility is based on tcpdump.

Capdns is a network capture utility designed specifically for DNS traffic. This utility is based on tcpdump. Some of its features include: Unde

Feb 26, 2022
HTTP API traffic recording and replay middleware based on GoReplay, can be used for migration and refactoring testing

gorc HTTP API traffic recording and replay middleware based on GoReplay, can be used for migration and refactoring testing. English | δΈ­ζ–‡ Requirements

Feb 13, 2022
Network cli based on urfave/cli package

go_network_cli network cli based on urfave/cli package available on Working for ip, cname and mx. Use --hel

Nov 28, 2021
Swiss Army knife Proxy tool for HTTP/HTTPS traffic capture, manipulation, and replay on the go.
Swiss Army knife Proxy tool for HTTP/HTTPS traffic capture, manipulation, and replay on the go.

Features β€’ Installation β€’ Usage β€’ Running Proxify β€’ Installing SSL Certificate β€’ Applications of Proxify β€’ Join Discord Swiss Army Knife Proxy for rap

Jan 8, 2023
A Golang program that receives DNSTAP traffic and relays it to multiple other listeners.

socket-proxy socket-proxy is a Golang program that is used to proxy dnstap messages from one socket to multiple other sockets. Overview Name Servers t

Jan 10, 2022
Package socket provides a low-level network connection type which integrates with Go's runtime network poller to provide asynchronous I/O and deadline support. MIT Licensed.

socket Package socket provides a low-level network connection type which integrates with Go's runtime network poller to provide asynchronous I/O and d

Dec 14, 2022
Magma is an open-source software platform that gives network operators an open, flexible and extendable mobile core network solution.
Magma is an open-source software platform that gives network operators an open, flexible and extendable mobile core network solution.

Connecting the Next Billion People Magma is an open-source software platform that gives network operators an open, flexible and extendable mobile core

Dec 31, 2022
Zero Trust Network Communication Sentinel provides peer-to-peer, multi-protocol, automatic networking, cross-CDN and other features for network communication.
Zero Trust Network Communication Sentinel provides peer-to-peer, multi-protocol, automatic networking, cross-CDN and other features for network communication.

Thank you for your interest in ZASentinel ZASentinel helps organizations improve information security by providing a better and simpler way to protect

Nov 1, 2022
Transfer 10Gbps http traffic over 1Gbps networks :)

httpteleport Teleports 10Gbps http traffic over 1Gbps networks. Built on top of fastrpc. Use cases httpteleport may significantly reduce inter-server

Nov 30, 2022
Reducing Malloc/Free traffic to cgo

CGOAlloc Reducing Malloc/Free traffic to cgo Why? Cgo overhead is a little higher than many are comfortable with (at the time of this writing, a simpl

Dec 24, 2022
Serve traffic (HTTP/gRPC) over SSH using Domain Sockets

Serve On SSH Introduction There is often a need to offer services for administrative purposes on servers or even for microservices that are running on

Nov 10, 2022
dumpr! is a tool to capture text based tcp traffic from the receivers point of view.
dumpr! is a tool to capture text based tcp traffic from the receivers point of view.

dumpr! dumpr! is a tool to capture text based tcp traffic. The project came about for the need to capture a web request from the back end. It was also

Dec 4, 2021
A minimal analytics package to start collecting traffic data without client dependencies.

go-web-analytics A minimal analytics package to start collecting traffic data without client dependencies. Logging incoming requests import "github.co

Nov 23, 2021