This manager helps handle the life cycle of your eBPF programs

eBPF Manager

License

This repository implements a manager on top of Cilium's eBPF library. This declarative manager simplifies attaching and detaching eBPF programs by controlling their entire life cycle. It was built with the intention of unifying how eBPF is used in large scale projects such as the Datadog Agent. By using the same declarative conventions, multiple teams can quickly collaborate on complex eBPF programs by sharing maps, programs or even hook points without having to worry about the setup of complex program types.

Requirements

Getting started

You can find many examples using the manager in examples/. For a real world use case, check out the Datadog Agent.

Useful resources

License

  • Unless explicitly specified otherwise, the golang code in this repository is under the MIT License.
  • The eBPF programs are under the GPL v2 License.
Comments
  • hook sys_execve fail

    hook sys_execve fail

    hi, i want to use ebpf-manage hook kprobe: sys_execve. (my machine is ubuntu21.04 x86).

    when i use this: image

    error info: {UID:KretprobeSysExecve EBPFSection:kretprobe/sys_execve EBPFFuncName:kretprobe_sys_execve}: symbol 'sys_execve' not found: invalid argument

    when i use this: image

    error_info: couldn't find program spec for func kretprobe___x64_sys_execve: unknown section or eBPF function name

    i confuse , how to correct use it. i hope someone can help me solve this problem. or tell me correct usage.

    thanks!

  • [Manager] Add KeepKernelBTF option

    [Manager] Add KeepKernelBTF option

    What does this PR do?

    This PR adds an option to keep the Kernel types so that CloneProgram can still work. This PR doesn't change the current default state of the manager.

    Motivation

    Fix CloneProgram usage with BTF.

  • Compare with cilium/ebpf

    Compare with cilium/ebpf

    I have a question on how it compare to cilum/ebpf. cilum/ebpf converts a bpf C file into Go file inside a docker container, and the container is independent with the target Linux host kernel. It doesn't require the Clang and kernel headers be installed on the dev machine. DataDog/ebpf-manager is based on cilum/ebpf but requires the Clang and kernel headers be installed on the dev machine. I'm confused what's the value this project? Correct me if I'm wrong. Thanks!

  • 【BUG】i can‘t get updated code

    【BUG】i can‘t get updated code

    hi,i am a use of ebpf-manager. i run 'go get github.com/DataDog/ebpf-manager', and found it's version is: v1.0.3. this version hav't ringbuffer.go.

    maybe, can set a updated tag, and sure we can get new version code anywhere.

    it's so mess, can someone help me?

  • selectors: Export RunValidator

    selectors: Export RunValidator

    What does this PR do?

    Exporting RunValidator method of the selectors.

    Motivation

    Adding uprobes in datadog-agent repository, and some of the uprobes might not exists (depending on the version of openssl). I want to use "BestEffort" selector on the "might be missing" uprobes, and "AllOf" selector on the rest. I want to validate the hooks are properly set, to do so I need to use RunValidator. Since those are uprobes, and I'm hooking openssl libraries, it must dynamic per process and during runtime (instead of setting the hooks during load time.

    Describe how to test your changes

    Write here in detail how you have tested your changes and instructions on how this should be tested.

  • Use ReadInto for RecordHandler

    Use ReadInto for RecordHandler

    What does this PR do?

    Use ReadInto to reduce allocations from reading from the perf buffer.

    Motivation

    Alternative to #47 which minimizes API change and pushes allocations to library user.

  • make use of readinto

    make use of readinto

    What does this PR do?

    A brief description of the change being made with this pull request.

    Motivation

    What inspired you to submit this pull request?

    Additional Notes

    Anything else we should know when reviewing?

    Describe how to test your changes

    Write here in detail how you have tested your changes and instructions on how this should be tested.

  • Fix unbounded retry and slowdowns

    Fix unbounded retry and slowdowns

    What does this PR do?

    A brief description of the change being made with this pull request.

    Motivation

    What inspired you to submit this pull request?

    Additional Notes

    Anything else we should know when reviewing?

    Describe how to test your changes

    Write here in detail how you have tested your changes and instructions on how this should be tested.

  • [manager] Add `DetachAndDeleteHook` method to the manager

    [manager] Add `DetachAndDeleteHook` method to the manager

    What does this PR do?

    This PR introduces a new method to the manager: DetachAndDeleteHook(id ProbeIdentificationPair). The goal of this method is to provide a way to override the deletion decision of DetachHook(id ProbeIdentificationPair) without breaking the existing API.

    Motivation

    Providing the ability to stop and remove a probe from the manager is useful in situation where you want to get rid of a Probe that you no longer need and won't need in the future.

  • Fix tests-and-benchmarks example

    Fix tests-and-benchmarks example

    What does this PR do?

    The program does 2 * x, so it should use an input of 64 if it expects 128.

    Motivation

    failing output:

    INFO[0000] Running tests ...
    INFO[0000] { Input:2 Output:4 } - PASS
    INFO[0000] { Input:10 Output:20 } - PASS
    INFO[0000] { Input:42 Output:128 } - FAIL (checkout /sys/kernel/debug/tracing/trace_pipe to see the logs)
    INFO[0000] { Input:42 Output:84 } - PASS
    INFO[0000] Running benchmark ...
    INFO[0000] { Input:2 Output:4 } - PASS (duration: 7ns)
    INFO[0000] { Input:10 Output:20 } - PASS (duration: 7ns)
    INFO[0000] { Input:42 Output:128 } - benchmark FAILED (checkout /sys/kernel/debug/tracing/trace_pipe to see the logs)
    INFO[0000] { Input:42 Output:84 } - PASS (duration: 7ns)
    
  • [examples] Remove go-bindata

    [examples] Remove go-bindata

    What does this PR do?

    This PR removes all usage of gobindata and uses the "embed" package added in go 1.16.

    Motivation

    Remove an unnecessary dependency.

    Describe how to test your changes

    Run all the examples and make sure they all work.

  • Using bugfixed cilium/ebpf commit to show full verifier output

    Using bugfixed cilium/ebpf commit to show full verifier output

    What does this PR do?

    The current cilium/ebpf release shows truncated verifier errors. A recently merged branch fixes this problem, and this commit uses that bugfix: https://github.com/cilium/ebpf/pull/851

    Motivation

    What inspired you to submit this pull request?

    Additional Notes

    Anything else we should know when reviewing?

    Describe how to test your changes

    Write here in detail how you have tested your changes and instructions on how this should be tested.

  • use ringbuf fail

    use ringbuf fail

    hi, i try to use ringbuf. but it load fail.

    err: map exec_events: map create without BTF: invalid argument

    my env is:

    os: ubuntu21.04 kern: linux 5.11 x86

    here is my code:

    So, someone can help me? maybe my usages is error. e.g: some params is correctly.

    tell me correct usages, if possible. thanks!

  • failed to run examples

    failed to run examples

    I tried several examples, none runs.

    eoi@ck16:/data01/zhichyu/ebpf-manager/examples/tests_and_benchmarks$ uname -a
    Linux ck16 3.10.0-1160.66.1.el7.x86_64 #1 SMP Wed May 18 16:02:34 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
    eoi@ck16:/data01/zhichyu/ebpf-manager/examples/tests_and_benchmarks$ clang --version
    clang version 14.0.5 (https://github.com/llvm/llvm-project.git c12386ae247c0d46e1d513942e322e3a0510b126)
    Target: x86_64-unknown-linux-gnu
    Thread model: posix
    InstalledDir: /usr/local/bin
    eoi@ck16:/data01/zhichyu/ebpf-manager/examples/tests_and_benchmarks$ make
    mkdir -p ebpf/bin
    clang -D__KERNEL__ -D__ASM_SYSREG_H \
            -Wno-unused-value \
            -Wno-pointer-sign \
            -Wno-compare-distinct-pointer-types \
            -Wunused \
            -Wall \
            -Werror \
            -I/lib/modules/$(uname -r)/build/include \
            -I/lib/modules/$(uname -r)/build/include/uapi \
            -I/lib/modules/$(uname -r)/build/include/generated/uapi \
            -I/lib/modules/$(uname -r)/build/arch/x86/include \
            -I/lib/modules/$(uname -r)/build/arch/x86/include/uapi \
            -I/lib/modules/$(uname -r)/build/arch/x86/include/generated \
            -O2 -emit-llvm \
            ebpf/main.c \
            -c -o - | llc -march=bpf -filetype=obj -o ebpf/bin/probe.o
    go-bindata -pkg main -prefix "ebpf/bin" -o "probe.go" "ebpf/bin/probe.o"
    go build -o bin/main .
    sudo bin/main
    FATA[0000] couldn't find asset: Asset /probe.o not found 
    make: *** [run] Error 1
    
    
  • [ebpf/manager] Resolve ELF symbol from memory (/proc/pid/maps) if the library file is not accessible/available

    [ebpf/manager] Resolve ELF symbol from memory (/proc/pid/maps) if the library file is not accessible/available

    What does this PR do?

    Resolve ELF symbol from memory (/proc/pid/maps) if the library file is not accessible/available

    Motivation

    FindSymbolOffsets() will try to extract symbols from the library ELF file. If it can't fallback to parse the ELF available from memory (proc/pid/maps) But the original ELF sections could not be avaiable (not mapped in memory) so the last chance is to get the table file offset (DYNTAB, SYMTAB) from the prog ELF section DT_DYNAMIC and reconstruct ELF section to extract dynamic symbols.

    OpenAndListSymbols(path) : read the ELF thanks to debug/elf

    OpenAndListSymbolsFromPID(pid, path) : read the ELF thanks to elfmap

    If the .so is not accessible, elfmap would ptrace() PEEK_DATA and copy the mapped lib. If the caller lookup to himself, so the same memspace, we just memcpy the sections.

  • [manager] verify program type for tail_call map

    [manager] verify program type for tail_call map

    What does this PR do?

    Tail call map must have an unique program type per program array (post check)

    Motivation

    Inserting multiple program (fd) type in a prog array is possible, but will not work

Hubble - Network, Service & Security Observability for Kubernetes using eBPF
Hubble - Network, Service & Security Observability for Kubernetes using eBPF

Network, Service & Security Observability for Kubernetes What is Hubble? Getting Started Features Service Dependency Graph Metrics & Monitoring Flow V

Jan 2, 2023
A K8s ClusterIP HTTP monitoring library based on eBPF

Owlk8s Seamless RED monitoring of k8s ClusterIP HTTP services. This library provides RED (rate,error,duration) monitoring for all(by default but exclu

Jun 16, 2022
Power-ups for the daily DevOps life

DevOps Loop Power-Ups Requirements Connected Kubernetes cluster. Some features need support for LoadBalancer services Permission to list, create and d

Nov 3, 2022
Metrics collector and ebpf-based profiler for C, C++, Golang, and Rust

Apache SkyWalking Rover SkyWalking Rover: Metrics collector and ebpf-based profiler for C, C++, Golang, and Rust. Documentation Official documentation

Jan 6, 2023
Kepler (Kubernetes-based Efficient Power Level Exporter) uses eBPF to probe energy related system stats and exports as Prometheus metrics
Kepler (Kubernetes-based Efficient Power Level Exporter) uses eBPF to probe energy related system stats and exports as Prometheus metrics

kepler Kepler (Kubernetes Efficient Power Level Exporter) uses eBPF to probe energy related system stats and exports as Prometheus metrics Architectur

Dec 26, 2022
Provide task runtime implementation with pidfd and eBPF sched_process_exit tracepoint to manage deamonless container with low overhead.

embedshim The embedshim is the kind of task runtime implementation, which can be used as plugin in containerd. With current shim design, it is used to

Dec 18, 2022
Terraform Controller manages the life cycles of a terraform resource, allowing developers to self-serve dependencies in a controlled manner.
Terraform Controller manages the life cycles of a terraform resource, allowing developers to self-serve dependencies in a controlled manner.

TERRAFORM CONTROLLER Terraform Controller manages the life cycles of a terraform resource, allowing developers to self-serve dependencies in a control

Dec 15, 2022
Terraform Provider for Azure (Resource Manager)Terraform Provider for Azure (Resource Manager)
Terraform Provider for Azure (Resource Manager)Terraform Provider for Azure (Resource Manager)

Terraform Provider for Azure (Resource Manager) Version 2.x of the AzureRM Provider requires Terraform 0.12.x and later, but 1.0 is recommended. Terra

Oct 16, 2021
This plugin helps you to use the AWS Command Line Interface (AWS CLI) to start and end sessions to your managed instances

Session Manager Plugin This plugin helps you to use the AWS Command Line Interface (AWS CLI) to start and end sessions to your managed instances. Sess

Dec 28, 2022
A simple download file manager that sorts your files into your desired folders, This was meant to be a small project and nothing big.

GoDFM Simply go to the tags and download the .exe file (or compile it yourself by running go build). Add it to your environment paths by going to sett

Aug 9, 2022
dockin ops is a project used to handle the exec request for kubernetes under supervision
dockin ops is a project used to handle the exec request for kubernetes under supervision

Dockin Ops - Dockin Operation service English | 中文 Dockin operation and maintenance management system is a safe operation and maintenance management s

Aug 12, 2022
A library for writing backup programs in Golang

Barkup godoc.org/github.com/keighl/barkup Barkup is a library for backing things up. It provides tools for writing bare-bones backup programs in Go. T

Nov 13, 2022
Collection of mini-programs demonstrating Kubernetes client-go usage.

Kubernetes client-go examples Collection of mini-programs covering various client-go use cases. The intention (at least so far) is to test (more or le

Jan 3, 2023
Simple tuning work for go programs in high concurrency scenarios.

go-program-tuning Simple tuning work for go programs in high concurrency scenarios. Installation Run the following command under your project: go get

Mar 15, 2022
Litmus helps Kubernetes SREs and developers practice chaos engineering in a Kubernetes native way.
Litmus helps Kubernetes SREs and developers practice chaos engineering in a Kubernetes native way.

Litmus Cloud-Native Chaos Engineering Read this in other languages. ???? ???? ???? ???? Overview Litmus is a toolset to do cloud-native chaos engineer

Jan 1, 2023
preflight helps you verify scripts and executables to mitigate chain of supply attacks such as the recent Codecov hack.
preflight helps you verify scripts and executables to mitigate chain of supply attacks such as the recent Codecov hack.

?? Mitigate chain of supply attacks ?? Verify your curl scripts and executables ?? What is it? preflight helps you verify scripts and executables to m

Nov 18, 2022
ControllerMesh is a solution that helps developers manage their controllers/operators better.
ControllerMesh is a solution that helps developers manage their controllers/operators better.

ControllerMesh ControllerMesh is a solution that helps developers manage their controllers/operators better. Key Features Canary update: the controlle

Jan 6, 2023
The GCP Enterprise Cloud Cost Optimiser, or gecco for short, helps teams optimise their cloud project costs.
The GCP Enterprise Cloud Cost Optimiser, or gecco for short, helps teams optimise their cloud project costs.

gecco helps teams optimise their cloud resource costs. Locate abandoned, idle, and inefficiently configured resources quickly. gecco helps teams build

Jan 9, 2022
A Kubernetes Operator, that helps DevOps team accelerate their journey into the cloud and K8s.
A Kubernetes Operator, that helps DevOps team accelerate their journey into the cloud and K8s.

A Kubernetes Operator, that helps DevOps team accelerate their journey into the cloud and K8s. OAM operator scaffolds all of the code required to create resources across various cloud provides, which includes both K8s and Non-K8s resources

Nov 30, 2021