Catch Performance in eBPF with Rust: XDP Programs 

This is the second in a five-part series. Read Part 1 here.

In this series we learned what eBPF is, the tools to work with it, why eBPF performance is important and how to track it with continuous benchmarking. In this entry of the series, we will discuss how to create a basic eBPF XDP program in Rust using Aya. All of the source code for the project is open source and is available on GitHub.

eBPF XDP programs allow for very efficient, custom packet handling. The eBPF XDP program is run before it reaches the kernel’s network stack. There are four different actions an eBPF XDP program can perform:

• XDP_PASS: Pass the packet to the normal network stack for processing. The packet contents can be modified.
• XDP_DROP: Drop the packet and do not process it. This is the fastest action.
• XDP_TX: Forward the packet to the same network interface it arrived on. The packet contents can be modified.
• XDP_ABORTED: Error when processing, so drop the packet and do not process it. This indicates an error in the eBPF program.

In our basic example, if everything goes well, we will only perform the first action, XDP_PASS, as we will be focusing more on the scaffolding and interprocess communication than the packet handling logic. The initial version of our eBPF XDP application will simply log the IPv4 source address for each packet that is received.

Let’s first look at the kernel side of things:

1. #[xdp(name = fun_xdp)]
2. pub fn fun_xdp(ctx: XdpContext) -> u32 {
3. match try_fun_xdp(&ctx) {
4. Ok(ret) => ret,
5. Err(_) => xdp_action::XDP_ABORTED,

Going line by line:

1. An Aya macro that tells us we are creating an eBPF XDP program named fun_xdp.
2. The function definition for our eBPF XDP program. This takes in the context as its only argument. The context tells us all the information that the kernel provides to us and returns an unsigned 32-bit integer.
3. There is a helper function called try_fun_xdp that we will discuss next. Based on what it returns, if it is Ok then everything is good and we return the value given. Otherwise, if we get an Err, we abort.

The Linux Foundation is the world’s leading home for collaboration on open source software, hardware, standards, and data. Linux Foundation projects are critical to the world’s infrastructure including Linux, Kubernetes, Node.js, ONAP, Hyperledger Foundation, PyTorch, RISC-V, and more.

Learn More

The latest from Linux Foundation Training

Now let’s look at that try_fun_xdp function:

1. 1. 1. fn try_fun_xdp(ctx: &XdpContext) -> Result<u32, ()> {
      2. let eth_hdr: *const EthHdr = unsafe { ptr_at(ctx, 0)? };
      3. unsafe {
      4. let EtherType::Ipv4 = (*eth_hdr).ether_type else {
      5. return Ok(xdp_action::XDP_PASS);
      6. };
      7. }
      8. let ipv4_hdr: *const Ipv4Hdr = unsafe { ptr_at(ctx, EthHdr::LEN)? };
      9. let source_addr = unsafe { (*ipv4_hdr).src_addr };
      10. info!(ctx, “IPv4 Source Address: {}”, source_addr);
      11. Ok(xdp_action::XDP_PASS)
      12. }
      13. }
      14. }

   Going line by line:

   1. The try_fun_xdp function takes in a reference to the context and returns a Result containing either an Ok unsigned 32-bit integer value or an empty Err.
   2. From the context, we get the ethernet header. Notice the unsafe ptr_at helper function here. We will look at that next.
   3. The following operations are also considered unsafe by the Rust compiler, so we have to explicitly opt in.
   4. We only care about IPv4 for our basic example, so for anything else we just go ahead and pass the packet along.
   5. –
   6. –
   7. –
   8. Extract the IPv4 header. Again using that unsafe ptr_at helper function.
   9. Get the source address from the IPv4 header.
   10. Log the IPv4 source address
   11. –
   12. Return pass!
   13. –
   14. –

   Finally let’s look at that ptr_at helper function:

   1. #[inline(always)]
   2. unsafe fn ptr_at<T>(ctx: &XdpContext, offset: usize) -> Result<*const T, ()> {
   3. let start = ctx.data();
   4. let end = ctx.data_end();
   5. let len = core::mem::size_of::<T>();
   6. if start + offset + len > end {
   7. return Err(());
   8. }
   9. Ok((start + offset) as _)
   10. }

   Going line by line:

   1. Because this operation is going to be performed a lot and within the critical path, we use a macro to ask the compiler to always inline our helper function.
   2. This is an unsafe function that reads a generic type, T, from the memory given by the context at a specific byte offset. For a successful read, the Result is an Ok pointer to a T. Otherwise, an empty Err is returned.
   3. The address of the start of the memory given by context.
   4. The address of the end of the memory given by the context.
   5. The size in bytes of the generic type T.
   6. If the sum of the start address, byte offset and length of T in bytes is greater than the end address, then return an empty Err, as we have exceeded the bounds of the context. If we didn’t perform this check, the eBPF verifier would get upset and likely fail our build.
   7. –
   8. –
   9. –
   10. –
   11. Read T from the memory given by the context at a specific byte offset
   12. –

   There is also one last function:

   1. #[panic_handler]
   2. fn panic(_info: &core::panic::PanicInfo) -> ! {
   3. unsafe { core::hint::unreachable_unchecked() }
   4. }

   Going line by line:

   1. Define a custom Rust panic handler.
   2. This function takes in the Rust panic info, which it never uses. This function should never return.
   3. Give a hint to the Rust compiler that this code path should be unreachable. That is, we are never expecting to panic. This is required to keep the eBPF verifier happy.
   4. –

   Now over to the userspace side of things. Let’s look at our main function:

   1. #[tokio::main]
   2. async fn main() -> Result<(), anyhow::Error> {
   3. let opt = Opt::parse();
   4. env_logger::init();
   5. let mut bpf = Bpf::load(include_bytes_aligned!(“../path/to/ebpf-bin”))?;
   6. BpfLogger::init(&mut bpf)?;
   7. let program: &mut Xdp = bpf.program_mut(fun_xdp).unwrap().try_into()?;
   8. program.load()?;
   9. program.attach(&opt.iface, XdpFlags::default())?;
   11. info!(“Waiting for Ctrl-C…”);
   12. signal::ctrl_c().await?;
   13. info!(“Exiting…”);
   14. Ok(())
   15. }
   16. #[derive(clap::Parser)]
   17. struct Opt {
   18. #[clap(long)]
   19. iface: String,
   20. }

   Going line by line:

   1. This macro creates an asynchronous runtime for our program using tokio.
   2. An asynchronous main function. In a Rust binary, main is the de facto entry point. The Result of this function is an empty Ok or a catch-all Err using the anyhow crate.
   3. Parse the command line arguments passed to our binary.
   4. Initialize logging for userspace.
   5. Load our compiled eBPF bytecode. Aya makes recompiling our eBPF source code into bytecode easy, so it automatically happens before our userspace code is compiled.
   6. Initialize logging from our eBPF program.
   7. From our eBPF bytecode, get our fun_xdp eBPF XDP program.
   8. Load the fun_xdp eBPF XDP program into the kernel using the default flags.
   9. Attach our fun_xdp eBPF XDP program to a network interface that was set by the iface command line argument to our binary.
   10. –
   11. Log how to exit our program.
   12. Wait for the user to enter Ctrl + C.
   13. Log that our program is exiting.
   14. Return an empty Ok as our Result.
   15. –
   16. –
   17. This macro uses clap to parse the command line arguments defined in the Opt struct.
   18. The command line argument struct named Opt.
   19. Another macro that tells clap that this field should be parsed as a long argument name, i.e., --iface.
   20. The name of argument is iface, and its value is a string.

   With that, we have created a very basic eBPF program. Again, all of the source code for the project is open source and is available on GitHub.

Everett Pompeii is a software engineer with a focus on real-time network systems. As a recovering C++ developer, he now enjoys working primarily in Rust. He is a contributor and maintainer for several open source projects, the most recent of...

Read more from Everett Pompeii