Catch Performance Regressions: Evolving eBPF Program 

This is the third in a five-part series. Read Part 1 and Part 2. 

In this series we learned what eBPF is, the tools to work with it, why eBPF performance is so important, and how to track it with continuous benchmarking. We created a basic eBPF XDP program, line by line in Rust using Aya. In this next entry we will discuss how to evolve this basic eBPF XDP program to new feature requirements. All of the source code for the project is open source and is available on GitHub.

Programs in eBPF are in and of themselves completely stateless. Every invocation is a fresh eBPF program. To keep state, report back on its operations or change its behavior, an eBPF program needs to use “maps”. Maps are persistent data structures available to both eBPF and userspace programs. There are several different kinds of eBPF maps: arrays, hash maps, stacks, queues and many more. They are the only way to reliably communicate between an eBPF program and userspace and vice versa. In the next iteration of our eBPF XDP program, we will use maps to communicate from our eBPF program back to our userspace program.

In the next version of our application, Version 1, we will implement a “Fizz Feature.” This Fizz Feature requires:

• Push “Fizz” into the queue if the IPv4 source address is divisible by 3.
• Otherwise, just return XDP_PASS.

To implement this Fizz Feature, we need to create the message that will be passed inside of the queue map:

1. #[repr(C)]
2. #[derive(Clone, Copy)]
3. #[cfg_attr(feature = “user”, derive(Debug))]
4. pub enum SourceAddr {
5. Fizz,
6. }
7. #[cfg(feature = “user”)]
8. unsafe impl aya::Pod for SourceAddr {}

Going line by line:

1. This macro tells the Rust compiler to represent this data just like a C program would.
2. The derive macro automatically implements the Clone (the ability to duplicate our message) and Copy (the ability to duplicate our message by simply copying bits).
3. This third macro is a bit more complex. It is conditional on whether the “user” feature is enabled. The “user” feature is only used by the userspace side of things, and it allows us to display our message type as Debug output.
4. Our message type is an enum named SourceAddr.
5. Fizz is the one and only message variant we currently have.
6. –
7. –
8. Again, we have a conditional macro. The line below is only used when the “user” feature is enabled, by userspace.
9. Implement a trait required by Aya that marks SourceAddr as being eBPF map friendly.

The Linux Foundation is the world’s leading home for collaboration on open source software, hardware, standards, and data. Linux Foundation projects are critical to the world’s infrastructure including Linux, Kubernetes, Node.js, ONAP, Hyperledger Foundation, PyTorch, RISC-V, and more.

Learn More

The latest from Linux Foundation Training

With our message type created, we can now make changes to the eBPF side of things:

1. #[map]
2. pub static mut SOURCE_ADDR_QUEUE: Queue<SourceAddr> = Queue::with_max_entries(1024, 0);
3. fn try_fun_xdp(ctx: &XdpContext) -> Result<u32, ()> {
4. …
5.  info!(ctx, “IPv4 Source Address: {}”, source_addr);
6. let opt_source_addr = (source_addr % 3 == 0).then_some(SourceAddr::Fizz);
7. if let Some(source_addr) = opt_source_addr {
8. unsafe {
9. if let Err(e) = SOURCE_ADDR_QUEUE.push(&source_addr, 0) {
10. error!(ctx, “Failed to push source address into queue: {}”, e);
11. }
12. }
13. }
14. Ok(xdp_action::XDP_PASS)
15. }

Going line by line:

1. This Aya macro creates an eBPF map from the line below.
2. The eBPF map for our SourceAddr messages is going to be a queue named SOURCE_ADDR_QUEUE with a maximum of 1,024 entries.
3. –
4. We are going to update our try_fun_xdp helper function that we [created in the previous part of the series](/2).
5. –
6. Remove the line that simply logs the IPv4 source address.
7. If the IPv4 source address is divisible by 3, then store Some Fizz message. Otherwise, store None.
8. –
9. If the source address message is Some then…
10. The following operations are considered unsafe, so we must explicitly opt in.
11. Try to push our Fizz message onto our SOURCE_ADDR_QUEUE. If this returns an error then…
12. Log the error along with the eBPF context.
13. –
14. –
15. –
16. –
17. Just as before, return pass!
18. –

Now to receive that message on the userspace side:

1. #[tokio::main]
2. async fn main() -> Result<(), anyhow::Error> {
3. …
4. spawn_agent(&mut bpf).await?;
5. info!(“Waiting for Ctrl-C…”);
6. signal::ctrl_c().await?;
7. info!(“Exiting…”);
8. Ok(())
9. }

The main function is going to stay mostly the same. Except we are going to add a spawn_agent helper function at Line 5. We will be evolving spawn_agent throughout the rest of this series.

1. async fn spawn_agent(bpf: &mut Bpf) -> Result<(), anyhow::Error> {
2. let mut xdp_map: Queue<_, SourceAddr> =
3. Queue::try_from(bpf.map_mut(“SOURCE_ADDR_QUEUE”)?)?;
4. loop {
5. while let Ok(source_addr) = xdp_map.pop(0) {
6. info!(“{:?}”, source_addr);
7. }
8. }
9. }

Going line by line:

1. The spawn_agent function is asynchronous, which is why it had to be .awaited in main. It takes in a mutable reference to our eBPF program; it returns a Result that is either an empty Ok or that same flexible Err used in main.
2. Create a userspace side of the SOURCE_ADDR_QUEUE eBPF map.
3. –
4. –
5. loop over the source address queue.
6. Attempt to pop off of the source address queue. If successful…
7. Log the source address message, ie Fizz.

With that complete, we have now finished Version 1 of our application. Fizz Feature complete. Now let’s not rest on our laurels for too long. Let’s add a simple update. Maybe you can see where this is going…

The next version of our application, Version 2, we will implement a “FizzBuzz Feature.” This FizzBuzz Feature requires:

• Push “Fizz” into the queue if the IPv4 source address is divisible by 3.
• Push “Buzz” into the queue if divisible by 5.
• Push “FizzBuzz” into the queue if divisible by both 3 and 5.
• Otherwise, just return XDP_PASS.

Just as before, were going to start by modifying the shared message type, SourceAddr:

1. #[repr(C)]
2. #[derive(Clone, Copy)]
3. #[cfg_attr(feature = “user”, derive(Debug))]
4. pub enum SourceAddr {
5. Fizz,
6. Buzz,
7. FizzBuzz,
8. }
9. #[cfg(feature = “user”)]
10. unsafe impl aya::Pod for SourceAddr {}

The only changes here are on line 6 and 7, where we added Buzz and FizzBuzz as potential SourceAddr messages.

Next, we will update the eBPF XDP program. It will also stay very similar, except we’re going to implement FizzBuzz.

1. fn try_fun_xdp(ctx: &XdpContext) -> Result<u32, ()> {
2. …
3. let opt_source_addr = (source_addr % 3 == 0).then_some(SourceAddr::Fizz);
4. let opt_source_addr = match (source_addr % 3, source_addr % 5) {
5. (0, 0) => Some(SourceAddr::FizzBuzz),
6. (0, _) => Some(SourceAddr::Fizz),
7. (_, 0) => Some(SourceAddr::Buzz),
8. _ => None,
9. };
10. …
11. Ok(xdp_action::XDP_PASS)
12. }

The only changes here are removing line 3 and replacing it with lines 4 through 9. Here we see if the source address is divisible by both 3 and 5. If both, FizzBuzz. If 3, Fizz. If 5, Buzz. Otherwise, None.

There are no changes to make in the userspace program. It will just log whatever message, Fizz, Buzz or FizzBuzz that is sent over to it. We are done with Version 2. Now we’re cracking the coding interview! Though we’re not quite done yet. There is one last feature we would like to add.

The next version of our application, Version 3, we will implement a “FizzBuzzFibonacci Feature.” This FizzBuzzFibonacci Feature requires:

• Push “Fizz” into the queue if the IPv4 source address is divisible by 3.
• Push “Buzz” into the queue if divisible by 5.
• Push “FizzBuzz” into the queue if divisible by both 3 and 5.
• Except if the remainder of the IPv4 source address divided by 256 is part of the fibonacci sequence, then push “Fibonacci”
• Otherwise, just return XDP_PASS.

Just as before, were going to start by modifying the shared message type, SourceAddr:

1. #[repr(C)]
2. #[derive(Clone, Copy)]
3. #[cfg_attr(feature = “user”, derive(Debug))]
4. pub enum SourceAddr {
5. Fizz,
6. Buzz,
7. FizzBuzz,
8. Fibonacci,
9. }
10. #[cfg(feature = “user”)]
11. unsafe impl aya::Pod for SourceAddr {}

The only changes made here are on line 8, where we added Fibonacci as a potential SourceAddr message.

Next we will update the eBPF XDP program try_fun_xdp function:

1. fn try_fun_xdp(ctx: &XdpContext) -> Result<u32, ()> {
2. …
3.  let opt_source_addr = match (source_addr % 3, source_addr % 5) { … };
4. let opt_source_addr = is_fibonacci(source_addr as u8)
5. .then_some(SourceAddr::Fibonacci)
6. .or(match (source_addr % 3, source_addr % 5) {
7. (0, 0) => Some(SourceAddr::FizzBuzz),
8. (0, _) => Some(SourceAddr::Fizz),
9. (_, 0) => Some(SourceAddr::Buzz),
10. _ => None,
11. });
12. …
13. Ok(xdp_action::XDP_PASS)
14. }
15. fn is_fibonacci(n: u8) -> bool {
16. let (mut a, mut b) = (0, 1);
17. while b < n {
18. let c = a + b;
19. a = b;
20. b = c;
21. }
22. b == n
23. }

Instead of just doing FizzBuzz, we will implement our FizzBuzzFibonacci logic. This will entail calling a is_fibonacci helper function. If that returns true, then our message is Fibonacci. Otherwise, we do the same FizzBuzz logic as before. The is_fibonacci function calculates the Fibonacci sequence until it reaches or exceeds the argument passed in n. It then checks if those two values are equal, indicating that the argument is indeed part of the Fibonacci sequence.

Again there are no changes to make in the userspace program. It will just log whatever message, Fizz, Buzz, FizzBuzz or FizzBuzzFibonacci that is sent over to it. We are done with Version 3. Now nothing is ever going to go wrong due to this change. Everything will be 🌈🦄🌞, I promise… until we get to our next installment in the series where production is on fire 🔥🔥🔥, and you’re reconsidering all of your life choices that have led you to this moment.

Everett Pompeii is a software engineer with a focus on real-time network systems. As a recovering C++ developer, he now enjoys working primarily in Rust. He is a contributor and maintainer for several open source projects, the most recent of...

Read more from Everett Pompeii