For newcomers to BPF tracing, bpftrace is the place to start. It lets you create new tools using several lines of concise code; see the example from this article:

https://github.com/devopsspiral/ebpf-bt-unlock/blob/master/b...

As BTF (BPF Type Format) becomes the default in kernels, this short tool will become even shorter: the "( struct hci_dev *) arg0" cast won't be necessary as the kernel will already know what type arg0 is. So line 9 can be skipped.

I see a lot of people start with other BPF tracers that made sense years ago as we hadn't built bpftrace yet. It'll really save you a lot of time to look at bpftrace first and then others only if need be. bpftrace is installed by default on all cloud instances at Netflix and other companies: it's the go-to tool for whipping up custom tracing programs.

Quick way to implement eBPF is via Falco Project. SDK support for higher level languages.

Cool to see this spinoff from SysDig evolve into a fully funded cloud native product ;)

https://falco.org/docs/

The article did instrumentation such as this:

  #!/snap/bin/bpftrace
  
  #include <net/bluetooth/bluetooth.h>
  #include <net/bluetooth/hci_core.h>
  
  kprobe:mgmt_device_found
  {
  $dev=( struct hci_dev *) arg0;
  printf("%s\n", $dev->name);
  }

AFAIK that's way beyond Falco's capabilities.

Disclaimer: I'm one of the organizers of eBPF summit

If you are interested in eBPF, consider attending the eBPF summit [0] on Oct 28-29. Community driven, free and virtual. I've spotted half a dozen "getting started with eBPF" type talks in the proposals already and we will select several of them.

Besides that, both eBPF maintainers, Brendan Gregg, as well as Cilium & Falco developers will speak. Definitely worth considering if you want to learn more about eBPF.

[0] https://ebpf.io/ebpf-summit-2020-cfp/

The article unfortunately does not say what eBPF stand for.

Edit: it is the Berkeley Packet Filter.

More valuable is actually describing what it does/what it's for. And the article does this:

""" eBPF is a functionality of linux kernel that allows lightweight execution of user code as a response to kernel events. The events could be hardware/software events, tracing events both static (compiled into code) and dynamic (attached in runtime), etc. The code itself is limited in a sense that it is guaranteed to finish (no loops) and is verified before loading into kernel. """

Arguably this is a much more useful description for people who don't know what eBPF is, since "packet filter" makes it sound like it's exclusively for firewalling and/or routing.

Awesome! I finally understood what the fuss about eBPF is all about :p

Thanks for pointing this out!

If I remember correctly, eBPF originally stood for "Extended Berkeley Packet Filter". However, it's usage changed from a packet filter to general purpose kernel hook system.

A little while after that, eBPF stopped being an acronym and started being the actual name. It, as I understand it, no longer stands for "Extended Berkeley Packet Filter".

In that case I wish they would make a new backronym or change the name. Having acronyms that used to stand for something but now stand for nothing feels like (somewhat common) antipattern.

In modern parlance I believe the desire is to move away from the acronym and have BPF just be a standalone name since its functionality goes beyond a packet filter now.

That makes sense. Thank You.

I thought they had something against Berkeley / BSD.

When my eyes first alit upon it, I thought it was something to do with extended backus-naur form[1].

1 -https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_f...

Ironically, if you spell it out it only gets more confusing, because it does a lot more than just packer filtering these days.

Originally BPF aka the Berkeley packet filter was written to provide a faster/more lightweight alternative to tcpdump. People realized it was an awesome idea, so they started adding more functionality and it became eBPF/extended berkeley packet filter.

I see snap is used over apt to install bpftrace and friends. Any reason to not use apt here?

I’d think for something kernel-related you want to use tools provided by the same package-tree as the kernel itself.

I haven't tried recently, but bpftrace is still changing pretty quickly, and the versions you can get via apt-get are far behind enough to break bpftrace programs due to API differences.

I guess it’s easier to distribuiere up to date versions that way.

While this is a neat idea, isn’t it insecure against just sniffing bt mac addresses and masquerading as the correct one?

Yes and they say as much in the post:

>Just remember that the script showed in this article was created for educational purposes (just for fun) and it shouldn’t be used in places where you are exposed to anybody that could reach your keyboard. This is mainly because of the fact that your Bluetooth MAC becomes effectively password to your system, and despite of techniques that hides Bluetooth MAC it should be treated as public information. It works ok when using it at home, during pandemic, when the only hackers nearby are 2 and 6 years old and mostly using brute-force keyboard attacks, not Bluetooth MAC spoofing.

It seems like a very cool thing, that's mostly inaccessible, due to being so archaic, to non kernel programmers.

Time to write JSeBPF, the new web API allowing kernel-side javascript...

You joke, but BPF is frequently likened to JavaScript: it's a small, limited language that runs relatively safely in a sandbox in order to extend the behaviour of existing functionality.

Brendan Gregg (the guy that gave that talk that the article linked to) wrote an extremely awesome book called systems performance. It is the best and most pragmatic operating system book I've read and is definitely worth a read. It will give you a better idea about how all of this eBPF stuff applies.

I work with some of the maintainers of bpf, bpftrace, etc. What can we do to make this stuff more accessible?

If you want people to use it at the kernel level (with libbpf and C programming), better documentation. XDP has a fantastic tutorial[0], but everything else can be quite difficult to get into (this is from personal experience). I've been toying with the idea of starting my own version of the XDP tutorial for other BPF functions, but I don't want to start without support from the larger BPF community. They've been somewhat supportive but seemingly not the most willing to help (which they're under no obligation to do so and I don't think they should be).

For BCC and bpftrace I personally think the documentation is actually quite good.

[0] https://github.com/xdp-project/xdp-tutorial

Honestly it's not.

What it is, is badly documented. If you can't work out how to use it from scratch you probably don't know what you want to use it for, if that makes sense.

The actual VM makes quite a lot of sense, you just have to read a lot of kernel samples and txt documentation to work out how to do it properly.