News for hackers. Stuff that matters.
Re:eBPF on Windows by tptacek on Monday May 10, 2021 @04:18PM (#27110401)
A fun detail of this work is that it uses a formal-methods-based verifier (designed outside of Microsoft) that accepts a wider range of programs than does the Linux verifier, which is itself kind of nightmare fuel.

https://vbpf.github.io/assets/prevail-paper.pdf

The verifier in this paper also has some biting limitations; for instance, you can't resize a packet in it, because they don't account for pointer invalidation. I wonder whether they've since implemented these verifier features, since they'd be problematic for compatibility otherwise.

Additionally, the PREVAIL paper explicitly doesn't verify program termination, which is kind of a dealbreaker for kernel BPF.

Re:eBPF on Windows by dthaler on Monday May 10, 2021 @07:48PM (#27112411)
The ebpf-for-windows maintainers have been working with the PREVAIL verifier maintainers for some time now on improving the implementation for everyone, since we agree that addressing such limitations is critical. For example, the PREVAIL maintainers added program termination verification back in January 2021. Check it out at https://github.com/vbpf/ebpf-verifier
Re:eBPF on Windows by tptacek on Monday May 10, 2021 @08:46PM (#27112830)
This is all super exciting. A weird aside: one thing that'd be really nice to have is a userland verifier that we could build into our CI system; testing verifiability involves a lot of really contrived steps in our current process.
Re:eBPF on Windows by pctammela on Monday May 10, 2021 @08:15PM (#27112606)
the PREVAIL implementation seems to be more robust then Linux's verifier, are there any plans to port it to Linux?

From the counter examples directory, this implementation seems very promising.

Re:eBPF on Windows by sanxiyn on Monday May 10, 2021 @07:06PM (#27111997)
Re: termination, can't you use Ethereum gas style termination? I always found termination verification strange, because "gas" is usually more practical alternative.
Re:eBPF on Windows by MarkSweep on Monday May 10, 2021 @09:30PM (#27113089)
The ZFS file system supports running Lua programs in Kernel mode. It supports setting a limit on the number of instructions run and the amount of memory used:

https://www.delphix.com/blog/delphix-engineering/zfs-channel...

So it’s been done before. I think traditional BPF programs were really short and ran in performance critical contexts. Disallowing backwards branches and limiting size was preferred to slowing down these code paths with instruction counting.

Re:eBPF on Windows by MichaelMoser123 on Monday May 10, 2021 @10:45PM (#27113651)
How would you maintain the gas cout? Per instruction or per basic block?
Re:eBPF on Windows by salawat on Monday May 10, 2021 @07:02PM (#27111954)
>Additionally, the PREVAIL paper explicitly doesn't verify program termination, which is kind of a dealbreaker for kernel BPF.

I'd be more alarmed if someone had solved the Halting Problem and I hadn't heard about it, to be honest.

Re:eBPF on Windows by sanxiyn on Monday May 10, 2021 @07:10PM (#27112038)
Microsoft did termination proof for Windows drivers in 2006. Termination proof does not require solving halting problem.

Termination proofs for systems code (PLDI 2006): https://dl.acm.org/doi/10.1145/1133255.1134029

Re:eBPF on Windows by salawat on Monday May 10, 2021 @11:52PM (#27114049)
As mentioned in a sibling reply, there goes my week. Thanks for the reference! Time to learn something I apparently did not know!
Re:eBPF on Windows by whizzter on Tuesday May 11, 2021 @07:53AM (#27116947)
I think you can reason it like this.

Assuming certain constraints (what you can define bounded analysis rules for) you can prove that things terminate. These rules things are usually far more advanced than we suspect initially, but they're not still not fully unbounded.(And we can often come up with more rules to make things practical even if unwieldy)

Deciding termination for an arbitrarily large program in a turing complete system CAN require no less than execution of such a program thus we CANNOT guarantee to be able to verify termination for EVERY program. (This is the halting problem)

I spent a lot of time on my thesis learning about type inference systems and as many such systems for uncooperative languages(JS,Python,etc) need to do abstract interpretation they fall into the same kind of issues and this is why practical systems use JIT compilation instead of AOT even if there are exceptions (Shed Skin Python is quite impressive for example).

Re:eBPF on Windows by ngcc_hk on Wednesday May 12, 2021 @01:40AM (#27127003)
I thought any general system contain arithmetic (as the proof based on transformation) cannot be proved it will be halted. Termination ... can it be terminated without halting?
Re:eBPF on Windows by bshanks on Wednesday May 12, 2021 @06:56PM (#27136183)
Halting is indeed the same as termination. The halting problem DOES allow you to prove that some programs terminate. The problem is that there will be other programs that do terminate, but you won't be able to prove that they terminate.

So there is no way to know, for EVERY possible program, whether that program will terminate or not. But there are still some programs that you can know for certain that those programs will terminate.

Re:eBPF on Windows by cmeacham98 on Monday May 10, 2021 @07:27PM (#27112191)
From my understanding eBPF is sufficiently constrained to where the halting problem is solvable for all representable programs (in eBPF).
Re:eBPF on Windows by tptacek on Monday May 10, 2021 @07:11PM (#27112048)
BPF verifiers don't accept arbitrary programs.
Re:eBPF on Windows by salawat on Monday May 10, 2021 @10:23PM (#27113495)
Awww shit.

I knew this day would come. Be back in like a week. Gotta get my neurons warmed up and translating Theoretic CompSci/discrete math notation again.

If I don't come back send a search party. I'll probably be stuck somewhere around pumping lemmas screaming "This is arbitrary bullshit!"

Re:eBPF on Windows by alexeldeib on Tuesday May 11, 2021 @02:54AM (#27114997)
It’s not magic, just constraints. there are some obvious tricks (which have since been relaxed).

No unbounded loops for one. I think no backward jumps was previously a rule? I think the ability to tail call another bpf program was added, so not sure if you can accidentally loop forever between individually terminating programs.

Re:eBPF on Windows by tptacek on Tuesday May 11, 2021 @03:22AM (#27115234)
You can't; each program is verified independently, and there's a small limit to the number of tail calls you can do.

Being able to run an unbounded loop in eBPF would be a big deal (doesn't mean it can't be done; in fact: it almost surely can be) --- a serious vulnerability.