This was really interesting! I've often wondered how one would do such a thing.

Seeing your name in the Linux changelog must be awesome!

Thank you! It is indeed!

Bravo. Awesome work.

> Seeing your name in the Linux changelog must be awesome!

Years ago when I was a teenager I had a couple patches accepted for ksnake and gedit.

Certainly not as prestigious as a patch to Linux, but still the idea of code I'd written running on millions of PCs around the world felt amazing.

Same. I found serious bugs in a couple of games, exploited some, had fun, wrote a patch and sent it to whoever. Problem is, I was so anxious I did it under an alias. I have my different aliases in the changelog of some games. Nice to look at them, like "wow, that's my contribution", and then it hits "but no one will believe it was me, it was not under my name". God damn it. :(

You know, back when I confused "hash function" with "encryption" and whatnot.

I got a patch to Debian’s vixie-cron accepted [0], which thus far has been my proudest achievement. It’s tiny and stupid, and should never happen to anyone reasonable, but nevertheless, you can now change your server’s TZ without restarting crond or waiting for a DST shift, and your jobs will fire at the new correct time.

Contributing to the kernel is definitely on my list of things to eventually do.

0: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1019716

Congrats. I had a commit in the kernel but it got refactored out.

I'd be very proud of that myself. Nice job!

This was a fun read, and well written. Thanks for sharing! Adding/improving support for some niche piece of hardware sounds like an ideal way to get started with kernel development, and something I'd like to try myself sometime.

It’s definitely something on my bucket list too. Delighted at recent rust developments as a result cause I know zero c

Compared to Rust, C is pretty easy

Kernel development has always been a bit of a mystery to me so I really appreciate this post walking through the process.

Did you try to use any AI tools during their process?

Thanks for the great write-up!

This was an absolutely awesome post, and it makes me want to do the work to fix the functionality on my Lenovo laptop. Though I'm sure the Lenovo drivers are a little more closely watched, so I'll make sure to do my due diligence first. Thank you for the write up!

This is the easiest way to hire engineers with high quality open source contributions with a public track record.

All it takes is just to check that the commit shows up in upstream projects such as Linux and anyone can see the code, the reviews and the authors email in the AUTHORS file which verify that this contribution / patch is indeed from the author who committed that change.

This is a very old form of social proof which saves lots time and makes Leetcode redundant. (Which can now be completely cheated with LLMs.)

Be careful.. any measure becomes a target, and in doing so voids its usefulness. Getting a kernel patch in is probably already somewhat afflicted by this, but imagine the lkml if every Bay Area wannabe company started soft-requiring this as a screen!

> Be careful.. any measure becomes a target, and in doing so voids its usefulness.

Except, this tests for a wide range of tasks related to a typical SWE role: programming proficiency, reasoning through rigorous code-review, clean code, open-source and testing.

Given that Leetcode and Hackerrank tests can be cheated / beaten with LLMs, it not only has been gamed to death, but it tests for nothing that is related to the role.

> but imagine the lkml if every Bay Area wannabe company started soft-requiring this as a screen!

Then this saves time and the proof is on the interviewee to show high quality functional open source changes to high profile projects like the Linux kernel for example and the bar is set high on purpose.

Imagine if hundreds of candidates have "interview cheating" software installed, just to pass the coding interview. Both the interviewer and the interviewee lose.

> Then this saves time and the proof is on the interviewee to show high quality functional open source changes to high profile projects like the Linux kernel for example and the bar is set high on purpose.

At the cost of the maintainer's time being your first round interview.

https://news.ycombinator.com/item?id=24643894

Again, Hacktoberfest is an example of a low quality programme which the bar is lowered on purpose to encourage trash contributions.

You would not be looking for "contributions" that have "edited README" or "fixed typo" in them, which these are examples of non-functional changes.

It is SWEs that have evidence of significant functional changes throughout a high profile open-source project (like Linux, Chromium) with extensive code-reviews and / or have core-contributor roles which is the bar that should be set.

as others have warned.. a scary counterexample is the npm ecosystem, where people are gaming it with questionable npm package spam, to get the 'astroturf footprint' of being the author of widely installed packages. (which get their footprint by piggybacking on actual useful packages, akin to stapled-together law packages in the us congress).

That was so cool!

I too went on this adventure with my laptop. Sadly I hit a wall while reverse engineering the ACPI stuff. With no logs, error messages or tools on the Windows side to intercept the ACPI events, I was at a loss but eventually gave up. Massive respect for managing it with your own laptop!!

I did manage to reverse engineer the keyboard's LEDs and drive them from user space! Studied the kernel to make a contribution but decided not to do so when I saw comments saying it is better to keep functionality in user space if at all possible.

If I had no problem with devoting the time and money, contributing to the kernel (especially in a topic as obscure as making the extra buttons work on a 20-year-old laptop) is at the top of my bucket list, and I am definitely going to be doing it in the near future when my calendar clears up a bit.

Exquisite write-up and OP's simple writing has a motivating ring to it, and I'm now on the local used marketplace looking for pieces of tech like this :-)

I feel most laptops still don't work completely out of the box with Linux, so you don't have to hunt for old hardware.

Maybe you won't find an issue as simple as fixing a button, though.

> Maybe you won't find an issue as simple as fixing a button, though.

Every laptop I've used with linux has had a few non-functioning buttons and keys. I think you underestimate the widespread issue.

We might have a different definition of issue.. I think 100% compatibile working would be launching bloatware installed by the manufacturer. I'm happy not to have the pavlovian training that may some day cause me to click one of these things on someone's windows machine.

> I think 100% compatibile [sic] working would be launching bloatware installed by the manufacturer.

Making a physical button work requires bloatware in your understanding?

> I'm happy not to have the pavlovian training that may some day cause me to click one of these things on someone's windows machine.

Do you know what you're trying to say here? I do not.

I think it's more of the buttons perform specialized tricks to launch bloatware in Windows.

Some of the issue here is the keys themselves have almost no standardization, even across models. Hell, possibly in the same model sometimes. Some backend windows driver captures these signals via a 50 mile long series of if statements that make grown men weep when viewed. This later can mean your totally working fix for the kernel doesn't actually work on a 1/3rd of that fleet of laptops.

> I think it's more of the buttons perform specialized tricks to launch bloatware in Windows.

The linked article is discussing play/pause buttons as well as a "mode-switch" button that allows the play/pause button to have a second function. I do not understand how any of these regular functions become bloatware in your estimation.

> Some of the issue here is the keys themselves have almost no standardization, even across models.

There is actually widespread standardization, which is why many important keys work by default. Laptops sometimes have buttons to disable the internal wifi or adjust the keyboard brightness. These keys are less universal, but still hard to categorize as bloatware.

> ome backend windows driver captures these signals via a 50 mile long series of if statements that make grown men weep when viewed.

I don't know any grown men who would weep when viewing this. I'm confused that you do not like a simple solution (if statements, which a computer has zero problems following precisely even if it is complex to you) nor the complex solution ("bloatware")

> This later can mean your totally working fix for the kernel doesn't actually work on a 1/3rd of that fleet of laptops.

Most devices used in fleets are well-supported in linux after a few years, specifically because of users like the linked article who spend time making buttons worked when pressed.

I have a calculator button on my Dell laptop. Some of these keys are just macros.

The calculator button is one of the "standardized" buttons, it isn't even as complex as macro as it turns out!

And very handy

Really? I had assumed it was running calc.exe via some hidden command line window

Yep! It's basically just a button that tells the OS "open the system calculator"

~~

I looked it up, the Human Interface Devices usage "Consumer Control" code assigned to "Application Launch - Calculator" is 0x0C0192 or 0x192

This keypress is sent as a scancode/keycode, not an ASCII character. On Windows, this opens calc.exe by default, but you can change which app opens in response to the calculator key by editing the media key mappings in the Registry

You can obviously map arbitrary key codes however you want on a custom OS and have extremely little fear of someone having embedded nonsense down to the bios.

On windows many of these laptop buttons were added like the Yahoo browser bar to specifically work with bloatware that might go on to make a meaningful action for non malicious software as well as what it is really for.

I prefer not to be in the habit of pressing footguns given that I might occasionally be placed in front of a consumers windows laptop that no one cleaned.

> I prefer not to be in the habit of pressing footguns given that I might occasionally be placed in front of a consumers windows laptop that no one cleaned.

If you're this anxious about security, you might not want to be anywhere near a Windows machine.

I'm also looking forward to telling a driver that I never I wanted to be near cars when they eventually run me down.

Please help me understand how your comment fits into this thread and not just random words?

I think they're trying to say avoiding a Windows computer is about as difficult as avoiding an automobile, and potentially just as fatal.

I think if they're honestly not being hyperbolic, they should find a less technical career or hobby. If you're afraid of flying, don't join the Air Force.

I'm not sure if there is a word for people who choose high risk activities because they have less fear, because I'm not a coroner.. Perhaps because I'm not sure I have a healthy enough fear of death.

If you examine the stats with the assumption that you could be one who dies early and probably won't take up opioids then there is some logical reasoning to do about your relationship with cars and death. "Mistaking" that for fear is a defense mechanism.

The situation of the worse is better crowd and computer security increasingly looks like one of these things that I don't want to apply such a defense mechanism to.

Linux has not really gotten us closer to eliminating Windows yet it has eliminated a lot of the other approaches to eliminating Windows. Maybe it is only on par with the escooters that can now run me down on the sidewalk but don't seem to have reduced my risk from cars.

> I'm not sure I have a healthy enough fear of death.

You seem to have a very healthy fear of other things, like drivers or scooters running you over, and media buttons on computers launching life-altering software. I'm betting your fear of death might not be healthy, but possibly unhealthy in the sense you're spending your life worrying about things that... well, likely aren't gonna happen to you.

Left right left and all that.

No external compass, something the group views as (or probably unavoidably knows to be) dangerous. Overcoming the fear of it and then belittling those who haven't completed the same process. Psychologists call that the normalization of deviance. A commercial airline pilot has the external compass in your questionable aviation example.

You don't want a group with initial success with the left, right left and so much enjoyment from the sense of fulfillment in social deviance process that they think they can take over everything.

When did every mundane choice became a life-or-death calculation for you? Media buttons aren't footguns. Scooters aren't assassins.

The gap between reasonable caution and this kind of hypervigilance isn't wisdom, it's self-imposed paralysis.

Running pedestrian safety and keyboard shortcuts through the same catastrophic framework is a system that's eating itself - every interaction becomes a referendum on survival.

When people point out that you aren't serving yourself with this worldview, you seem to take the disagreement as proof that others just don't see what's coming.

That's not living... that's white-knuckling through existence, mistaking the grip for control.

If someone wants to tackle this on a laptop where this is the case, WMI is where you want to start.

https://docs.kernel.org/next/wmi/driver-development-guide.ht...

I've never had that problem with a Thinkpad.

Lucky you! There are thousands of ThinkPad models and they're not magically exempt from this issue:

(Not a new issue... here's the problem on an R60) https://unix.stackexchange.com/questions/475968/thinkpad-vol...

E14 Gen4 https://forum.manjaro.org/t/thinkpad-e14-gen4-special-keys-m...

E14 Gen2 issues https://unix.stackexchange.com/questions/609942/thinkpad-spe...

T510 issues https://bbs.archlinux.org/viewtopic.php?id=268269

Fn Volume Control Keys Not Working https://forums.linuxmint.com/viewtopic.php?t=412947

It just takes a cursory glance of search results to see that your ThinkPad experience is not everyone's.

Yep, it definitely is well above 'basememt full of model trains' on my (sadly far-off) list of retirement activities.

That's the spirit! :-)

By the way, delving into obscure and hardware-specific kernel code, sometimes yields quite interesting generally-applicable problems. For example, @dougg3 did an (excellent!) series of articles about his work on bringing mainline kernel support to "Chumby 8", a somewhat obscure, but historically significant piece of hardware, and as a side-quest he stumbled into this:

https://www.downtowndougbrown.com/2024/04/why-is-my-cpu-usag...

...which is applicable to quite a few other machines.

If you want to find devices that still need hardware support under Linux, I highly recommend trying to get a mobile Linux distribution to work on an old smartphone or tablet.

postmarketOS in particular has a really good devices page [1] showing missing feature support at a glance, as well as guides for porting to new devices [2] and porting features from an outdated vendor-provided Linux fork to the upstream kernel [3].

[1] https://wiki.postmarketos.org/wiki/Devices [2] https://wiki.postmarketos.org/wiki/Porting_to_a_new_device [3] https://wiki.postmarketos.org/wiki/Mainlining

I genuinely want to work on postmarketOS but I don't have the technical know-how right now but one day.

I would prefer a sort of dual-boot or just a simple ability to run linux in "android phones"

Like, if we were to build a linux phone somehow hacking it through a raspberry pi or the alikes, they would be so much more costly and subpar.

Android phones have whole globalization working on it and the only reason why they don't work is lack of drivers support/software side, something which can be worked on.

I think if society rewards something like PostMarketOS more/make it easier to install it, then things can be so great as well.

I know I can run a terminal inside android but till now it was only possible through qemu which had its issues...

https://www.androidauthority.com/android-linux-terminal-app-...

I am not sure but I have never really appreciated having linux run inside a vm, I'd much rather run something like waydroid in a postmarketOS phone than linux inside android in an ideal world technically speaking from strictly performance but we don't live in one and installing waydroid on postmarketos or even installing postmarketos can be a very huge issue whereas installing linux on android can be a single step with termux or userLand.

You could also look into running mobile Linux on top of libhybris - it's a proprietary compatibility layer, but some people use it to get support for mobile Linux runtimes on more recent devices.

It's possible to dual-boot. My favorite method is to have a device with a SD-card slot and have android on the eMMC but PostmarketOS on the SD-card.

Currently only do that on one of my older devices, would love to do it on my main device but when I bought it I was in a hurry to get it going and did not have time to unlock the bootloader. Unlocking the bootloader now means having to factory default the device but I simply have too much important stuff set up that it's not worth it. Apparently there's also no great way to backup all partitions of the Android device which I find to be quite strange.

I'm actually trying this right now, but it is becoming much harder than I expected. The fact that I keep needing to recompile kernel and rebooting device doesn't help. I'm trying to make the display and touchscreen work consistently on a qualcomm soc.

And the pmos wiki is severely lacking IMHO.

What I did was buy a wifi router with a chipset supported by OpenWrt but with no build target yet. One of the OpenWrt maintainers then guided me in what I had to do to get the info he needed for a patch. I followed his instructions, he wrote the patch and there it was, a new build target. Later I wrote a patch myself to fix the LED control. It was a rewarding and educational experience. Later I even found someone using OpenWrt on the router I helped get supported which put a smile on my face.

Nice! That's something I've kind of thought I should try one day. Unsupported routers aren't exactly hard to come by these days.

Hieno homma

Kiitos!

I like your website design! Have you documented the design/setup anywhere?

Thanks! I haven't, but I probably should, since you're the second person asking about it. The site is built with Zola[0], and it's using the Radion[1] theme, with small modifications.

[0]: https://www.getzola.org/ [1]: https://github.com/micahkepe/radion

Great write up and easy to follow. I appreciate the extra details related to related things.

I've been procrastinating a trivial fix for years, thanks for having listed the commands to run to format and send the patch, that might help me find my way out from procrastination because this is exactly what's been blocking me.

Such a lovely read and extremely well written! Thanks for the post.

Neat! I don't know much about the Linux ecosystem so I didn't realize how Linus himself is still so deeply involved in the day-to-day review and release process.

This was encouraging! Thanks for explaining the process!

Well written! Enjoyed the detailed report.

Wait… Koskivuori? Well of course they took his Linux patch right away, this is blatant Finnish favoritism! Imagine if some poor Estonian tried similar…

;-)

Just kidding, very cool to see a blow by blow of landing a Linux patch. I felt similar excitement landing a mere emacs patch.

The joke gets even better when you consider that the subsystem maintainer that reviewed my patch is also Finnish :]

I've never really done anything with the kernel, and at this point it feels kind of overwhelming to start contributing.

I'm sure if I went to the source tree and asked people for a low-hanging-fruit task someone would be kind enough to guide me to get started, but it's still kind of overwhelming to a point where I've just avoided it.

I should probably should stop coming up with excuses and just do it, as I would like to do a lot more with filesystems and having an understanding of the kernel would probably help with that.

https://kernelnewbies.org/FAQ/WhereDoIBegin (and https://kernelnewbies.org/ in general) has some good pointers...

This is a nice writeup. I hope we see more great blog posts from this person here in the future.

At the bottom, there is a timeline, and I noticed this entry with a LWN link:

    > 2025-05-27: Sasha Levin selects my patch (and a few others) for backporting...

https://lwn.net/Articles/1020203/ ... which leads to a LKML link: https://lwn.net/ml/all/aBj_SEgFTXfrPVuj@lappy/

The new version of this tool (AUTOSEL) looks very interesting!

    > AUTOSEL leverages modern large language models and

embedding technology to provide significantly more accurate recommendations.

Having your name as a Linux contributor is the highest level of accolade I can think of when it comes to being a programmer.

Even if it's small it demonstrates a level of commitment to working with a human system of senior engineers and ambition for software to become better just because. Always good on a resume.

Agreed. There's also getting a hexadollar from Donald Knuth for finding errors in The Art of Computer Programming

I've never done either, so I'm not bragging or anything

I wonder why they haven't upgraded the spinning hard disk to an SSD. Even on old hardware you'll find that often the HDD still presents a bottleneck.

I wanted to hold off on that upgrade until I found a source of reliable, high-quality PATA SSDs. Haven't looked into that yet, suggestions welcome!

If it really is PATA, my course of action would be to get a pata-to-cf adapter and get a fast and large CF card.

This was a great read. Congratulations on your first contribution to the kernel!

Does `usbhid-dump` show anything when pressing the buttons?

Pretty cool writeup - wish I had time or need to hack on stuff like this - maybe someday ;)

Thank you for this. I have a USB desktop speaker whose volume controls don't work and I was thinking of writing a patch, but didn't know where to start; now I do.

My favourite contribution to the Linux kernel, which I witnessed myself, was a 1-character fix to a macro that my boss found, related to reading ADC, and which we spent two weeks checking and double-checking and re-double-checking to be sure that it was actually a bug.

It was, he submitted it, and that one character fix got him into the contributors file .. we were all highly amused, because that particular boss didn't program much, but found the bug during testing and it was, nevertheless, a huge win for him .. ;)

EDIT: it was a 2-character fix. ;)

https://lkml.iu.edu/2103.2/08109.html

(W., if you read this, I still love telling the story of how you found this bug..)

you can't say that and then keep the story to yourself!

Lets just say that it involved a lot of fluidic particulate quantification.

All over the desk.

I am doing the same but with LLVM. For starters, I filtered issues with the "good first issue" label and found something to work on [1]. It took 2 months in review and about a month to research and write the code. Finally got merged yesterday. I am writing about it so that when someone else bumps into the same problem, they hopefully take 1 hour instead of 1 month. Compilers, like the kernel, are a lot of fun!

[1] https://github.com/llvm/llvm-project/pull/154914

congratulations. as someone who is a bit of a finnaboo and lives there, it's my dream to have had contributed something to the linux kernel. I was wondering if you used the same email for the commit as your github profile so that it shows up in the contributor list in the github mirror of the kernel and on your profile? I don't think I see the contribution when taking a quick glance at your github

I just checked this, and it seems like GitHub is a bit weird about tracking these. It didn't show up on my profile at all until I just now forked[0] torvalds/linux on GitHub. Now it shows up on my profile on May 9th (author date), but the link to the commit is broken, because repository authors page it points to filters on the commit date, not the author date.

Broken link points here and shows no commits: https://github.com/torvalds/linux/commits?author=vkoskiv&sin...

If I correct the date range to include the maintainer commit date (May 15th), the commit shows up: https://github.com/torvalds/linux/commits?author=vkoskiv&sin...

Also strange that if I only filter on author[1], nothing shows up. Though I suspect that dataset might only be updated periodically.

[0]: https://docs.github.com/en/account-and-profile/reference/pro...

[1]: https://github.com/torvalds/linux/commits?author=vkoskiv

That's really awesome - well done. Here's hoping you add a bunch more patches :)

Fantastic read! I've alas been frightened of contributing to the Linux kernel in spite of my interests, this instills both confidence and inspiration while serving as a simple "guide" on how to even approach debugging a hardware issue. I particularly like how the author expresses his nervousness in submitting the patch as he recognizes the importance of the work of the maintainers

Nice, torille!

I also got my first and so far only kernel patch submitted years ago:

On my MacBook Pro 2010 with GeForce 320M running any OpenGL application, even glxgears, would crash because it ran out of memory. So I dug into nouveau drm code and found out that some memory related function was using wrong code path for that GPU.

It took some time to figure out how to submit a patch but it felt nice after I got it accepted.

I didn't even know about those patch sanity-checking scripts back then, they look useful for potential future patches!

cool stuff and congrats. nice reading about the whole journey to it :) always humbled by people digging so deeply into things. thanks for the writeup!