In my opinion, video4linux is already a decent kernel API which exposes cameras to userspace. However, as embedded camera setups are becoming more and more complex, with more and more programmable image processors and configurable subdevice node graphs, the userspace code to interact with the mess is becoming more and more complex. My understanding is that libcamera builds on top of the video4linux kernel API, but presents a much simpler interface.

And they already work with gsteramer. Libcamera provides a gstreamer plug-in. But because libcamera is also a separate library, people who don't want to add all of gstreamer as a dependency can still get the benefits.

I'd love it if people who are more involved could correct me if anything I said is inaccurate.

Ok, a simpler library on top of, and in collaboration with, the existing library I can get behind. Perhaps they should add this to their FAQ.

From the project documentation there's no indication of it being based "on top of" V4L2, in fact the exact opposite. libcamera has compatability layers that add V4L2 and "Android Camera HAL v3" (which I'm not familiar with) on top of libcamera.

https://libcamera.org/docs.html

A lot of software uses the video4linux kernel interface directly. That v4l2 compatibility layer is to make those applications go through libcamera even though they think they're using the kernel interface directly.

Exactly - that's the definition of a compatibility layer!

If you need to use a camera that's supported by libcamera, but not natively by v4l2, and prefer to use the v4l2 API, then libcamera's v4l2 compatability layer will let you do that.

But it seems the goals of libcamera are quite different than v4l2. v4l2 (which I used to use years ago) seems more about supporting the minimum common feature set of cameras - basic video streaming, while libcamera appears more about supporting the greatest common feature set - with CPU implementation of features where necessary (hence the Mesa comparison).

The thing is, libcamera uses video4linux, because video4linux (together with the rest of the media infrastructure userspace API[1]) is how camera devices are exposed by the kernel. It's just that not all cameras are as simple as "find the /dev/videoX device and throw ioctls at it" anymore; you need to set up the configurable subdevice node graph (also using video4linux APIs), and maybe the frames the device gives you need post-processing, etc. There's nothing in libcamera which you couldn't do with video4linux APIs + userspace code, since libcamera is itself just something which sits in userspace and uses video4linux APIs.

The compatibility layer simply makes applications which use video4linux in the "find the /dev/videoX device and throw ioctls at it" way work even with cameras which are much more complicated to set up and configure. But it just replaces one usage pattern of the video4linux kernel interface with a different, more complex usage pattern of the video4linux kernel interface (and the media controller kernel interface) + potential userspace processing.

So libcamera is absolutely "on top of" video4linux is my point.

[1] https://www.kernel.org/doc/html/latest/userspace-api/media/i...

OK I'll take your word for it (libcamera using v4l2), but are you sure there arn't also any cameras supported by libcamera directly? There must be some reason for libcamera's v4l2 compatability layer ...

I hadn't heard of libcamera until reading this story/thread, and it seems to provide useful functionality, so I'm trying to understand the amount of apparent hostility there seems to be towards it.. saying it should be part of v4l2. It certainly seems to provide added functionality "over and above" v4l2, and makes sense if it uses v4l2 for lower level camera access, although from a user perspective that really makes no difference.

To answer the first part: There's a lot of software out there which iterates through `/dev/videoX` devices, opens one of them (maybe by presenting the user with some UI to choose between them), and then just starts using it. You can query which pixel formats are supported and set one, you can query the range of supported frame rates and set a frame rate, you can query and set resolutions, etc. by interacting only with the /dev/videoX device you chose. However, this only works with some cameras; notably, it works with USB webcameras and most laptop webcameras.

But in the embedded/phone world, and with some recent laptops, this can't work. There is simply no single device which is "the" camera. The camera system is a complex pipeline of different image processing nodes. Here's how the graph looks on the hardware I usually work with: https://i.imgur.com/NSmu4Tj.png. The actual camera is the ov5645 node at the top, but that's not really useful in itself. You need to configure the media graph; in this case, I have set it up as ov5645 -> csiphy1 -> csid1 -> ispif1 -> vfe0_pix, and finally, /dev/video3 is a video4linux device (as opposed to a "subdev", which the others are) which an application can interact with using the video4linux IOCTLs. And importantly, you can't configure things like resolution, cropping, etc. on /dev/video3; you have to use the media control API to configure the ov5645 node's resolution, then you can additionally configure scaling and cropping and other forms of image processing on the vfe0_pix node. If you want to access your frames without doing any processing, you can instead link up the ispif1 node to a vfe0_rdiX node (RDI = Raw Dump Interface). My graph is relatively simple because only the vfe0_pix node does actual configurable image processing, but there's no reason the hardware couldn't be structured in a way where different nodes do different kinds of processing.

This is the stuff libcamera understands, and the stuff which normal "open a /dev/videoX device and throw ioctls at it" style application doesn't understand. The goal is for libcamera to figure out what graph it has to build to do what you want, to figure out what image processing operations are available at your various nodes, maybe apply various post-processing steps such as debayering if that's not handled by the image processing hardware, etc.

So the value of the compatibility layer is that an application which expects to just care about a /dev/videoX device can have its API calls intercepted, and libcamera will basically try to make it look to the application as if it's talking to a simple device where framerate/resolution/pixel format control/post processing/etc is all handled by the /dev/videoX device. In the background, libcamera will build the media graph, configure the graph nodes, do post processing, and whatever else is necessary to make the camera work. It basically makes a complex camera system look to the application as if it is a simple USB webcam.

I hope some of this makes the problem that's being solved here a bit more clear.

As to why people are hostile to it, I'd just chock that up to the general culture in Hacker News being of making fun of and demeaning other people's work, especially when they don't understand it.

Obligatory:

https://xkcd.com/927/

This is a case of "We already have a good kernel interface for interacting with cameras (video4linux, but the userspace code which uses that kernel interface is getting increasingly complex as hardware evolves. Let's factor that complexity out into its own library."

libcamera: from the people who made v4l2 a mess

That's both uncharitable & incorrect

from the name I guess v4l2 is the second version of something? So third time's the charm?

Video 4 Linux, and yes it's the second version. v4l was added to the kernel in the late 90s, and v4l2 replaced it in the early 2000s.

https://en.wikipedia.org/wiki/Video4Linux

Interesting statement from the author of megapixels https://git.sr.ht/~martijnbraam/megapixels

The people who downvote this comment probably have not used libcamera. All the smart pointer stuff makes it needlessly difficult. Ironically, the first example on how to use libcamera that I could find leaks memory.

I don't know the reasons of this specific group or person, so I'm obviously not speaking for them. When I work on something I like to do it my way. I don't think I or anyone else has the moral obligation to donate their time to existing projects, just because we have similar interests or goals.

Libcamera does work with gstreamer.

Libcamera is a glue framework between proprietary IPA-packaged code and v4l2 subsystem. Many of the tasks you'd want to use libcamera for are outside the scope of v4l2.

API glue: https://libcamera.org/api-html/namespacelibcamera.html