News for hackers. Stuff that matters.
Re:Abseil: C++ Common Libraries by RcouF1uZ4gsC on Tuesday September 26, 2017 @04:47PM (#15342360)
I think one of the biggest issues for adoption will be the Bazel build system. The C++ world seems to be slowly standardizing on CMake. Having to use another build system to incorporate this library will give people second thoughts about incorporating this library.
Re:Abseil: C++ Common Libraries by haberman on Tuesday September 26, 2017 @05:06PM (#15342518)
Hi there, I work at Google on the Protocol Buffers team. The Abseil team will soon be adding CMake support (and automake, I hope). We need this for Protocol Buffers, which currently supports CMake and automake.
Re:Abseil: C++ Common Libraries by shahms on Tuesday September 26, 2017 @05:39PM (#15342751)
Why would you wish maintenance of either of those things on your coworkers?
Re:Abseil: C++ Common Libraries by haberman on Tuesday September 26, 2017 @06:38PM (#15343221)
As grandparent illustrates, this matters to users. So it matters to me too.
Re:Abseil: C++ Common Libraries by shahms on Tuesday September 26, 2017 @06:57PM (#15343356)
As one of your users, I'd really like to stop maintaining our own BUILD files for proto :-)
Re:Abseil: C++ Common Libraries by haberman on Tuesday September 26, 2017 @07:03PM (#15343397)
I don't follow what you are saying. Protobuf has a BUILD file: https://github.com/google/protobuf/blob/master/BUILD Do you mean something else?

A GitHub issue might be a better place for this discussion.

Re:Abseil: C++ Common Libraries by packetslave on Tuesday September 26, 2017 @07:37PM (#15343622)
As another of your users, I really wish the proto_library BUILD rules were open-source :-)
Re:Abseil: C++ Common Libraries by DannyBee on Tuesday September 26, 2017 @09:45PM (#15344240)
If you feel like you are missing implicit dependencies on building haskell in your project, you could always just add it.
Re:Abseil: C++ Common Libraries by mintplant on Tuesday September 26, 2017 @10:26PM (#15344406)
I wasn't aware Protocol Buffers had any relation to Haskell; would you mind explaining what you're referring to here?
Re:Abseil: C++ Common Libraries by 7197ghr918hf on Tuesday September 26, 2017 @10:31PM (#15344437)
Googler-on-googler trolling. The internal version of the build rule they're discussing has some busted dependencies. Because of historical cruft, it brings in a lot of unrelated crap. For example, building a C++ binary might build bring in Haskell dependencies because of a protocol buffer. This is being fixed, but it's been a thorn in our sides for years.
Re:Abseil: C++ Common Libraries by DannyBee on Wednesday September 27, 2017 @12:25PM (#15349383)
To expand a bit, the proto_library rules were "famously" broken in that it never knew which outputs were actually used (Or so i believe that was the cause, it's been a while)

Thus, it assumed the output was needed for any language specified in the proto library rule. So if your proto library rule offered a haskell version, anything using that proto library rule would do what was necessary (usually, building haskell stuff) to make that output.

The more people offered more languages in their rules, the worse it got.

I believe this was eventually fixed (i stopped coding in the main repo years ago, these days, i only hack on open source projects).

I expect one reason the rule was not open sourced was to not expose this cruft to the rest of the world :)

Re:Abseil: C++ Common Libraries by RcouF1uZ4gsC on Tuesday September 26, 2017 @06:39PM (#15343233)
Glad to hear. This will make using it with our existing code base much easier.
Re:Abseil: C++ Common Libraries by Camillo on Tuesday September 26, 2017 @08:59PM (#15344052)
It would be nice if Bazel could generate CMake projects.
Re:Abseil: C++ Common Libraries by gumby on Tuesday September 26, 2017 @09:17PM (#15344134)
Bazel make Cmake projects which then make make/xcode etc projects? If you have to program Bazel anyway why have it insert CMake?
Re:Abseil: C++ Common Libraries by ithkuil on Wednesday September 27, 2017 @03:38AM (#15345632)
so that other people can consume/build the library without having Bazel installed.

(disclaimer: I love Bazel; I wish more people would just not be afraid of its size)

Re:Abseil: C++ Common Libraries by malkia on Wednesday September 27, 2017 @12:11AM (#15344862)
I think the opposite - the descriptive language of bazel (buck, pants, build.gn and others too) is the key feature. I haven't seen anything that comes so close to defining in very precise, and clean manner your exact dependencies across multiple build platforms.

It is a foreign lang for folks used to MSBuild, .vcxproj/.sln, etc. - like the debug/release, or CRT options are not right away visible (just saw a commit 15-20 hrs ago in bazel to support dynamic CRT linking), but I'm sure this is coming.

Re:Abseil: C++ Common Libraries by vvanders on Wednesday September 27, 2017 @01:04AM (#15345057)
My experience has been anything but positive in those systems. I spent ~3 hours trying to get build.gn to work for a "non-googler" config to build Chromium, gave up on it. Ditto Buck, hit some random dependency issue and would fail to run.

Honestly CMake gets a lot of hell but it's the only one I've seen work wide-scale. Short of a greenfield implementation (like say Rust's package manager) I don't think you'll do much better in C++.

Re:Abseil: C++ Common Libraries by weberc2 on Tuesday September 26, 2017 @05:23PM (#15342638)
CMake is the biggest reason I left C++ for Rust (there are others, of course), which is a shame because I actually came to like modern C++ (after years of using it). I just don't have time to maintain a complex build program (in a crappy, slow language to boot) when Cargo just works out of the box.
Re:Abseil: C++ Common Libraries by pjmlp on Wednesday September 27, 2017 @02:42AM (#15345407)
Cargo still doesn't work out of the box for linking against binary libraries.

That, alongside the lack of incremental compiler and linking, means my C++ work projects (native libraries for Java and .NET) actually do compile much faster than Rust occasional weekend projects.

Re:Abseil: C++ Common Libraries by weberc2 on Wednesday September 27, 2017 @07:51AM (#15346830)
Looks like incremental compilation hit beta in Feb: https://internals.rust-lang.org/t/incremental-compilation-be...

For me it's just fine that cargo doesn't support every odd use case of of the box in exchange for supporting the majority of use cases with virtually no effort. CMake hardly supports anything without a bunch of scripting--this includes reproducible builds, unit testing, etc.

Re:Abseil: C++ Common Libraries by erichocean on Thursday September 28, 2017 @10:40PM (#15362368)
I freaking LOVE Bazel. Convert every library I use from CMake to Bazel on first use. Fork + BUILD and you're good to go.
Re:Abseil: C++ Common Libraries by 0xbear on Tuesday September 26, 2017 @04:51PM (#15342403)
Bazel is not just a build system though. It’s also dependency management system, parallel test system, cross language build system, cross platform build system, and a bunch of other things, with build file syntax a 10 year old could learn in an hour. Some of those things you could plausibly do with CMake, but a lot more effort. Once you learn how to use it (which doesn’t take long) Bazel is pretty darn nice.
Re:Abseil: C++ Common Libraries by mikepurvis on Tuesday September 26, 2017 @05:01PM (#15342471)
Nonetheless, Bazel is very much set up for Google's mentality of building giant static binaries from head and shipping them off to a server somewhere, where they'll be replaced in two weeks with the next release, and then never thought of again. There's some software out here in the world that works like that, but most of it doesn't.

In any case, CMake is _also_ a dependency management system, but works with libraries in the way that they are shipped on a typical Linux distribution. It lets you find common stuff easily, and has multiple extension points in terms of adding support for new libraries. CMake itself can ship your find module, the library you're linking against can ship it, or you can ship it yourself.

Re:Abseil: C++ Common Libraries by star-trek-fleet on Tuesday September 26, 2017 @06:45PM (#15343271)
Trend wise, applications are expanding more and more into controlling its whole life-cycle from binary to deployment. Repeatable build Containers VM Unikernels Standardized libraries

There are other means to share build artifacts or code across language boundary, not necessarily through availability from OS distros. For example, Docker registry solves this issue with container images and overlay FS.

I think one way or another, Google's model has been proven to work for large-scale (probably the largest C++ code base on earth) c++ code base. In that I have my full confidence of adopting Google's proven approach.

disclaimer: Work at google, but not the c++ team

Re:Abseil: C++ Common Libraries by vvanders on Tuesday September 26, 2017 @06:53PM (#15343324)
> Trend wise, applications are expanding more and more into controlling its whole life-cycle from binary to deployment. Repeatable build Containers VM Unikernels Standardized libraries

Maybe in the cloud, however that's only a portion of where you see C++ deployed. For embedded systems(where C & Rust are the only other options) I don't see it going this way anytime soon.

Re:Abseil: C++ Common Libraries by jsolson on Wednesday September 27, 2017 @01:56AM (#15345234)
Aren't embedded systems already there? What is a firmware blob if not a monolithic static build artifact? Bazel is perfect for this. I recently taught it how to build boot images for Xilinx FPGAs including a bitstream, first stage bootloader, kernel, and initial ramdisk, all hermetically built and bottled up into a boot.bin. I can reproduce my firmware from source control and get out the same bits. How neat is that?
Re:Abseil: C++ Common Libraries by 0xbear on Thursday September 28, 2017 @07:33PM (#15361643)
Speaking of which, any possibility of open sourcing this? We are looking to get into FPGAs and would rather not reinvent a bicycle.
Re:Abseil: C++ Common Libraries by jsolson on Saturday September 30, 2017 @01:54PM (#15374080)
Yes.

It's a little complicated as I did it for a personal project, but I'm also pulling it into Google for my 20% project. It is by FAR easier/faster for me to release under Google's copyright, but there's a bit of "paperwork" I need to go through to get it staged and approved for the Google GitHub.

Since it seems there's interest I'll take a stab at that on Monday. It usually takes a week or two, but no promises. I don't anticipate any blocking issues, though, as the bazel bits don't expose anything about Google.

One caveat -- as released it won't quite be bitwise hermetic, as it depends on an external Xilinx toolchain, and Xilinx does non-hermetic things to their bits that are awkward to work around (you can, but my code to do that is a much less mature shell script full of dd and cat commands to rip apart bitstreams and glue them back together -- I've since found open source tooling for accomplishing the same task, but haven't adopted it yet).

Drop me an email (addr in profile) and I'll tell you when it's up.

Re:Abseil: C++ Common Libraries by jopsen on Tuesday September 26, 2017 @11:40PM (#15344745)
Repeatable builds are pretty universally desirable, why wouldn't you want it for embedded systems?

It seems pretty important for security, etc..

Note: Rust and cargo.lock files another example of things going in this direction.

Re:Abseil: C++ Common Libraries by pjmlp on Wednesday September 27, 2017 @02:37AM (#15345387)
> where C & Rust are the only other options

Java, Ada, Pascal, Basic also happen to be viable options, with compiler vendors still doing quite well.

Re:Abseil: C++ Common Libraries by flukus on Tuesday September 26, 2017 @08:53PM (#15344028)
> Trend wise, applications are expanding more and more into controlling its whole life-cycle from binary to deployment. Repeatable build Containers VM Unikernels Standardized libraries

Isn't this the reason behind the equifax breach? From what I understand (not a java dev) Apache struts is deployed along side the application, so the patch couldn't be applied until someone manually updated the dependencies and deployed a new version of the app.

If this was a "traditional" shared library on a distro then it would have been patched by the system administrators.

When you bundle dependencies via any method you are also bundling the exploits.

Re:Abseil: C++ Common Libraries by DannyBee on Tuesday September 26, 2017 @09:49PM (#15344259)
traditional shared libraries never have good enough interfaces to not break stuff. So tradtitionally, the shared library isn't updated anyway until the app is fixed and deployed for the new version of the shared lib.

Bundled dependencies are easy to handle from a security perspective.

Build stamping tells you what CLs are contained (or just go by monotonically incrementing number if it's bundled deps). Someone commits the CL that fixes the dependency. In any sane bundled system, this increments that number. You say that any binary in prod must be something past that CL (which can be verified using the build stamping). Done.

If it's running, it's fixed. Modify to have whatever grace period you want. The same works for shared dependencies, of course.

In practice, both shared dependencies and static dependencies have tradeoffs, but security is not one of them. Both can be and should be easy to make secure in the sense of "make sure it contains the fix".

Both may require app redeployment in practice, and expecting them not to is not a good strategy right now (for most programming languages)

Re:Abseil: C++ Common Libraries by flukus on Tuesday September 26, 2017 @11:09PM (#15344625)
> traditional shared libraries never have good enough interfaces to not break stuff. So tradtitionally, the shared library isn't updated anyway until the app is fixed and deployed for the new version of the shared lib.

I agree and disagree. Traditionally a lot of shared libraries are in the OS and quite good at not breaking things, you can run windows update or apt-get upgrade and apps shouldn't break. Breakages generally happen in higher level libraries, but that is a cultural issue more than a technical one. If more care is put into the interface and managing breaking changes then it is doable. But regardless of how libraries are being deployed "move fast and break things" is not an attitude I want from the developers of them.

Most language specific package managers I've used do a terrible job of telling developers which packages are obsolete and no longer supported or of what the ramifications for updating a package is. Most libraries in these repositories don't even support previous versions.

> Bundled dependencies are easy to handle from a security perspective.

But not from a user or corporate perspective. I shudder to think at how many old and unpatched libraries are currently in use in the corporate world where the have an attitude of "deploy, then don't touch until we request a feature". It's an uphill battle to even upgrade when there are no compatibility issues, if they need development time to update then they're likely to stick to an old version forever.

Users are unlikely to upgrade as well for various reasons, so if a shared library is bundled with an app it is unlikely to be patched.

Re:Abseil: C++ Common Libraries by jonstewart on Wednesday September 27, 2017 @12:10AM (#15344860)
However large and robust the deployment may be, it's still a narrow use case/environment compared. It may be very well outside the norm of C++ use cases, frankly. A lotta people make Windows apps still.
Re:Abseil: C++ Common Libraries by krona on Tuesday September 26, 2017 @05:15PM (#15342583)
> Once you learn how to use it (which doesn’t take long) Bazel is pretty darn nice.

I've spent a few hundred hours integrating bits of CMake build scripts in to various Bazel builds.

If you're doing something which is explicitly catered for by Bazel itself then it works well, so long as you avoid the bugs.

Otherwise, you'll have to confront the hermetic build abstractions which prevent otherwise simple operations. Bazel toolchain files are just as bad as CMake, if not worse.

I'll give Bazel another try in a few years. For now, I still prefer CMake+Ninja, worts and all.

Re:Abseil: C++ Common Libraries by 0xbear on Tuesday September 26, 2017 @05:27PM (#15342670)
Hermetic is good though. Hermetic means reproducible.
Re:Abseil: C++ Common Libraries by pgeorgi on Thursday September 28, 2017 @03:01AM (#15355268)
> Hermetic means reproducible.

Hermetic is a subset of reproducible builds (bit-identical results). You can have a hermetic build and it calls getrandom() as a side-effect that ends up in the binary.

The only thing that bazel prevents is that your build rules do something like that accidentally, without it being glaringly obvious (eg. accessing $RANDOM, $$, /dev/urandom, time, ... in a shell script)

Re:Abseil: C++ Common Libraries by krona on Tuesday September 26, 2017 @05:37PM (#15342734)
Use Docker?
Re:Abseil: C++ Common Libraries by 0xbear on Tuesday September 26, 2017 @06:23PM (#15343100)
Bazel is on its way to be more like Google’s internal implementation: distributed builds, object file cache, distributed tests. That means your C++ build and test runs will take barely any time at all, same as they do at Google. That, in turn, enables the kind of velocity unachievable through other means.
Re:Abseil: C++ Common Libraries by candiodari on Thursday September 28, 2017 @11:53PM (#15362611)
I seem to find many sources that Googlers spend heaps of time waiting for test runs. Apparently they're required for about everything you do with code at Google and are the but of dozens of jokes.

All of which makes me think that maybe Google's internal building of code isn't that fast. Or perhaps it's fast, but only in relative terms (as in you're still waiting a few minutes for hello world, but huge programs build "fast", as in a few dozen minutes)

I did test it on hello world, and euhm ... It's not fast. The first build of it is horribly slow (double digit minutes, wtf is it doing ?). Second build is "fast" which should be understood to still be double digit seconds. Running the same program in VSCode and just compiling it with the system-installed compiler is so fast you see the output before the VSCode has finished drawing the terminal window.

Reading the site I also get the impression that Bazel's singular focus is correctness. They don't mind things being ridiculously slow if it increases the odds of being a correct build.

Re:Abseil: C++ Common Libraries by dekhn on Wednesday September 27, 2017 @10:22AM (#15348027)
I'm sorry but, "bwahahahha". C++ build and test times at Google are astronomical. It's one of the reasons go exists: they built it to compile and link faster.
Re:Abseil: C++ Common Libraries by 0xbear on Wednesday September 27, 2017 @04:50PM (#15352204)
If a minute to build and link half a million lines of C++ is “astronomical”, then sure. Try to build that on your local box, then get back to me. Go exists solely because Rob Pike had nothing better to do, and a lot of googlers hate Java.
Re:Abseil: C++ Common Libraries by dekhn on Thursday September 28, 2017 @11:57AM (#15358117)
Yeah, I actually build locally. It takes a while up front (about half an hour) to build Hello World but incrementals are fast.
Re:Abseil: C++ Common Libraries by paulddraper on Tuesday September 26, 2017 @08:04PM (#15343757)
Bazel does use namespaces on Linux. It's how it can prevent you from accessing files you didn't declare as dependencies.