News for hackers. Stuff that matters.
Re:F-Droid build servers can't build modern Android apps due... by csdreamer7 on Wednesday August 13, 2025 @07:57AM (#44887365)
This means their servers are very old ones that do not support x86-64-v2. Intel Core 2 Duo days?

https://developers.redhat.com/blog/2021/01/05/building-red-h...

Think of how much faster their servers would be with one of those Epyc consumer cpus.

I was about to ask people to donate, but they have $80k in their coffers. I realize their budget is only $17,000 a year, but I am curious why they haven't spent $2-3k on one of those Zen4 or Zen5 matx consumer Epyc servers as they are around under $2k under budget. If they have a fleet of these old servers I imagine a Zen5 one can replace at least a few of them and consume far less power and space.

https://opencollective.com/f-droid#category-BUDGET

Not sure if this includes their Librapay donations either:

https://liberapay.com/F-Droid-Data/donate

Re:F-Droid build servers can't build modern Android apps due... by FirmwareBurner on Wednesday August 13, 2025 @08:33AM (#44887665)
>they have $80k in their coffers but I am curious why they haven't spent $2-3k on one of those Zen4 or Zen5 matx consumer Epyc servers

I would also like to know this.

Re:F-Droid build servers can't build modern Android apps due... by Perz1val on Wednesday August 13, 2025 @10:13AM (#44888710)
Yeah and everybody was complaining how slow the builds are for years. I really want to know too
Re:F-Droid build servers can't build modern Android apps due... by pastage on Wednesday August 13, 2025 @08:40AM (#44887731)
I would much rather they spent that on having the devs network and travel, the servers work.
Re:F-Droid build servers can't build modern Android apps due... by melodyogonna on Wednesday August 13, 2025 @09:10AM (#44888002)
Why are the builds failing then?
Re:F-Droid build servers can't build modern Android apps due... by tcfhgj on Wednesday August 13, 2025 @01:10PM (#44891080)
planned obsolescence by Google
Re:F-Droid build servers can't build modern Android apps due... by shadowgovt on Wednesday August 13, 2025 @03:04PM (#44892457)
Beginning to use a CPU opcode that is 19 years old doesn't feel like planned obsolescence. if anything, it feels like unplanned obsolescence... "Oh hell what do you mean your CPU doesn't have that opcode no we've just been running the compiler with the default flags and that opcode got added to the default two months ago after a 10-year fight about the possible consequences of changing defaults!"

Although I'm a little surprised to learn that the binary itself doesn't have enough information in its header to be able to declare that it needs SSSE3 to be executed; that feels like something that should be statically-analyzed-and-cached to avoid a lot of debugging headaches.

Re:F-Droid build servers can't build modern Android apps due... by tcfhgj on Wednesday August 13, 2025 @03:56PM (#44893060)
> "Oh hell what do you mean your CPU doesn't have that opcode [...]"

hobbyst dev? sure

Google? nope

Re:F-Droid build servers can't build modern Android apps due... by shadowgovt on Wednesday August 13, 2025 @04:11PM (#44893247)
Did they make any explicit guarantees that their newly-cut binaries would continue to support 20-year-old architectures?

Googlers aren't gods. It's a 100,000-person company; they're as vulnerable to "We didn't really think of that one way or the other" as anyone else.

Re:F-Droid build servers can't build modern Android apps due... by Angius on Wednesday August 13, 2025 @04:01PM (#44893121)
They clearly don't
Re:F-Droid build servers can't build modern Android apps due... by lupusreal on Wednesday August 13, 2025 @08:36AM (#44887695)
Probably a case of "don't fix it if it ain't broke" keeping old machines in service too long, so now they broke.
Re:F-Droid build servers can't build modern Android apps due... by FirmwareBurner on Wednesday August 13, 2025 @01:47PM (#44891515)
That's like ignoring your 'Check Engine' light because the engine still runs.
Re:F-Droid build servers can't build modern Android apps due... by Timshel on Wednesday August 13, 2025 @09:12AM (#44888021)
$2-3k ? That’s barely the price of a lower end Threadripper bare cpu not a full Epyc server ???
Re:F-Droid build servers can't build modern Android apps due... by wongarsu on Wednesday August 13, 2025 @09:58AM (#44888537)
At our supplier $2k would pay for a 1U server with a 16 core 3GHz Epyc 7313P with 32GB RAM, a tiny SSD and non-redundant power.

$3k pays for a 1U server with a 32 core 2.6GHz Epyc 7513 with 128GB RAM and 960GB of non-redundant SSD storage (probably fine for build servers).

All using server CPUs, since that was easier to find. If you want more cores or more than 3GHz things get considerably more expensive.

Re:F-Droid build servers can't build modern Android apps due... by Timshel on Wednesday August 13, 2025 @12:04PM (#44890253)
Yes but thoose are Zen 3 Milan cpu released in 2021 I believe.

Not that they are bad and would not be way better than what they have, just that I though the parent was quite the optimist with his Zen4/Zen5 pricing.

Re:F-Droid build servers can't build modern Android apps due... by wtallis on Wednesday August 13, 2025 @12:23PM (#44890488)
OP did say "consumer Epyc", so presumably referring to the parts using the AM5 socket. From a quick check on Newegg, it looks like barebones servers for that platform start at under $1000, to which you need to add CPU, RAM, and storage. So a $3000 budget to assemble a low-end Zen4/5 EPYC server is realistic: $570 for the 16-core EPYC 4565P, a few hundred for DDR5 ECC unbuffered modules, a few hundred for an enterprise SSD, and you have a credible current-gen server from readily available parts at retail prices, without any of the enterprise pricing and procurement hassle.
Re:F-Droid build servers can't build modern Android apps due... by csdreamer7 on Wednesday August 13, 2025 @12:37PM (#44890671)
That was my intention; mATX AM5 parts.
Re:F-Droid build servers can't build modern Android apps due... by BizarroLand on Wednesday August 13, 2025 @01:24PM (#44891229)
I imagine they would need quite a few servers to replace their current setup.

Then there's also the overhead of setting up and maintaining the hardware in their location. It's not just a "solve this problem for ~$2,000 and be done with it".

I don't know the actual specs or requirements. Maybe 1 build server is sufficient, but from what I know there's nearly 4,000 apps on FDroid. 1 server might be swamped handling that much overhead in a timely manner.

Re:F-Droid build servers can't build modern Android apps due... by wtallis on Wednesday August 13, 2025 @02:05PM (#44891769)
One server with today's tech can easily replace several servers that are 12+ years old. 4000 apps doesn't sound like a lot of work for one machine, unless you assume almost all of them are releasing new builds more than once a week. A 16-core CPU can rebuild a full Gentoo desktop OS multiple times a week.
Re:F-Droid build servers can't build modern Android apps due... by speckx on Wednesday August 13, 2025 @01:38PM (#44891395)
Is that $2k/$3k for the year?
Re:F-Droid build servers can't build modern Android apps due... by wongarsu on Wednesday August 13, 2025 @02:37PM (#44892165)
That's $2k/3k to get a box with fully assembled hardware delivered to your doorstep or to a DC of your choice.

Space in your basement or the colo rack of a datacenter along with power, data and cooling is an expense on top. But whatever old servers they have are going to take up more space and use more power and cooling. Upgrading servers that are 5+ years old frequently pays for itself because of the reduced operating costs (unless you opt for more processing power at equal operating cost instead)

Re:F-Droid build servers can't build modern Android apps due... by c0balt on Wednesday August 13, 2025 @11:29AM (#44889760)
Low end EPYC (16-24 cores) especially for older generations are not that expensive 800-1.2K ime. Less when in a second hand server.
Re:F-Droid build servers can't build modern Android apps due... by doublepg23 on Wednesday August 13, 2025 @09:25AM (#44888149)
Perhaps the servers run Coreboot / Libreboot?
Re:F-Droid build servers can't build modern Android apps due... by bayindirh on Wednesday August 13, 2025 @09:32AM (#44888222)
> This means their servers are very old ones that do not support x86-64-v2. Intel Core 2 Duo days?

This is not always a given. In our virtualization platform, we have upgraded a vendor supplied VM recently, and while it booted, some of the services on it failed to start despite exposing a x86_64v2 + AES CPU to the said VM. Minimum requirements cited "Pentium and Celeron", so it was more than enough.

It turned out that one of the services used a single instruction added in a v3 or v4 CPU, and failed to start. We changed the exposed CPU and things have returned to normal.

So, their servers might be capable and misconfigured, or the binary might require more that what it states, or something else.

Re:F-Droid build servers can't build modern Android apps due... by lucb1e on Wednesday August 13, 2025 @10:16AM (#44888756)
A developer on the ticket writes: "Our machines run older server grade CPUs, that indeed do not support the newer SSE4_1 and SSSE3"
Re:F-Droid build servers can't build modern Android apps due... by bayindirh on Wednesday August 13, 2025 @10:18AM (#44888783)
Ooh. They are at least ~15 years old, then. Maybe they have scored on some old, 4 socket Dell R815s. 48 cores ain't that bad for a build server.
Re:F-Droid build servers can't build modern Android apps due... by lucb1e on Wednesday August 13, 2025 @10:21AM (#44888819)
It's kinda good they use such old systems, as the vast majority of pollution occurs during manufacturing of devices since we usually use them only a handful of years. Iirc the break-even point was somewhere around 25 years, as in, upgrading for energy efficiency then becomes worth it (source: https://wimvanderbauwhede.codeberg.page/articles/frugal-comp...). 15 goes a long way towards that!

On the other hand, I didn't dig very deep into the ticket history now but it sounds like this could have been expected: it broke once already 4 years ago (2021), so maybe planning an upgrade for when this happens again would be good foresight. Then again, volunteers... It's not like I picked up the work as an f-droid user either

Re:F-Droid build servers can't build modern Android apps due... by NewJazz on Wednesday August 13, 2025 @12:08PM (#44890301)
While I appreciate the sentiment, I think you may be misreading the "Emissions from production of computational resources" section of that link.

It says for servers that 13-21 years is the break even for emissions from production vs consumption.

The 25 year number is for consumer devices like phones and laptops.

I would also argue that average load on the servers comes into play.

Re:F-Droid build servers can't build modern Android apps due... by miladyincontrol on Thursday August 14, 2025 @12:08AM (#44896652)
Moot point imo, no one says they have to buy new hardware. Used, affordable, but still much more modern hardware could still save them plenty on power usage and replace several systems with one.
Re:F-Droid build servers can't build modern Android apps due... by pclmulqdq on Wednesday August 13, 2025 @09:45AM (#44888386)
I'm not even sure mainline Linux supports machines this old at this point. The cmpxchg16b instruction isn't that old, and I believe it's required now.
Re:F-Droid build servers can't build modern Android apps due... by csdreamer7 on Wednesday August 13, 2025 @12:40PM (#44890695)
32 bit Linux is still supported by the kernel and Debian, Arch, and Fedora still supports baseline x86_64.

RHEL 8 is still supported and Ubuntu is still baseline x86_64 I believe for commercial distros. Not sure about SuSE.

Re:F-Droid build servers can't build modern Android apps due... by squiffsquiff on Wednesday August 13, 2025 @02:11PM (#44891852)
> 32 bit Linux is still supported by the kernel and Debian

Deprecated for Debian

https://www.debian.org/releases/stable/release-notes/issues....

Re:F-Droid build servers can't build modern Android apps due... by csdreamer7 on Thursday August 14, 2025 @09:51AM (#44900368)
>> 32 bit Linux is still supported by the kernel and Debian

> Deprecated for Debian

> https://www.debian.org/releases/stable/release-notes/issues....

32 bit Linux is still supported by the kernel... and... 'Debian, Arch, and Fedora still supports baseline x86_64'.

Please do not take things out of context.

Re:F-Droid build servers can't build modern Android apps due... by cwillu on Wednesday August 13, 2025 @02:09PM (#44891815)
CMPXCHG8B is required as of a month or two ago, not 16B (i.e., the version from the 90's is now required)

See https://lkml.org/lkml/2025/4/25/409

Re:F-Droid build servers can't build modern Android apps due... by ignoramous on Wednesday August 13, 2025 @10:12AM (#44888700)
> about to ask people to donate, but they have $80k in their coffers

I'd still ask folks to donate. £80k isn't much at all given the time and effort I've seen their volunteers spend on keeping the lights on.

From what I recall, they do want to modernize their build infrastructure, but it is as big as an investment they can make. If they had enough in their "coffers", I'm sure they'd feel more confident about it.

It isn't like they don't have any other things to fix or address.

Re:F-Droid build servers can't build modern Android apps due... by csdreamer7 on Wednesday August 13, 2025 @12:41PM (#44890714)
I would too but do you have a link to them talking about it?
Re:F-Droid build servers can't build modern Android apps due... by ignoramous on Sunday August 17, 2025 @01:18PM (#44933192)
They publish weekly reports now on the work they're doing: https://f-droid.org/en/news

For example, here's a recent NLNet sponsorship that helped reproducible builds ship (an effort that began in 2023): https://f-droid.org/en/2025/05/21/making-reproducible-builds...

Their 2025 year in review: https://f-droid.org/en/2025/01/21/a-look-back-at-2024-f-droi...