danvittegleo

this is the first thing I list in the readme of the repo "For fun only. This is a proof-of-concept exploring what's possible..."

Absolutely - there are plenty of more cost-effective, cloud-agnostic ways to build something like this. This is just an experiment to explore Lambda networking and push it beyond its intended use cases.

Like most things with AWS, it’s tricky to pin down exactly - and it’ll probably cost more than you’d expect once you factor in Lambda invocations, GB-seconds of execution time, S3 request fees, and especially data‐transfer charges.

I did toy around with Tailscale initially trying to get it to spin up as an exit node but wasn't able to get that functional. I did manage to get Tailscale Funnel to work as the tunnel mechanism to Lambda, but unfortunately the performance was really poor.

It scales amazingly well with the number of cores you throw at it. Definitely the 3950x would work great.

I investigated EBS snapshot as an option, but there were two problems. 1) cost as i mentioned initially - for just AOSP source tree alone you are looking at > 250GB and at a cost of $0.05 per GB you are already at > $10/month and 2) EBS snapshots lazy load from S3 which gives TERRIBLE performance which means you end up with far far slower builds. AWS released a feature "EBS Fast Snapshot Restore" to workaround this issue, but it's extremely expensive.

In order to limit costs, everything is pulled from source on each build and there is nothing cached. This strategy takes advantage of the fact that AWS doesn't charge for ingress traffic and unfortunately puts additional load on Google's servers. I've attempted a few different strategies on caching AOSP and Chromium source trees, but since you have to incur the storage costs on an ongoing basis, it's just not very economical.

Yes, building AOSP requires a fairly powerful machine (at least to do it quickly): https://source.android.com/setup/build/requirements. It's definitely possible to do on a local machine with decent specs though.

With rattlesnakeos-stack, it uses spot instances and defaults to a c5.4xlarge which takes about 7-8 hours to build AOSP and Chromium (for an up to date webview) and equates to about ~$1 a build. I typically build on a c5.24xlarge instance which takes about 2.5 hours and costs about ~$2 per build. Unfortunately both AOSP and Chromium are massive projects that require a ton of computing power to build quickly. It's definitely still possible to do on less powerful machines, but it's just going to take a lot longer to do builds.

CalyxOS is an awesome project. I have worked with the lead developer a bit over the past few years and it's been such a pleasure. We share some bits of code between our projects here: https://github.com/AOSPAlliance.

If anyone is interested in building their own custom android OS in the cloud (AWS) with same ability to lock your bootloader like CalyxOS, you can checkout my project I've been maintaining for a few years now called RattlesnakeOS: https://github.com/dan-v/rattlesnakeos-stack.

And if you prefer to not build in the cloud, there is also a really great project called robotnix (https://github.com/danielfullmer/robotnix) which provides a way to build many flavors of OS (AOSP, GrapheneOS, LineageOS, etc).

Running it with pypy sped it up quite a bit and got it far enough to figure out the remaining portion of the URL.

For anyone interested in doing their own customizable builds of AOSP for Pixel devices check out: https://github.com/dan-v/rattlesnakeos-stack. This doesn't have any of the security hardening features of GrapheneOS, but does have some of the same security properties like verified boot, OTA software updates that included updated firmware/drivers, support for remote attestation, etc.

Agreed. For anyone interested in doing their own monthly signed AOSP builds for Pixel phones with OTA updates, take a look at a project that I built that fully automates the process in AWS: https://github.com/dan-v/rattlesnakeos-stack.

For those that prefer algo, there is an open issue to add Pi-Hole (https://github.com/trailofbits/algo/issues/1258) and in my fork I've added Pi-Hole support: https://github.com/dan-v/algo. It's a really nice setup especially on mobile devices where you typically have to choose between using a VPN or using adblock as they typically require a local VPN hack to function.

Ha, you definitely have some years on me, but I can appreciate that perspective :)

If you are looking to maintain a secure device similar to stock Android without google services, a Pixel phone with verified boot using your own signing keys, latest AOSP, and up to date drivers/firmware is a far better option. I've been working on a project that automates the entire process in AWS and it supports all Pixel phones now: https://github.com/dan-v/rattlesnakeos-stack.

For sure. I'd love to have this script be able to run standalone as a local build process that would be portable across platforms and only run the AWS specific bits if you passed a flag for example. Anyways, happy to chat - you can shoot me an email [email protected]

Right now if a spot instance terminates mid-build it doesn't attempt to save any progress. I do have an open item to at least alert users that this has happened though (https://github.com/dan-v/rattlesnakeos-stack/issues/41). Although if Chromium is built it will checkpoint that by saving the built artifact to S3 and would then skip the Chromium build next time. I've found it's much cheaper to do full builds on AWS each time rather than trying to store the source tree for AOSP and Chromium as they are just so large.

If you were looking to just do something similar but running on another platform, knowledge of Go isn't required. The core build process is a shell script, which just has some variables injected through Go templates: https://raw.githubusercontent.com/dan-v/rattlesnakeos-stack/...

I actually recently added a feature to search for the cheapest spot instance price across a number of different regions. Right now for example the cheapest price for the instance type used c5.4xlarge is ~$0.15/hour in us-east-2. This equates to a full AOSP build with Chromium build included (~5.25 hours) costing $0.80 and without Chromium (~1.75 hours) costing $0.30

I made an alternative to CopperheadOS after it fell apart for those interested: https://github.com/dan-v/rattlesnakeos-stack. Rather than providing random binaries of an OS to install on your phone, I've gone the route of creating a cross platform tool, rattlesnakeos-stack, that provisions all of the AWS infrastructure needed to continuously build your own personal RattlesnakeOS, with your own signing keys, and your own OTA updates. It uses AWS Lambda to provision EC2 Spot Instances that build RattlesnakeOS and upload artifacts to S3. Resulting OS builds are configured to receive over the air updates from this environment.

I haven't spent much time looking at this part of the project as my phone isn't supported but it does look really neat. On my next phone upgrade I will probably investigate.

Agreed that if lower level hardware is compromised it doesn't matter what you run on top of it, and RattlesnakeOS doesn't protect from that other than keeping components up to date with latest security patches. I think if you are going to buy a dumb flip phone and solder out components for privacy, you are probably better off just not having a phone at all. This project is more about having a useful smartphone experience while still focusing on privacy and security.

That's a great question. I would say it is a bit limiting, and there are definitely some bumps in the road for the average user coming from stock Android. There are a lot of great open source applications in the F-Droid app store, but it definitely doesn't cover everything you'd expect from a stock phone these days. There is an app, Yalp Store (it downloads APKs directly from Google servers), that can be used to supplement this with normal Play Store apps. Many apps in the Play Store will rely on Google services though, so you will have a wide variety of results ranging from apps running just fine, crashing on boot, or not supporting push notifications. For me, not getting push notifications in Slack was an issue as it relies on Google Cloud Messaging (GCM). Luckily the encrypted chat messengers that I use Signal/Telegram have support for their own polling mechanism and don't rely on GCM. For Slack I ended up just writing a quick tool (https://github.com/dan-v/slack-to-telegram) to forward messages to Telegram so I get immediately alerts. But it's things like that this that you will definitely run into when running without Google services.

Unfortunately there aren't very many phones that currently support verified boot (https://source.android.com/security/verifiedboot/), which would be a requirement to be supported by this project.

Nexus 5x is end of life unfortunately, which means it won't get security updates from Google and therefore isn't a great fit for this project.

I like this idea and could definitely get behind something like this. The signing process is done after builds complete, so it might be possible. Although on Pixel and Pixel XL it is likely not possible as the kernel must be built with the signing key to support the earlier version of Android Verified Boot (AVB 1.0).

Thanks for the response! Unfortunately, I'm personally not interested in distributing binaries of the OS for public consumption, as I think installing an OS from a unknown person on the internet is probably not the best approach for a privacy/security focused Android OS. I personally would prefer to do my own builds with my own keys so that I know what is running on my phone - hence why i created this tool. That said, it doesn't mean someone else can't spin up a public version of this setup using this tool.