On Glass, we actually went to Broadcom and had them on the ropes for fixing parts of their firmware. Sadly, we couldn't bring those fixes out of the closed source world of hardware, so it's still up to the system integrator to fight those battles...

Serious question, why doesn't Google build its own bluetooth & BLE chips? Please put some competition on Broadcoam and the like and either push them entirely out of the market (good riddance), or force them to step up their game.

I can't speak to this directly because of numerous reasons, chiefly among them being that I don't get to make those decisions. Standard disclaimer follows: I rejoined in the last two years, what follows are my opinions, these opinions are my own, blah blah.

Android has never been about driving the hardware narrative -- it's always been about building a phone with mostly open contributions and driving the start of a wedge to open up the phone industry a bit. It's always been a software answer to a hardware problem, even today. Prior to Android, all we had were closed source low powered feature phones and Blackberries.

That being said, building silicon is non trivial work, and building a BLE stack and controller is even more so. Will a solid BLE stack sell phones? Hard to say how it could drive that narrative, realistically, and even harder to say if such a controller could be made cost effectively. Given Android's archetype (software solution to closed hardware), this puts such a project into a much more difficult position politically and financially.

I can't see this kind of thing having much in the way of legs in a large corp. That being said, I do think if a startup could challenge this landscape, it is a HUGE opportunity.

    Prior to Android, all we had were closed source low 
    powered feature phones and Blackberries.

I...what? Even if we want to ignore the iPhone for whatever reason, the Palm Treo, Nokia N900, and Windows Phone were firmly established by the time Android started getting demoed--and that was the variant that was very much reminiscent of Windows Phone, with a strong emphasis on the cursor keys over a touchscreen.

The rest of your comment makes sense, but the cognitive dissonance of that sentence was so extreme I had to respond.

I missed a comma in that: closed source phones, low power feature phones, and Blackberries. The N900 is a different, rare breed that did not see widespread adoption.

Fun fact: the first Android phone was the sooner, and it looked and behaved much like a blackberry. Still have mine, in fact.

I loved my Symbian E62

Yeah from the Android platform side it would be weird to build chips. For products like the Pixel phone though, that would be a great place to innovate. And realistically, Google needs to get into the custom chip game sooner rather than later... GCE needs to start competing with Amazon's Graviton ARM processors. The sooner you get the expertise and talent to churn out chips (and maybe even a fab or two?), the better. The global shortage for fab usage and chips could kind of force your hand soon.

Google doesn't have the volume on its first-party hardware to drive something like this yet & they would never start with BLE chips. Additionally, there's some really good vendors out of China already challenging BCOM & QCOM FWIW, which further complicates the "build it yourself" narrative (look at the Pixel buds which have an AirPods-like experience running a Chinese BLE chip for the buds which other Western vendors weren't able to match).

The Android org generally isn't the right org to build hardware, let alone do chip design. Maybe the camera team got closest when I worked there?

Source: worked on Pixel Buds @ Google & was one of several engineers responsible for selecting the chip vendor. We got source access to the entire stack/OS except for the microcode & some parts of the stack they hid. I found BES a way better partner to work with than the BCOM/QCOM mess.

Are you talking about the 1st or 2nd Gen Buds? Because the 2nd gens seem to have severe connection issues:

a) they frequently flip flop between at least two BT profiles, leading to a lackluster listening experience b) they lose connection to each other (leading to intermittend profile switches during reconnects, possibly due to lack of available bandwidth) c) they have severe signal quality issues, leading to limited range and audio interruptions

All these issues only happen with the 2nd Gen Pixel Buds, but not with a random sample of various other true wireless earbuds (I tested Sony WF-1000XM3, 1More True Wireless ANC, Airpods Pro) leading me to believe that this must be a hardware issue on Google's side, especially because the amount of people with the same issues is pretty high.

No other pair of true wireless earbuds hat any issues with music playback while I'm on a road bike, with my phone being on my back, only the Pixel Buds do - and I'm on my 2nd replacement already.

Aren’t they already with TPUs?

Huge difference between designing for their own data centers vs consumer chips.

Yes. And several generations in.

> Prior to Android, all we had were closed source low powered feature phones and Blackberries.

Symbian was open source, ran on millions of smartphones - most of which had app stores and web browsers. Some of which had touchscreens, GPS, augmented reality features etc.

Don't get me wrong - Android has been brilliant. But let's not completely rewrite history, eh?

Right. I didn't think I was trying to rewrite history -- I'm simply pointing out a gap in the market that Android was attempting to fill. Asking me to enumerate everything out there at the time is silly -- especially given the haze of memory.

Symbian was far more popular in Europe than it ever was Stateside, so bear that in mind. I had to import my Nokia E70 gullwing phone before I received my sooner, and what functionality it had was okay, but the browser was hardly more than a WAP browser in a feature phone. The app store was barely there as well, including only a handful of very simple apps at the time.

The browser in S60 phones was actually WebKit. In fact it was Nokia that started the work of fitting WebKit into memory-constrained devices.

Oh, neat! I had no idea it was WebKit on there. Still, what I had wasn't exactly what you'd be comfortable using for more than a few moments. I bought one of the original Nokia Internet Tablets and put together a full wearable system back then to make things a bit better for myself, but I never used the browser in S60 for anything serious because it was so cut down.

Google throws money at problems which don't generate revenue all the time. I feel like all it takes is someone inside Google with enough leverage to push it through without the thing having to make business sense

That could work but the other side of that coin is: if it stops making financial sense Google will do the responsible thing and end it. "Your BT hardware's software is no longer supported" will do more damage to the problem they're trying to solve here

I’ve already had Android phones that didn’t get more than a single major update where the BT/WiFi blobs were locked to some ancient kernel.

tbh, Android consistently having a good bluetooth experience would probably be good for the platform.

> Android has never been about driving the hardware narrative

Apple has been building its own hardware from the beginning, but still also uses Broadcom chips.

They have been integrating hardware from the beginning, not making it all themselves. What they do, though, is demand the ability to vet and fix vendor firmware.

>Apple has been building its own hardware from the beginning

do you mean designing? I'm not familiar with any point in time where Apple was building phones, but maybe i'm mistaken.

A quick google search indicates that even the first generation phones were built by Hon Hai.

They acquired Intel's modem division, so that may change in the future as well.

> What a hopelessly naive perspective.

Maybe, but I was there, helping it develop in the early days, which is the time span we're talking about. I can tell you the core of the Android team was fighting to keep it open -- so much so that about three years in one of the guys who dedicated his job to open sourcing drivers and kernel patches burnt out because of it.

What Android is about is decidedly not what Play Services and GCM core are about.

> Try seeing how willing they are to add support for chipsets on phones that don't include Google Services

It's always been the system integrator's job to work with the OEM vendors to integrate drivers and functionality into Android -- not the Android team's. Even on Glass we had to do this as though we were outside system integrators.

too bad more and more of android was moved into play services over time

It wasn't. Play services is just add on features that proxy app permissions and centralize push notifications -- no actual Android features moved into Play services that I know of. Ie: auto filling SMS OTP codes. It feels like Android is being sucked up that way, but that's because there are a lot of really nice features in there, like push notifications, geofencing, etc.

You can still run Android without GMS core and Play services -- I do that, myself, on a Pixel 3a running Graphene. The trick is that you lose some nice functionality (which Android actually makes up for in some cases, ie: SMS OTP copy buttons), and the mapping experience is god-awful (mostly because the OSS mapping scene is hopelessly stuck in the 1990s GPS model)

one example i can think of without having to look things up is that the music player used to be part of AOSP and then got replaced with a google play variant

e: here's an article from 2018 with some more examples:

https://arstechnica.com/gadgets/2018/07/googles-iron-grip-on...

So the old music player isn't still included by default. It should still work just fine on modern android if you install it, and there's plenty of foss media players on fdroid (ie not Google-dependent), most of them thin wrappers around the android media apis with a few extras like playlists. Kodi and VLC should work fine without Google services, and LineageOS Eleven doesn't require Google, though I'm not sure if it still works on modern Android.

Lack of media playback options shouldn't be able to stop you from using AOSP.

What would get open-source mapping out of this "1990s GPS model"?

Stop geocoding by separating out addresses into singular fields, for one. Stop showing mostly irrelevant contours on maps, for another. OsmAnd~ is unfortunately the only game in town, and the UX is freaking terrible.

how do i get SMS OTP copy buttons on my google-free android device?

> Serious question, why doesn't Google build its own bluetooth & BLE chips?

There is a lot of overlap between WiFi and Bluetooth & BLE such that you just have a Wireless chip that does both. In fact I think with Bluetooth 4 the file transfer profile just establishes an adhoc WiFi network. You don't have separate Bluetooth and WiFi chips anymore.

Furthering that, in the mobile space the Wireless capabilities are usually integrated into the mobile SoC. So you don't even have separate chips for CPU and Wireless.

About the only time you see a separate Wireless chip is when a new technology is emerging like 5G and it's usually only an external chip for a generation or two until it can be integrated into the SoC.

So, if you were to design your own Bluetooth chip today you would also be designing a WiFi chip and then you'd probably just roll all that into a SoC with a CPU. No small feat.

> In fact I think with Bluetooth 4 the file transfer profile just establishes an adhoc WiFi network.

This was a feature of Bluetooth 3.0, but almost nothing ever used it. I was once at a big BT testing company and asked about it, and they had like one device that could do it (a crazy feature-packed HTC WinMo device I think).

And then Bluetooth 4.0 added BLE, and it seems like there hasn't been much development of classic BT since then.

Building radio ASICs is a world rife with patents. Pretty much nobody new can enter the market.

Aren't they considered "essential patents" that must be made available at a reasonable price?

Generally yes. But you would need to spend 3 years in court to get that to work.

Follow-on question for the sake of debate: what if you got a bunch of smaller wannabe startups and interests backing a group effort to do exactly this for a pool of things? Would the legal fees scale linearly? :/

Care to expand? Is that why Software Define Radio is still so much a niche and expensive?

Nearly all modern radio chipsets are mostly software defined. That includes WiFi, LTE and GPS.

The radio frontend is typically a downmixer and then straight into digital.

Some of the typically "software" bits like FFT's, various encodings, checksums, clock recovery etc. are frequently done in digital hardware acceleration blocks for performance, and saving power. If you were writing the firmware of the device, you needn't use them though.

With enough human years of effort, you could take almost any radio hardware for sale today and repurpose it to speak nearly any other radio protocol in similar frequency bands. Performance will probably be terrible though!

It's rare people do this though - all the chips don't have their firmware documented (again mostly to avoid publishing documentation that proves they are violating someone elses patents), and many have various cryptographic elements that makes reverse engineering hard.

The one exception to this is WiFi chips used in the Nexus 5 by Broadcom, which has had a reasonable amount of reverse engineering because Broadcom accidentally published the source code because the firmware code was in part shared with published Linux kernel driver source code.

> all the chips don't have their firmware documented (again mostly to avoid publishing documentation that proves they are violating someone elses patents)

:O

Enlightenment moment :(

There are a gazillion patents on radio related stuff. Nearly all standards have associated patents.

Companies already in the industry typically have cross-licensing agreements - ie. I can use your patents if you can use mine. Either that or they just violate each others patents knowing that a patent war would be mutual destruction and in neither companies interests.

But a newcomer has nothing to offer - the minute they release any product, every incumbent company will go through their patent portfolio and sue them out of the water.

Yikes, that sounds pretty harsh. So how would a new player be able do enter?

I thought only the hardware could be patented, not the software, and so SDR would level the playing field, but that's perhaps too naive?

Google's involvement in any kind of hardware is already a distraction from their core product line. Making their own chips is a distraction on a distraction.

Distraction might not be the right word but I can't conjure up the right one.

Hardware is one of the end goals for Apple, for example. For Google, Android hardware is not. It's just there to serve their goal of selling ads.

IMHO those dynamics are changing. Custom chips are becoming table stakes for new products. Look at Apple M1, W1, or Amazaon's Graviton2 chips. All of those are core parts of products and services which would not be possible without the custom silicon. The pool of talent and resources to build these things is extremely small, and there are geopolitical issues putting ever more pressure and scarcity on them (i.e. China pivoting to reduce dependency on Western designed chips and hoovering up as much chip design talent as possible). The TL;DR is amazing new hardware and services need custom silicon, but custom silicon is only getting more difficult and more challenging to build in the near future.

Getting good yield/performance on the radio is non trivial. Aside from having domain experts you have a pretty significant investment in equipment to do the testing. Broadcom and other semis split that cost over many customers.

Tangential - in the BLE land nrf52832 and friends are pretty neat, and there is an ongoing work on the rust stack: https://github.com/jonas-schievink/rubble

I tried it and it can do a few basic things, though it’s in the early stages, apparently.

Nrf52 are great BLE chips, however the full bluetooth spec (classic+BLE) is orders of magnitude more complex...

Even if some courageous developer there fixes the bugs and updates the firmware; how many end-users would actually receive the update and actually apply the update? That's the problem Linux's LVFS[1] solves but it's unfortunate that not all manufacturers support it.

I got update for my half a decade old Logitech's 2.4 GHz receiver (nRF24L) for wireless keyboard as soon as I plugged it on Linux, I've used the same keyboard on Mac and the official Logitech software doesn't even detect the device properly let alone update the receiver's firmware(no issues using the device though).

[1] https://fwupd.org/

Do you have any more info about ROM patch slots? I have never heard of this before. I assume this is a small amount of r/w memory that is somehow overlaid over specific locations in the ROM?

Correct. It's a small table of:

  address1, 4 bytes overlay data
  address2, 4 bytes overlay data
  etc

The data is overlayed over the specified addresses, in runtime. On some chips its 8 bytes instead of 4. On a typical Broadcom/Cypress chip you have 128 or 256 entries.

By the time the chip is 2-3 years in the market and still getting firmware updates, ~98% of them are used by existing firmware, so there are only 5-10 free entries by the time the chip is considered "obsolete".

Case in point: the Broadcom/Cypress BCM43455 chip on the raspberry pi is almost out of patch entries. Broadcom have switched to their usual tactic of stalling for years on known, reproducible bug reports.

> Case in point: the Broadcom/Cypress BCM43455 chip on the raspberry pi is almost out of patch entries. Broadcom have switched to their usual tactic of stalling for years on known, reproducible bug reports.

And it's still really buggy. I had to write a service on the RPI and the only way to reliably connect was to restart bluetooth before every attempt.

That kind of fix makes a person feel dirty.

Sadly that's common in the hardware world.

Step 1. Have a reliable hardware watchdog that restarts everytime there's a software problem.

Step 2. There is no step 2.

Such is the sad world of Bluetooth. The dirty secret to this industry is that this, while seeming hacky, is the bare minimum de-facto standard in most cases.

So given these data points, isn’t it reasonable for Apple to refuse to play along this broken tune and just roll out their own dialect of a wireless protocol? Why, if not in the name of scarcely affirmed “standards”, drag suppliers through an endless contractual game, when you can direct your own capacity toward the quality standards that fit you?

I don’t follow the leap. The grandparent’s point was about the quality and terrible lack of long term support of Broadcom chips. How does that translate to issues of the standard itself?

Nobody would complaining about Apple creating their own radio chips (which they seem to plan for 5G/6G). Apple creating their own standard protocols is an issue though.

Well, if the implementations of the standard are such a garbage fire what's the point chasing them? Just to check the box "standards compliant" and likely providing an abysmal UX and poor interoperability?

I fixated on Apple because they're often picked on for taking the highway, but on the other hand what's the point doing otherwise? What's a common ground if it's just a pipe dream?

Just because a chip is shitty doesn't mean it's worthless. In practice, Bluetooth is quite interoperable, and reliable enough for many use cases (especially the common, better-tested ones).

Breaking compatibility with that ecosystem out of spite is not conducive to getting adoption for a better product.

Well, the original posts report some frankly tragic scenarios - so bad that they “reboot to initialize” just to keep sane - in what are some pretty ubiquitous devices. Or not?

"Reboot to initialize" is ugly as hell and very brittle, but it's good enough for most I/O devices like keyboards or headphones. If the kernel is able to properly reinitialize the chip with all of its old association information, it might even be indistinguishable from a few hundred ms of interference. (Rebooting on errors is in fact quite common for all kinds of hardware in high-radiation environments, and is a pretty standard kernel technique for working around buggy hardware.)

Now, of course, multiple nines of uptime would be very nice to have (and open up new use-cases), but 2-3 nines is still a lot better than 0.

Fine, but you're rationalizing living with poor execution. I'm rationalizing with deep (perhaps goldplating) correctness.

I'm not "rationalizing", and neither are you.

You're arguing that it's not enough to make a correct implementation, but that it's also important to break compatibility with incorrect implementations.

I'm arguing that a better implementation that is compatible with current protocols is strictly better than a better implementation that is not Bluetooth-compatible.

If you make a piece of hardware that is good (e.g. doesn't randomly crash and need to be rebooted by the kernel), why is it a bad thing for it to try to connect to some flaky BT headset?

The real brokenness here seems to be that the chips are not engineered with say ten times the patch capacity.

And the root of the brokenness is that there isn't the end-to-end awareness and acceptance that ten times the capacity is obviously needed.

Owww.

Easy to say, and I can’t know for sure exactly what factored impacted Broadcom’s decisions here, but I can tell you that chip manufacturers are under extreme pressure to keep costs down, which means that they may under-spec systems at times. Also, with the long design cycles involved in chip design the patch capabilities may have been decided years in advance, before realizing how much would be needed.

In general I agree with your comment, though it’s a lot easier to say this in hindsight.

For more illustrations of bugs/vulns in the firmware: https://www.youtube.com/watch?v=7tIQjPjjJQc&t=1986s

Broadcom's firmware seems to be just absolutely terrible across the stack, NIC included. They seem to have solid product design / engineering chops, but firmware just defies them.

My question would be what were the motivations to move into a stack written in Rust, if the much of the bugs are in the closed source FW running in the peripheral?

What would happen if consistency is lost outside the Rust domain but still in the BT stack?