The last section of the article is the most relevant: the hardware, tools, and documentation for USB3 are of mediocre quality. The whole history of moving bits over cables is adding and adapting tricks for getting more and more data through, we’ve always been at the point where poorly implemented solutions would cause problems, and we’re several orders of magnitude away from it being possible for a novice to implement a hardware and software solution from scratch (bit banging a serial interface on a microcontroller GPIO is possible for a reasonable person to do in a week to achieve less than a megabit connection). Super fast data rates are all over and most of them are very reliable, USB isn’t up to the same quality. You can make any speed reliable and fool proof, you just have to do it.
A large number of problems appear to be bad hardware compliance, documentation, firmware or driver. For example, I have a PC with an early Renesas USB 3.0 controller. Once in a while, the controller could become completely unresponsive. The reason was that the controller has a firmware bug, and you can actually get some reverse-enginnered firmware update tool on GitHub to fix it.
I'm not denying physics. Funny that you use Gigabit Ethernet as an example... I'm still debugging a hobby Gigabit Ethernet project. For some complicated reasons, It has to use a standard BASE1000-T PHY, but instead of communicating over a CAT-6 cable, it has to communicate over the circuit board backplane. Previous prototypes couldn't even reliably auto-negotiate. Apparently, in Ethernet, not only a long cable and heavy attenuation can be the problem, an extremely short link with low attenuation can also confuse the "Feed-Forward Equalization" DSP algorithm in the PHY and create problems. I totally understand the inherent challenges of high-speed designs and have some hand-on experience.
But I have to say, yes, you can make any speed reliable, given a controlled environment and a reasonable technical constraints. It's called engineering. Howard Johnson (the author of the famed engineering handbook High-Speed Digital Design - A Handbook of Black Magic), in a 1997 article on Gigabit Ethernet, which he heavily involved in its standardization, he discussed its enormous challenges [0]...
> Key ingredients in any Gigabit Ethernet design will include:
> Careful control of clock skew on the 125-MHz 10-bit bus.
> Attention to crosstalk between massive, 125 MHz TTL-level parallel buses and critical high-speed serial circuits.
> EMI considerations on the serial cables (if you are not an expert in this area, I suggest choose fiber)
> Huge pin counts, especially on complex switching or routing ASICS, some of which will be using 512-bit bus interconnections
> More than usual emphasis on time-to-market; everybody will want their products ASAP
> If you thought Fast Ethernet 100BASE-T layouts were tricky, think again. Gigabit Ethernet is going to be faster, with more parallel signals, and tighter layout constraints.
However, he also said,
> It will work, and it will work reliably, but you will have to follow the rules.
You can make any speed reliable within a reasonable engineering constrict, and in a controlled setup. Using off-the-shelf PHYs and controllers, doing hardware compliance testing, and installing good cables are all supposed to create this controlled setup. Failures to do so is what we call "hardware compliance problems". I agree that it's definitely not foolproof. Saying it is would be denying physics, but I didn't say that. EMI/EFI is a serious problem for USB 3.o. I'm only saying that a lot of USB 3.0 problems in practice is bad hardware compliance, true electrical problems can be fewer.
It took a while to debug because I was absolutely convinced the slowdown was a software or GPU issue. After all, wouldn't you expect most digital ports to be binary: either be plugged in or not? It turns out with USB3 there's a third state.
No. USB-A plugs for 2.x have 4 contacts that extend close to the outer edge. USB-A plugs for 3.x have additional 5 contacts close to the inner edge. If you don't insert the plug fully, you physically just get a USB 2.x connection. Nothing magic there.
https://www.techspot.com/guides/235-everything-about-usb-30/
There are more subtle problems with 3.0.
Mmm, I am surprised about that because USB 2.0 is fully capable of 1080p60 without issues. I have Logitech C910 and it does 1080p60 flawlessly. I wonder if it is not actually 2.0 but USB 1.1 which made sense since the transfer rate is abysmally low (1.1 does up to 12 Mbps, 2.0 is up to 480 Mbps).
But again, computers does have their quirks and can do weird & strange things that seem impossible but it is possible.
I am not who you were responding to, but I can confirm that at 480 Mbps our Intel Realsense does not work, 5000 Mbps are mandatory. We use an Infrared stereo mode at 30 fps. I am not a my desk now and don't remember the exact details by heart.
mjpeg compressed
https://telestreamforum.forumbee.com/t/6363z0/psa-windows-10...
It's always a good idea to have a known-good extra cable or two and laptop that you know has good ports. All the time and frustration you will save is definitely worth the hassle.
In contrast to the sheer idiocy of demanding a "pocket change" fee of $1 per port, which made FireWire an instant turnoff for most device makers not targeting Western markets, and singlehandedly killed FireWire as a standard.
FireWire was possibly better engineered, but USB was better marketed.
Yes, perhaps, but it had a big drawback. You could not build a Firewire device with a single die because of the high voltages. This added cost to the USB device (memory stick, etc) meant it would never be price competitive with USB. I was a member of a USB working group and was amazed when a member from a large Japanese company explained that to me. Up to that point, I had not been exposed to the low-end (as in price) of the USB market and didn't realize the significance of this issue. These manufactures would bring the 5V straight into their IC and drop the voltage on-chip. That is actually fairly difficult in the age of such small chip geometries, but basically impossible for Firewire which is 8V to 30V. The USB market is driven by the low-end, so Firewire never had a chance because of an engineering specification.
To be fair to them though, it's very easy to get caught up in the beauty of those theoretical designs and forget that the best-selling car is a Toyota Corolla, not a Lamborghini.
Or, to put that another way: is the total profit per Lamborghini sold really higher than the total profit per Corolla sold, after the CapEx and OpEx of any maintenance + spare-parts supply logistics + recall costs + training + supply contracts + etc. are taken into account? (I would assume, if it were, Lamborghini would have a higher market cap.)
A Lamborghini is full of flashy, pie-in-the-sky engineering that often ignores practicality in favour of flash and performance. On paper, it's a beautiful machine, but much like in the case of FireWire, it doesn't meet eye to eye with the real world around it.
The Toyota is less ambitious, more constrained by reality, and while it certainly doesn't match the on-paper specs of the Lamborghini, it is in every way more practical and viable as a system.
I would consider the Toyota to be the better engineered of the two when looking at them as cars and not as toys, even though it lacks the technical wow factor.
In that sense, Firewire is less analogous to a Lambo ("a better car"), and more akin to a truck. But a very tiny truck, designed for both off-roading and portability. Which is to say: an ATV.
An ATV isn't better at being a car than a car is, but it is better at being a truck than a car is (though worse at being a truck than a truck is.) But sometimes, especially in constrained environments (e.g. search-and-rescue in the Alaskan wilderness), a truck won't fit, and a car won't work. So you need an ATV. It's well-engineered for that particular set of constraints.
Which is also the deal with Thunderbolt: it's PCI-e, but smaller and ruggedized, for places PCI-e itself wouldn't work (like a laptop), but where you nevertheless still need PCI-e -alike performance and bus semantics.
Even including an optional 5V power pin would have improved this situation, and could have put FireWire in a similar situation to Thunderbolt 3 now, capable of functioning with both high-end professional equipment and ordinary consumer devices.
With engineering solutions like Firewire it could have been as fast and more reliable. But more expensive, i.e. not mass market or consumer friendly.
The problem with USB 1 and 2 is that it was designed by insane committee. This resulted in stuff like those versions not using true differential signaling (like USB3 does) even though that was common technology at the time. Instead they have end of packets delimited by single-ended conditions. That, coupled with a single ground wire for power and data makes for a world of ground loop issues.
And then they made the frame rate 1kHz, which is right in the peak of human hearing sensitivity. And so, USB's terrible design is single-handedly responsible for a huge amount of, even most, noise and interference issues in audio systems involving it. Especially in low cost consumer hardware, but it has made its way into many professional productions. Any time you hear a constant 1kHz beep of interference, mixed with some fuzzy crackling, nine times out of ten that's a ground loop with USB and audio involved. It's everywhere once you listen closely. Headphone output on your laptop or phone is noisy when you access a USB drive, or has a 1k beep? USB. USB audio widget has background fuzz and a beep? USB. Slight 1k beep in the background of a news cast? Someone had the bright idea of using USB somewhere along the line.
It didn't have to be this way. Ethernet, SATA, plenty of other protocols use proper differential signaling and don't have this issue. How to do this properly was well established long before USB existed.
And to add insult to injury, while USB 2 specifies a transaction translator, so you can run USB 1 devices on a USB 2 bus, USB 3 does not. And so, while USB 3 devices don't have this problem any more, you can't plug in a USB 2 device into a USB 3 bus or a USB 3 hub. And nobody uses USB 3 for low channel count audio interfaces, so this problem will remain forever. Oh sure, you can plug it into a USB 3 port, which is actually a USB 3 port and a USB 2 port in one. And you can plug it into a hub labeled "USB 3" which actually has a whole USB 2 hub along side it in parallel. The communication from USB 2 devices remains USB 2 all the way to the host. And so, not only do USB 2 devices only get 480Mbps of total bandwidth on a USB "3" hub (minus a ton of overhead, because the protocol is terrible), it doesn't fix this issue! And did I mention USB 2 is impossible to galvanically isolate in any kind of cost-effective way?
I could go on and on about how the polled architecture causes performance bottlenecks, about how it's impossible to tunnel without unfixable corner cases, about how its latency requirements preclude lots of innovative ideas, about how the descriptor and device inquiry protocols are practically designed to guarantee that implementations are vulnerable and have exploitable bugs (yes, your USB stack is vulnerable. They all are. This is how the PS3 got hacked. And the Switch. And tons of other devices. USB developers like me know misbehaving USB devices crash hosts all the time).
I don't like USB.
Real transaction translators in USB 2 work with host cooperation and specific support for USB 1 devices in the protocol, while this is trying to do transparent protocol conversion. Sadly, due to USB's design, this runs into the same corner case problems as any other kind of tunneling.
The main problem is the polled architecture. Host asks device for data; translator has to reply immediately so it has to say there is no data yet and ask the device for data. Translator buffers data and delivers it to the host the next time it asks. Except the host is allowed to never ask, again. Now the translator has data it has to drop on the floor. Not good. And this all requires heuristics to decide what to do. It's a mess.
TT = Transaction Translator
You might say Firewire could have provided the same for me, but Firewire wasn't widely deployed or adopted. Therefore it could not have replaced USB for my use cases. That may not be a technical issue, but I don't really care as a user if the failure of Firewire is technical or not. Technical superiority is only a small piece of the puzzle. Plus, I think you are underestimating the technical advantages which led to USB's success (e.g. low cost).
USB1 and USB2 still work fine despite being less than ideal engineering. Tolerances are big enough at lower speeds. USB3 can be unreliable (e.g. for cameras) because limits set by the laws of physics get too close for cheap implementations. That's no longer good. Especially if we are forced to use that in professional set-ups, where a couple of cents or even Euros/dollars would not harm. The problem is now that you cannot put that tiny bit of money on the table and you get a reliable product. As the discussion showed even certified and known brands' USB3 products can be unreliable.
Sounds like it delivered what it said on the tin!
Do we?
My phone, headphones, drawing tablet, personal laptop, work laptop, and monitors all plug in using a single port.
Not just for connectivity, but also for power.
It's incredibly liberating. Everything charges from the same charger, everything plugs together.
I vividly remember the days of trying to find the right wall wart for the right device. I can't tell you how happy I am that my work laptop and personal laptop can both share chargers. That alone would make me ecstatic. That my phone/headphones/tablet also all work with just that charger is candy on top. That my monitor can provide that power while also using the same cable for display in? Incredible!
There are absolutely still issues, and it requires more research than it should, but once it works, it's SO DAMN NICE that I simply can't imagine going back.
https://dl.acm.org/doi/pdf/10.1145/3230543.3230560 "For 64B DMA Reads it drops by almost 70% and even for 256B DMAs the drop is still a significant 30%".
There is crack the PC open physical access and just connect a device to a readily available connection access. With the later the adversary doesn't even need physical access himself, he just needs an unwary victim.
Smartphones had been plagued by something as simple as a public charging station getting compromised when Android was configured to just grant access to the file system and debug interface without requiring user confirmation. Before that Windows stopped auto playing software because it was used to infect systems from compromised devices.
Wikipedia seems to state no OS has managed to do so (as of 2019). Thunderbolt had been around since 2011.
> Intel stopped using IOMMU support on CPUs for product segmentation by the time Thunderbolt 3 arrived
So it couldn't possibly be made secure before 2015 and still wont be secure until $UNIVERSE_HEATDEATHDATE .
You're wildly misunderstanding things. Up to 2015, it was possible to build a new PC that lacked the necessary hardware features, primarily by choosing one of Intel's enthusiast-oriented overclockable CPU models (which used to have the IOMMU disabled for product segmentation reasons). But it was still very much possible to get all the necessary hardware features by selecting the next slower speed grade of the same chip, or by buying almost any pre-built system equipped with Thunderbolt ports. And since then, it's been almost impossible to build a PC that lacks the necessary hardware security features, because pretty much the only Intel CPUs still in production that lack IOMMU capabilities are Atom parts which aren't socketed and probably aren't used in any Thunderbolt-capable machines.
> Wikipedia seems to state no OS has managed to do so (as of 2019). Thunderbolt had been around since 2011.
You should read more carefully, and preferably follow the link to the source paper. That paper points out how Mac OS X started using the IOMMU to protect against DMA attacks in 2012, Linux and FreeBSD likewise have such capabilities, Windows pretty much doesn't (no longer true).
The point of that paper cited on Wikipedia was to show that when operating systems poke holes in the IOMMU protection to allow certain devices to perform DMA, it's possible to open up a more exploitable hole than you might expect. In particular, data structures may need to be rearranged to align with the 4kB page protection granularity, or else information that should remain inaccessible to peripherals may be exposed alongside legitimate DMA payload. And it's also possible for a malicious device to spoof the device IDs of a trusted device type and be granted access by default, but that's a class of vulnerability that exists for plain old USB, too.
So as I originally stated, the IOMMU is the hardware feature needed to block DMA attacks. But it's only effective when used properly by the OS. That's not trivial—but also not impossible.
That doesn't seem obvious to me. Is there evidence it wasn't just binning?
Features like HyperThreading or IOMMU are integral to the design of functional units that aren't optional. Those features aren't implemented as separate physical regions of silicon that take up non-trivial space, so it's very unlikely that a defect could break HyperThreading without breaking the entire CPU core, or that a defect could break IOMMU functionality without breaking the entire PCIe root complex. Disabling features like this does not meaningfully change the fraction of dies that pass quality control.
The IOMMU in particular is probably not going to be affected by overclocking, because it's probably in the same clock domain as the rest of the PCIe IO stuff, and that clock domain doesn't usually get overclocked when the CPU or DRAM is overclocked. So there's no reason to expect IOMMU functionality to affect what CPU clock speed a given die can reach.
In a most rudimentary implementation, a device doing DMA is physically wired to the main memory in parallel with CPU. The device will set something like `data, addr, CS, WR = 0xCAFE, 0x1000, TRUE, TRUE` and send a clock pulse. Then when CPU comes back to check address `0x1000`, the data `0xCAFE` is magically “there”, so programs on CPU don’t have to waste cycles polling the device to get the value. Naturally, some arbitration is necessary because trying to change bus states from both sides will mess things up.
Above was probably the case until 1980s or 90s, and DMA in modern days instead goes through bus masters and memory controllers. Since it’s not going directly to physical RAM anymore, such controllers can be configured in such ways as to prevent unwanted accesses.
But it’s more of “can be used to protect against”, not “won’t be possible if done correctly”.
Go look up what IOMMU means. We're not talking about a feature inside an individual CPU core, but in the PCIe Root Complex that implements DMA capabilities in the first place; it's the thing that sits between peripheral devices and the DRAM controller. That functionality is on the CPU die these days.
That's an interesting domain. What are you working on? Any reason for choosing Realsense over Azure Kinect or other sensors?
Firstly, RealSense cameras are really affordable. I think the cameras we were using were around $80 a piece. Cost was an important consideration for us because we were building a multi camera device and to do something similar with Azure Kinect would have cost us several thousand dollars.
Another thing we like was how great the Intel RealSense SDK is. They have official wrappers for multiple languages along with sample code to get you started.
The RealSense ecosystem is quite healthy too – at least for a niche technology. RealSense cameras have a proven track record in various commercial devices and the development forums are fairly active. We found we could find answers to most common questions and problems we had.
Another nice thing about RealSense is that they have a great range of cameras so they have something perfectly suited for every application. For example, some of their cameras use stereo vision while others use LIDAR or structured light. They also use the same SDK for every camera which makes swapping out cameras fairly easy. We were using structured light cameras, but we were able to swap them out with Stereo and LIDAR cameras with no problems at all.
Azure Kinect looks interesting. I've wanted to play around with one of a while, although given its price and feature set it wouldn't have been the right camera for us. It's probably a good all-rounder for general robotics / vision projects though.
As a SW engineer my interpretation has always been that USB3 speeds are just too high to work really reliably with consumer grade hardware. This article gives technical details that tell me I have been correct.