News for hackers. Stuff that matters.
Re:Musk explains why SpaceX prefers clusters of small engines by mhandley on Thursday February 08, 2018 @11:46AM (#16333443)
There are two other reasons which SpaceX has mentioned before:

- When you're landing a rocket, you need to be able to throttle down quite low. Even a single Merlin 1-D engine, throttled down, is too much thrust to be able to hover with a nearly empty booster. It's really hard to get stable combustion at very low throttle settings. Having only one engine out of nine running for landing makes this much more manageable.

- There are economies of scale and reliability when you're building large numbers of something. Ariane 5 only launched 6 times in 2017. It uses one first stage engine, so they're only building one engine every 2 months. Falcon 9 launched 18 times in 2017, with 9 first stage engines per launch, that's roughly an engine every 2 days. More continuous construction, better economies of scale, more repeatable.

Re:Musk explains why SpaceX prefers clusters of small engines by vannevar on Thursday February 08, 2018 @12:07PM (#16333612)
There are economies of scale and reliability when you're building large numbers of something.

Yes. One of the biggest predictors of reliability in aerospace systems in general is time in operation---the longer you've actually run something, the closer it approaches the upper limit of reliability for that component. Running 27 small copies gives you 27x the time in operation vs a single big-ass engine.

The downside is complexity, and in particular unknown failure interactions which could bring down the entire system in a cascade. As long as many of the components are identical, though, an improvement in reliability of that component should translate directly into lower probability of failure interactions since there are fewer failures, period.

Re:Musk explains why SpaceX prefers clusters of small engines by sehugg on Thursday February 08, 2018 @01:30PM (#16334351)
Yes, one of the poorly-understood problems with early NASA missions was "pogo", a propellant vibration problem which affects the entire vehicle: https://www.popsci.com/how-little-vibrations-break-big-rocke...
Re:Musk explains why SpaceX prefers clusters of small engines by bunderbunder on Thursday February 08, 2018 @01:10PM (#16334178)
How does one quantify the risk of cascading failure?

My initial instinct, informed by computing, is to say that it's easier to avoid cascading failures in the system that is composed of more, smaller parts. All other things being equal, in a rocket with 5 engines, if one of them fails, then each of the remaining ones needs to pick up 1/4 of the slack to compensate. In a rocket with 9 engines, not only would each of the remaining engines need to pick up only 1/8 of the slack, but the total size of the gap would be a little over half as much, too.

(This is obvs ignoring the possibility that the failure in question is catastrophic.)

Re:Musk explains why SpaceX prefers clusters of small engines by neltnerb on Thursday February 08, 2018 @01:50PM (#16334517)
To answer the first question, you use a fault tree analysis to predict potential failure starting points (like a broken component) and then describe how those failures will propagate through the system.

https://en.wikipedia.org/wiki/Fault_tree_analysis

For an example, say I'm building a system that needs to hold a block of aluminum at 550C, 99% of the time. Okay, so you add a thermocouple and a heater to it, easy.

What if the thermocouple fails?

Well, if the thermocouple fails open then the temperature will read infinity and the heater will shut down and probably produce a non-catastrophic failure.

If the thermocouple fails closed, the temperature will read room temp and the heater will blast full on until the aluminum melts at 660C, which is a catastrophic failure.

If the relay in the temperature controller fails, the furnace probably turns off but theoretically could fail on if the relay switch gets fused.

Okay, so I can see that there is an unlikely but possible chain of events that could cause a catastrophic failure. So I add a second thermocouple to act as a safety shutoff using a second redundant relay and controller if it reads a temperature above 600C.

Total probability then is estimated by either using real world performance metrics or best-guesses. I'd say the odds of a thermocouple failing in 10 years of operation at 550C is nearly 100%, so this failure will nearly certainly occur.

Or consider an LED array with 10 of them in parallel. If one blows open, the remaining 9 each get 10% more current so are more likely to fail. So your first branch of the tree might be that the odds are 10% that an LED will fail at design current within five years. That may well not qualify as a failure, especially since the other 9 LEDs are ~10% brighter due to the higher-than-spec current. But now your probability for the next failure is 20% within five years. So you do need to define different outcomes, usually by severity of impact and probability of outcome in event of a predicted possible failure point.

Re:Musk explains why SpaceX prefers clusters of small engines by ethbro on Thursday February 08, 2018 @03:12PM (#16335196)
Is there a name for (or keywords to search for) weighing the tradeoffs between attempting to reduce failure effects in a component itself vs addressing them at the system level?

E.g. while you could mechanically debounce a button itself, it's usually easier to engineer the system in such a way that trigger bounce doesn't cause issues

Wondering how the call is made on where the appropriate fix to increase reliability should be made? Or is all bespoke / gut?

Re:Musk explains why SpaceX prefers clusters of small engines by neltnerb on Thursday February 08, 2018 @05:00PM (#16335982)
So the proper way to think about this is in terms of where you put abstractions. Much like you'd write a function or library, you can abstract physical machines by idealizing the component in a system.

I don't think it's really a separate keyword to search for. This is all probabilities.

The math isn't complex, the hard part is writing down a complete graph of all the connections between different components, environments, and failure scenarios. If your valve is made of five parts, and one of them has a 10% chance of failing per year, then your valve has a 10% chance of failing per year. If it has two parts that have a 10% chance of failing in a year, then assuming independent failures the total probability of failure of that component is 19% in a year.

These numbers are rarely known with so much precision during initial design. Consider it akin to estimating the probability of certain kinds of predictable bugs in a library you're using. How much do you trust that github repository vs intel? The most robust thing to do is typically to design around your best guesses but then do validation testing to refine your guesses.

So if I think a critical valve or seal has a high probability of failure but have low confidence in what the probability is, I'll take that valve or seal and literally set up a test case to make sure it performs as expected. Then I can collect real statistics and go from there. Data >> Guesses, but the systems are so complex that guesses are where you have to start.

Then you'd basically put the system together, one part at a time, and validate with each added part that the entire system still behaves as expected. And you throw in some edge cases to ensure that controls are working properly, like perhaps in the aluminum heater case you'd simply break a thermocouple yourself to ensure that the safety system works. But you'd do that in testing, not in production.

It's really very analogous to unit tests, unsurprisingly because the need is similar. I've had vendors ship me special custom thermocouples that they claimed would run for 10 years at 600C. We threw them in an oven as a trivial validation test. They caught fire. We didn't use that vendor again. But by analogy that's how the firmware blob you get from a vendor is too. They sure claim it does something, but until you've done real testing with it who knows?

As you pin down the true probabilities of different failures, you just propagate them through your graph of possible failures to estimate the probability of different scenarios and focus on the high risk and high likelihood events. Sometimes the risk is as simple as "the system will be down for an hour while we replace a failed component". No biggie, maintenance is an expected cost. Sometimes the risk is a nuclear plant meltdown.

EDIT: The goal of the above is to identify which causes result in critical failures with high likelihood. Once you've identified them, then you focus down on addressing the root cause. It's more about identifying where problems would start if there were a bad scenario, so you know where to spend more attention in quality control.

If you identify that the debouncing is a cause where if it doesn't work your machine doesn't work as needed, the actual solutions could be software or hardware. What's the relative probability that each solution will work? How costly is a failure? How much does it cost to implement? At that point you're talking cost models with reliability requirements as an input.

Re:Musk explains why SpaceX prefers clusters of small engines by thisacctforreal on Friday February 09, 2018 @04:05AM (#16338876)
These comments are incredibly helpful, thank you for taking your time to write them.

Please correct me if I'm wrong, but I think is named Reliabilty Engineering / Safety Engineering? Those might be some good things to search for people interested.

Re:Musk explains why SpaceX prefers clusters of small engines by physcab on Thursday February 08, 2018 @01:27PM (#16334318)
The difference between aerospace and computing is that aerospace is heavily materials dependent. Getting a bad pixel in a monitor is rare but when it happens it’s not catastrophic. When you’re producing a ton of parts you have to also maintain quality. Not doing so will introduce impurities which will alter the material properties. An alloy with different material properties could potentially burn, fracture, expand etc more differently than represented in their acceptable failure models.
Re:Musk explains why SpaceX prefers clusters of small engines by avmich on Thursday February 08, 2018 @12:16PM (#16333707)
> When you're landing a rocket, you need to be able to throttle down quite low.

When R-7 was developed in 1950-s, as a weapon, one of requirements was the precision of attained speed at the moment of engine cut-off. It was solved by shutting down main chambers on the upper (second) stage, powered by RD-108 - 4 chambers about 20 tons of thrust each - and maintaining thrust from steering chamber, 4 of them, 3 tons of thrust each.

This is another example of deep throttling.

Re:Musk explains why SpaceX prefers clusters of small engines by Gravityloss on Thursday February 08, 2018 @03:24PM (#16335287)
That'd be quite something, improving the Soyuz to do powered landings. Talk about getting mileage from your design.

On the other hand, it can't even stand on the pad fully fueled, it's hanging from an arm...

Re:Musk explains why SpaceX prefers clusters of small engines by pjscott on Thursday February 08, 2018 @01:56PM (#16334561)
Another reason is that, because the Merlin 1-D is small enough, they can use a vacuum variant of it as the upper stage engine rather than designing another one.
Re:Musk explains why SpaceX prefers clusters of small engines by marktangotango on Thursday February 08, 2018 @12:58PM (#16334072)
The landing case makes me wonder about the wisdom of Blue Origins choice of going with a larger engine, the BE-4 with 550,000 lbf for the New Glen vehicle. Much more thrust than a Falcon 9 landing on a single M1D. Surely they've run the numbers and find it viable though.
Re:Musk explains why SpaceX prefers clusters of small engines by Robotbeat on Thursday February 08, 2018 @01:46PM (#16334487)
Blue Origin also has worse mass fraction on the booster stage than SpaceX does, which means they don't need to throttle down as much. If you look at New Shepard as a preview to New Glenn, there are a whole bunch of aerosurfaces which add dry mass, essentially ballast. New Glenn will add big side fins, which will help increase lift, drag, and provide more ballast.

Also, the New Glenn booster will have 7 engines. Not so different from Falcon 9's 9 engines per booster.

Overall, New Glenn is a good design (and will eventually have a reusable 2nd stage). If SpaceX stopped with Falcon 9 and Falcon Heavy, New Glenn could easily give SpaceX a run for their money. Luckily, SpaceX won't stop with Falcon Heavy. BFR, once they finish getting Block 5 out the door in a couple months and crew Dragon flown sometime by around the end of the year, will be almost their sole engineering/development effort (Starlink being kind of a separate division).

Of course, Blue Origin also won't sit still with New Glenn. They'll make the upper stage reusable, then also add a hydrogen/oxygen kick stage for very high energy payloads, and then they'll be working on the New Armstrong monster.

SpaceX is super far ahead, but luckily Bezos is so rich that I expect both companies now to deliver on their equally grand visions.

Re:Musk explains why SpaceX prefers clusters of small engines by marktangotango on Thursday February 08, 2018 @07:40PM (#16336930)
The thing about these reusable boosters that’s surprising to me when I think about is; they really don’t need very many of them. Like 10 or 50?
Re:Musk explains why SpaceX prefers clusters of small engines by Robotbeat on Friday February 09, 2018 @08:48AM (#16340107)
Bingo. That allows them to switch over to BFR much faster and for much less money than an expendable launch provider.
Re:Musk explains why SpaceX prefers clusters of small engines by ygra on Thursday February 08, 2018 @01:18PM (#16334242)
Thrust isn't that much of an issue. Yes, Falcon 9 cannot hover. But they don't want it to anyway, as it's too wasteful on fuel.
Re:Musk explains why SpaceX prefers clusters of small engines by philipwhiuk on Thursday February 08, 2018 @06:18PM (#16336440)
It's slightly ironic that SpaceX has brought economies of scale to the space industry and Telsa is struggling with scale in the automotive industry.
Re:Musk explains why SpaceX prefers clusters of small engines by greglindahl on Thursday February 08, 2018 @07:35PM (#16336905)
Yes and no -- SpaceX has the greatest scale in the space industry, and Tesla has small scale in the car industry. That's mainly a statement that cars are a huge industry, and rocket launchers is a small one.