That sounds logical, but isn't really a thing in aerospace/space. Complicated high-energy systems have thousands of failure points. So to have any chance of success each failure point needs to be engineered below, by way of example, a 0.0001% chance of failure. That costs lots of money. But say one decides to accept more risk for less cost. Ok. So you switch from 0.0001 to 0.001 failure rates. You risk is now 10-fold higher at each failure point, but with thousands of failure points adding up you are now essentially doomed. And you haven't saved anything. The cost of 0.001 components isn't fundamentally different than the 0.0001 components were. SpaceX can save money though different business practices, by trimming people/money/contracts/compliance and such, but if you look at their rockets they are not fundamentally any less-perfect than anyone else's. They cannot afford to be. This is why rocket failures, like aircraft failures, are taken so seriously. There is an extremely fine line between "works ever time" and "never worked twice" with very little money to be saved between the two.
Across many areas, a risk-v-cost math never really happens. It is either go or no go. Take CPU production. Intel spends billions at each of hundreds of fabrication step to push down miniscule error rates because any of a million errors can destroy a chip. There is no money to be saved by allowing any one process to become less than as perfect as it can possibly be. The marginal savings of a less-than-perfect process isn't worth the exponential increase in the risk of total system failure.
And you can look all the way back to their little hopper version of the Falcon 9 and see that this strategy has been the key to them undercutting the launch market significantly.
My prediction: SpaceX will have a 5th 100% successful launch of Starship before the SLS has a 5th successful launch. They'll just have ten not-100%-success launches before then.
Really? I make is 189 successful F9 launches since the last issue in 2018 (there's a couple of landing failures, but given that everyone else apart from the space shuttle has a 100% loss...)
If you look at the "finished product" of block 5 that makes 162 launches and zero failures.
That's reliability far beyond any other launch system, including the space shuttle and Ariane 5 which are the only ones to come close in numbers of launches. Ariane 5 is certainly a reliable system as far as spaceflight goes, but it flies 3 times a year, Falcon 9 flies 3 times a month
These systems all are quite good, and they have tended to get better over time.
Landing the shuttle is like landing the Dragon Crew capsule. SpaceX landed the booster, where shuttles ditch theirs into the ocean.
So even shuttles have a pretty high loss rate :)
They got great publicity from it, but landing vertically is a major compromise.
Falcon 9 has the record for most consecutive successful orbital launches. Their last failure was AMOS-6 in September of 2016. Since then they've had 189 successful launches in a row.[1] In that same time Soyuz has had 113 launches with 3 failures. Soyuz's longest success streak was 100 launches from 1983 to 1986.[2] The US's Delta II had 100 consecutive successes from 1997 to 2018, though it has since been retired. A total of 155 Delta IIs were launched with 2 failures.
Falcon 9's current successful landing streak of 110 missions exceeds the competition's best launch streak. By any metric one can measure, SpaceX has the most reliable rocket.
1. See https://en.wikipedia.org/wiki/List_of_Falcon_9_and_Falcon_He... and https://en.wikipedia.org/wiki/List_of_Falcon_9_and_Falcon_He... for the list of launches and outcomes.
2. https://arstechnica.com/science/2022/02/spacexs-falcon-9-roc...
Current vehicles are vastly different from the originals. What we’re trying to do is predict the probability of the next launch failing. Equally weighting far historicals and recents is bogus statistics.
I thought we were comparing methods. Unless the next payload is yours then the odds of the next launch failing is meaningless to most of us, but we can learn something from the methods used.
But sure, if you have a bet in Vegas or something then feel free try and calculate things as closely as possible. Just understand that several of Soyuz failures didn’t kill the crew so there’s other metrics people might care about.
What does this mean? The question most of us care about is which method resulted in a more reliable rocket. And SpaceX’s track record shines uniquely in that respect. The frequency, moreover, makes the results robust. Legacy rockets like Ariadne will never be able to reach that confidence level, because the likelihood of fluke successes won’t have been minimised by the time the rocket is retired.
So sure, we can reasonably assume that Starship will get to a state of reliability similar to current Falcon rocket, eventually. We can’t assume the first few commercial Starship launches are going to even approach that level of reliability. And in fact the best point of comparison may be the early days of Falcon 9.
This was a partially successful test nothing more and nothing less. I get people really really think SpaceX had done an excellent job and I don’t disagree but people who are comparing the end result of a long process Aka the current state of falcon 9 with a new system like Starship are going to be disappointed.
Starship is extremely likely to fail repeatedly before achieving anything close to the same streak as the Falcon 9 has. That’s not an issue with SpaceX that’s an inherent aspect of doing something really difficult.
There are other new space companies, but they're just not as good.
It does though. During the development process. SpaceX will do five launches with lower reliability vehicles before the big launch providers will even do a single launch, and those five will have be cheaper in total. Of course, they aren't putting essential cargo (or god forbid people) on these higher risk test flights. By the time they're doing that they have developed certainty in the design.
A number of Tesla-related deaths can be attributed to Autopilot malfunction. Real people - really dead.
Except Tesla, unlike SpaceX, is willing to put passengers in its test vehicles. The SpaceX approach would be to let a bunch of FSD Teslas crash into things and each other before giving them payloads.
Citation needed. Not Tesla's "stats" that if the people compiling them completed anything more than high school statistics are intensely misleading.
Comparing a subset of miles driven on the simplest and easiest roads (because the systems can't be used and are turned off) and comparing to accident stats across ALL roads is disingenuous to the extreme, and Tesla continues to tout it.
Short of pulling over, humans don't have the opportunity to say "let's disengage, because it's a bit challenging", and then not have to worry about "counting" any accidents from there forward.
SpaceX does not have a difference in intended reliability, or a difference in design reliability. (At least for the rocket as a whole. One could argue that 33 engines allows lower engine reliability through redundancy)
What they DO have is a significant difference in prototype reliability for live launch. This is clear when you look at their launch history.
I don't think that is accurate. There is a difference between 9%, 99%, and 99.999% confidence of success going into a launch.
You can almost always delay builds and launces to run more simulations, tests, and studies and increase confidence.
A simple example is SpaceX could have chosen to wait until they had a booster test with 100% engine ignition before moving on a full launch. Instead they choose move forward anyways without more stationary booster testing.
Why? If you're launching people I see why you want near-perfect, but if you're launching something with a low replacement cost (ex: getting fuel to orbit to support other projects) it seems to me that as volumes get large enough eventually "use lower quality and accept a slightly higher probability of failure" starts to be cost effective.
Your comments in this thread all go against the development approach of that rocket— Falcon9 is a stable platform now as it is used in production. In the development phase there were tons of explosions. This all happened in the past.
Is unarguably cheaper to forgo the additional testing and cost required for human flight.
But this is totally false. There are entirely different standards applied if you want to design an aircraft for commercial airline passenger transport vs for general aviation. There are entirely different requirements for instrumentation reliability if you're building a day-VFR aircraft vs one that is allowed to fly in instrument conditions. And there are entirely different requirements for aircraft that you sell to the public vs ones that you build yourself.
The aviation side is full of examples of exactly the sliding scale you're saying doesn't exist.
Of course there is. The famous example is radiation hardening. SpaceX opted for redundancy instead. Not only cheaper, but more modern, too.
> in such as complex high-energy environment, there is no money to be saved by building less-than-perfect machines
SpaceX has launched zero humans. It aims to, so target reliability is high. (I would argue their track record in production is a product of their willingness to push the envelope in tests.) But there is a large market for cheap, if unreliable, launches. Because there is an emerging market of cheap satellite makers.
It is the only US entity with the capability to do so.
Sorry, had a brain fart, no idea how I typed that out.
substitute rocket and spaceflight with airplane and flight and see how that sounds
Yes ultralights have some regulations, but even general aviation has less regulations resulting in cheaper aircraft, to say nothing of ultralights.
I saw in another comment that each SLS launch costs $2B. Do you think SpaceX spent $2B on this launch? I find it hard to believe.
Indeed, if we assume that the each launch is an IID binomial coin flip (which isn’t really the right way to evaluate right-censored data), and observe (by reading Wikipedia) that SpaceX has had at least 450 successful Falcon 9 launches since the last in-flight failure, then they have at least five nines of reliability:
0.999995^450 ~= 449/450.
Which appears to be an industry-leading stat.
For context (excluding the Columbia re-entry failure), the space shuttle only had 4 nines:
0.99994^125 ~= 124/125
I'm not the best way to frame this is "first one to a single successful launch". Reliability matters when you're dealing with high-risk scenarios, so a better measure is "probability of a given launch being successful".
I'm not asking as a 'gotcha' question, I'm acknowledging that the sample size matters. A lot of these statistics are bandied about without really elaborating on the full context of the way reliability is actually measured in industry.
Without checking for the most up-to-date numbers, I believe the F9 is slightly better than Soyuz on raw numbers (successful flights / total attempted). But Soyuz probably has around 6-7x more flights, meaning we have much more confidence that the Soyuz numbers accurately reflect reality.
A new vehicle will always be unproven. But one that's flown 5 times in testing (4 of them unsuccessfully) before its first flight will have had more flight time on most of its systems than something like the SLS where you take 10 years to think about what could go wrong and then do one test launch and call it good.
Having said that I do think propulsive landing for crewed vehicles is pretty scary. F9 firsts stage landings have been pretty reliable for a while, and they now have several boosters with 10 or more flights, so we’ll see. It’s not like capsule landings are 100% safe either.
There has been over 140 crewed launches. For context, you seem to be counting both test/demo, crewed, and uncrewed F9 launches. Again, I don't know the exact number off the top of my head, but there's probably ~10 crewed F9 launches, so it's an order magnitude difference using the same metric. It's gets better though, when comparing total launches.
Not that many options for landing something that size on Mars. That’s what’s really driving many of the design choices.
Lots of falcons failed, but you don't see anybody worried about their payload or crew on Falcon 9 these days.
I agree with the first part, but the second half extrapolates too far IMO. One successful launch tells us very little about operational reliability.
From an outcome perspective, it's hard to ever see the lower launch rate dominating from any perspective.
You don't have economies of manufacturing scale, because your assembly rate is so low it doesn't make sense not to treat each as exquisite.
You don't have rapid iteration on manufacturing improvements, because the tyranny of safety checks on manufacturing time balloons {time from fix to flight}, after a proposed fix is identified.
And most importantly, you leave yourselves extremely vulnerable to unknown-unknowns, that you can't imagine in the design phase.
For example, if NASA had been launching the shuttle more rapidly, with the bulk of those being uncrewed launches, they probably would have picked an uncrewed launch to test expanding the temperature bounds at Cape Canaveral, and Challenger would have exploded without a crew.
As was, NASA's shuttle launches were so rare that there wasn't acceptable launch rate and weren't low-impact launch opportunities to do so. So they tested it on a crewed mission with disastrous results.
Point being: they backed themselves into a low-volume/high-risk corner of their own strategic design
SpaceX's most brilliant achievement was using Starlink to artificially boost launch demand and give them a minimally-profitable/break-even place to sink higher-risk launches.
>You don't have rapid iteration on manufacturing improvements, because the tyranny of safety checks on manufacturing time balloons
Rapid iterations has obvious benefits. But there are also downsides because it makes it harder to arrive at a stable, reliable design, it introduces vendor issues, etc. Tesla is also known for iterating faster compared to their major competitors but it has resulted in logistics and reliability issues.
IMO SpaceX's most brilliant achievement was leveraging govt contracts to work out the kinks of their designs, which could then be leveraged at a lower risk for Starlink. In effect, they let the public take the burden of the risk (because the govt is really the only entity capable of shouldering that size of a risk for an unproven quantity) and then transitioned to a private means of revenue in Starlink. (I'm not saying that as a slight btw, I think it's mutually beneficial).
SpaceX's strategy for the Falcon 9 worked and it's one of the most reliable rockets in the world, flying the most often.
You're focused on the wrong takeaway. I used Tesla because it's easy to see how one culture translates to the other. Insert any company that uses rapid iteration in place of Tesla if you prefer.
The point is that there are certain circumstances where rapid iteration is useful and others less so. When reliability matters, rapid iteration may be working against you. (It's a continuum, of course, so the real question is where is that tipping point)
>SpaceX's strategy for the Falcon 9 worked
The point I'm driving at is there is a distinction to be made when finding out why certain iterations didn't work vs. just changing the design without fully understanding the failure mechanisms of the first. One leads to a greater understanding than the other. It's a difference in an engineers mindset and a scientists mindset.
You are driving at a point by pretending there's some important difference because "in this industry".
It's the same industry. Building the same type of product. By the same company.
Now go look at the history of Shuttle for the equivalent number of launches at that risk level. Would you claim they are equivalent in terms of human-rated safety?
If not, it’s only because you have the benefit of knowing the long-tail probabilities with the Shuttle.
True, few of those F9 missions were crewed, but that's the point. There's no difference between a crewed and a noncrewed F9 launch vehicle, so there's no reason to think the presence of humans would change the risk. So, you get to accumulate most of that reliability data without putting people at risk doing so.
Because the nature of the two programs was fundamentally different. The Shuttle was a product contract, while CCP is a service contract. On the former, the govt has much more control, and will detail more rigorous acceptance criteria. This generally gives a much higher pedigree on quality control. On the latter, they take a much more hands off approach and have limited insight.
As an analogy, imagine you are making a big bet on acquiring a software company. One company gives you their source code, shows you all their most recent static analysis, unit test results, to interview their programmers etc. The other company allows you none of that, but gives you a chance to play around on their website to see for yourself. Both systems seem to work when you try the end product, but which do you have higher confidence in?
At the end of the day, "reliability" is just a measure of how much confidence we have that a product will do what we ask of it, when we ask.
I'm not saying you can get by without thinking, but it's difficult for humans to estimate the reliability of a complicated system. Reality, though, has no problem doing it.
Plus, I think your analogy is flawed. NASA surely has a more hands-off approach on the CCP than on the Shuttle, but to say it's hands-off is misleading. They do have a lot of access.
Take the example of the F9 strut failure. They could have tested the material outside of the final test configuration and saved themselves a lot of trouble. They chose to forego that testing, and instead 'tested ' it as part of part of their launch configuration. (I put it in quotes because it's not clear to me that this was a conscious testing decision).
There’s also a difference between “we’re not completely sure of the fundamental principles, but our testing indicates it works” and “our testing indicates it works and we have a solid understanding why”. The latter allows you to know the limits of your application much more readily. The risk in the former is that you don’t know what you don’t know, so you can never be wholly sure if you’re good or lucky. And luck can be fickle. And this is also where rapid iteration can lead to issues: the more you change, the less sure you can be about whether your results are attributable to luck.
>Plus, I think your analogy is flawed. NASA surely has a more hands-off approach on the CCP than on the Shuttle, but to say it's hands-off is misleading. They do have a lot of access.
They have many engineers who want more access and are effectively told to back off because it's not their place in this type of contract. So I'm not saying they have no access, I'm saying they have very limited access by comparison. It would have been better if I made the analogy that they get the results to a small select number of tests, but not all.
Because they knew they had to "get it right" the first time because a bunch of buffoons(congress) would consider any crash or explosion as a failure and pull funds immediately.
That’s not the way the CCP contracting works.
It's the X-origin vs Slope issue - steeper slope always wins.
The problem is maintaining that aggressive problem-seeking culture after long periods of success
As an example, they had issues with failures related to their COPVs rupturing. On the one hand, they addressed the problem by redesigning their system. On the other, they never really investigated fully why the COPVs were failing in the first place. Instead, NASA decided to fund that investigation on their own. One possible consequence is that their redesign didn't fully address the risk because they never fully investigated the root cause.
I don’t think SpaceX has smarter engineers. I don’t think they even have smarter managers, since many of their executives used to work for NASA and/or traditional aerospace firms (e.g. President+COO Gwynne Shotwell started out in aerospace at a research centre doing contract engineering work for the NASA Space Shuttle and the US military space program)
One big difference is Elon Musk started out by telling his executives to take big risks (as in “if this fails we go bankrupt”). I think Tory Bruno at ULA is a great CEO, but no way is Boeing or Lockheed-Martin ever saying to him “we want you to take such big risks that we might go bankrupt if they fail”. He, and all the people under him, are only allowed to take small-to-medium sized risks, so that’s all the risks they are allowed to take. But that puts a definite limit on what they can achieve compared to SpaceX whose executives and engineers have the freedom to make much riskier decisions
[0] https://upload.wikimedia.org/wikipedia/commons/b/b2/Survivor... :)
The Soviets proved that methodology works and SpaceX continues it to this day. I have a feeling most that follow SpaceX think they are trying a new revolutionary approach here.
They were working with 1960s material science.
They were working with 1960s understanding of rocketry.
Korolev died.
We've come a long way in the last 50+ years, and sometimes you can use materials with substantially higher limits to paper over design risks.
>Adverse characteristics of the large cluster of thirty engines and its complex fuel and oxidizer feeder systems were not revealed earlier in development because static test firings had not been conducted.[9]
They planned and expensed the N1 as if it was another LEO Soyuz variant.
N1 might have simply been before its time.
After they get to orbit they can start putting payloads on board, and the tests will pay for themselves. I’d say the risk of bankruptcy is very low.
Edit: Just saw photos of the crater under stage zero. Looks like they do need a flame diverter lol.
It's totally feasible for them to keep testing and keep refilling that hole while in parallel they build a perfect launch pad with a flame diverter somewhere else. That way they don't have a gap in testing cadence.
Oh yes: https://en.wikipedia.org/wiki/Unapproved_aircraft_part
For example:
The crash of Partnair Flight 394 in 1989 resulted from the installation of counterfeit aircraft parts. Counterfeit bolts, attaching the vertical stabilizer of a Convair CV-580 to the fuselage, wore down excessively, allowing the tail to vibrate to the extent that it eventually broke off.
This happens to aircraft, and the only way to ensure it doesn't happen to rockets is to have extreme control of your supply chain.
Sometimes in aerospace you need to collect data, and sometimes that means explosions.
[0] https://www.space.com/spacex-starship-first-space-launch#:~:....
Triple redundancy is not a thing in general aviation while, for some systems, it is in commercial. That's a risk vs cost calculation.
Semiconductor manufacturers do risk vs cost calculations through the entire development and manufacturing process. Source: I've worked in semiconductor, doing those calculations.
Your authoritative tone doesn't make any sense when Falcon 9 exists and has been actively dominating the entire industry for half a decade.
It seems like a strawman to suggest that you would let failure probability increase by a factor of ten as the first step.
>but with thousands of failure points adding up you are now essentially doomed.
And it was an easy one to tear down.
>And you haven't saved anything. The cost of 0.001 components isn't fundamentally different than the 0.0001 components were.
There is no reason to believe that at all. Invert the question: if a system is 99.999% reliable, does that mean it should be free to make it more reliable? And why?
>It is either go or no go. Take CPU production. Intel spends billions at each of hundreds of fabrication step to push down miniscule error rates because any of a million errors can destroy a chip. There is no money to be saved by allowing any one process to become less than as perfect as it can possibly be.
This is completely wrong. Chip factories do produce chips with defects; those chips are sold for cheaper, since they still work, but not as fast. IIRC most of TSMC's modern processes are designed to sell the good chips at a high price and the bad ones cheaper.
If the failure points are arranged in parallel - X OR Y OR Z must happen to have a successful outcome, with multiple redundant paths to success, your total failure rate is the chance that ALL of X, Y, and Z fail. This is a much lower number than when they are in series.
To use concrete math - say that Starship has 33 raptor engines with a failure rate of 0.01%, 3 grid fins with a failure rate of 0.05%, and a fuel tank with a failure rate of 0.001%. If it's engineered so that all 33 raptor engines, all 3 grid fins, and the fuel tank all need to work for a successful launch, the success rate of the whole system = 0.9999^33 * 0.9995^3 * 0.99999 = 0.9952 = ~0.5% chance of failure. If it's engineered so that it can get to orbit on 28 out of the 33 raptor engines, 2 out of the 3 grid fins, and there is a double-hull to the fuel tank with a failure rate of 0.005%, then the chance of failure for each subsystem is 0.0001^5 = 10^-20, 0.0005^2 = 2.5 * 10^-7, and 0.00001 * 0.00005 = 5 * 10^-10, and when you multiply out those subsystem failure rates you get 1 - (1 - 10^-20) * (1 - 2.510^-7) * (1 - 510^-10) = 0.9999997495 = ~0.000025% chance of failure.
Moreover, lets look what happens if you take the multiply-redundant design above and then increase the chance of failure of each component 100x. Raptor engines are now 99% reliable, grid fins are now 95% reliable, and fuel tanks are now 99.9% reliable. The overall failure rate for each subsystem becomes 0.01^5 = 10^-10, 0.05^2 = 2.5 * 10^-3 and 0.001 * 0.005 = 0.000005. When you multiply out those subsystem failure rates you get 1 - (1 - 10^-10) * (1 - 2.510^-3) * (1 - 510^-6) = 0.00250498759 = ~0.2% chance of failure. The multiply-redundant system, even with component failure rates 100x higher, still has better reliability than the perfectly-engineered system where every component must perform exactly to spec.
This principle is used all the time in practical engineering. It's why Google builds server farms out of thousands of commodity PCs, hooked up in primary/replica clusters with replication and transparent failover. It's why ships have watertight compartments and double-hulls. It's why passenger jets have multiple engines, multiple hydraulic control systems, and multiple flight computers. Any engineer worth their salt is going to avoid SPOFs and assume that components will fail, then build redundancies into the design so that a partial failure does not endanger overall mission success.
exactly. optimizing for an acceptable risk for space launches, esp space travel, seems still to be leading to $2B launches. at which point, what's the difference?
Our ex ante ability to estimate these probabilities is lacking. Especially when they’re coupled. And redundancy, instead of tighter tolerances, is often cheaper than the classic approach. These are proven lessons from SpaceX.
So SLS does a LOT of analysis and manages to find and rectify failure points prior to flight. They pay a lot in analysis and inflexible design to do this.
SpaceX does some analysis, but then flies to confirm the analysis early. That way they identify actual failure points and can use sensors to see how close they got to failure.
For instance today I'm sure they learnt a lot from each failed engine, parts on the booster that stopped working, detailed telemetry on ship behavior on flight and sensors for the non-separation of booster from ship.
SpaceX builds to the same high tolerances as other rocket manufacturers, but they don't try to avoid testing through overly rigorous analysis. They also don't gold-plate their manufacturing, instead making tradeoffs to allow cheaper volume production with recovery instead.
These sorts of statements are infuriating. You're dropping all of the context of past achievements (budget, political will, design goals, need, etc) and then castigating modern engineers.
The SR-71, X-15, and Saturn V achieved their design goals. There's no modern need to go back and re-achieve those goals.
The SR-71 is an amazing plane. When it was originally built there was a need for a reconnaissance plane that could fly high enough and fast enough to evade Soviet high altitude air defense systems. In the intervening years reconnaissance satellites have obviated that need. Outside of a few other tasks there's not much need for a Mach 3+ manned aircraft. Even for the tasks where one is useful the SR-71 still exists. That's not to say a replacement couldn't be built today. In fact many believe one exists in the form of the rumored Aurora.
It's just ridiculous to make assertions about "no one" being able to accomplish those projects today. If NASA or the DOD put out an RFP and had a budget for a modern X-15 it could definitely be built.
> Unusually, the ejection system was tested with live bears and chimpanzees; it was qualified for use during 1963 and a bear became the first living being to survive a supersonic ejection.
If you're talking about the launch escape tower, that's for during launch. It's gone by the time you reach orbit, leaving you in the same scenario of Lunar Starship in the event of a failure; in space, but no way to get down.
There is a section about Space Shuttle's crew bailout at https://en.wikipedia.org/wiki/Space_Shuttle_abort_modes
Before the "Ejection escape systems" there are a few paragraphs about "inflight crew escape system"
> The vehicle was put in a stable glide on autopilot, the hatch was blown, and the crew slid out a pole to clear the orbiter's left wing. They would then parachute to earth or the sea
But 100 people is a different matter IMHO.
Ejections seats and capsule were not pursued, the Wikipedia page explains the reasons.
Airliners don't have an in-flight escape system, anyways.
Right now we don’t even have that many astronauts.
The booster presumably like you will have been thrown into the ocean. Very wasteful system.
From https://en.wikipedia.org/wiki/Space_Shuttle_abort_modes#Tran...
"Preparations of TAL sites took four to five days and began one week before launch, with the majority of personnel from NASA, the Department of Defense and contractors arriving 48 hours before launch. Additionally, two C-130 aircraft from the space flight support office from the adjacent Patrick Space Force Base (then known as Patrick Air Force Base) would deliver eight crew members, nine pararescuers, two flight surgeons, a nurse and medical technician, and 2,500 pounds (1,100 kg) of medical equipment to Zaragoza, Istres, or both. One or more C-21S or C-12S aircraft would also be deployed to provide weather reconnaissance in the event of an abort with a TALCOM, or astronaut flight controller aboard for communications with the shuttle pilot and commander."
If you look through all the failures and close calls in aerospace they are often rooted in human psychological errors. The pressures that lead to them may change with different organizations, but they don't go away.
Keeping it brief SpaceX/engineer is genuinely trying to get people to Mars, largely driven by ideological reasons with extensive technical creativity/competence backing them up. Accountant/Boeing wants to make more money. Outsource our software development to guys in India bidding $9/hour? Awesome! That's another 0.037% profit, what could go wrong!? Something doesn't work? Who cares!? We're on a cost+ contract baby, what you call "failure to deliver", I call delivering value to my shareholders!
And then there's the politician. In this particular case, he's not only a life long politician but also 80 years old on top. The only 'bright side' is that, due to his political influence, he's gone to space before. On the other hand Charles Bolden was a genuine astronaut and absolutely everything one would think they would want from a NASA head, yet he was a miserable failure. It may simply be that political style leadership (even when not a politician) isn't really conducive to meaningful progress in modern times.
I'd argue the incentive wasn't there until CCP. That was the fundamental difference in the last 20 years. Without CCP, I don't think SpaceX would be successful, either. But I will say they've done much more than the Boeing at executing on that incentive.
I do think you may be overly cynical in your characterization, though. It wasn't too many years ago that Boeing was listed as the most desirable company to work for by college students. The reason isn't that they thought it was because they couldn't wait to gouge the public coffers, it was because aerospace has always been considered a sexy engineering discipline. You'll almost never find a civil engineering firm on those lists because "roads and commodes" just aren't considered cool.
Back to the question, I'll weigh in with my perspective. They are all responding to incentives, albeit different ones. But we have to acknowledge the downsides of each. SpaceX is awesome, but they aren't without their own psychological pressures and biases. I've brought it up elsewhere in this thread, but they have wanted to rapidly iterate rather than fundamentally understand some of their design issues. I suspect this is partly cultural (where operational tempo matters more than scientific rigor...i.e., "we don't need to know why it works, as long as it works") and some of it is business (i.e., they have specific contractual deadlines to consider). Those are also some of the issues that lead to mishaps dating back to Apollo and Shuttle.
Considering you seem to think the legacy downsides are due to the business/shareholder side, do you see the same issues encroaching on SpaceX if they go public?
The point I'd make is that leadership really matters. Boeing, at its peak, almost certainly had orders of magnitude more talent than SpaceX did in its early years - in no small part because of what you mentioned. And they absolutely had many orders of magnitude more money and access to funding. But their leadership was just absolutely abysmal, and consequently the potential of that talent was left completely untapped.
But on the other hand, like you mentioned there are incentive problems. Even though Boeing failed to tap into their potential, their stock price has been constantly and steadily going up for decades. Even their planes literally falling out of the skies was but a brief stumble, the damage there largely repaired owing to the start of a profitable new war. So in this regard I doubt their leadership is particularly disappointed with their results. They achieved what they set out to do after all. And that's pretty disappointing.
I wholeheartedly agree. You're probably aware, but there have been a number of good write-ups detailing an overall erosion of engineering leadership at Boeing [1].
>Elon has stated that he will not be taking SpaceX public until transit between Earth and Mars is well established.
I think maybe the difference between you and I is that I take many of Musk's promises with a boulder of salt. If I was a gambler, I'd bet that we see some wordsmithing about what the definition of what a "well established Mars transit" means when it comes time for an IPO.
[1] https://www.theatlantic.com/ideas/archive/2019/11/how-boeing...
You say this with seeming knowledge of the thoroughness of SpaceX's failure investigations. Care to elaborate how you can say this with such authority?
NASA is about making doing right the first time though careful design.
For SpaceX approach to work, they need to test a lot, with a lot of explosions, so they need to make a lot of rockets, they also need repeatability. That's why they are using assembly lines and mass production techniques.
Which approach is the best, I don't know. The traditionally designed Delta IV never failed, but on the other hand, we lost two Space Shuttles and their crews. As for the Falcon 9, it had a couple of failed missions, but it has proven very reliable over time.
> On the other hand, the SLS — having a launch escape system and not requiring sophisticated acrobatics
> in order to reach land safely — is totally the ship I'd rather be on.
That said, I do agree about the flip maneuver. I am very interested to see how that evolves.
I would think weight of fuel required to land booster + legs + steering fins would be much greater than a parachute or two and a water landing. That weight savings would be a cost savings as well as more payload that could be lifted (also a cost savings).
Would the booster be destroyed landing in the ocean by parachute?
Quote is towards the end, but the whole vid is worth a watch.
http://heroicrelics.org/info/all-up/all-up-flight-testing.ht...
I think this could have been more successful
sidibe: “I think this could have been more successful”
:)
SpaceX said their goal was to get off the launch pad going into this, which indeed may have been setting a low bar for success. However, there is no other alternative definition.
If you are tasked with recording X, and anything else beyond is bonus, doing X is success.
The main goal here was to clear the pad and get some atmospheric experience with the entire stack. Goal met. But of course they had contingency plans to get as much more experience with it as possible, if things continued to function nominally (as they have with some rovers, or indeed some previous first SpaceX test flights, like the Falcon Heavy).
Btw this isn’t moving the goalposts. Clearing the launch tower was always the success criterion; the rest is gravy.
What would be a failure is if the next one has the same problem.
That seems like such a low bar? One wonders if the engineers/management at SpaceX cringed when he said that.
StarShip is so huge and the reuse will be so cheap that it's going to be 100x or more cheaper. We could send heavy equipment into space and on to other planets that just didn't make sense before.
Once Starship can land 100 tons on the moon, the question isn't "what can we fit" but "how do we fill all this capacity usefully?!" So the science objectives we can achieve grow enormously.
Also, yes: larger space telescopes. And eventually,manned missions to Mars.
https://en.wikipedia.org/wiki/RS-25#Space_Shuttle_program ("contract extension to manufacture 18 additional RS-25 engines, with associated services, for $1.79 billion")
And that doesn't count the 2nd stage which, for 10 flights, should be 3 * 1 + 7 * 4 = 31 RL-10s.
But yeah - a very different architecture.
NASA has to do everything in the public eye and their process reflects this. Private enterprise is allowed to hide information, which implies they are allowed to massage information.
(... this probably says unfortunate things about the nature of governance by public sentiment that it's best not to think about over-much if one is super-fond of democracies as engines of progress...)
That would be a good point if we were talking about NASA vs Blue Origin. But SpaceX is arguably building more in the open than NASA.
The person I replied to was making the case that NASA can't take risks because they develop in the open whereas private companies can develop in hiding, so they can take more risks.
But a quick glance on reddit, nasaspaceflight, and youtube tells me that SpaceX is very much developing in the open, and to a greater extent than NASA.
Which means that there is some other reason why SpaceX feels more free to take risks than NASA.
In contrast, Blue Origin is famous for not disclosing almost any information to the public. For a long time Tory Bruno, the CEO of ULA, was the one providing the most public information on Blue Origin's BE-4 engines.
Not everything is there, but enough to get a very decent picture of what's going on. Sometimes relevant information takes YEARS to be published.
But it's not easy to find. I've had to directly email people sometimes. Sometimes stuff is classified for no good reason other than someone thought at one point that a particular program was dual-use (commercial/military). Sometimes you DO have to file FOIA agreements.
I'm completely not coming down on anyone for not spending their time doing this. It takes a fair amount of time to piece things together from spreadsheets and contract addenda.
But it is possible.
What SpaceX does is they make it easy to see what they're doing. They upload videos to YouTube and Shotwell speaks at events and conferences from time to time. But you are getting their side of the story. Every now and again Casey Dreier over at the Planetary Society will dig up some previously difficult to find nugget of information about how various programs are being funded and exactly what they're being funded for. But Dreier's job is to focus more on planetary exploration missions rather than aerospace development, so I don't think he's focusing on SpaceX, SLS and Blue Origin.
The SLS has spent 23.8 billion in nominal dollars so far. SpaceX has received about 9.8 Billion total investment, which puts an upper limit on operating losses.
I don't follow your NASA example though. We know the NASA/government spend amount for the SLS (23.8 billion). We don't need to speculate on if there is double counting of government spend and contractor spend.
With that, they developed Falcon 1, Merlin engines, Falcon 9, Falcon Heavy, first full-flow staged combustion engine in the methalox Raptor engine, Starship, Starship Heavy, Dragon, Crew Dragon, Re-usability, Starlink satellite manufacturing and 4,000 satellites launched to orbit, and quite a bit more.
Rocket specialists know this is part for the course, but most of the public isn't rocket specialists.
In congress, maybe about 25% of them can't be bothered to remember what state they're from. They don't really matter as they'll pretty much vote however the last person to talk to them wants them to vote. This is why congressional legislative directors try to schedule appointments immediately before their congress-person votes.
About 10% of congress-people are SUPER sharp and will understand this whole "you have to break eggs to make an omelet" concept. 50% more could be educated. The remainder will act randomly depending on what the internal polling says.
SSL and Boeing and the old school guys know a critical part of their job is to lobby congress-persons and staffers. They're not paying them off, they're just making campaign contributions to ensure they get access to pitch their side of the story.
As for the modal American voter? They don't care about space. They care about whether they're getting a raise next year, the mortgage is paid and inflation doesn't price them out of a good meal every now and again. When they get economic security, THEN they start caring about other things like who goes in what bathroom, why they can't buy TANG or light-bulbs at the grocery store anymore or whether they're getting value for money in their national space program.
My gut feeling after doing polling for a few years is the numbers are about the same: 25% of American voters can't tie their shoe-laces, 10% will understand you sometimes fail when you try to do something innovative and the remainder will need some convincing.
The good news is (effectively) no-one in the US looks at national budgets. Heck, most congressional staffers never read the whole thing, just the bits they're interested in. Many (most?) voters (and congress-persons) look to membership organizations for direction. If someone is a member of the Planetary Society, and they happen to be chatting w/ their elected representative, you can be pretty sure they'll mention how important the space program is. If someone is a member of Drunk Middle-Aged Regressive Science Haters of America, you can probably guess what they think about anything with the pong of science about it. Fortunately, this latter class of Americans usually doesn't know who their congress-person is or that they have a congressional representative.
So... to make an already long post short... I don't think you have to convince EVERYBODY, just the people that matter. The message that "it's okay to fail from time to time as long as you're making substantial forward progress" is something at least half of the people who affect US budget priorities can get their brainstems around.
[And as an aside... having worked with US congress-people in the past, I can report they're frequently much smarter than you give them credit for. And you don't get to be a Legislative Director in a congressional staff without being reasonably sharp. But they do worry about how large donors will respond to their votes. We're entering a phase of debt-ceiling debates. The GOP controls congress at the moment and we'll see a lot of wheeling and/or dealing. It's quite telling to see what each party thinks is important. Biden seems to have invested a small portion of personal reputation in the SLS, probably because of it's history over the Obama administration. Despite it's actual benefits, various GOP members may use that to rail against it (SLS) as a proxy for railing against Biden. (I'm trying to avoid being partisan here, some Dems railed against the Constellation program during the Bush years, so I'm not saying it's ONLY something the GOP does. It's just the typical railing against the other party to try and fire up your base so you don't have to explain why you didn't follow-through on your campaign promises.) But I ramble...]
Today's rocket was a byproduct of what they're actually working on, so it's hard to pin a price on it. I'd guess a few million dollars in engines, and a few hundred thousand in steel? This is all stuff they would've needed to find a way to dispose of anyway; the next booster in line (B9) already has hundreds of improvements. Among these are improvements to thrust vector control and engine shielding, both of which were involved in the issues with today's flight.
I'm sure they're not just dumping 2.9B in Elon's bank account (I mean... if they did he would just buy back all the Tesla stock) but there are a series of milestones that need to be met for the government to release the next chunk of funds. I don't know if NASA published the exact milestones, but maybe "not exploding on the launch-pad" was enough to release the next block of funds.
https://www.businessinsider.com/elon-musk-spacex-starship-ro...
By comparison, Falcon Heavy costs about $62M per launch, but Gwynne Shotwell predicted they can shave 40% off that price tag when they start re-using heavies.
https://spacedotbiz.substack.com/p/is-starship-really-going-...
But Elon has supposedly said the TDC (total development cost) of the Starship is projected to be 2-10 billion. [I found this at the Daily Mail website, so take it with as large a grain of salt as you wish.]
https://www.dailymail.co.uk/sciencetech/article-11994755/Wha...
If it costs $10B to build one and $10M per launch and you get 10 launches per vehicle, that's $1.1B per launch if you amortize the development costs. But if it takes $2B to build one and $2M per launch and you get 10 launches, then it's only $220M per launch. But if you build five operational starships that each have 10 launches each, things start to get downright affordable.
So if the question is how much has SpaceX spent on the Starship to date? I don't know that's public info. If the question is how much will they spend on development, the answer is $2B-$10B (though that's from the Daily Mail.) If the question is how much will each launch cost (including amortizing development costs)? It could be as cheap as $42M. Or depending on how much of the $2.9B from NASA they're able to apply to previous development costs... who knows!? They could make $1.6M per launch as long as the GSA doesn't audit them too closely.
There's a myth in US government purchasing that competition drives costs down, and that's probably true for commodities. But Starship class super-heavy-launch vehicles aren't commodities. There's a TREMENDOUS amount of cost (both opportunity cost and dollars) associated with fiscal oversight of large projects. The hope has always been that when Blue Origin built something to compete, it would bring total costs down. I am skeptical.
The design targets 10$/kg to orbit.
If you hang the entire R&D plus materials cost for Starship on this one launch, it's a huge amount.
If you consider this as R&D that will be amortized over decades of launches, not that much.
It's the same math that gets you the military buying $5000 hammers: it all depends how you allocate fixed costs across units.
I expect their math was something like:
95% chance of enough engines lighting to launch 90% chance of clearing launch pad 60% chance of making 60 seconds of flight 30% chance of 2nd stage separation 15% chance of re-entry
Yes, but let's not take the analogy too far. For people who work in safety-critical design (including software), it takes a different mindset than what is often prevalent in silicon valley. When the stakes are high, you don't really want to "move fast and break things".
You can actually see this in SpaceX. In development when stakes are relatively low (e.g., no payload or passengers), the risk threshold is high. But they start taking a more measured approach when that risk starts to ratchet up. The danger being, advocates of one approach don't always know when/how to transition to the other.
This corroborates "fail fast". They achieve reliability and safety by launching far more than traditional space companies and seeing failure in those launches. They prove it out before adding the risk of human lives or expensive cargo. Meanwhile traditional companies will develop their rocket for 15 years till it's "perfect". SpaceX figured out that achieving perfection is best done through actual attempts, not laboratory experiments.
But it also shows "fail slow" later. My point is that "fail fast" is not some hard rule to abide by, but contextually dependent on the risk at stake.
Put another way, what would global exploration have looked like if sailors refused to accept the risks of early ocean crossings?
There is no consummate economic incentive for being an astronaut. The incentive is the experience and making some impact on science, and while that motivates many people the probability of attracting the best and the brightest goes down as the probability of exploding goes up.
I'd say there's unlikely to be enough of an economic incentive to justify riskier manned space travel until the earth becomes a whole lot less habitable.
People volunteer for the military, and I don't think anyone's under any illusion this is risk free. The important thing is not lying to people about the risk involved.
They acknowledged this point, though:
>The incentive is the experience and making some impact on science
I think you might be interpreting their last point differently than me. I took it to mean that the "Mars or bust" is an inspiring, but impractical, narrative.
What you don't want is a culture that is either a) afraid to take any risk because they are afraid of career consequences or b) willing to roll the dice with bad decision processes due to biases and bad incentives.
Example for a): bureaucrats who are unwilling to push the envelope because a bad outcome would effectively end their career
Example for b): making high-risk decisions due cost/schedule pressure, like competition for a contract
Take their example of a failed F9 strut, where the material supplied by a vendor didn't come close to meeting the necessary specs. A mature aersopace company would have processes in place to check the material for these specs before use. SpaceX has since levied these new process checks, but prior to that failure lots people may have pointed to them as being more efficient because of their 'streamlined' process.
If you don't know what you don't know, you must find out.
Framed this way, I'm not surprised younger Elon's company missed it.
It would have been much more economical to test a coupon of the material upon receipt, like what is considered standard practice throughout aerospace companies. Or, like you said, you can blow up a rocket and launch pad instead. Same result, different risk profiles.
The question is how you transition from the state of not knowing to knowing.
If you have poor processes and a lack of knoledge, how do you get better?
In this specific instance, the root cause analysis and remediation are vastly more complicated that presented in this thread. It is not like SpaceX wasnt doing testing on incoming materials at all or ignorant of the concept.
Regarding knowing if you have a poor process or not, it depends on the uniqueness of your problem. For proteins that are relatively common, like material checks, you can shorten your learning process by looking at other organizations that have been through it for decades. For more exotic non-standard problems, you might have to learn the hard way.
Would a "mature aerospace company" also know to not use O-rings outside the temperature range specified by its engineers? Or know to test whether foam traveling at high velocity would penetrate the TPS?
Look, this is hard stuff. It's very easy to tell when you screw up, but very difficult to tell how close you are to screwing up. You're deluding yourself if you think some entities are immune to screwing up just because they've been around.
The difference is I don’t allow “gosh, space is hard!” as a rationale for thinking one organization is immune to those shortcomings. So instead of taking a look and asking something like, “Hmmm. Every other organization seems to have a supplier vetting process for safety critical stuff, I wonder if we should too?” We can instead just pretend we’re smart and different and be forced to learn already solved problems the hard way. The supplier thing is very standard quality control process stuff that transcends industry. Knowing if foam can penetrate tile or o-rings operate out of spec are not, precisely because they were non-standard conditions. That’s not to say that the decisions weren’t flawed, but I don’t think it’s as good of an analogy as you may think. Besides, the investigations largely pointed to broken cultures so I don’t know if that’s the type of company you want SpaceX associated with.
What’s the saying? “A fool learns from his own mistakes. A wise man learns from somebody else’s”
It's good to learn from other people's mistakes, but you also can't let everything people have done before go unchallenged or no progress would ever be made from rockets that cost $2B per launch.
For the real launch they've already mapped out the failure modes and are able to prevent them when it really matters.
One small BS detector is when there's some unplanned/unmitigated test outcome that gets characterized as a "test anomaly" rather than being transparent about the details.
>they've already mapped out the failure modes and are able to prevent them when it really matters
This remains to be seen. The shuttle also had all their failure modes mapped out. As did CST-100. Yet massive failures still occurred.
Mercury and Gemini both had good track records. Apollo and Shuttle, not so much.
Put differently, how much would you be willing to bet that their FMEA has caught literally every failure mode possible?
Regarding the foam, they had a difficult time even recreating it after the fact. It was apparently only on a lark that they decided to turn the gun up to 11 and, viola, now the foam had the physical properties capable of damaging the tile catastrophically. So, yes, they knew the mechanism of foam shedding but did not realize the effect properly.
With the o-rings, they similarly just didn’t have the test data for this conditions. They incorrectly extrapolated on the test data they did have.
The knee jerk mindset that most people have in safety-critical design, is being ultra conservative.
In the fields of medical and space (and likely others), you have an asymmetrical risk-reward profile. Think about it this way, if an engineer takes a risk on refactoring some software logic, and it speeds the system up 3%, what is their reward? A raise? A promotion? Fat chance. If that refactor instead breaks 3% of the time; engines blow up, people die, customers yell at them, they get fired, perhaps in some situations they even get sued.
The engineers then converge to a local maxima by the means of: "If it ain't broke, don't fix it", and various other ultra conservative leanings. This mindset also will often get selected for in hiring, and rewarded.
Now take the limit of this as time goes to infinity, you have bloated, legacy software that is full of spaghetti code and can't take new features easily (if at all). In the case of NASA's space shuttle program, it was extremely expensive, and the cost wasn't falling significantly over time either.
One might view ultra conservatism as the problem, but the real issue is the asymmetrical risk-reward profile. Solving that takes a head-on approach with great leadership, deploying capital, state of the art testing/QA, great deployment pipelines, and more. Shield people from the risk, and intentionally reward people when they push the envelope.
Imagine if you had a software testing process and product specification that was 99.9999% effective (and no, yours is not even close to that), you could then move at a silicon valley "fail fast" pace and advance the technology and architecture rapidly.
I can't personally attest to this but another commenter mentioned that this was what the soviets did as part of a space race with a smaller budget.
You should note that success rate is crucial once you are in production and ready to go - judging overall success against that bar while developing and testing is nonsense.
Finally - I think everyone learns faster by trying more things and making more mistakes? I wouldnt blame you for wanting to see that studied - kind of a neat question after all - but it seems like you are assuming one method or another was picked because it was superior as opposed to a reality of material constraints.
I feel like if NASA had budget for blowing up more rockets to prove/disprove points that they would be doing so. Actually now that I say it out loud I can't imagine anyone choosing 'not rocket'.
Iteration is key here. Nobody is talking about putting a human on Starship until it's quite reliable.
Where in the past, people would be like "OH NO!!" -- but the crew knew how much data they got out of that launch.
I wonder how stripped down the starship was though in an effort to save costs in the event such as this RUD.
I would love a piece of the shrapnel from the exploded device though...
Hopefully they get those pieces and auction them off.
It would be AWESOME if elon included actual fireworks which would go off in the event of a RUD to add some fabulousness to such events.
In fact, all the devices that are stripped out for these test launches should be replaced with their equivalent weights in fireworks. Detonate them when possible during a RUD to allow for more spectacle!
There's something called FTS. Said fireworks were responsible for todays RUD... ;-)
That is what detonated the StarShip today.
You'll find it was notably absent in Soviet rocket tests, for instance with the N1 which veered off course, returned to the ground intact and exploded there instead.
I would be very surprised if Shotwell didn't have an all-hands to set the whole team's expectations beforehand.
Otherwise you get Boeing.
I've have my own version of this more times than I can count.
For me, I run a trading firm and when I started it was "if we can survive the day without losing a ton of money, its a win." I was happy if the positions even made it to the exchange the way we wanted, haha.
https://austinkleon.com/2020/12/10/quantity-leads-to-quality...
[A] ceramics teacher announced on opening day that he was dividing the class into two groups. All those on the left side of the studio, he said, would be graded solely on the quantity of work they produced, all those on the right solely on its quality. His procedure was simple: on the final day of class he would bring in his bathroom scales and weigh the work of the “quantity” group: fifty pound of pots rated an “A”, forty pounds a “B”, and so on. Those being graded on “quality”, however, needed to produce only one pot — albeit a perfect one — to get an “A”. Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the “quantity” group was busily churning out piles of work – and learning from their mistakes — the “quality” group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.
McD's never tried to build the best burger, they built one of the best supply chains in the world.
SpaceX doesn't have to build the best rocket, in whatever that means, it just needs not to blow up and allow its cargo to reach space.
If you build lots or burgers or lots of rockets its highly likely you'll reach and pass 'good enough'.
The counterpoint to it is the probably apocryphal Soviet nail factory (but with more verifiable examples nearby):
Once upon a time, there was a factory in the Soviet Union that made nails. Unfortunately, Moscow set quotas on their nail production, and they began working to meet the quotas as described, rather than doing anything useful. When they set quotas by quantity, they churned out hundreds of thousands of tiny, useless nails. When Moscow realized this was not useful and set a quota by weight instead, they started building big, heavy railroad spike-type nails that weighed a pound each.
https://skeptics.stackexchange.com/questions/22375/did-a-sov...
mabbo: this is why elon musk is actually a genius
> Musk has said the entire program will cost anywhere from $3 billion to $10 billion.
These can't both be true.
The initial launches are gone be more like 5 billion. Its pure fantasy that it will go to $2 billion anytime soon.
That is not always a bad thing. I mean you typed this on a computer running an ISA that started in the 70's or 80's. Maybe 90's if you are on PowerPC ;)
You’d think NASA would have read their own studies and given up rather than proceed on the predictable Artemis boondoggle.
SLS has had a 100% success rate. One launch, one success. But it costs $2B per launch (and climbing) and has taken 12 years to get here. Plus, it's stealing pieces of the Space Shuttle, which was developed in the 70s over a long period of time.
Starship will launch a half a dozen "failures" before they achieve success, but they will have a bigger, better launch system that's fully reusable and costs orders of magnitude less per launch.
"Fail fast" applies to more than just software.