A 3% chance of impact right after discovery with the initial error ellipse isn't all that unusual; it will almost certainly be revised to 0 with more observation (and if it's not, you'll hear about it).
If these large numbers happen so often that asteroids with initial impact probabilities of 3% are known to actually impact much less frequently than that, then the model is poorly calibrated, no? In other words, the reported probabilities aren't really probabilities and that is what has caused the confusion and anxiety in these comments.
"The measurement data has noise" does not explain why the noise has a bias towards "the asteroid will hit earth" whereas reality so far has been biased towards "the asteroid will not hit earth".
(This assumes that significantly more than 33 asteroids have had >= 3% impact probability predicted at some point. The opposite would not be less concerning.)
You wait 5 sec and it's still only imperceptibly closer. You realize there is no way it could possibly hit you. You cross the street unconcerned.
That's exactly it. And at the speeds these objects are going and the uncertainty of the observations you would have to be observing an object for a really long time to get the kind of accuracy required to pick a mitigation method that would work. And even then, assuming you could nail the point of impact of something going 2000 km / second of unknown mass in a strong gravity field: given the COVID response I have a hard time believing that the response to 'Houston, Texas is going to be obliterated on Jun 1st 2024' would be met with anything but skepticism and laughter. Right up to the moment of impact.
A 3% chance that never occurs is an inaccurate prediction.
It's wrong because the measurements are suggestive of possibility, rather than certain of it.
If we observe an asteroid that with two poor measurements is determined to be headed away from Earth, that's the end. Look no further.
If we observe an asteroid with two poor measurements that has some significant chance of hitting, more and better measurements are made. Then very often those better measurements show it was never actually going to hit anyhow.
But we never would have known without the better measurements, and we never would have devoted more time to making better measurements without a reason to do so.
A 3% chance that never occurs is because that 3% is based on data that's at the limit of what the telescopes can provide, not based upon bad math.
Hence it seems that it would lead to more accurate predictions if the measurements and their uncertainties were fitted to a model that corrects for the prior probability of observing an asteroid on a given trajectory/making a certain observation.
This discrepancy between distribution of measurement error vs distribution of actual trajectories is what people are wondering about, because it seems interesting to know more about (e.g. "why are certain trajectories less likely?").
It's is similar to P hacking.
If you just want to argue with people, feel free. But based on how this conversation has been going it doesn't seem like you want to learn.
I understand that calculating trajectories is difficult.
If someone claims something like a 3% impact probability, and they are wrong 99.999% of the time, that speaks to a methodological error in how the numbers are conveyed and or defined.
I work in medical devices and testing. I perform tests like X percentage of patients will die based on the statistical calculations. You may undergo treatment with a medical device that I have worked on.
Wait a week and get another sample and your arc is now approx 5x as long. Wait a month and get another and now your arc is 30x as long as the original. More observations shrink your error bars.
There are systemic errors here for sure. Two kinds, really:
1. Limits of resolution of telescopes 2. Short sample lengths
You absolutely can't do anything about error type 1. You can fix 2 by getting more data. But there's no point in getting data on asteroids that have absolutely no possibility of hitting. So only asteroids that have some probability with limited measurements get enough better measurements that are high quality in order to find out where they're really headed.
All of these measurements of trajectories are completely uncorrelated, so you can't use the priors to adjust probabilities. I mean you can do whatever you want, but we haven't been hit by a big asteroid yet since we've had telescopes and tracking databases.
If we made adjustments based on priors we'd have to discount all collisions down to 0 irrespective of the trajectories. Seems absurd, so there must be something else going on here.
Illustrate the point, imagine a pass/fail AIDs test with 99% accuracy and 1% false positive. If you test one patient only and they are positive, You can conclude that is 99% likely to be correct. However, if you test a hundred different people and one of them comes up positive, you can no longer claim the 99% certainty for that patient. You know that you administered a hundred different tests to different people and would have to reduce your confidence accordingly because you expect one false positive. This second statistical approach is what is not happening with the asteroids, and why asteroids with a 3% chance of hitting Earth suspiciously get revised down to zero more than 97% of the time.
>If we made adjustments based on priors we'd have to discount all collisions down to 0 irrespective of the trajectories. Seems absurd, so there must be something else going on here
Not quite true. If you measure a million asteroids in the data from one says it has a trajectory towards Earth, you need to Discount that observation by the fact that you made 1 million different measurements. The outlier might still be close to zero statistically, but it did have a outlier data. This would be a reason to remeasure the asteroid multiple times. It is only through that process that the number will climb from zero, or stay at zero.
It's not that you're applying the prior that we have never observed Earth colliding asteroid. You're simply accounting for the fact that with the error bars on your measurement system, you expect one false positive in 1 million measurements.
My inference is that the 3% number we are talking about for this specific asteroid what's not calculated using the proper statistical treatment, and that's why it wasn't published in the first place.
This is also why it is similar to P hacking. If you run 20 experiments and analyze them as if they were the only experiment you did, you will get one of them that says a wrong result with 95% confidence, which is the common threshold for publishing outside of physics.
Consider a 6-sided dice roll. What is the chance it will roll a 1?
A person might think, "1 in 6". But what if this is a loaded die? In that case, we need more information before we can classify it as "a die like other dice". We can observe two rolls, and try to ascertain whether or not it is like other fair 6-sided dice; however, two rolls is not enough to be sure.
So as we're gathering data, we start to classify this instance of a thing (a die, an asteroid) as part of a series of things we already know about. The more rolls we observe, the more sure we can be that this is a fair die or a loaded die, for instance.
If I'm understanding how asteroids' trajectories are calculated, we can simulate THIS asteroid's trajectory (3% chance of hitting you, based on a little data), or we can just decide to classify it (perhaps prematurely?) in the series "an asteroid like every other asteroid that we've observed" and arrive at a 0.000001% chance of hitting you (I'm making up a number here).
You have one confidence margin for a single single measurement and a different confidence margin if you make 1 million measurements.
Let's say you can measure marble diameters and your tool has a calibrated standard deviation of 1 mm.
If you pull one marble and measure it to be 10 mm larger than expected, you can calculate the chance you are wrong using only the standard deviation of your measurement tool.
However, if you pull 1 million marbles and measure one to be 10 mm larger than expected, you need to take into account the number of marbles you have measured.
When it gets a little closer, you can tell at least which half it's on, the left or the right. Now your estimate is either 0% or 50%.
Closer still and you tell which lane it's in, so now you're sure.
What wouldn't make sense is if you repeat this 1000 times and a car is never in your lane.
That means that something is wrong about how you are modeling the road and cars.
The claim that people are confused by is (asteroids with a 3% chance of hitting get the change revised to 0% more than 97% of the time).
There are parts of the road you can't see. If you were forced to estimate the chance you'd have to move for this particular purported car with just this information, you might throw up your hands and say P(they're telling the truth) * 1/4 until you can see it.
If a bunch of people like to lie, and you have nothing but one person's word, your move/no-move prediction for this car is likely to be wrong despite your model of the road-car-person system being perfectly right.
As more people say they saw the car, the estimate from these reports approaches 1/4, dominated by your understanding of the system. But you might have fewer early estimates that say you need to move per actual car and actual move, and many estimates for cars that don't exist.
I hope we're in a world where that makes sense and I'm actually right about this. But who knows!
Now think of what that kind of force would do on a much lighter object that moves faster.
It's just that anything traveling through the earth/moon gravitational sphere of influence will have it's trajectory tweaked just a bit. How close to the center of gravity the pass is determines exactly how much of a tweak. There is a small section of space, we'll call it the keyhole, where if the asteroid happens to pass exactly through that area the tweak will result in a collision next time the asteroid comes around. That could be decades hence.
There could even be a case where an unlucky keyhole pass this time lines up another unlucky keyhole pass the next time to an eventual collision in the distance future.
The technology to nudge the asteroid just far enough to miss a critical keyhole pass is within the realm of possibility with today's knowhow. We just need to have these missions ready to go on short (order of a few months to a year) notice.
Most likely: this will never come up.
Less likely: if it does we're fucked.
Even less likely: if it does and surprisingly we see it in time we will act for the good of all and not bicker about who pays and we'll make things better rather than worse. If not, see above.
[0] https://arxiv.org/abs/0707.1919
[edit] FWIW, I actually corresponded with one of the authors of this paper back in 2007, and from what I could tell, this wasn't an attempt at parody, although now it might be dismissed as one. Personally I'm not willing to declare my (non)commitment to the theory either way.
So the best guess you have is that the true asteroid is 99% likely to be somewhere within a 2km box centered at the observation point.
For each possible location in this box you use it as a hypothetical starting point and run a simulation forward creating a trajectory. In 3% of these trajectories the asteroid hits the earth.
The 3% is only a probability over the measurement uncertainty. It represents our knowledge about the system in a bayesian sense. The true asteroid was always ever going to hit the earth or not. There is no uncertainty inherent in the system.
That many asteroids have non negligible probability only means the physics is sensitive to initial conditions or that the measurements are loose. (Both are true)
What your analysis is not touching on is the prior probability that an asteroid will hit earth (you collapse this to "any asteroid will either hit or not", but that is not helpful for "model calibration" or whatever you want to call this) - or, equivalently, the prior probability of making (a series of) observations with a certain uncertainty/error distribution. If that prior were actually as uniform as each measurement error suggests, I don't see any Bayesian wiggle room left for why we don't have those 3% of impact actually happen.
(I'm no expert, but presumably you need multiple measurements to predict a trajectory, and while their measurement error distributions may be independent, it seems plausible to me that the prior probability of making two specific noise-affected observations, i.e. of the asteroid being on a certain trajectory, is most likely not so uniform. That's the part that I'd like to learn more about though.)
-Then what does 3% mean? Surely it means "given the data we have, one in every 33 will hit" -Given everything you said is true, under those assumptions 3% of those asteroids that we identify as being in said 2km box will hit earth.
Both of these statements are false. The probability density is over our knowledge of the state variables/state space for this asteroid, not over asteroids. The hypothetical sample of asteroids is not drawn from the distribution I'm talking about.
Going back to the simplified example: With the uniform prior on the box, our probability means that 3% of the volume of this box would lead to an impact if an asteroid was centered at a point in that volume at this time of measurement.
It doesn't say anything about hypothetical realizations of this asteroid (it is not clear what this would be sampled from or what it means in a precise sense to repeat a 1 time event) and says even less about the sample of (nearly) independent asteroids observed in the past. The probability measure only describes the measurement uncertainty on properties of this particular asteroid. It is not conditioned on or related to statistics on impacts of "general asteroids".
But "presumably you need multiple measurements to predict a trajectory" and your notes about independence and uniformness being bad assumptions are absolutely correct tho. I agree 100%
My comment above is mostly an attempt to make a simple example to clarify what the probability measure being measured here is. It's not a physically realistic example :) and definitely doesn't make good assumptions about what information is needed and what error distributions that information would have! I don't do space and didn't want to make guesses
Calibration here would have to be over multiple measurements of the same asteroid (which my example doesn't touch on). Likely by predicting trajectories at different intervals and matching the likelihood of later observations.
Verifying multiple observations leading up to a 1 time event is a very different than, say, verification of simulations of an internal combustion engine design where measurements of a real world prototype can be conducted repeatedly and independently to learn/calibrate some fundamental properties or initial conditions like chemical kinetic coefficients and such.
For general interest/lectures/fun, the general field that studies how to push uncertainties forwards/backwards/calibrate a mathematical model and simulation is called "Uncertainty Quantification". Also not an expert lol, I was just surrounded by a bunch in my cohort
There would be a ~63.4% chance that at least one would hit us if there were 33 such asteroids. To compute this, take 1-(0.97^33). I agree with your broader point though.
In many situations, erring on one side results in worse outcomes than erring on the other side. In our case, a false positive has pretty much zero consequences, while a false negative could wipe out the dinosaurs.
There would be a ~63.4% chance that at least one would hit us if this happened 33 times. To compute this, take 1-(0.97^33). But I agree with your broader intuition that these predictions must be getting inflated.
By "This makes sense", I meant that this kind of thing can happen; as more data are gathered, the Bayesian probability of a candidate value can increase and then suddenly decrease. Here's a Colab notebook demonstrating the general phenomenon: https://colab.research.google.com/drive/1Eb1_humiGPdKb0c3qr_...
"Calibration" in this context means "statistical consistency between distributional forecasts and observations" in the words of https://sites.stat.washington.edu/raftery/Research/PDF/Gneit... . If the model's early forecasts predict impact with probability >3% for a class of objects that end up impacting with frequency much less than 3%, then the model is not well calibrated with respect to its early forecasts for those objects.
Based on the GP, it sounds like these early impact "probabilities" are no one's subjective (Bayesian) probability of impact because people who are closely familiar with this model know it is not well calibrated. The reported probabilities may still be useful to them as indicators or flags. However, those of us who are _not_ closely familiar with the model have found it confusing to see things that are not really probabilities reported as probabilities.
Anything that size aimed straight down would most likely not reach the ground but burn up in the atmosphere and any remaining bits would just fall at regular terminal velocity.
But from 10 meters and up things change and the Chelyabinsk meteor is remarkable in that it (1) was large enough to have been detected but wasn't and (2) struck while we were apparently focused on one that was more visible but that ultimately missed us. We were very lucky that it impacted where and at the angle that it did, otherwise the airburst might have happened far closer to the ground or to might have impacted directly over much more populated territory. That would have been very bad news.
It doesn't matter how many 1 through 5 meter objects we can track because we have the atmosphere to protected us from the worst of these if we miss the 20 meter ones (or apparently even much larger) that travel at speeds high enough to give their relatively modest mass tremendous energy and for which the atmosphere does not give sufficient (or even any) protection.
- What is the probability that this asteroid will hit us?
- What is the time interval where that probability applies?
> [TDB] is a relativistic coordinate time scale, intended for astronomical use as a time standard to take account of time dilation when calculating orbits and astronomical ephemerides of planets, asteroids, comets and interplanetary spacecraft in the Solar System. TDB is now (since 2006) defined as a linear scaling of Barycentric Coordinate Time (TCB). A feature that distinguishes TDB from TCB is that TDB, when observed from the Earth's surface, has a difference from Terrestrial Time (TT) that is about as small as can be practically arranged with consistent definition: the differences are mainly periodic, and overall will remain at less than 2 milliseconds for several millennia.
https://en.wikipedia.org/wiki/Terrestrial_Time
> TT is distinct from the time scale often used as a basis for civil purposes, Coordinated Universal Time (UTC). TT is indirectly the basis of UTC, via International Atomic Time (TAI). Because of the historical difference between TAI and [Ephemeris Time] ET when TT was introduced, TT is approximately 32.184s ahead of TAI.
Phew!
2ms at 10km/s is off by 20m.
If there's <1% confidence - this is not exactly news.
"If it is real this IS the worst asteroid threat ever discovered and the impact location and times are ugly (will post pics shortly). Note that the impact is in the next couple days! However, an Italian colleague of one of our astronomers suspects there is an error in the reported observations and there has been no chatter about this object and no followup. This probably means that it is not real. " - Joel C. Sercel, PhD
I'm guessing the impact death area will be around 3600 km2 (I have no idea, please correct), so ultimately the chance of it falling on me is 1 in 10,000 to 1 in 100,000 provided the asteroid hits. Which means 1 in 1,000,000 to 1 in 10,000,000 in total? (assuming 1% hit probability)
I’m hoping this ztm0038 is an error or hoax.
>> I would like to make an apology to the small body community. Some information was shared with me on an internal company message board that I did not have full perspective on and I posted a tweet, which I now understand was not appropriate. It was preliminary data and I did not have full perspective on it.
Does that sound like everything's gonna be okay?
Land impact would be catastrophic (1x-100x 50 megaton explosion)
> I would like to make an apology to the small body community. Some information was shared with me on an internal company message board that I did not have full perspective on and I posted a tweet, which I now understand was not appropriate. It was preliminary data and I did not have full perspective on it.
“Preliminary data” is not very reassuring.