News for hackers. Stuff that matters.
Re:Semiconducting Transport in LK99 by dmvdoug on Tuesday August 01, 2023 @01:35AM (#36952711)
Not just materials science though. I’m a longtime reader of Derek Lowe’s In the Pipeline blog, and he often emphasizes the trickiness of lab work and lab techniques. But the model we have of science from elementary and high school doesn’t give us room for the random, ephemeral things that come into play in actual experimental work. Our model is methods equal result, every time, even though jiggling your foot a little bit to get the feeling back may have somehow made your small molecule production have less binding affinity or something. And yes, eventually you figure that out and stop jiggling your foot but two years have passed.
Re:Semiconducting Transport in LK99 by lucubratory on Tuesday August 01, 2023 @01:37AM (#36952725)
Absolutely, ultimate failures to replicate are generally multiple labs across dozens and dozens of attempts and continuous communication with the authors. And yeah, it's a thing in science in general, materials science is just particularly susceptible because of the nature of the work.
Re:Semiconducting Transport in LK99 by tetha on Tuesday August 01, 2023 @02:10AM (#36952878)
NileRed and other Maker Channels are great to demonstrate that. In every video, there's 1 - 2 of these situations - Nile has been following some documented, established procedure which should result in a specific reaction and it looks decent to the layman - and then it just doesn't work at all. And then he re-does pretty much the same thing and it works.

I can't imagine how frustrating that can be. And then you have to do it with something you don't even know works. Kind of like dealing with obscure retro computers. Does it still work? No one knows. What does it do if it works? No one knows. Can it even function or are parts missing? That's right - no one knows.

Re:Semiconducting Transport in LK99 by teaearlgraycold on Tuesday August 01, 2023 @02:24AM (#36952958)
Re:Semiconducting Transport in LK99 by MichaelZuo on Tuesday August 01, 2023 @09:03AM (#36955419)
Well to be fair it's very difficult to nail down every possible parameter to 99.99% accuracy and precision from the original successful experiment. There's a lot of randomness even in a highly controlled, well funded, modern laboratory setting, let alone less well equipped facilities.
Re:Semiconducting Transport in LK99 by whimsicalism on Tuesday August 01, 2023 @10:21AM (#36956266)
It is incredibly frustrating, my experience in an experimental physics lab involved in creation of certain types of wafers for SEM characterization, there are so many things that can go wrong for certain materials. just a certain speck of dust being in the wrong place at the wrong time

It is why I quit experimental physics.

Re:Semiconducting Transport in LK99 by oceanplexian on Tuesday August 01, 2023 @06:08PM (#36963855)
Anyone who has dived into FDM 3D printing knows this pain well.

Filament is too wet, the air is too cold, a batch of filament might behave wrong. Or everything goes 99% right and the axis of the moon and the constellations are off and it ruins your entire print after 57 hours. Then you start again and it breaks at 59.5 hours. I’m sure it’s not as hardcore as synthesizing superconductors but the real world is incredibly messy and imprecise.

Re:Semiconducting Transport in LK99 by stevage on Tuesday August 01, 2023 @06:39PM (#36964240)
But Jesus in this case the payoff is immense.
Re:Semiconducting Transport in LK99 by Loic on Tuesday August 01, 2023 @02:41AM (#36953067)
For computer science people, I always use the yield on wafers as analogy to explain how hard it can be to nail an experiment right.

Usually they understand directly because this is something they can easily relate to.

Re:Semiconducting Transport in LK99 by eric-hu on Tuesday August 01, 2023 @03:43AM (#36953453)
For those of us unfamiliar with the numbers, what do typical yields look like on silicon wafers?
Re:Semiconducting Transport in LK99 by peyton on Tuesday August 01, 2023 @03:56AM (#36953532)
50–97% with 90% considered “good.”
Re:Semiconducting Transport in LK99 by XorNot on Tuesday August 01, 2023 @06:32AM (#36954416)
It's also worth contextualizing percentages. 90% still means 1 in every 10 parts fails.
Re:Semiconducting Transport in LK99 by OJFord on Tuesday August 01, 2023 @11:15AM (#36957053)
It's not always outright failure, but rather different performance binning.
Re:Semiconducting Transport in LK99 by Chabsff on Tuesday August 01, 2023 @11:38AM (#36957435)
But the performance binning is a direct result of careful design that goes out of its way to ensure that a subset of failures can still lead to a workable/binnable products.

It's still a failure to produce a part that matches the full design.

Re:Semiconducting Transport in LK99 by crote on Tuesday August 01, 2023 @05:58AM (#36954227)
It really varies.

Samsung 3nm had a yield rate well below 20% back in early 2022, and that's not unheard of while still in the development phase. To be economically viable the yield rates have to be around 80%, so mature nodes can be expected to be in that ballpark.

Re:Semiconducting Transport in LK99 by djtango on Tuesday August 01, 2023 @02:57AM (#36953170)
We also don't have a culture of replication. Basically all of the post docs I worked with in the lab lamented how hard it can be to replicate reported syntheses in chemistry wet labs.

Everything from humidity and altitude to undocumented "magic tricks" (eg baking something in an oven under vaccuum for a week to get.rid.of.all.the.water.)

I heard anecdotes that experiments were sometimes literally only reproducible in one exact lab because reasons.

Turns out there's just as much art as there is science in experimentation...

Re:Semiconducting Transport in LK99 by mschuster91 on Tuesday August 01, 2023 @03:31AM (#36953372)
The problem IMHO is funding related. Often enough the original lab's grant is barely enough to account for the research and a shoddily put together paper, the rest is filled with a crapload of un(der)paid/"voluntold" labour.

It's rare for researchers to get the budget to replicate someone else's work - this case might be one of the few exceptions given that everyone and their dog want to be the first one to show the world a successful replication. Hell, there are Youtube and Tiktok streamers on the task as well, that way you know just how nuts everyone is going.

Re:Semiconducting Transport in LK99 by michaelt on Tuesday August 01, 2023 @06:24AM (#36954371)
Even in well funded fields, nobody wants to give away easy follow-on publications to other labs.

Oh, you want to document things well enough that people can build on your work and give you citations eventually. But if you've just demonstrated a groundbreaking technique for Foo and the obvious next step is to test it on Bar, you've got an opportunity to get your lab two papers instead of one, and all you have to do is less documentation?

Re:Semiconducting Transport in LK99 by techdragon on Tuesday August 01, 2023 @03:52AM (#36953512)
The simple truth is we don’t have a culture of replication because no one is paying and no one is paying because no one is demanding it as part of the incentive system that the money is part of…

If a prerequisite of publication in a top tier journal was someone having replicated the work… anyone at all… then the quality of write up would improve dramatically because all the stuff that never gets written up would be written up so that the second lab costs them less money to do the replication work, so the second lab doesn’t spent a month failing because they didn’t bake the powder in a vacuum for a week before performing the entire synthesis under dry nitrogen purge when the methodology just said the powder had been dried and the synthesis was performed at ambient pressure in a zero humidity atmosphere…

Re:Semiconducting Transport in LK99 by grall on Tuesday August 01, 2023 @09:58AM (#36956028)
> If a prerequisite of publication in a top tier journal was someone having replicated the work… anyone at all

For a SciFi take on this see the section "A Note on Governance" at https://eldraeverse.com/2018/05/25/approach-vector-2-2/. Specifically, the "Dean of Evidence".

Re:Semiconducting Transport in LK99 by Someone on Tuesday August 01, 2023 @10:21AM (#36956272)
> If a prerequisite of publication in a top tier journal was someone having replicated the work… anyone at all… then the quality of write up would improve dramatically

I would expect pairs of labs to gang up, with one of them giving the other the details so that they can reproduce the result, and the contents of the publication staying more or less the same.

Reason is that everybody wants to prevent others from being able to build on one’s work faster than one can do themselves.

Even if the lab you give more information isn’t run by a buddy of you, having a single competitor is better than having the whole world compete with you.

For example, in this case, people will start cooking all kinds of variations on this stuff to find one that’s easier to produce, allows for higher currents before losing superconductivity, doesn’t have lead in it, etc.

Re:Semiconducting Transport in LK99 by techdragon on Tuesday August 01, 2023 @11:15AM (#36957062)
Oh absolutely, there’s a lot of potential issues with the simple suggestion of requiring replication. I was tempted to write a bunch more to outline strategies and counter strategies from a game theory perspective but realised it was basically unnecessary since it will never happen.
Re:Semiconducting Transport in LK99 by pjc50 on Tuesday August 01, 2023 @12:23PM (#36958185)
> The simple truth is we don’t have a culture of replication because no one is paying and no one is paying because no one is demanding it as part of the incentive system that the money is part of…

A darker comment: the non-replication of results implies that nobody's actually making use of the science output, and an alarmingly large fraction of unreplicated positive results are simply a waste of time.

Re:Semiconducting Transport in LK99 by Vespasian on Tuesday August 01, 2023 @03:58AM (#36953538)
In that light, industrial processes seem even more impressive.

The super power of modern technology is the ability to perform follow precise steps on a massive scale and basically forever.

Obviously it takes a lot of effort to keep the machinery running but still, the mere fact that it's running in the first place blows your mind if you think about it.

Modern Machinery multiplies the impact on the world of any human who operates it.

Re:Semiconducting Transport in LK99 by djtango on Tuesday August 01, 2023 @04:13AM (#36953619)
Research experimentation is like doing a gymnastics performance, you're trying to prove the limits of what can be done.

Industry is all about doing it cheap, consistently and at scale. So by comparison you don't want any flourish or special conditions that take lots of practice and super skilled operators.

The magic of industrial engineering is figuring out how to go from research to scale. But the flipside is all the crazy stuff that is achievable at the frontier that never gets productionised

Re:Semiconducting Transport in LK99 by ilc on Tuesday August 01, 2023 @09:02AM (#36955405)
Look up FOGBANK (nuclear material), if you want to see what happens when people get REALLY motivated to replicate the impossible to replicate.
Re:Semiconducting Transport in LK99 by kitd on Tuesday August 01, 2023 @09:58AM (#36956021)
A chemical well-it-works-on-my-computer experiment. Nice!
Re:Semiconducting Transport in LK99 by whimsicalism on Tuesday August 01, 2023 @10:23AM (#36956301)
This was my experience as well, my job was basically to specialize in reverse engineering those exact “baking” techniques in the tube furnace and it is so challenging and subject to variability.
Re:Semiconducting Transport in LK99 by paulmd on Tuesday August 01, 2023 @12:24PM (#36958205)
> baking something in an oven under vaccuum for a week to get.rid.of.all.the.water.

vacuum oven also works great for drying 3d printer filament!

Re:Semiconducting Transport in LK99 by OJFord on Tuesday August 01, 2023 @11:07AM (#36956932)
> But the model we have of science from elementary and high school doesn’t give us room for the random, ephemeral things that come into play in actual experimental work. Our model is methods equal result, every time, even though jiggling your foot a little bit to get the feeling back may have somehow made your small molecule production have less binding affinity or something.

That wasn't my experience, in either high school physics or undergrad electronics. I hated keeping a lab book (just wanted to 'do' & see results I suppose) but the emphasis was always on recording as much as possible to understand what's important and what happened etc. when you review later.

I suppose looking through another lens you can call that an expectation of repeatability, but it's just a matter of emphasis isn't it? You know it might be hard to repeat, so you record as much as possible about the setup & process to remove variables and control what needs to be controlled (especially if you don't yet understand what that is, or the full scope).

Re:Semiconducting Transport in LK99 by sterlind on Tuesday August 01, 2023 @11:07AM (#36956936)
FOGBANK, the classified styrofoam-like material used as a filler in US thermonuclear warheads, had a similar issue. They needed to restart production, but the new samples didn't work! It turns out an impurity present in the original batch was critical. Once they reproduced that impurity, the material acted correctly.
Re:Semiconducting Transport in LK99 by AaronM on Tuesday August 01, 2023 @11:39AM (#36957459)
I am not involved in lab work at all, but his "Things I won't work with" posts are wonderful