For stuff like the bad social science you mentioned, often you can catch it if you know basic statistics along with the common tricks people use to p-hack. See, for example, "The garden of forking paths" by Gelman.
Even if the statistical analysis is not given explicitly, or the issue is more subtle, for social science that touches on everyday life, you can usually tell if it'll hold up just with common sense. Try yourself using this quiz: https://80000hours.org/psychology-replication-quiz/
Anything that is either out of range on scale, or that ducks the question, is worthy of further evaluation and at least some mild skepticism. Theranos promising hundreds of blood tests out of a thimble of blood would be the former. The article referring to Volts of electricity is like a car salesman replying to 'how many seats does it have?' with 'The seats are upholstered in Fine Corinthian Leather' the statement isn't incorrect, it is just useless and, in context, intentionally misleading. Salesmen do it to sell cars, scientists to sell research, journalists to sell papers, and conspiracy theorists for, whatever reasons, I guess.
[0] A few examples: How many piano tuners are in New York City? How many ping pong balls could fit in a school bus?
I really don’t get when people bemoan these. Do they not get that it’s supposed to be an estimation, and it’s not actually a quiz of whether you’ve memorized a seemingly useless bit of trivia? Do they not see how it can be a useful skill to have, especially in an engineering field where you frequently deal with many orders of magnitude? (If acquiring this lock takes 100 ns, am I doing this often enough for this to cause a user-perceivable delay longer than doing it single-threaded?)
The way you get better at this is by doing it, a lot. Every time you read such an article, or some one relays such information to you, run the math through your head as much as you can. Make notes of the things you don’t know that prevent you from pushing the thinking further, and methodically learn about those. Don’t think of yourself in terms of particular labels - eg “I’m not much of a physics person” or “I’m a software guy” - your job is to understand the world, solve problems, answer questions.
It’s a constant effort and it’s not easy. It helps to surround yourself with others with the same drive. It helps to teach, as it forces you to constantly reassess the fundamentals, and the common mistakes beginners make (see Feynman’s writing on that topic).
In this particular case, there's the knowledge of the first basic law of thermodynamics (You can't win, the limiting case on energy conversion) and Kinematics, typically taught as the first half/third of physics. (Broadly: Mechanics/Kinematics, E/M, Optics and Waves)
[1] https://ocw.mit.edu/courses/mathematics/18-098-street-fighti...
That's definitely how I feel about it at least. Others might feel differently.
That said, how does one actual build skills in this area. It feels like the author is able to do this here because it's a research paper in an area they have knowledge in.
How does one build up a general ability to critically evaluate numbers in a case like this? Does one have to know how to go look up formulas?
I recall an article from some time ago where authors released a paper about chocolate and health benefits or something similar but they intentionally gamed the numbers to find something that correlated. The press ate it up before the authors revealed the truth.
How does one go about becoming more literate in evaluating those kinds of papers and conclusions?
I'm willing to put in the time to read carefully and calculate. I just wish I knew what skills to acquire.