I'm not trying to refute what he says about how he feels, whether that saves him money, or his mental ability - those aren't my areas of expertise. I do know a little about medical tests, and I think it's valid to point out that he clearly does not and how the results he reports are either incorrect or irrelevant. I'm just suggesting you shouldn't consider his medical tests in deciding how good this product is.
I have no horse in this race, but I'll tell you if this racetrack is not standard regulation. Self experimentation might be useful, but only if you understand the metrics you measure and whether it's stochastic noise or meaningful results.
I feel you miss the whole point of self experimentation. If your aim is for improved mental function, more energy, less health problems, etc., you don't need to know how to interpret any obscure tests. You'll know.
A low-risk food product (such as this) can simply be trialed in a n=1 trial. If the results are positive, great. If not, go back to what you were doing before.
All the studies in the world only suggest how something might work for you. It is only the n=1 test that matters.
Perception is different from reality, as exemplified by an Apple anecdote:
http://www.asktog.com/TOI/toi06KeyboardVMouse1.html
We’ve done a cool $50 million of R & D on the Apple Human Interface. We discovered, among other things, two pertinent facts:
- Test subjects consistently report that keyboarding is faster than mousing.
- The stopwatch consistently proves mousing is faster than keyboarding.
This contradiction between user-experience and reality apparently forms the basis for many user/developers’ belief that the keyboard is faster.
Interesting article. I'm sure the $50M in studies were of non-power users, though, and so irrelevant for developers with sufficiently powerful keyboard shortcuts.
I can guarantee it doesn't take me 2 seconds for my top 20 keyboard shortcuts, any more than it takes 2 seconds for me to hit "return" at the end of a line, or to move around with arrow keys. That said, I frequently use the mouse as well, and always prefer to work in a GUI instead of a text-mode editor.
What I found most revealing, though, was the admission that part of the problem was the awkward location of the command key. Whenever I'm on a Mac I end up remapping it so that I can hit Control (in the corner of the keyboard) -- Control-ZXCV are all easier to hit than Command-ZXCV.
Which just goes to prove that most studies are wrong [1], or at least can't always be generalized.
From observing developers I've worked with over the years, I doubt most developers know more keyboard shortcuts than "normal people".
In fact, I've come to realise just how appallingly bad most of us are at using our tools.
I don't disagree. I know developers who use Notepad to develop, which boggles the mind.
In fact, I don't know why I bothered posting. I'm enough of an outlier in keyboard performance that, even if I'm right, it proves nothing. I've had many developers' jaws hit the floor as they watch over my shoulder. I've even shocked vi and emacs users into no longer trying to convince me their ways were better.
Maybe I should teach a class on how to use a GUI efficiently.
Without measuring it A/B I cannot guarantee anything. Those people were just as sure. And I don't think they were keyboard un-savvy -- at that point in computing, they were far more proficient with a keyboard (AppleII etc) than a mouse.
You don't need to guarantee anything; I know my navigation and coding are way faster my way than if I were to ignore my shortcuts.
>And I don't think they were keyboard un-savvy
As a sister comment to your mentions, MOST users, even developers, are still extremely limited in the number of keyboard shortcuts they use. I'm an outlier.
If I want to save, my fingers hit Ctrl-S before I could even FIND where the mouse pointer is on the screen. Same with Ctrl-A for Select All. Or Shift-Ctrl-Left for select the previous word. No A-B testing required; it's seconds using the mouse for any of those, and 300-700ms for the keyboard.
And those aren't even my most powerful shortcuts; they're just the ones ANYONE can use on Windows. If you get into "complete this word from [above/below] in this file", "jump to definition of this function" or "show references for this function", or "jump to previous bookmark", or "set bookmark" even, all of these are sub-500ms key sequences that would all take longer than a second if I had to use a mouse. It's not even close.
Part of the problem with the study is that it was comparing using shortcuts vs. mouse on the Mac, where shortcuts are far less useful than they are on Windows. Navigating through menus by keyboard is easier for me with the keyboard on Windows as well; that's not even an option on Mac.
If the changes are great enough in magnitude to be obvious, then you know you're on to something. Also, measures like weight and computer mental agility tests can be quite reliable if you are consistent in your testing, and they aren't subject to misinterpretation or wishful thinking.
Edit:
Below, skore is skeptical that a person can notice a significant change to their own health. I respectfully disagree.
Check out the blog of Seth Roberts. He is a professor who is involved in the quantified self movement. He records many subjective measures qualitatively, and feels he has made some significant discoveries doing so. I won't try to summarize anything here, but anyone interested can read for themselves on his blog. Roberts studies rats, and he is the ultimate human rat.
No, no you don't. That's the whole point of doing scientific experiments. Your brain is very, very, extremely bad at judging objectively. Any subjective "test" is simply no test at all.
> Also, numbers like weight and computer mental agility tests also can be quite reliable if you are consistent in your testing...
That's a pretty big 'if', plus a 'can' and a 'quite'.
> ...and they aren't subject to misinterpretation or wishful thinking
That's the whole point of the OT in this discussion thread. They very much clearly are subject to misinterpretation. A zillion factors can impact weight and "mental agility".
People aren't lab rats because they don't live in labs and aren't rats.
> Below, skore is skeptical that a person can notice a significant change to their own health.
Yes, a point which still stands.
> Roberts studies rats, and he is the ultimate human rat.
No, Robert might be the ultimate rat conducting experiments on himself. He cannot be his own lab rat, he can only be running his own asylum.
> He records many subjective measures qualitatively, and feels he has made some significant discoveries doing so.
That sums it up perfectly. I suppose all this boils down to is whether you want your science science-y or feel-y. I prefer the science science. Because while I trust them in many things, I don't trust human emotions when making important decisions that are supposed to influence millions over millions of other humans.
I don't know Seth Roberts and I hope I don't come across as overly disrespectful. I'm not saying that it's impossible to make scientific discoveries this way. The problem is that any discovery you make is N=1 and as far away from a double-blind trial as it can be. So this might be an important tool to pre-select what you want to concentrate your actual science on.
But as with the Soylent stuff, I would never in a million years base my own health decisions on some quantified self guy on the internet.
This is exactly why we have science: people have prior beliefs that affect how they perceive the thing they're trying to measure. People cannot objectively judge their own health in response to a variable. There are thousands of placebo studies that demonstrate this.
You mean like the people who take homeopathic 'medicine'? They know. They're wrong, but they know.
I found this drug that makes me feel way relaxed and totally content with the world. I don't do anything while on the drug, 'cause life's great just sitting here on the couch. As a consequence of sitting on my couch enjoying my great life, I don't spend any money, so I'm totally saving compared to before I took this drug. Pass the joint, man.
I'm legitimately not sure if your argument in this thread is serious. You are basically recommending we throw out any pretensions of objectivity or science or knowledge in favor of "do what you feel".
Not so. Mental agility, weight, blood pressure are all easy and inexpensive to track.
A person's perception of how well they slept, overall feeling of well being, etc., can also be tracked if they are careful to quantify and record them.
I think you underestimate the power of carefully recorded data, which is at the core of the quantified self movement.
>If your aim is for improved mental function, more energy, less health problems, etc., you don't need to know how to interpret any obscure tests. You'll know.
The above directly contradicts your most recent reply, since it obviously precludes the possibility of any meaningful quantification.
"Screw the tests, you'll know" is precisely the opposite of objective analysis. We'll know what? Nothing objective, that's for sure.
Since you appear to be arguing both for and against quantified self-style living in this thread, I can't help but wonder what your angle is. Are you actually just arguing for Soylent? Your position makes no sense, but you're enthusiastically defending the product, and you've done it in prior threads about Soylent, too.
Many argue against Soylent by arguing that we need some careful studies before anyone should try it. I think that is overkill for a food product like Soylent.
As for how to tell if Soylent or anything else is helping you, big changes should be obvious, smaller changes would require careful quantified-self style tracking to detect. Sorry if I wasn't more clear, but by obscure test I'm referring to blood work and other harder to interpret tests. Easy to interpret tests are reaction time, weight, fast arithmetic tests for mental agility, as well as recording qualitative feelings of health quantitatively, in a consistent and rigorous manner.
Edit:
As for this article on the Soylent trial, my take was it had "some promising initial signs given the short duration of the trial". Hardly blind enthusiasm.
I'd like to see some proper triple-blind studies involving people who haven't strongly invested themselves in the outcome.
People will see in this what they want to see. This is why case reports are evidence of the lowest grade. This is more a curio than it is evidence, and there is nothing wrong with writing a blog post about a curio.
1 = I think the creatinine bump is quite likely due to stochastic fluctuation, but if for discussion's sake we attribute all positive changes to Soylent, we should do the same for the negative ones.
It could be just fluctuating in his case, or it could very well be, given his past diet (how could he survive on that), that he was finally getting decent amounts of protein.
For creatinine, there is peer-reviewed data demonstrating that in people without renal disease, increasing the protein in your diet tends to improve your eGFR, not decrease it (which is what happened here). So for him, it either fluctuated or was a marker of true kidney disease, but in any case this is not just a marker of adequate protein intake.
Also, while you may very well be right that GFR would improve, the little bits of reading I've had time to do indicates that unless they did eGFR by measuring his urine over 24 hours, then they likely estimated it from his creatinine levels, and so anything increasing creatinine levels would presumably cause the eGFR value to drop whether or not the increased creatinine levels are due to renal function or anything else.
Also, creatinine is measured in serum to estimate gfr. Urine creatinine is more used as part of a test to look for nephrotic syndrome.
Creatinine is produced constantly, filtered and not resorbed, without active secretion, which is why it is an excellent proxy for glomerular filtration rate.
The article doesn't give many details, but it looks like changes in diet can be dangerous.
Of course changes in diet can be dangerous (you just try switching to an all cyanide diet).
But "liquid protein products"? Like milk? Or protein shakes? There are certainly plenty of "liquid protein products" on the market, and heavily used, and tons more protein products sold as powders to be mixed with water, milk or juice by consumers.
Somehow we don't have weightlifters dropping like flies.
It is far easier to stomach large amounts of protein (due to taste and consistency) when mixed with a lot of carbs than alone, and unless you're cutting you generally need a lot of carbs too when training hard, so there's little incentive for most general protein supplements to keep carbs down.
In fact, there are a large number of protein supplements on the market where the majority of calories comes from carbs and fat, and many will have more carbs alone by weight than protein (many often also are specifically advertised for weight gain / bulking).
They are still commonly referred to as protein products because the main basis for calling something a protein supplement amongst suppliers and people using them is generally large amounts of protein, not low amounts of carbs and fats. (And yes, that means some of them - intentionally - have massive amounts of calories)
You also probably talk about "pure" milk, but there's also a number of milk products on the market that are milk + extra casein extracted from milk, for very high protein concentrations - my favourite has about 40g of protein per 300ml; you are free to not accept that as "milk", but it tastes like milk and is generally marketed as "protein milk" or similar.
But milk alone is also frequently used by weight lifters for the decent protein content and good ration of protein, carbs and fat (a well known "supplemental diet" for bulking for lifters is called GOMAD: Gallon Of Milk A Day)
The main point being that large amounts of protein is being ingested in liquid form every day all around the world, and has been for decades, and we're not seeing mass deaths.
Barring actual evidence to the contrary, there's very little reason to believe that the deaths referenced in the article had anything to do with getting "liquid protein", and very good reasons to believe it was hysteria.
He used
> Finally, I took some tests on Quantified-Mind.com to measure my mental alertness while I was eating my typical diet of burritos and Diet Wild Cherry Pepsi. In this way, I could try to reproduce Rhinehart’s claim that Soylent improves mental acuity.
Those tests tend to exhibit substantial practice effects until you've done them a lot.
And then there's the obvious placebo effect. I present to you a training regime which due essentially to an expectancy effect by subjects can increase your IQ by 9 points... or 2 points: http://www.gwern.net/DNB%20meta-analysis Quite a difference.
Sell people a green pill and tell them it'll make them want fast food less. Done.
Make it more dramatic and you'll get a better effect. Expensive helps, requiring it to be injected would help even more.
Benefits are not self-assessed for drugs.
Imagine if there was a new drug that gave the mental benefits the guy reported after two weeks of use and saved you money too. It would be front page news.
The guy doesn't claim to be a medical expert, he just did his best to report the results of a bunch of miscellaneous tests that were suggested to him.
With a drug for which if you took 40x the daily dose you might die, or a drug that might cause significant harmful side effects at normal dose, you need careful studies to prove a benefit and quantify risks. With a dietary change, the risks are orders of magnitude lower. I'd be surprised if the average American wouldn't improve their nutritional status doing this for a while, with an existing diet likely low in magnesium, copper, zinc, vitamin c and d and probably more.
The need for careful studies is greatly reduced for a food product like this one. People can try it, report how they feel. If other think is worth trying, they can see exactly how it works for them by giving it a try. That is the power of self experimentation.