The 'problem' with modern science is that a large finite amount of stuff has been discovered, and new discoveries are most often built on that stuff. I splash around in the backwaters of machine learning, however I have to pick my targets carefully and maintain a very tight focus - the defined knowledge base has already grown to more than anyone can ingest in detail. Given the amount of knowledge that has to be absorbed to achieve mastery in a scientific discipline, and the fact that knowledge base keeps growing, it takes longer and longer to lay a foundation for just understanding what the hell people are talking about. Now, there are opportunities, for example the use of genetic algorithms as part of ML solutions, because for any 'real' researcher, GA's aren't the cool kid in the opinions of reviewers or funding agents. It's easier when you are the PI and the funding authority, even if the budget is quite a bit smaller

The problem with modern science is that there is no room for the unusual.

Faraday had no formal training, but his natural intuition, interest, and tenacity made him standout.

Newton was a brilliant, paranoid asshole.

Instead funding goes to credentialed career scientists whose greatest ability is self-promotion, fund raising, and stringing-along the public.

As an example: The next big particle accelerator sucks up billions; while alternative approaches to QM never get any attention because it’s a guaranteed way to kill a career and become a pariah.

So nobody is available to even try to create the theoretical framework at the investment of a few million.

There is still plenty of room for the unusual, it just isn't as easy as it was back in the day when most important discoveries were made by people working by themselves at home with little organizational funding.

It's a nice narrative that overgrown institutions are ruthlessly repressing all creativity. And I do believe there is some truth to that. We should work hard to understand why and then improve the situation. However, unfortunately, reality is usually more complex and also more mundane (in some ways) than any nice narratives we can come up with.

From my experiences in the life sciences going through the academic credentialing process (PhD to postdoc), there was room for the unusual but the way it worked was a little convoluted. Basically the grant funding agencies give you money for a project that they can understand and follow your logic on why it will succeed. Then when they fund you, you cut back on the resources required to get to that success and spend the savings on new ideas. The fun part about the "new idea" spending is you can look at most anything that your instrumentation can look for. The idea being all the tools in your lab, your departments lab, even collaborating institutions are available to play around with and probe. You can even build new instrumentation to look at new things with that money. This is how modern life science pushes ahead.

Or the other option is you get funding for an idea that falls within the same realm as your unusual idea. Then you spend the money on those overlapping projects.

A great example is a chemist I knew that love research with selenium (an uncommon element). He was most interested in what role it plays in organic chemistry. That’s it.

But when he wrote up grant proposals it was always about the anti-cancer properties of selenium compounds. Never mind the fact he had zero plans to actually pursues that end.

So what happens if you get caught? Or someone calls you out?

Nobody's checking. The focus is on what you're going to spend next year's money on, not how you spent last year's money. And if you're savvy, you can use tools that were already bought for some other purpose, or that don't cost much.

Working a day job in industry isn't categorically different, except that someone is probably watching your spending more closely. You have to figure out a way to set aside some time to work on your own interest, whether you do it at work or at home.

As for money, you can get technology made for 1/10 of what it costs your employer, by choosing your battles, cutting out all of the overhead, and using free stuff.

So I moved into industry out of post doc and I can tell you as long as you're getting your day job done and it isn't that expensive you can test most any idea. I'd say it's even easier to do it in industry because "not that expensive" to industry is like 10-fold more than in academia.

Except that you’re not allowed to be bored and mentally daydream.

Newton and Einstein both hit on some big ideas during lulls (@home to avoid plague for the former; patent office work for the latter).

In Newton's day a single person could probably learn all the math and science there was to be learned in the entire world.

Now things are much more specialized and require years of formal study just to get a base level of knowledge in one tiny aspect.

I doubt this is the case at all.

We have the luxury of hindsight to know which peculiar schools of thought were rubbish, so we don’t even teach them.

I’m sure there are many, many theorems built up with Euclidean Geometry that we don’t bother teaching anymore.

Because modern methods make the results trivial.

Today, what are we burdening upcoming scientists with unnecessarily?

Part of it has to also do with science becoming more empirical.

The number of unanswered questions grows even faster as more questions are answered.

Good questions are even harder to find than good answers.

When you come up with a good question, make sure lots of people hear it.

> a large finite amount of stuff has been discovered

https://en.wikipedia.org/wiki/Lists_of_unsolved_problems

Neural networks are a good example of something that was once ignored as an academic curiosity at best.

Neural networks followed a typical hype cycle — before they were ignored they were the next big thing.

Then they got ignored, and then they finally got useful.

It's worse than this. The careerists have taken control and use it to demand you work as they would like. They kill the creativity and passion and will not allow the real work to happen and make progress.

There is a formula you must fit into, 5 days a week, probably 50 hours, a specific attitude, and everything they can get from you. Try this: in your next offer negotiation, ask to cut your salary by 20% and get one day back.

Can't blame them. It is how Tesla and many other inventive minds have been made subject. It's not slavery but it doesn't honor the contributor acting in good faith.

> The consequence of this is that these fields are filled up with careerists - people who are socially driven rather than curiosity driven.

I think it's worse than this: school and academia is no longer the sole option for the ultra curious. Curiosity is better fed by the internet, which means we end up lacking a social institution which captures and unifies people like this.

Isn't the internet the social institution then?

There's a much more obvious difference between now and 100 years ago: 100 years ago the education and financial support for scientific research was largely a closed shop for European-descended upper middle class men. There are certainly many geniuses obsessively researching into areas that stimulate their intellectual interest today that wouldn't have had the opportunity to do so back then.

I'm not sure being a scientist was that unfashionable 100 years ago anyway: research labs were a thing, academic work was arguably more prestigious than it is today, fortunes were being made from inventions especially in fields where there were low hanging fruit, competitions and societies and exhibitions to celebrate scientific achievement had come into existence and the idea of inventiveness was even tied up in popular contemporaneous notions of national and racial superiority. There might be more subfields and more research to build on nowadays, but the stereotype of scientists being underpaid eccentrics certainly hasn't gone away and nor has the fact a mathematics prodigy can make a lot more money working in financial services than academic research.

I'm also unconvinced scientific progress has slowed because today's geniuses are working on interpreting our genetic code, modelling solutions to climate change and solving scaling problems in computation rather than drawing taxonomy diagrams, inventing new types of consumer electronics and breaking the land speed record.

> 100 years ago the education and financial support for scientific invention was largely a closed shop for European-descended upper middle class men.

i.e. exactly the people who were, by far, most likely to be living a comfortable, stress-free life back then. We should not underestimate the sheer amount of material progress and economic growth that has occurred since then. A lot more people can have the luxury of getting playfully obsessed about something than could back then.

800 karma in 73 days. I wonder what the highest karma gain in two months was, in HN’s history. (From comments only, and with the top comment removed from consideration to avoid one-offs.)

I guess I’m a bus ticket collector of HN facts.

tptacek has 323873 since november 2007 [0] or an average of 2,249 per month so probably quite a bit higher.

[0]: https://news.ycombinator.com/user?id=tptacek

I probably should have specified “besides tptacek.” He’s a fascinating outlier. He once claimed he was worried some Russians were going to harm him due to his HN comments. I still wonder if he was just saying that to sound cool, but truth is often strange.

That average does put things into perspective though.

I said what? If this was in person, I'm confident I was both drunk and joking.

Hey, please don't swerve into personal attack here. Friendly banter needn't turn ugly.

https://news.ycombinator.com/newsguidelines.html

How did you read 'hate' into that? Genuinely curious.

As an aside, I recently feel at odds with the language many people are using. The mere word 'hate' seems to have grown weird political connotations, while is suddenly okay to hate 'hate'. Is this not fostering the very emotion you revile, under twisted pretense? I remember 'love thy enemy' to have been the twist of the knife in peaceful protest, not sinking to their level with 'justified hate'. It feels hollow. If we want a change, we need to reach out first. /rant

There is a trend to demonize the Chinese and Russians governments (it is always about governments but then it translates to companies and then to people). This trend is very useful to promote different kinds of online censorship and put people into political boxes (e.g., you like this then you are that). Basically, when every single piece of material about a country is negative, you get a set of feeling where hate is not the least common. Then you go full paranoia cycle and expect that these same people also will punish you for what you write about them. One can benefit from these tendencies and ride them to get attention and respect of like-minded people, basically sell hate.

I’ve consistently gained karma faster than that. By my estimate, there’s still a handful of people who are currently accumulating karma even faster than I am (one of whom has been mentioned here), though I expect to only really be sure in a year or so.

I would say:

100 years ago the education and financial support for scientific research by European-descended upper middle class men

was a thing.

50 years ago I wasn't expecting it to come back, it wouldn't have helped me anyway, overall scientific opportunities were far worse by then and the trend has continued. I wasn't waiting around for an uptick, I just started right away putting in the effort to try and compensate.

And I agree with this completely:

>There are certainly many geniuses obsessively researching into areas that stimulate their intellectual interest today that wouldn't have had the opportunity to do so back then.

There is so much brilliance in so many cultures through so many millennia, it can be seen that education and financial support are not even essential for genius, mainly for documenting, recognizing, and leveraging the influence of a very very small percentage of geniuses through history, only those few whose works were preserved and/or applied.

Surviving largely within a system of lesser thinkers whose works were better preserved and/or more stongly applied.

Which is why I think

>inventing new types of consumer electronics

can be a good thing if the consumer is given careful enough consideration.

Science has a level of technical difficulty and detail to not be obsessed about something in order to discover anything. Try one of those extract DNA from a strawberry kit for example. Even in big data science you find people who know a lot of math or applied math that have the technical insights. A lot of science today also requires collaborative cross discipline work to make progress as well. I don’t think careerism is an issue. We should encourage wacky ideas though through competitive grants.

The people driven by curiosity will still be in those unfashionable backwaters of any field. I would think true progress is made by those individuals, not ones that chase ‘hot’ fields.

Bingo. I would add that the movers tend to define their own fields, and by action and driven by need/practicality, not marketing. Those seeking to enter a career, generally have a different set of goals and priorities, reguardless of overlap.

> In the past, science was an eccentric hobby, not particularly valued and certainly not a viable career option. Today it's all of these things - like becoming a doctor.

Really! That's news to me. Are you aware of how much scientists are paid and their general career prospects?

The average tenured professor salary in the US is $141k. Star research directors with a proven record of bringing in grant money can easily earn two or three times that. In engineering, physics, med, and bio there are also lucrative consultancy opportunities.

The average scientist outside academia is probably doing grunt work and is paid badly - unless they're working in fintech, or something with an obvious financial upside.

The reality is that the entire research system is optimised for direct and indirect cash accumulation, not for genuine innovation or invention. There's some interest in blue sky funding, but if you're a fresh postgrad no grant body is going to give you a lot of money to go off and design a working warp drive unless you also have the bureaucratic skills to make your project sound like something they want to fund - probably for other reasons.

Researchers with good bureaucratic skills and genius-level scientific insights are exceptionally rare. And the publishing system isn't welcoming to talented semi-outsiders

Newton, Einstein, Darwin, and Ramanujan would really struggle in today's environment. Newton might be okay if he managed to get tenure, but the others not so much.

The big difference is that currently scientists are paid and their research equipment funded by someone else; contrasting to the earlier times where usually you could be a scientist if and only if had "passive income" (usually, inherited wealth), so that you would not need to work to earn money and could instead study (and pay for studies in a world without student loans) and research instead of that.

>Telling your mother you want to be a scientist is far better than saying you want to be a musician.

Are you joking? A small child literally has a better chance of growing up to become a professional athlete than a tenured research professor!

This doesn’t seem true.

In 2016, American universities hired about 20,000 tenure-track faculty [1]. Adding up the major pro sports leagues, there are very very generously 10,000 American pro athletes, (i.e. people who don’t have day jobs).

I guess things change if you include all the minor league and farm teams, especially for the MLB.

[1] https://www.insidehighered.com/news/2018/10/12/about-three-q...

Of course very few tenured research professors grow up to be professional athletes, but that hardly seems relevant.

> filled up with careerists

I agree but I think it is driven by academia's incentive structure which promotes and rewards such behavior.

thats why we have bad doctors?

The BIGGEST problem with research is this: it's HARD, not easy, to understand. Most papers written are absolute trash. Not because the content is. Because it's written in a shitty, overly braggy way (especially mathematics and physics), that's mostly shouting "I'm better than you and if you don't get it, you're an idiot". They are not written with any USER, let alone, READER, in mind. Anyone would immediately be fired by a remotely consumer-centric company.

Wikipedia was a huge step in the right direction: making everything easier to understand. With lots of proofs and examples.

I think you could easily become billionaire by improving Wikipedia and the "make science and knowledge easily absorbable" 10X easier

Please don't use uppercase for emphasis.

https://news.ycombinator.com/newsguidelines.html.

That's one very interesting observation. I've commented in that vein a short while ago where I noticed that once I finally understood what some paper was about my usual response (definitely not always) would be 'That's it?'.

Wikipedia has been a godsend for me, to be able to understand core principles without having to wade through what seems to be obfuscated English in order to hide something relatively trivial at the heart of the document.

Wikipedia for certain topic is often just as inscrutable as papers would be. I have no idea why this is, but my guess is that it’s written by people with passing interest in the field who lack the experience to effectively distill their knowledge like paper authors might.

You could not "easily" become a billionaire by improving Wikipedia, which is the result of millions of hours of writing and editing. The fact is most low-hanging fruit has already been plucked, thus every scientific discovery relies upon more and more background knowledge. Yes, we do make innovations in explaining/teaching science more quickly to successive generations, but I doubt if we'll ever see a Newton or Darwin who can single-handedly, obsessively write and observe and calculate, by themselves, and then push science forward by leaps and bounds. The best research today is all done by teams, with experts on statistics, study design, clinicians, hardware experts - there are just so many niche fields that we MUST collaborate on extremely advanced work.

I was actively discouraged from writing readable text in my thesis. That kind of behavior would erode the power of the guild.

Your advisors were acting in your best interest. There is a time and place for readable science, and your thesis is not it. It has nothing to do with the power of the guild, and everything to do with assessing your knowledge and preparation.

The purpose of your thesis is to demonstrate to them (and perhaps the larger scientific community) that you can communicate to other scientist in the language of the field, that you possess the requisite knowledge, and that you are prepared to advance that field.

There is no guild, but there are gate keepers (reviewers of various sorts) and you must be prepared well enough to make convincing cases (for publication, funding, etc.) Your advisors were training you for this role.

Well, so sorry. If things are as bad as you sketch them then there may as well be a guild.

Science is first and foremost about understanding, and writing in a way that purposefully obfuscates and makes it harder to understand what is communicated is anathema to true science.

You misunderstand the purpose - it's not intentional obfuscation. Its the lingua franca of the field. A technical term can define in one phrase an entire concept that would be tedious to spell out each and every time. It can define one 'chunk' that you can then combine with other chunks to develop deeper understanding. Surely you can agree with that?

While I fully agree that ability to communicate science to the general public is incredibly important, the thesis is not necessary the place for this. Plenty of other places are (blogs, twitter, etc) and this ability is crucial for a publicly funded scientist. General large conferences that I am aware of often encourage non-technical translations of abstracts.

I don't think that's what 'jacquesm is talking about. It's not about the technical term that can communicate a lot of meaning in few characters. It's about constant use of obscure technical phrases that communicate the same or less than a plain-language description, except you have to work to decipher it. It's obfuscation, in a sense similar to what a JS obfuscator does to readable code.

I long for voice, for playful humor, for that je ne sais quoi of good writing! Robert Anton Wilson wrote a marvelous book called Quantum Psychology, where he uses the insights of quantum physics to upend the prevailing Aristotelian view of is / is not logic in psychology and in scientific thinking generally. It was a genuine pleasure to read, as I sensed the intelligent, interested, living being doing the writing all throughout. It felt intimate despite its serious nature. I just don't understand why writing about a scientific subject, even in an obscure and rarified field, gives you a pass on crafting a piece of writing someone might actually want to read, might actually connect with on an emotional level.