In those days, we would usually read the manual when we needed to dig past a surface-level understanding of how things work. The manuals are dry and dense, but they're much easier to read with modern niceties like "Ctrl+F".
IMO, this is one of the reasons that people recommend using software with a long track record. If you have a question about some parameter in a systemd service script, and the internet doesn't have a ready made answer, the details are all written down in the manual.
https://www.man7.org/linux/man-pages/man5/systemd.service.5....
Plenty of modern tools have comprehensive detailed docs like this. Python, Go, even most widely-used JS frameworks. The primary sources are often downranked in search engines because they don't sell ads, but it's a good idea to find and bookmark them when you start working with a new technology. RTFM!
Then, after that didn't work, we had to go to find the weird bearded guy that didn't like to talk to people, but knew everything. He would usually tell you the answer, but in a condescending way that made you understand that you only came to him in "emergencies" and that you were kind of stupid for asking an "obvious" question. Actually, now that I think of it, that is stackoverflow now!
Thank god message boards started taking off! ... but sadly, then the "real" Internet came and killed it. :-(
"Why would you want to do that?"
"What's this for?"
"Did you try searching before asking such a stupid question?"
OP bemoaned how people on StackOverflow would tell you not to do what you're asking about. Reference material is not the cause of that "weird bearded guy" problem, but it is one possible solution.
We're talking about software developers, not wizards. It's not like you have to offer a dram of blood and draw a pentagram with salt before you approach the ancient grimoires.
On the good threads that become helpful, yeah.
On most of it, no, those are not the questions people ask there. They will focus on unrelated marginal issues, try to refuse to answer the question, just assume the reason the person is doing it (even when explicitly told on the question), and just throw wrong answers on the wall to see if they stick. (Granted, throwing things on the wall is a useful way to answer some questions, but not all.)
It's good that Google rewards the first kind of thread, but the second one is what sends people away from the platform.
Q: "I'm trying to configure Foo to produce Bar, but it's giving me this error!"
A1: "What are you really trying to do?" [Unsaid: I don't know how to produce Bar either]
A2: "You should not be trying to produce Bar. Instead you should produce Baz. I know how to do that--follow the following steps..."
A3: "Producing Bar will not solve what I imagine your goal is. In the general case, you may need to produce many different results, which I would rather answer about..."
Q: "Uhh, thanks everyone, but I'm just trying to configure Foo to produce Bar."
Many such cases!
The Elixir community talked about this a lot on the forums back when I was active on there, because language rankings look at Stack Overflow questions and Stack Overflow questions proliferate in languages with footguns. "Don't use obviously_named_function(), use obscr_wrd_fcn() instead" fits the SO formula quite well. Well designed languages, Elixir included, don't produce many of those kinds of question/answers. That leaves significantly more room for discourse around architecture and other higher-level conceptual stuff, which doesn't really have one correct answer, and thus is ill-suited for SO, being relegated to the lower-discoverability forums (and worse, the un-discoverable Discord).
Back then, it seemed a lot like that!
>"Why would you want to do that?" because it seems like the right thing to do? Is there a better way.
>"What's this for?" the boss wants it
>"Did you try searching before asking such a stupid question?" I read the manual and thought about it a lot (remember this was pre-Internet)
so, there was a lot of "rough" hand-holding to learn the ropes for edge cases.
The thing is... languages were smaller because they hadn't started the accretion of thousands of libraries and frameworks.
In my experience, every language out there is somewhat easy to learn and master. The ecosystem around it is an insane and ever-growing Katamri Damacy of (largely) crap. We all must know it, must use it, and must contribute to it because no one trusts the work of the individual and only the work of an aggregate of individuals... people often don't even trust their own code.
I wouldn't make a blanket statement either way, but there are certainly counterexamples to this:
- The mpv manual[1] is a work of art.
- The Arch Linux wiki[2] is a treasure trove of information for not just Arch-specific topics, but Linux in general.
- MDN[3] is the defacto standard for any web documentation.
- The PostgreSQL[4] documentation is quite thorough and high quality.
What I think backs up your point are two things:
1. There's just a vast amount of software since those early days. "Software is eating the world", and it's realistically impossible for most of it to be well documented.
2. A lot of information is spread out in books from users of the software, on blogs, tutorials, forums, videos, etc. Sure, this might be seen as a failure of the software authors to produce good documentation, but many of these resources didn't exist before the web made them accessible. In some ways this is better than having a single source of reference, as you can benefit from the collective wisdom of the hivemind, rather than only from what the author thought relevant to document.
[2]: https://wiki.archlinux.org/
Granted, there's a lot of chaff in books as well, but their quality to junk ratio is wildly better than Google, and they generally go much deeper.
Only problem is that pesky copyright that makes you buy them
It's the nature of the incentive system that they set up. Their only grand vision for the internet was one where Google hit its growth targets every quarter. If you do the math on that, then Google has to be enormous in 2022. Guess what, Google is enormous in 2022, and the only way they could figure out how to do that was to keep growing their ad business. So here we are.
Free means we the users become the commodity.
Our imaginations have run out because those sweet ad dollars are too good to pass up.
Don't pollute the frigging index with SEO garbage. Google didn't tell everyone to SEO. People didn't even start doing SEO.
Then some arsehole went and had a brainwave about "what if we sold people on tweaking their pages to try to get all our customers fighting each other to show up on the front for the front page"?
I still remember my first time overhearing some SEO guys pitching to somebody while eating lunch. Spat out my food when it dawned on me what I was hearing unfold and started to realize that guy was likely not the only doing that.
The librarian's assistant in me had to be held back that day.
Why? Why would you pollute the Index that way? Monsters... Convincing people poisoning the town well was a good idea...
Google gave lots and lots of ad money to websites that did SEO, so they effectively did tell everyone to SEO - crippling their own search product in favor of their ad machine.
In this discussion, the people making the good content don't do it for the money, but the people creating spam do. Kill the money, and the commercial SEO crap goes away, and only the enthusiast content stays - as it was in the 'good old days' when simply getting clicks on your site could not get you any money.
> the people making the good content don't do it for the money
citation needed. there're two driving forces:
- money
- clout
they're interchangeable. if you have one the other becomes much easier to obtain. if the writer don't do it directly for the money, they're most definitely doing it for the clout.
Don't worry, it won't be long until we're lamenting that most of ChatGPT's output caps out at about the level a human could deliver.
I think this is due to the fact that the overall concepts are usually quite easy to understand and therefore easy to talk about.
As soon as you have to dig deeper you need to invest significantly more time to talk about it and your audience becomes really small.
This is amplified by people creating newsletters, content (marketing) in general, but the actual knowledge/wisdom is not given away. It's incredibly frustrating but happens in almost all technological field, especially when they are fast moving.
In mechanical engineering for example a lot of wisdom is also put into norms, standards and so on. Books are written about it. But 10 year old books are still completly valid while in software every couple of years a new framewokr is the new hot shit.
The more I look at it, the more it feels like software hasn’t had a fundamentally new idea in decades. Frameworks improve, networks get bigger, compute gets cheaper, and we spend most of our time dealing with domain modeling, cache, and statistical inference.
We’ve made great strides in the ease of building and encoded many lessons in our frameworks, scale is bigger too, but the underlying guts of software engineering have been very stable for decades.
And there are good books about software architecture and so on. Unfortunately, for each good one there are at least 50 that are really really bad. There are a lot of technology-related books as well, that teach you about responsive web design with react or stuff like this.
Encoding lessons in frameworks isn't always helpful either in my opinion. Your mental model of the software needs to be correct, otherwise it's just a really painful experience. What works for large scale software is often just pain for beginners.
I do feel like wisdom is inherently an "internalize for your own mind" process, and on some level, nobody can do that for you. So of course you will be scrabbling around trying to piece things together until it clicks in your own mind.
I've found having a discussion with another human, who understands the topic, is very helpful. And it is one case where an "academic" setting can be useful, assuming you can actually talk to the professor rather than being forced to parse some unfathomable textbook.
Even that example — a textbook — is similar to what you said. It is literally designed to help you understand a topic. And yet, my experience with them mimics yours. A lot of the material is uselessly general, and a lot of it is uselessly specific and technical. And it almost never answers the specific question you actually want to know at a given moment. So, you have to internalize the information (slowly) and eventually get there.
Maybe there is an "event horizon" of me being able to absorb new material, rather than something external.
Partly, because so many developers famously hate writing documentation and hate commenting their code.
If developers followed Donald Knuth's Literate Programming [1] then it would be a big improvement.
People rely on a lot of "tribal knowledge" without ever bothering to write it down, and out of all the fields, software development seems to be particularly notorious for its anti-documentation bias.
Example: it's 2022 and we still somehow upvote blog posts detailing one's stack as if that has any impact on anyone else.
(It's very impressive how it rewires the logic, but it can't really know anything, by design.)
And yes, that horizon is very real. It's what the internet rewards, so it's what almost everybody delivers.
When you google for some programming story you typically look for a story that describes how to combine concept A, B, C and maybe F. The more concepts you add the likelyhood of somebody having described a good implementation story for the combination of those concepts goes down. But you don't need to add many such concepts before the number of potential stories become very large. So for any non trivial number of concepts a blog article is not likely to exist.
We could only really change the game by going from text representation to intelligent representations. Rules or general AI.
However, my main surprise in all this: why do we need this many combinations of concepts in the first place? Why isn't the total domain of things that are interesting to do for utilitarian reasons more limited in size?
I view documentation as a set facts and stack overflow as additional color.
What type of projects require this workflow?
Dictionary says there are 3 components to wisdom: knowledge, experience, and good judgment. Machines can help with the first requirement, knowledge, to an extent but leave you frustrated (as you note). They can even help you dig out wise chestnut buried somewhere (as you did) but then you hit the other two.
This is how it is supposed to work: armed with knowledge, you apply this and gain experience, and after a few close calls the fortunate also develop good judgment, informed by experience of application of knowledge.
You want all that squeezed out how?
Most responses are there just to get your attention.
This whole thing runs on attention.
"Back in the day", this problem was addressed by having significant hurdles in order to publish material on a topic. E.g., books used to be the primary way to learn about a topic, and in order to publish a book you must 1) deeply engage with the material for quite a while simply to write the book and 2) become fairly credentialed / convince a publisher you're worth their time, etc. Same thing with academic papers.
The internet has completely removed all barriers to publishing, meaning it's up to us to sort the wheat from the chaff.
i wouldn't be surprised if a very large percentage of the last decade of tech blog posts are just remnants of someone's promotion, and is now hyperlinked on some resume, or linkedin page
I think this is only true if the commentors are SME's & operating in good faith. We've all seen whole threads hijacked by erroneous groupthink, while the 1 or 2 SME's were buried.
> meaning it's up to us to sort the wheat from the chaff.
I was thinking about this the other day. How do you sort out conflicting information when your not an expert your self. I don't know who to trust sometimes.
If you don't find an expert in the field and you publish a crappy book you tend to get a number of reviews (hopefully before you send it out to retail) that your book sucks.
When selling books about fake evolution you're not looking for an expert in the field, you're looking to see if you have an audience that would eat it up.
I've been feeling this more and more in relation to software. It feels like the more popular a piece of software is and the more people use it, the harder it is to find and talk to users of said software who have engaged with it past an extremely "shallow" level. I constantly encounter significant, easy to reproduce bugs and issues in very popular software used by millions of people, maybe even tens or hundreds of millions, but if I try to look for any sort of discussion about it online, I usually get nothing.
It feels much easier to discuss software with other real people who use it and think about how they use it when it's something really niche used by maybe a few thousand people at most.
My suspicion is that most people who engage with Unity are amateur engineers. Nothing wrong with that, of course.
What you’re describing takes a lot of time and nerve. Those are two of the foremost requisites of the pursuit of knowledge.
The Web is working. It’s enabling you to even access those old manuals via the Internet Archive, papers via Sci-hub, source code on GitHub and of course that old blog post on page 7 of a DuckDuckGo query.
The Web works y’all!
Wisdom comes from an awareness of the greater whole. The insights do not come from analysis, but rather, synthesis. It engages with the intuition rather than the intellect.
Although it can seem mystical, I think there are authors who have been able to express ways to engage in wisdom even if they are not directly talking about it. For example, Christopher Alexander has some interesting things to say about wholes, centers, and unfolding:
Seeing Wholes - https://iamronen.com/blog/2018/01/14/christopher-alexander-s...
Centers - https://iamronen.com/blog/2018/01/08/christopher-alexander-c...
Unfolding - https://www.livingneighborhoods.org/ht-0/whatisanunfolding.h...
Also, many so called prophets can easily exploit this skill, which they might well do if that’s their personality type.
Just typing this out has made me think it’s almost like passing on wisdom and the religious experience are probably inseparable.
What's amazing to me about Christopher Alexander and his work is that he's able to describe the process of generating such art (in the form of architecture) to ordinary people, using plain, relatable language. The links I posted in the earlier comment are examples.
In other words, you don't need to be someone with a rare skill.
As a result these changes can destabilize a persons identity, which causes said person to look for the closest source of stability, often the 'peophet'.
This gives the 'prophet' enourmous power over the person, not just because of the identity destabilization, but because when you change someones fundamental assumptions about the world, those changes don't nessecerily have to be truthful or helpful for the person. The person only has to think the changes do.
This identity change and stability dependence is probably what causes the cult like appearance/ behaviour.
The next step in the process is to undo your abstractions to determine which of the previous solutions (if any) is the best one for your particular situation. It's not entirely identical, but I tend to think of intelligence and wisdom as deductive and inductive parts of the same process.
So I don't think that means that wisdom is so far removed from scientific enquiry that it becomes mystical, far from it: instead, it requires so much more rigour and discipline to codify wisdom into laws that it just doesn't happen that much; and even when it does happen, the most you'll find is fuzzy frameworks on how to deal with certain problems rather than the hard and accurate rules you can find in deductive analysis.
However, wholes are nested. The computer you are using to read these words, wherever you are, are part of a larger whole. Further, there is a paradox in which, while parts make up the whole, it's the whole that makes the parts.
Taking those all the way, "The" Whole in which all wholes are parts of, then, is boundless (no edge), and it is beginningless, (no causal origin).
My understanding of the scientific method is that it is ultimately limited in what it can find. It is not necessarily true that the scientific method is capable of explaining everything, though it is broadly applicable. That method is very good for analysis, but not synthesis, and focused on the origin in causal chains rather than the teleos.
Sure, one also needs understanding to get something out of search. But Google when was in it's sweet-spot, it could get a researcher extra knowledge and insight. After all, a researcher needs both a holistic perspective and information they'd know at the start of an exploration.
I was so tickled that I put it on my profile here on HN and elsewhere -- but all of those things I put down are different ways of saying "Wisdom".
Google has aspirations about organizing the world's knowledge, and making it easy to search for facts and knowledge. What I'm saying here is that people have never been able to find wisdom in a Google search -- and never will. Each person finds it within themselves through awareness of wholes.
So in my view, it is not that it's getting harder to find wisdom from a Google search, but rather, the proliferation of knowledge and facts over the years have increasingly made it even more distracting to find wisdom within themselves.
The Göögle Incompleteness Theorem
As someone who once thought this was true of my experience, yet have since learned they were just common phenomenon, I'm curious what these experiences were. And how do you think they have effected your perspective?
I don't have any experiences I would call "verifiably paranormal", but I've had a few I've not been able to fully account for to my own satisfaction from a purely materialist perspective (despite a total collapse of my childhood evangelical Christian beliefs).
One was sudden, total cessation of a destructive habit I'd struggled to stop for decades, after a single prayer I hadn't realized was related to that struggle ("Make me more like You"), a very unexpected turn of improbable events immediately following the prayer, and an intense spiritual experience following the unexpected events.
I have devised pure materialist accountings for it, but they aren't rigorous and haven't persuaded me.
I don't have any experiences I would call "verifiably paranormal", but I've had a few I've not been able to fully account for to my own satisfaction from a purely materialist perspective (despite a total collapse of my childhood evangelical Christian beliefs).
One was sudden, total cessation of a destructive habit I'd struggled to stop for decades, after a single prayer I hadn't realized was related to that struggle ("I don't care what it costs - make me more like You"), a very unexpected turn of improbable events immediately following the prayer, and an intense spiritual experience following the unexpected events.
I have devised pure materialist accountings for it, but they aren't rigorous and haven't persuaded me.
...all that said, I actually found the GP post very understandable and thought it made tremendous amounts of sense - perhaps even was "wise." I certainly wouldn't call it "bafflegab".
I should perhaps clarify that while I had the total collapse and found myself staring nihilism in the face, I currently am drifting towards a very heretical, Bible-is-flawed-and-maybe-not-divine-at-all form of faith in Jesus (in part due to the above-mentioned experience).
I’ve also experienced telepathy, seen a ghost (with multiple corroborating witnesses) and witnessed poltergeist activity. The last of those I can kind of doubt though. The mirror falling off the wall could maybe have been some kind of microquake.
It's good to see this surface up here, though it's the kind of stuff I discuss in other communities.
There are more things in Heaven and Earth, Horatio, than are dreamt of in your philosophy
Email is my profile, if you have the time and inclination to get in touch.
Given your background, I think you might find Christopher Wallis's Tantra Illuminated -- in particular, the chapter on the View of Tantra -- more sensible. Although it does not talk about wholes, centers, and unfoldings in those terms that Christopher Alexander does.
I think the key is in the non-duality of creator and creation. That within all parts are the Whole. I can't tell from the wikipedia page on Leibniz's monadology if monads are distinct from each other, whereas that stuff about wholes and centers point to apparent distinctions on a continuum that is the Whole.
wisdom, 1. in general, the best form of knowledge.
2. specifically, the intellectual virtue
or science concerning the first or
supreme causes of all things.
philosophical wisdom, intuitive knowledge combined
with scientific knowledge of
the objects which are
naturally the highest;
metaphysics, including
natural theology.
practical wisdom, 1. prudence.
2. any excellent form of
practical knowledge, as of
one of the architectonic
arts.
You see this a lot in recipes, and I've heard tales of military being susceptible to this, large manufacturing processes, etc where the tweaks are susceptible to being lost if large cutbacks/layoffs were to affect the people with that tribal knowledge.
Finding something more complex than few cm deep is becoming a hellish task.
I blame Google for lack of moderation and getting their SOE gamed like a little pipsqueek.
Juniors are dying happy coz of how fast they can find trivial answers.
Seniors get shafted. Its so bad that I started to build my own github repo woth various answers I needed over the years.
That doesn't happen for trivial topics or problems though! It's a sort of new frontier where we have to re-solve potentially solved problems because the solution isn't pervasive enough to be trivial.
We have to push ourselves to overcome a knowledge limit, which throws a nice fat monkey wrench into the idea of having "All human knowledge in the palm of your hand".
There's undiscovered territory out there and it's hard to see unless you get above the trees.
It succeeded in eliminating spam from a lot of small sites and instead resulting in an even bigger spam from the same big sites for every category. Great for people who were doing shoe-shopping at, well, Amazon, probably, but bad for everyone else.
Then they increasingly personalized search results via AI. And that changes your search results so fast that you may not get the same seach results for the same exact keyword in a week's time.
Im increasingly of the opinion that corporate, 'business mentality' should not run any major tech corp. It sees everything in numbers and extreme abstractions, juggling them to make a desirable false reality happen in paper and in metrics, screwing up the real world for every user.
Just before it hit, I had a small site that was on the rise in terms of traffic. A hobbyist site, no ads, all original content, and very much not "thin". An enthusiast site, much like the original internet. Incoming links were organic and spontaneous, from serious sites.
Panda crushed it. Massive pagerank downgrade and traffic decimated to about 5%. In mere hours. A false positive, I suppose. I spent 3 months trying to figure out the reason but nobody, including Google or SEO specialists were able to provide any tangible answer.
That day etched a few important lessons into my long term memory:
1) The size of Google's power. It effectively has the power to decide whether you digitally exist or not. The ultimate traffic controller.
2) Even a good faith move by Google may have 1% of false positives. Which seems totally rational until you realize that at Google's scale this could mean millions of people being totally fucked out of the blue.
3) Worse, when that happens, there's no accountability. You can't go anywhere for help, undo being a false positive, you're just randomly fucked and that's it. You cannot depend on anything ran by Google.
Although I'm fine and it wasn't exactly a pivotal moment in my life, it radically changed my view on Google. Early on, I was a fanboy. Google kicking lame Microsoft's butt with miraculous products like search, maps, gmail. I was cheering them on. As of my "lesson", I've grown increasingly cynical.
That was 10 years ago. They've become infinitely worse.
Yep. I also had various such legitimate sites. They all sank to oblivion. Worse, I had a small web development outfit - we used to put a link back to our site at the footers of the websites of our clients, like 'Designed by', or 'Developed by' etc. As then-advised by Google as good practice.
All of our clients' websites sunk. Small businesses, small ecommerce sites, blogs, open source project sites. All of them became 'low quality'. Not only their livelihood was sunk by Panda, but also our, the developers' websites were sunk because those sites were linked to us per what Google advised. We lost incoming business. Independent web development, small businesses, blogs etc that were thriving before Google murdered them were forced to use the centralized marketplaces - elance, upwork, amazon etc. Causing the consolidation of the internet and the rise of the algorithm.
Its amazing how no class action lawsuit ensued over that. Google literally backstabbing its own users who adhered to the best practices it advocated, and killing their businesses and livelihoods to amplify shopping searches for Amazon and other corporate brands.
> nobody, including Google or SEO specialists were able to provide any tangible answer.
It was because of 'low quality links' that were pointing to websites. Ie, small sites linking to your site. I remember clients fervently going through all the other websites that they asked to link back to them, and begging them to take off those links so their rankings would improve. Some of them succeeded and rose some in ranking, but nothing near enough to keep their small shops alive, leave aside near anything before Panda.
> 3) Worse, when that happens, there's no accountability.
Yep. One sociopathic old-school corporate exec and a host of totally disconnected corporate executives decided the fate and livelihoods of tens of millions of people directly. Maybe up to hundreds of millions of people if you counted in the business ecosystem that those small businesses supported. Literal feudalism, albeit exercised through private tyrannies without any accountability.
That made me conclude that any corporation that holds the livelihoods of people in its hands has to be tightly regulated by the the democratically elected governments.
> it wasn't exactly a pivotal moment in my life
It was a pivotal moment in the lives of countless people and their families. They lost their businesses, their livelihoods. Even worse, the Internet los the thriving grassroots businesses, communities and creative ecosystem that it had. It may be regaining it with the rise of the creator economy, but that will take time. Nothing can make up for the harm that Google did to the Internet and actual people's livelihoods.
You make a great point that as of then, the great centralization of the internet began. It's only heads and the tail is cut off. Social media accelerated it even further. This is why 90% of the internet feels dead.
On the plus side, that means if you're willing to dig through the research papers and other hard but useful sources, you have an opportunity to make a difference by writing that book.
Working on a honest straight reality is like drinking water, has no calories in it. Coke is better. Wines are more better, and everyone other than than I these days seems to be always on a magic fissile cocktails, and while I don’t get that, they seem well off doing it.
You see this on Reddit all the time. There will be a Q&A sub for a topic dominated by people with scant if any experience who think they're helping people by googling stuff for them and there will be a comedy subreddit for the same topic. Smart people ask questions in the joke sub because they know that having sufficient understanding to be amused by highly contextual stuff weeds out a lot of the casuals.
The cure for me has been two fold: 1) Focusing on books from reputable publishing houses (curated long form content) - but even that takes filtering. I noticed O'Riley has been putting out a lot of noise lately. 2) Focusing on open-standards, sound engineering. I'd rather focus on understanding hardware, operating systems, networking etc then myriad software as a service offerings full of vendor lock-in dark patterns.
Comparison of ChatGPT and Google on 500 coding queries: https://news.ycombinator.com/item?id=34089423
After that it always ends in hard to parse research papers, specifications, jumping around the source code of multiple libraries, debugging or reverse engineering. If I get lucky I might land on a 15 year old blog post that no man has seen since inception and it describes exactly what I wanted to know.
And that's just software. Humans know exactly what parts went into it and how it works. Why must wisdom always be squeezed out of a rock? Why is only the fluff copied thousands of times making research harder each day? Why are there never indicators which way to dig for more details?
T__T