I would be willing to pay an annual fee to have access to well-curated search results with all the clickbait, blogspam, etc. filtered out.
Until then, I recommend uBlacklist[0], which allows you to hide sites by domain in the search results page for common search engines.
This gives me the idea to build a search engine that only contains content from domains that have been vouched for. Basically, you'd have an upvote/downvote system for the domains, perhaps with some walls to make sure only trusted users can upvote/downvote. It seems like in practice, many people do this anyway. This could be the best of both worlds between directories and search engines.
You could counter that somewhat by having a "people who liked X also like Y" mechanism, but that quickly brings you back to search bubbles.
In that sense Google probably should/could do a better job by profiling you and if you never click through to a page lower it in the rankings. Same with preferences, if I am mainly using a specific programming language and search for "how to do X" they could only give me results on that language.
In the end that will probably make my search results worse, as I am not only using one language ... and sometimes I actually click on Pinterest :-(
Unfortunately, this doesn't entirely solve the problem. Counting clicks doesn't work because you don't know if the result was truly worthwhile or if the user was just duped by clickbait.
As you say, clicking when they stop trying sites is better, but I don't know how good that signal to noise ratio is. A low-quality site might answer my question, sort of, perhaps in the minimal way that gets me to stop searching. But perhaps it wouldn't answer it nearly as well as a high-quality site that would really illuminate and explain things and lead to more useful information for me to explore. Both those scenarios would look identical to the click-tracking code of the search engine.
And new clone sites launch all the time, so I’m always clicking on search results to new clone sites that I’ve never seen before so can’t avoid them in results.
Just giving us personal blocklists would help a lot.
Then if search engines realize most people block certain websites they could also let it affect ranking.
Problem is people usually want one general search engine, not a collection of niche ones.
Problem is people usually want one general search engine, not a collection of niche ones.
In my opinion, the reason they want a general search engine is that they think in their box (search -> general search). What they really want is a way to discover things and quick summaries about them: “$section: $what $what_it_does $see_also”. Search engines abuse this necessity and suggest deceitfully summarized ads instead.
It's very possible. In general Google will penalize you for duplicate content, but that might not apply to code snippets since code is often re-used within projects or between projects.
The code snippet sites also typically have 2 or more snippets on the same page. When combined, it might then look unique to Google since their algorithm probably can't understand code as well as it understands natural text. Just a guess
Between the constrained syntax, literal keywords, standard APIs and common coding conventions it seems like even independently-authored source code listings will be superficially similar. Certainly the basic/naive markov-chain-style logic that works really well for detecting even small examples of plagiarism in natural language content isn't going to as effective on `public static void main(String[] args)` or whatever.
Obviously there are strategies that could distinguish superficial/boilerplate stuff from truly duplicated code, but between the volume of actual (and often legitimate) duplication on the internet and the (maybe?) low ROI for duplicate content penalties for code relative to other value metrics/signals, maybe this just isn't terribly important as a quality factor?
ABC news, the NYT, WP and the WHO are considered high profile but the topic is not their area of expertise. Non of them would consider themselves the go-to place for people searching for it.
If only... it seems like the search results have gotten far worse for exact-matching. I regularly search for part numbers and markings, somewhere where exact matches are pretty much the only thing I want, and I can clearly see the decline over the years as it starts including more and more results that don't even have the string I'm looking for.
Searching just now I do see some other sources making a similar claim, so maybe I'm just out of the loop. But in my cursory scan I haven't found much detail or real evidence beyond "google says they matter" either. I mean, that's not why those internal links were created in the first place, but it sure would be nice to get a ranking boost out of them too. I wonder what I'm doing wrong :)
It's beyond simple for Google to fix. Just drop those sites from the search index. But Google won't do that because it's in their interests to send you to those shit holes because they're littered with Google ads.
One thing Google has gotten really good at lately is knowing that when I search for "John Smith" I mean John Smith the baseball player not the video game character or vice-versa.
GH also breaks down the results into types which is very helpful when you only want code or are looking for documentation or discussion.
Offtopic but I'm curious why this is the case? Is the Free Software movement responsible for this mindset?
With knowledge, the problem can be worse. You can't even evaluate if it's any good until it's been given to you. At that point, you don't need to pay for it because you have it. The number of paid knowledge things that I see that don't really have good information and avoid all the real problems that they promised they were going to solve for you can be high.
I think sites can build trust with users, but it can mean creating a lot of free content in the hopes of converting some people to paid users. Of course, if that model is too successful, then there will be an army of people dumping free content hoping to do the same, but then your paid content is competing with the free content available from many trying to do the same business model. If Bob creates a business doing this, do Pam, Sam, Bill, James, and Jess try to do that a few months later which then means that the amount of free content is now 5x what it was and there's no need to pay for it because it'll be a freebie on one of those sites?
They won't do it.
I've had multiple programmers literally shocked and avoid my outstretched book. Once I got a question, said "I just read the answer to this in this book right here"... and the programmer refused to read the book to answer his question.
I don't get it.
This, coupled with companies lack of investment in their expensive engineers, is mystifying.
None of the above has anything to do with the FSF.
I dunno, I’m not really a reading type but I do own programming-related books. It’s the only type of book I own. I learned a lot from books like The Clean Coder, The Pragmatic Programmer and some 90’s book about Design Patterns with c++ examples and I don’t even write c++
The recommendations I got from lecturers at university were thick academic textbooks. The 'good programming books' popular on HN etc. - without taking anything away from them - tend to be more waffley blog-turned-book type things.
Aside from classics from the former category that are also popular in the latter. Sipser, the dragon book, Spivak, etc.
Best example that comes to mind is the first edition of SQl in a nutshell. It was thin reference that covered SQL and the major differences between different SQL databases that I picked up back in 2002 ish. Quite useful if your working with several at the same time, but not something to teach you SQL. I haven’t looked at a recent edition because I am not in that situation any more, but I presume it’s still useful for that purpose.
Granted most books are significantly more verbose, but the physical limitations of books can promote extremely precise writing with minimal extraneous information.
So as far as I see it, the problem is not that the content is uneconomical to produce. The problem is that searching doesn’t discover the content I want. It brings up geeksforgeeks or worse.
If I pay for something else, I don't get source code, I get borderline malware that spies on me constantly, I'm subjected to idiotic non-negotiable terms and conditions and I'm at the mercy of some gigantic corporation that can kill the product at any time.
We don't pay for "something better" because it's not actually better at all. We're not laymen, we actually understand the fact it's a shitty deal. We're not some mindless consumer companies can just dump these products on. We have the power to create. The more we use this power, the better off we are.
Also if you code for fun anyways, you might as well build what you need, and get chance to use that shiny new technology while you do it. You save money, have fun, improve your resume, share projects with your friends and communities for kudos, all at the same time.
Reinventing wheels out of plain curiosity has exposed me to a variety of problems and their solutions, and I believe it exercises my general problem solving skills.
1. How to use that tool effectively
2. How to build better products with it.
You are never done learning those. And the best way to learn is to (at least try) to build it yourself. Therefore I think it makes sense for programmers to try to build it.
It's about freedom to access, modify and distribute the source code.
I've also seen Bing & Google citing StackOverFlow and the replies in SO awarding or agreeing on a solution comes straight from this "text book" "The Gentleman's Guide To Forum Spies" https://cryptome.org/2012/07/gent-forum-spies.htm
Perhaps it would be useful to dig into a posters history on a site and then decide who you trust instead of just trusting a random on the internet?
How many people have download code from SO into VS and found it doesnt even do what its purported to do? I've seen plenty of that.
Resource Burning the population, in this case programmers, is a perfectly valid technique for a variety of reasons, but the main one being, you are stuck in front of computer and that means you cant get into mischief away from work. Religions have been using that technique for hundreds of years and colloquially its know as "The devil makes work for idle hands to do" or something to that effect.
Choose carefully what you want to believe and trust.
Well, that's also the side effect of taking Cunningham's Law to heart, which says "the best way to get the right answer on the Internet is not to ask a question, it's to post the wrong answer."
I wonder if there's an opportunity for a paid bundle of programming-related sites. I indeed will not pay for a single site (nor for a news site for that matter), but a $10-20/month subscription that covers quality programming websites could be interesting.
I think part of the problem is that the content producers are trying to cater to everyone: google algo (by artifically inflating word count, using specific keywords), beginner programmers, advanced programmers, potentially paid users, ephemeral users that arrive on the site via a referral or via googling. In the end you end up catering to no one.
As a side note, RealPython.com is going to go down even more if they're going to keep their "register to view" policy I've started to see recently.
I don’t like the SEO industry very much.
If you could create a site with both interview prep and quality code tips I'd pay for it.
- The search results for programming content has been very volatile the last year or so. Google has released a lot of core algorithm updates in the last year, which has caused a lot of high-quality sites to either lose traffic or stagnate.
- These low-quality code snippet sites have always been around, but their traffic has exploded this year after the algorithm changes. Just look at traffic estimates for one of the worst offenders - they get an estimated 18M views each month now, which has grown almost 10x in 12 months. Compare that to SO, which has stayed flat or even dropped in the same time-frame
- The new algorithm updates seem to actually hurt a lot of high-quality sites as it seemingly favors code snippets, exact-match phrases, and lots of internal linking. Great sites with well-written long-form content, like RealPython.com, don't get as much attention as they deserve, IMO. We try to publish useful content, but consistently have our traffic slashed by Google's updates, which end up favoring copy-pasted code from SO, GitHub, and even our own articles.
- The programming content "industry" is highly fragmented (outside of SO) and difficult to monetize, which is why so many sites are covered in ads. Because of this, it's a land grab for traffic and increasing RPMs with more ads, hence these low-quality snippet sites. Admittedly, we monetize with ads but are actively trying to move away from it with paid content. It's a difficult task as it's hard to convince programmers to pay for anything, so the barrier to entry is high unless you monetize with ads.
- I'll admit that this is likely a difficult problem because of how programmer's use Google. My guess is that because we often search for obscure errors/problems/code, their algorithm favors exact-match phrases to better find the solution. They might then give higher priority to pages that seem like they're dedicated to whatever you searched for (i.e. the low-quality snippet sites) over a GitHub repo that contains that snippet _and_ a bunch of other unrelated code.
Just my two cents. Interested to hear your thoughts :)