News for hackers. Stuff that matters.
Re:The Web We Have to Save by sergiosgc on Monday August 03, 2015 @07:26AM (#9994999)
I really hope the current centralization of the web is an evolutionary step. At first, the web was decentralized with newsgroups as open as NNTP provided. Then, the lack of controlling features killed the decentralized protocols and medium: namely the inability to control spam and the inability to have participants pay for the cost of the medium.

Blogs suffered from the inability to provide a good reading platform, which is why the death of Google Reader was so mourned. GReader, not G+, was Google's best weapon against Facebook (with proper development, not stagnant as it was).

It is easier to add features to centralized architectures, but it does not mean the death of distributed protocols. I do believe a distributed publishing platform will be born sometime in the future. It is unthinkable that Facebook will still be the top destination, say, 20 years from now.

Re:The Web We Have to Save by Sideloader on Monday August 03, 2015 @11:00AM (#9996330)
Feedly does what GoogleReader used to do and it's based on the GoogleReader API. Unlike GoogleReader there is a subscription based premium membership available in addition to the basic ad supported service. I was an avid GR user and Feedly (basic) has served me very well after it was shuttered. (I am not affiliated in any way with Feedly. Have tried many RSS readers and Feedly IMO is the best of the bunch.)
Re:The Web We Have to Save by cxseven on Monday August 03, 2015 @11:45AM (#9996866)
One thing Google Reader did great was "sort by magic" where it somehow knew VERY well according to my individual tastes what article I'd be most interested in reading from my massive set of subscriptions. Feedly and other readers' auto-selected articles don't seem to have come close, but that may be a function of their reduced userbase and shrinking RSS activity.
Re:The Web We Have to Save by sombremesa on Monday August 03, 2015 @11:53AM (#9996958)
In my opinion Feedly is hands down better than any other RSS reader I have ever used, including Google Reader. I don't read auto-selected articles, however, I subscribe to and curate feeds on my own.

I'm afraid RSS readers aren't very popular, however, and Google was indeed in a better place to change that.

Re:The Web We Have to Save by Yhippa on Monday August 03, 2015 @02:37PM (#9998693)
The best part about Google Reader was I could easily share articles with select friends and it would show up in their streams and comment on it. Feedly is pretty awesome but I miss that.
Re:The Web We Have to Save by vanderZwan on Monday August 03, 2015 @05:31PM (#9999997)
Nothing has ever come close to the level of, for the lack of a better word, intellectual intimacy I felt with my small circle of Google Reader friends. I still miss their daily share of thought-provoking articles to this day.
Re:The Web We Have to Save by greggman on Monday August 03, 2015 @02:01PM (#9998342)
I would love to use Feedly but Feedly requires an extension to use who's only possible purpose is to spy on you. The extension provides zero features that could not be provided without it. Since it's required (as opposed to optional) the only logical explanation is its spying on you.
Re:The Web We Have to Save by thomasmarcelis on Monday August 03, 2015 @02:22PM (#9998529)
What? I use feedly daily but don't have a feedly extension. You can just go to feedly.com
Re:The Web We Have to Save by sombremesa on Monday August 03, 2015 @04:07PM (#9999381)
They used to require an extension but they don't any longer. Also, their mobile apps are amazing. I don't believe Google Reader had anything like that.
Re:The Web We Have to Save by greggman on Monday August 03, 2015 @06:39PM (#10000383)
Good to know. Sorry. I should have checked to see if they got rid of that requirement
Re:The Web We Have to Save by ProCynic on Monday August 03, 2015 @06:33PM (#10000364)
I used the google reader android app before it was shuttered.
Re:The Web We Have to Save by tajen on Tuesday August 04, 2015 @02:59AM (#10002048)
Correct! I use The Old Reader, free, because it's awesome. I remember now why I did this choice, and indeed invasive browser extensions which ask for * access to all my sites are a big no-no. Whenever I want to use a web-screenshot tool, I do it in a separate profile.
Re:The Web We Have to Save by manigandham on Tuesday August 04, 2015 @03:23AM (#10002096)
This is no longer true, Feedly is completely accessible by browser, no extension needed.
Re:The Web We Have to Save by yoz-y on Tuesday August 04, 2015 @03:00AM (#10002051)
The extension was required to cache articles back in time when local storage sucked and feedly's infrastructure was not big enough to handle influx of all of the people who had to leave Google Reader.
Re:The Web We Have to Save by DiabloD3 on Monday August 03, 2015 @02:35PM (#9998659)
Citation needed.
Re:The Web We Have to Save by iamcurious on Monday August 03, 2015 @03:59PM (#9999332)
I really hope the current centralization of the web is an evolutionary step.

The degree of centralization is directly correlated with its reach. It was decentralized when most people in it were tech savvy, it became centralized as more people got in that are not tech savvy. To make it decentralized again we could 1) expel all non tech savvy people 2) increase the level of tech savvyness 3) create a parallel web where mostl people are tech savvy. I don't see either 1 or 2 happening any time soon. (3) is happening with bitcoin and tor, but as soon as they get popular enough they get centralized again.

Re:The Web We Have to Save by nightpool on Monday August 03, 2015 @04:22PM (#9999495)
So you're saying its impossible to have decentralized software that non-tech-savvy people can use? That seems a little backwards from a cause and effect perspective.

Making software centralized makes it easier for non-tech-savvy people to use, as there are incentives to people to make their websites more accessible (larger profits). Decentralized services aren't somehow fundamentally incompatible with usability—its just a challenge we haven't tackled yet.

Re:The Web We Have to Save by iamcurious on Monday August 03, 2015 @05:31PM (#9999996)
So you're saying its impossible to have decentralized software that non-tech-savvy people can use?

Decentralized services aren't somehow fundamentally incompatible with usability—its just a challenge we haven't tackled yet.

I don't say it is impossible. Just a lot harder due, for example, to the CAP theorem and friends. But that isn't that important because...

Making software centralized makes it easier for non-tech-savvy people to use, as there are incentives to people to make their websites more accessible (larger profits).

Yes! It isn't a matter of usability but of incentives. By having more tech savvy people you change the incentives, there is less to win by centralization (because they can do it alone) and more to lose (for instance they can't fiddle with it any longer). Also it is less attractive to make a centralization because a larger percent of the users will work around the attempts to control them.

Re:The Web We Have to Save by derefr on Monday August 03, 2015 @08:21PM (#10000870)
I think it's more like, non-tech-savvy people don't care about centralization. And decentralized things are harder to build. So, the natural tendency is to do the easier thing (build a centralized service) if-and-when the people you want to target will put up with it.
Re:The Web We Have to Save by motters on Tuesday August 04, 2015 @02:30PM (#10005265)
I think this is a straw man, because whether the internet is centralised or decentraised doesn't depend on how knowledgeable its users are.

We have a decentralised transport system, and modern vehicles are very sophisticated, but that doesn't mean that every driver needs to be an expert mechanic or embedded systems engineer.

Re:The Web We Have to Save by iamcurious on Tuesday August 04, 2015 @03:32PM (#10005681)
Most people understand the basics of car security much more than they understand the basics of computer security. It is much easier to see that losing control of your car is dangerous. It is much easier to recognize when you are losing control of the car. There is a tangible sense of being in power of their own vehicle that is really hard to convey with something as abstract as a general purpose computer network.

Even so, as cars gets more complicated there is a tendency for centralization. There are more laws and regulations, there is a bigger dependency on the company that designed it, some cars even update their firmware remotely, and so on.

Re:The Web We Have to Save by VT_Drew on Monday August 03, 2015 @04:52PM (#9999717)
>(3) is happening with bitcoin and tor

You can't centralize bitcoin, that would be like trying to centralize p2p sharing, the protocol doesn't allow it. An alternative digital currency may emerge that is centralized, or more likely, a single currency exchange will dominate bitcoin. But in reality exchanges would become obsolete if bitcoin really took off (you would just get paid and purchase goods all in bitcoin, no need to exchange to another currency).

Re:The Web We Have to Save by iamcurious on Monday August 03, 2015 @05:36PM (#10000044)
You can't centralize bitcoin, that would be like trying to centralize p2p sharing, the protocol doesn't allow it.

You can't centralize the protocol, but you can centralize the most popular exchanges. You still got the protocol down there, but it is wrapped up in a nice centralized single point of failure interface.

Re:The Web We Have to Save by yegortimoshenko on Friday August 14, 2015 @03:34PM (#10062345)
It is about Blockchain and Xapo wallets or even about centralized validation in wallets like Mycelium.

Such centralized systems had not existed in Bitcoin before it became popular.

Re:The Web We Have to Save by Semaphor on Monday August 03, 2015 @07:37AM (#9995032)
> Blogs suffered from the inability to provide a good reading platform, which is why the death of Google Reader was so mourned.

It was mourned, but even then there were people saying it's a good thing. And they turned out to be right, we now have more and better RSS readers to choose from than ever before.

Re:The Web We Have to Save by summerdown2 on Monday August 03, 2015 @07:53AM (#9995080)
Interesting, because my experience has been exactly the opposite. I used to read so many web articles via rss, but that's reduced month on month since the death of Google Reader. One of the main reasons is that I've found it hard to read on tablet, phone, laptop and desktop and know what I've read and what I haven't.

I'd really like to get back into reading RSS, but found it really difficult to manage.

What suggestion would you make for a good, free, cross-platform rss reader?

Re:The Web We Have to Save by zufallsheld on Monday August 03, 2015 @08:16AM (#9995181)
I suggest TinyTinyRSS[0] (ttrss), a good and free rss-reader. It's open-source, so you can host it yourself (a short tutorial is on my blog[1]). Since it's a web-application, you can run it on any device that has a browser, ttrss even has an offical android client[2].

There's even a hoster that provides ttrss as a service [3].

[0] https://tt-rss.org

[1] https://www.zufallsheld.de/2013/11/03/tiny-tiny-rss-centos-n...

[2] https://tt-rss.org/gitlab/fox/tt-rss-android

[3] https://tinytinyrss-servers.com/

Re:The Web We Have to Save by puzzlingcaptcha on Monday August 03, 2015 @09:43AM (#9995691)
I've been running it for years and heartily recommend it. Both the web and android clients are very robust. The killer feature for me is the ability to share arbitrary content to your own public feed provided by the ttrss instance. (https://tt-rss.org/gitlab/fox/tt-rss/wikis/ShareAnything)

I'd recommend using Postgres as opposed to Mysql, as that is what author specifically recommends (mysql support has been a bit subpar)

Re:The Web We Have to Save by EvanAnderson on Monday August 03, 2015 @06:05PM (#10000225)
Another endorsement for ttrss. I've been running it since 2005, albeit I forked my version years ago after finding the original author a little abrasive to deal with re: patches.
Re:The Web We Have to Save by ivank on Monday August 03, 2015 @08:34AM (#9995252)
https://www.inoreader.com/ is very much like Google Reader except better (e.g. detection of dead feeds)
Re:The Web We Have to Save by joosters on Monday August 03, 2015 @09:39AM (#9995663)
Seconded! Inoreader is completely free to use (but there are reasonably priced upgrades if you want to support them), and offers web-based and app-based reading.

I tried several RSS readers after Google Reader closed, and Inoreader is the one I've stuck with.

Re:The Web We Have to Save by david-given on Monday August 03, 2015 @12:43PM (#9997544)
Thirded. I use inoreader to track my billion or so webcomics, and I like it a lot --- enough that I actually gave them money.
Re:The Web We Have to Save by scrollaway on Monday August 03, 2015 @08:17AM (#9995187)
Not free, sorry (although cheap), but as far as web-based alternatives go I recommend BazQux (bazqux.com). It's essentially like Google Reader, lets you auth with your Google account, and works with a bunch of RSS apps.

I was happy to pay for a lifetime account. It has a 30 day demo period if you want to check it out before deciding.

Re:The Web We Have to Save by acemarke on Monday August 03, 2015 @12:54PM (#9997673)
I found BazQux when Google Reader died, and have been thrilled with it ever since. It looks and feels much like GReader did, and it just _works_. I continue to be amazed that it's not more widely mentioned when these sorts of discussions pop up.

BazQux and Pinboard are two simple services that I am extremely happy to pay for, based on their usefulness, reliability, and value.

Re:The Web We Have to Save by losvedir on Tuesday August 04, 2015 @03:51AM (#10002143)
Oooh, I remember hearing about BazQux. IIRC, it's actually written in some really cool language, I think. I wanna say... Ur/Web + Haskell? They don't publicize it on their site or anything, which I love, since it's actually a real product that happens to be written with it, rather than an "RSS feed reader written in X".
Re:The Web We Have to Save by scrollaway on Tuesday August 04, 2015 @10:09AM (#10003454)
Yes, correct, Ur/Web + Haskell. The author posted it in /r/haskell, IIRC.
Re:The Web We Have to Save by deskglass on Monday August 03, 2015 @01:26PM (#9997977)
Bazqux is also great because it can fetch content from social media - Twitter, Google Plus, some Facebook, etc. Unfortunately, you can't just get your account's entire feed from it, but you can, for example, give it a Twitter page and get that page's tweets.
Re:The Web We Have to Save by coffeecheque on Monday August 03, 2015 @08:18AM (#9995190)
I self host via a Tiny Tiny RSS install. It's got a good reader app for iOS (iPhone and iPad). It hasn't caused me any issues at all. I prefer it to NewsBlur too.
Re:The Web We Have to Save by NoGravitas on Monday August 03, 2015 @10:47AM (#9996230)
Agreed. The Android app is very good, too.
Re:The Web We Have to Save by gernerz on Monday August 03, 2015 @08:25AM (#9995222)
Feedly is is excellent.
Re:The Web We Have to Save by _rpd on Monday August 03, 2015 @11:55AM (#9996973)
I agree. The default UI is tailored to tablet-style formatting, but with a bit of configuration it is a very compact and efficient RSS reader.
Re:The Web We Have to Save by johnchristopher on Monday August 03, 2015 @09:06AM (#9995423)
I like https://digg.com/reader/ but it's not free/cross-platform but it's the closest to Google Reader I found.

If I had to pick up a client application I'd choose Feeddemon on Windows.

Re:The Web We Have to Save by Semaphor on Monday August 03, 2015 @07:58AM (#9995110)
Sorry, don't really know about free. Back then I tried pretty much every solution there was, including the self hosted ones (which I assume are probably far better now then then), but stuck with NewsBlur.com which is not really usable for free (limited amount of feeds).
Re:The Web We Have to Save by cc81 on Monday August 03, 2015 @08:58AM (#9995383)
I use digg reader (yes, really). It works just like the google reader did.

http://digg.com/reader

Re:The Web We Have to Save by vog on Monday August 03, 2015 @10:43AM (#9996195)
I'm using QuiteRSS and I'm quite satisfied with it.

https://quiterss.org/

I would have preferred a browser plugin, but all plugins had a really bad user interface. So I tried RSS tools that are separate from the browser, and settled with QuiteRSS.

Re:The Web We Have to Save by jay_kyburz on Monday August 03, 2015 @08:04PM (#10000791)
Its not the platform that's the problem. Its that the content is not appearing on Blogs and in the feed. Its happening on Social Networks.
Re:The Web We Have to Save by b0ner_t0ner on Monday August 03, 2015 @09:33AM (#9995615)
Re:The Web We Have to Save by Eric_WVGG on Monday August 03, 2015 @11:56AM (#9996982)
if you don’t mind paying a token amount of cash for good software, Reeder for Mac and iOS is brilliant
Re:The Web We Have to Save by mindcrime on Monday August 03, 2015 @11:03AM (#9996372)
If you're OK with a web based reader, I've actually found Digg Reader to be pretty nice. I know, I know... it's Digg, but still... it's not bad.

Of the desktop client readers, I still favor RSSOwl, but if you have a LOT of feeds, then performance can be a bit of a drag.

Re:The Web We Have to Save by chei0aiV on Monday August 03, 2015 @08:48AM (#9995333)
rss2email
Re:The Web We Have to Save by rm_-rf_slash on Monday August 03, 2015 @09:38AM (#9995656)
Perhaps the centralization is a necessary function of technological progress. Once Web 2.0 was a thing, everybody had comment sections. Now the dial is being turned back, and comments sections are beginning to be seen as tacky.

In some ways, the web is already decentralizing, or at least moving away from monolithic control by social networks. For younger people, Facebook is basically dead and snapchat does the job.

I doubt we will see significant technological innovation in the next 20 years that matches the scale of interactivity the web brought for the first 20. Maybe VR will be that next step, but even that won't likely change the dynamics of casual web browsing in between work, for example.

So in short, I think that since we've developed all the basics, the state of the web will start to resemble fashion, rather than technological progress.

Re:The Web We Have to Save by dreamfactory2 on Monday August 03, 2015 @03:12PM (#9999010)
Salmon protocol provides very powerful decentralised commenting - http://www.w3.org/2005/Incubator/federatedsocialweb/wiki/SWA...
Re:The Web We Have to Save by nly on Monday August 03, 2015 @10:20AM (#9995978)
> Once Web 2.0 was a thing, everybody had comment sections. Now the dial is being turned back, and comments sections are beginning to be seen as tacky.

I suppose the 'Web 1.0' equivalent, if we persist in these ridiculous version numbers, would be Guestbooks.

Re:The Web We Have to Save by rm_-rf_slash on Monday August 03, 2015 @10:27AM (#9996046)
It depends on what the needs are. We have the infrastructure and libraries to do pretty much anything. Our limit is our imagination. Cheesy, I know, but it's true.
Re:The Web We Have to Save by nbadg on Monday August 03, 2015 @02:36PM (#9998684)
Agreed, and I think the critical inflection point will be the development of an open, decentralized hosting+sharing+identity protocol.

Why all three? Well, I can't imagine an open hosting protocol without the hosted information being private by default (otherwise the proverbial cat starts outside of the bag, and good luck getting it back in). That implies end-to-end encryption. To do that and still be able to communicate requires a key exchange system -- that's the sharing bit. And to know who you're sharing with requires an identity definition, which has to be stored somewhere. And now we've closed the loop.

There are definitely people working on that problem (I know, because I'm one of them).

Re:The Web We Have to Save by brutuscat on Friday August 07, 2015 @09:55AM (#10022467)
What about http://ipfs.io/

> IPFS is a new hypermedia distribution protocol, addressed by content and identities. IPFS enables the creation of completely distributed applications. It aims to make the web faster, safer, and more open.

> The InterPlanetary File System (IPFS) is a peer-to-peer distributed file system that seeks to connect all computing devices with the same system of files. In some ways, IPFS is similar to the Web, but IPFS could be seen as a single BitTorrent swarm, exchanging objects within one Git repository. In other words, IPFS provides a high throughput content-addressed block storage model, with content-addressed hyperlinks. This forms a generalized Merkle DAG, a data structure upon which one can build versioned file systems, blockchains, and even a Permanent Web. IPFS combines a distributed hashtable, an incentivized block exchange, and a self-certifying namespace. IPFS has no single point of failure, and nodes do not need to trust each other.

Re:The Web We Have to Save by PeterisP on Monday August 03, 2015 @06:10PM (#10000253)
Perhaps I misunderstand the problem, but can you elaborate what does "an open, decentralized hosting+sharing+identity protocol" enable that a set of https servers does not?

I mean, pretty much anyone can set up a server to securely host data right now, the main limiting factor is lack of desire, not lack of ability.

Only the identity part is not decentralized but any significantly useful identity anyway needs a solid tie back to the pysical world, as enforcing commitments requires a reasonably reliable ability to trace that identity down to a specific person for legal consequences and expelling harmful actors requires that it is hard to create many fresh identities; I am not aware of any successful identity system that does not ultimately piggyback off of some existing, very centralized sources.

Re:The Web We Have to Save by nbadg on Tuesday August 04, 2015 @01:43AM (#10001872)
Some clarifications: first, I need to be clear that I'm not saying three protocols (one for each), I'm saying a single protocol that does all three. Approaching the problems separately leads you to something quite similar to the mess we're already in. Second, security is a necessary but insufficient condition for privacy -- just look at Facebook. You can access it with https; that doesn't make it private. And third, the problem is one of agency (tangent: good recent article re privacy vs agency https://medium.com/message/mark-zuckerberg-s-personal-news-s...). The root goal of decentralization is divestiture of agency.

Now, for the sake of argument let's say everyone who wants to host something online has a button that automatically deploys an SSL cert and an https server with domain name, etc. Let's say they throw up a blog on that site. How do they create private posts? That's the critical part, and that's where https isn't just unwieldy, but downright inadequate.

The goal of a network is to facilitate connections between agents, not between topological nodes, and https only provides confidentiality between those topological nodes. If you want everyone to be able to host their own stuff on their own servers, and still retain the ability to have a private post, you need a defined way to share things between those agents. And you could, in theory, write a decentralized protocol for that kind of sharing mechanism, but that requires a sense of who you're sharing things with -- an identity protocol. So you need all three.

Now as you mentioned, identity validation is a very complicated, highly variable problem. Facebook has much lower physical verification requirements than your bank does. There can never be a perfect guarantee that this digital entity is the same as the person you have in your head, in much the same way that you can never truly guarantee that the person sitting next to you at a bar isn't, in fact, impersonating your best friend. You just mitigate that risk as much as possible, which is best done with a protocol that allows people to implement as strict (or nonexistent! can't rule out anonymity) a verification as they need.

And yeah, you could definitely write some protocols that, along with https, accomplishes the above. Arguably that's exactly what Facebook has done (in a closed, sleazy, really not private way). But even if you wrote an open copy of facebook, it's impractical for every single user, or even a large percentage of them, to have their own servers (this, I think, is the fundamental failing of Diaspora). So now you're describing, almost exactly, email over https -- and it still doesn't stop gmail from reading your messages.

Sorry if that's a little meandering, it's been a very long month and I'm exhausted right now, but I wanted to give you at least some kind of response. Does that help clear things up?

Re:The Web We Have to Save by wildster on Monday August 03, 2015 @02:59PM (#9998888)
Is there anything open source like this?
Re:The Web We Have to Save by nbadg on Monday August 03, 2015 @04:06PM (#9999368)
That effectively implement storage+sharing+identity? Not that I'm aware of, which is why I'm working on one (https://github.com/Muterra/doc-muse). Plenty of things try and tackle one or two of those but I've never seen anything address all three. I would really, really like to know if one exists so I can pick it apart and learn from it.

A big part of the difficulty is that you truly can't be everything for everyone, but you really need to be something for anyone. If you approach it as a distributed social networking problem (which is most of what I've seen) then you're really missing the indescribably large forest for one specific tree.

The blackbox I've focused on for this protocol is (arbitrary insecure physical bytestream in) -> [box] -> (arbitrary private, authenticated, integrity-verified bytestream out). We use networks to communicate between persons/services/things, not between IP addresses -- the right place to address the problem is between network and application.

Re:The Web We Have to Save by Kalium on Monday August 03, 2015 @03:46PM (#9999251)
Yes. There is. Several, in fact.

It shouldn't surprise you that in practical terms, nobody outside of tech cares enough to use them.

Re:The Web We Have to Save by nbadg on Monday August 03, 2015 @03:56PM (#9999313)
Can you give some examples? I'm not aware of any protocols that fit the storage+identity+sharing triad I mentioned above (except the one I'm working on).
Re:The Web We Have to Save by dublinben on Monday August 03, 2015 @06:31PM (#10000351)
You can do all of that today on Freenet.
Re:The Web We Have to Save by nota_bene on Monday August 03, 2015 @11:53AM (#9996961)
> I really hope the current centralization of the web is an evolutionary step.

In our Western economic system, the question boils down to:

Can a company make more money based on the concept of centralization or decentralization?

The answer seems obvious (to me).

Re:The Web We Have to Save by wolfgke on Monday August 03, 2015 @08:29PM (#10000900)
Skype was originally (before being bought by Microsoft) very decentralized (though the software was heavily obfuscated, so that "decentralized" did not mean "user has much control"). This saved the company the money to build an expensive infrastructure.

So for me the answer is not obvious, whether there is more money to be made on the concept of centralization or decentralization. So I think the problem is rather in terms of "amount of user control" instead of centralization or decentralization.

Re:The Web We Have to Save by pjc50 on Monday August 03, 2015 @07:33AM (#9995019)
Blogs suffered from the inability to provide a good reading platform

I think you're going to have to explain what you mean by good here, given that you mention there was a good reading platform and it was killed in favour of the monetizable web by the chief web monetizer.

Re:The Web We Have to Save by sergiosgc on Monday August 03, 2015 @08:02AM (#9995122)
No, I don't think GReader was good. It had the potential to be great, but it was the bare minimum. It had, however, the necessary lead to leverage the distributed Web into something that could threaten Facebook. Instead they chose to start from scratch and do Facebook in shades of red.
Re:The Web We Have to Save by wtbob on Monday August 03, 2015 @09:39AM (#9995662)
> the inability to control spam

Actually, I think that could have been fixed. If the originating site signed a message, and the each relaying site signed it, then one would be able to calculate a reputation score for sites. Spammy originating sites (or originating domains pretending to be relaying sites) would be easily identifiable, and thus blockable.

Re:The Web We Have to Save by toyg on Monday August 03, 2015 @10:51AM (#9996259)
Time to dust off http://craphound.com/spamsolutions.txt

"Here is why it won't work.

(X) Requires immediate total cooperation from everybody at once

(X) Anyone could anonymously destroy anyone else's career or business

Specifically, your plan fails to account for

(X) Lack of centrally controlling authority for email

(X) Dishonesty on the part of spammers themselves

and the following philosophical objections may also apply:

(X) Ideas similar to yours are easy to come up with, yet none have ever been shown practical

(X) Blacklists suck

(X) Countermeasures must work if phased in gradually

(X) Why should we have to trust you and your servers?

Furthermore, this is what I think about you:

(X) Sorry dude, but I don't think it would work."

Re:The Web We Have to Save by wtbob on Monday August 03, 2015 @02:45PM (#9998763)
Maybe I should explain the idea in more detail.

Recall that the way Usenet works is that each site gets its new feeds from one or more other sites.

Imagine that I'm site A, connected to sites B and C. B is connected to D and E; C is connected to D, F & G. Whenever a user at a site submits an article, that site attests 'this message was sent via site X.' Whenever a site sends a message to another site that it received from somewhere else, it attests 'this message was sent via site X.' As part of the setup process, sites exchange signing keys (recall that Usenet peering was fairly static, something arranged by email).

So, any messages local to Site A don't (need) any attestation: they were generated locally, and read locally. Any messages submitted to Site B and sent to A are provably from Site B (and A would in fact reject any message claiming to be from Site B but without a signature). Likewise, any message submitted to Site E would be signed by E, then signed by B, then arrive at A. In the normal course of events, A doesn't care about Site E at all; he cares about the quality of his feed from B.

Say a spam arrives at A. He knows that it didn't come from C, because it's signed by B. So he returns it to B, stating, 'yo, police your users!' B can see his own signature on the spam, so he knows that he can trust the origination of the message from his point of view (whether it's his own user, or one from Sites D or E); in fact, B could automatically forward the spam notification to the originating site if it weren't his own.

Let's say that B is actually malicious, and has invented a hundred fake sites that he pretends to forward messages from. I don't care: any message I get from B is signed by B: if I receive enough spams from him, I can choose to just refuse articles from him.

> (X) Requires immediate total cooperation from everybody at once

Some solutions do require complete cooperation. Given that Usenet's effectively dead, is an example which kills Usenet and replaces it with Usenet-Prime all that different?

> (X) Anyone could anonymously destroy anyone else's career or business

Completely and totally wrong, since in the model I'm talking about reputation is local to my site, not shared.

> (X) Lack of centrally controlling authority for email

Nope: no centrally-controlling authority is needed: each site only cares about the relationships it has established, and doesn't care about downstream relationships ('police yourself!').

> (X) Dishonesty on the part of spammers themselves

Nope: dishonesty doesn't matter because each site only cares about the quality of the feed it receives from other sites. A dishonest site will have a poor-quality feed, and may be disciplined by being ignored.

> (X) Ideas similar to yours are easy to come up with, yet none have ever been shown practical

Seems quite practical to me. 'We've not bothered to try' doesn't count as a proof of impracticality.

> (X) Blacklists suck

Better than whitelists, no? At some point one has to refuse to do business with bad actors; that's a blacklist.

> (X) Countermeasures must work if phased in gradually

Why? As noted, Usenet is basically dead by now; how would killing it and recreating a new, more trustworthy one not have been better?

> (X) Why should we have to trust you and your servers?

I think maybe you thought that I was proposing some sort of centralised reputation system, which would indeed be crazy.

Re:The Web We Have to Save by toyg on Monday August 03, 2015 @04:04PM (#9999354)
> A dishonest site will have a poor-quality feed, and may be disciplined by being ignored.

So you're really just doing current antispam, with some additional identification which may or may not be respected (its value relies in forcing manual labor on initial key exchange) and may or may not be scalable (say you put a new honest mailserver online -- are you going to statically exchange keys with all other servers out there? And how do you verify a key is still valid at any point? Are you going to pull something out of DNS? this sounds familiar...). Hardly a silver bullet IMHO, sorry.

> Completely and totally wrong, since in the model I'm talking about reputation is local to my site, not shared.

It doesn't matter: you're a stolen key away from suffering. If I get your key and start signing spam with it, sending it all over the place, you'll soon be overflowed by "yo!"s, you'll have to regenerate keys, rebuild trust (a process you've forced to be slow), and your mail won't go through for days or weeks. The typical exactness of caching algorithms will ensure that minor servers will still be "yo!"ing you for months or years, and again your mail won't go through.

We really have no idea how many private keys get stolen every day. They are just not deployed unless they produce valuable activity (accessing a control panel, signing a malicious driver and so on). Spamming is an immediately-valuable activity, so they would become prized loot overnight. Also, we've seen enough attacks on HTTPS and other "secure" implementations by now, that we should know signatures are not a panacea.

And all this effort just to add "another signal" to the pile of checks we already do...

> Better than whitelists, no?

They are effectively the same thing, which is why the following option in the list is "Whitelists suck".

> (X) Countermeasures must work if phased in gradually

Why?

Are you going to have "email switch day"? Good luck with that. Look at IPv4 vs IPv6, email is on that scale.

EDIT: apologies, I see you were concentrating on NNTP. However, I don't think the two worlds are particularly dissimilar -- NNTP just never reached the scale and criticality of SMTP, at which point your solution becomes basically unmanageable.

Re:The Web We Have to Save by wtbob on Monday August 03, 2015 @07:50PM (#10000727)
> its value relies in forcing manual labor on initial key exchange

Setting up NNTP peers has always been manual, and one has always gotten one's newsfeed through one's peers which is why this can work; as your edit noted, this can't work for SMTP because it's a different model.

> If I get your key and start signing spam with it, sending it all over the place, you'll soon be overflowed by "yo!"s, you'll have to regenerate keys, rebuild trust (a process you've forced to be slow), and your mail won't go through for days or weeks.

And for NNTP that would be okay.

Solving the SMTP spam problem is something else entirely, and I wouldn't want to taken on that burden. But Usenet could have been saved.

Re:The Web We Have to Save by tomjen3 on Monday August 03, 2015 @11:01AM (#9996355)
Please don't dust that of. It did way more damage to the web than any spammer ever did.

In addition to that you failed to understand what signatures do: there is no way you can have sign a message with anybody else signature. And it doesn't require immediate cooperation if a signed message with a good reputation is used as one of several spam signals. You want to discuss viagra? Well better sign your message with a public key and have a track record of not being a spammer.

Blacklists are the de facto solution that we implemented because using that list prevented any consensus of any other method of combating spam.

Re:The Web We Have to Save by toyg on Monday August 03, 2015 @12:06PM (#9997107)
How is "a list of bad signatures" different from existing blacklists? It isn't. Spammers can just generate new signatures faster than you can mark them as bad... same as IPs, really. And if I manage to get somebody else's private key (which happens so often these days, it's not even funny), I can ruin someone else's career while I push my spam.

Unless, of course, you have a closed system everyone cooperates in; which is nothing like current email and cannot be rolled on gradually.

Honestly now? That list hurts because it's true, and you know it. You cannot wipe out spam without losing the decentralization element built into current protocols. Your "solution" would just add yet another "signal" to the long list of hacks spammers routinely implement.

(Not that I really care: I use gmail, trading my privacy for some of the best antispam systems money can buy, and I'm happy enough. I'm just a bit too old to still believe spam can be fixed, or that "webs of trust" can work at significant scale.)

Re:The Web We Have to Save by wtbob on Monday August 03, 2015 @02:49PM (#9998804)
> Unless, of course, you have a closed system everyone cooperates in; which is nothing like current email and cannot be rolled on gradually.

I was writing about NNTP, not SMTP—they are different.

> You cannot wipe out spam without losing the decentralization element built into current protocols.

NNTP was never decentralised like SMTP is. Each NNTP site had a very limited number of feeds, set up on a social basis. There was no concept that anyone could submit articles anywhere; rather, users submitted articles to their local site, which forwarded them on. In this environment, cutting bad feeds out would have worked (or at least, kept spam to a manageable level, which is all anyone really cares about).

Re:The Web We Have to Save by JupiterMoon on Monday August 03, 2015 @01:31PM (#9998027)
Just so that we're all on the same page I assume by signatures we mean PGP type signing? This would not be spoofible at all (unless a key was lost).
Re:The Web We Have to Save by CodeMage on Monday August 03, 2015 @01:42PM (#9998139)
Which is why GP said: "And if I manage to get somebody else's private key (which happens so often these days, it's not even funny), I can ruin someone else's career while I push my spam."

Also, the biggest problem with PGP-type signing as a mechanism to control spam is not whether you can spoof the signature or not, it's that there's nothing in it that stops the spammer from generating a new identity once the current identity is tarnished.

Re:The Web We Have to Save by tracker1 on Monday August 03, 2015 @04:16PM (#9999453)
With NNTP, you pretty much have to login to a system in order to post... this means, you are trackable, and if the origin server is doing the signing, this can be traced... It's generally very easy to trace a message through it's server chain.
Re:The Web We Have to Save by the8472 on Monday August 03, 2015 @10:30AM (#9996071)
That would require a central authority to verify signatures. Otherwise intermediate sites could just make up new identities once old ones became tarnished.

And a central authority is not really all that decentralized, is it?

Re:The Web We Have to Save by wtbob on Monday August 03, 2015 @02:46PM (#9998779)
> That would require a central authority to verify signatures.

Nope. Each site would be its own verifying authority. All I care about as a site is that the sites which feed me are responsible for what they send me; all each of them cares about is the same. If any one site gets bad enough, I'll start to reject their articles.

Re:The Web We Have to Save by the8472 on Monday August 03, 2015 @05:20PM (#9999909)
doesn't work for things like email, especially for people who get lots of previously-unknown senders.
Re:The Web We Have to Save by wtbob on Monday August 03, 2015 @07:52PM (#10000732)
> doesn't work for things like email

Which is why I was writing about Usenet☺

Controlling spam in general is a much tougher problem, but in the directed cyclic graph which is Usenet, it could have been done.

Re:The Web We Have to Save by xanderjanz on Monday August 03, 2015 @12:57PM (#9997702)
Signed, Sillicon Valley's Collective Naivite
Re:The Web We Have to Save by roymurdock on Monday August 03, 2015 @10:54AM (#9996286)
Centralization is a law of nature. It allows for more efficiency through economies of scale.

It's not unthinkable that Facebook will be the top destination 20 years from now if they are (1) successful with internet.org and (2) first-to-market with a viable virtual reality product.

Re:The Web We Have to Save by FilterSweep on Monday August 03, 2015 @11:32AM (#9996710)
Hardly a law of nature. Its a school of thought. Centralization leads to other inefficiencies such as Economic deadweight loss, and a lack of competition. The cartel/monopolist economy, as created by Centralization, has it's own issues.

Does Economies of scale outweigh the economic surpluses gained from a more competitive economy? Well, that's something one should ask an ivory tower Economist.

See also: Google's increasingly long list of discontinued products, many of which were acquires from burgeoning young businesses.

Re:The Web We Have to Save by roymurdock on Monday August 03, 2015 @12:07PM (#9997114)
Not making any value judgments. I agree that centralization leads to problems (from the consumer's viewpoint) of collusion, lack of choice, etc. That's why the government has a duty to intervene when monopolies threaten to burden consumers - a competing law of nature called decentralization through regulation.

Ask an economist and you'll get different normative answers of how we should craft policy, but the positive answer will be the same across the board: it has happened with clothing manufacturing, publishing, radio, TV, food production, etc. and is now happening with the internet. A nascent and initially fragmented medium/industry attracts investment and develops standards; large conglomerations begin to form and M&A smaller players to gain market share, leading to a saturated market of heavy-weight players.

Re:The Web We Have to Save by sbov on Monday August 03, 2015 @02:35PM (#9998676)
This would be ignoring history though. The concept of "being online" started centralized with heavy weight players - AOL, prodigy, compuserve, etc.
Re:The Web We Have to Save by nostrademons on Monday August 03, 2015 @03:47PM (#9999257)
Also virtualization - VMWare started out as the dominant player, but then Xen came out, which first led to AWS becoming dominant but now results in a number of cloud hosting and VPS providers, many of which are more cost-effective than AWS.

And PCs - it started out as a decentralized market between Apple, Commodore, Tandy, Atari, Osborne, etc, but then centralized under IBM before being decentralized again with IBM clones like Dell, Compaq, etc. That in turn forced the centralization of the OS market under Microsoft (driving out Apple, CP/M, Acorn, Atari) and microprocessors under Intel (driving out Motorola and MOS technologies).

I think it's more accurate to say that the point of centralization moves up and down the value chain based on technological developments, market deals, and corporate strategy. When things get too centralized, a savvy entrepreneur will come in as a small-time complement, then find a way to encourage competition and commoditize his complements, riding the wave to a very prosperous monopoly. When competition becomes too intense in those markets, the bigger players will start eating the smaller players, and eventually the market consolidates.

Re:The Web We Have to Save by Dn_Ab on Monday August 03, 2015 @12:48PM (#9997615)
In nature, decentralized networks are more the norm. You find them in neural, gene, protein and metabolic networks. You find them in mycorrhizal networks of forests and in various self-organizing systems. Food webs, IIRC are even more random, as far from centralized as you can get.

These networks with small world properties strike a highly pragmatic balance. They are far more robust to insult when compared to centralized networks (though not so much as random) while having much more efficient propogation of information than in more random networks.

Re:The Web We Have to Save by amelius on Monday August 03, 2015 @11:19AM (#9996563)
> Centralization is a law of nature. It allows for more efficiency through economies of scale.

However, as technology progresses, we might reach a point where "distributed" can be sufficiently efficient.

Re:The Web We Have to Save by lectrick on Monday August 03, 2015 @11:24AM (#9996625)
> if they are (1) successful with internet.org

no self-respecting third-world intellectual is going to be satisfied by a catered whitelist of websites they can access. to me this seems like an attempt to repeat what we did with confusing "the web" with "the internet" by having third world people confuse "facebook" with "the web"

Re:The Web We Have to Save by roymurdock on Monday August 03, 2015 @11:52AM (#9996951)
Indonesians surveyed by Galpaya told her that they didn’t use the internet. But in focus groups, they would talk enthusiastically about how much time they spent on Facebook. Galpaya, a researcher (and now CEO) with LIRNEasia, a think tank, called Rohan Samarajiva, her boss at the time, to tell him what she had discovered. “It seemed that in their minds, the Internet did not exist; only Facebook,” he concluded. [1]

It doesn't matter what intellectuals are satisfied with, the internet (like TV and Radio) will be driven by what the mass-market consumer wants. If they can get all they need from Facebook for free (running a business & talking to friends and family), then why pay more to develop a network that they don't want?

[1] http://qz.com/333313/milliions-of-facebook-users-have-no-ide...

Re:The Web We Have to Save by dreamfactory2 on Monday August 03, 2015 @03:35PM (#9999186)
This is even true with SE Asian web developers - if there isn't an active Facebook community, a technology can't get traction there
Re:The Web We Have to Save by lectrick on Tuesday August 04, 2015 @10:20AM (#10003507)
wow.

so what I predicted has basically already happened. sigh.

Re:The Web We Have to Save by pjc50 on Monday August 03, 2015 @11:56AM (#9996985)
This depends very much on whether the local facebook censorship/real names policy makes it hard to post political content. If it doesn't, then it may well be popular.
Re:The Web We Have to Save by kristianc on Monday August 03, 2015 @01:02PM (#9997739)
What specific expertise do you have with what self-respecting third world intellectuals want?