News for hackers. Stuff that matters.

A comment on Three Months to Scale NewsBlur.

Re:Three Months to Scale NewsBlur by conesus on Sunday March 17, 2013 @08:48PM (#5391770)
And I just want to remind the technical crowd here that NewsBlur is 100% open-source: http://github.com/samuelclay

I've been asked quite a few times why I open-source the code. The answer is simple. Let me use an example from Joel Spolsky.

(From http://www.joelonsoftware.com/articles/fog0000000052.html)

    The dominant spreadsheet, with 100% market share, is Lotus 123. You're the
    product manager for Microsoft Excel. Ask yourself: what are the barriers to
    switching? What keeps users from becoming Excel customers tomorrow?

    ... <snip: barriers to entry> ...

    That's the barrier to entry. Not how hard it is to switch in: it's how hard it
    might be to switch out.

    And this reminded me of Excel's tipping point, which happened around the time of
    Excel 4.0. And the biggest reason was that Excel 4.0 was the first version of
    Excel that could write Lotus spreadsheets transparently.

    Yep, you heard me. Write. Not read. It turns out that what was stopping people
    from switching to Excel was that everybody else they worked with was still using
    Lotus 123. They didn't want a product that would create spreadsheets that nobody
    else could read: a classic Chicken and Egg problem. When you're the lone Excel
    fan in a company where everyone else is using 123, even if you love Excel, you
    can't switch until you can participate in the 123 ecology.
If you know that in the absolute worst case that you can still use the product even if it's shut down, then by golly, you have even less of a reason to not switch to it.
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by mynegation on Sunday March 17, 2013 @09:00PM (#5391796)
I tried to install NewsBlur, but instructions were not very helpful. I tried to get as far as I can, editing fabfile and installing gajillions of new python modules, but still failed.

Don't get me wrong, it is wonderful and very commendable of you to open source NewsBlur! Nor do I feel entitled to the source code or free ride (I will probably end up paying for the hosted solution). Just providing some feedback, given that you indicated that low barriers to entry are important to you.

I am not a web developer so that can partially explain my struggle with fabric, Django and multitude of webdev and devops modules that you use.

I wish you good luck and I hope that you take this wonderful product even further.

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by conesus on Sunday March 17, 2013 @09:08PM (#5391813)
This is something I'll be working on soon enough. There are many, many dependencies (not only for the web app, but there are three DBs that you have to have installed as prerequisites - mongo, postgres/mysql, and redis).

The real problem with setting up your own instance of NewsBlur is that you'll have to do your own feed fetching. This is effectively what you're paying for when you pay for NewsBlur. Let me break it down.

500 feeds updated once every five minutes is 144,000 feed fetches a day. Couple that with the original page and icon fetches, you're looking at almost two feeds every second just to keep your feeds up to date. And the average feed takes 5 seconds to completely update (feed, page, differences in stories, updating unread counts). So you have to run 10 processes in parallel, and that's already beyond the capabilities of most single machines. Then good luck keeping your DBs running clean and backed up.

Or you could pay $2 / month and never have to worry about setting up a very big stack. And you get the social community of shared stories on newsblur.com.

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by mnutt on Sunday March 17, 2013 @10:51PM (#5392137)
Have you considered releasing VM images? I've thought about it for open source projects that have a bunch of dependencies, but haven't actually tried it.

On the other hand, you may end up with people running the app that have no way to maintain/support it.

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by duncan_bayne on Sunday March 17, 2013 @10:22PM (#5392034)
Three DBs? I don't suppose you have a blog post or something that explains your architecture decisions? Not criticising - I'm sure you had good reasons - but I'm rather curious as to what they were :)
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by BoyWizard on Monday March 18, 2013 @01:26AM (#5392617)
Based on the Github page, I would imagine Redis as a cache, MongoDB to store the 'non-relational data' (quoted from the GH page - I'm guessing the actual articles and such) and Postgres for the other application stuff - accounts, preferences, subs, etc.
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by ceol on Monday March 18, 2013 @01:37AM (#5392650)
https://github.com/samuelclay/NewsBlur/blob/master/apps/rss_...

It looks like MongoDB is being used for fetch/push history, feed icon data, and article data. It's also being used for some other stuff: https://github.com/search?l=Python&q=%22mongo.Document%2...

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by bsg75 on Monday March 18, 2013 @12:46AM (#5392514)
I too am interested, esp what goes into Mongo and what into Postgres (assuming Redis is a form of cache?)

I suppose I could also look at the source ;)

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by boyter on Sunday March 17, 2013 @10:34PM (#5392063)
Just to add to this, the size of the data grows far more quickly then you would expect. I crawl 30 or so feeds for a personal project and database after 2 months of operation is over 200 meg in size.

For the cost of hosting/development/work you are MUCH better paying for newsblur and forgoing 6 outside coffee's a year.

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by laureny on Monday March 18, 2013 @01:33AM (#5392640)
> So you have to run 10 processes in parallel, and that's already beyond the capabilities of most single machines.

Er... really? My laptop is three years old and I run 150 threads on it every time I launch the server I'm working on (dozens of times a day). No problems at all.

It's in Java, in case it matters.

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by MikeAmelung on Monday March 18, 2013 @05:10AM (#5393151)
I believe he means that each process requires ~5 seconds of CPU/system time per update, meaning that even a single 8-core machine wouldn't keep up. It's not really related to how many threads or processes can be created, rather how many can actually be doing work in parallel.
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by VLM on Monday March 18, 2013 @08:56AM (#5393793)
There's an old saying that the true definition of supercomputing merely means moving the bottleneck from the CPU to the IO subsystem. The modern "big data" world seems about the same way. On a regular basis I find strange new ways at work to saturate any I/O system I get access to... Give me a faster I/O system, I'll find an exciting new way to bring it to its knees.
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by lgp171188 on Monday March 18, 2013 @02:04PM (#5395978)
I tried very hard to setup my own instance as well going through the fabfile.py to figure out things, but I hit a point where I simply couldn't go ahead on my Debian Squeeze installation. I'd love to have a very good documentation on the installation process.

Also there seem to be a lot of stuff that need to be installed and configured that might not be necessary for single-user/<10 user instances. Yeah the 'fetching the feed part' still needs to be done, but I guess right now it is close to impossible for anyone apart from yourself to be able to understand how to do that.

Due to the mass exodus from Google Reader, I understand that NewsBlur has faced challenges in scaling and hence is slow. But things can only get better from here.

Since it in open source software, you should at least setup a getting started developer's guide so that you will get a lot more contributions from the community.

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by georgeoliver on Sunday March 17, 2013 @09:20PM (#5391853)
Does NB fetch feeds that have no new articles?
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by UberMouse on Sunday March 17, 2013 @09:48PM (#5391929)
Doesn't it have to fetch feeds to determine there are no new articles?
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by tcwc on Sunday March 17, 2013 @09:58PM (#5391952)
Feed readers can send If-Modified-Since or If-None-Match as part of the request so the server only sends back the full feed if there's something new (Otherwise a 304 Not Modified)
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by mikro2nd on Monday March 18, 2013 @03:11AM (#5392853)
Assuming that the feed implementer bothered... many news CMSs don't bother. The world of RSS is a horrorshow of standards noncompliance; Atom only somewhat better. (My experience is, admittedly,a couple of years out of date, but feed generation tends to be a minimum spend item for news organisations.)
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by Flenser on Tuesday March 19, 2013 @07:06AM (#5400110)
Could you federate some of the feed fetches to your users? Get the UI to check for updates to some subset of that user's feeds and then ping the newsblur server to tell it when there's been an update.
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by rssident on Sunday March 17, 2013 @11:05PM (#5392197)
You should make an intelligent feed updating algorithm.

If you have at least fifty previous entries from a feed it's easy to predict fairly accurately when that feed will need to be updated again.

That's how the indexer for http://rssident.com works. Saves you tons of cpu cycles.

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by sergiosgc on Monday March 18, 2013 @07:38AM (#5393530)
Most blog providers support pubsubhubbub(the worst protocol name ever). It allows you to avoid polling, by having the content producer notify you of feed updates.
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by dpcx on Monday March 18, 2013 @08:58AM (#5393805)
Except that, even with PSHB, you get polling. I wrote about it here http://www.dp.cx/blog/pubsubhubbub-and-polling.html#.UUcPWLp...
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by sergiosgc on Monday March 18, 2013 @11:07AM (#5394616)
The fact that two feed clients, Superfeedr and Guzzle, implement PuSH wrong in no way leads to the conclusion that, when creating a feed reader, you can avoid polling for every content source that does support PuSH.
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by btipling on Sunday March 17, 2013 @11:25PM (#5392256)
> I am not a web developer so that can partially explain my struggle with fabric, Django and multitude of webdev and devops modules that you use.

GitHub is not an app store. You can't expect to "install" a software as a service's code base. Open sourcing a project is opening it up to engineers, providing a self-hosted option on the other hand should be an easy to install alternative that probably requires another full-time engineer to handle as it needs to come with ability to upgrade, get logs, support and documents...

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by tmzt on Monday March 18, 2013 @12:28AM (#5392452)
Seems like there's a need for a cloud infrastructure/language independent packaging format, to build a "github of installable applications"

Maybe container images that can also be used as bootable images?

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by ryen on Monday March 18, 2013 @12:44AM (#5392510)
> Seems like there's a need for a cloud infrastructure/language independent packaging format, to build a "github of installable applications"

I think Chef and Puppet try to fill that role, somewhat

Edit: actually, this recent HN discussion looks promising https://news.ycombinator.com/item?id=5392041

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by jzelinskie on Monday March 18, 2013 @12:53AM (#5392540)
Something like JuJu?
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by est on Sunday March 17, 2013 @10:25PM (#5392039)
> I tried to install NewsBlur, but instructions were not very helpful.

NewsBlur should distribute in an .egg form, or via PIP directly, with VirtualEnv and everything.

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by pestaa on Monday March 18, 2013 @02:33AM (#5392772)
There is a difference between open sourcing a product and releasing it through multiple channels as binaries. I do not think the latter is in the best interest of the developer.
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by jscn on Sunday March 17, 2013 @11:03PM (#5392191)
Just wanted to say, NewsBlur and Pinboard are both big inspirations for me. It's awesome to see people making money off of open source software.
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by rb2k_ on Monday March 18, 2013 @05:30AM (#5393230)
pinboard isn't open source?
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by codexon on Sunday March 17, 2013 @09:03PM (#5391801)
This analogy seems to be describing a different principle. Microsoft was trying to break the network effect that Lotus had.
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by conesus on Sunday March 17, 2013 @09:10PM (#5391821)
I agree it's not a 1-to-1 analogy, but the idea is the same. Knowing you're free to come back to Lotus by using Excel frees you from the chicken-and-egg constraint of waiting until your entire office is on Excel. With NewsBlur you know that you can host your own instance if your dependency on the newsblur.com hosted instance breaks.
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by codexon on Sunday March 17, 2013 @10:15PM (#5392015)
I see what you mean by that, but open-sourcing all your software instead of just having an export function seems risky to me.

In our area of business, our features get copied really quickly.

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by vineet on Sunday March 17, 2013 @10:51PM (#5392140)
It might be risky, but there are a lot of other companies that are doing exactly the same thing and and have shown a great path for newsblur.... (for example, wordpress, sugar-crm, etc).
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by boyter on Sunday March 17, 2013 @10:46PM (#5392117)
You can still copy the feature without looking at the code. Sometimes its even preferable.
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by roc on Monday March 18, 2013 @12:23PM (#5395156)
An export function is hardly the same thing. We're not talking about the concern that an RSS client might go away. There's no shortage of clients. No-one worries too much about those. The concern is what happens when the service goes away.
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by micshasan on Sunday March 17, 2013 @10:37PM (#5392075)
This analogy only helps for developers that would be comfortable installing and operating their own instance. I don't see many folks being excited about the opportunity for personal use.

For other folks hoping to build their own service, commercially or otherwise, having access to your code provides a huge savings in time to market.

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by conesus on Sunday March 17, 2013 @10:48PM (#5392128)
You would be better off by just copying features and writing your own code than taking NewsBlur's highly coupled code and trying to forge your own business out of it. I think they reserve that task for developers sent to hell.
[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by ergo14 on Monday March 18, 2013 @07:58AM (#5393590)
Hello Samuel,

I'm sure you will need all the help you can get with upcoming traffic so I've tried contacting you recently about helping you scale better with https://errormator.com, but I somehow miss you when you are online on IRC, so I've sent you mail to the mailbox i've found.

Its great since your app is python based and we maintain an awesome python client.

Drop me a line if you are interested, we love open-source at Errormator so you would get a completly free plan for non-profits :-)

I could also help you personally if you have any integration questions, you can find me on #errormator on Freenode IRC server.

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by dpcx on Monday March 18, 2013 @09:01AM (#5393817)
How does this compare to [1]?

[1]: https://github.com/errbit/errbit

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by ergo14 on Monday March 18, 2013 @09:08AM (#5393847)
Hey, you could use airbrake/errbit compatible client to stream errors to errormator by changing the endpoint, but it is 30% of what errormator provides (just exceptions), we aim to be a lot more than that.

Please see this inteview with us that was posted recently, the article is in polish but the screenshots should give you a better overview of what errormator can do:

http://translate.google.com/translate?sl=pl&tl=en&js...

In short:

- exception monitoring

- performance metrics

- slow request and slow call aggregation

- log aggregation

We def. need a new index page and a features page to show all the functionality in a nice way.

[ Parent | on HN ]
Re:Three Months to Scale NewsBlur by ergo14 on Tuesday March 19, 2013 @07:04AM (#5400095)
Errormator launched its new index page yesterday - some things are still missing, but the images should give you a better overview of what to expect vs. other solutions.

Cheers!

[ Parent | on HN ]