News for hackers. Stuff that matters.

Ask HN: Is there a list of open event streams to do stream processing on?

There are curated lists of open datasets (like https://datahub.io/) for anyone to in (side) projects and/or visualizations. I couldn't find a list of open streams of events (like https://wikitech.wikimedia.org/wiki/EventStreams or https://github.com/HackerNews/API or https://dev.twitter.com/streaming/overview ) that I could use for developing stream processing jobs. If you know any other, please share :)
( 5 comments | 32 points | on HN )
Re:Ask HN: Is there a list of open event streams to do strea... by sclangdon on Wednesday April 12, 2017 @03:05AM (#14095431)
Perhaps not a perfect fit, but what I've done in the past, and found to work quite well, is to turn those big datasets you mention into an event stream.

You can do it quite trivially by simply raising an event on every line read, or you can add some kind of transform to the data beforehand, etc. Whatever you need. You can obviously take this further and turn it into a service or something, but that's up to you.

Here's a simple example in C# - https://ghostbin.com/paste/bzgez

Re:Ask HN: Is there a list of open event streams to do strea... by juancampa on Tuesday April 11, 2017 @09:06PM (#14093972)
Would this work? https://www.satori.com/
Re:Ask HN: Is there a list of open event streams to do strea... by hallmark on Tuesday April 11, 2017 @11:37PM (#14094555)
I assumed this Ask HN was a ploy for free research toward the $1M Satori contest.
Re:Ask HN: Is there a list of open event streams to do strea... by juancampa on Wednesday April 12, 2017 @02:59PM (#14100697)
Nah, my brother sent me that link because he thought I might be interested in the contest and this question just happened to pop up soon after
Re:Ask HN: Is there a list of open event streams to do strea... by b5u on Wednesday April 12, 2017 @05:50AM (#14096162)
Didn't know about that Juan! Thanks But those are only samples and available for limited period of time...