News for hackers. Stuff that matters.
Re:Deepstream.io – Open Realtime Server by kabes on Wednesday September 21, 2016 @08:25AM (#12547515)
I really like that the documentation is quite honest and unbiased. Like with the integrations: https://deepstream.io/tutorials/integrations it always mentions downsides next to the upsides.
Re:Deepstream.io – Open Realtime Server by benjaminjt on Wednesday September 21, 2016 @09:09AM (#12547788)
This is a really good point actually. I particularly liked the overview of other realtime libraries[1] for the honesty factor, and it really helped me understand how it relates to more familiar frameworks too.

[1] https://deepstream.io/blog/realtime-framework-overview/

Re:Deepstream.io – Open Realtime Server by asadlionpk on Wednesday September 21, 2016 @07:05AM (#12547044)
This is great! I wish somebody would write a Firebase API compatible wrapper around this so escaping Firebase lock-in would be possible without altering code.

I might start it myself when I find some time.

Re:Deepstream.io – Open Realtime Server by cheriot on Wednesday September 21, 2016 @08:13AM (#12547428)
Or even just integration testing Firebase without hitting a remote service!
Re:Deepstream.io – Open Realtime Server by asadlionpk on Wednesday September 21, 2016 @09:33AM (#12547966)
that might be overkill. Here is a better solution: https://github.com/urish/firebase-server
Re:Deepstream.io – Open Realtime Server by cheriot on Wednesday September 21, 2016 @10:29AM (#12548441)
I looked into it. firebase-server doesn't work with recent firebase clients and it's not a priority for the author.
Re:Deepstream.io – Open Realtime Server by MasterScrat on Wednesday September 21, 2016 @05:10AM (#12546571)
From a quick look at it it looks like a self-hosted Firebase.

Would that be a good approximation?

Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @05:12AM (#12546577)
yup, that's not too far off. It also gives you Pub/Sub and Request/Response though. Also Firebase uses one huge data-structure for data-sync whereas deepstream breaks data down into smaller objects called "records" with individual lifecycles
Re:Deepstream.io – Open Realtime Server by duncanawoods on Wednesday September 21, 2016 @06:01AM (#12546766)
Huh. I visited your site and reading the tag line "a fast, secure and scalable websocket & tcp server", I assumed it was just a socket server and had no idea it was actually a data-store.
Re:Deepstream.io – Open Realtime Server by MasterScrat on Wednesday September 21, 2016 @06:05AM (#12546783)
Yeah I mostly understood that from the example.

You really need to make the explanation clearer ^^

Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @06:18AM (#12546839)
Very much agree. Would you have any recommendation?
Re:Deepstream.io – Open Realtime Server by MasterScrat on Wednesday September 21, 2016 @08:27AM (#12547535)
I would show what it does step by step: one box with data-sync, one with real-time data sharing, etc... with an animation for each.

Your animation "built for a new generation of apps" is awesome but it doesn't not show clearly what role deepstream plays in it.

I don't think it is possible to explain everything clearly in one sentence so I wouldn't start with a tagline like that.

Re:Deepstream.io – Open Realtime Server by rambos on Wednesday September 21, 2016 @03:36PM (#12551479)
+1 something like this is defintley needed
Re:Deepstream.io – Open Realtime Server by benjaminjt on Wednesday September 21, 2016 @09:15AM (#12547825)
I know the question wasn't directed to me, but the animation on the 'Cluster Messaging'[1] page really helped me understand.

[1] https://deepstream.io/tutorials/core/cluster-messaging/

Re:Deepstream.io – Open Realtime Server by partycoder on Wednesday September 21, 2016 @04:23PM (#12551867)
While it looks interesting, I honestly don't feel like building a system like this on top of node. Not because of the library itself, which seems properly implemented (hard to see libraries using proper formatting and JSDoc these days).

node is a simple, friendly technology, until you to profile it. Then it's not so friendly. With a realtime system you require a great deal of profiling and quite honestly the tooling is not there.

"Oh but what about Flamegraphs and DTrace?"... it is nothing compared to what you have in Java. And node tooling is better on obscure operating systems like IllumOS.

If I had to implement a system like this I would opt for Scala + Akka, or Elixir/Erlang. If it's for simplicity/budget reasons, try PubNub.

Re:Deepstream.io – Open Realtime Server by dragonne on Wednesday September 21, 2016 @06:23PM (#12552802)
What tools do you use to profile a Scala + Akka service? IME they are quite opaque to traditional Java tools like VisualVM. What am I missing?
Re:Deepstream.io – Open Realtime Server by partycoder on Wednesday September 21, 2016 @07:40PM (#12553230)
Scala can be profiled with VisualVM.

You can monitor Akka with kamon.io, or using the tool offered by the authors.

Re:Deepstream.io – Open Realtime Server by reactor on Wednesday September 21, 2016 @09:51AM (#12548117)
Genuine question, how does this different from https://github.com/centrifugal/centrifugo ?
Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @09:59AM (#12548181)
I'm afraid I don't know too much about centrifugo, but it looks like a pub-sub server you can connect to Redis. deepstream on the other hand provides datasync (stateful, persistent JSON objects you can subscribe and listen to), request-response and pubsub in a scalable way with granular permissions and authentication strategies and an ecosystem of connectors that make it work with different databases, caches and message buses.
Re:Deepstream.io – Open Realtime Server by reactor on Wednesday September 21, 2016 @10:58AM (#12548732)
Thanks, any plan for the android client?
Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @11:12AM (#12548889)
sorry, not quite following. There is an android client here:

https://deepstream.io/install/android/

Re:Deepstream.io – Open Realtime Server by reactor on Wednesday September 21, 2016 @11:41AM (#12549214)
Looks quite interesting, I was evaluating few real-time frameworks (esp socket.io) for the past couple of days . Could you pls shed some lights on how easy to use deepstream to setup a one-to-one in-app chat/messaging setup for android? I guess its the event's are the way to go.

For eg: client.event.subscribe('channel1', eventCallback) client.event.emit('channel1', 'hello how are you?')

How scalable is that? can it support thousands of channels without any hiccups on a single medium digital ocean instance?

Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @12:12PM (#12549509)
It scales very far. We've conducted benchmarks lately that showed that a single deepstream on an AWS t2.medium instance can comfortably handle 160.000~200.000 messages a second and a cluster of six ~ 4 Billion an hour. Please find the detailed results here

https://deepstream.io/info/performance/four-billion-messages...

For your chat though you need to consider whether your messages will be persistent and mutable (changeable after they've been posted) or are just one-off messages. In the first case using a list (https://deepstream.io/tutorials/core/datasync-lists/) with references to records (https://deepstream.io/tutorials/core/datasync-records/) is the way to go, in the second case pub/sub (https://deepstream.io/tutorials/core/pubsub-events/) might be better

Re:Deepstream.io – Open Realtime Server by stemuk on Wednesday September 21, 2016 @01:52PM (#12550556)
Do lists offer pagination for lazy-loading?
Re:Deepstream.io – Open Realtime Server by reactor on Wednesday September 21, 2016 @11:28AM (#12549068)
Ah sorry, didn't notice.
Re:Deepstream.io – Open Realtime Server by akanet on Wednesday September 21, 2016 @07:35AM (#12547200)
Do you provide a version of Firebase's onDisconnect? https://firebase.google.com/docs/reference/js/firebase.datab...

It's very hard to correctly manage distributed state without a guarantee similar to this.

Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @07:42AM (#12547233)
Yes, there are granular connection states and associated events. Messages are queued during connection loss. Please find more information here.

https://deepstream.io/docs/client-js/client/#connectionstate...

https://deepstream.io/docs/common/constants/#connection-stat...

Re:Deepstream.io – Open Realtime Server by akanet on Wednesday September 21, 2016 @07:44AM (#12547243)
Sorry, I should have been clearer. I don't mean "does Deepstream tell the user when it's disconnected", I mean:

Does the Deepstream client have a facility (like Firebase) to instruct the remote server to perform an action in the event the client disconnects in the future and the client is unable to get any further message to the server?

Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @07:46AM (#12547255)
ehm...this is not Horizon. But its a good question none the less. No, it does currently not have that facility, but it would certainly be a good addition
Re:Deepstream.io – Open Realtime Server by akanet on Wednesday September 21, 2016 @07:48AM (#12547266)
Hah, sorry, too many tabs open lately. I'll edit. Thanks for the answer.
Re:Deepstream.io – Open Realtime Server by asadlionpk on Wednesday September 21, 2016 @09:14AM (#12547824)
Since this is not built-in, it may be possible to make with deepstream's onClientDisconnect.

https://github.com/deepstreamIO/deepstream.io/issues/191

Also they seem to be working on 'presence'. https://github.com/deepstreamIO/deepstream.io/issues/118

Re:Deepstream.io – Open Realtime Server by samblr on Wednesday September 21, 2016 @05:06PM (#12552253)
Could you please shed some light on concurrent connections benchmarks ? Im looking for a scalable solution - I am trying out socket.io and I read from others there are some performance issues as it scales.

edit: added socket.io part

Re:Deepstream.io – Open Realtime Server by oakesm9 on Wednesday September 21, 2016 @08:45AM (#12547651)
How does this compare to something like ShareDB (https://github.com/share/sharedb)? It seems on the face of it that the data part of it is pretty similar (albeit Deepstream is more fully featured with auth etc.), however, there is no mention of how Deepstream handles conflict resolution if two clients are modifying the state.

Internally ShareDB uses Operational Transformation with special operations which are designed for use in JSON documents.

Does anyone know how this works in Deepstream?

Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @08:51AM (#12547686)
There's a choice of merge algorithms as well as the ability to hook in your own.

https://deepstream.io/tutorials/core/handling-data-conflicts...

Re:Deepstream.io – Open Realtime Server by oakesm9 on Wednesday September 21, 2016 @08:58AM (#12547728)
Thanks! I hadn't found that before.

Sounds like having those merge algorithms/hooks and splitting up the document into multiple "records" might actually be enough for my use case.

Re:Deepstream.io – Open Realtime Server by marknadal on Wednesday September 21, 2016 @12:26PM (#12549672)
Wow, deepstream looks really good (and I'm a competitor, http://gun.js.org/ ). Your guy's product seems to have significantly matured over the last year. I'm very impressed! Up for chatting? mark AT gunDB DOT io (I still like your guys old website better though with the server/client graphic animation).
Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @12:29PM (#12549699)
Ooh, thanks so much. Just heading out for the inevitable "we've made the frontpage drink", but let's get in touch tomorrow. The diagram is still there, but buried in https://deepstream.io/tutorials/core/cluster-messaging/
Re:Deepstream.io – Open Realtime Server by marknadal on Wednesday September 21, 2016 @12:32PM (#12549742)
Yeah that one! I love that graphic! Your profile doesn't have your email address in it, so you'll have to ping me. Tomorrow sounds good. Congrats on the launch!
Re:Deepstream.io – Open Realtime Server by pubnub on Wednesday September 21, 2016 @06:38PM (#12552889)
I love NodeJS. So many NPM's to be had. I want to open source our ANSI-C Realtime Server - maybe someday. Our https://www.pubnub.com/blocks-catalog/ BLOCKS engine actually starting in Java. That was a M-I-S-T-A-K-E-E-E-E. It's better now.
Re:Deepstream.io – Open Realtime Server by makkesk8 on Wednesday September 21, 2016 @01:19PM (#12550228)
I like it but coming from socket.io I have a hard time with it. For instance, Where/how would I put my server logic? The "Node API" as its called isn't really helpful or not documented enough. But then again, I could have misunderstood what deepstream is all about or how you are supposed to be using it.
Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @01:29PM (#12550338)
It's a different beast from socket.io :-) your server logic is just another client, written in java or node. There's a lot of functionality that let's you listen in on what clients request etc. Please find more details at https://deepstream.io/tutorials/core/active-data-providers/
Re:Deepstream.io – Open Realtime Server by firdaus on Thursday September 22, 2016 @12:46AM (#12554435)
Can you state your stance on AGPL[1] on the website like MongoDB and RethinkDB do to put people's minds at ease.

[1] https://github.com/deepstreamIO/deepstream.io/issues/298

Re:Deepstream.io – Open Realtime Server by dschiptsov on Wednesday September 21, 2016 @07:27AM (#12547150)
Realtime? In nodejs?

It must be a webscale realtime, or at least reactive realtime..

Re:Deepstream.io – Open Realtime Server by hackerboos on Wednesday September 21, 2016 @07:36AM (#12547201)
If you're willing to put the work in to load balance then it can probably scale quite well.

I'm always baffled why people choose JS over Erlang/Elixir which is designed to do this and comes with the tools/libs required to scale.

Re:Deepstream.io – Open Realtime Server by elcct on Wednesday September 21, 2016 @07:44AM (#12547248)
Exactly word "probably" is a deal breaker in this case. I am also curious how someone was thinking JS will be a good idea for such project. But hey, there are people building cars out of plastic bottles...
Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @07:50AM (#12547279)
We are working with Alex,creator of µWebSockets (https://github.com/uWebSockets/uWebSockets) on moving all connectivity and networking aspects to low level C
Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @08:00AM (#12547349)
yes, that's one major aspect of it. engine.io adds is own protocol and isn't exactly the most performance tuned library
Re:Deepstream.io – Open Realtime Server by stemuk on Wednesday September 21, 2016 @08:09AM (#12547411)
Is there some data available on which factor the perfomance benefitted from this change (like some benchmark results)?
Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @08:13AM (#12547427)
Sorry, it doesn't let me reply for some reason :-( Development is in progress, there will be figures soon
Re:Deepstream.io – Open Realtime Server by stemuk on Wednesday September 21, 2016 @08:23AM (#12547500)
It seems like you have to wait some seconds for the server to process the comment, if you refresh it should let you comment again. Anyway, looking forward to the data release, through a blog post I suppose?
Re:Deepstream.io – Open Realtime Server by wyldfire on Wednesday September 21, 2016 @08:59AM (#12547735)
That feature of HN is by design. It's there to force you to think about your replies instead of engage in knee-jerk flamewars. :)
Re:Deepstream.io – Open Realtime Server by stemuk on Wednesday September 21, 2016 @07:57AM (#12547325)
Would this mean you are getting rid of engine.io as the base, or is this just meant to be a general addition?
Re:Deepstream.io – Open Realtime Server by stemuk on Wednesday September 21, 2016 @06:29AM (#12546873)
Are there any plans for an iOS/Swift client? So far it seems only js and Java clients are available...
Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @06:31AM (#12546881)
Yup, we're in the process of finalizing it. There's a related issue at https://github.com/deepstreamIO/deepstream.io/issues/68 with links to the beta artefacts.
Re:Deepstream.io – Open Realtime Server by stemuk on Wednesday September 21, 2016 @06:43AM (#12546944)
Nice, really appreciate the progress here! However, is there any reason the artifacts aren't on github? It may be more convenient when it comes to issues/changelogs etc.
Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @06:47AM (#12546966)
agreed, they will be soon. At the moment the core is cross-compiled from the Java-Client using j2obc, hence the underlying repo is the java one. We are in the process of evaluating how well this worked.
Re:Deepstream.io – Open Realtime Server by stemuk on Wednesday September 21, 2016 @07:03AM (#12547037)
:D I am actually trying to do something similar with the js client. I recently tried to convert it to Dart using jsparser, since i am working with http://flutter.io, a hybrid app development project from Google. It worked out just fine for now, currently testing it to get the weak spots fixed.
Re:Deepstream.io – Open Realtime Server by phelm on Wednesday September 21, 2016 @11:11AM (#12548864)
How does this compare to RethinkDB / Horizon ?
Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @11:16AM (#12548935)
Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @04:02AM (#12546246)
oops, thanks so much, fixed it
Re:Deepstream.io – Open Realtime Server by aw3c2 on Wednesday September 21, 2016 @06:54AM (#12546995)
Can't read the text, please review the contrast part of https://www.w3.org/TR/WCAG20/
Re:Deepstream.io – Open Realtime Server by stemuk on Wednesday September 21, 2016 @07:07AM (#12547052)
This might help you out for now (if you are using Firefox): https://support.mozilla.org/en-US/kb/accessibility-features-...
Re:Deepstream.io – Open Realtime Server by lpinca on Wednesday September 21, 2016 @05:15AM (#12546581)
I'm confused. 1.1 has been released 13 days ago https://github.com/deepstreamIO/deepstream.io/releases/tag/v...
Re:Deepstream.io – Open Realtime Server by wolframhempel on Wednesday September 21, 2016 @05:17AM (#12546588)
yes, but we wanted to give it an initial test phase before announcing it to make sure we didn't miss anything major
Re:Deepstream.io – Open Realtime Server by andrewwhartion on Wednesday September 21, 2016 @05:18AM (#12546595)
Isn't that what a release candidate is for?
Re:Deepstream.io – Open Realtime Server by lpinca on Wednesday September 21, 2016 @05:19AM (#12546599)
I'm still confused. You are announcing 1.0 but the actual version is 1.1
Re:Deepstream.io – Open Realtime Server by 120bits on Wednesday September 21, 2016 @03:33AM (#12546108)
Page Not Found. When I clicked on Getting started

https://deepstream.io/core/getting-started-quickstart/

Edit: Clicked on tutorials and then Getting started.