News for hackers. Stuff that matters.
Re:Node Streams: How do they work? by tantalor on Tuesday April 17, 2012 @10:41PM (#3856210)
Reminds me much of channels in Go: http://golang.org/doc/effective_go.html#channels

Are Node streams pure, generic JavaScript? Do they rely much on Node internals or outside libs?

I ask because I experimented with "porting" Go channels to generic JavaScript in http://www.johntantalo.com/blog/go-flavored-javascript/

Re:Node Streams: How do they work? by firefoxman1 on Tuesday April 17, 2012 @10:54PM (#3856239)
From what I remember when I looked at Node's Stream source, I believe they're pure JS. They depend on the EventEmitter and they basically emit a "data" event with a chunk of data as many times as needed.

My main question is that since it's an instance of EventEmitter, and EventEmitter uses a synchronous loop to "emit" events, wouldn't stream-intensive code be slowed down by lots of requests due to the synchronous nature of EventEmitter?

Re:Node Streams: How do they work? by wulczer on Wednesday April 18, 2012 @03:45AM (#3856956)
> Reminds me much of channels in Go: http://golang.org/doc/effective_go.html#channels

Or producers and consumers in Twisted: http://twistedmatrix.com/documents/current/core/howto/produc...

Re:Node Streams: How do they work? by tantalor on Wednesday April 18, 2012 @08:24AM (#3857579)
This API appears to match the Stream API nearly perfectly.

Max, you should add a reference to Twisted Producers and Consumers API in the article as an example of a similar interface.

Re:Node Streams: How do they work? by maxogden on Tuesday April 17, 2012 @10:53PM (#3856236)
they are pure js, though streams in node emit Buffer objects which can be mostly emulated using html5 Int32Arrays
Re:Node Streams: How do they work? by tantalor on Tuesday April 17, 2012 @11:14PM (#3856291)
Had to look that up,

> A Buffer is similar to an array of integers but corresponds to a raw memory allocation outside the V8 heap. http://nodejs.org/docs/v0.3.1/api/buffers.html

Are streams limited to binary buffers, or may they also emit JavaScript objects or primitives? Binary may be great for low-level I/O streams, but I'd expect more nuanced events for high-level streams.

Re:Node Streams: How do they work? by maxogden on Tuesday April 17, 2012 @11:20PM (#3856309)
You can emit anything but it just so happens that 99% of the streams used in node core are Buffers for performance reasons (Buffers are allocated outside of V8's heap). An example of emitting objects is in here https://github.com/maxogden/dominode/blob/master/example.htm...
Re:Node Streams: How do they work? by mjor on Wednesday April 18, 2012 @05:14AM (#3857114)
Node.js Buffers are actually built on top of V8's Uint8Array. Using 32-bit integers to represent 8-bit values seems quite wasteful.
Re:Node Streams: How do they work? by tantalor on Wednesday April 18, 2012 @08:21AM (#3857565)
Source please? The docs say a Buffer is "outside the V8 heap".
Re:Node Streams: How do they work? by mjor on Wednesday April 18, 2012 @10:31AM (#3858089)
Yes, the chunk of memory is managed by node, but each Buffer instance is linked with a slice of the heap through `v8::Object->SetIndexedPropertiesToExternalArrayData()`, the same way a JS `Uint8Array` would.

Here's the relevant link: https://github.com/joyent/node/blob/master/src/node_buffer.c....