News for hackers. Stuff that matters.
Re:Pipelines – a guided tour of the new IO API in .NET by nathanaldensr on Monday July 02, 2018 @03:30PM (#17444681)
As Tychus Findlay might say, "It's about damned time."

The Stream API has to be one of the ugliest warts in .NET, and it's been around since the beginning. I am so glad Microsoft finally addressed it. Every time I use Stream, I am reminded of how badly the Stream API violates SOLID principles.

I like how Microsoft approached the problem from below by introducing new types like Span<T> and Memory<T>, then the Pipes API. It shows that the future of the platform is in good hands.

Re:Pipelines – a guided tour of the new IO API in .NET by quiq on Monday July 02, 2018 @06:27PM (#17445895)
First job out of school my coworker was talking about how nice dotnet Stream API was. I had a Stream. Streams have Position. A GZipStream is a Stream. A GZipStream does not have position. Countless hours of Java style "OOP" indoctrination flashed before my eyes and I realized that even something as big as Dotnet could do such a thing. It made me feel like a kid that just realized their parents aren't infallible.
Re:Pipelines – a guided tour of the new IO API in .NET by sk5t on Monday July 02, 2018 @09:27PM (#17446761)
I dunno, I'd prefer a consistent Stream interface that just sometimes doesn't have a position, and wraps other Streams with little fuss, rather than have to change gears into Source/Sink lexicon.
Re:Pipelines – a guided tour of the new IO API in .NET by quiq on Monday July 02, 2018 @10:10PM (#17446945)
I see where you're coming from, and hey, the thing works. Not everything is perfect, but the Stream API could definitely be better.

I agree it would be nice to have a consistent interface. The problem is it's neither consistent nor an interface, it's an inconsistent hierarchy of subclasses. There's always fuss because everyone is going to get bitten by it at some point. It's just a poor abstraction, and it leaves a bad taste in your mouth as a library (well, framework) consumer.

Microsoft promised me that they gave me a thing that had certain features and capabilities using industry standard terminology and knowledge. There's a contract. If you go out for a car loan that says "0% APR for 12 months!" You expect that to be the first 12 months, not "0% APR only on days divisible by 7 during leap years, up to a total of 12 * 30 days over the course of the next millennium, otherwise 7.5%"

Re:Pipelines – a guided tour of the new IO API in .NET by nathanaldensr on Tuesday July 03, 2018 @12:51PM (#17451705)
Thank you for eloquently summarizing my "violates SOLID principles" comment. Rigid OOP class hierarchies suck, in general.
Re:Pipelines – a guided tour of the new IO API in .NET by zamalek on Monday July 02, 2018 @08:38PM (#17446569)
It's probably one of the best out there, for likely no other reason than it being included in the box. I still remember back in .Net 1.0, Googling how to load a PNG to GDI from memory and being being met with "just load it from a MemoryStream." At the time this was profound (maybe not so much for Haskell coder, but we didn't all have the fortune of growing up with that). A few years later I was developing something that used STARTTLS (XMPP) and actually starting that TLS stream was just a matter of wrapping the current stream in a SslStream and swapping it out. It's very consistent.

Contrast this with, say, C where there is nothing. Encryption would look vastly different to compression, but both are a similar abstraction: transformations. Now if you wanted to load that result into a device bitmap, you'd likely have to dump the entire stream into memory on onto disk - depending on what different pattern the other library you are using thinks about streams.

Go did a much better job out of the box, but you'd have a hard time convincing me that .Net streams weren't an inspiration there; especially given the nomenclature used.

Re:Pipelines – a guided tour of the new IO API in .NET by pjmlp on Tuesday July 03, 2018 @01:25AM (#17447751)
Smalltalk, CLU, Modula-3, Oberon, Eiffel, C++, Java, Delphi already had the streams concept.

Plenty of stuff for inspiration for.NET devs.

Re:Pipelines – a guided tour of the new IO API in .NET by josteink on Monday July 02, 2018 @04:16PM (#17444981)
> The Stream API has to be one of the ugliest warts in .NET, and it's been around since the beginning.

While true, at least it was less ugly than Java, and that was the initial design goal.

I agree though. Lots of things have changed and been further improved on since then, and maybe it was about time these parts got fixed up too.

Re:Pipelines – a guided tour of the new IO API in .NET by mindfulplay on Monday July 02, 2018 @06:45PM (#17446009)
Maybe the absence of lambdas, async/await and 'using' in the initial release be a reason for this?

A lot of these features evolved over time, so the fundamental building blocks might have been missing.

I am not sure....

Re:Pipelines – a guided tour of the new IO API in .NET by WorldMaker on Monday July 02, 2018 @07:55PM (#17446377)
Right, it was a good API for its time, especially compared to alternatives, most particularly Java's Streams API at the time.

That said, the number of projects I worked on where this new API would have made things so much easier is staggering. It is a shame we can't just fax this API back to our past selves.

The other big building block this uses is Span<T>/Memory<T> which is recent as in this year. Span<T>'s the sort of huge paradigm shift that likely would never have occurred to early .NET design, but starts to seem obvious in retrospect.

Re:Pipelines – a guided tour of the new IO API in .NET by FlorianRappl on Tuesday July 03, 2018 @12:42AM (#17447586)
While I agree in total (e.g., how Microsoft approached the problem is great) I do not agree that the Stream API is that ugly. Yes, it is not nice and has a lot of pitfalls, but it is one of the "lower level" APIs in .NET and certainly did its job. Now its (finally) time to move on.

The future here looks bright!

Re:Pipelines – a guided tour of the new IO API in .NET by Chronos309 on Monday July 02, 2018 @09:52PM (#17446866)
Credit where it is deserved. Span is a great idea, and building fixed sized streams on top of them is a great idea (I haven't read it, but I assume this is the thing they're going after).
Re:Pipelines – a guided tour of the new IO API in .NET by steveadoo on Monday July 02, 2018 @04:53PM (#17445259)
I had some trouble using ReadOnlySequence in a library I am writing (mostly because of no documentation). After looking through how Kestrel uses it(they started using Pipelines in ASP.NET Core 2.1) I found their BufferReader[1] class. It made using ReadOnlySequence much easier.

Overall, pipelines have been really solid for me so far. Working out how to use it without documentation was a total pain though, hopefully they get that out soon. If anyone is interested I can throw the library up on GitHub if anyone wants some real world examples of using pipelines.

[1] https://github.com/aspnet/Common/blob/master/shared/Microsof...

Re:Pipelines – a guided tour of the new IO API in .NET by Haldir on Monday July 02, 2018 @04:57PM (#17445285)
Re:Pipelines – a guided tour of the new IO API in .NET by algorithmsRcool on Monday July 02, 2018 @04:21PM (#17445014)
David Fowler (The architect of this API) is brilliant! The Pipelines API turned out better than I ever expected.
Re:Pipelines – a guided tour of the new IO API in .NET by skc on Monday July 02, 2018 @04:28PM (#17445060)
His work rate is pretty impressive.
Re:Pipelines – a guided tour of the new IO API in .NET by moomin on Monday July 02, 2018 @04:22PM (#17445022)
This is literally the most documentation on the internet for this API. AFAICT, though, it doesn't have compression or encryption stages, yet?
Re:Pipelines – a guided tour of the new IO API in .NET by algorithmsRcool on Monday July 02, 2018 @04:28PM (#17445062)
Since the entire design of Pipelines is 0-copy, I figure you would have to manually pull from one pipe, operate on it and push the resulting data into another pipe.

Im sure there is an example of connecting pipes together like this somewhere. Maybe in Kestrel or SignalR (Core)?

Re:Pipelines – a guided tour of the new IO API in .NET by josteink on Monday July 02, 2018 @04:33PM (#17445095)
> Im sure there is an example of connection pipes together like this somewhere. Maybe in Kestrel or SignalR?

Last I checked, SignalR haven’t had a release in years, so I wouldn’t go there looking for news.

Re:Pipelines – a guided tour of the new IO API in .NET by algorithmsRcool on Monday July 02, 2018 @04:36PM (#17445119)
SignalR Core is what i meant.

It is alive an under active development on .NET Core

https://github.com/aspnet/SignalR

Re:Pipelines – a guided tour of the new IO API in .NET by josteink on Monday July 02, 2018 @04:43PM (#17445178)
That’s definitely something different and you appear to be correct:

https://github.com/aspnet/SignalR/search?q=pipelines&unscope...

Re:Pipelines – a guided tour of the new IO API in .NET by moomin on Monday July 02, 2018 @04:47PM (#17445212)
Yes, but assume you’re lazy: CompressionStream and EncryptionStream already exist.
Re:Pipelines – a guided tour of the new IO API in .NET by CodesInChaos on Tuesday July 03, 2018 @03:10AM (#17448149)
If you're referring to [`CryptoStream`](https://msdn.microsoft.com/en-us/library/system.security.cry...), forget it. Using it securely (in particular against chosen ciphertext attacks) is nigh impossible.
Re:Pipelines – a guided tour of the new IO API in .NET by Const-me on Tuesday July 03, 2018 @02:08PM (#17452409)
I wonder which of that applies to SslStream class from the framework?

https://msdn.microsoft.com/en-us/library/system.net.security...

Re:Pipelines – a guided tour of the new IO API in .NET by manigandham on Wednesday July 04, 2018 @05:34AM (#17456175)
Re:Pipelines – a guided tour of the new IO API in .NET by tigershark on Monday July 02, 2018 @03:12PM (#17444553)
Very nice post, I’m eagerly waiting for the next parts.