Stream Split and Join

The addition of System.IO.Stream, a well-designed abstraction of binary data sources and sinks, into the .NET base class library is one of the best design choices Microsoft could make.

In C and C++ libraries, you often encounter methods expecting a filename — so even if you had all the data in memory already, you’d have to store it in a temporary file, only so the method can retrieve it again from the file. The only alternative that’s part of an official standard were C++’ iostreams (*shudder*). For .NET developers, it’s easy to design methods that don’t care where the data comes from: just pass a System.IO.Stream to the method.

Joining Streams

Now even with streams, you sometimes reach a dead-end: assume, for example, you had a method that sent a file to another user:

void SendFile(string ipOrHost, Stream contents) { /* ... */ }

What if you wanted to add some custom headers in front of the file’s contents? You’d be forced to create a MemoryStream of the combined size of your headers and the entire file, write your headers into it and then load the entire file into memory.

That’s were my ChainStream becomes useful: just write your headers into a small MemoryStream and then virtually join them with the FileStream opened from the file you want to transmit. The ChainStream will first read from the MemoryStream and, upon reaching its end, transparently continue with the FileStream – the SendFile() method won’t even notice that half its data came from memory and the other half from a file!

Splitting Streams

Sometimes, you want to do the exact opposite and provide only a portion of a file to a method. If you have merged several files into a packages, this might be useful in order to completely isolate a piece of code from that fact (so it can even do a Seek(0, SeekOrigin.Begin);, ending up at the file’s starting location and not at the beginning of the package file).

Assume you were creating a video editor that enabled users to cut movies into small segments and then join these segments back together. The segments may well be hundreds of megabytes in size, so writing them out to temporary files takes too long and storing them in memory uses up too much space. What you’d need would be a way to isolate a part of the original movie.

This is what the PartialStream class does:

In the above illustration, both types of streams are used in combination: two PartialStreams are used to virtually cut out two segments from the larger FileStream and then joined together with a MemoryStream providing the new headers. All without copying a single byte whether in memory or on disk :)