Thoughts on Scramjet 5.0
Plans on what needs to be changed in the next stable major version of the framework.
The time has come to start development of the next major version of the Scramjet framework. 4.x versions have brought stability and clear vision of the range of applications.
- Scramjet v4 will be still be supported with security updates provided until April 2019.
- Most of the software written against v4 should work flawlessly in v5.
-compatmodule may be released if sufficient interest shows (it'll be v4 interface using v5 underneath).
- Deprecated usages will result in immediate error if possible.
With the upcoming v5 we expect the following changes:
One of the issues in using current version is that usage of
DataStream.pipeline or a number of transforms returns just the last of a chain of streams instead of a full stream.
In scramjet v5 it will be possible to write new chunks into the result of a stream, like this:
const stream = new DataStream() .map(x => x + 1) .each(console.log) .run(); stream.write(1); stream.write(2); stream.write(3); stream.end(4); // 2 // 3 // 4 // 5
In most of the operations a stream that will be returned will be a Duplex stream that will allow writing data before any transforms and will allow reading data already transformed.
Multiple transforms are being merged in a single stream object. In v5, as long as the stream just gets piped to another stream that is able to receive an array of methods it will get that. Slicing of that array will happen at runtime so that you'll be able to pipe any point in the stream flow.
Streams will no longer accept any options, only those defined will be accepted.
- Any access (read or write) to an undeclared option will result in error
- Options will be declared per class and will inherit along the classes' inheritance chain
Some methods, especially
map allowed passing of a
Clazz argument. This looks fairly nice on the face of it, but it does make the method quite unpredictable. This is also a feature that is inconsistently implemented in the framework.
structure_args will be provided to allow more complex operations to be simplified on sub-objects.
Usage of the
WindowStream introduced in 4.14 series did not pick up too much, similarly
NumberStream will be removed.
These streams will still be possible for use with the help of scramjet
plugins as those will be introduced as separate modules.
Currently plugins inject their methods into prototypes of core streams. This causes that no two conflicting plugins may run in the same process. The refactor will address that issue.
Every method for each stream will be placed into a separate file - the resulting class will be constructed by
Core currently contains way too much methods. In
scramjet-core v5.0.0 all main streams will be removed and only the
PromiseTransformStream will be exposed.
All main methods will get moved to the main
All streams will now expose a
stack property showing place of construction.
Array.prototype inspired methods like map/reduce/find etc. will now follow the ECMA interface. For example the
MapCallback will now receive
self arguments providing index in stream and the stream being mapped respectively.
map method will accept
thisObj argument, as well as any additional arguments that will be passed to the
Currently there's the
referrer option and
graph methods available. These work one way, so you can see the object that constructed the prior object, but finding where the stream is targeted (what other streams are consuming it) is rather hard.
graph will change it's operation returning a soup of all stream inter-connections - graph object will be shared between all streams.
targets methods will be introduced (names to be clarified).