Akka Stream and HTTP Experimental Java Documentation

Transcription

1 Akka Stream and HTTP Experimental Java Documentation Release 1.0 Typesafe Inc July 20, 2015

2 CONTENTS 1 Streams Introduction Quick Start Guide: Reactive Tweets Design Principles behind Akka Streams Basics and working with Flows Working with Graphs Modularity, Composition and Hierarchy Buffers and working with rate Custom stream processing Integration Error Handling Working with streaming IO Pipelining and Parallelism Testing streams Overview of built-in stages and their semantics Streams Cookbook Configuration Akka HTTP Configuration HTTP Model Low-Level Server-Side API Server-Side WebSocket Support High-level Server-Side API Consuming HTTP-based Services (Client-Side) i

3 CHAPTER ONE STREAMS 1.1 Introduction Motivation The way we consume services from the internet today includes many instances of streaming data, both downloading from a service as well as uploading to it or peer-to-peer data transfers. Regarding data as a stream of elements instead of in its entirety is very useful because it matches the way computers send and receive them (for example via TCP), but it is often also a necessity because data sets frequently become too large to be handled as a whole. We spread computations or analyses over large clusters and call it big data, where the whole principle of processing them is by feeding those data sequentially as a stream through some CPUs. Actors can be seen as dealing with streams as well: they send and receive series of messages in order to transfer knowledge (or data) from one place to another. We have found it tedious and error-prone to implement all the proper measures in order to achieve stable streaming between actors, since in addition to sending and receiving we also need to take care to not overflow any buffers or mailboxes in the process. Another pitfall is that Actor messages can be lost and must be retransmitted in that case lest the stream have holes on the receiving side. When dealing with streams of elements of a fixed given type, Actors also do not currently offer good static guarantees that no wiring errors are made: type-safety could be improved in this case. For these reasons we decided to bundle up a solution to these problems as an Akka Streams API. The purpose is to offer an intuitive and safe way to formulate stream processing setups such that we can then execute them efficiently and with bounded resource usage no more OutOfMemoryErrors. In order to achieve this our streams need to be able to limit the buffering that they employ, they need to be able to slow down producers if the consumers cannot keep up. This feature is called back-pressure and is at the core of the Reactive Streams initiative of which Akka is a founding member. For you this means that the hard problem of propagating and reacting to back-pressure has been incorporated in the design of Akka Streams already, so you have one less thing to worry about; it also means that Akka Streams interoperate seamlessly with all other Reactive Streams implementations (where Reactive Streams interfaces define the interoperability SPI while implementations like Akka Streams offer a nice user API). Relationship with Reactive Streams The Akka Streams API is completely decoupled from the Reactive Streams interfaces. While Akka Streams focus on the formulation of transformations on data streams the scope of Reactive Streams is just to define a common mechanism of how to move data across an asynchronous boundary without losses, buffering or resource exhaustion. The relationship between these two is that the Akka Streams API is geared towards end-users while the Akka Streams implementation uses the Reactive Streams interfaces internally to pass data between the different processing stages. For this reason you will not find any resemblance between the Reactive Streams interfaces and the Akka Streams API. This is in line with the expectations of the Reactive Streams project, whose primary purpose is to define interfaces such that different streaming implementation can interoperate; it is not the purpose of Reactive Streams to describe an end-user API. 1

4 1.1.2 How to read these docs Stream processing is a different paradigm to the Actor Model or to Future composition, therefore it may take some careful study of this subject until you feel familiar with the tools and techniques. The documentation is here to help and for best results we recommend the following approach: Read the Quick Start Guide: Reactive Tweets to get a feel for how streams look like and what they can do. The top-down learners may want to peruse the Design Principles behind Akka Streams at this point. The bottom-up learners may feel more at home rummaging through the Streams Cookbook. For a complete overview of the built-in processing stages you can look at the table in Overview of built-in stages and their semantics The other sections can be read sequentially or as needed during the previous steps, each digging deeper into specific topics. 1.2 Quick Start Guide: Reactive Tweets A typical use case for stream processing is consuming a live stream of data that we want to extract or aggregate some other data from. In this example we ll consider consuming a stream of tweets and extracting information concerning Akka from them. We will also consider the problem inherent to all non-blocking streaming solutions: What if the subscriber is too slow to consume the live stream of data?. Traditionally the solution is often to buffer the elements, but this can and usually will cause eventual buffer overflows and instability of such systems. Instead Akka Streams depend on internal backpressure signals that allow to control what should happen in such scenarios. Here s the data model we ll be working with throughout the quickstart examples: public static class Author { public final String handle; public Author(String handle) { this.handle = handle; //... public static class Hashtag { public final String name; public Hashtag(String name) { this.name = name; //... public static class Tweet { public final Author author; public final long timestamp; public final String body; public Tweet(Author author, long timestamp, String body) { this.author = author; this.timestamp = timestamp; this.body = body; 1.2. Quick Start Guide: Reactive Tweets 2

5 public Set<Hashtag> hashtags() { return Arrays.asList(body.split(" ")).stream().filter(a -> a.startswith("#")).map(a -> new Hashtag(a)).collect(Collectors.toSet()); //... public static final Hashtag AKKA = new Hashtag("#akka"); Note: If you would like to get an overview of the used vocabulary first instead of diving head-first into an actual example you can have a look at the Core concepts and Defining and running streams sections of the docs, and then come back to this quickstart to see it all pieced together into a simple example application Transforming and consuming simple streams The example application we will be looking at is a simple Twitter fed stream from which we ll want to extract certain information, like for example finding all twitter handles of users who tweet about #akka. In order to prepare our environment by creating an ActorSystem and ActorMaterializer, which will be responsible for materializing and running the streams we are about to create: final ActorSystem system = ActorSystem.create("reactive-tweets"); final Materializer mat = ActorMaterializer.create(system); The ActorMaterializer can optionally take ActorMaterializerSettings which can be used to define materialization properties, such as default buffer sizes (see also Buffers in Akka Streams), the dispatcher to be used by the pipeline etc. These can be overridden withattributes on Flow, Source, Sink and Graph. Let s assume we have a stream of tweets readily available, in Akka this is expressed as a Source: Source<Tweet, BoxedUnit> tweets; Streams always start flowing from a Source<Out,M1> then can continue through Flow<In,Out,M2> elements or more advanced graph elements to finally be consumed by a Sink<In,M3>. The first type parameter Tweet in this case designates the kind of elements produced by the source while the M type parameters describe the object that is created during materialization (see below) BoxedUnit (from the scala.runtime package) means that no value is produced, it is the generic equivalent of void. The operations should look familiar to anyone who has used the Scala Collections library, however they operate on streams and not collections of data (which is a very important distinction, as some operations only make sense in streaming and vice versa): final Source<Author, BoxedUnit> authors = tweets.filter(t -> t.hashtags().contains(akka)).map(t -> t.author); Finally in order to materialize and run the stream computation we need to attach the Flow to a Sink<T, M> that will get the flow running. The simplest way to do this is to call runwith(sink) on a Source<Out, M>. For convenience a number of common Sinks are predefined and collected as static methods on the Sink class. For now let s simply print each author: authors.runwith(sink.foreach(a -> System.out.println(a)), mat); or by using the shorthand version (which are defined only for the most popular sinks such as Sink.fold and Sink.foreach): 1.2. Quick Start Guide: Reactive Tweets 3

6 authors.runforeach(a -> System.out.println(a), mat); Materializing and running a stream always requires a Materializer to be passed in explicitly, like this:.run(mat). The complete snippet looks like this: final ActorSystem system = ActorSystem.create("reactive-tweets"); final Materializer mat = ActorMaterializer.create(system); final Source<Author, BoxedUnit> authors = tweets.filter(t -> t.hashtags().contains(akka)).map(t -> t.author); authors.runwith(sink.foreach(a -> System.out.println(a)), mat); Flattening sequences in streams In the previous section we were working on 1:1 relationships of elements which is the most common case, but sometimes we might want to map from one element to a number of elements and receive a flattened stream, similarly like flatmap works on Scala Collections. In order to get a flattened stream of hashtags from our stream of tweets we can use the mapconcat combinator: final Source<Hashtag, BoxedUnit> hashtags = tweets.mapconcat(t -> new ArrayList<Hashtag>(t.hashtags())); Note: The name flatmap was consciously avoided due to its proximity with for-comprehensions and monadic composition. It is problematic for two reasons: firstly, flattening by concatenation is often undesirable in bounded stream processing due to the risk of deadlock (with merge being the preferred strategy), and secondly, the monad laws would not hold for our implementation of flatmap (due to the liveness issues). Please note that the mapconcat requires the supplied function to return a strict collection (Out f -> java.util.list<t>), whereas flatmap would have to operate on streams all the way through Broadcasting a stream Now let s say we want to persist all hashtags, as well as all author names from this one live stream. For example we d like to write all author handles into one file, and all hashtags into another file on disk. This means we have to split the source stream into 2 streams which will handle the writing to these different files. Elements that can be used to form such fan-out (or fan-in ) structures are referred to as junctions in Akka Streams. One of these that we ll be using in this example is called Broadcast, and it simply emits elements from its input port to all of its output ports. Akka Streams intentionally separate the linear stream structures (Flows) from the non-linear, branching ones (FlowGraphs) in order to offer the most convenient API for both of these cases. Graphs can express arbitrarily complex stream setups at the expense of not reading as familiarly as collection transformations. A graph can be either closed which is also known as a fully connected graph, or partial which can be seen as a partial graph (a graph with some unconnected ports), thus being a generalisation of the Flow concept, where Flow is simply a partial graph with one unconnected input and one unconnected output. Concepts around composing and nesting graphs in large structures are explained explained in detail in Modularity, Composition and Hierarchy. It is also possible to wrap complex computation graphs as Flows, Sinks or Sources, which will be explained in detail in Constructing Sources, Sinks and Flows from Partial Graphs. FlowGraphs are constructed like this: 1.2. Quick Start Guide: Reactive Tweets 4

7 Sink<Author, BoxedUnit> writeauthors; Sink<Hashtag, BoxedUnit> writehashtags; FlowGraph.factory().closed(b -> { final UniformFanOutShape<Tweet, Tweet> bcast = b.graph(broadcast.create(2)); final Flow<Tweet, Author, BoxedUnit> toauthor = Flow.of(Tweet.class).map(t -> t.author); final Flow<Tweet, Hashtag, BoxedUnit> totags = Flow.of(Tweet.class).mapConcat(t -> new ArrayList<Hashtag>(t.hashtags())); b.from(tweets).via(bcast).via(toauthor).to(writeauthors); b.from(bcast).via(totags).to(writehashtags); ).run(mat); As you can see, we use graph builder to mutably construct the graph using the addedge method. Once we have the FlowGraph in the value g it is immutable, thread-safe, and freely shareable. A graph can be run() directly - assuming all ports (sinks/sources) within a flow have been connected properly. It is possible to construct PartialFlowGraph s where this is not required but this will be covered in detail in Constructing and combining Partial Flow Graphs. As all Akka Streams elements, Broadcast will properly propagate back-pressure to its upstream element Back-pressure in action One of the main advantages of Akka Streams is that they always propagate back-pressure information from stream Sinks (Subscribers) to their Sources (Publishers). It is not an optional feature, and is enabled at all times. To learn more about the back-pressure protocol used by Akka Streams and all other Reactive Streams compatible implementations read Back-pressure explained. A typical problem applications (not using Akka Streams) like this often face is that they are unable to process the incoming data fast enough, either temporarily or by design, and will start buffering incoming data until there s no more space to buffer, resulting in either OutOfMemoryError s or other severe degradations of service responsiveness. With Akka Streams buffering can and must be handled explicitly. For example, if we are only interested in the most recent tweets, with a buffer of 10 elements this can be expressed using the buffer element: tweets.buffer(10, OverflowStrategy.dropHead()).map(t -> slowcomputation(t)).runwith(sink.ignore(), mat); The buffer element takes an explicit and required OverflowStrategy, which defines how the buffer should react when it receives another element while it is full. Strategies provided include dropping the oldest element (drophead), dropping the entire buffer, signalling failures etc. Be sure to pick and choose the strategy that fits your use case best Materialized values So far we ve been only processing data using Flows and consuming it into some kind of external Sink - be it by printing values or storing them in some external system. However sometimes we may be interested in some value that can be obtained from the materialized processing pipeline. For example, we want to know how many tweets we have processed. While this question is not as obvious to give an answer to in case of an infinite stream of tweets (one way to answer this question in a streaming setting would to create a stream of counts described as up until now, we ve processed N tweets ), but in general it is possible to deal with finite streams and come up with a nice result such as a total count of elements. First, let s write such an element counter using Flow.of(Class) and Sink.fold to see how the types look like: final Sink<Integer, Future<Integer>> sumsink = Sink.<Integer, Integer>fold(0, (acc, elem) -> acc + elem); 1.2. Quick Start Guide: Reactive Tweets 5

8 final RunnableGraph<Future<Integer>> counter = tweets.map(t -> 1).toMat(sumSink, Keep.right()); final Future<Integer> sum = counter.run(mat); sum.foreach(new Foreach<Integer>() { public void each(integer c) { System.out.println("Total tweets processed: " + c);, system.dispatcher()); First we prepare a reusable Flow that will change each incoming tweet into an integer of value 1. We combine all values of the transformed stream using Sink.fold will sum all Integer elements of the stream and make its result available as a Future<Integer>. Next we connect the tweets stream though a map step which converts each tweet into the number 1, finally we connect the flow using tomat the previously prepared Sink. Remember those mysterious Mat type parameters on Source<Out, Mat>, Flow<In, Out, Mat> and Sink<In, Mat>? They represent the type of values these processing parts return when materialized. When you chain these together, you can explicitly combine their materialized values: in our example we used the Keep.right predefined function, which tells the implementation to only care about the materialized type of the stage currently appended to the right. As you can notice, the materialized type of sumsink is Future<Integer> and because of using Keep.right, the resulting RunnableGraph has also a type parameter of Future<Integer>. This step does not yet materialize the processing pipeline, it merely prepares the description of the Flow, which is now connected to a Sink, and therefore can be run(), as indicated by its type: RunnableGraph<Future<Integer>>. Next we call run() which uses the ActorMaterializer to materialize and run the flow. The value returned by calling run() on a RunnableGraph<T> is of type T. In our case this type is Future<Integer> which, when completed, will contain the total length of our tweets stream. In case of the stream failing, this future would complete with a Failure. A RunnableGraph may be reused and materialized multiple times, because it is just the blueprint of the stream. This means that if we materialize a stream, for example one that consumes a live stream of tweets within a minute, the materialized values for those two materializations will be different, as illustrated by this example: final Sink<Integer, Future<Integer>> sumsink = Sink.<Integer, Integer>fold(0, (acc, elem) -> acc + elem); final RunnableGraph<Future<Integer>> counterrunnablegraph = tweetsinminutefromnow.filter(t -> t.hashtags().contains(akka)).map(t -> 1).toMat(sumSink, Keep.right()); // materialize the stream once in the morning final Future<Integer> morningtweetscount = counterrunnablegraph.run(mat); // and once in the evening, reusing the blueprint final Future<Integer> eveningtweetscount = counterrunnablegraph.run(mat); Many elements in Akka Streams provide materialized values which can be used for obtaining either results of computation or steering these elements which will be discussed in detail in Stream Materialization. Summing up this section, now we know what happens behind the scenes when we run this one-liner, which is equivalent to the multi line version above: final Future<Integer> sum = tweets.map(t -> 1).runWith(sumSink, mat); Note: runwith() is a convenience method that automatically ignores the materialized value of any other stages except those appended by the runwith() itself. In the above example it translates to using Keep.right as the combiner for materialized values Quick Start Guide: Reactive Tweets 6

9 1.3 Design Principles behind Akka Streams It took quite a while until we were reasonably happy with the look and feel of the API and the architecture of the implementation, and while being guided by intuition the design phase was very much exploratory research. This section details the findings and codifies them into a set of principles that have emerged during the process. Note: As detailed in the introduction keep in mind that the Akka Streams API is completely decoupled from the Reactive Streams interfaces which are just an implementation detail for how to pass stream data between individual processing stages What shall users of Akka Streams expect? Akka is built upon a conscious decision to offer APIs that are minimal and consistent as opposed to easy or intuitive. The credo is that we favor explicitness over magic, and if we provide a feature then it must work always, no exceptions. Another way to say this is that we minimize the number of rules a user has to learn instead of trying to keep the rules close to what we think users might expect. From this follows that the principles implemented by Akka Streams are: all features are explicit in the API, no magic supreme compositionality: combined pieces retain the function of each part exhaustive model of the domain of distributed bounded stream processing This means that we provide all the tools necessary to express any stream processing topology, that we model all the essential aspects of this domain (back-pressure, buffering, transformations, failure recovery, etc.) and that whatever the user builds is reusable in a larger context. Resulting Implementation Constraints Compositionality entails reusability of partial stream topologies, which led us to the lifted approach of describing data flows as (partial) FlowGraphs that can act as composite sources, flows (a.k.a. pipes) and sinks of data. These building blocks shall then be freely shareable, with the ability to combine them freely to form larger flows. The representation of these pieces must therefore be an immutable blueprint that is materialized in an explicit step in order to start the stream processing. The resulting stream processing engine is then also immutable in the sense of having a fixed topology that is prescribed by the blueprint. Dynamic networks need to be modeled by explicitly using the Reactive Streams interfaces for plugging different engines together. The process of materialization may be parameterized, e.g. instantiating a blueprint for handling a TCP connection s data with specific information about the connection s address and port information. Additionally, materialization will often create specific objects that are useful to interact with the processing engine once it is running, for example for shutting it down or for extracting metrics. This means that the materialization function takes a set of parameters from the outside and it produces a set of results. Compositionality demands that these two sets cannot interact, because that would establish a covert channel by which different pieces could communicate, leading to problems of initialization order and inscrutable runtime failures. Another aspect of materialization is that we want to support distributed stream processing, meaning that both the parameters and the results need to be location transparent either serializable immutable values or ActorRefs. Using for example Futures would restrict materialization to the local JVM. There may be cases for which this will typically not be a severe restriction (like opening a TCP connection), but the principle remains Interoperation with other Reactive Streams implementations Akka Streams fully implement the Reactive Streams specification and interoperate with all other conformant implementations. We chose to completely separate the Reactive Streams interfaces (which we regard to be an SPI) from the user-level API. In order to obtain a Publisher or Subscriber from an Akka Stream topology, a corresponding Sink.publisher or Source.subscriber element must be used Design Principles behind Akka Streams 7

10 All stream Processors produced by the default materialization of Akka Streams are restricted to having a single Subscriber, additional Subscribers will be rejected. The reason for this is that the stream topologies described using our DSL never require fan-out behavior from the Publisher sides of the elements, all fan-out is done using explicit elements like Broadcast[T]. This means that Sink.fanoutPublisher must be used where multicast behavior is needed for interoperation with other Reactive Streams implementations What shall users of streaming libraries expect? We expect libraries to be built on top of Akka Streams, in fact Akka HTTP is one such example that lives within the Akka project itself. In order to allow users to profit from the principles that are described for Akka Streams above, the following rules are established: libraries shall provide their users with reusable pieces, allowing full compositionality libraries may optionally and additionally provide facilities that consume and materialize flow descriptions The reasoning behind the first rule is that compositionality would be destroyed if different libraries only accepted flow descriptions and expected to materialize them: using two of these together would be impossible because materialization can only happen once. As a consequence, the functionality of a library must be expressed such that materialization can be done by the user, outside of the library s control. The second rule allows a library to additionally provide nice sugar for the common case, an example of which is the Akka HTTP API that provides a handlewith method for convenient materialization. Note: One important consequence of this is that a reusable flow description cannot be bound to live resources, any connection to or allocation of such resources must be deferred until materialization time. Examples of live resources are already existing TCP connections, a multicast Publisher, etc.; a TickSource does not fall into this category if its timer is created only upon materialization (as is the case for our implementation). Resulting Implementation Constraints Akka Streams must enable a library to express any stream processing utility in terms of immutable blueprints. The most common building blocks are Source: something with exactly one output stream Sink: something with exactly one input stream Flow: something with exactly one input and one output stream BidirectionalFlow: something with exactly two input streams and two output streams that conceptually behave like two Flows of opposite direction Graph: a packaged stream processing topology that exposes a certain set of input and output ports, characterized by an object of type Shape. Note: A source that emits a stream of streams is still just a normal Source, the kind of elements that are produced does not play a role in the static stream topology that is being expressed The difference between Error and Failure The starting point for this discussion is the definition given by the Reactive Manifesto. Translated to streams this means that an error is accessible within the stream as a normal data element, while a failure means that the stream itself has failed and is collapsing. In concrete terms, on the Reactive Streams interface level data elements (including errors) are signaled via onnext while failures raise the onerror signal. Note: Unfortunately the method name for signaling failure to a Subscriber is called onerror for historical reasons. Always keep in mind that the Reactive Streams interfaces (Publisher/Subscription/Subscriber) are modeling 1.3. Design Principles behind Akka Streams 8

11 the low-level infrastructure for passing streams between execution units, and errors on this level are precisely the failures that we are talking about on the higher level that is modeled by Akka Streams. There is only limited support for treating onerror in Akka Streams compared to the operators that are available for the transformation of data elements, which is intentional in the spirit of the previous paragraph. Since onerror signals that the stream is collapsing, its ordering semantics are not the same as for stream completion: transformation stages of any kind will just collapse with the stream, possibly still holding elements in implicit or explicit buffers. This means that data elements emitted before a failure can still be lost if the onerror overtakes them. The ability for failures to propagate faster than data elements is essential for tearing down streams that are backpressured especially since back-pressure can be the failure mode (e.g. by tripping upstream buffers which then abort because they cannot do anything else; or if a dead-lock occurred). The semantics of stream recovery A recovery element (i.e. any transformation that absorbs an onerror signal and turns that into possibly more data elements followed normal stream completion) acts as a bulkhead that confines a stream collapse to a given region of the flow topology. Within the collapsed region buffered elements may be lost, but the outside is not affected by the failure. This works in the same fashion as a try catch expression: it marks a region in which exceptions are caught, but the exact amount of code that was skipped within this region in case of a failure might not be known precisely the placement of statements matters The finer points of stream materialization Note: This is not yet implemented as stated here, this document illustrates intent. It is commonly necessary to parameterize a flow so that it can be materialized for different arguments, an example would be the handler Flow that is given to a server socket implementation and materialized for each incoming connection with information about the peer s address. On the other hand it is frequently necessary to retrieve specific objects that result from materialization, for example a Future[Unit] that signals the completion of a ForeachSink. It might be tempting to allow different pieces of a flow topology to access the materialization results of other pieces in order to customize their behavior, but that would violate composability and reusability as argued above. Therefore the arguments and results of materialization need to be segregated: The Materializer is configured with a (type-safe) mapping from keys to values, which is exposed to the processing stages during their materialization. The values in this mapping may act as channels, for example by using a Promise/Future pair to communicate a value; another possibility for such information-passing is of course to explicitly model it as a stream of configuration data elements within the graph itself. The materialized values obtained from the processing stages are combined as prescribed by the user, but can of course be dependent on the values in the argument mapping. To avoid having to use Future values as key bindings, materialization itself may become fully asynchronous. This would allow for example the use of the bound server port within the rest of the flow, and only if the port was actually bound successfully. The downside is that some APIs will then return Future[MaterializedMap], which means that others will have to accept this in turn in order to keep the usage burden as low as possible Design Principles behind Akka Streams 9

12 1.4 Basics and working with Flows Core concepts Akka Streams is a library to process and transfer a sequence of elements using bounded buffer space. This latter property is what we refer to as boundedness and it is the defining feature of Akka Streams. Translated to everyday terms it is possible to express a chain (or as we see later, graphs) of processing entities, each executing independently (and possibly concurrently) from the others while only buffering a limited number of elements at any given time. This property of bounded buffers is one of the differences from the actor model, where each actor usually has an unbounded, or a bounded, but dropping mailbox. Akka Stream processing entities have bounded mailboxes that do not drop. Before we move on, let s define some basic terminology which will be used throughout the entire documentation: Stream An active process that involves moving and transforming data. Element An element is the processing unit of streams. All operations transform and transfer elements from upstream to downstream. Buffer sizes are always expressed as number of elements independently form the actual size of the elements. Back-pressure A means of flow-control, a way for consumers of data to notify a producer about their current availability, effectively slowing down the upstream producer to match their consumption speeds. In the context of Akka Streams back-pressure is always understood as non-blocking and asynchronous. Non-Blocking Means that a certain operation does not hinder the progress of the calling thread, even if it takes long time to finish the requested operation. Processing Stage The common name for all building blocks that build up a Flow or FlowGraph. Examples of a processing stage would be operations like map(), filter(), stages added by transform() like PushStage, PushPullStage, StatefulStage and graph junctions like Merge or Broadcast. For the full list of built-in processing stages see Overview of built-in stages and their semantics When we talk about asynchronous, non-blocking backpressure we mean that the processing stages available in Akka Streams will not use blocking calls but asynchronous message passing to exchange messages between each other, and they will use asynchronous means to slow down a fast producer, without blocking its thread. This is a thread-pool friendly design, since entities that need to wait (a fast producer waiting on a slow consumer) will not block the thread but can hand it back for further use to an underlying thread-pool. Defining and running streams Linear processing pipelines can be expressed in Akka Streams using the following core abstractions: Source A processing stage with exactly one output, emitting data elements whenever downstream processing stages are ready to receive them. Sink A processing stage with exactly one input, requesting and accepting data elements possibly slowing down the upstream producer of elements Flow A processing stage which has exactly one input and output, which connects its up- and downstreams by transforming the data elements flowing through it. RunnableGraph A Flow that has both ends attached to a Source and Sink respectively, and is ready to be run(). It is possible to attach a Flow to a Source resulting in a composite source, and it is also possible to prepend a Flow to a Sink to get a new sink. After a stream is properly terminated by having both a source and a sink, it will be represented by the RunnableGraph type, indicating that it is ready to be executed. It is important to remember that even after constructing the RunnableGraph by connecting all the source, sink and different processing stages, no data will flow through it until it is materialized. Materialization is the process of allocating all resources needed to run the computation described by a Flow (in Akka Streams this will often involve starting up Actors). Thanks to Flows being simply a description of the processing pipeline they are immutable, thread-safe, and freely shareable, which means that it is for example safe to share and send them 1.4. Basics and working with Flows 10

13 between actors, to have one actor prepare the work, and then have it be materialized at some completely different place in the code. final Source<Integer, BoxedUnit> source = Source.from(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)); // note that the Future is scala.concurrent.future final Sink<Integer, Future<Integer>> sink = Sink.fold(0, (aggr, next) -> aggr + next); // connect the Source to the Sink, obtaining a RunnableFlow final RunnableGraph<Future<Integer>> runnable = source.tomat(sink, Keep.right()); // materialize the flow final Future<Integer> sum = runnable.run(mat); After running (materializing) the RunnableGraph we get a special container object, the MaterializedMap. Both sources and sinks are able to put specific objects into this map. Whether they put something in or not is implementation dependent. For example a FoldSink will make a Future available in this map which will represent the result of the folding process over the stream. In general, a stream can expose multiple materialized values, but it is quite common to be interested in only the value of the Source or the Sink in the stream. For this reason there is a convenience method called runwith() available for Sink, Source or Flow requiring, respectively, a supplied Source (in order to run a Sink), a Sink (in order to run a Source) or both a Source and a Sink (in order to run a Flow, since it has neither attached yet). final Source<Integer, BoxedUnit> source = Source.from(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)); final Sink<Integer, Future<Integer>> sink = Sink.fold(0, (aggr, next) -> aggr + next); // materialize the flow, getting the Sinks materialized value final Future<Integer> sum = source.runwith(sink, mat); It is worth pointing out that since processing stages are immutable, connecting them returns a new processing stage, instead of modifying the existing instance, so while constructing long flows, remember to assign the new value to a variable or run it: final Source<Integer, BoxedUnit> source = Source.from(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)); source.map(x -> 0); // has no effect on source, since it's immutable source.runwith(sink.fold(0, (agg, next) -> agg + next), mat); // 55 // returns new Source<Integer>, with `map()` appended final Source<Integer, BoxedUnit> zeroes = source.map(x -> 0); final Sink<Integer, Future<Integer>> fold = Sink.fold(0, (agg, next) -> agg + next); zeroes.runwith(fold, mat); // 0 Note: By default Akka Streams elements support exactly one downstream processing stage. Making fan-out (supporting multiple downstream processing stages) an explicit opt-in feature allows default stream elements to be less complex and more efficient. Also it allows for greater flexibility on how exactly to handle the multicast scenarios, by providing named fan-out elements such as broadcast (signals all down-stream elements) or balance (signals one of available down-stream elements). In the above example we used the runwith method, which both materializes the stream and returns the materialized value of the given sink or source. Since a stream can be materialized multiple times, the MaterializedMap returned is different for each materialization. In the example below we create two running materialized instance of the stream that we described in the runnable variable, and both materializations give us a different Future from the map even though we used the same sink to refer to the future: 1.4. Basics and working with Flows 11

14 // connect the Source to the Sink, obtaining a RunnableGraph final Sink<Integer, Future<Integer>> sink = Sink.fold(0, (aggr, next) -> aggr + next); final RunnableGraph<Future<Integer>> runnable = Source.from(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)).toMat(sink, Keep.right()); // get the materialized value of the FoldSink final Future<Integer> sum1 = runnable.run(mat); final Future<Integer> sum2 = runnable.run(mat); // sum1 and sum2 are different Futures! Defining sources, sinks and flows The objects Source and Sink define various ways to create sources and sinks of elements. The following examples show some of the most useful constructs (refer to the API documentation for more details): // Create a source from an Iterable List<Integer> list = new LinkedList<Integer>(); list.add(1); list.add(2); list.add(3); Source.from(list); // Create a source form a Future Source.from(Futures.successful("Hello Streams!")); // Create a source from a single element Source.single("only one element"); // an empty source Source.empty(); // Sink that folds over the stream and returns a Future // of the final result in the MaterializedMap Sink.fold(0, (Integer aggr, Integer next) -> aggr + next); // Sink that returns a Future in the MaterializedMap, // containing the first element of the stream Sink.head(); // A Sink that consumes a stream without doing anything with the elements Sink.ignore(); // A Sink that executes a side-effecting call for every element of the stream Sink.foreach(System.out::println); There are various ways to wire up different parts of a stream, the following examples show some of the available options: // Explicitly creating and wiring up a Source, Sink and Flow Source.from(Arrays.asList(1, 2, 3, 4)).via(Flow.of(Integer.class).map(elem -> elem * 2)).to(Sink.foreach(System.out::println)); // Starting from a Source final Source<Integer, BoxedUnit> source = Source.from(Arrays.asList(1, 2, 3, 4)).map(elem -> elem * 2); source.to(sink.foreach(system.out::println)); // Starting from a Sink final Sink<Integer, BoxedUnit> sink = Flow.of(Integer.class) 1.4. Basics and working with Flows 12

15 .map(elem -> elem * 2).to(Sink.foreach(System.out::println)); Source.from(Arrays.asList(1, 2, 3, 4)).to(sink); Illegal stream elements In accordance to the Reactive Streams specification (Rule 2.13) Akka Streams do not allow null to be passed through the stream as an element. In case you want to model the concept of absence of a value we recommend using akka.japi.option (for Java 6 and 7) or java.util.optional which is available since Java 8. Back-pressure explained Akka Streams implement an asynchronous non-blocking back-pressure protocol standardised by the Reactive Streams specification, which Akka is a founding member of. The user of the library does not have to write any explicit back-pressure handling code it is built in and dealt with automatically by all of the provided Akka Streams processing stages. It is possible however to add explicit buffer stages with overflow strategies that can influence the behaviour of the stream. This is especially important in complex processing graphs which may even contain loops (which must be treated with very special care, as explained in Graph cycles, liveness and deadlocks). The back pressure protocol is defined in terms of the number of elements a downstream Subscriber is able to receive and buffer, referred to as demand. The source of data, referred to as Publisher in Reactive Streams terminology and implemented as Source in Akka Streams, guarantees that it will never emit more elements than the received total demand for any given Subscriber. Note: The Reactive Streams specification defines its protocol in terms of Publisher and Subscriber. These types are not meant to be user facing API, instead they serve as the low level building blocks for different Reactive Streams implementations. Akka Streams implements these concepts as Source, Flow (referred to as Processor in Reactive Streams) and Sink without exposing the Reactive Streams interfaces directly. If you need to integrate with other Reactive Stream libraries read Integrating with Reactive Streams. The mode in which Reactive Streams back-pressure works can be colloquially described as dynamic push / pull mode, since it will switch between push and pull based back-pressure models depending on the downstream being able to cope with the upstream production rate or not. To illustrate this further let us consider both problem situations and how the back-pressure protocol handles them: Slow Publisher, fast Subscriber This is the happy case of course we do not need to slow down the Publisher in this case. However signalling rates are rarely constant and could change at any point in time, suddenly ending up in a situation where the Subscriber is now slower than the Publisher. In order to safeguard from these situations, the back-pressure protocol must still be enabled during such situations, however we do not want to pay a high penalty for this safety net being enabled. The Reactive Streams protocol solves this by asynchronously signalling from the Subscriber to the Publisher Request(int n) signals. The protocol guarantees that the Publisher will never signal more elements than the signalled demand. Since the Subscriber however is currently faster, it will be signalling these Request messages at a higher rate (and possibly also batching together the demand - requesting multiple elements in one Request signal). This means that the Publisher should not ever have to wait (be back-pressured) with publishing its incoming elements. As we can see, in this scenario we effectively operate in so called push-mode since the Publisher can continue producing elements as fast as it can, since the pending demand will be recovered just-in-time while it is emitting elements Basics and working with Flows 13

16 Fast Publisher, slow Subscriber This is the case when back-pressuring the Publisher is required, because the Subscriber is not able to cope with the rate at which its upstream would like to emit data elements. Since the Publisher is not allowed to signal more elements than the pending demand signalled by the Subscriber, it will have to abide to this back-pressure by applying one of the below strategies: not generate elements, if it is able to control their production rate, try buffering the elements in a bounded manner until more demand is signalled, drop elements until more demand is signalled, tear down the stream if unable to apply any of the above strategies. As we can see, this scenario effectively means that the Subscriber will pull the elements from the Publisher this mode of operation is referred to as pull-based back-pressure. Stream Materialization When constructing flows and graphs in Akka Streams think of them as preparing a blueprint, an execution plan. Stream materialization is the process of taking a stream description (the graph) and allocating all the necessary resources it needs in order to run. In the case of Akka Streams this often means starting up Actors which power the processing, but is not restricted to that - it could also mean opening files or socket connections etc. depending on what the stream needs. Materialization is triggered at so called terminal operations. Most notably this includes the various forms of the run() and runwith() methods defined on flow elements as well as a small number of special syntactic sugars for running with well-known sinks, such as runforeach(el -> ) (being an alias to runwith(sink.foreach(el -> )). Materialization is currently performed synchronously on the materializing thread. The actual stream processing is handled by actors started up during the streams materialization, which will be running on the thread pools they have been configured to run on - which defaults to the dispatcher set in MaterializationSettings while constructing the ActorMaterializer. Note: Reusing instances of linear computation stages (Source, Sink, Flow) inside FlowGraphs is legal, yet will materialize that stage multiple times. Combining materialized values Since every processing stage in Akka Streams can provide a materialized value after being materialized, it is necessary to somehow express how these values should be composed to a final value when we plug these stages together. For this, many combinator methods have variants that take an additional argument, a function, that will be used to combine the resulting values. Some examples of using these combiners are illustrated in the example below. // An empty source that can be shut down explicitly from the outside Source<Integer, Promise<BoxedUnit>> source = Source.<Integer>lazyEmpty(); // A flow that internally throttles elements to 1/second, and returns a Cancellable // which can be used to shut down the stream Flow<Integer, Integer, Cancellable> flow = throttler; // A sink that returns the first element of a stream in the returned Future Sink<Integer, Future<Integer>> sink = Sink.head(); // By default, the materialized value of the leftmost stage is preserved RunnableGraph<Promise<BoxedUnit>> r1 = source.via(flow).to(sink); 1.4. Basics and working with Flows 14

17 // Simple selection of materialized values by using Keep.right RunnableGraph<Cancellable> r2 = source.viamat(flow, Keep.right()).to(sink); RunnableGraph<Future<Integer>> r3 = source.via(flow).tomat(sink, Keep.right()); // Using runwith will always give the materialized values of the stages added // by runwith() itself Future<Integer> r4 = source.via(flow).runwith(sink, mat); Promise<BoxedUnit> r5 = flow.to(sink).runwith(source, mat); Pair<Promise<BoxedUnit>, Future<Integer>> r6 = flow.runwith(source, sink, mat); // Using more complext combinations RunnableGraph<Pair<Promise<BoxedUnit>, Cancellable>> r7 = source.viamat(flow, Keep.both()).to(sink); RunnableGraph<Pair<Promise<BoxedUnit>, Future<Integer>>> r8 = source.via(flow).tomat(sink, Keep.both()); RunnableGraph<Pair<Pair<Promise<BoxedUnit>, Cancellable>, Future<Integer>>> r9 = source.viamat(flow, Keep.both()).toMat(sink, Keep.both()); RunnableGraph<Pair<Cancellable, Future<Integer>>> r10 = source.viamat(flow, Keep.right()).toMat(sink, Keep.both()); // It is also possible to map over the materialized values. In r9 we had a // doubly nested pair, but we want to flatten it out RunnableGraph<Cancellable> r11 = r9.mapmaterializedvalue( (nestedtuple) -> { Promise<BoxedUnit> p = nestedtuple.first().first(); Cancellable c = nestedtuple.first().second(); Future<Integer> f = nestedtuple.second(); // Picking the Cancellable, but we could also construct a domain class here return c; ); Note: In Graphs it is possible to access the materialized value from inside the stream processing graph. For details see Accessing the materialized value inside the Graph Stream ordering In Akka Streams almost all computation stages preserve input order of elements. This means that if inputs {IA1,IA2,...,IAn cause outputs {OA1,OA2,...,OAk and inputs {IB1,IB2,...,IBm cause outputs {OB1,OB2,...,OBl and all of IAi happened before all IBi then OAi happens before OBi. This property is even uphold by async operations such as mapasync, however an unordered version exists called mapasyncunordered which does not preserve this ordering. However, in the case of Junctions which handle multiple input streams (e.g. Merge) the output order is, in general, not defined for elements arriving on different input ports. That is a merge-like operation may emit Ai before emitting Bi, and it is up to its internal logic to decide the order of emitted elements. Specialized elements such as Zip however do guarantee their outputs order, as each output element depends on all upstream elements having been signalled already thus the ordering in the case of zipping is defined by this property. If you find yourself in need of fine grained control over order of emitted elements in fan-in scenarios consider using MergePreferred or FlexiMerge which gives you full control over how the merge is performed Basics and working with Flows 15

18 1.5 Working with Graphs In Akka Streams computation graphs are not expressed using a fluent DSL like linear computations are, instead they are written in a more graph-resembling DSL which aims to make translating graph drawings (e.g. from notes taken from design discussions, or illustrations in protocol specifications) to and from code simpler. In this section we ll dive into the multiple ways of constructing and re-using graphs, as well as explain common pitfalls and how to avoid them. Graphs are needed whenever you want to perform any kind of fan-in ( multiple inputs ) or fan-out ( multiple outputs ) operations. Considering linear Flows to be like roads, we can picture graph operations as junctions: multiple flows being connected at a single point. Some graph operations which are common enough and fit the linear style of Flows, such as concat (which concatenates two streams, such that the second one is consumed after the first one has completed), may have shorthand methods defined on Flow or Source themselves, however you should keep in mind that those are also implemented as graph junctions Constructing Flow Graphs Flow graphs are built from simple Flows which serve as the linear connections within the graphs as well as junctions which serve as fan-in and fan-out points for Flows. Thanks to the junctions having meaningful types based on their behaviour and making them explicit elements these elements should be rather straightforward to use. Akka Streams currently provide these junctions (for a detailed list see Overview of built-in stages and their semantics): Fan-out Broadcast<T> (1 input, N outputs) given an input element emits to each output Balance<T> (1 input, N outputs) given an input element emits to one of its output ports UnzipWith<In,A,B,...> (1 input, N outputs) takes a function of 1 input that given a value for each input emits N output elements (where N <= 20) UnZip<A,B> (1 input, 2 outputs) splits a stream of Pair<A,B> tuples into two streams, one of type A and one of type B FlexiRoute<In> (1 input, N outputs) enables writing custom fan out elements using a simple DSL Fan-in Merge<In> (N inputs, 1 output) picks randomly from inputs pushing them one by one to its output MergePreferred<In> like Merge but if elements are available on preferred port, it picks from it, otherwise randomly from others ZipWith<A,B,...,Out> (N inputs, 1 output) which takes a function of N inputs that given a value for each input emits 1 output element Zip<A,B> (2 inputs, 1 output) is a ZipWith specialised to zipping input streams of A and B into a Pair(A,B) tuple stream Concat<A> (2 inputs, 1 output) concatenates two streams (first consume one, then the second one) FlexiMerge<Out> (N inputs, 1 output) enables writing custom fan-in elements using a simple DSL One of the goals of the FlowGraph DSL is to look similar to how one would draw a graph on a whiteboard, so that it is simple to translate a design from whiteboard to code and be able to relate those two. Let s illustrate this by translating the below hand drawn graph into Akka Streams: 1.5. Working with Graphs 16

19 Such graph is simple to translate to the Graph DSL since each linear element corresponds to a Flow, and each circle corresponds to either a Junction or a Source or Sink if it is beginning or ending a Flow. final Source<Integer, BoxedUnit> in = Source.from(Arrays.asList(1, 2, 3, 4, 5)); final Sink<List<String>, Future<List<String>>> sink = Sink.head(); final Sink<List<Integer>, Future<List<Integer>>> sink2 = Sink.head(); final Flow<Integer, Integer, BoxedUnit> f1 = Flow.of(Integer.class).map(elem -> elem + 10); final Flow<Integer, Integer, BoxedUnit> f2 = Flow.of(Integer.class).map(elem -> elem + 20); final Flow<Integer, String, BoxedUnit> f3 = Flow.of(Integer.class).map(elem -> elem.tostring()); final Flow<Integer, Integer, BoxedUnit> f4 = Flow.of(Integer.class).map(elem -> elem + 30); final RunnableGraph<Future<List<String>>> result = FlowGraph.factory().closed( sink, (builder, out) -> { final UniformFanOutShape<Integer, Integer> bcast = builder.graph(broadcast.create(2)); final UniformFanInShape<Integer, Integer> merge = builder.graph(merge.create(2)); builder.from(in).via(f1).via(bcast).via(f2).via(merge).via(f3.grouped(1000)).to(out); builder.from(bcast).via(f4).to(merge); ); Note: Junction reference equality defines graph node equality (i.e. the same merge instance used in a FlowGraph refers to the same location in the resulting graph). By looking at the snippets above, it should be apparent that the builder object is mutable. The reason for this design choice is to enable simpler creation of complex graphs, which may even contain cycles. Once the FlowGraph has been constructed though, the RunnableGraph instance is immutable, thread-safe, and freely shareable. The same is true of all flow pieces sources, sinks, and flows once they are constructed. This means that you can safely re-use one given Flow in multiple places in a processing graph. We have seen examples of such re-use already above: the merge and broadcast junctions were imported into the graph using builder.graph(...), an operation that will make a copy of the blueprint that is passed to it and return the inlets and outlets of the resulting copy so that they can be wired up. Another alternative is to pass existing graphs of any shape into the factory method that produces a new graph. The difference between these approaches is that importing using b.graph(...) ignores the materialized value of the imported graph while importing via the factory method allows its inclusion; for more details see stream-materialization-scala. In the example below we prepare a graph that consists of two parallel streams, in which we re-use the same instance of Flow, yet it will properly be materialized as two connections between the corresponding Sources and Sinks: final Sink<Integer, Future<Integer>> topheadsink = Sink.head(); final Sink<Integer, Future<Integer>> bottomheadsink = Sink.head(); final Flow<Integer, Integer, BoxedUnit> shareddoubler = Flow.of(Integer.class).map(elem -> elem * final RunnableGraph<Pair<Future<Integer>, Future<Integer>>> g = FlowGraph.factory().closed( topheadsink, // import this sink into the graph bottomheadsink, // and this as well Keep.both(), 1.5. Working with Graphs 17

20 (b, top, bottom) -> { final UniformFanOutShape<Integer, Integer> bcast = b.graph(broadcast.create(2)); b.from(source.single(1)).via(bcast).via(shareddoubler).to(top); b.from(bcast).via(shareddoubler).to(bottom); ); Constructing and combining Partial Flow Graphs Sometimes it is not possible (or needed) to construct the entire computation graph in one place, but instead construct all of its different phases in different places and in the end connect them all into a complete graph and run it. This can be achieved using FlowGraph.factory().partial() instead of FlowGraph.factory().closed(), which will return a Graph instead of a RunnableGraph. The reason of representing it as a different type is that a RunnableGraph requires all ports to be connected, and if they are not it will throw an exception at construction time, which helps to avoid simple wiring errors while working with graphs. A partial flow graph however allows you to return the set of yet to be connected ports from the code block that performs the internal wiring. Let s imagine we want to provide users with a specialized element that given 3 inputs will pick the greatest int value of each zipped triple. We ll want to expose 3 input ports (unconnected sources) and one output port (unconnected sink). final Graph<FanInShape2<Integer, Integer, Integer>, BoxedUnit> zip = ZipWith.create((Integer left, Integer right) -> Math.max(left, right)); final Graph<UniformFanInShape<Integer, Integer>, BoxedUnit> pickmaxofthree = FlowGraph.factory().partial(builder -> { final FanInShape2<Integer, Integer, Integer> zip1 = builder.graph(zip); final FanInShape2<Integer, Integer, Integer> zip2 = builder.graph(zip); builder.edge(zip1.out(), zip2.in0()); // return the shape, which has three inputs and one output return new UniformFanInShape<Integer, Integer>(zip2.out(), new Inlet[] {zip1.in0(), zip1.in1(), zip2.in1()); ); final Sink<Integer, Future<Integer>> resultsink = Sink.<Integer>head(); final RunnableGraph<Future<Integer>> g = FlowGraph.factory().closed(resultSink, (builder, sink) -> { // import the partial flow graph explicitly final UniformFanInShape<Integer, Integer> pm = builder.graph(pickmaxofthree); builder.from(source.single(1)).to(pm.in(0)); builder.from(source.single(2)).to(pm.in(1)); builder.from(source.single(3)).to(pm.in(2)); builder.from(pm.out()).to(sink); ); final Future<Integer> max = g.run(mat); As you can see, first we construct the partial graph that describes how to compute the maximum of two input streams, then we reuse that twice while constructing the partial graph that extends this to three input streams, then we import it (all of its nodes and connections) explicitly to the FlowGraph instance in which all the undefined elements are rewired to real sources and sinks. The graph can then be run and yields the expected result Working with Graphs 18

21 Warning: Please note that a FlowGraph is not able to provide compile time type-safety about whether or not all elements have been properly connected this validation is performed as a runtime check during the graph s instantiation. A partial flow graph also verifies that all ports are either connected or part of the returned Shape Constructing Sources, Sinks and Flows from Partial Graphs Instead of treating a PartialFlowGraph as simply a collection of flows and junctions which may not yet all be connected it is sometimes useful to expose such a complex graph as a simpler structure, such as a Source, Sink or Flow. In fact, these concepts can be easily expressed as special cases of a partially connected graph: Source is a partial flow graph with exactly one output, that is it returns a SourceShape. Sink is a partial flow graph with exactly one input, that is it returns a SinkShape. Flow is a partial flow graph with exactly one input and exactly one output, that is it returns a FlowShape. Being able to hide complex graphs inside of simple elements such as Sink / Source / Flow enables you to easily create one complex element and from there on treat it as simple compound stage for linear computations. In order to create a Source from a partial flow graph Source provides a special apply method that takes a function that must return an Outlet. This unconnected sink will become the sink that must be attached before this Source can run. Refer to the example below, in which we create a Source that zips together two numbers, to see this graph construction in action: // first create an indefinite source of integer numbers class Ints implements Iterator<Integer> { private int next = 0; public boolean hasnext() { return true; public Integer next() { return next++; final Source<Integer, BoxedUnit> ints = Source.fromIterator(() -> new Ints()); final Source<Pair<Integer, Integer>, BoxedUnit> pairs = Source.factory().create( builder -> { final FanInShape2<Integer, Integer, Pair<Integer, Integer>> zip = builder.graph(zip.create()); builder.from(ints.filter(i -> i % 2 == 0)).to(zip.in0()); builder.from(ints.filter(i -> i % 2 == 1)).to(zip.in1()); return zip.out(); ); final Future<Pair<Integer, Integer>> firstpair = pairs.runwith(sink.<pair<integer, Integer>>head(), mat); Similarly the same can be done for a Sink<T>, in which case the returned value must be an Inlet<T>. For defining a Flow<T> we need to expose both an undefined source and sink: final Flow<Integer, Pair<Integer, String>, BoxedUnit> pairs = Flow.factory().create( b -> { final UniformFanOutShape<Integer, Integer> bcast = b.graph(broadcast.create(2)); final FanInShape2<Integer, String, Pair<Integer, String>> zip = 1.5. Working with Graphs 19

22 b.graph(zip.create()); b.from(bcast).to(zip.in0()); b.from(bcast).via(flow.of(integer.class).map(i -> i.tostring())).to(zip.in1()); return new Pair<>(bcast.in(), zip.out()); ); Source.single(1).via(pairs).runWith(Sink.<Pair<Integer, String>>head(), mat); Bidirectional Flows A graph topology that is often useful is that of two flows going in opposite directions. Take for example a codec stage that serializes outgoing messages and deserializes incoming octet streams. Another such stage could add a framing protocol that attaches a length header to outgoing data and parses incoming frames back into the original octet stream chunks. These two stages are meant to be composed, applying one atop the other as part of a protocol stack. For this purpose exists the special type BidiFlow which is a graph that has exactly two open inlets and two open outlets. The corresponding shape is called BidiShape and is defined like this: /** * A bidirectional flow of elements that consequently has two inputs and two * outputs, arranged like this: * * {{{ * * In1 ~> ~> Out1 * bidi * Out2 <~ <~ In2 * * */ final case class BidiShape[-In1, +Out1, -In2, +Out2](in1: Inlet[In1], out1: Outlet[Out1], in2: Inlet[In2], out2: Outlet[Out2]) extends Shape { // implementation details elided... A bidirectional flow is defined just like a unidirectional Flow as demonstrated for the codec mentioned above: static interface Message { static class Ping implements Message { final int id; public Ping(int id) { this.id = id; public boolean equals(object o) { if (o instanceof Ping) { return ((Ping) o).id == id; else return false; public int hashcode() { return id; static class Pong implements Message { final int id; public Pong(int id) { this.id = id; public boolean equals(object o) { if (o instanceof Pong) { 1.5. Working with Graphs 20

23 return ((Pong) o).id == id; else return false; public int hashcode() { return id; public static ByteString tobytes(message msg) { // implementation details elided... public static Message frombytes(bytestring bytes) { // implementation details elided... public final BidiFlow<Message, ByteString, ByteString, Message, BoxedUnit> codecverbose = BidiFlow.factory().create(b -> { final FlowShape<Message, ByteString> top = b.graph(flow.<message> empty().map(bidiflowdoctest::tobytes)); final FlowShape<ByteString, Message> bottom = b.graph(flow.<bytestring> empty().map(bidiflowdoctest::frombytes)); return new BidiShape<>(top, bottom); ); public final BidiFlow<Message, ByteString, ByteString, Message, BoxedUnit> codec = BidiFlow.fromFunctions(BidiFlowDocTest::toBytes, BidiFlowDocTest::fromBytes); The first version resembles the partial graph constructor, while for the simple case of a functional 1:1 transformation there is a concise convenience method as shown on the last line. The implementation of the two functions is not difficult either: public static ByteString tobytes(message msg) { if (msg instanceof Ping) { final int id = ((Ping) msg).id; return new ByteStringBuilder().putByte((byte) 1).putInt(id, ByteOrder.LITTLE_ENDIAN).result(); else { final int id = ((Pong) msg).id; return new ByteStringBuilder().putByte((byte) 2).putInt(id, ByteOrder.LITTLE_ENDIAN).result(); public static Message frombytes(bytestring bytes) { final ByteIterator it = bytes.iterator(); switch(it.getbyte()) { case 1: return new Ping(it.getInt(ByteOrder.LITTLE_ENDIAN)); case 2: return new Pong(it.getInt(ByteOrder.LITTLE_ENDIAN)); default: throw new RuntimeException("message format error"); In this way you could easily integrate any other serialization library that turns an object into a sequence of bytes. The other stage that we talked about is a little more involved since reversing a framing protocol means that any received chunk of bytes may correspond to zero or more messages. This is best implemented using a PushPullStage (see also Using PushPullStage) Working with Graphs 21

24 public static ByteString addlengthheader(bytestring bytes) { final int len = bytes.size(); return new ByteStringBuilder().putInt(len, ByteOrder.LITTLE_ENDIAN).append(bytes).result(); public static class FrameParser extends PushPullStage<ByteString, ByteString> { // this holds the received but not yet parsed bytes private ByteString stash = ByteString.empty(); // this holds the current message length or -1 if at a boundary private int needed = -1; public SyncDirective onpull(context<bytestring> ctx) { return run(ctx); public SyncDirective onpush(bytestring bytes, Context<ByteString> ctx) { stash = stash.concat(bytes); return run(ctx); public TerminationDirective onupstreamfinish(context<bytestring> ctx) { if (stash.isempty()) return ctx.finish(); else return ctx.absorbtermination(); // we still have bytes to emit private SyncDirective run(context<bytestring> ctx) { if (needed == -1) { // are we at a boundary? then figure out next length if (stash.size() < 4) return pullorfinish(ctx); else { needed = stash.iterator().getint(byteorder.little_endian); stash = stash.drop(4); return run(ctx); // cycle back to possibly already emit the next chunk else if (stash.size() < needed) { // we are in the middle of a message, need more bytes return pullorfinish(ctx); else { // we have enough to emit at least one message, so do it final ByteString emit = stash.take(needed); stash = stash.drop(needed); needed = -1; return ctx.push(emit); /* * After having called absorbtermination() we cannot pull any more, so if we need * more data we will just have to give up. */ private SyncDirective pullorfinish(context<bytestring> ctx) { if (ctx.isfinishing()) return ctx.finish(); else return ctx.pull(); public final BidiFlow<ByteString, ByteString, ByteString, ByteString, BoxedUnit> framing = 1.5. Working with Graphs 22

25 BidiFlow.factory().create(b -> { final FlowShape<ByteString, ByteString> top = b.graph(flow.<bytestring> empty().map(bidiflowdoctest::addlengthheader)); final FlowShape<ByteString, ByteString> bottom = b.graph(flow.<bytestring> empty().transform(() -> new FrameParser())); return new BidiShape<>(top, bottom); ); With these implementations we can build a protocol stack and test it: /* construct protocol stack * * stack * * * ~> O~~o ~> o~~o ~> * Message codec ByteString framing ByteString * <~ O~~o <~ o~~o <~ * * */ final BidiFlow<Message, ByteString, ByteString, Message, BoxedUnit> stack = codec.atop(framing); // test it by plugging it into its own inverse and closing the right end final Flow<Message, Message, BoxedUnit> pingpong = Flow.<Message> empty().collect(new PFBuilder<Message, Message>().match(Ping.class, p -> new Pong(p.id)).build() ); final Flow<Message, Message, BoxedUnit> flow = stack.atop(stack.reversed()).join(pingpong); final Future<List<Message>> result = Source.from(Arrays.asList(0, 1, 2)).<Message> map(id -> new Ping(id)).via(flow).grouped(10).runWith(Sink.<List<Message>> head(), mat); final FiniteDuration onesec = Duration.create(1, TimeUnit.SECONDS); assertarrayequals( new Message[] { new Pong(0), new Pong(1), new Pong(2), Await.result(result, onesec).toarray(new Message[0])); This example demonstrates how BidiFlow subgraphs can be hooked together and also turned around with the.reversed() method. The test simulates both parties of a network communication protocol without actually having to open a network connection the flows can just be connected directly Accessing the materialized value inside the Graph In certain cases it might be necessary to feed back the materialized value of a Graph (partial, closed or backing a Source, Sink, Flow or BidiFlow). This is possible by using builder.materializedvalue which gives an Outlet that can be used in the graph as an ordinary source or outlet, and which will eventually emit the materialized value. If the materialized value is needed at more than one place, it is possible to call materializedvalue any number of times to acquire the necessary number of outlets. final Sink<Integer, Future<Integer>> foldsink = Sink.<Integer, Integer> fold(0, (a, b) -> { return a + b; ); final Flow<Future<Integer>, Integer, BoxedUnit> flatten = Flow.<Future<Integer>> empty().mapasync(4, x -> { return x; 1.5. Working with Graphs 23

26 ); final Flow<Integer, Integer, Future<Integer>> foldingflow = Flow.factory().create(foldSink, (b, fold) -> { return new Pair<>( fold.inlet(), b.from(b.materializedvalue()).via(flatten).out()); ); Be careful not to introduce a cycle where the materialized value actually contributes to the materialized value. The following example demonstrates a case where the materialized Future of a fold is fed back to the fold itself. // This cannot produce any value: final Source<Integer, Future<Integer>> cyclicsource = Source.factory().create(foldSink, (b, fold) -> { // - Fold cannot complete until its upstream mapasync completes // - mapasync cannot complete until the materialized Future produced by // fold completes // As a result this Source will never emit anything, and its materialited // Future will never complete b.from(b.materializedvalue()).via(flatten).to(fold); return b.from(b.materializedvalue()).via(flatten).out(); ); Graph cycles, liveness and deadlocks Cycles in bounded flow graphs need special considerations to avoid potential deadlocks and other liveness issues. This section shows several examples of problems that can arise from the presence of feedback arcs in stream processing graphs. The first example demonstrates a graph that contains a naive cycle. The graph takes elements from the source, prints them, then broadcasts those elements to a consumer (we just used Sink.ignore for now) and to a feedback arc that is merged back into the main via a Merge junction. // WARNING! The graph below deadlocks! final Flow<Integer, Integer, BoxedUnit> printflow = Flow.of(Integer.class).map(s -> { System.out.println(s); return s; ); FlowGraph.factory().closed(b -> { final UniformFanInShape<Integer, Integer> merge = b.graph(merge.create(2)); final UniformFanOutShape<Integer, Integer> bcast = b.graph(broadcast.create(2)); b.from(source).via(merge).via(printflow).via(bcast).to(sink.ignore()); b.to(merge).from(bcast); ); Running this we observe that after a few numbers have been printed, no more elements are logged to the console - all processing stops after some time. After some investigation we observe that: through merging from source we increase the number of elements flowing in the cycle by broadcasting back to the cycle we do not decrease the number of elements in the cycle Since Akka Streams (and Reactive Streams in general) guarantee bounded processing (see the Buffering section for more details) it means that only a bounded number of elements are buffered over any time span. Since our cycle gains more and more elements, eventually all of its internal buffers become full, backpressuring source forever. To be able to process more elements from source elements would need to leave the cycle somehow. If we modify our feedback loop by replacing the Merge junction with a MergePreferred we can avoid the deadlock. MergePreferred is unfair as it always tries to consume from a preferred input port if there are 1.5. Working with Graphs 24

27 elements available before trying the other lower priority input ports. Since we feed back through the preferred port it is always guaranteed that the elements in the cycles can flow. // WARNING! The graph below stops consuming from "source" after a few steps FlowGraph.factory().closed(b -> { final MergePreferredShape<Integer> merge = b.graph(mergepreferred.create(1)); final UniformFanOutShape<Integer, Integer> bcast = b.graph(broadcast.create(2)); b.from(source).via(merge).via(printflow).via(bcast).to(sink.ignore()); b.to(merge.preferred()).from(bcast); ); If we run the example we see that the same sequence of numbers are printed over and over again, but the processing does not stop. Hence, we avoided the deadlock, but source is still back-pressured forever, because buffer space is never recovered: the only action we see is the circulation of a couple of initial elements from source. Note: What we see here is that in certain cases we need to choose between boundedness and liveness. Our first example would not deadlock if there would be an infinite buffer in the loop, or vice versa, if the elements in the cycle would be balanced (as many elements are removed as many are injected) then there would be no deadlock. To make our cycle both live (not deadlocking) and fair we can introduce a dropping element on the feedback arc. In this case we chose the buffer() operation giving it a dropping strategy OverflowStrategy.dropHead. FlowGraph.factory().closed(b -> { final UniformFanInShape<Integer, Integer> merge = b.graph(merge.create(2)); final UniformFanOutShape<Integer, Integer> bcast = b.graph(broadcast.create(2)); final FlowShape<Integer, Integer> droppyflow = b.graph( Flow.of(Integer.class).buffer(10, OverflowStrategy.dropHead())); b.from(source).via(merge).via(printflow).via(bcast).to(sink.ignore()); b.to(merge).via(droppyflow).from(bcast); ); If we run this example we see that The flow of elements does not stop, there are always elements printed We see that some of the numbers are printed several times over time (due to the feedback loop) but on average the numbers are increasing in the long term This example highlights that one solution to avoid deadlocks in the presence of potentially unbalanced cycles (cycles where the number of circulating elements are unbounded) is to drop elements. An alternative would be to define a larger buffer with OverflowStrategy.fail which would fail the stream instead of deadlocking it after all buffer space has been consumed. As we discovered in the previous examples, the core problem was the unbalanced nature of the feedback loop. We circumvented this issue by adding a dropping element, but now we want to build a cycle that is balanced from the beginning instead. To achieve this we modify our first graph by replacing the Merge junction with a ZipWith. Since ZipWith takes one element from source and from the feedback arc to inject one element into the cycle, we maintain the balance of elements. // WARNING! The graph below never processes any elements FlowGraph.factory().closed(b -> { final FanInShape2<Integer, Integer, Integer> zip = b.graph(zipwith.create((integer left, Integer right) -> left)); final UniformFanOutShape<Integer, Integer> bcast = b.graph(broadcast.create(2)); b.from(source).to(zip.in0()); b.from(zip.out()).via(printflow).via(bcast).to(sink.ignore()); b.to(zip.in1()).from(bcast); ); Still, when we try to run the example it turns out that no element is printed at all! After some investigation we realize that: 1.5. Working with Graphs 25

28 In order to get the first element from source into the cycle we need an already existing element in the cycle In order to get an initial element in the cycle we need an element from source These two conditions are a typical chicken-and-egg problem. The solution is to inject an initial element into the cycle that is independent from source. We do this by using a Concat junction on the backwards arc that injects a single element using Source.single. FlowGraph.factory().closed(b -> { final FanInShape2<Integer, Integer, Integer> zip = b.graph(zipwith.create((integer left, Integer right) -> left)); final UniformFanOutShape<Integer, Integer> bcast = b.graph(broadcast.create(2)); final UniformFanInShape<Integer, Integer> concat = b.graph(concat.create()); b.from(source).to(zip.in0()); b.from(zip.out()).via(printflow).via(bcast).to(sink.ignore()); b.to(zip.in1()).via(concat).from(source.single(1)); b.to(concat).from(bcast); ); When we run the above example we see that processing starts and never stops. The important takeaway from this example is that balanced cycles often need an initial kick-off element to be injected into the cycle. 1.6 Modularity, Composition and Hierarchy Akka Streams provide a uniform model of stream processing graphs, which allows flexible composition of reusable components. In this chapter we show how these look like from the conceptual and API perspective, demonstrating the modularity aspects of the library Basics of composition and modularity Every processing stage used in Akka Streams can be imagined as a box with input and output ports where elements to be processed arrive and leave the stage. In this view, a Source is nothing else than a box with a single output port, or, a BidiFlow is a box with exactly two input and two output ports. In the figure below we illustrate the most common used stages viewed as boxes. The linear stages are Source, Sink and Flow, as these can be used to compose strict chains of processing stages. Fan-in and Fan-out stages have usually multiple input or multiple output ports, therefore they allow to 1.6. Modularity, Composition and Hierarchy 26

29 build more complex graph layouts, not just chains. BidiFlow stages are usually useful in IO related tasks, where there are input and output channels to be handled. Due to the specific shape of BidiFlow it is easy to stack them on top of each other to build a layered protocol for example. The TLS support in Akka is for example implemented as a BidiFlow. These reusable components already allow the creation of complex processing networks. What we have seen so far does not implement modularity though. It is desirable for example to package up a larger graph entity into a reusable component which hides its internals only exposing the ports that are meant to the users of the module to interact with. One good example is the Http server component, which is encoded internally as a BidiFlow which interfaces with the client TCP connection using an input-output port pair accepting and sending ByteString s, while its upper ports emit and receive HttpRequest and HttpResponse instances. The following figure demonstrates various composite stages, that contain various other type of stages internally, but hiding them behind a shape that looks like a Source, Flow, etc. One interesting example above is a Flow which is composed of a disconnected Sink and Source. This can be achieved by using the wrap() constructor method on Flow which takes the two parts as parameters. The example BidiFlow demonstrates that internally a module can be of arbitrary complexity, and the exposed ports can be wired in flexible ways. The only constraint is that all the ports of enclosed modules must be either connected to each other, or exposed as interface ports, and the number of such ports needs to match the requirement of the shape, for example a Source allows only one exposed output port, the rest of the internal ports must be properly connected Modularity, Composition and Hierarchy 27

30 These mechanics allow arbitrary nesting of modules. For example the following figure demonstrates a RunnableGraph that is built from a composite Source and a composite Sink (which in turn contains a composite Flow). The above diagram contains one more shape that we have not seen yet, which is called RunnableGraph. It turns out, that if we wire all exposed ports together, so that no more open ports remain, we get a module that is closed. This is what the RunnableGraph class represents. This is the shape that a Materializer can take and turn into a network of running entities that perform the task described. In fact, a RunnableGraph is a module itself, and (maybe somewhat surprisingly) it can be used as part of larger graphs. It is rarely useful to embed a closed graph shape in a larger graph (since it becomes an isolated island as there are no open port for communication with the rest of the graph), but this demonstrates the uniform underlying model. If we try to build a code snippet that corresponds to the above diagram, our first try might look like this: Source.single(0).map(i -> i + 1).filter(i -> i!= 0).map(i -> i - 2).to(Sink.fold(0, (acc, i) -> acc + i)); //... where is the nesting? It is clear however that there is no nesting present in our first attempt, since the library cannot figure out where we intended to put composite module boundaries, it is our responsibility to do that. If we are using the DSL provided by the Flow, Source, Sink classes then nesting can be achieved by calling one of the methods withattributes() or named() (where the latter is just a shorthand for adding a name attribute). The following code demonstrates how to achieve the desired nesting: final Source<Integer, BoxedUnit> nestedsource = Source.single(0) // An atomic source.map(i -> i + 1) // an atomic processing stage.named("nestedsource"); // wraps up the current Source and gives it a name final Flow<Integer, Integer, BoxedUnit> nestedflow = Flow.of(Integer.class).filter(i -> i!= 0) // an atomic processing stage.map(i -> i - 2) // another atomic processing stage.named("nestedflow"); // wraps up the Flow, and gives it a name final Sink<Integer, BoxedUnit> nestedsink = nestedflow.to(sink.fold(0, (acc, i) -> acc + i)) // wire an atomic sink to the nestedflow.named("nestedsink"); // wrap it up 1.6. Modularity, Composition and Hierarchy 28

31 // Create a RunnableGraph final RunnableGraph<BoxedUnit> runnablegraph = nestedsource.to(nestedsink); Once we have hidden the internals of our components, they act like any other built-in component of similar shape. If we hide some of the internals of our composites, the result looks just like if any other predefine component has been used: If we look at usage of built-in components, and our custom components, there is no difference in usage as the code snippet below demonstrates. // Create a RunnableGraph from our components final RunnableGraph<BoxedUnit> runnablegraph = nestedsource.to(nestedsink); // Usage is uniform, no matter if modules are composite or atomic final RunnableGraph<BoxedUnit> runnablegraph2 = Source.single(0).to(Sink.fold(0, (acc, i) -> acc + i)); Composing complex systems In the previous section we explored the possibility of composition, and hierarchy, but we stayed away from nonlinear, generalized graph components. There is nothing in Akka Streams though that enforces that stream processing layouts can only be linear. The DSL for Source and friends is optimized for creating such linear chains, as they are the most common in practice. There is a more advanced DSL for building complex graphs, that can be used if more flexibility is needed. We will see that the difference between the two DSLs is only on the surface: the concepts they operate on are uniform across all DSLs and fit together nicely. As a first example, let s look at a more complex layout: 1.6. Modularity, Composition and Hierarchy 29

32 The diagram shows a RunnableGraph (remember, if there are no unwired ports, the graph is closed, and therefore can be materialized) that encapsulates a non-trivial stream processing network. It contains fan-in, fanout stages, directed and non-directed cycles. The closed() method of the FlowGraph factory object allows the creation of a general closed graph. For example the network on the diagram can be realized like this: FlowGraph.factory().closed(builder -> { final Outlet<Integer> A = builder.source(source.single(0)); final UniformFanOutShape<Integer, Integer> B = builder.graph(broadcast.create(2)); final UniformFanInShape<Integer, Integer> C = builder.graph(merge.create(2)); final FlowShape<Integer, Integer> D = builder.graph(flow.of(integer.class).map(i -> i + 1)); final UniformFanOutShape<Integer, Integer> E = builder.graph(balance.create(2)); final UniformFanInShape<Integer, Integer> F = builder.graph(merge.create(2)); final Inlet<Integer> G = builder.sink(sink.foreach(system.out::println)); builder.from(f).to(c); builder.from(a).via(b).via(c).to(f); builder.from(b).via(d).via(e).to(f); builder.from(e).to(g); ); In the code above we used the implicit port numbering feature to make the graph more readable and similar to the diagram. It is possible to refer to the ports, so another version might look like this: FlowGraph.factory().closed(builder -> { final Outlet<Integer> A = builder.source(source.single(0)); final UniformFanOutShape<Integer, Integer> B = builder.graph(broadcast.create(2)); final UniformFanInShape<Integer, Integer> C = builder.graph(merge.create(2)); final FlowShape<Integer, Integer> D = builder.graph(flow.of(integer.class).map(i -> i + 1)); final UniformFanOutShape<Integer, Integer> E = builder.graph(balance.create(2)); final UniformFanInShape<Integer, Integer> F = builder.graph(merge.create(2)); final Inlet<Integer> G = builder.sink(sink.foreach(system.out::println)); builder.from(f.out()).to(c.in(0)); builder.from(a).to(b.in()); builder.from(b.out(0)).to(c.in(1)); builder.from(c.out()).to(f.in(0)); builder.from(b.out(1)).via(d).to(e.in()); 1.6. Modularity, Composition and Hierarchy 30

33 builder.from(e.out(0)).to(f.in(1)); builder.from(e.out(1)).to(g); ); Similar to the case in the first section, so far we have not considered modularity. We created a complex graph, but the layout is flat, not modularized. We will modify our example, and create a reusable component with the graph DSL. The way to do it is to use the partial() method on FlowGraph factory. If we remove the sources and sinks from the previous example, what remains is a partial graph: We can recreate a similar graph in code, using the DSL in a similar way than before: final Graph<FlowShape<Integer, Integer>, BoxedUnit> partial = FlowGraph.factory().partial(builder -> { final UniformFanOutShape<Integer, Integer> B = builder.graph(broadcast.create(2)); final UniformFanInShape<Integer, Integer> C = builder.graph(merge.create(2)); final UniformFanOutShape<Integer, Integer> E = builder.graph(balance.create(2)); final UniformFanInShape<Integer, Integer> F = builder.graph(merge.create(2)); builder.from(f.out()).to(c.in(0)); builder.from(b).via(c).to(f); builder.from(b).via(builder.graph(flow.of(integer.class).map(i -> i + 1))).via(E).to(F); return new FlowShape(B.in(), E.out(1)); ); The only new addition is the return value of the builder block, which is a Shape. All graphs (including Source, BidiFlow, etc) have a shape, which encodes the typed ports of the module. In our example there is exactly one input and output port left, so we can declare it to have a FlowShape by returning an instance of it. While it is possible to create new Shape types, it is usually recommended to use one of the matching built-in ones Modularity, Composition and Hierarchy 31

34 The resulting graph is already a properly wrapped module, so there is no need to call named() to encapsulate the graph, but it is a good practice to give names to modules to help debugging. Since our partial graph has the right shape, it can be already used in the simpler, linear DSL: Source.single(0).via(partial).to(Sink.ignore()); It is not possible to use it as a Flow yet, though (i.e. we cannot call.filter() on it), but Flow has a wrap() method that just adds the DSL to a FlowShape. There are similar methods on Source, Sink and BidiShape, so it is easy to get back to the simpler DSL if a graph has the right shape. For convenience, it is also possible to skip the partial graph creation, and use one of the convenience creator methods. To demonstrate this, we will create the following graph: The code version of the above closed graph might look like this: 1.6. Modularity, Composition and Hierarchy 32

35 // Convert the partial graph of FlowShape to a Flow to get // access to the fluid DSL (for example to be able to call.filter()) final Flow<Integer, Integer, BoxedUnit> flow = Flow.wrap(partial); // Simple way to create a graph backed Source final Source<Integer, BoxedUnit> source = Source.factory().create(builder -> { final UniformFanInShape<Integer, Integer> merge = builder.graph(merge.create(2)); builder.from(builder.source(source.single(0))).to(merge); builder.from(builder.source(source.from(arrays.aslist(2, 3, 4)))).to(merge); // Exposing exactly one output port return merge.out(); ); // Building a Sink with a nested Flow, using the fluid DSL final Sink<Integer, BoxedUnit> sink = Flow.of(Integer.class).map(i -> i * 2).drop(10).named("nestedFlow").to(Sink.head()); // Putting all together final RunnableGraph<BoxedUnit> closed = source.via(flow.filter(i -> i > 1)).to(sink); Note: All graph builder sections check if the resulting graph has all ports connected except the exposed ones and will throw an exception if this is violated. We are still in debt of demonstrating that RunnableGraph is a component just like any other, which can be embedded in graphs. In the following snippet we embed one closed graph in another: final RunnableGraph<BoxedUnit> closed1 = Source.single(0).to(Sink.foreach(System.out::println)); final RunnableGraph<BoxedUnit> closed2 = FlowGraph.factory().closed(builder -> { final ClosedShape embeddedclosed = builder.graph(closed1); ); The type of the imported module indicates that the imported module has a ClosedShape, and so we are not able to wire it to anything else inside the enclosing closed graph. Nevertheless, this island is embedded properly, and will be materialized just like any other module that is part of the graph. As we have demonstrated, the two DSLs are fully interoperable, as they encode a similar nested structure of boxes with ports, it is only the DSLs that differ to be as much powerful as possible on the given abstraction level. It is possible to embed complex graphs in the fluid DSL, and it is just as easy to import and embed a Flow, etc, in a larger, complex structure. We have also seen, that every module has a Shape (for example a Sink has a SinkShape) independently which DSL was used to create it. This uniform representation enables the rich composability of various stream processing entities in a convenient way Materialized values After realizing that RunnableGraph is nothing more than a module with no unused ports (it is an island), it becomes clear that after materialization the only way to communicate with the running stream processing logic is via some side-channel. This side channel is represented as a materialized value. The situation is similar to Actor s, where the Props instance describes the actor logic, but it is the call to actorof() that creates an actually running actor, and returns an ActorRef that can be used to communicate with the running actor itself. Since the Props can be reused, each call will return a different reference. When it comes to streams, each materialization creates a new running network corresponding to the blueprint that was encoded in the provided RunnableGraph. To be able to interact with the running network, each 1.6. Modularity, Composition and Hierarchy 33

36 materialization needs to return a different object that provides the necessary interaction capabilities. In other words, the RunnableGraph can be seen as a factory, which creates: a network of running processing entities, inaccessible from the outside a materialized value, optionally providing a controlled interaction capability with the network Unlike actors though, each of the processing stages might provide a materialized value, so when we compose multiple stages or modules, we need to combine the materialized value as well (there are default rules which make this easier, for example to() and via() takes care of the most common case of taking the materialized value to the left. See flow-combine-mat-scala for details). We demonstrate how this works by a code example and a diagram which graphically demonstrates what is happening. The propagation of the individual materialized values from the enclosed modules towards the top will look like this: To implement the above, first, we create a composite Source, where the enclosed Source have a materialized type of Promise. By using the combiner function Keep.left(), the resulting materialized type is of the nested module (indicated by the color red on the diagram): // Materializes to Promise<BoxedUnit> (red) final Source<Integer, Promise<BoxedUnit>> source = Source.<Integer> lazyempty(); // Materializes to BoxedUnit (black) final Flow<Integer, Integer, BoxedUnit> flow1 = Flow.of(Integer.class).take(100); // Materializes to Promise<BoxedUnit> (red) final Source<Integer, Promise<BoxedUnit>> nestedsource = source.viamat(flow1, Keep.left()).named("nestedSource"); Next, we create a composite Flow from two smaller components. Here, the second enclosed Flow has a materialized type of Future, and we propagate this to the parent by using Keep.right() as the combiner function (indicated by the color yellow on the diagram): // Materializes to BoxedUnit (orange) final Flow<Integer, ByteString, BoxedUnit> flow2 = Flow.of(Integer.class).map(i -> ByteString.fromString(i.toString())); 1.6. Modularity, Composition and Hierarchy 34

37 // Materializes to Future<OutgoingConnection> (yellow) final Flow<ByteString, ByteString, Future<OutgoingConnection>> flow3 = Tcp.get(system).outgoingConnection("localhost", 8080); // Materializes to Future<OutgoingConnection> (yellow) final Flow<Integer, ByteString, Future<OutgoingConnection>> nestedflow = flow2.viamat(flow3, Keep.right()).named("nestedFlow"); As a third step, we create a composite Sink, using our nestedflow as a building block. In this snippet, both the enclosed Flow and the folding Sink has a materialized value that is interesting for us, so we use Keep.both() to get a Pair of them as the materialized type of nestedsink (indicated by the color blue on the diagram) // Materializes to Future<String> (green) final Sink<ByteString, Future<String>> sink = Sink.fold("", (acc, i) -> acc + i.utf8string()); // Materializes to Pair<Future<OutgoingConnection>, Future<String>> (blue) final Sink<Integer, Pair<Future<OutgoingConnection>, Future<String>>> nestedsink = nestedflow.tomat(sink, Keep.both()); As the last example, we wire together nestedsource and nestedsink and we use a custom combiner function to create a yet another materialized type of the resulting RunnableGraph. This combiner function just ignores the Future part, and wraps the other two values in a custom case class MyClass (indicated by color purple on the diagram): static class MyClass { private Promise<BoxedUnit> p; private OutgoingConnection conn; public MyClass(Promise<BoxedUnit> p, OutgoingConnection conn) { this.p = p; this.conn = conn; public void close() { p.success(scala.runtime.boxedunit.unit); static class Combiner { static Future<MyClass> f(promise<boxedunit> p, Pair<Future<OutgoingConnection>, Future<String>> rest) { return rest.first().map(new Mapper<OutgoingConnection, MyClass>() { public MyClass apply(outgoingconnection c) { return new MyClass(p, c);, system.dispatcher()); // Materializes to Future<MyClass> (purple) final RunnableGraph<Future<MyClass>> runnablegraph = nestedsource.tomat(nestedsink, Combiner::f); Note: The nested structure in the above example is not necessary for combining the materialized values, it just demonstrates how the two features work together. See Combining materialized values for further examples of combining materialized values without nesting and hierarchy involved Modularity, Composition and Hierarchy 35

38 1.6.4 Attributes We have seen that we can use named() to introduce a nesting level in the fluid DSL (and also explicit nesting by using partial() from FlowGraph). Apart from having the effect of adding a nesting level, named() is actually a shorthand for calling withattributes(attributes.name("somename")). Attributes provide a way to fine-tune certain aspects of the materialized running entity. For example buffer sizes can be controlled via attributes (see stream-buffers-scala). When it comes to hierarchic composition, attributes are inherited by nested modules, unless they override them with a custom value. The code below, a modification of an earlier example sets the inputbuffer attribute on certain modules, but not on others: final Source<Integer, BoxedUnit> nestedsource = Source.single(0).map(i -> i + 1).named("nestedSource"); // Wrap, no inputbuffer set final Flow<Integer, Integer, BoxedUnit> nestedflow = Flow.of(Integer.class).filter(i -> i!= 0).via(Flow.of(Integer.class).map(i -> i - 2).withAttributes(Attributes.inputBuffer(4, 4))) // override.named("nestedflow"); // Wrap, no inputbuffer set final Sink<Integer, BoxedUnit> nestedsink = nestedflow.to(sink.fold(0, (acc, i) -> acc + i)) // wire an atomic sink to the nestedflow.withattributes(attributes.name("nestedsink").and(attributes.inputbuffer(3, 3))); // override The effect is, that each module inherits the inputbuffer attribute from its enclosing parent, unless it has the same attribute explicitly set. nestedsource gets the default attributes from the materializer itself. nestedsink on the other hand has this attribute set, so it will be used by all nested modules. nestedflow will inherit from nestedsink except the map stage which has again an explicitly provided attribute overriding the inherited one Modularity, Composition and Hierarchy 36

39 This diagram illustrates the inheritance process for the example code (representing the materializer default attributes as the color red, the attributes set on nestedsink as blue and the attributes set on nestedflow as green). 1.7 Buffers and working with rate Akka Streams processing stages are asynchronous and pipelined by default which means that a stage, after handing out an element to its downstream consumer is able to immediately process the next message. To demonstrate what we mean by this, let s take a look at the following example: Source.from(Arrays.asList(1, 2, 3)).map(i -> {System.out.println("A: " + i); return i;).map(i -> {System.out.println("B: " + i); return i;).map(i -> {System.out.println("C: " + i); return i;).runwith(sink.ignore(), mat); Running the above example, one of the possible outputs looks like this: A: 1 A: 2 B: 1 A: 3 B: 2 C: 1 B: 3 C: 2 C: 3 Note that the order is not A:1, B:1, C:1, A:2, B:2, C:2, which would correspond to a synchronous execution model where an element completely flows through the processing pipeline before the next element enters the flow. The next element is processed by a stage as soon as it is emitted the previous one. While pipelining in general increases throughput, in practice there is a cost of passing an element through the asynchronous (and therefore thread crossing) boundary which is significant. To amortize this cost Akka Streams uses a windowed, batching backpressure strategy internally. It is windowed because as opposed to a Stop-And- Wait protocol multiple elements might be in-flight concurrently with requests for elements. It is also batching because a new element is not immediately requested once an element has been drained from the window-buffer but multiple elements are requested after multiple elements have been drained. This batching strategy reduces the communication cost of propagating the backpressure signal through the asynchronous boundary. While this internal protocol is mostly invisible to the user (apart form its throughput increasing effects) there are situations when these details get exposed. In all of our previous examples we always assumed that the rate of the processing chain is strictly coordinated through the backpressure signal causing all stages to process no faster than the throughput of the connected chain. There are tools in Akka Streams however that enable the rates of different segments of a processing chain to be detached or to define the maximum throughput of the stream through external timing sources. These situations are exactly those where the internal batching buffering strategy suddenly becomes non-transparent Buffers in Akka Streams Internal buffers and their effect As we have explained, for performance reasons Akka Streams introduces a buffer for every processing stage. The purpose of these buffers is solely optimization, in fact the size of 1 would be the most natural choice if there would be no need for throughput improvements. Therefore it is recommended to keep these buffer sizes small, and increase them only to a level suitable for the throughput requirements of the application. Default buffer sizes can be set through configuration: 1.7. Buffers and working with rate 37

40 akka.stream.materializer.max-input-buffer-size = 16 Alternatively they can be set by passing a ActorMaterializerSettings to the materializer: final Materializer materializer = ActorMaterializer.create( ActorMaterializerSettings.create(system).withInputBuffer(64, 64), system); If the buffer size needs to be set for segments of a Flow only, it is possible by defining a separate Flow with these attributes: final Flow<Integer, Integer, BoxedUnit> flow1 = Flow.of(Integer.class).map(elem -> elem * 2) // the buffer size of this map is 1.withAttributes(Attributes.inputBuffer(1, 1)); final Flow<Integer, Integer, BoxedUnit> flow2 = flow1.via( Flow.of(Integer.class).map(elem -> elem / 2)); // the buffer size of this map is the default Here is an example of a code that demonstrate some of the issues caused by internal buffers: final FiniteDuration onesecond = FiniteDuration.create(1, TimeUnit.SECONDS); final Source<String, Cancellable> msgsource = Source.from(oneSecond, onesecond, "message!"); final Source<String, Cancellable> ticksource = Source.from(oneSecond.mul(3), onesecond.mul(3), "tick"); final Flow<String, Integer, BoxedUnit> conflate = Flow.of(String.class).conflate( first -> 1, (count, elem) -> count + 1); FlowGraph.factory().closed(b -> { final FanInShape2<String, Integer, Integer> zipper = b.graph(zipwith.create((string tick, Integer count) -> count)); b.from(msgsource).via(conflate).to(zipper.in1()); b.from(ticksource).to(zipper.in0()); b.from(zipper.out()).to(sink.foreach(elem -> System.out.println(elem))); ).run(mat); Running the above example one would expect the number 3 to be printed in every 3 seconds (the conflate step here is configured so that it counts the number of elements received before the downstream ZipWith consumes them). What is being printed is different though, we will see the number 1. The reason for this is the internal buffer which is by default 16 elements large, and prefetches elements before the ZipWith starts consuming them. It is possible to fix this issue by changing the buffer size of ZipWith (or the whole graph) to 1. We will still see a leading 1 though which is caused by an initial prefetch of the ZipWith element. Note: In general, when time or rate driven processing stages exhibit strange behavior, one of the first solutions to try should be to decrease the input buffer of the affected elements to 1. Explicit user defined buffers The previous section explained the internal buffers of Akka Streams used to reduce the cost of crossing elements through the asynchronous boundary. These are internal buffers which will be very likely automatically tuned in future versions. In this section we will discuss explicit user defined buffers that are part of the domain logic of the stream processing pipeline of an application. The example below will ensure that 1000 jobs (but not more) are dequeued from an external (imaginary) system and stored locally in memory - relieving the external system: 1.7. Buffers and working with rate 38

41 // Getting a stream of jobs from an imaginary external system as a Source final Source<Job, BoxedUnit> jobs = inboundjobsconnector; jobs.buffer(1000, OverflowStrategy.backpressure()); The next example will also queue up 1000 jobs locally, but if there are more jobs waiting in the imaginary external systems, it makes space for the new element by dropping one element from the tail of the buffer. Dropping from the tail is a very common strategy but it must be noted that this will drop the youngest waiting job. If some fairness is desired in the sense that we want to be nice to jobs that has been waiting for long, then this option can be useful. jobs.buffer(1000, OverflowStrategy.dropTail()); Instead of dropping the youngest element from the tail of the buffer a new element can be dropped without enqueueing it to the buffer at all. jobs.buffer(1000, OverflowStrategy.dropNew()); Here is another example with a queue of 1000 jobs, but it makes space for the new element by dropping one element from the head of the buffer. This is the oldest waiting job. This is the preferred strategy if jobs are expected to be resent if not processed in a certain period. The oldest element will be retransmitted soon, (in fact a retransmitted duplicate might be already in the queue!) so it makes sense to drop it first. jobs.buffer(1000, OverflowStrategy.dropHead()); Compared to the dropping strategies above, dropbuffer drops all the 1000 jobs it has enqueued once the buffer gets full. This aggressive strategy is useful when dropping jobs is preferred to delaying jobs. jobs.buffer(1000, OverflowStrategy.dropBuffer()); If our imaginary external job provider is a client using our API, we might want to enforce that the client cannot have more than 1000 queued jobs otherwise we consider it flooding and terminate the connection. This is easily achievable by the error strategy which simply fails the stream once the buffer gets full. jobs.buffer(1000, OverflowStrategy.fail()); Rate transformation Understanding conflate When a fast producer can not be informed to slow down by backpressure or some other signal, conflate might be useful to combine elements from a producer until a demand signal comes from a consumer. Below is an example snippet that summarizes fast stream of elements to a standart deviation, mean and count of elements that have arrived while the stats have been calculated. final Flow<Double, Tuple3<Double, Double, Integer>, BoxedUnit> statsflow = Flow.of(Double.class).conflate(elem -> Collections.singletonList(elem), (acc, elem) -> { return Stream.concat(acc.stream(), Collections.singletonList(elem).stream()).collect(Collectors.toList()); ).map(s -> { final Double mean = s.stream().maptodouble(d -> d).sum() / s.size(); final DoubleStream se = s.stream().maptodouble(x -> Math.pow(x - mean, 2)); final Double stddev = Math.sqrt(se.sum() / s.size()); return new Tuple3(stdDev, mean, s.size()); ); This example demonstrates that such flow s rate is decoupled. Element rate at the start of the flow can be much higher that the element rate at the end of the flow Buffers and working with rate 39

42 Another possible use of conflate is to not consider all elements for summary when producer starts getting too fast. Example below demonstrates how conflate can be used to implement random drop of elements when consumer is not able to keep up with the producer. final Double p = 0.01; final Flow<Double, Double, BoxedUnit> sampleflow = Flow.of(Double.class).conflate(elem -> Collections.singletonList(elem), (acc, elem) -> { if (r.nextdouble() < p) { return Stream.concat(acc.stream(), Collections.singletonList(elem).stream()).collect(Collectors.toList()); return acc; ).mapconcat(d -> d); Understanding expand Expand helps to deal with slow producers which are unable to keep up with the demand coming from consumers. Expand allows to extrapolate a value to be sent as an element to a consumer. As a simple use of expand here is a flow that sends the same element to consumer when producer does not send any new elements. final Flow<Double, Double, BoxedUnit> lastflow = Flow.of(Double.class).expand(d -> d, s -> new Pair(s, s)); Expand also allows to keep some state between demand requests from the downstream. Leveraging this, here is a flow that tracks and reports a drift between fast consumer and slow producer. final Flow<Double, Pair<Double, Integer>, BoxedUnit> driftflow = Flow.of(Double.class).expand(d -> new Pair<Double, Integer>(d, 0), t -> { return new Pair(t, new Pair(t.first(), t.second() + 1)); ); Note that all of the elements coming from upstream will go through expand at least once. This means that the output of this flow is going to report a drift of zero if producer is fast enough, or a larger drift otherwise. 1.8 Custom stream processing While the processing vocabulary of Akka Streams is quite rich (see the Streams Cookbook for examples) it is sometimes necessary to define new transformation stages either because some functionality is missing from the stock operations, or for performance reasons. In this part we show how to build custom processing stages and graph junctions of various kinds Custom linear processing stages To extend the available transformations on a Flow or Source one can use the transform() method which takes a factory function returning a Stage. Stages come in different flavors swhich we will introduce in this page. Using PushPullStage The most elementary transformation stage is the PushPullStage which can express a large class of algorithms working on streams. A PushPullStage can be illustrated as a box with two input and two output ports as it is seen in the illustration below Custom stream processing 40

43 The input ports are implemented as event handlers onpush(elem,ctx) and onpull(ctx) while output ports correspond to methods on the Context object that is handed as a parameter to the event handlers. By calling exactly one output port method we wire up these four ports in various ways which we demonstrate shortly. Warning: There is one very important rule to remember when working with a Stage. Exactly one method should be called on the currently passed Context exactly once and as the last statement of the handler where the return type of the called method matches the expected return type of the handler. Any violation of this rule will almost certainly result in unspecified behavior (in other words, it will break in spectacular ways). Exceptions to this rule are the query methods isholding() and isfinishing() To illustrate these concepts we create a small PushPullStage that implements the map transformation. Map calls ctx.push() from the onpush() handler and it also calls ctx.pull() form the onpull handler resulting in the conceptual wiring above, and fully expressed in code below: public class Map<A, B> extends PushPullStage<A, B> { private final Function<A, B> f; public Map(Function<A, B> f) { this.f = f; public SyncDirective onpush(a elem, Context<B> ctx) { return ctx.push(f.apply(elem)); 1.8. Custom stream processing 41

44 public SyncDirective onpull(context<b> ctx) { return ctx.pull(); Map is a typical example of a one-to-one transformation of a stream. To demonstrate a many-to-one stage we will implement filter. The conceptual wiring of Filter looks like this: As we see above, if the given predicate matches the current element we are propagating it downwards, otherwise we return the ball to our upstream so that we get the new element. This is achieved by modifying the map example by adding a conditional in the onpush handler and decide between a ctx.pull() or ctx.push() call (and of course not having a mapping f function). public class Filter<A> extends PushPullStage<A, A> { private final Predicate<A> p; public Filter(Predicate<A> p) { this.p = p; public SyncDirective onpush(a elem, Context<A> ctx) { if (p.test(elem)) return ctx.push(elem); else return ctx.pull(); public SyncDirective onpull(context<a> ctx) { return ctx.pull(); To complete the picture we define a one-to-many transformation as the next step. We chose a straightforward example stage that emits every upstream element twice downstream. The conceptual wiring of this stage looks like this: 1.8. Custom stream processing 42

45 This is a stage that has state: the last element it has seen, and a flag oneleft that indicates if we have duplicated this last element already or not. Looking at the code below, the reader might notice that our onpull method is more complex than it is demonstrated by the figure above. The reason for this is completion handling, which we will explain a little bit later. For now it is enough to look at the if(!ctx.isfinishing) block which corresponds to the logic we expect by looking at the conceptual picture. class Duplicator<A> extends PushPullStage<A, A> { private A lastelem = null; private boolean oneleft = false; public SyncDirective onpush(a elem, Context<A> ctx) { lastelem = elem; oneleft = true; return ctx.push(elem); public SyncDirective onpull(context<a> ctx) { if (!ctx.isfinishing()) { // the main pulling logic is below as it is demonstrated on the illustration if (oneleft) { oneleft = false; return ctx.push(lastelem); else return ctx.pull(); else { // If we need to emit a final element after the upstream // finished if (oneleft) return ctx.pushandfinish(lastelem); else return ctx.finish(); public TerminationDirective onupstreamfinish(context<a> ctx) { return ctx.absorbtermination(); Finally, to demonstrate all of the stages above, we put them together into a processing chain, which conceptually would correspond to the following structure: 1.8. Custom stream processing 43

46 In code this is only a few lines, using the transform method to inject our custom processing into a stream: final RunnableGraph<Future<List<Integer>>> runnable = Source.from(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)).transform(() -> new Filter<Integer>(elem -> elem % 2 == 0)).transform(() -> new Duplicator<Integer>()).transform(() -> new Map<Integer, Integer>(elem -> elem / 2)).toMat(sink, Keep.right()); If we attempt to draw the sequence of events, it shows that there is one event token in circulation in a potential chain of stages, just like our conceptual railroad tracks representation predicts. Completion handling Completion handling usually (but not exclusively) comes into the picture when processing stages need to emit a few more elements after their upstream source has been completed. We have seen an example of this in our Duplicator class where the last element needs to be doubled even after the 1.8. Custom stream processing 44

47 upstream neighbor stage has been completed. Since the onupstreamfinish() handler expects a TerminationDirective as the return type we are only allowed to call ctx.finish(), ctx.fail() or ctx.absorbtermination(). Since the first two of these available methods will immediately terminate, our only option is absorbtermination(). It is also clear from the return type of onupstreamfinish that we cannot call ctx.push() but we need to emit elements somehow! The trick is that after calling absorbtermination() the onpull() handler will be called eventually, and at the same time ctx.isfinishing will return true, indicating that ctx.pull() cannot be called anymore. Now we are free to emit additional elementss and call ctx.finish() or ctx.pushandfinish() eventually to finish processing. The reason for this slightly complex termination sequence is that the underlying oncomplete signal of Reactive Streams may arrive without any pending demand, i.e. without respecting backpressure. This means that our push/pull structure that was illustrated in the figure of our custom processing chain does not apply to termination. Our neat model that is analogous to a ball that bounces back-and-forth in a pipe (it bounces back on Filter, Duplicator for example) cannot describe the termination signals. By calling absorbtermination() the execution environment checks if the conceptual token was above the current stage at that time (which means that it will never come back, so the environment immediately calls onpull) or it was below (which means that it will come back eventually, so the environment does not need to call anything yet). The first of the two scenarios is when a termination signal arrives after a stage passed the event to its downstream. As we can see in the following diagram, there is no need to do anything by absorbtermination() since the black arrows representing the movement of the event token is uninterrupted. In the second scenario the event token is somewhere upstream when the termination signal arrives. In this case absorbtermination needs to ensure that a new event token is generated replacing the old one that is forever gone (since the upstream finished). This is done by calling the onpull() event handler of the stage Custom stream processing 45

48 Observe, that in both scenarios onpull() kicks off the continuation of the processing logic, the only difference is whether it is the downstream or the absorbtermination() call that calls the event handler. Using PushStage Many one-to-one and many-to-one transformations do not need to override the onpull() handler at all since all they do is just propagate the pull upwards. For such transformations it is better to extend PushStage directly. For example our Map and Filter would look like this: public class Map2<A, B> extends PushStage<A, B> { private final Function<A, B> f; public Map2(Function<A, B> f) { this.f = f; public SyncDirective onpush(a elem, Context<B> ctx) { return ctx.push(f.apply(elem)); public class Filter2<A> extends PushStage<A, A> { private final Predicate<A> p; public Filter2(Predicate<A> p) { this.p = p; public SyncDirective onpush(a elem, Context<A> ctx) { if (p.test(elem)) return ctx.push(elem); else return ctx.pull(); The reason to use PushStage is not just cosmetic: internal optimizations rely on the fact that the onpull method only calls ctx.pull() and allow the environment do process elements faster than without this knowledge. By extending PushStage the environment can be sure that onpull() was not overridden since it is final on PushStage Custom stream processing 46

49 Using StatefulStage On top of PushPullStage which is the most elementary and low-level abstraction and PushStage that is a convenience class that also informs the environment about possible optimizations StatefulStage is a new tool that builds on PushPullStage directly, adding various convenience methods on top of it. It is possible to explicitly maintain state-machine like states using its become() method to encapsulates states explicitly. There is also a handy emit() method that simplifies emitting multiple values given as an iterator. To demonstrate this feature we reimplemented Duplicator in terms of a StatefulStage: public class Duplicator2<A> extends StatefulStage<A, A> { public StageState<A, A> initial() { return new StageState<A, A>() { public SyncDirective onpush(a elem, Context<A> ctx) { return emit(arrays.aslist(elem, elem).iterator(), ctx); ; Using DetachedStage The model described in previous sections, while conceptually simple, cannot describe all desired stages. The main limitation is the single-ball (single event token ) model which prevents independent progress of an upstream and downstream of a stage. Sometimes it is desirable to detach the progress (and therefore, rate) of the upstream and downstream of a stage, synchronizing only when needed. This is achieved in the model by representing a DetachedStage as a boundary between two single-ball regions. One immediate consequence of this difference is that it is not allowed to call ctx.pull() from onpull() and it is not allowed to call ctx.push() from onpush() as such combinations would steal a token from one region (resulting in zero tokens left) and would inject an unexpected second token to the other region. This is enforced by the expected return types of these callback functions. One of the important use-cases for DetachedStage is to build buffer-like entities, that allow independent progress of upstream and downstream stages when the buffer is not full or empty, and slowing down the appropriate side if the buffer becomes empty or full. The next diagram illustrates the event sequence for a buffer with capacity of two elements Custom stream processing 47

50 The very first difference we can notice is that our Buffer stage is automatically pulling its upstream on initialization. Remember that it is forbidden to call ctx.pull from onpull, therefore it is the task of the framework to kick off the first event token in the upstream region, which will remain there until the upstream stages stop. The diagram distinguishes between the actions of the two regions by colors: purple arrows indicate the actions involving the upstream event token, while red arrows show the downstream region actions. This demonstrates the clear separation of these regions, and the invariant that the number of tokens in the two regions are kept unchanged. For buffer it is necessary to detach the two regions, but it is also necessary to sometimes hold back the upstream or downstream. The new API calls that are available for DetachedStage s are the various ctx.holdxxx() methods, ctx.pushandpull() and variants, and ctx.isholdingxxx(). Calling ctx.holdxxx() from onpull() or onpush results in suspending the corresponding region from progress, and temporarily taking ownership of the event token. This state can be queried by ctx.isholding() which will tell if the stage is currently holding a token or not. It is only allowed to suspend one of the regions, not both, since that would disable all possible future events, resulting in a dead-lock. Releasing the held token is only possible by calling ctx.pushandpull(). This is to ensure that both the held token is released, and the triggering region gets its token back (one inbound token + one held token = two released tokens). The following code example demonstrates the buffer class corresponding to the message sequence chart we discussed. class Buffer2<T> extends DetachedStage<T, T> { final private Integer SIZE = 2; final private List<T> buf = new ArrayList<>(SIZE); private Integer capacity = SIZE; private boolean isfull() { return capacity == 0; 1.8. Custom stream processing 48