Concurrency in Server-Side Dart

server-side-Dart

Let’s discuss concurrency in server-side Dart. I’ve been a fan of Erlang for many years, but it wasn’t until last year that I finally got to build a serious production application with it. That application maintains about 40,000 persistent connections, actively transmitting data in a binary, proprietary format to physical devices in people’s homes across the world. The exchanged information is posted to a service bus that stores data in CouchDB, sends push notifications and collects historical data in another document store.

The Erlang runtime is a great choice for managing many simultaneous persistent connections. If you are unfamiliar with Erlang’s model, it’s quite simple: every connection spawns a new, lightweight process that has its own event queue. These processes all get their fair share of CPU time, which works out well in a system where each connection is equally as valuable as another.

Much of the functionality required for this behavior is just baked into the Erlang VM and writing this code on another platform would have required more work. However, while Erlang is a beautifully simple language, it just isn’t as productive as other languages for many common tasks. While Elixir is an improvement, it still carries some of the burden of the Erlang VM.

Dart Efficiencies Drawn From Erlang

Concepts from Erlang have rightfully made their way into other platforms. Dart isolates are similar to Erlang’s concept of processes. While an isolate is not quite the same lightweight construct as an Erlang process, it is still a VM-managed thread that has its own heap and communicates with other isolates through messaging and an event queue. And while Google is marketing Dart as the frontend solution to end all problems, which it very well could be, I was drawn to its potential on the server because of isolates.

Unfortunately, there weren’t any mature server frameworks for Dart, and isolates weren’t even in the discussion for those that did exist. So, we gradually built our own framework – Aqueduct – over the course of the last two years.

Related: Releasing Aqueduct 2.0 – A Server-Side Web Framework Written in Dart

One of the core features of Aqueduct for Dart is spreading request handling across many isolates. Each isolate is a carbon-copy replica of the others – with the same routing table, database connections, etc. When the overall application receives an HTTP request, it is delivered to one of the replica isolates. Connection pooling is achieved without actually pooling connections – the entire application is ‘pooled,’ and each instance has its own set of resources, like database connections, to fulfill requests.

aqueduct_tree.pngAqueduct Tree

An important requirement for this feature is that it shouldn’t require the developer to necessarily understand isolates or multi-threading to reap the benefits. The structure of an Aqueduct application effectively enforces an application that can be distributed across many isolates. The number of isolates is a configurable value at startup, and there isn’t anything special to be done when building the application:

aqueduct serve --isolates 3

Of course, centralized resources and resource pools are sometimes necessary. For example, an application that opens a streaming connection to Nest should only have one connection in total, not one for each isolate. This is all possible through isolate messages. As an example, Aqueduct handles logging with a separate logging isolate.

When the application starts up, a logging isolate is spun up – its only job is to receive log messages and write them somewhere. A reference to the logging isolate is passed to each web server isolate. A web server isolate sends a message to the logging isolate when it wants to log something. The web server isolate moves on to do its job, while the logging isolate works on writing the message to its destination.

Setting this up is straightforward – a one-time initialization callback starts the logging isolate and stores the reference in a configuration object that is sent to each isolate on startup:

class class AppSink extends RequestSink {
  static Future initializeApplication(ApplicationConfiguration config) async {
    var loggingServer = new LoggingServer([new ConsoleBackend()]);

    // Spawns a new isolate
    await loggingServer.start();

    // Stores reference to isolate in config
    config.options["loggingTarget"] = loggingServer.getNewTarget();
  }

  AppSink(ApplicationConfiguration config) : super(config) {
    // Unpacks isolate reference, routes all logging messages
    // to logging isolate.
    var target = config.options["loggingTarget"];
    target.bind(logger);
  }

  Logger logger = new Logger("app");

  @override
  void setupRouter(Router router) {
    router
      .route("/log")
      .listen((req) async {
        // This sends the request’s body to the logging isolate
        logger.info("${await req.body.decodeAsString()}");

        return new Response.accepted();
      });
  }

(By the way, the above code is technically a complete, albeit simple and silly, Aqueduct application.)
We sincerely hope you’ll give Dart’s frameworks – Angular2 and Flutter and Aqueduct – a shot.
There’s a lot more to Aqueduct and you can find clear and pragmatic documentation here.

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *