Swim - What is SWIM

Swim developer logo developer.

What is SWIM?


SWIM is a set of libraries that empowers the creation of transparently scalable, persistent, real-time applications. SWIM intrinsically combines the best features of web servers, application servers, stream processing software, and databases in a vertically integrated stack, allowing developers to write applications without stressing over infrastructure.

SWIM has three core libraries, each of which addresses a different set of the problems involved in building decentralized real-time applications:

  • SWIM-Server is a Java library that enables ingesting and processing data, transforming data into distributed objects, and persisting these objects. The objects form a stateful streaming API over websockets on top of which real-time applications can be built. Some core features include:

    • Websocket and HTTP support: Lightweight and ultra-fast http and websocket server that is backed by a buffer-free network stack which eliminates buffer bloat.
    • Messaging: Peer-to-peer messaging between components which uses eventual consistency semantics over guaranteed message delivery.
    • Persistence: Embedded, non-blocking, log structured data store with parallel compacting that bounds memory use.
    • Scheduling: Prioritized, demand-driven scheduling that bounds CPU use.

  • The SWIM-Client library allows external processes to access the aforementioned API. It currently has both Java and Javascript implementations, and it features:

    • Network stack: Uses the same network stack as, and thus receives the same benefits of, SWIM-Server.
    • Multiplexing: Multiplexes multiple connections to the same server with a single websocket connection.
    • Reconnects: Supports seamless reconnects in the case of a network outage.
    • Simple Embeddability: Contains non-intrusive scripts and libraries that are available after a simple import.

  • The SWIM-UI library, built on top of SWIM-Client, is a set of lightweight web components that can ingest and display real-time data from any Swim Server API. In addition to having all SWIM-Client advantages, SWIM-UI features

    • Trivial Embeddability: Web components can be directly added into existing HTML pages.

Programming Model


Building a SWIM API

Developers use SWIM-Server to create a streaming API. This API follows an object model where

  • Service definitions specify the behavior for all of their instances, much like "classes" do for "objects".
  • Lanes are non-static Service members that store mutable data, much like "fields" inside "objects". They are also the API endpoints.
  • Every Lane comes with a customizable callbacks that will trigger during stages of its update lifecycles. User-defined callback implementations are analogous to "methods". To those unaccustomed to the observer pattern, this may require some getting used to. The mindset should not be, "I will invoke this function", but rather, "Whenever I feed a value to Lane, my function will be invoked during a specific stage of the Lane's lifecycle".

This model was designed with scale in mind. Every Lane callback is asynchrously invoked; consequently, if we instantiate an arbitrary number of Services, the code in their Lanes' callbacks will run in parallel at no extra cost of development. SWIM performs well regardless of how many Services are instantiated, so the desired API should drive Service structure, not the other way around.

There is one more element in the SWIM model, but no analogy to classical object-oriented models exists for it. Addressing a Lane requires URIs for both its Lane and its containing Service. Each Lane has statically-known URI called the laneUri. Every Service, likewise, has a nodeUri, but it can't possibly be statically known if we want to dynamically instantiate Services with shared behavior.

In a SWIM application, the Plane stores URI patterns for Services and correspondingly routes incoming requests. A Service is lazily instantiated when a Plane routes a request to it for the first time, and subsequent requests will always route to this instantiation.

Plane
import swim.api.*; class SolarSystem extends AbstractPlane { // Declare nodeUri patterns with @SwimRoute // Accepts /planet/saturn, /planet/earth, etc. @SwimRoute("/planet/:name") final ServiceType<?> planet = serviceClass(PlanetService.class); // No dynamic components => only one sun! @SwimRoute("/sun") final ServiceType<?> sun = serviceClass(SunService.class); }
  • HTML
  • recon
Service with Lanes
import recon.*; import swim.api.*; class PlanetService extends AbstractService { // Declare laneUris with @SwimLane @SwimLane("co2") ValueLane<Double> co2 = valueLane() .valueForm(Form.DOUBLE) .didSet((cur,prev) -> { long now = System.currentTimeMillis(); // Can directly refer to same-instance lanes this.co2Hist.put(now, cur); }); @SwimLane("co2History") MapLane<Long, Double> co2Hist = mapLane() .keyForm(Form.LONG).valueForm(Form.DOUBLE); @SwimLane("self-destruct") CommandLane<Value> sd = commandLane(); }
  • HTML
  • recon

To summarize, the object model in SWIM-Server consists of:

  • Service definitions and declarations, analogous to object-oriented "classes" and "objects". Instances are identified with nodeUris. Services are the fundamental dynamically scalable units in SWIM.
  • Lanes, analogous to "fields". Lanes are identified with laneUris. Lanes are API endpoints in SWIM; addressing one requires both its laneUri and its enclosing Service's nodeUri.
  • User-defined Lane callback implementations, analogous to "methods".
  • A Plane that routes all requests into and within the SWIM Server.

Further reading:

Using a SWIM API

Once a SWIM API is built, it is accessible from both SWIM-Server and SWIM-Client (and transitively SWIM-UI), and even outside these libraries, to a very limited extent. But before we discuss API utilization, let's establish something that's easy to overlook:

  • Code inside a Service definition can directly refer to any Lanes within the same instance, just like with regular instance variables.

We show this in the PlanetService class above. Therefore, while it is possible to use the API to communicate with a Lane inside the same Service instance, it is usually unnecessary.

Communication between Service instances or from an external process, on the other hand, requires the API. The API is symmetric in both cases and exposes two options:

  • A Lane can be commanded with a value; this will send a "fire-and-forget" websocket message containing that value to the Lane. This is the only option without SWIM-Server or SWIM-Client.
  • A Downlink is a stateful reference to some Lane. Downlinks can synchronously read from and write to their underlying Lanes, and, similarly to Lanes themselves, they can be supplemented with callbacks that will trigger during specific lifecycle stages.

Enhanced Lanes
import recon.*; import swim.api.*; class PlanetService extends AbstractService { @SwimLane("co2") ValueLane<Double> co2 = valueLane() .valueForm(Form.DOUBLE) .didSet((cur,prev) -> { long now = System.currentTimeMillis(); // Don't need API here this.co2Hist.put(now, cur); if (cur > .97) { // But need it here command("/planet/pluto", "self-destruct", Value.of("password")); } }); @SwimLane("co2History") MapLane<Long, Double> co2Hist = mapLane() .keyForm(Form.LONG).valueForm(Form.DOUBLE); @SwimLane("self-destruct") CommandLane<Value> sd = commandLane() .onCommand(v -> { // A no-op. Pluto lives to see another day. }); }
  • HTML
  • recon
Sample SWIM-Client
import recon.*; import swim.api.*; import swim.client.*; class Client { static SwimClient sc = new SwimClient(); static String HOST = "ws://localhost:5620"; static { sc.start(); } final MapDownlink<Long, Double> history = sc .downlinkMap() .keyForm(Form.LONG).valueForm(Form.DOUBLE) .hostUri(HOST) .nodeUri("/planet/venus") .laneUri("co2History") .didUpdate((k,n,o) -> { if (n < .20) { client.command(HOST, "/planet/venus", "self-destruct", Value.of("password")); } }) .open(); }
  • HTML
  • recon

Further reading:

Recon

Recon is a simple, expressive, structural data format with the simplicity of JSON and the expressiveness of XML. Recon is, among other things, the sole message format in the SWIM API and the internal storage structure of Lanes.

This pervasiveness throughout the SWIM stack affects how a few code patterns, such as a Lane declarations (notice the Forms in PlanetService), are written. While it is completely possible to build SWIM applications knowing absolutely nothing about Recon other than these patterns, you will likely find its out-of-the-box features helpful during data processing.

Further reading:

Application Lifecycle


The SWIM API is bidirectional. Generally, the overview of data flow in a SWIM application is as follows:

  1. Through either a SWIM-Client instance or a websocket connection, external sources send data to specified Lanes in specified Service instances. Only when the sources cannot use the SWIM API directly do we need an ingress bridge.
  2. This triggers Lane callbacks, which can process this data and update any other Lanes, even those outside their Service instances, by using the SWIM API.
  3. A SwimClient can subscribe to, display, and modify data inside any of these Lanes.
  4. Optionally, Lanes themselves can push data to external data sinks via an egress bridge.
This flow is visualized in the diagram below. Solid lines denote usage of the SWIM API; note the bidirectionality of most these arrows. If ingestion into SWIM-Server is achieved by SWIM-Client rather than websockets, then the only unidirectional solid arrows can optionally become bidirectional. If the data sources themselves can use the SWIM API, then a Bridge does not need to be built.

There are just three steps to enable the above data flow.

  1. Write SWIM Services definitions with appropriate Lane declarations and configurations. Service instantiation is completely demand-driven at runtime.
  2. Define a Plane with URI route definitions of all the Services.
  3. Send data into Lanes, using Commands or Downlinks if you have a SwimClient instance or an external program with websockets if you don't.