Rx.js and d3.js in the Red Hat Summit Middleware Keynote Demo

Brian Leathem

4 minute read

At this year’s DevNation/Red Hat Summit I was part of the team that created the Red Hat Summit Middleware keynote demo. I made the custom front-ends in the demo using Reactive Extensions for javascript (Rx.js) to manipulate the datastreams from the various back-end systems and to transform that data into a form that can drive the UI using d3.js.

If you missed the keynote, you can watch it on YouTube below (the keynote demo starts at 19m). In this blog post I will cover at a high level the UIs I created for the keynote demo, setting the stage for a more in-depth analysis with subsequent posts.

Rx.js in the Keynote Demo

This year’s Red Hat Summit Middleware keynote demo showcased 3 major technologies at Red Hat:

  • IoT using Red Hat JBoss A/MQ as the message broker, and Apache Spark for streaming analytics

  • Docker at Scale using OpenShift

  • Mobile with the Red Hat Mobile Application platform

To showcase the data we collected from the systems, I built 3 user interfaces using Rx.js to manipulate the data, and d3.js to visualize it. Developing these UIs provided me with an excellent opportunity to dive deep into these libraries and learn many of their ins and outs.

IoT Beacon Location

The data collected for the IoT portion of our keynote demo consisted of millions of scan events from 300 Gimbal BLE beacons handed out to DevNation attendees. The scanners were made using Raspberry PIs that transmitted the scan events to a Red Hat A-MQ message broker. These numerous events were then processed using Apache Spark, reducing the millions of beacon events into tens-of-thousands of meaningful "business events" representing attendee movements throughout the conference.

IoT Map Visualization

The Spark-processed data was retrieved from the message broker into a node.js back-end, was stored in a mongodb datastore, and then was re-broadcast to the browser clients. I used Rx.js throughout both the node.js back-end and in the browser to:

  • wrap a STOMP over Websocket protocol connection.

  • wrap the websocket broadcasts and clients

  • filter and map the data streams

  • bind to DOM events

  • etc.

Using Rx.js throughout the application meant that any idiosyncrasies of the the various protocols were encapsulated within the Observable definition, and the rest of the application adhered to a consistent API for authoring Asynchronous javascript code.

The map UI uses d3.js to bind the beacon location data to SVG circles in the browser. I then used the d3.js force layout to animate the circles and move them around the map, mimicking the movement of the real-world beacons.

Watch the video of the keynote linked above to see the UI backed with real data. I additionally embedded a code-pen that shows the beacon-location visualization backed by simulated data. (If the animation is already complete, hover over the codepen and hit the "rerun" button in the bottom right).

See the Pen Beacon Location by Brian Leathem (@bleathem) on CodePen.

IoT broker Visualization

The map UI worked really well to display the business value of the beacon data collected. We needed a second UI however to demonstrate both the sheer volume of data that our A-MQ message broker was receiving from the Raspberry Pis, and the dramatic reduction in data volume as Spark processed the data.

Again using a STOMP over websockets connection to the message broker, I used Rx.js to manipulate the data, and d3.js to visualize it. This broker UI is running in the codepen below backed by simulated data.

See the Pen Broker by Brian Leathem (@bleathem) on CodePen.

1k Containers Visualization

The power of docker, kubernetes, and OpenShift was demonstrated with a rapid scale-up of 1026 containers live on stage. To visualize this scale-up, we developed a custom "hexboard" UI, where each hexagon represents an individual kubernetes pod. The lifecycle of each individual pod was reflected as a falling hexagon indicating each stage transition, culminating with the shadowman Red Hat logo when all pods were online.

I’ve included a codepen below showing the hexboard UI, again backed by "simulated" data.

See the Pen Hexboard by Brian Leathem (@bleathem) on CodePen.

Claim you container

Audience members were then provided an opportunity to claim their own piece of "the cloud" by drawing a sketch on their mobile devices and posting those sketches to an individual kubernetes pod via the Red Hat Mobile Application Platform back-end. The audience participation was incredible, and we quickly filled the hexboard with user-generated sketches!

The source

The source code for the beacon location UI, and the 1k container UI are both available on github. The goal over the next few weeks is to clean up the keynote demo code, and provide it in a form where folks can run it themselves. Stay tuned for further developments in this regard.