KSQL - Visualizing windows (tumbling)

Windowing in KSQL allows grouping of the data stream into “windows” based on a unique key. The window can be overlapping or non-overlapping. In this example we will consider non-overlapping time windows.

Currently with Confluent KSQL 4.1 you can define tumbling window within a KTable as such:

CREATE TABLE clickstream.clickstream_window_1min AS
SELECT user_id,
count(*)
FROM clickstream.clickstream
WINDOW TUMBLING (SIZE 1 MINUTE)
GROUP BY user_id;

What this does is is basically create a non-overlapping windows of 1min each, which means a new “window” is created (appended) every 1 min, which reports the number of records for each user_id within that window (More info on the KSQL syntax here: https://docs.confluent.io/current/ksql/docs/syntax-reference.html#select)

Arcadia can be used to visualize these tumbling window groups in real time.

Here’s an example dashboard showing the raw events from the KTable along with a trellis bar chart.

image

The tabular has been sorted on ROWTIME so the most recent records appear on top. The trellis bar chart is setup is shown below:

image

A few things to note about the bar chart:

  1. The Y-axis shelf includes ROWTIME as a dimension this will generate the trellis
  2. You have to manually apply “agg” function to the measure.
  3. Sorts have been applied to the ROWTIME and user_id to make it easier to compare windows.

Once configured, as new data streams in, new rows are added to the trellis bar chart - one row for each time “window”.