Definition
- Window TVF syntax – Call the window function in the FROM clause (e.g.,
TABLE(TUMBLE(...))) to add window_start and window_end columns, then GROUP BY them. - TUMBLE – Fixed-size, non-overlapping windows; each event in exactly one window.
- HOP – Fixed-size, overlapping windows that advance by a hop interval; one event can land in multiple windows.
- SESSION – Gap-based windows that close after a period of inactivity.
- CUMULATE – Growing windows sharing a start, emitting early partial results up to a final size.
- Watermark requirement – The time descriptor column must have a watermark defined; windows fire when the watermark passes the window end.
The National Bank of India wants to summarize transactions over time in different shapes. TUMBLE is like totaling transactions in back-to-back one-minute boxes. HOP is like a guard who every thirty seconds totals the last full minute, so the minutes overlap. SESSION groups a customer's burst of activity and closes after five quiet minutes. CUMULATE is like a running daily total reported every hour, each report covering from the top of the day up to now.
Understanding the Concepts
What is the window TVF syntax?
The modern Flink SQL approach wraps the source table in a window table-valued function inside the FROM clause, adding window_start, window_end, and window_time columns. You then GROUP BY those window columns alongside your keys.
-- TUMBLE window TVF: 1-minute non-overlapping windows
SELECT
window_start,
window_end,
account_id,
COUNT(*) AS txn_count,
SUM(amount) AS total_amount
FROM TABLE(
TUMBLE(
TABLE nbi_transactions,
DESCRIPTOR(event_time),
INTERVAL '1' MINUTE
)
)
GROUP BY window_start, window_end, account_id;
Result:
+---------------------+---------------------+------------+-----------+--------------+
| window_start | window_end | account_id | txn_count | total_amount |
+---------------------+---------------------+------------+-----------+--------------+
| 2024-03-15 10:00:00 | 2024-03-15 10:01:00 | ACC-001 | 5 | 250000.00 |
| 2024-03-15 10:01:00 | 2024-03-15 10:02:00 | ACC-001 | 3 | 180000.00 |
+---------------------+---------------------+------------+-----------+--------------+
How does TUMBLE work?
TUMBLE takes the table, the time descriptor, and a size. Each event belongs to exactly one non-overlapping window.
-- Per-branch total every 5 minutes
SELECT
window_start, window_end, branch_code,
SUM(amount) AS total
FROM TABLE(
TUMBLE(TABLE nbi_transactions,
DESCRIPTOR(event_time),
INTERVAL '5' MINUTE)
)
GROUP BY window_start, window_end, branch_code;
How does HOP (sliding) work?
HOP takes a slide and a size. A one-minute window sliding every thirty seconds means each event can belong to two windows, giving a smoother, more frequent view.
-- Sliding window: 1-minute size, 30-second slide
SELECT
window_start, window_end, account_id,
AVG(amount) AS avg_amount
FROM TABLE(
HOP(TABLE nbi_transactions,
DESCRIPTOR(event_time),
INTERVAL '30' SECOND, -- slide
INTERVAL '1' MINUTE) -- size
)
GROUP BY window_start, window_end, account_id;
-- Each event lands in size/slide = 60/30 = 2 windows
-- Output frequency: every 30 seconds
How does SESSION work?
SESSION groups events separated by less than a gap. A session window closes after a period of inactivity and merges dynamically.
-- Session window: 5-minute gap
SELECT
window_start, window_end, account_id,
COUNT(*) AS session_txn_count
FROM TABLE(
SESSION(TABLE nbi_transactions,
DESCRIPTOR(event_time),
INTERVAL '5' MINUTE) -- gap
)
GROUP BY window_start, window_end, account_id;
-- Rajesh's shopping burst: 5 transactions in 3 minutes → 1 session
-- 6 minutes of silence → session closes
-- Next transaction starts a new session
How does CUMULATE work?
CUMULATE takes a step and a max size, producing windows that all share a start and grow by the step up to the max. This is the clean way to emit early running totals.
-- Running daily total, reported every hour
SELECT
window_start, window_end, branch_code,
SUM(amount) AS running_total
FROM TABLE(
CUMULATE(TABLE nbi_transactions,
DESCRIPTOR(event_time),
INTERVAL '1' HOUR, -- step
INTERVAL '1' DAY) -- max size
)
GROUP BY window_start, window_end, branch_code;
CUMULATE windows for a day starting at midnight:
[00:00, 01:00) step 1: first hour's total
[00:00, 02:00) step 2: first two hours' total
[00:00, 03:00) step 3: first three hours' total
... ... progressive running total
[00:00, 24:00) final: full day's total
All share the same start (midnight)
How do you define watermarks for window TVFs?
The time descriptor column must be a time attribute with a watermark defined on the source table. Windows fire when the watermark passes the window end.
-- CREATE TABLE with watermark
CREATE TABLE nbi_transactions (
transaction_id STRING,
account_id STRING,
amount DECIMAL(15,2),
event_time TIMESTAMP(3),
WATERMARK FOR event_time AS event_time - INTERVAL '10' SECOND
) WITH (
'connector' = 'kafka',
'topic' = 'nbi-transactions',
'properties.bootstrap.servers' = 'broker1:9092',
'format' = 'json'
);
-- Watermark: event_time - 10 seconds
-- Windows fire when watermark passes window_end
What is the older syntax vs TVF syntax?
The older syntax used grouped window functions like TUMBLE_START in the GROUP BY directly. The TVF approach is now preferred because it is more expressive and supports windowing joins and top-N.
-- OLD syntax (still works but not recommended)
SELECT
TUMBLE_START(event_time, INTERVAL '1' MINUTE) AS wstart,
account_id,
COUNT(*) AS cnt
FROM nbi_transactions
GROUP BY TUMBLE(event_time, INTERVAL '1' MINUTE), account_id;
-- NEW TVF syntax (recommended)
SELECT window_start, account_id, COUNT(*) AS cnt
FROM TABLE(
TUMBLE(TABLE nbi_transactions,
DESCRIPTOR(event_time),
INTERVAL '1' MINUTE))
GROUP BY window_start, window_end, account_id;
What are the performance considerations?
HOP windows with a small slide relative to the size multiply work because each event lands in many windows. CUMULATE is the clean alternative for progressive early results without the overlap cost.
-- HOP: 1-minute window, 1-second slide
-- Each event lands in 60 windows! Very expensive.
HOP(TABLE t, DESCRIPTOR(event_time),
INTERVAL '1' SECOND, INTERVAL '1' MINUTE)
-- CUMULATE: progressive results without overlap cost
-- Each event lands in exactly 1 growing window per step
CUMULATE(TABLE t, DESCRIPTOR(event_time),
INTERVAL '10' MINUTE, INTERVAL '1' HOUR)
What should a senior engineer know?
A windowed aggregation generally produces an append-only result because each window emits once when it closes, which means you can write it to a plain append sink. This is unlike an unbounded GROUP BY that produces retractions. A stalled watermark means windows never fire. Size HOP windows carefully. Reach for CUMULATE when you want progressive early results.
Visual Overview
Flink SQL Window Types
TUMBLE (1 min):
|-------|-------|-------|-------|
| W1 | W2 | W3 | W4 | non-overlapping
HOP (1 min size, 30s slide):
|-------| |-------|
|-------| |-------|
|-------| |-------|
overlapping, each event in 2 windows
SESSION (5 min gap):
|--txns--| gap>5min |--txns--|
| Sess 1 | | Sess 2 | gap-based
CUMULATE (1 hour step, 1 day max):
|-------| step 1
|---------------| step 2
|----------------------| step 3
|------------------------------| final
All share the same start point
TVF Syntax:
FROM TABLE(TUMBLE(TABLE t, DESCRIPTOR(ts), INTERVAL '1' MINUTE))
GROUP BY window_start, window_end, key
Key Takeaway
Windowed aggregations in Flink SQL group an unbounded stream into finite time windows and aggregate over each. The modern syntax is the window table-valued function: you wrap your source in a window TVF in the FROM clause, and Flink adds window_start and window_end columns, then you GROUP BY them alongside your keys.
There are four shapes. TUMBLE gives fixed, non-overlapping windows. HOP gives overlapping windows that advance by a slide, providing a smoother view at the cost of more computation per event. SESSION groups bursts of activity and closes after an inactivity gap. CUMULATE produces growing windows that share a start and emit progressive running totals, which is the clean way to get early results without HOP's overlap cost.
The window TVF requires a watermark on the time descriptor column; windows fire when the watermark passes the window end. A windowed aggregation produces append-only results because each window emits once, so you can write to a plain append sink. Size HOP windows carefully because a small slide multiplies per-event work.
At the National Bank of India TUMBLE one-minute windows gave per-account counts for the fraud dashboard. CUMULATE with hourly steps up to a day gave leadership a running daily total that updated through the day. HOP was used sparingly for smooth operations views, with careful attention to the overlap cost.
Remember one sentence: Use the window TVF syntax with TUMBLE, HOP, SESSION, or CUMULATE in the FROM clause, GROUP BY window_start and window_end, and ensure a watermark is defined so windows actually fire.
---