Data pipeline
Real-Time Data Lakehouse. 10M+ events a day flow from Redpanda through Flink into versioned Iceberg tables, queried with Trino. Simulated live below.
Events / sec115
Events ingested0
Snapshots0
Quarantined0
lakehouse / simulatedhealthy
Click any stage to see what it does.
Time travel
trino> run a query to read the lake at any snapshot.
RedpandaApache FlinkApache IcebergTrinoMinIONessie
How it works
01
Ingest
Redpanda takes in 10M+ events a day with Kafka-compatible streaming.
02
Transform
Flink processes events in flight and quarantines bad ones in a dead-letter queue.
03
Store
Iceberg tables on MinIO, versioned by Nessie, commit a snapshot as data lands.
04
Query
Trino reads the lake with SQL, including any past snapshot.