Query Live Data Using Open Source SQL Engines

Streaming data is rapidly becoming a key component in modern applications, and Apache Kafka, Redpanda and Apache Pulsar have emerged as a popular and powerful platform for managing and processing these data streams. However, as the volume and complexity of streaming data continue to grow, it becomes increasingly critical to have efficient and effective ways of querying and analyzing this data.

This is where query engines like Apache Flink, ksqlDB, Trino, Timeplus Proton, RisingWave, Materialize, etc come in. These powerful tools offer flexible and scalable ways of processing and analyzing streaming data in real-time, enabling users to extract valuable insights from their data streams.

In this talk, we will introduce the audience to the world of querying streaming data on Apache Kafka with SQL, compare and contrast the features and capabilities of each of these tools, and provide an in-depth analysis of their respective Pros and Cons. We will also discuss the best practices and scenarios where each tool is most effective.

In conclusion, query engines like Apache Flink, Trino, ksqlDB, Proton, RisingWave are useful tools in processing and analyzing streaming data on Kafka or other message buses. With their ability to extract valuable insights from real-time data streams, these tools are a valuable asset for modern data-driven applications.