risingwavelabs/awesome-stream-processing: A collection of demos showcasing how stream processing can be used to solve real-world problems.

The term “stream processing” might sound intimidating to many people. We often hear statements like:

• “Stream processing is too difficult to learn and use!” 😱
• “Stream processing is very expensive!” 😱
• “I don’t see any business use cases for stream processing!” 😱

However, we believe this isn’t true. ❌

Streaming data is everywhere, generated from operational databases, messaging queues, IoT devices, and many other sources. People can leverage modern stream processing technology to easily address classic real-world problems, using SQL as the programming language.

In this repository, we provide a series of executable demos demonstrating how stream processing can be applied in practical scenarios:

0. Getting started ✅

   • Install Kafka, PostgreSQL, and RisingWave, and run minimal toy examples on your device.
   • Integrate RisingWave with other data platforms.

1. Basic stream processing examples ✅

   Learn the fundamentals of ingesting, processing, transforming, and offloading data from streaming systems.

   1. Querying and processing event streaming data (👈 Kafka users, you may start here! 💡)
   • Directly query data stored in event streaming systems (e.g., Kafka, Redpanda).
   • Continuously ingest and analyze data from event streaming systems.
   2. Bringing analytics closer to operational databases (👈 Postgres users, you may start here! 💡)
   • Offload event-driven queries (e.g., materialized views and triggers) from operational databases (e.g., MySQL, PostgreSQL).
   3. Real-time ETL (Extract, Transform, Load)
   • Perform ETL continuously and incrementally.

2. Simple demonstrations ✅

   • A collection of simple, self-contained demos showcasing how stream processing can be applied in specific industry use cases.

3. Solution demonstrations ✅

   • A collection of comprehensive demos showcasing how to build a stream processing pipeline for real-world applications.

We use RisingWave as the default stream processing system to run these demos. We also assume that you have Kafka and/or PostgreSQL installed and possess basic knowledge of how to use these systems. These demos have been verified on Ubuntu and Mac.

All you need is a laptop 💻 – no cluster is required.

Any comments are welcome. Happy streaming!

Join our Slack community to engage in discussions with thousands of stream processing enthusiasts!

Keep your files stored safely and securely with the SanDisk 2TB Extreme Portable SSD. With over 69,505 ratings and an impressive 4.6 out of 5 stars, this product has been purchased over 8K+ times in the past month. At only $129.99, this Amazon’s Choice product is a must-have for secure file storage.

Help keep private content private with the included password protection featuring 256-bit AES hardware encryption. Order now for just $129.99 on Amazon!

Help Power Techcratic’s Future – Scan To Support

If Techcratic’s content and insights have helped you, consider giving back by supporting the platform with crypto. Every contribution makes a difference, whether it’s for high-quality content, server maintenance, or future updates. Techcratic is constantly evolving, and your support helps drive that progress.

As a solo operator who wears all the hats, creating content, managing the tech, and running the site, your support allows me to stay focused on delivering valuable resources. Your support keeps everything running smoothly and enables me to continue creating the content you love. I’m deeply grateful for your support, it truly means the world to me! Thank you!