Real-time Stream Processing
As more and more data is being streamed, there is a growing need for the organizations to process the data instantly or near real time; this course will cover stream processing concepts and technologies that will allow participants to query, analyze, and process data as it flows.
The course aims to equip participants with knowledge and hands-on skills needed for a streaming data architecture and will also cover popular use cases driven by streaming data with the aim of helping participants learn to identify opportunities at work that could leverage stream processing.
Courage Noko
Courage Noko has over 12 years experience in Data engineering. He started at the New York Times where he was a Hadoop administrator/developer maintaining Hadoop clusters as well as writing MapReduce pipelines in Java and Pig. I also designed and implemented the first real time recommendation engine for nytimes.com.
He has led a team at Spotify to build a scalable real-time streaming platform that allows engineers to write real time recommendation engines, receive over 100s of millions of live events per seconds from streaming devices and run real time performance diagnostics. Over time, he has worked on several streaming frameworks: Apache Beam/Dataflow, Spark, Flink. And Storm.
Lately, he has taken interest in real-time analytics, leading teams to build and deploy platforms that support OLAP databases such as Druid, ClickHouse and Pinot.
What you'll learn
Module 1: Introduction to basic streaming concepts
Batch vs Streaming data - comparing the main differences between bounded and unbounded data processing.
Real world examples - a look at some real world examples of real-time stream processing.
Overview of streaming architecture - detailed look at the various components that form the core parts of a streaming infrastructure
Examples of streaming engines.
Module 2: Real time data processing
Event processing - a flow of events in the streaming pipeline.
Stream window operations -a detailed look at how different execution engines handle aggregations.
Module 3: Advanced concepts
Stream Joins - joining multiple sources to streaming events.
Unit tests for streaming pipelines.
Real world streaming project covering concepts in modules 1-3.
Module 4: Operations
Monitoring streaming jobs.
Best practices.
Review of Real world streaming project from module 3.
Prerequisites:
This course is for anyone with knowledge of basic data processing which includes the following data transformation concepts:
- Filtering
- Map
- GroupBy
- Sum
- Count
This course is for:
Data Scientists, Data Engineers and ML Engineers
What you will be able to do after this course:
- Understand the differences between streaming and batch processing.
- Learn about different streaming platforms and their features.
- Learn different aggregation types and operations.
- Learn how to join streams with several sources.
- Write streaming pipelines for each concept covered to help participants understand the concepts through hands-on exercises.
- Write tests for streaming pipelines.
- Work on a real world streaming project.
Frequently Asked Questions
Are all sessions live?
Yes, all sessions during the cohort will be live with the instructor. However, we will record each session and make them available for everyone in the cohort.
What is the time commitment?
Our courses typically have 2-4 modules, with each module lasting for approximately 2h per week which you can block out on your calendar during your work day. You also get some take home projects that you can complete at your own pace.
Do I earn a certificate for this course?
Of course! Once you’ve completed the course modules, you will get a certificate of completion that you can showcase to the world.
Can I expense this course?
Yes. Most of our cohort participants have expensed this course through their Learning & Development budget, similar to how you expense conferences. You can use this email template to request expense approval from your manager.
Do you offer discounts for students?
Yes. Email us at hello@learncrunch.com and we'll be happy to give you a discount.
I have more questions. Get in touch with us!
If you have more questions, email us at hello@learncrunch.com.
Book a call with us