4.19 out of 5
4.19
498 reviews on Udemy

Apache Kafka for Beginners – Learn Kafka by Hands-On

Kafka - Deep dive in to Apache Kafka concepts and learn to build Kafka producers/consumers using Java,Camel,Spring etc.,
Instructor:
Dilip S
2,646 students enrolled
Understand all the Kafka concepts and Kafka core internals.
Create Topics, Post data in to topic and Read from Topic using different technologies.
Build a complete working app with Kafka producer and consumer using Java, Apache Camel, Spring Boot and Docker.
Learn to develop Kafka Clients using Apache Spark
Learn to code Kafka Producer and Consumers using Apache Camel.
Learn to run Dockerized Kafka Broker, Zookeeper and Dockerized Spring Boot App
Implementation of Kafka security using SSL in Java.
Download , Install and Usage of Kafka Client GUI Tool.
By the end of the course you will have a complete understanding of Apache Kafka and hands on experience on how to implement these principles.

Description:

Learn the fundamentals and advanced concepts of Apache Kafka in this course. There will be a hands on for each concept using inbuilt shell scripts that are available inside the Kafka download and using Java, Camel,Spark Spring Boot and Docker.

  • This course will walk you through the concepts step by step and immediate hands on for each topic discussed.

  • This course will help increase your technical expertise in Kafka.

  • This course will be an eyeopener for people think Kafka is a complex system, eventually you will understand how simple Kafka is.

  • This course can be a career breakthrough and assure you that you will find yourself in a better place after you complete this course.

  • This course will transfer your from novice Kafka user to a experienced Kafka user.

Why I should take this course ?

  • This course will walk you through the Kafka architecture, different components of the Kafka architecture and advantages  of Kafka over other systems.

  • This course will focus on the internals of the Kafka and how to use Kafka and implement the concepts that are learnt in real time.

  • This course will teach you how to implement the concepts using Java, Camel, Spark, Spring Boot and Docker.

Course Overview:

Section 1: Getting Started

  • This section explains about the author and the course objectives.

Section 2: Kafka Architecture

  • This section explains about the following:

    • Why do you need Kafka ?

    • What is Kafka ?

    • Kafka Architecture and the different components in the Kafka architecture.

      • Role of zookeeper, Kafka Broker, Kafka Cluster, Producers and Consumers.

Section 3: Download Kafka

  • This sections explored about the different approaches of downloading the Kafka distribution in your local.

Section 4: Core Internals of Apache Kafka + Hands on

  • This section shows you the demo on  how to start zookeeper and Kafka broker.

  • Detailed explanation about Topics, Partitions, Consumers and Producers.

  • Hands on creating topic  and how to connect producer and consumers to the created topic using Console Producer and Consumer.

  • Hands  on about  Kafka commit log.

Section 5 : Kafka API’s

  • This section explains about the different Core API’s and its use cases.

Section 6 : Behind the scenes zookeeper, Broker , Producer and Consumers

  • This section explains about what are the different Kafka components that interact with each to create a topic using a flow diagram

  • This sections also covers how producers and consumers connect to the Kafka and what are the different components that interact with each other behind the scenes.

Section 7 : Running Multiple Brokers

  • Hands on how to run multiple brokers in your machine.

  • How does the consumer and producer while running it against the multiple brokers.

  • Hands on Leader, Replica and ISR attributes in a Topic.

Section 8 : Kafka Producers

  • Detailed explanation about Kafka Producer and how to connect to the Kafka Cluster.

  • How to implement the Kafka producer using Java?

  • Exploring different partitioning mechanism and implementation in Java.

  • How to alter the configuration of a topic?

Section 9 : Kafka Consumer

  • Detailed explanation about Kafka Consumer.

  • How to implement the Kafka Consumer using Java?

  • Different types of Offset Management in Kafka Consumer.

    • Automatic Offset Management using Java.

    • Manual Offset Management using Java.

  • Consumer Groups in Kafka consumer and its Advantages.

  • Consumer Group id and its benefits.

  • Implementation of consumer group in Java.

  • How to reset the offset to a particular value in Kafka Consumer ?

Section 10: Kafka Client – GUI Tool

  • In this section we will download, install and demo about the Kafka GUI client tool that will be used to connect and Manage the Kafka cluster.

Section 11: Apache Camel + Kafka Integrationa

  • Quick Introduction to Apache Camel

  • Apache Camel simple example

  • Apache Camel Architecture

  • Hands on – How to Build a Kafka consumer using Apache Camel Framework ?

  • Hands on – How to Build a Kafka producer using Apache Camel Framework ?

Section 12: Apache Spark + Kafka Integrationa

  • Quick Introduction to Apache Spark

  • Hands on – How to Build a Kafka consumer using Apache Spark Framework ?

Section 12: Additional Configurations

How to delete a topic ?

Section 13: Apache Kafka -Spring Boot Implementation

This section explains about the Spring boot Implementation of Apache Kafka modules

  • Quick Introduction to Spring

  • Quick Introduction to Spring boot

  • Simple Spring Boot App via hands on

  • Kafka Consumer using Spring Boot

  • Kafka Producer using Spring Boot

Section 14: Docker – Dockerize Kafka Broker, Zookeeper, Producer and Consumer

In this section we will run the dockerized version of kafka broker, zookeeper and we will create the docker image of the Spring boot App.

  • Quick Introduction to Docker.

  • Installation of Docker Tool Box on Windows 10.

  • Installation of Docker Tool Box on Mac.

  • Creating a docker image of the spring boot App

  • Creating an Account in Docker Hub and push/pull from Docker Hub.

  • Exploring Different Docker commands.

  • How to run Kafka Broker/Zookeeper in Docker ?

  • Complete Integration and working app of Dockerized Kafka Broker, Zookeeper and dockerized Spring boot app.


Section 15: Apache Kafka Security and Authentication

In this section we will explore about the Kafka security 

  • Why do you need Kafka security ?

  • Hands on session on how to enable SSL in Kafka Broker ?

  • Hands on session on how to access the SSL secured broker using Console Consumer/Producer?

  • Hands on session on how to configure SSL in Kafka Producer using Java ?

  • Hands on Session on how to configure SSL in Kafka Consumer using Java ?

  • How to delete a topic ?

Section 16: Conclusion

  • Congratulations and Thank You !

By the end of this you will have a complete understanding of how Apache Kafka works and how to implement the concepts in real time using Java, Apache Camel, Apache Spark , Spring Boot and Docker.

Getting Started

1
About the Course

Kafka Architecture

1
Why Kafka ?
2
What is Kafka?
3
Kafka Architecture

This lecture explains about Kafka Architecture, Distributed Systems and Role of Zookeeper.

Download Kafka

1
Download and Install Kafka from Kafka Website
2
Download and Install Kafka using curl command

Core Internals of Apache Kafka + Kafka Concepts Hands On

1
Hands on - How to start Zookeeper and Kafka Broker ?

This tutorial will show you a demo on How to start Zookeeper and Kafka Broker?

2
Kafka Topics, Partitions,Producers,Consumers and Retention Period in Topic

This tutorial explains about Topic, Partitions, Multi Partitions, Retention Period in Topic.

3
Hands on - Create Topic, Producer and Consume Messages, Kafka Log Folder

This hands on session explains about how to create a topic, how to publish the message using console producer, how to consume the message using console consumer and Kafka logs folder.

4
Kafka Commit Log

Kafka API's

1
Kafka Core API's

In this tutorial, we will discuss about Kafka core Apis and their uses.

Behind the Scenes - Zookeeper, Broker, Producers and Consumers

1
Behind the Scenes - Create topic, Zookeeper and Broker

This tutorial explains about what happens behind the scenes when a create topic command is issued.

2
Behind the scenes - Producers and Consumers

This tutorial explains about what happens when how the consumers and producers interact to the Kafka.

Running Multiple Kafka Brokers

1
Hands On - Running Multiple Kafka Brokers

This tutorial explains about how to run multiple Kafka brokers in your machine, how to create multiple partitions in a topic and how does the consumer behaves for multiple partitions in a topic.

2
Hands on - Replication Factor, Leader and ISR

Kafka Producers

1
Download and Install Eclipse
2
Hands on - Kafka Producer coding using Java

In this tutorial, we will have hands on  how to code the Kafka producer in Java. I will walk you through the code line by line.

3
Hands on - How to alter the configuration of a topic ?
4
Hands on - Producer Partitioning Mechanism using Java

In this tutorial, we will have hands on  about different partitioning approaches in the producer end,

Kafka Consumers

1
Hands on - Kafka Consumer using Java

This tutorial is going to show you a demo of Kafka consumer using Java and walk you through the code line by line.

2
Hands On - Offset Management in Kafka consumer using Java

In this tutorial, we will discuss about the differences between manual and automatic offset management. I will show you a demo on default offset management by Kafka and override the default management by manually committing the offset in the consumer end.

3
Consumer Group in Kafka Consumer - Insight and Benefits
4
Hands on - Consumer Group using Java

This tutorial show you the demo on  the benefits of consumer groups using Java. How does the zookeeper rebalances if the consumer goes down ?  

5
Hands on - Group Id and its Impacts

In this tutorial, I will show you hands on about group id and its impacts in the Kafka Consumer.

6
Hands on - Reset to a specific Offset value in Kafka Consumer using Java

In this tutorial, I will show you a demo on how to reset the offset to a specific value and re read the messages from it.

7
Kafka Consumer - Additional Configurations

Kafka Client - GUI Tool

1
Kafka - GUI Client

In this tutorial I will show you demo on how to download, install and use the Kafka client tool.

Apache Camel + Kafka Integration

1
Camel - Quick Introduction
2
Hands on - Simple Camel Example

Hands on coding - First Simple Camel application.

3
Camel Architecture
4
Hands on - Kafka Consumer using Apache Camel

In this we will have a hands on coding exercise on how to implement the Kafka Consumer using Apache Camel.

5
Hands on - Kafka Producer using Apache Camel

Apache Spark + Kafka Integration

1
Spark - Spark Streaming - Quick Introduction
2
Hands on - Kafka Consumer using Spark Streaming

Apache Kafka - Additional Configurations

1
Hands on - How to delete a topic ?

Apache Kafka + Spring Boot Integration

1
About this Section
2
Spring - Quick Introduction

In this tutorial, we will have a quick introduction to Spring and it features.

3
Spring Boot - Quick Introduction

This tutorial gives you a quick introduction about Spring Boot.

4
Simple Spring Boot App - Hands on Part 1

In this tutorial , we will have a hands on how to build a simple Spring boot App.

5
Simple Spring Boot App - Hands on Part 2
6
Spring Boot - Kafka Consumer - Hands On

This lecture will show you hands on session on how to implement a Kafka consumer in Spring Boot.

7
Spring Boot - Kafka Producer Hands On

This tutorial will show you a hands on session on how to implement the Kafka Producer using Spring Boot.

This will show you a complete work flow of how to publish and consume message from a topic using a Spring boot App.

8
Spring Boot - Kafka Consumer Manual Offset Commit - Hands on

In this tutorial we will have hands on session on how to implement "Manually managing the Offset" in Kafka consumer using Spring boot.

9
Spring Boot - Kafka Consumer Group - Hands on

Docker - Dockerize Kafka, Zookeeper, Producer and Consumer

1
Docker - Quick Introduction

In this tutorial, We will discuss about Why Docker ? What is Docker ? What is container ?

Advantages of Docker and Container.

2
How to Install Docker Tool Box on Mac ?
3
How to Install Docker Tool Box on Windows 10?

In this tutorial, we will have a demo on "How to install the Docker Tool Box on Windows 10 ?

4
Hands on - How to create a docker image for Kafka Producer and Consumer?

In this tutorial, We will have a hands on session on how to dockerize Kafka produce and Consumer that are part of the spring boot app.

5
How to create an account in Docker Hub and push the images to Docker Hub ?
6
Hands on - Running basic docker commands

In this tutorial, we will explore different docker commands that are needed for this tutorial.

7
Hands on - How to run a zookeeper/Kafka in Docker ?

In this tutorial, we will have hands on session on How to run the Kafka Broker and Zooker in Docker.

8
Demo - Connect to Dockerized kafka/zookeeper using Dockerized Spring boot app

Apache Kafka - Security and Authentication

1
Why Kafka security ?
2
Hands on - How to enable SSL in Kafka Broker?

In this tutorial, I will show you the demo on how to set up SSL in Kafka broker.

3
Hands on - How to access the SSL secured topic using Console Producer/Consumer ?

In this tutorial we will have Hands-On session about how to connect to the secured broker using console consumer and console producer.

4
Hands on - How to configure SSL in Kafka Producer using Java?

In this tutorial, we will have hands on session on how to configure SSL in Kafka Producer using Java.

5
Hands on - How to configure SSL in Kafka Consumer using Java?

In this tutorial, we will have hands on session on how to configure SSL in Kafka Consumer using Java.

Bonus Section

1
Coupon for my Other Courses

Conclusion

1
Congratulations !
You can view and review the lecture materials indefinitely, like an on-demand channel.
Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don't have an internet connection, some instructors also let their students download course lectures. That's up to the instructor though, so make sure you get on their good side!
4.2
4.2 out of 5
498 Ratings

Detailed Rating

Stars 5
212
Stars 4
176
Stars 3
86
Stars 2
16
Stars 1
8
010eae5f7100f642d931d153757796bc
30-Day Money-Back Guarantee

Includes

6 hours on-demand video
1 article
Full lifetime access
Access on mobile and TV
Certificate of Completion