Apache Kafka for Beginners – Learn Kafka by Hands-On
Learn the fundamentals and advanced concepts of Apache Kafka in this course. There will be a hands on for each concept using inbuilt shell scripts that are available inside the Kafka download and using Java, Camel,Spark Spring Boot and Docker.
This course will walk you through the concepts step by step and immediate hands on for each topic discussed.
This course will help increase your technical expertise in Kafka.
This course will be an eyeopener for people think Kafka is a complex system, eventually you will understand how simple Kafka is.
This course can be a career breakthrough and assure you that you will find yourself in a better place after you complete this course.
This course will transfer your from novice Kafka user to a experienced Kafka user.
Why I should take this course ?
This course will walk you through the Kafka architecture, different components of the Kafka architecture and advantages of Kafka over other systems.
This course will focus on the internals of the Kafka and how to use Kafka and implement the concepts that are learnt in real time.
This course will teach you how to implement the concepts using Java, Camel, Spark, Spring Boot and Docker.
Section 1: Getting Started
This section explains about the author and the course objectives.
Section 2: Kafka Architecture
This section explains about the following:
Why do you need Kafka ?
What is Kafka ?
Kafka Architecture and the different components in the Kafka architecture.
Role of zookeeper, Kafka Broker, Kafka Cluster, Producers and Consumers.
Section 3: Download Kafka
This sections explored about the different approaches of downloading the Kafka distribution in your local.
Section 4: Core Internals of Apache Kafka + Hands on
This section shows you the demo on how to start zookeeper and Kafka broker.
Detailed explanation about Topics, Partitions, Consumers and Producers.
Hands on creating topic and how to connect producer and consumers to the created topic using Console Producer and Consumer.
Hands on about Kafka commit log.
Section 5 : Kafka API’s
This section explains about the different Core API’s and its use cases.
Section 6 : Behind the scenes zookeeper, Broker , Producer and Consumers
This section explains about what are the different Kafka components that interact with each to create a topic using a flow diagram.
This sections also covers how producers and consumers connect to the Kafka and what are the different components that interact with each other behind the scenes.
Section 7 : Running Multiple Brokers
Hands on how to run multiple brokers in your machine.
How does the consumer and producer while running it against the multiple brokers.
Hands on Leader, Replica and ISR attributes in a Topic.
Section 8 : Kafka Producers
Detailed explanation about Kafka Producer and how to connect to the Kafka Cluster.
How to implement the Kafka producer using Java?
Exploring different partitioning mechanism and implementation in Java.
How to alter the configuration of a topic?
Section 9 : Kafka Consumer
Detailed explanation about Kafka Consumer.
How to implement the Kafka Consumer using Java?
Different types of Offset Management in Kafka Consumer.
Automatic Offset Management using Java.
Manual Offset Management using Java.
Consumer Groups in Kafka consumer and its Advantages.
Consumer Group id and its benefits.
Implementation of consumer group in Java.
How to reset the offset to a particular value in Kafka Consumer ?
Section 10: Kafka Client – GUI Tool
In this section we will download, install and demo about the Kafka GUI client tool that will be used to connect and Manage the Kafka cluster.
Section 11: Apache Camel + Kafka Integrationa
Quick Introduction to Apache Camel
Apache Camel simple example
Apache Camel Architecture
Hands on – How to Build a Kafka consumer using Apache Camel Framework ?
Hands on – How to Build a Kafka producer using Apache Camel Framework ?
Section 12: Apache Spark + Kafka Integrationa
Quick Introduction to Apache Spark
Hands on – How to Build a Kafka consumer using Apache Spark Framework ?
Section 12: Additional Configurations
How to delete a topic ?
Section 13: Apache Kafka -Spring Boot Implementation
This section explains about the Spring boot Implementation of Apache Kafka modules
Quick Introduction to Spring
Quick Introduction to Spring boot
Simple Spring Boot App via hands on
Kafka Consumer using Spring Boot
Kafka Producer using Spring Boot
Section 14: Docker – Dockerize Kafka Broker, Zookeeper, Producer and Consumer
In this section we will run the dockerized version of kafka broker, zookeeper and we will create the docker image of the Spring boot App.
Quick Introduction to Docker.
Installation of Docker Tool Box on Windows 10.
Installation of Docker Tool Box on Mac.
Creating a docker image of the spring boot App
Creating an Account in Docker Hub and push/pull from Docker Hub.
Exploring Different Docker commands.
How to run Kafka Broker/Zookeeper in Docker ?
Complete Integration and working app of Dockerized Kafka Broker, Zookeeper and dockerized Spring boot app.
Section 15: Apache Kafka Security and Authentication
In this section we will explore about the Kafka security
Why do you need Kafka security ?
Hands on session on how to enable SSL in Kafka Broker ?
Hands on session on how to access the SSL secured broker using Console Consumer/Producer?
Hands on session on how to configure SSL in Kafka Producer using Java ?
Hands on Session on how to configure SSL in Kafka Consumer using Java ?
How to delete a topic ?
Section 16: Conclusion
Congratulations and Thank You !
By the end of this you will have a complete understanding of how Apache Kafka works and how to implement the concepts in real time using Java, Apache Camel, Apache Spark , Spring Boot and Docker.
This lecture explains about Kafka Architecture, Distributed Systems and Role of Zookeeper.
Core Internals of Apache Kafka + Kafka Concepts Hands On
This tutorial will show you a demo on How to start Zookeeper and Kafka Broker?
This tutorial explains about Topic, Partitions, Multi Partitions, Retention Period in Topic.
This hands on session explains about how to create a topic, how to publish the message using console producer, how to consume the message using console consumer and Kafka logs folder.
In this tutorial, we will discuss about Kafka core Apis and their uses.
Behind the Scenes - Zookeeper, Broker, Producers and Consumers
This tutorial explains about what happens behind the scenes when a create topic command is issued.
This tutorial explains about what happens when how the consumers and producers interact to the Kafka.
Running Multiple Kafka Brokers
This tutorial explains about how to run multiple Kafka brokers in your machine, how to create multiple partitions in a topic and how does the consumer behaves for multiple partitions in a topic.
In this tutorial, we will have hands on how to code the Kafka producer in Java. I will walk you through the code line by line.
In this tutorial, we will have hands on about different partitioning approaches in the producer end,
This tutorial is going to show you a demo of Kafka consumer using Java and walk you through the code line by line.
In this tutorial, we will discuss about the differences between manual and automatic offset management. I will show you a demo on default offset management by Kafka and override the default management by manually committing the offset in the consumer end.
This tutorial show you the demo on the benefits of consumer groups using Java. How does the zookeeper rebalances if the consumer goes down ?
In this tutorial, I will show you hands on about group id and its impacts in the Kafka Consumer.
In this tutorial, I will show you a demo on how to reset the offset to a specific value and re read the messages from it.
Kafka Client - GUI Tool
In this tutorial I will show you demo on how to download, install and use the Kafka client tool.
Apache Camel + Kafka Integration
Hands on coding - First Simple Camel application.
In this we will have a hands on coding exercise on how to implement the Kafka Consumer using Apache Camel.
Apache Spark + Kafka Integration
Apache Kafka - Additional Configurations
Apache Kafka + Spring Boot Integration
In this tutorial, we will have a quick introduction to Spring and it features.
This tutorial gives you a quick introduction about Spring Boot.
In this tutorial , we will have a hands on how to build a simple Spring boot App.
This lecture will show you hands on session on how to implement a Kafka consumer in Spring Boot.
This tutorial will show you a hands on session on how to implement the Kafka Producer using Spring Boot.
This will show you a complete work flow of how to publish and consume message from a topic using a Spring boot App.
In this tutorial we will have hands on session on how to implement "Manually managing the Offset" in Kafka consumer using Spring boot.
Docker - Dockerize Kafka, Zookeeper, Producer and Consumer
In this tutorial, We will discuss about Why Docker ? What is Docker ? What is container ?
Advantages of Docker and Container.
In this tutorial, we will have a demo on "How to install the Docker Tool Box on Windows 10 ?
In this tutorial, We will have a hands on session on how to dockerize Kafka produce and Consumer that are part of the spring boot app.
In this tutorial, we will explore different docker commands that are needed for this tutorial.
In this tutorial, we will have hands on session on How to run the Kafka Broker and Zooker in Docker.
Apache Kafka - Security and Authentication
In this tutorial, I will show you the demo on how to set up SSL in Kafka broker.
In this tutorial we will have Hands-On session about how to connect to the secured broker using console consumer and console producer.
In this tutorial, we will have hands on session on how to configure SSL in Kafka Producer using Java.
In this tutorial, we will have hands on session on how to configure SSL in Kafka Consumer using Java.