4.56 out of 5
4.56
31 reviews on Udemy

Master Flume, Sqoop, Hive for Big Data Ingestion & Analytics

Complete course on Sqoop, Flume & Hive- Great for CCA175 & Hortonwork Spark Certification preparation.
Instructor:
Navdeep Kaur
1,996 students enrolled
English [Auto-generated]
Hadoop distributed File system and commands. Lifecycle of sqoop command. Sqoop import command to migrate data from Mysql to HDFS. Sqoop import command to migrate data from Mysql to Hive. Working with various file formats, compressions, file delimeter,where clause and queries while importing the data. Understand split-by and boundary queries. Use incremental mode to migrate the data from Mysql to HDFS. Using sqoop export, migrate data from HDFS to Mysql. Using sqoop export, migrate data from Hive to Mysql. Understand Flume Architecture. Using flume, Ingest data from Twitter and save to HDFS. Using flume, Ingest data from netcat and save to HDFS. Using flume, Ingest data from exec and show on console. Flume Interceptors.

In this course, you will start by learning what is hadoop distributed file system and most common hadoop commands required to work with Hadoop File system.

Then you will be introduced to Sqoop Import

  • Understand lifecycle of sqoop command.

  • Use sqoop import command to migrate data from Mysql to HDFS.

  • Use sqoop import command to migrate data from Mysql to Hive.

  • Use various file formats, compressions, file delimeter,where clause and queries while importing the data.

  • Understand split-by and boundary queries.

  • Use incremental mode to migrate the data from Mysql to HDFS.

Further, you will learn Sqoop Export to migrate data.

  • What is sqoop export

  • Using sqoop export, migrate data from HDFS to Mysql.

  • Using sqoop export, migrate data from Hive to Mysql.

Further, you will learn about Apache Flume

  • Understand Flume Architecture.

  • Using flume, Ingest data from Twitter and save to HDFS.

  • Using flume, Ingest data from netcat and save to HDFS.

  • Using flume, Ingest data from exec and show on console.

  • Describe flume interceptors and see examples of using interceptors.

  • Flume multiple agents

  • Flume Consolidation.

Finally, we will start with our last section about Apache Hive

  • Hive Intro

  • External &┬áManaged Tables

  • Working with Different Files – Parquet,Avro

  • Compressions

  • Hive Analysis

  • Hive String Functions

  • Hive Date Functions

  • Partitioning

  • Bucketing

Hadoop Introduction

1
HDFS and Hadoop Commands

Sqoop Import

1
Sqoop Introduction
2
Managing Target Directories
3
Working with Different File Formats
4
Working with Different Compressions
5
Conditional Imports
6
Split-by and Boundary Queries
7
Field delimeters
8
Incremental Appends
9
Sqoop Hive Import
10
Sqoop List Tables/Database
11
Sqoop Import Practice1
12
Sqoop Import Practice2
13
Sqoop Import Practice3

Sqoop Export

1
Export from Hdfs to Mysql
2
Export from Hive to Mysql

Apache Flume

1
Flume Introduction & Architecture
2
Exec Source and Logger Sink
3
Moving data from Twitter to HDFS
4
Moving data from NetCat to HDFS
5
Flume Interceptors
6
Flume Interceptor Example
7
Flume Multi-Agent Flow
8
Flume Consolidation

Apache Hive

1
Hive Introduction
2
Hive Database
3
Hive Managed Tables
4
Hive External Tables
5
Hive Inserts
6
Hive Analytics
7
Working with Parquet
8
Compressing Parquet
9
Working with Fixed File Format
10
Alter Command
11
Hive String Functions
12
Hive Date Functions
13
Hive Partitioning
14
Hive Bucketing
You can view and review the lecture materials indefinitely, like an on-demand channel.
Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don't have an internet connection, some instructors also let their students download course lectures. That's up to the instructor though, so make sure you get on their good side!
4.6
4.6 out of 5
31 Ratings

Detailed Rating

Stars 5
13
Stars 4
14
Stars 3
4
Stars 2
0
Stars 1
0
6cc626bb82878b3a9db81804990d3cd0
30-Day Money-Back Guarantee

Includes

3 hours on-demand video
Full lifetime access
Access on mobile and TV
Certificate of Completion