4.45 out of 5
4.45
28 reviews on Udemy

Big Data Internship Program – Data Processing – Hive and Pig

Provide higher-level language to facilitate large-data processing.
Instructor:
Big Data Trunk
701 students enrolled
English [Auto-generated]
Have excellent understanding of Apache Hive and Pig tool with hands-on experience .
Understand the working of a project in real-world scenario.
Work experience in end-to-end Project ( Data Masking) and can mention in Resume .

This course is part of “Big data Internship Program”  which is aligned to a typical Big data project life cycle stage.

  • Foundation
  • Ingestion
  • Storage
  • Processing
  • Visualization

This course is focused on Data Processing in Big data.This course is suitable for developers, data analysts and business analysts. Experience with SQL and scripting languages is recommended, but is not required. 

You will learn 

  • Understanding of Hive core concept and architecture.
  • How to create and manipulate tables using Hive.
  • Advanced features of Hive.
  • Hive Best Practices
  • Performing real-time, complex queries on datasets
  • Pig’s Architecture
  • Reading and Writing Data with Pig
  • Pig Best Practices

Project work

  1. Provide Data in Hive and manipulate the data for Our Book Recommendation project
  2. One Ad-on project — Data Masking with hive and sqoop

Data Processing Introduction in Big Data

1
Introduction to the course

In This video, we have explained the course structure of course, How our course is useful for Big data experts and beginners.

2
Introduction to Data processing

In This video, we have explained what is data processing, how data processing is done in big data environment, what is big data cycle. Why big data processing is important in different Areas.

HIve

1
Hadoop in Retail Industry

In this video, we have explained how Hadoop is applicable in the retail market, how hadoop can play important role in customer analysis.

2
Hive Introduction

In this video, we have given a small introduction of the hive, why
Facebook uses hive, where the hive was developed, what are hive
features.

3
Hive Introduction
4
Hive Architecture

In this video, we have explained what is hive Architecture, how the hive is integrated with other hive components, how hive executes hive query.

5
Hive Architecture
6
DataTypes in Hive

In this video we have explained, what are the data type available in
HiveQL, what are primitive data type available, Collection data types
etc

7
Managed table and External table in Hive

In this video, we have explained what is an internal table, what is an
external table, How they are used, what is the feature of hive internal
table and external table.

8
Manage and external tables in hive
9
Demonstrating difference between Internal and external table

In this video, we have explained what is an external table and internal
table, what is the benefit of an external table over an internal table.

10
Demonastrating diffrent between external and interrnal
11
Hands on Lab Hive External Vs InternalTable
12
Partitions in Hive with demo

In this video we have explained partitioning in the hive, what is the meaning of partitioning table.

13
Hands on Lab Hive Partition
14
Partition in hive
15
HIve Dynamic Partitioning

In this video we have explained Dynamic partition in hive, what is the usage of hive dynamic table, how to create dynamic table etc.

16
Hands on Lab HIve Dynamic Partitioning
17
Hive dynamic Partition

Pig

1
What is Apache PIG?
In this video, we have given a brief introduction of Pig, why it is used, and what are the characteristic of the pig.
2
PIG Architecture

In this video, we have shown Pig architecture, how Pig statement is executed in Pig grunt mode.

3
Pig Data Types

In this video, we have shown what are the different data types available in Pig, Type of data types, what are relations, bag, tuple etc.

4
Pig Data Types
5
Pig Latin
In this video, we have shown what is Pig Latin, what are various basic commands which are used in Pig Script, Pig latin Map Reduce, Use of Python with Pig
6
Pig Latin
7
PIG Running Modes
In this video, we have described different running mode, and their usage, How pig script is executed in different modes
8
PIG Running Modes
9
PIG Operators

In this video we have shown the different type of operators available in Pig Latin, like binary, ternary, flatter, how to load data using PigStorage(), Dump operator, store operator, limit and distinct, order by, grouping etc.

10
PIG Operators
11
PIG Wordcount example

In this video, we have shown how we can execute word count task in pig latin.

12
Pig Wordcount lab

Data Processing in Recommendation Project

1
BookRecommendationProject

In this video we have explain how to execute our Book Recommendation Project by using hive, sqoop, mysql. How to upload data in system for processing.

2
BookRecommendationProject-2

In this video we have explained some attribute of table, how we can access them and how we can optimize query execution in hive, we have done some hands-on recommendation database, for analysis of tables and seen the results.

Ad-on Project Data Masking

1
Data Masking Project Overview

In this video, we have explained the data masking project, which components we are going to use, what is the use of data masking etc, what are project requirement, what is a flow of the project.

2
Data Masking Project Solution Design

In this video, we have explained data masking solution, how this project
is executed, what is the goal of the project, what Softwares/tools we
are going to use in execution.

3
Data Masking Project Solution Walkthrough

In this video we have explain the step-by-stpe flow of Data Masking project, and different stages of project.

4
DataMaskingProject-step1(Create tables inMySql)
In this video we have explained how to create table in mysql, how to load data in mysql from file.
5
Hands on Lab DataMaskingProject-step1-Document
6
Step2:Creating and importing data in hive external tables

In this video, we have explained how to create an external table, and
how to load data in the external table, how to import data from external
table to hive table.

7
Hands on Lab DataMaskingProject-step2- Document
8
Step3:Creating UDFs in Java
In this video we have explained, how to crreate UDFs, and how to jar for Data Masking Project
9
Hands on Lab DataMaskingProject-step3(udf)- Document
10
Step4:Exporting data to MySql in masked database
In this video, we have shown you actual execution of data masking, by using MySQL, sqoop
11
Hands on Lab DataMaskingProjectStep4 - Document
You can view and review the lecture materials indefinitely, like an on-demand channel.
Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don't have an internet connection, some instructors also let their students download course lectures. That's up to the instructor though, so make sure you get on their good side!
4.5
4.5 out of 5
28 Ratings

Detailed Rating

Stars 5
11
Stars 4
10
Stars 3
5
Stars 2
2
Stars 1
0
c51fbbcc689d939cacb084ffea761008
30-Day Money-Back Guarantee

Includes

2 hours on-demand video
8 articles
Full lifetime access
Access on mobile and TV
Certificate of Completion