Big Data Analytics Using Sqoop and Hive
Data is the new oil of this Digital Age.
Are you thinking to start your career in Big Data Analytics ? Then you have landed in the right place.
In this course, you will learn how to organize, analyze and interpret vast sources of information.
We cover all essential fundamental understanding of Big Data Analytics and provide end-to-end real life project practice.
Course contains
- Understanding of Big Data and MapReduce
- How to transfer Data from different sources using Sqoop
- Manipulate Data in HDFS using HIVE
- Real Hands experience project
You will get only in this course :
1) 24*7 support for any issue or doubt raised by student.
2) Exercises in well written document with screenshot.
Introduction
In this video we are providing all features of the course
Big Data Trunk is the leading Big Data focus consulting and training firm founded by industry veterans in data domain. It helps is customer gain competitive advantage from open source, big data, cloud and advanced analytics. It provides services like Strategy Consulting, Advisory Consulting and high quality classroom individual and corporate training.
Newly added section- Installation and Configuration of Cloudera
How to install cloudera in 64 bit Window Machine.
Step 1: Download the virtual box- executable file from https://www.virtualbox.org/wiki/Downloads
Download Virtual Box 5.0 for Windows hosts The page look like:
Step2: Click on x86/amd64 link, then it starts downloading virtualBox-5.0.0-101573-Win.exe
Step 3: Click on arrow to open folder contains the file
Step4: Then folder gets open, then double click on that folder to run
Step5 : Installation process starts. Click on Next button.
Step 6: It show default location ; if you want to change location, click on Browse button. Otherwise click on Next button
Step 7: It shows list of options, click on Next button.
Step8 : Then it asks for installation, Click on Next button.
Step9: It asks for review or change in Installation setting If you want any changes in previous process, click on Back button or else, click on Install button
Step10: It starts processing.
Step11: Click on Finish button
Step12: Virtual Box Manager Window appear.
Step13: Click on below link to download Cloudera Quick Start VM
http://www.cloudera.com/downloads/quickstart_vms/5-7.html
The page look like:
Step14: Fill below screenshot page and Download the required version from menu (Virtual Box in this case)
Step15: Extract files from cloudera-quickstart-vm-5.7.0-virtualbox.
Step16: Double click on cloudera-quickstart-vm-5.7.0-virtualbox.
Step17: Double Click on setup as shown on below
Step18: Click on Import button
Step19: Shows Progress of importing.
Step20: After completion of process of importing, the following window opens; then click on highlighted area to start cloudera on Oracle Virtualbox.
Step21: Running Cloudera on Virtualbox.
Step 22: Finally completed installation.
Understand Big Data
In this video, we have explained what is Hadoop, how we can use MapReduce in our day-to-day life.
In this video, we have explained Big data ecosystem, and Big data terms, what are different tools of Hadoop Eco-system.
Understating of MapReduce
In this video we have explained, what is mapreduce, what is use of mapreduce, how mapreduce is executed.
In this video, we have explained what are the daemons run while executing map reduce job, how daemons communicate with each other to do the job of map reduce.
Ingestion
In this video, we have explained to you what is sqoop, what is flume, sqoop work flow, sqoop architecture
In this video, we have explained what is import command, how sqoop import command is executed.
In this video we have explained how to execute commands in terminal,how
to get table list, how to get list of data bases, how to import data in
hdfs.
In this video we have explained what is sqoop export, and how it is used.
In this video we have explained what is incremental sqoop, and how it works.what are the incremental import parameters etc.
In this video, we have explained what is sqoop jobs how it used and when
it is used. how to create jobs, how to list sqoop jobs available.
In this video, we have shown you, what is an incremental import, how it is executed, what is a benefit of incremental import.
Hive
In this video, we have given a small introduction of the hive, why Facebook uses hive, where the hive was developed, what are hive features.
In this video, we have explained what is hive Architecture, how the hive is integrated with other hive components, how hive executes hive query.
In this video, we have explained what is an internal table, what is an external table, How they are used, what is the feature of hive internal table and external table.
In this video, we have explained what is an external table and internal table, what is the benefit of an external table over an internal table.
In this video we have explained Dynamic partition in hive, what is the usage of hive dynamic table, how to create dynamic table etc.
Project
In this video, we have explained the data masking project, which components we are going to use, what is the use of data masking etc, what are project requirement, what is a flow of the project.
In this video we have explain the step-by-stpe flow of Data Masking project, and different stages of project.
In this video we have explained how to create table in mysql, how to load data in mysql from file.
In this video, we have explained how to create an external table, and how to load data in the external table, how to import data from external table to hive table.
In this video we have explained, how to crreate UDFs, and how to jar for Data Masking Project.
In this video, we have shown you actual execution of data masking, by using MySQL, sqoop.
