Section 9 : Case Study # Objectives of this Session The Motivation For Hadoop What problems exist with traditional large-scale computing systems What requirements an alternative approach should have How Hadoop addresses those requirements Hadoop: Basic Concepts What Is Hadoop? The Hadoop Distributed File System (HDFS) How Google MapReduce Algorithm works Anatomy of a Hadoop Cluster Who uses Hadoop? db.suven.net # Not a part of 1Z0-061 or 1Z0-144 Certification test, but very important technology in BIG DATA Analysis compiled by Rocky Jagtiani Tech Head for 1
Objectives of this Session contd Hadoop Solutions The most common problems Hadoop can solve The types of analytics often performed with Hadoop Where the data comes from? The benefits of analyzing data with Hadoop How some real-world companies use Hadoop Hadoop Ecosystem Cloudera Software (All Open-Source) compiled by Rocky Jagtiani Tech Head for 2
The Motivation For Hadoop compiled by Rocky Jagtiani Tech Head for 3
* MPI: Message Passing Interface PVM: Parallel Virtual Machine compiled by Rocky Jagtiani Tech Head for 4
Major Problem compiled by Rocky Jagtiani Tech Head for 5
1 GB = 1000 MB, 1 TB = 1000 GB, 1 PT = 1000 TB, 1 Exabyte = 1000 PT PT => petabyte, TB => terabyte compiled by Rocky Jagtiani Tech Head for 6
compiled by Rocky Jagtiani Tech Head for 7
The Motivation For Hadoop compiled by Rocky Jagtiani Tech Head for 8
1. 2. compiled by Rocky Jagtiani Tech Head for 9
compiled by Rocky Jagtiani Tech Head for 3. 4. 5. 10
Hadoop History compiled by Rocky Jagtiani Tech Head for 11
Core Hadoop Concepts compiled by Rocky Jagtiani Tech Head for 12
Hadoop Components compiled by Rocky Jagtiani Tech Head for 13
HDFS compiled by Rocky Jagtiani Tech Head for 14
HDFS Concepts compiled by Rocky Jagtiani Tech Head for 15
HDFS : How Files Are Stored? compiled by Rocky Jagtiani Tech Head for 16
How Files Are Stored: Example compiled by Rocky Jagtiani Tech Head for 17
IMP : How MapReduce Work? compiled by Rocky Jagtiani Tech Head for 18
MapReduce: The Mapper compiled by Rocky Jagtiani Tech Head for 19
Example : compiled by Rocky Jagtiani Tech Head for 20
compiled by Rocky Jagtiani Tech Head for 21
compiled by Rocky Jagtiani Tech Head for 22
compiled by Rocky Jagtiani Tech Head for 23
compiled by Rocky Jagtiani Tech Head for 24
Anatomy of a Hadoop Cluster : compiled by Rocky Jagtiani Tech Head for 25
compiled by Rocky Jagtiani Tech Head for 26
compiled by Rocky Jagtiani Tech Head for 27
Who uses Hadoop? compiled by Rocky Jagtiani Tech Head for 28
Hadoop Solutions compiled by Rocky Jagtiani Tech Head for 29
A compiled by Rocky Jagtiani Tech Head for 30
B What is Problem if the data is coming? compiled by Rocky Jagtiani Tech Head for 31
C compiled by Rocky Jagtiani Tech Head for 32
D The most common problems Hadoop can solve : We understand how each problem is solved using Hadoop in brief compiled by Rocky Jagtiani Tech Head for 33
compiled by Rocky Jagtiani Tech Head for 34
compiled by Rocky Jagtiani Tech Head for 35
compiled by Rocky Jagtiani Tech Head for 36
compiled by Rocky Jagtiani Tech Head for 37
compiled by Rocky Jagtiani Tech Head for 38
compiled by Rocky Jagtiani Tech Head for 39
compiled by Rocky Jagtiani Tech Head for 40
compiled by Rocky Jagtiani Tech Head for 41
E How some real-world companies use Hadoop compiled by Rocky Jagtiani Tech Head for 42
Hadoop Ecosystem compiled by Rocky Jagtiani Tech Head for 43
Cloudera Software (All Open-Source) compiled by Rocky Jagtiani Tech Head for 44
Conclusion : *enterprise data warehouse (EDW) compiled by Rocky Jagtiani Tech Head for 45
Questions 1) Input to mapper is "Google is one of the richest companies " "one who works with the Google is technical expert " what will be the out put after reducing? compiled by Rocky Jagtiani Tech Head for 46
2) Input to mapper is "Cat is eating milk" "Cat is very sweet and she likes milk" "milk is in bottle" what will be the out put after reducing? compiled by Rocky Jagtiani Tech Head for 47
3) Input to mapper is "Dollar is national currency for USA" "Rupee is national currency for India" "Dollar is ahead of Rupee in economy" "India is developing country" what will be the out put after Mapping? compiled by Rocky Jagtiani Tech Head for 48
what will be the out put after shuffling? what will be the out put after reducing? compiled by Rocky Jagtiani Tech Head for 49