Big Data and Hadoop. Module 1: Introduction to Big Data and Hadoop. Module 2: Hadoop Distributed File System. Module 3: MapReduce

Transcription

1 Big Data and Hadoop Module 1: Introduction to Big Data and Hadoop Learn about Big Data and the shortcomings of the prevailing solutions for Big Data issues. You will also get to know, how Hadoop eradicates the Big Data problem by discovering the Hadoop ecosystem tools and Hadoop Cluster Architecture. Common Big Data Customer Scenarios Core Components of Hadoop Hadoop Cluster Modes Module 2: Hadoop Distributed File System In this module, you will learn about HDFS, which is eminently fault-tolerant and designed to be deployed on economical hardware. You will further discover about where HDFS is suitable and where not, Interruptions in NameNode, DataNode, etc. HDFS Files and Blocks Replication across multiple data node centers HDFS read/write Operations Configuration Files in Hadoop HDFS Admin Commands Module 3: MapReduce In this module, you will learn about MapReduce, which is a distributed programming paradigm. It is an allied implementation for analysing and processing huge data sets with a parallel, distributed algorithm on a cluster. You will discover: Data Flow in MapReduce

2 MapReduce Programming Model Map and Reduce Operations Job Submission Flow of MapReduce Functioning of Job Tracker and Task Tracker Module 4: Advanced MapReduce I In this module, you will learn about the Advanced Concepts of MapReduce. This module will include the theoretical and practical concepts of: GenericOptionsParser, Tool and ToolRunner Writables in Hadoop JUnit and MRUnitTesting Counters and Schedulers Data compression Techniques in Hadoop Module 5: Advanced MapReduce II In this module, you will dive deep into MapReduce concepts by learning about Input Splits and Record Reader, Tuning and Inverted Indexing. You will learn how to join huge data sets using Map side and Reduce side joins. Profiling Map and Reduce Tasks Custom Practitioner in MapReduce Data Serialization using Protocol Buffers, Thrift and Avro Joining in MapReduce: MapSide and ReduceSide Module 6: Apache Pig

3 Learn about Pig a platform for processing and analysing huge datasets using Pig Latin Language. Discover where to use Pig, Pig Data Models, Pig data types and Data Flow in Pig. Group, JOIN and COGROUP operator Pig Latin-File Loaders Pig Latin-Creating UDF Module 7: Apache Hive In this module you will learn about Hive a warehouse infrastructure, build on top of Hadoop to provide data summarization and analysis. You will dive deep into the Components of Hive, Hive Architecture, Data types and Models. Hive Metastore and Hive QL External tables, HiveQL: Data Manipulation and Queries Different kinds of UDF s in Hive. Module 8: Apache Zookeeper Learn about Zookeeper, How Zookeeper handles partial failure and Need of Coordination service. You will discover the problems with the old approach and how Zookeeper solved the issues.- Group Membership in Zookeeper Zookeeper API Zookeeper Service : Data Model, Operation, Implementation Zookeeper State transitions Module 9: Apache HBase You will learn about HBase which is a distributed column-oriented database built on the top of HDFS. You will discover how Facebook uses HBase, Data Model in HBase, HBase DDL and DML Heartbeat Sends by RegionServers

4 Working of HBaseArchitecture Compaction in HBase Data Loading Techniques used in HBase Module 10: Apache Sqoop Learn about a tool designed for efficiently transferring bulk of data. You will also discover the Sqoop Import and Export Process, about Sqoop connectors which are modular components. SqoopConnectors SqoopCommands SqoopImport and Export Processes Import Data from MySQL to HDFS, Hive and HBase SqoopConnectors Module 11: Hadoop 2.x with YARN You will learn about YARN concept in MapReduce. You will discover problems with Hadoop 1.x, Schedulers in Hadoop 2.x and how YARN Ecosystem differs from Hadoop 1.x. Hadoop 2.x Cluster Architecture Different Processing Applications in YARN Job Execution Flow on YARN Module 12: Apache Oozie Learn about Oozie, which is a workflow scheduler to manage Hadoop jobs. You will discover Databases in Oozie and various Oozie Jobs.- OozieArchitecture OozieWorkflow, OozieCoordinator and OozieBundle Jobs

5 Action and Control Nodes in Oozie OozieCoordinator Lifecycle Operations Module 13: Cloudera Impala and Project In this module, you will learn about Impala MPP SQL query engine for Apache Hadoop. You will also discover Working of ClouderaImpala Engine and Impala Daemons. This module will also cover Fume and multiple datasets and Specifications of the project too. - Impala Architecture Impala Storage Working of ClouderaImpala Engine File Formats, supported by Impala