HADOOP. Session 1: Introduction to Hadoop & Big Data. Session 2: Hadoop Distributed File Systems. Session 3: Administering Hadoop Cluster

Transcription

1 HADOOP Sessin 1: Intrductin t Hadp & Big Data What is Hadp? Big data and Big Data Analytical services Histry f Hadp Hadp Ec-Systems Hadp Framewrk Hadp Distributin Hadp Cre Cmpnents Hadp Vs RDBMS Sessin 2: Hadp Distributed File Systems HDFS architecture Hadp file systems HDFS Daemns Master and Slave nde architecture HDFS webui Hw t lad the data int HDFS HDFS using Java API Serializatin in Hadp Reading and writing files frm/t Hadp URL Sessin 3: Administering Hadp Cluster Intrductin t Quality Center. Cluster specificatin, Hadp cluster setup and installatin standalne pseud distributed mde fully distributed mde Step-by-step multinde installatin Hadp Cnfiguratins namende and datande directry structure User cmmands: fs, fsck,distcp,archive, Administratin cmmands dfsadmin 1 P a g e

2 balancer jbtracer tasktracker namende Sessin 4: Imprt and Exprt data using SQOOP Sqp installatins Sqp1 vs Sqp2 Sqp cmmands Sqp cnnectrs fr racle, Mysql, Teradata and ther Rdbms Imprting the data frm Mysql with all pssible imprts Exprting the data frm HDFS t Rdbms Creating hive tables by imprting data Sessin 5: HIVE Hive cncepts Hive installatin Hive cnfiguratin, Hive services Hive query prcessing Hive Metastre cnfiguratins Hive datatypes - primitive and cmplex types Hive peratrs Hive Built-in functins Static and dynamic partitins Bucketing in Hive Hive Tables Creating tables Partitins and buckets Brwsing tables and partitins Strage frmats Lading data Jins Aggregatins and srting Insert int lcal files Altering, drpping tables Imprting data Wrd cunt in Hive Writing UDFs Sub Queries Views Indexes Hive Cmpressin Hive Security Hive Patterns and AntiPatterns Sessin 6: PIG Why PIG? What is PIG PIG Vs Map Reduce 2 P a g e

3 PIG executin mdes ( Lcal and Distributed) PIG installatin PIG Latin Relatinal peratrs Diagnstic peratrs Schemas Datatypes Operatrs Functins Filtering Gruping and Jining Srting Basic cmmands (Lad, Stre, Dump) Data types and Expressins Built-in functins UDFs in PIG Nested freach Sessin 7: MapReduce MapReduce Overview and Architecture Develping MapRed Jbs Cmbiner Partitiner Map Reduce Chain MR Unit Cunters Distributed Cache Wrd cunt prgram in MapReduce Input and Output Frmats Jb Cnfiguratin Packaging, Launching, Debugging jbs Jb Submissin MapReduce wrkflws MapReduce types Input and Output Frmats Running MapReduce jbs in Eclipse Practicing MapReduce Prgrams Sessin 8: YARN architecture and the New MapReduce (MRV2) Yarn: Yet Anther Resurce Negtiatr Hadp YARN - a resurce-management platfrm respnsible fr managing cmpute resurces in clusters and using them fr scheduling f users' applicatins. Yarn Architecture Yarn Cnfiguratins Yarn and MapReduce 2.0 Daemns Yarn New MapReduce API Yarn cmmand line tls 3 P a g e

4 Sessin 9: NSQL Prgramming with MngDB Intrductin t NSql Prgramming MngDB Overview Installing mngdb n Windws, Linux, Mac OS cmmunicatin between Hadp and mngdb MngDB management services MngDB Prgramming Creating Dcuments, adding Data, Retrieving data frm Dcuments CRUD in mngdb Datatypes, Arrays, Regular Expressins Sessin 10: Selenium WebDriver Intrductin t Selenium RC. Intrductin t Selenium WebDriver. Difference between Selenium RC and Selenium WebDriver. Dwnlading and Installatin f Selenium WebDriver with Eclipse. First WebDriver script in Eclipse. WebDriver script with JUnit. Using JUnit anntatins in WebDriver. JUnit Timeut and Expected Exceptin test. Selenium WebDriver example in Chrme brwser. Selenium WebDriver Element Lcatrs. Selenium WebDriver Accessing Frm Elements. Sessin 11: Advanced Selenium WebDriver Accessing Links and Tables. Selenium WebDriver Keybard and Muse events. Flight reservatin example using Selenium WebDriver. Intrductin t TestNG. Advantages f TestNG ver JUnit. Installing TestNG in Eclipse. TestNG- Selenium WebDriver example using anntatins. Selenium WebDriver reprts using TestNG. Selenium WebDriver real wrld example. Fitnesse + Selenium WebDriver Integratin example. Sessin 12: Reprting Tls Reprts using Jaspersft Pentah BI Tls Sessin 13: Hadp Cluster Mnitring Tls Cludera Manager Hadp cluster mnitring using Ganglia Sessin 14: HBase 4 P a g e

5 What is HBase HBase architecture HBase cmpnents Regin and Regin Server HBase strage HBase API Sample prgrams Sessin 15: Misc Hadp Subprjects OOzie Impala Registratin & Inquiries: P a g e