Hadoop, (Part 3 of 4): YARN and NiFi
Interactive

Hadoop, (Part 3 of 4): YARN and NiFi

Biz Library
Updated Feb 04, 2020

Tackle Hadoop tools and services like NiFi, YARN, and Flume as well as the Spark shell, an alternative to MapReduce. Discover why Hadoop has such a large and growing following among sys admins and data scientists. Learning how Hadoop has something for just about everybody to gain and maintain competitive advantage.


Lesson 1:

  • Scalable Computing
  • Scalable Computing - YARN
  • YARN
  • Hadoop = YARN + HDFS
  • Managing Data and Processes
  • Limitations of MapReduce v1
  • YARN Processing.

Lesson 2:

  • YARN Daemons
  • Resource Manager
  • Node Manager
  • Job History Server
  • Application Master
  • Tasks in YARN
  • YARN Architecture
  • Demo: YARN Tools
  • Demo: YARN Parameters.

Lesson 3:

  • Hadoop Structure
  • Tez
  • What is Apache Spark?
  • Benefits of Spark vs. MapReduce
  • Spark Framework
  • Spark Languages.

Lesson 4:

  • Demo: The Spark Shell
  • Demo: Using Scala.

Lesson 5:

  • What Is Flume?
  • What Does Flume Do?
  • Flume Overview
  • Linux ETL
  • What is Nifi?
  • Working with Nifi
  • Nifi Capabilities
  • Demo: Linux ETL
  • Demo: More Linux Commands.

Lesson 6:

  • Demo: Install Nifi
  • Demo: install-nifi.sh
  • Demo: Run Nifi.

Lesson 7:

  • Nifi Components
  • The FlowFile
  • The GetFile Component
  • The UnpackContent Component
  • The ControlRate Component
  • The EvaluateXPath Component
  • The SplitXML Component
  • The UpdateAttribute Component
  • The AttributesToJSON Component
  • The MergeContent Component
  • The PutFile Component
  • The RouteOnAttribute Component.

Lesson 8:

  • Demo: Nifi Workflow
  • Demo: Processors
  • Demo: Source File.

Lesson 9:

  • Demo: GetFile, Unpack Content, and ControlRate
  • Demo: Evaluate XPath and Split XML
  • Demo: Update Attribute and Parse Records
  • Demo: Route on Attribute
  • Demo: MergeContent and PutFile
  • Demo: Debugging.