Data Engineering Training

Announcement

New Batches Starts For Data Engineering Training From November

Enroll Now For Free Demo

JOIN INDIA’S NO.1 OUTCOME FOCUSED

Data Engineering

This course will nurture and transform you into a highly-skilled data engineer who builds and maintains data structures and architectures for data ingestion, processing, and deployment for large-scale, data-intensive applications. It’s a promising career for both new and experienced professionals with a passion for data, including graduates from all streams.

Download Brochure

Share This Course

Weeks Duration

Hr/Week Therory

Hr/Week Lab

Students

Big Data Introduction, Hadoop Introduction

What is Big Data
Bigdata Analytics
Bigdata Challenges
Technologies for Bigdata
What is Hadoop?
History of Hadoop
Basic Concepts
Future of Hadoop
The Hadoop Distributed File System
Future of Hadoop
Breakthroughs of Hadoop
Breakthroughs of Hadoop

Apache Hadoop
Cloudera Hadoop (CDH)
Horton Networks Hadoop (HDP)
MapR Hadoop (mapR)
AWS - EMR

Hadoop Daemon Processes, HDFS (Hadoop Distributed File System)

Hadoop Architecture
High Availability
Name Node
DataNode
Secondary Name Node
Job Tracker/Resource Manager
Task Tracker/Node Manager

Blocks and Input Splits
Data Replication
Hadoop Rack Awareness
Hadoop Cluster Architecture and Block Placement
Accessing HDFS
CLI Approach
HDFS basic file operations
Basic Administration commands

Hadoop Installation Modes, YARN

Local Mode
Pseudo-distributed Mode
Fully distributed mode

What is YARN
How YARN Works
- Resource Manager
- Node Manager
- Application Master
- Containers
Advantages of YARN
Fail Over Mechanizm

Big Data On Cloud, Course Deliverables

Introduction to Cloud - Models, Service Categories, AWS security, IAM
AWS platform
EC2
S3
Databases on AWS
AWS EMR
Workshop style coaching
Interactive approach
Course material
Hands on practice exercises for each topic
Quiz at the end of each major topic
Tips and techniques on Certification Examinations
Linux concepts and basic commands on demand

Introduction to Spark, Spark Basics

What is Spark?
Spark Overview
Setting up PySpark environment
Using Spark Shell
Resilient Distributed Datasets (RDDs)
Spark Context
Spark Ecosystem
In-Memory Computations in Spark

Working with RDDs

Creating, Loading and Saving RDD
Transformations on RDD
Actions on RDD
Key-Value Pair Transformation on RDDs
RDD Partitioning
RDD Persistence

Spark Streaming

Spark Streaming with DStreamAPI

Spark Streaming Architecture
Spark Streaming Transformations
- Stateless
- Stateful Transformations
Rolling Window and Check pointing
Integrating Spark with Kafka Streaming Data
Structured Streaming
Integrating Spark with Twitter Streaming Data
Spark Streaming Performance Considerations

Structured Streaming

Structured Streaming Overview
Advantages of Structured Streaming
Other Streaming system Vs. Structured Streaming
Stateful operations
Spark Structured Streaming
Output Modes
Spark Structured Streaming Example
Window API
Event Time
Late Events
Watermark

Structured Streaming and Kafka integrations
Integrating Spark with Twitter Streaming Data
Spark Streaming Performance Considerations

Performance Tuning and Debugging

Shared Variables: Broadcast Variables
Shared Variables: Accumulators
Common Performance Issues
Performance Tuning Tips
Spark WebUI
Monitoring Driver and Executor Logs

Testimonial's

Aman Y

I have completed the Data Engineering Training at Invictus. I would like to express my gratitude to the trainer for their knowledge and expertise which has helped me gain a strong understanding of the subject matter. Thank you, Invictus

Aditya R

This was a very good hands-on data engineering cloud learning experience. The training sessions and detailed course curriculum were both helpful. The trainer did a great job of covering each and every topic with detailed hands-on sessions

Manutha R

I thoroughly enjoyed the data engineering sessions - they were both informative and interactive. I learned about the in-depth workings of data engineering on the cloud, and got to do a lot of hands-on assignments with the help of the trainer. I found the whole experience to be very valuable and would recommend it to anyone interested in learning more about this field

Vidhi A

I found the data engineering sessions to be both informative and interactive. I learned about the in-depth workings of data engineering on the cloud and got to do several hands-on assignments with the help of the trainer. I found the whole experience to be very valuable and would recommend it to others.

Sathyajith N

The training was excellent. The explanations were clear and easy to understand. The trainer did a great job of explaining the theory and then demonstrating it with practical examples. This was really helpful in learning and understanding Data Engineering on cloud.