Workshops :

1. Master The Python Interview 1-Day workshop Know More | 2.One Day Workshop - Python Project (Learn how to approach programming) Know More | 3. The Extraordinary Python Coder - Workshop Know More | 4. Artificial Intelligence for Everyone Know More | 5. Data Analytics for Solving Business Problems Know More | 6. Machine Learning for Predictive Analytics Know More

Data Engineering Training

Announcement

New Batches Starts For Data Engineering Training From November

New Batches Starts For Data Engineering Training From November

Enroll Now For Free Demo

JOIN INDIA’S NO.1 OUTCOME FOCUSED

Data Engineering

This course will nurture and transform you into a highly-skilled data engineer who builds and maintains data structures and architectures for data ingestion, processing, and deployment for large-scale, data-intensive applications. It’s a promising career for both new and experienced professionals with a passion for data, including graduates from all streams.

Share This Course

5
Weeks Duration
8
Hr/Week Therory
24
Hr/Week Lab
50
Students
  • What is Big Data
  • Bigdata Analytics
  • Bigdata Challenges
  • Technologies for Bigdata
  • What is Hadoop?
  • History of Hadoop
  • Basic Concepts
  • Future of Hadoop
  • The Hadoop Distributed File System
  • Future of Hadoop
  • Breakthroughs of Hadoop
  • Breakthroughs of Hadoop
  1. Apache Hadoop
  2. Cloudera Hadoop (CDH)
  3. Horton Networks Hadoop (HDP)
  4. MapR Hadoop (mapR)
  5. AWS - EMR
  • Hadoop Architecture
  • High Availability
  • Name Node
  • DataNode
  • Secondary Name Node
  • Job Tracker/Resource Manager
  • Task Tracker/Node Manager
  • Blocks and Input Splits
  • Data Replication
  • Hadoop Rack Awareness
  • Hadoop Cluster Architecture and Block Placement
  • Accessing HDFS
  • CLI Approach
  • HDFS basic file operations
  • Basic Administration commands
  • Local Mode
  • Pseudo-distributed Mode
  • Fully distributed mode
  • What is YARN
  • How YARN Works
    • Resource Manager
    • Node Manager
    • Application Master
    • Containers
  • Advantages of YARN
  • Fail Over Mechanizm
  • Introduction to Cloud - Models, Service Categories, AWS security, IAM
  • AWS platform
  • EC2
  • S3
  • Databases on AWS
  • AWS EMR
  • Workshop style coaching
  • Interactive approach
  • Course material
  • Hands on practice exercises for each topic
  • Quiz at the end of each major topic
  • Tips and techniques on Certification Examinations
  • Linux concepts and basic commands on demand
  • What is Spark?
  • Spark Overview
  • Setting up PySpark environment
  • Using Spark Shell
  • Resilient Distributed Datasets (RDDs)
  • Spark Context
  • Spark Ecosystem
  • In-Memory Computations in Spark
  • Creating, Loading and Saving RDD
  • Transformations on RDD
  • Actions on RDD
  • Key-Value Pair Transformation on RDDs
  • RDD Partitioning
  • RDD Persistence

 Spark Streaming with DStreamAPI

  • Spark Streaming Architecture
  • Spark Streaming Transformations
    • Stateless
    • Stateful Transformations
  • Rolling Window and Check pointing
  • Integrating Spark with Kafka Streaming Data
  • Structured Streaming
  • Integrating Spark with Twitter Streaming Data
  • Spark Streaming Performance Considerations

Structured Streaming

  • Structured Streaming Overview
  • Advantages of Structured Streaming​
  • Other Streaming system Vs. Structured Streaming​
  • Stateful operations​
  • Spark Structured Streaming​
  • Output Modes​
  • Spark Structured Streaming Example​
  • Window API​
  • Event Time​
  • Late Events​
  • Watermark
  • Structured Streaming and Kafka integrations
  • Integrating Spark with Twitter Streaming Data
  • Spark Streaming Performance Considerations
  • Shared Variables: Broadcast Variables
  • Shared Variables: Accumulators
  • Common Performance Issues
  • Performance Tuning Tips
  • Spark WebUI
  • Monitoring Driver and Executor Logs

Testimonial's

Aman Y

I have completed the Data Engineering Training at Invictus. I would like to express my gratitude to the trainer for their knowledge and expertise which has helped me gain a strong understanding of the subject matter. Thank you, Invictus

Aditya R

This was a very good hands-on data engineering cloud learning experience. The training sessions and detailed course curriculum were both helpful. The trainer did a great job of covering each and every topic with detailed hands-on sessions

Manutha R

I thoroughly enjoyed the data engineering sessions - they were both informative and interactive. I learned about the in-depth workings of data engineering on the cloud, and got to do a lot of hands-on assignments with the help of the trainer. I found the whole experience to be very valuable and would recommend it to anyone interested in learning more about this field

Vidhi A

I found the data engineering sessions to be both informative and interactive. I learned about the in-depth workings of data engineering on the cloud and got to do several hands-on assignments with the help of the trainer. I found the whole experience to be very valuable and would recommend it to others.

Sathyajith N

The training was excellent. The explanations were clear and easy to understand. The trainer did a great job of explaining the theory and then demonstrating it with practical examples. This was really helpful in learning and understanding Data Engineering on cloud.