Spark and Scala training offers you to create Spark applications using the Scala programming. The course provides you techniques to increase application performance and enable high-speed processing using Spark RDDs.The course includes Big Data, Hadoop Core Concepts, Scala Basics, Apache Spark, Working with RDD’s, Spark SQL structure data ( Hive with spark sql) batch processing, Spark Streaming unstructured data , and real time processing.
Understanding Data & Hadoop: Basic Concepts
What is BigData.
Characteristics of BigData..
Problems with BigData.
Know the concepts of classes in scala.
The Hadoop Project and Hadoop Components.
Object orientation in scala.
Scala simple build tool – SBT.
Functional programming in scala – Closures,Currying,Anonymous functions.
Exploring mutable and immutable variables.
Execution of Scala code through REPL or CLI.
Working on basic programming constructs.
Collections – array,set.
Introduction to Apache Spark
Hadoop vs Spark.
Batch Vs. Real Time Big Data Analytics.
Spark Installation and Configuration.
Spark Execution Architecture.
Components of Spark – SQL,Streaming,Storm,GraphX.
Understanding Spark Context.
Resilient Distributed Data (RDD) – Partitions,Features ,Parallelism.
RDD operations – Transformations and Actions.
RDD - DeepDive,Persistance/Caching,Lineage.
Types of RDD -Pair RDD,chain RDD.
Spark API programming.
Executing spark program with SBT and spark-assembly.
Running spark program in local mode and in cluster.
Spark SQL overview.
Dataframes Vs RDD’s.
Processing data using Dataframes.
Custom case classes.
Temp tables Vs Persistent tables.
Inferring Schema programmatically.
Querying files as tables – CSV,Text,JSON,Parquet.
Standard transformations in querying.
Analytics and Window functions in sql.
Working of Spark SQL in Native and Hive context.he from...import Statement.
Features of Spark Streaming.
Use case 1:- Streaming data from netcat server.
Use case 2:- Flume and spark streaming integration
Use case 3:- Kafka and Spark streaming integration (kafka -messaging service).
Sliding window operations.
Transformers and Estimators.
A dynamic and self - motivated Trainer and System Administrator. Aspiring for a Bright and challenging career in the field of Training and Networking Technology, which could enable me to upgrade myself with emerging trends and technologies to benefits of the professional growth and accomplishment of organizational goals.