Unlock the Power of Spark SQL and Boost Your Data Engineering Career.
Prerequisites:
- Scala Programming Basics
Want to Become a Spark Pro but Don't Know Where to Begin? You're in the Right Place!
π€ Spark feels complex? We make it simple. Start from scratch with step-by-step guidance.
πββοΈ Worried about keeping up? Learn at your own pace with clear, easy-to-follow lessons.
πββοΈ Want to keep learning after the course? Weβll build your foundation and teach you how to find the best resources.
π° Is this worth the money? Gain essential, in-demand skills that make this course worth every penny.
Zero to Pro: Become a Professional Spark Engineer
Start from Scratch: No previous Spark knowledge? No problem! We start at the very beginning.
Develop a Deep Understanding: Gain a profound understanding of Spark's API structure, essential for mastering the framework.
Professional Coding Practices: Learn how to code professionally, ensuring your skills are workplace-ready.
Navigate Documentation: Learn where to find the best information and how to effectively read and utilize Spark documentation.
What You Will Learn
βοΈ Setting Up Your Environment: Get hands-on with setting up your development environment, including a Spark Zeppelin Docker setup.
βοΈ Working with DataFrames and Datasets: Master DataFrames and Datasets, the backbone of Spark SQL.
βοΈ Code Examples for All Transformations: Learn by doing with code examples for every transformation you'll need.
βοΈ Testing Spark SQL Code: Ensure your code works correctly by learning the best practices for testing.
βοΈ Reading and Writing Data with Spark: Understand how to read and write data using Spark, including CSV and Parquet file formats.
βοΈ Deep Dive into Encoders: Get to grips with encoders and how they affect your data processing.
βοΈ Develop High-Quality Code: Elevate your coding standards to produce clean, efficient, and high-quality Spark applications.
Upon completion of this course, you will feel confident about your Spark skills and ready to develop applications on your own.
Learn from an Expert
Philipp Brunenberg
- Bachelor's and Master's degree in computer science
- 10 years of experience as freelance big data software engineer
- Expert-level experience in the distributed data processing framework Spark
- Publishing content on his blog and YouTube channel
- Conference speaker
- Helped many students to learn Spark to a professional level
What's Included
High-Quality Video Course
Learn to write Spark SQL code in Scala from 5 hours of video lessons.
Code Repository
Access the complete code base used in the course for reference and practice.
Assignments with Solutions
Hands-on assignments to test your knowledge, complete with solutions to guide you.
Spark SQL Cheat Sheet
A handy reference to keep your skills sharp and re-access what we covered quickly.
ο»ΏZeppelin Docker Setup
Simplified setup instructions for interactive Spark development in Zeppelin using Docker.
Join a Community of Learners
Engage with fellow learners, share knowledge, and get support as you progress through the course.
Your Curriculum
- Getting to know DataFrames: Schema (i) (6:12)
- Introduction to the Dataset API (6:37)
- SparkSQL DSL (i): Working with Columns (6:52)
- SparkSQL DSL (ii): Column functions (10:44)
- SparkSQL DSL (iii): sql.functions (5:47)
- SparkSQL DSL (iv): SQL expressions (10:18)
- First assignment: Solving a business question using SparkSQL (12:19)
- Concepts (i): What is Spark? (8:35)
- Working with groupBy, sort and aggregations (11:01)
- Understanding window functions (6:31)
- Concepts (ii): Partitions, AST, Logical Plan & Optimizations (13:04)
- Joining DataFrames (10:28)
- Union on DataFrames (4:11)
- Using map and flatMap (9:20)
- MapGroups on DataFrames (10:59)
- Working with UDFs (user-defined functions) (10:42)
- Concepts (iii): The relation of SparkSQL and Spark Core (3:04)
Sign-up for the Free Spark Rockstar Newsletter π€
Receive weekly, bite-sized, high quality content on learning Spark.