The logo of the product. In the background the Spark learning roadmap. In front the title of the video course: Mastering Spark Internals

Video Course

Mastering Spark Internals

Understand how Spark distributes workload and optimizes performance.



Prerequisites:

  • Spark Programming Fundamentals

The best video course on Spark internals you'll find.


This video course is the result of years of research, studying source code and working with Spark in professional setups. I have put everything I learned into this course to help you develop a profound understanding of Spark.

Outperform 90% of Spark developers out there.

An example physical plan as Spark uses it in its core execution model.

Develop a profound understanding of how Spark works.

👩‍💻 Write better Spark code and reason about design decisions.

🧑‍🏫 Become an expert yourself.

📈 Lay an important foundation for optimizing performance.

💰 Be confident in theoretical interview questions.

Become a Pro-Level Spark developer.

Developing a deep understanding of internals will help you ...

... develop your skills in Spark even further,

... to get your job in Data Engineering,

... and to radiate a sense of confidence and expertise when talking to colleagues.

An example Spark program written in Scala, linking to a parallel execution on multiple workers in a cluster.

What's in the box?

Shows an example DAG (directed acyclic graph) as Spark's core execution model will create it for parallel execution.
Deep-dive into Spark Core's execution model.

We will develop a profound understanding of how Spark executes workload in a distributed manner.

On the left side: An example physical plan of a simple Spark application. On the right side: Visualization of the transformations applied to a small example data set.
Learn how Spark automatically optimizes applications.

We will explore in detail, how SparkSQL's powerful query optimization engine works.

On the left side: A visualization of Spark memory regions on an executor. On the right side: The calculation of the region sizes in gigabytes.
Understand resource allocation and execution on clusters.

We will cover in-depth what is happening, when Spark executes distributed applications on a cluster.

Find a detailed outline below.

Your journey to mastering Spark internals ...

  Introduction
Available in days
days after you enroll
  Understanding Spark's Execution Model
Available in days
days after you enroll
  Understanding SparkSQL
Available in days
days after you enroll
  Conclusion
Available in days
days after you enroll

Enroll Today!

A picture showing a portrait of Philipp Brunenberg in front of a glass window.
Your Teacher

Philipp Brunenberg


  • Bachelor's and Master's degree in computer science
  • Almost a decade of experience as freelance big data software engineer
  • Expert-level experience in the distributed data processing framework Apache Spark
  • Publishing content on his blog and YouTube channel
  • Conference speaker
  • Helped many of his students becoming professional Spark developers

Sign-up for the Free Spark Rockstar Newsletter 🤘

Receive weekly, bite-sized, high quality content on learning Spark.