Part 1: Getting Started
Part 2: Working with the SparkSQL API
-
Getting to know DataFrames: Schema (i) (6:12)
-
Introduction to the Dataset API (6:37)
-
SparkSQL DSL (i): Working with Columns (6:52)
-
SparkSQL DSL (ii): Column functions (10:44)
-
SparkSQL DSL (iii): sql.functions (5:47)
-
SparkSQL DSL (iv): SQL expressions (10:18)
-
First assignment: Solving a business question using SparkSQL (12:19)
-
Concepts (i): What is Spark? (8:35)
-
Working with groupBy, sort and aggregations (11:01)
-
Understanding window functions (6:31)
-
Concepts (ii): Partitions, AST, Logical Plan & Optimizations (13:04)
-
Joining DataFrames (10:28)
-
Union on DataFrames (4:11)
-
Using map and flatMap (9:20)
-
MapGroups on DataFrames (10:59)
-
Working with UDFs (user-defined functions) (10:42)
-
Concepts (iii): The relation of SparkSQL and Spark Core (3:04)
Part 3: Writing Tests
Part 4: Reading and Writing Files
Part 5: The Dataset API
Part 6: Software Engineering with SparkSQL
