Model a small business
Design tables for a shop, load sample data and answer ten questions with SQL.
Build the pipelines and platforms that move, store and prepare data for analysis and AI.
You have reached the end of the beginner stages. Prove what you learned with a project.
You have reached the end of the intermediate stages. Prove what you learned with a project.
You have reached the end of the advanced stages. Prove what you learned with a project.
Design tables for a shop, load sample data and answer ten questions with SQL.
Pull data from a public API every day and load it into PostgreSQL without duplicates.
Load raw data into a warehouse, transform it with dbt and add tests that fail on bad data.
Run your pipeline in Airflow with retries, alerts and a backfill for past dates.
Process a multi-gigabyte open dataset with Spark and write it out as partitioned Parquet.
Stream events through Kafka and keep a live summary table up to date.
The same roadmap as a reading list: every stage, what each topic covers, and which ones are optional.
The core skill: query, join, aggregate and change data in relational databases.
Write scripts and pipeline code: files, APIs, errors and tests.
Move files, read logs, schedule jobs and work on remote servers.
Version pipeline code and review changes with your team.
Tables, keys, constraints and normal forms that keep data correct.
Read query plans and make slow queries fast.
Facts, dimensions and star schemas for analytics.
Document and key-value stores, and when they fit better than tables.
Extract, load and transform data, and keep transformations tested and documented.
Schedule pipelines, manage dependencies and retry failures.
Newer orchestrators with a focus on data assets and developer experience.
Catch nulls, duplicates and broken assumptions before anyone sees a wrong number.
Column stores that query billions of rows: BigQuery, Snowflake, Redshift and others.
Cheap, durable storage for raw files at any scale.
CSV, JSON, Parquet and Avro, and why columnar formats save time and money.
Package pipeline code so it runs the same on your laptop and in production.
Process data too large for one machine with distributed DataFrames.
Move events between systems in real time, reliably and in order.
Add transactions and versioning on top of files in a data lake.
Compute results continuously as events arrive.
Create databases, buckets and permissions from reviewed code, not clicks.
Test and deploy pipeline changes automatically.
Access control, personal data rules, retention and knowing where data came from.
Partitioning, clustering and query habits that keep cloud bills down.
They build and run the systems that collect, clean, store and serve data, so analysts, data scientists and AI products can rely on it.
Data engineers build the pipes and storage. Data scientists use the data to answer questions and build models. Both need SQL and Python.
Pick one (AWS, Azure or Google Cloud) and learn it well. The concepts transfer, and job posts in your area will tell you which is most in demand.
© 2026 KamerLink. All rights reserved.