DataBurst Data Engineering Wiki - Roadmap Step-by-Step: https://wiki.databurst.tech/docs/roadmap/ - Introduction: https://wiki.databurst.tech/docs/roadmap/introduction/ - Programming Language: https://wiki.databurst.tech/docs/roadmap/programming-language/ - Python: https://wiki.databurst.tech/docs/roadmap/programming-language/python/ - Java: https://wiki.databurst.tech/docs/roadmap/programming-language/java/ - Scala: https://wiki.databurst.tech/docs/roadmap/programming-language/scala/ - Go: https://wiki.databurst.tech/docs/roadmap/programming-language/go/ - Rust: https://wiki.databurst.tech/docs/roadmap/programming-language/rust/ - Operating System: https://wiki.databurst.tech/docs/roadmap/operating-system/ - Windows: https://wiki.databurst.tech/docs/roadmap/operating-system/windows/ - Linux: https://wiki.databurst.tech/docs/roadmap/operating-system/linux/ - Networking: https://wiki.databurst.tech/docs/roadmap/networking/ - Protocols: https://wiki.databurst.tech/docs/roadmap/networking/protocols/ - TCP/IP: https://wiki.databurst.tech/docs/roadmap/networking/protocols/tcp/ - DNS: https://wiki.databurst.tech/docs/roadmap/networking/protocols/dns/ - HTTP: https://wiki.databurst.tech/docs/roadmap/networking/protocols/http/ - HTTPS: https://wiki.databurst.tech/docs/roadmap/networking/protocols/https/ - SSH: https://wiki.databurst.tech/docs/roadmap/networking/protocols/ssh/ - UDP: https://wiki.databurst.tech/docs/roadmap/networking/protocols/udp/ - Concepts: https://wiki.databurst.tech/docs/roadmap/networking/concepts/ - IP addressing: https://wiki.databurst.tech/docs/roadmap/networking/concepts/ip-addressing/ - DHCP: https://wiki.databurst.tech/docs/roadmap/networking/concepts/dhcp/ - Firewalls: https://wiki.databurst.tech/docs/roadmap/networking/concepts/firewalls/ - VPN: https://wiki.databurst.tech/docs/roadmap/networking/concepts/vpn/ - General Software Engineering Skills: https://wiki.databurst.tech/docs/roadmap/general-software-engineering-skills/ - OAuth: https://wiki.databurst.tech/docs/roadmap/general-software-engineering-skills/oauth/ - JWT: https://wiki.databurst.tech/docs/roadmap/general-software-engineering-skills/jwt/ - GraphQL: https://wiki.databurst.tech/docs/roadmap/general-software-engineering-skills/graphql/ - REST: https://wiki.databurst.tech/docs/roadmap/general-software-engineering-skills/rest/ - gRPC: https://wiki.databurst.tech/docs/roadmap/general-software-engineering-skills/grpc/ - Caching: https://wiki.databurst.tech/docs/roadmap/general-software-engineering-skills/caching/ - Web Frameworks and API Development: https://wiki.databurst.tech/docs/roadmap/web-frameworks-and-api-development/ - Python: https://wiki.databurst.tech/docs/roadmap/web-frameworks-and-api-development/python/ - Go: https://wiki.databurst.tech/docs/roadmap/web-frameworks-and-api-development/go/ - Java: https://wiki.databurst.tech/docs/roadmap/web-frameworks-and-api-development/java/ - Version Control System: https://wiki.databurst.tech/docs/roadmap/version-control-system/ - git: https://wiki.databurst.tech/docs/roadmap/version-control-system/git/ - Version Control System Hosting: https://wiki.databurst.tech/docs/roadmap/version-control-system-hosting/ - Github: https://wiki.databurst.tech/docs/roadmap/version-control-system-hosting/github/ - Gitlab: https://wiki.databurst.tech/docs/roadmap/version-control-system-hosting/gitlab/ - BitBucket: https://wiki.databurst.tech/docs/roadmap/version-control-system-hosting/bitbucket/ - Distributed Systems Concepts: https://wiki.databurst.tech/docs/roadmap/distributed-systems-concepts/ - CAP: https://wiki.databurst.tech/docs/roadmap/distributed-systems-concepts/cap/ - PACELC: https://wiki.databurst.tech/docs/roadmap/distributed-systems-concepts/pacelc/ - Sharding: https://wiki.databurst.tech/docs/roadmap/distributed-systems-concepts/sharding/ - Clock: https://wiki.databurst.tech/docs/roadmap/distributed-systems-concepts/clock/ - ACID: https://wiki.databurst.tech/docs/roadmap/distributed-systems-concepts/acid/ - Replication: https://wiki.databurst.tech/docs/roadmap/distributed-systems-concepts/replication/ - Distributed Transactions: https://wiki.databurst.tech/docs/roadmap/distributed-systems-concepts/transaction/ - Indexing: https://wiki.databurst.tech/docs/roadmap/distributed-systems-concepts/indexing/ - Batch and Stream Processing: https://wiki.databurst.tech/docs/roadmap/distributed-systems-concepts/batch-and-stream-processing/ - Consistency vs Eventual Consistency: https://wiki.databurst.tech/docs/roadmap/distributed-systems-concepts/consistency-vs-eventual-consistency/ - Normalization or Denormalization: https://wiki.databurst.tech/docs/roadmap/distributed-systems-concepts/normalization-or-denormalization/ - SQL Fundamentals: https://wiki.databurst.tech/docs/roadmap/sql-fundamentals/ - Basic SQL: https://wiki.databurst.tech/docs/roadmap/sql-fundamentals/basic-sql/ - Advanced SQL: https://wiki.databurst.tech/docs/roadmap/sql-fundamentals/advanced-sql/ - Databases: https://wiki.databurst.tech/docs/roadmap/databases/ - Relational: https://wiki.databurst.tech/docs/roadmap/databases/relational/ - NoSQL: https://wiki.databurst.tech/docs/roadmap/databases/nosql/ - Columnar: https://wiki.databurst.tech/docs/roadmap/databases/nosql/columnar/ - Document Based: https://wiki.databurst.tech/docs/roadmap/databases/nosql/document-based/ - Key-Value: https://wiki.databurst.tech/docs/roadmap/databases/nosql/key-value/ - Time Series: https://wiki.databurst.tech/docs/roadmap/databases/nosql/time-series/ - Vector: https://wiki.databurst.tech/docs/roadmap/databases/nosql/vector/ - Graph: https://wiki.databurst.tech/docs/roadmap/databases/nosql/graph/ - NewSQL: https://wiki.databurst.tech/docs/roadmap/databases/nosql/newsql/ - Data Modeling: https://wiki.databurst.tech/docs/roadmap/data-modeling/ - Star Schema (Ralph Kimball): https://wiki.databurst.tech/docs/roadmap/data-modeling/star-schema/ - Snowflake: https://wiki.databurst.tech/docs/roadmap/data-modeling/snowflake/ - Data Vault: https://wiki.databurst.tech/docs/roadmap/data-modeling/data-vault/ - Data Management Architecture: https://wiki.databurst.tech/docs/roadmap/data-management-architecture/ - Relational Data Warehouse: https://wiki.databurst.tech/docs/roadmap/data-management-architecture/relational-data-warehouse/ - Data Lake: https://wiki.databurst.tech/docs/roadmap/data-management-architecture/data-lake/ - Modern Data Warehouse: https://wiki.databurst.tech/docs/roadmap/data-management-architecture/modern-data-warehouse/ - Data Fabric: https://wiki.databurst.tech/docs/roadmap/data-management-architecture/data-fabric/ - Data Mart: https://wiki.databurst.tech/docs/roadmap/data-management-architecture/data-mart/ - Data Lakehouse: https://wiki.databurst.tech/docs/roadmap/data-management-architecture/data-lakehouse/ - Data Mesh: https://wiki.databurst.tech/docs/roadmap/data-management-architecture/data-mesh/ - Data Integration: https://wiki.databurst.tech/docs/roadmap/data-integration/ - ETL: https://wiki.databurst.tech/docs/roadmap/data-integration/etl/ - ELT: https://wiki.databurst.tech/docs/roadmap/data-integration/elt/ - CDC: https://wiki.databurst.tech/docs/roadmap/data-integration/cdc/ - Reverse ETL: https://wiki.databurst.tech/docs/roadmap/data-integration/reverse-etl/ - Storage: https://wiki.databurst.tech/docs/roadmap/storage/ - Block Storage: https://wiki.databurst.tech/docs/roadmap/storage/block-storage/ - Object Storage: https://wiki.databurst.tech/docs/roadmap/storage/object-storage/ - File Formats and Serialization: https://wiki.databurst.tech/docs/roadmap/file-formats-and-serialization/ - File Formats: https://wiki.databurst.tech/docs/roadmap/file-formats-and-serialization/file-formats/ - CSV: https://wiki.databurst.tech/docs/roadmap/file-formats-and-serialization/file-formats/csv/ - JSON: https://wiki.databurst.tech/docs/roadmap/file-formats-and-serialization/file-formats/json/ - Parquet: https://wiki.databurst.tech/docs/roadmap/file-formats-and-serialization/file-formats/parquet/ - XML: https://wiki.databurst.tech/docs/roadmap/file-formats-and-serialization/file-formats/xml/ - Serialization: https://wiki.databurst.tech/docs/roadmap/file-formats-and-serialization/serialization/ - Avro: https://wiki.databurst.tech/docs/roadmap/file-formats-and-serialization/serialization/avro/ - Protocol Buffers: https://wiki.databurst.tech/docs/roadmap/file-formats-and-serialization/serialization/protobuf/ - Query Engine: https://wiki.databurst.tech/docs/roadmap/query-engine/ - Apache Hive: https://wiki.databurst.tech/docs/roadmap/query-engine/hive/ - SparkSQL: https://wiki.databurst.tech/docs/roadmap/query-engine/sparksql/ - Trino: https://wiki.databurst.tech/docs/roadmap/query-engine/trino/ - Presto: https://wiki.databurst.tech/docs/roadmap/query-engine/presto/ - Data Architecture: https://wiki.databurst.tech/docs/roadmap/data-architecture/ - Kappa: https://wiki.databurst.tech/docs/roadmap/data-architecture/kappa/ - Lambda: https://wiki.databurst.tech/docs/roadmap/data-architecture/lambda/ - Data Processing: https://wiki.databurst.tech/docs/roadmap/data-processing/ - Pipeline Orchestration: https://wiki.databurst.tech/docs/roadmap/pipeline-orchestration/ - Messaging and Streaming Platform: https://wiki.databurst.tech/docs/roadmap/messaging-and-streaming-platform/ - Data Governance: https://wiki.databurst.tech/docs/roadmap/data-governance/ - Data Quality: https://wiki.databurst.tech/docs/roadmap/data-governance/data-quality/ - Data Visualization: https://wiki.databurst.tech/docs/roadmap/data-visualization/ - Containerization: https://wiki.databurst.tech/docs/roadmap/containerization/ - Docker: https://wiki.databurst.tech/docs/roadmap/containerization/docker/ - Podman: https://wiki.databurst.tech/docs/roadmap/containerization/podman/ - containerd: https://wiki.databurst.tech/docs/roadmap/containerization/containerd/ - Container Orchestration: https://wiki.databurst.tech/docs/roadmap/container-orchestration/ - Kubernetes: https://wiki.databurst.tech/docs/roadmap/container-orchestration/kubernetes/ - Docker Swarm: https://wiki.databurst.tech/docs/roadmap/container-orchestration/docker-swarm/ - Apache Mesos: https://wiki.databurst.tech/docs/roadmap/container-orchestration/apache-mesos/ - CI/CD: https://wiki.databurst.tech/docs/roadmap/cicd/ - DataOps: https://wiki.databurst.tech/docs/roadmap/cicd/dataops/ - Infrastructure as Code: https://wiki.databurst.tech/docs/roadmap/infrastructure-as-code/ - Configuration Management: https://wiki.databurst.tech/docs/roadmap/infrastructure-as-code/configuration-management/ - Infrastructure Provisioning: https://wiki.databurst.tech/docs/roadmap/infrastructure-as-code/infrastructure-provisioning/ - Observability: https://wiki.databurst.tech/docs/roadmap/observability/ - Logging: https://wiki.databurst.tech/docs/roadmap/observability/logging/ - Monitoring: https://wiki.databurst.tech/docs/roadmap/observability/monitoring/ - Tracing: https://wiki.databurst.tech/docs/roadmap/observability/tracing/ - Glossary: https://wiki.databurst.tech/docs/roadmap/glossary/ # Data Modeling **Data modeling** is deciding how data is split into tables, what each row means, and how tables relate, so that the data is correct to load and easy to query. A good model makes the right query obvious and the wrong one hard. In analytics the model matters more than the engine: a fast warehouse cannot fix a fact table with an undefined grain. ## Three levels of a model | Level | Answers | Example | |---|---|---| | Conceptual | What things does the business care about? | Customer places Order, Order contains Product | | Logical | Which entities, attributes and keys, independent of a database? | `order(order_id PK, customer_id FK, ordered_at)` | | Physical | How is it stored in this engine? | Types, partitioning, clustering keys, Parquet/Iceberg layout | ## Two goals that pull in different directions - **Normalization** (3NF) removes redundancy so each fact is stored once. Ideal for transactional systems where many small writes must stay consistent. - **Denormalization** repeats data so reads need fewer joins. Ideal for analytics where queries scan and aggregate. Most analytical platforms do both: a normalized or raw integration layer, then denormalized models for consumers. See [normalization vs denormalization]({{< relref "/docs/roadmap/distributed-systems-concepts/normalization-or-denormalization" >}}). ## What this section covers - [Star Schema]({{< relref "/docs/roadmap/data-modeling/star-schema" >}}): Kimball's fact and dimension tables, grain, surrogate keys and slowly changing dimensions (SCD Type 2). The default for BI. - [Snowflake Schema]({{< relref "/docs/roadmap/data-modeling/snowflake" >}}): a star with normalized dimensions. Less redundancy, more joins. - [Data Vault]({{< relref "/docs/roadmap/data-modeling/data-vault" >}}): hubs, links and satellites for auditable, change-friendly integration of many sources. ## Choosing an approach | Situation | Good fit | |---|---| | Dashboards and self-service BI on a handful of sources | Star schema | | Large shared hierarchies, normalized sources, semantic layer hides joins | Snowflake schema where it helps | | Many volatile sources, strict audit requirements, big team | Data Vault for integration, stars on top | | One dashboard that must be very fast | One big table built from a star | A common 2026 layering, often called **medallion** (bronze, silver, gold) on lakehouses, maps onto these: raw data lands as-is, a cleaned and integrated layer (sometimes a vault) sits in the middle, and star schemas serve the gold layer. ## Learning Resources ### Books - [The Data Warehouse Toolkit, 3rd Edition](https://www.wiley.com/en-us/The+Data+Warehouse+Toolkit%3A+The+Definitive+Guide+to+Dimensional+Modeling%2C+3rd+Edition-p-9781118530801): Kimball and Ross - [Designing Data-Intensive Applications](https://www.oreilly.com/library/view/designing-data-intensive-applications/9781491903063/): chapter 2 on data models - [Data Modeling, A Beginner's Guide](https://www.oreilly.com/library/view/data-modeling-a/9780071623988/) ### Videos - [Introduction to Data Models](https://www.youtube.com/watch?v=OwQoj3GJfNY) - [What is Data Modelling? Beginner's Guide to Data Models and Data Modelling](https://www.youtube.com/watch?v=CUR6rKrIEGc)