GitHub
GitHub is the largest Git hosting platform, owned by Microsoft since 2018. It hosts repositories and wraps them with pull requests, code review, issues, project boards, CI/CD (GitHub Actions), package and container registries, and security scanning. It is also the home of most open-source data tooling: Airflow, Spark, dbt, DuckDB, Trino and Iceberg all develop on GitHub, so knowing it well is part of contributing to and tracking that ecosystem.
Features that matter for data teams
| Feature | What it does |
|---|---|
| Pull requests | Diff, review comments, suggested changes, required approvals. See the PR flow. |
| Rulesets / branch protection | Block direct pushes to main, require passing checks, reviews, linear history or signed commits |
| CODEOWNERS | A file mapping paths to teams, e.g. models/finance/ @acme/finance-data, who are auto-requested as reviewers |
| GitHub Actions | CI/CD defined in .github/workflows/*.yml, triggered by pushes, PRs, schedules or manual runs |
| Merge queue | Tests each PR combined with the ones ahead of it before merging, so main never breaks on a busy repo |
| Environments | Deployment targets (staging, prod) with required reviewers and their own secrets |
| GHCR | Container registry at ghcr.io for the images your pipelines run |
| Security | Dependabot updates, secret scanning with push protection, CodeQL code scanning |
| Codespaces | Cloud dev environments from a devcontainer.json, handy for onboarding |
| Copilot | AI assistance in the editor and for PR summaries and reviews |
GitHub Actions in one example
name: deploy-dags
on:
push:
branches: [main]
permissions:
id-token: write # OIDC: short-lived cloud credentials, no stored keys
contents: read
jobs:
deploy:
runs-on: ubuntu-latest
environment: prod # waits for an approval if the environment requires one
steps:
- uses: actions/checkout@v5
- uses: aws-actions/configure-aws-credentials@v4
with: { role-to-assume: arn:aws:iam::123456789012:role/dag-deployer, aws-region: eu-central-1 }
- run: aws s3 sync dags/ s3://acme-airflow/dags/ --deleteWorkflows are built from reusable actions published in the Marketplace. Pin third-party actions to a full commit SHA: a compromised tag can run arbitrary code with your secrets.
The gh CLI
gh repo clone acme/data-platform
gh pr create --fill --reviewer acme/data-eng
gh pr checks # watch CI on your PR
gh pr merge --squash --delete-branch
gh run watch # follow an Actions runCommon mistakes
- Long-lived cloud keys as repository secrets. Use OIDC federation so each run gets short-lived credentials.
- Workflows triggered by
pull_request_targetthat check out PR code. This runs untrusted fork code with your secrets. Read GitHub’s security hardening guide before using it. - Admins bypassing protection. Keep administrators off the ruleset bypass list so “just this once” pushes to
maindo not happen. - Giant monorepo checks on every PR. Use path filters so the dbt checks run only when
models/changes.
Learning Resources
- GitHub Docs
- GitHub Skills: free interactive courses (replaced the old Learning Lab)
- GitHub Actions documentation and security hardening for Actions
- About rulesets
- GitHub CLI manual