$ git log --oneline --graph --all

Git

Git is a distributed version control system: every clone of a repository holds the complete history, so you can commit, branch and compare versions offline and sync with others later. Linus Torvalds wrote it in 2005 for Linux kernel development; today it is the standard for virtually all software and data projects.

The mental model

Three ideas explain most of Git:

  1. A commit is a snapshot of the whole project, plus metadata (author, message) and a pointer to its parent commit(s). Its ID is a hash of all of that, so changing anything in history creates new commits with new IDs.
  2. A branch is just a movable pointer to a commit. main and feature are labels; creating a branch costs a few bytes.
  3. Four places: your working tree, the staging area, your local repository and a remote. Every everyday command moves changes from one place to the next.

Git is not GitHub. Git is the tool on your machine and works fully offline; GitHub, GitLab and Bitbucket host a shared copy and add pull requests, review and CI on top.

Four places your code lives

$ git add · git commit · git push · git fetch · git pull
One change travels from your editor to the team, and the team’s work comes back
your machine GitHub · GitLab WORKING TREE your files on disk STAGING AREA index: the next commit LOCAL REPO .git: every commit REMOTE origin: shared copy report.pyuntracked orders.sqlmodified git add orders.sqlstaged git commit 3f2a1c9main git push 3f2a1c9yours 8b4e0d2teammate git fetch origin/main8b4e0d2 git pull = fetch + merge into your files
  1. You edited orders.sql and created report.py. git status calls the first modified and the second untracked: Git has never stored it.
  2. git add orders.sql copies the change into the staging area. Only staged changes go into the next commit; report.py stays out.
  3. git commit turns the staging area into a permanent snapshot in .git, with an ID like 3f2a1c9. Still only on your laptop.
  4. git push uploads your new commits to the remote, so the team can see them.
  5. A teammate pushed 8b4e0d2. git fetch downloads it and moves origin/main, but your files do not change.
  6. git pull is fetch plus merge: the teammate's change lands in your working tree. Pull before you start, and before you push.

git status tells you where each file is:

State Meaning Next command
Untracked New file Git has never stored git add file, or list it in .gitignore
Modified Tracked file changed, not staged git add file, or git restore file to drop the edit
Staged Will be in the next commit git commit, or git restore --staged file to unstage
Committed (clean) Working tree matches the last commit git push to share it
git status
git diff                            # unstaged changes, line by line
git add models/orders.sql           # or: git add -p to pick hunks
git diff --staged                   # what will be committed
git commit -m "Deduplicate orders on order_id"
git log --oneline

A .gitignore file keeps secrets and junk out of the staging area: .env, *.log, __pycache__/, target/, local data files.

One-time setup

Tell Git who you are, once per machine. The name and email are written into every commit.

git config --global user.name "Alice Example"
git config --global user.email "[email protected]"
git config --global init.defaultBranch main
git config --global pull.rebase true        # optional: git pull rebases instead of merging

Then let GitHub or GitLab recognize your machine with an SSH key, so you never type a password to push:

ssh-keygen -t ed25519 -C "[email protected]"   # creates ~/.ssh/id_ed25519 and id_ed25519.pub
cat ~/.ssh/id_ed25519.pub                      # paste into GitHub › Settings › SSH and GPG keys
ssh -T [email protected]                          # "Hi alice! You've successfully authenticated..."

Start a repository with git init in an existing folder, or copy one with git clone [email protected]:example/pipelines.git.

Branches

A branch lets you work on a change without touching main. HEAD is the pointer to the branch you are on; new commits go on that branch, and the branch label moves forward with them.

$ git switch -c feature/login
A branch is a label that moves; HEAD says which label you are on
your laptop A B main HEAD feature/login HEAD B C feature/login HEAD D feature/login HEAD origin · GitHub A B C D feature/login main origin/feature/login $ git push -u origin feature/login HEAD
  1. History so far: commits A and B on main. HEAD points at main, so that is the branch you are on.
  2. git switch -c feature/login creates a new label at the same commit and moves HEAD to it. No files are copied.
  3. Edit, git add, git commit: the new commit C goes on feature/login, and the label moves forward with HEAD. main stays at B.
  4. Another commit, D. Each commit points back to its parent, so the branch is simply the chain behind its label.
  5. git push -u origin feature/login uploads C and D and creates the branch on the remote. -u links the two, so later a plain git push is enough.
  6. git switch main moves HEAD back to B and rewrites your files to match it. Your feature work is safe on its own branch, ready for a pull request.
git branch                     # list local branches; * marks the current one
git switch -c feature/login    # create and switch
git switch main                # go back (commit or stash your changes first)
git branch -d feature/login    # delete after it has been merged

Merge vs rebase

Sooner or later main moves on while you work on a branch. There are two ways to combine the histories.

$ git merge feature vs git rebase main
Same starting point, two ways to integrate a branch
git merge feature git rebase main ABCDEmainfeature ABCDEmainfeature Mmain M has two parents: C and E.D and E keep their hashes. D′E′featureold D, E: unreferenced mainfast-forward: linear
  1. Both panels start the same: feature branched off at B and added D and E, while main gained C.
  2. Merge: Git creates a new merge commit M that joins both lines, and main moves to it.
  3. M has two parents. Nothing was rewritten: the history shows what really happened, including the parallel work.
  4. Rebase: Git replays D and E on top of C as new commits D′ and E′, with new hashes. The old D and E are left behind.
  5. Now main can fast-forward to E′: a straight, linear history with no merge commit. The cost: D′ and E′ are different commits from what you pushed before.
Merge Rebase
History shape True shape, with merge commits Linear, easy to read and bisect
Rewrites commits No Yes: new hashes
Safe on shared branches Yes No: others have the old commits
Conflicts Resolved once, in the merge commit Possibly once per replayed commit

The rule of thumb: rebase your own local branch onto main to keep it current and tidy; never rebase commits others have already pulled. Many teams avoid the question by using squash merge in pull requests: the whole branch lands on main as one commit.

Everyday commands

Command What it does
git status, git diff, git diff --staged What changed, staged or not
git log --oneline --graph Compact history
git switch <branch>, git switch -c <new> Change or create branches
git pull --rebase Fetch and replay your local commits on top
git restore <file> Discard working-tree changes to a file
git commit --amend Fix the last commit (before pushing)
git revert <commit> Undo a commit with a new commit: safe on shared branches
git reset --soft HEAD~1 Un-commit, keep the changes staged
git stash Park uncommitted work temporarily
git bisect Binary-search history for the commit that broke something
git reflog Every position HEAD has been: how you recover “lost” commits

Your first push

From an empty folder to a merged pull request, using the commands above:

  1. Create an empty repository on GitHub called workshop, with no README, so the first push has nothing to conflict with.
  2. Start the local repository and make the first commit:
    cd ~/workshop
    git init
    git add backup.sh
    git commit -m "Add backup script"
  3. Keep generated files out: echo "*.tar.gz" > .gitignore, then add and commit .gitignore.
  4. Check the SSH key, connect the remote and push main:
    ssh -T [email protected]
    git remote add origin [email protected]:<you>/workshop.git
    git remote -v                  # origin is just a nickname for that URL
    git push -u origin main
  5. On a new branch, add README.md, commit it and push the branch: git switch -c add-readme, git add README.md, git commit -m "Add README", git push -u origin add-readme.
  6. Open a pull request on GitHub, merge it, then update your local copy: git switch main and git pull.
  7. Look at the result: git log --oneline --graph --all.
  8. Bonus: edit the same line on GitHub and locally, pull, and resolve the conflict. Git marks it between <<<<<<<, ======= and >>>>>>>; keep what you want, delete the markers, then add and commit.

A workflow for data repositories

  • Trunk-based development: short-lived branches (hours to days), merged to main through pull requests with CI. Avoid long-lived develop branches; they drift and merge painfully.
  • Keep each change small and focused: one model, one DAG, one migration.
  • Write commit messages that say why: “Deduplicate orders on order_id (late CDC events cause doubles)” beats “fix”.
  • Use pre-commit hooks for formatting (ruff, sqlfluff) and secret scanning.
  • Large files that must live with code (small seed CSVs are fine; models and media are not) belong in Git LFS.

Common mistakes

  • Committing secrets. Deleting the file later does not remove it from history. Rotate the secret immediately; then clean history if needed.
  • git push --force on a shared branch. It silently discards others’ commits. Use --force-with-lease, and only on your own branches.
  • Huge, mixed commits. A commit that changes ten models and the CI config is hard to review and impossible to revert cleanly.
  • Committing data and notebooks with outputs. Repository size explodes and diffs become unreadable. Strip notebook outputs (nbstripout) and keep data elsewhere.

Learning Resources

Books

Interactive

Reference