CI/CD and Continuous Deployment

A small breakdown every Tuesday

"Instead of heading for a big integration breakdown, I decided to have a small breakdown every Tuesday evening."

Adapted from Graham Parke, No Hope for Gomez!

Knight Capital, 1 August 2012

• A technician copies new code to the order router's servers and misses one of eight
• The eighth still holds Power Peg, code unused since 2003
• The new code reuses Power Peg's old flag
• The flag goes live: 4 million executions, 397 million shares, about 45 minutes
• More than $460 million lost

More pull requests, same hours

Continuous integration is a habit

• Merge small changes, at least daily
• Every change runs the same checks
• A red main gets fixed before anything else

What gets shipped

• Source code goes in
• ./mvnw package builds one executable JAR: your classes, every dependency, an embedded Tomcat
• Project Pulse also builds the Vue frontend into that same JAR
• java -jar starts it anywhere Java 21 is installed

It works on my machine

Image and container

GitHub Actions in five words

classDiagram
    direction LR
    Repository "1" --> "0..*" Workflow : holds
    Event "1..*" --> "0..*" Workflow : triggers
    Workflow "1" *-- "1..*" Job
    Job "1" *-- "1..*" Step
    Job "1" --> "1" Runner : runs on

Project Pulse: maven-build.yml

on:
  pull_request:
    branches: [main]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@11d5960a...   # v4.4.0
      - uses: actions/setup-java@cf277c60... # v4.9.1
        with: { java-version: "21", distribution: "temurin", cache: maven }
      - run: mvn --batch-mode verify
        working-directory: ./backend
  frontend:   # npm ci, ESLint, type check, Prettier
  dockerfile: # Hadolint

The runner starts empty

• No local database
• No file you forgot to commit
• No library you installed by hand last month

Keep it fast

• Three jobs in parallel: the slowest sets the wait
• Wait half an hour, and people stop waiting
• Aim for Extreme Programming's ten-minute build

Two workflows, one path

flowchart LR
    pr["Pull request"] --> ci["maven-build.yml<br/>build · frontend · dockerfile"]
    ci -->|"all green, up to date"| main["Merge to main"]
    main --> build["azure-webapps-deploy.yml<br/>image tagged with the SHA"]
    build --> deploy["staging slot"]
    deploy -->|"a person swaps"| prod["production slot"]

The checks ran. Nothing listened.

• Every pull request ran build, frontend, and dockerfile
• No ruleset on main: Merge stayed clickable next to a red X
• After merge, the deploy built with -DskipTests
• Trusting tests that might never have passed

Protect main

Leave the bypass list empty

Green alone, broken together

• Monday: your pull request goes green against Monday's main
• Tuesday: a teammate merges something else
• Your green check tested a main that no longer exists
• Both changes pass alone and break together

Know what each check checks

- name: ESLint (report only)
  continue-on-error: true
  run: npm run lint:check

Flaky checks destroy the gate

• A test fails at random
• Everyone learns to press Re-run
• A real failure gets re-run too

CD means two things

Project Pulse does both

Build once, promote that build

tags: ghcr.io/${{ env.REPO }}:${{ github.sha }}

• The image is built once, tagged with the commit SHA
• Staging runs that image; the swap moves that same image
• Nothing is rebuilt for production

When is -DskipTests safe?

Bad changes ship fast too

• 18 November 2025: a permissions change makes a query return duplicates
• A generated config file doubles; it passes a hardcoded limit of 200 (normal: about 60)
• Rust code calls unwrap() on the error; the proxy panics
• The file is "rapidly propagated across the network"
• Errors from 11:20 UTC; normal again at 17:06

"Hardening ingestion of Cloudflare-generated configuration files in the same way we would for user-generated input."

Each slot keeps its own database

• Database address and password live in Azure Key Vault
• SPRING_PROFILES_ACTIVE picks application-staging.yml or application-prod.yml
• AZURE_KEY_VAULT_ENDPOINT picks the vault
• Both are deployment slot settings: they stay with the slot, not the code

Rolling back

V5 and V6

Expand and contract

• Expand: add the new column, nullable, used by nothing
• Write: ship code that writes both
• Read: ship code that reads the new one, handling empty rows
• Contract: once every running build uses it, remove the old one

Secrets in the pipeline

Forks get no secrets

• A pull request from a fork runs with no secrets and a read-only token
• That's why Project Pulse's checks need none

Short-lived beats long-lived

An agent in CI

• Guard: it acts only for people with write access (leave that default on)
• Guard: a read-only GITHUB_TOKEN
• Guard: Actions pinned to SHAs

Same commit, same build

Versions and tags

• A tag marks a baseline: this commit is what we released
• MAJOR.MINOR.PATCH: breaks, adds, fixes
• v1.4.2: a patch about thirteen hours after v1.4.1, notes naming the bug and issue #28
• Today: last tag v1.5.0, 61 merged pull requests ago; pom.xml says 2.0.0

Tag your Checkpoint 2

git tag -a v0.1.0 -m "Checkpoint 2: proving slice"
git push origin v0.1.0

Then: Releases, Draft a new release, choose the tag, Generate release notes.

Feature flags rot

• A flag ships code switched off: on and off without a deploy
• Project Pulse: app.reminders.enabled (off in staging)
• Knight Capital: a reused flag woke Power Peg

Where will your project run?

• Whose account hosts it, and who can turn it off?
• Who pays, and what happens when a credit runs out?
• Who owns it after December, and after May?
• What is staging for you?
• What data will it hold?

Over the wall

DevOps: you build it, you run it

"You build it, you run it."

Werner Vogels, Amazon

CALMS

How DevOps measures itself

DevOps and SRE

class SRE implements DevOps { }

In an interview: "What is DevOps?"

• What: one team owns software from commit to production, instead of handing releases to operations
• How: automation and small, frequent changes, so each failure is small and quick to fix
• How you know: the DORA metrics
• Then one example from your project: your pipeline, your gate, your rollback

Red made green

The agent and the pipeline

Your week

• Friday studio: the gate on your repository, the Protect main rules, your charter line, and an open deployment question in section 7
• Checkpoint 2, by Friday: demo from a tagged v0.1.0 with release notes
• Assignment 4, Triage the analyzers: out today; your pull request leaves the pipeline green