Implementing CI/CD Pipelines

A CI/CD pipeline on AWS is an explicit state machine in which each stage either succeeds and passes its output forward or fails and stops the train. The senior decision is not "should I have a pipeline" but how to split the pipeline into stages so that the failure cost of each stage is bounded, the artifacts flowing between stages are immutable, and the production deployment stage can be aborted on signal without manual intervention. AWS CodePipeline is the orchestrator; CodeBuild is the compute layer for builds and tests; CodeDeploy is the deployment layer for compute targets. Action providers such as AWS Lambda invocation actions, CloudFormation deploy actions, and ECS deploy actions plug into the pipeline as terminal or intermediate steps.

The cost of a poorly partitioned pipeline shows up in two places. First, blast radius: if your "deploy" stage updates 30 microservices in a single CloudFormation stack, a failure in one resource rolls back the other 29 even when they are unrelated. Second, mean time to recovery: if your pipeline lacks an approval action between staging and production but pushes the same artifact to both, an incident detected in production forces you to re-run the entire pipeline from the source stage to ship a hotfix, instead of re-invoking only the deploy stage with a known-good artifact. CodePipeline addresses this by letting you scope action providers, run actions in parallel within a stage, and trigger pipelines from specific branches with filtering.

A canonical AWS pipeline contains a Source stage (CodeConnections to GitHub, Amazon S3, or Amazon ECR), a Build stage (CodeBuild project that compiles and runs unit tests), a Test stage (CodeBuild or Lambda action that runs integration and acceptance suites against a deployed copy), a Staging deploy stage (CloudFormation or CodeDeploy targeting a staging environment), an Approval stage (manual approval action), and a Production deploy stage (CodeDeploy with traffic shifting). The pipeline definition itself is infrastructure code: it lives in CloudFormation or AWS CDK, is version-controlled alongside the application, and runs as a pipeline-of-pipelines pattern when the platform team wants to update the pipeline structure without manual console edits.

CodePipeline V2 (the default pipeline type for new pipelines) adds pipeline-level triggers with branch and file-path filters, named parameters for executions, and the ability to start an execution with a specific commit SHA. The V2 trigger model means a single pipeline can react to a push on main differently from a tag push on a release tag, branching to different stages or different artifact storage paths. V1 pipelines that only support polling or simple webhook triggers should be migrated when the workload needs filtered triggers or git-clone style source actions.

Running Different Types of Tests in Pipelines

The AWS-native compute for in-pipeline tests is CodeBuild and Lambda. Pick CodeBuild when the test needs a build environment, a non-trivial language runtime, container support, or more than 15 minutes of execution time. Pick Lambda when the test is a quick assertion (verifying that a deployed API returns a 200, checking that a CloudFormation output matches an expected value, calling a third-party verification API). A CodeBuild project for unit tests uses a buildspec.yml with explicit phases (install, pre_build, build, post_build) and emits test reports via the reports block; CodeBuild surfaces those reports in the CodeBuild console as pass/fail counts and trends. A Lambda invocation action passes user parameters through UserParameters and reports success or failure based on the return code.

version: 0.2

env:
 variables:
 PYTHONPATH: src
 secrets-manager:
 NEXUS_TOKEN: prod/build/nexus:token

phases:
 install:
 runtime-versions:
 python: 3.11
 commands:
 - python -m pip install --upgrade pip
 - pip install -r requirements-dev.txt
 pre_build:
 commands:
 - echo "Running syntactic analysis"
 - ruff check src tests
 - mypy --strict src
 build:
 commands:
 - echo "Running unit tests with coverage"
 - pytest tests/unit --junitxml=reports/junit-unit.xml --cov=src --cov-report=xml:reports/coverage.xml --cov-fail-under=80
 post_build:
 commands:
 - echo "Packaging artifact"
 - zip -r app.zip src/ requirements.txt

reports:
 unit-tests:
 files:
 - reports/junit-unit.xml
 file-format: JUNITXML
 coverage:
 files:
 - reports/coverage.xml
 file-format: COBERTURAXML

artifacts:
 files:
 - app.zip
 - appspec.yml
 - scripts/**/*

Deployment Models For the Workload

AWS CodeDeploy and the CodePipeline action ecosystem support five canonical deployment models, and the supported set narrows by compute target. The model dictates how new code reaches users and how a failure is reversed. On Amazon EC2 with CodeDeploy, supported models are in-place (replace existing instances one at a time using a deployment configuration like CodeDeployDefault.OneAtATime, HalfAtATime, or AllAtOnce) and blue/green (provision a parallel green fleet, shift traffic via an Elastic Load Balancer, terminate the blue fleet after a wait window). On Amazon ECS, the native rolling update varies minimumHealthyPercent and maximumPercent to swap tasks; the CodeDeploy ECS controller adds blue/green with shifting traffic over an Application Load Balancer or Network Load Balancer listener. On AWS Lambda, traffic shifting on an alias supports Canary10Percent5Minutes, Canary10Percent15Minutes, Linear10PercentEvery1Minute, Linear10PercentEvery10Minutes, and AllAtOnce.

Immutable deployments deserve a separate mention because they are not a CodeDeploy mode but a strategy. An immutable deployment replaces the entire compute fleet with a freshly built AMI or container image, never patching an existing instance. Elastic Beanstalk supports an explicit Immutable deployment policy; for an EC2 Auto Scaling Group, the immutable equivalent is to create a new launch template version, create a new Auto Scaling Group, attach it to the load balancer target group, drain the old group, and delete it. CloudFormation handles this pattern with the AutoScalingReplacingUpdate policy on the Auto Scaling Group, which creates a new group, runs health checks, and deletes the old one on success.

💡 Exam Trap: A CodePipeline action runs only after every action in the previous stage finishes. Two actions in the same stage run in parallel by default, but you can chain them by setting RunOrder to increasing integers. If the question asks how to run an integration test only after the staging deploy completes, the answer is "place the test action in the same stage as the deploy with a higher RunOrder", not "put them in different stages".

💡 Exam Trap: CodeBuild's buildspec.yml reports block requires file-format to be one of JUNITXML, NUNITXML, NUNIT3XML, CUCUMBERJSON, TESTNGXML, VISUALSTUDIOTRX, or JSON, and code coverage reports must use COBERTURAXML, JACOCOXML, SIMPLECOV, CLOVERXML, or LCOV. A free-form text report will be silently ignored, and the build will pass with no coverage signal.

⚠️ Anti-Pattern: Running production deployments from the same pipeline action that built the artifact. The build context has elevated permissions (artifact write, container registry push), and reusing it for deploy violates least privilege. Split into a Build stage with a build role and a Deploy stage with a deploy role assumed via sts:AssumeRole from CodePipeline into the target account.

🎯 Scenario: A platform team wires a CodePipeline V2 pipeline with a Source action from a GitHub repository, a Build action in CodeBuild that produces a CloudFormation template and a Lambda ZIP, a Test action that runs a Lambda invocation, and a Deploy action that creates a CloudFormation change set. A push to the release/* branch must run the pipeline but a push to feature/* must not. The answer is to configure the V2 pipeline trigger with a branch filter release/*. A V1 webhook would deliver every push and the source action would still execute, wasting build minutes.

Decision Anchor: Choose CodeBuild for in-pipeline test execution when the test needs a custom runtime, longer than 15 minutes, or a container build context. Choose Lambda invocation actions when the test is an HTTP probe, an API contract check, or any quick assertion that completes in under 15 minutes and does not need a build environment.

The CodePipeline execution model is critical to understand for failure analysis. Each pipeline execution carries an executionId, and each artifact produced in a stage is keyed by that executionId in the artifact bucket. When you re-run a failed stage with "Retry" in the console, CodePipeline reuses the input artifacts from the previous stage; it does not re-run the source stage. When you "Release Change", CodePipeline pulls the latest source and creates a new execution. The difference matters when an integration test fails because of a flaky network condition: a Retry costs nothing in build minutes; a Release Change rebuilds and reships, potentially introducing new code.