Container Hardening Process
General requirements
- Automate the process as much as possible (e.g., via CI/CD pipelines).
- Document implicitly by using infrastructure-as-code (IaC).
- Use version control for all your work products (Containerfiles, IaC definitions, scan reports, etc.).
- Strive for hardened build environments (like defined in SLSA Level 3 - SLSA Level 3 on GitLab).
- Strive for transparency and make your assessment results freely available through standardized channels (VeX, container attestations, etc.)
A General Process
Producer or consumer?
As in supply chain security you can be the producer of a container image (building it yourself for your app) and/or the consumer (using a supplied image for an endproduct app, e.g. Redis Image).
Producer: Building your own application container image
When you are building your own container image (e.g., for your application), you should follow a process like this:
Prepare your application
Before building the container image, scan your application code and its dependencies for vulnerabilities, e.g. using DevGuard. You want to check for leaked secrets in your code, bad coding practices (SAST), and known vulnerabilities in your dependencies (SCA) (see the OWASP DevSecOps Pipeline for more details).
Make sure to adhere to the twelve factors for building web applications. These apply pretty good to containerized applications as well. Ensure that you do not store any configs as parts of the container image, but rather inject them as environment variables or mount them as files during runtime. Ensure to not write logs to the container file system, but rather write them to stdout. Ensure to not store any state in the container file system, but rather use external state services (e.g., databases, object storage, etc.).
Result of this step should be your code risk handled application code and a SBOM of your application dependencies. DevGuard can help you to identify and handle risks in your code and dependencies and can generate a SBOM for you.
This SBOM should be used to scan for known vulnerabilities in your dependencies (DevGuard does that for you). Follow the process described in the Container Hardening CVE Decision Guide to handle any vulnerabilities found. Ensure your solution can perform continuous vulnerability monitoring and you become aware of new vulnerabilities even when the application is already in use (DevGuard can do that for you).
Another use case of the SBOM is to ensure your license compliance. A violation can lead to cuts for availability requirements as well as legal issues. DevGuard can help you to identify and handle license compliance issues in your dependencies.
Plan a multi stage build
Typically you want to have a build environment with all the necessary tools and dependencies to compile your application, but you don’t want to ship all these tools in the final image. Therefore, plan a multi-stage build where you have a build stage and a final stage. In the final stage you only copy the necessary artifacts from the build stage. A multi stage build can also reduce the risk of leaking unwanted artifacts or information in the final image. Ensure to choose a trusted base image for the build stage. Specify the digest of the base image to pin the exact image version (tags can be overwritten).
You can find examples in the docs, in the DevGuards Containerfile or in the Container Hardening Work Bench Project.
Choose a minimal final base image
Choose a minimal base image as your final build target (e.g., Distroless, Scratch, Alpine) or use a hardened base image from a source you trust. The Iron Bank project for example provides a collection of DoD-approved, hardened container images.
Ensure to specify a non root user
Ensure to specify a non-root user in your Containerfile (e.g., USER 1000), so that the application does not run as root.
This contraints the potential damage an attacker can do when compromising the application and makes it harder to escape the container.
You can find a example in the DevGuards Containerfile
where you will find the USER directive and the --chown flags in the COPY directives.
Scan your containerfile and IaC definitions
Scan your Containerfile and any IaC definitions (e.g., Kubernetes manifests, Helm charts, Terraform scripts) for security issues and best practices (e.g., using checkov).
Using DevGuard you can ingest scan-reports in SARIF format from various tools like checkov and manage the findings, track their resolution, and fulfill compliance requirements.
Scan your final container image
After building your final container image, scan it for vulnerabilities using DevGuard or another vulnerability scanner. If vulnerabilities are found, follow the process described in the Container Hardening CVE Decision Guide to handle them.
Ensure your solution can perform continuous vulnerability monitoring and you become aware of new vulnerabilities even when the image is already in use (DevGuard can do that for you).
Ensure your license compliance for the image and its components too. A violation can lead to cuts for availability requirements as well as legal issues. DevGuard can help you to identify and handle license compliance issues.
Document, sign and attest your image
Ensure to attest your images SBOM and vulnerability status (VeX) to the final image and make it available transparently.
Ensure to add a release attestation to your image to state which application version is contained in the image, to be able to identify known vulnerabilities for that application version (you can find details why in the SBOM Problem Statement).
Ensure to sign your image (e.g., using cosign) to ensure its integrity and authenticity.
Consumer: Using a supplied container image
When you are using a supplied container image (e.g., Redis image), you should follow a process like this:
Check the source of the image
Ensure to use images from a trusted source, and in best case, they are signed.
Scan the image
Scan the image for vulnerabilities using DevGuard or another vulnerability scanner. If vulnerabilities are found, follow the process described in the Container Hardening CVE Decision Guide to handle them.
Ensure your solution can perform continuous vulnerability monitoring and you become aware of new vulnerabilities even when the image is already in use (DevGuard can do that for you).
Ensure your license compliance for the image and its components too. A violation can lead to cuts for availability requirements as well as legal issues. DevGuard can help you to identify and handle license compliance issues.
Modify the image if necessary
If the image does not meet your security requirements, consider modifying it by creating a new Containerfile that uses the supplied image as a base and applies necessary hardening steps (e.g., removing unnecessary components, adding security configurations, setting a non-root user, etc.). Then build and scan the modified image.
Document, sign and attest your image
Ensure to attest your images SBOM and vulnerability status (VeX) to the final image and make it available transparently.
Ensure to add a release attestation to your image to state which application version is contained in the image, to be able to identify known vulnerabilities for that application version (you can find details why in the SBOM Problem Statement).
Ensure to sign your image (e.g., using cosign) to ensure its integrity and authenticity.