CI/CD Python – Part 3: Going further with Poetry, Docker and advanced publishing
Published on 19 July 2025
In the first two parts of this series, we have: - Established a functional CI/CD pipeline for a Python application using GitHub Actions and PyPI. - Industrialized this pipeline with multi-version testing, a gradual release via Test PyPI, and quality and security tools.
In this third part, we will go.beyond just publishing on PyPIto build a complete, robust, and professional CI/CD pipeline with : - Poetryfor a modern packaging and optimized dependency management. - Dockerto create reproducible and cross-platform builds. - Conditional publicationto manage scenarios such as release candidates. - Automation tools(Renovate, Dependabot) to keep the pipeline up to date effortlessly.
Integration with Poetry
Poetry replaces the old packaging tools (setup.py, requirements.txt) by centralizing the management of dependencies and the build in`pyproject.toml`.
Installation of Poetry
# Installer Poetry
curl -sSL https://install.python-poetry.org | python3 -
# Vérifier la version
poetry --versionProject initialization
# Initialiser un nouveau projet avec Poetry
poetry init
# Suivre l'assistant pour renseigner : nom, version, description, licence, dépendances.This generates a file`pyproject.toml` :
[tool.poetry]
name = "playlist-downloader"
version = "0.1.0"
description = "CLI tool for managing YouTube playlists"
authors = ["Christophe Hérolivier <[email protected]>"]
[tool.poetry.dependencies]
python = ">=3.8"
typer = "^0.9.0"
yt-dlp = "^2023.7.6"
google-api-python-client = "^2.0.0"
google-auth-oauthlib = "^1.0.0"
[tool.poetry.group.dev.dependencies]
pytest = "^7.0"
mypy = "^1.0"
bandit = "^1.7"
safety = "^2.3"
black = "^23.0"
ruff = "^0.1"Adding and installing dependencies
poetry add typer yt-dlp google-api-python-client google-auth-oauthlib
poetry add --group dev pytest mypy black bandit safety ruffPublication with Poetry
Poetry natively manages publishing:
# Publication sur Test PyPI
poetry publish --build --repository test-pypi
# Publication sur PyPI
poetry publish --buildThis command automatically uses the information present in`pyproject.toml`.
Reproducible builds with Docker
To ensure identical executions in development, CI/CD and production, Docker integrates perfectly with Poetry.
Dockerfile example
FROM python:3.11-slim
WORKDIR /app
COPY pyproject.toml poetry.lock ./
RUN pip install poetry
RUN poetry install --no-root --only main
COPY . .
CMD ["poetry", "run", "python", "cli.py"]This guarantees: - A frozen Python environment. - Locked dependencies via`poetry.lock`. - An executable image on any system supporting Docker
Integration in GitHub Actions
name: Docker Build
on:
push:
branches: [main]
jobs:
build-docker:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build Docker image
run: docker build -t ghcr.io/${{ github.repository }}:latest .
- name: Push Docker image
run: docker push ghcr.io/${{ github.repository }}:latestConditional publication
In a professional pipeline, one must be able to publish only in certain cases: - Release candidates to Test PyPI. - Stable versions to PyPI. - Builds Docker triggered only for`main`.
- name: Publish to Test PyPI
if: contains(github.ref, '-rc')
run: poetry publish --build --repository test-pypi
- name: Publish to PyPI
if: startsWith(github.ref, 'refs/tags/v')
run: poetry publish --buildThis approach avoids accidental posts.
Dependency automation with Renovate and Dependabot
To prevent your dependencies from becoming obsolete, integrate automatic update tools.
Dependabot
version: 2
updates:
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"Dependabot opens PRs every week to update Python and GitHub Actions dependencies.
Renovate
{
"extends": ["config:base"],
"packageRules": [
{
"matchManagers": ["pip"],
"groupName": "python-dependencies",
"schedule": ["before 6am on monday"]
}
]
}Renovate allows for finer control: dependency grouping, scheduling, and advanced rules.
Diagrams PlantUML
Use Case – Advanced CI/CD
@startuml actor Developer actor GitHub as "GitHub Actions" actor PyPI actor DockerRegistry as "Docker Registry" actor Automation as "Renovate/Dependabot" Developer --> (Push code) (Trigger CI) --> GitHub GitHub --> (Run Tests & Lint) GitHub --> (Build Docker Image) GitHub --> DockerRegistry GitHub --> (Publish to Test PyPI) GitHub --> (Promote to PyPI) Automation --> (Update Dependencies) @enduml
Sequence – Conditional Publication
@startuml !theme vibrant left to right direction Developer -> GitHub: Push tag v1.2.3-rc GitHub -> CI: Run tests CI -> CD: Check release type CD -> Test PyPI: Publish RC Developer -> GitHub: Push tag v1.2.3 GitHub -> CD: Publish to PyPI CD -> Docker Registry: Push Docker image @enduml
States – CI/CD Pipeline
@startuml [*] --> Idle Idle --> CI_Running : push CI_Running --> CI_Success : tests ok CI_Running --> CI_Failed : tests fail CI_Success --> CD_Running : tag detected CD_Running --> CD_Success : publish ok CD_Running --> CD_Failed : error CD_Success --> [*] @enduml
Deployment – CI/CD Architecture
@startuml
node "Developer Machine" {
component "Git Client"
}
node "GitHub Actions" {
component "CI Workflow"
component "CD Workflow"
}
node "Package Repositories" {
artifact "Test PyPI"
artifact "PyPI"
artifact "Docker Registry"
}
"Developer Machine" --> "GitHub Actions"
"GitHub Actions" --> "Test PyPI"
"GitHub Actions" --> "PyPI"
"GitHub Actions" --> "Docker Registry"
@enduml
Conclusion
In this third part, we saw how: - Modernize Python packaging with Poetry. - Ensure reproducible builds via Docker. Set up a conditional publication. - Automate dependency updates.
You now have a CI/CD pipelinecomplete, industrialized and secure, ready to evolve with your projects.