Part 1: Setting up a simple CI/CD pipeline for Python and PyPI

----
on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
----

The pipeline triggers on: - Every push to the branch`main` - Every pull request to`main`

This approach ensures that the main code remains stable and that every contribution is validated before integration.

==== 2. Execution Environment

```yaml
----
runs-on: ubuntu-latest
----

Ubuntu Latest offers a good compromise between performance, cost, and compatibility for most Python projects.

==== 3. Code Checkout

```yaml
----
- name: Checkout code
  uses: actions/checkout@v4
----

The action`checkout@v4`retrieves the source code from the repository. Version v4 brings performance and security improvements.

==== 4. Python Configuration

```yaml
----
- name: Set up Python
  uses: actions/setup-python@v5
  with:
    python-version: '3.x'
----

Using`'3.x'`allows for the automatic use of the latest stable version of Python 3, simplifying maintenance.

==== 5. Dependency Installation

```yaml
----
- name: Install dependencies
  run: |
    python -m pip install --upgrade pip
    pip install -r requirements.txt
    pip install ruff pytest pytest-mock
----

This step: - Updates pip to the latest version - Installs project dependencies - Adds development tools (linting and tests)

==== 6. Linting with Ruff

```yaml
----
- name: Run Linting (Ruff)
  run: ruff check .
----

**Ruff**is an ultra-fast Python linter written in Rust. It combines the features of several tools (Flake8, Black, isort) into a single high-performance tool.

==== 7. Running Tests

```yaml
----
- name: Run Tests (Pytest)
  run: pytest
----

Pytest executes the entire test suite, ensuring that changes do not introduce regressions.

== Deployment (CD) Pipeline Configuration

=== CD Workflow Structure

The CD workflow triggers only during GitHub releases and automates publication to PyPI:

[source,yaml]
----
name: Publish to PyPI

on:
  release:
    types:
      - published

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: '3.x'

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install setuptools wheel twine

    - name: Build and publish
      env:
        TWINE_USERNAME: __token__
        TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
      run: |
        python setup.py sdist bdist_wheel
        twine upload dist/*
----

=== CD Steps Analysis

==== 1. Release Trigger

```yaml
----
on:
  release:
    types:
      - published
----

The CD pipeline only triggers upon the publication of a GitHub release. This approach ensures precise control over deployments.

==== 2. Build Tools Installation

```yaml
----
pip install setuptools wheel twine
----

- **setuptools**: Python packaging tools - **wheel**: Modern Python distribution format - **twine**: Secure tool for uploading to PyPI

==== 3. Authentication Configuration

```yaml
----
env:
  TWINE_USERNAME: __token__
  TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
----

Authentication uses a PyPI API token stored as a GitHub secret, which is more secure than classic credentials.

==== 4. Build and Publication

```yaml
----
run: |
  python setup.py sdist bdist_wheel
  twine upload dist/*
----

- `sdist`: Creates a source distribution - `bdist_wheel`: Creates a wheel (binary distribution) - `twine upload`: Publishes the distributions to PyPI

== Python Package Configuration

=== setup.py Structure

For the pipeline to work, your project must include a`setup.py`file:

[source,python]
----
from setuptools import setup, find_packages

with open("README.adoc", "r", encoding="utf-8") as fh:
    long_description = fh.read()

setup(
    name="playlist-downloader",
    version="1.0.0",
    author="Votre Nom",
    author_email="votre.email@example.com",
    description="CLI tool for managing YouTube playlists",
    long_description=long_description,
    long_description_content_type="text/plain",
    url="https://github.com/cheroliv/playlist-downloader",
    packages=find_packages(),
    classifiers=[
        "Development Status :: 4 - Beta",
        "Intended Audience :: Developers",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
        "Programming Language :: Python :: 3",
        "Programming Language :: Python :: 3.8",
        "Programming Language :: Python :: 3.9",
        "Programming Language :: Python :: 3.10",
        "Programming Language :: Python :: 3.11",
    ],
    python_requires=">=3.8",
    install_requires=[
        "typer>=0.9.0",
        "yt-dlp>=2023.1.6",
        "google-api-python-client>=2.70.0",
        "google-auth-oauthlib>=0.7.1",
        "pymonad>=2.4.0",
        "pyyaml>=6.0",
    ],
    entry_points={
        "console_scripts": [
            "playlist-downloader=cli:app",
        ],
    },
)
----

=== Key Points of setup.py

1. **Metadata**: Name, version, author, description
2. **Dependencies**: List of required packages
3. **Entry Points**: Exposed CLI commands
4. **Classifiers**: Metadata for PyPI

== Securing with GitHub Secrets

=== PyPI Token Configuration

1. **Create an API token on PyPI**:

- Log in to PyPI - Go to Account Settings > API tokens - Create a new token with the appropriate permissions

1. **Add the secret in GitHub**:

- Repository Settings > Secrets and variables > Actions - Create a new secret named`PYPI_API_TOKEN` - Paste your PyPI token

[plantuml, secrets-flow, svg]
----
@startuml
!theme plain

actor Developer as dev
participant "GitHub Repository" as repo
participant "GitHub Actions" as actions
participant PyPI

dev -> repo : Configure PYPI_API_TOKEN secret
repo -> actions : Trigger CD pipeline on release
actions -> actions : Access secret securely
actions -> PyPI : Authenticate with token
PyPI -> PyPI : Validate and publish package

note over actions, PyPI
  Token never exposed in logs
  Automatic rotation possible
end note

@enduml
----

== Complete Deployment Workflow

=== Deployment Sequence

[plantuml, deployment-sequence, svg]
----
@startuml
!theme plain

actor Developer as dev
participant "Local Git" as git
participant "GitHub" as github
participant "GitHub Actions" as actions
participant PyPI
participant "End Users" as users

dev -> git : git tag v1.0.0
dev -> git : git push origin v1.0.0
git -> github : Push tag
dev -> github : Create release from tag
github -> actions : Trigger CD pipeline
actions -> actions : Checkout code
actions -> actions : Setup Python environment
actions -> actions : Install build tools
actions -> actions : Build distributions (sdist + wheel)
actions -> PyPI : Upload to PyPI with token
PyPI -> PyPI : Validate and publish
users -> PyPI : pip install playlist-downloader

note over dev, github
  Release creation can be automated
  or done manually through GitHub UI
end note

@enduml
----

=== Pipeline States

[plantuml, pipeline-states, svg]
----
@startuml
!theme plain

[*] --> Idle

Idle --> CI_Running : Push/PR created
CI_Running --> CI_Success : All checks pass
CI_Running --> CI_Failed : Linting/Tests fail
CI_Success --> Idle : Merge completed
CI_Failed --> Idle : Fix and retry

Idle --> CD_Running : Release published
CD_Running --> CD_Success : Package published
CD_Running --> CD_Failed : Build/Upload error
CD_Success --> Idle : Package available on PyPI
CD_Failed --> Idle : Fix and retry release

note on link #red : Blocks merge
note on link #green : Allows deployment

@enduml
----

== Best Practices and Optimizations

=== 1. Version Management

Use semantic Git tags:

```bash
----
git tag -a v1.2.3 -m "Release version 1.2.3"
git push origin v1.2.3
----

=== 2. Matrix Tests

To test on multiple Python versions:

```yaml
----
strategy:
  matrix:
    python-version: [3.8, 3.9, "3.10", "3.11"]
----

=== 3. Dependency Caching

Speed up builds with caching:

```yaml
----
- name: Cache pip dependencies
  uses: actions/cache@v3
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
----

=== 4. Deployment Environments

Use GitHub environments for controlled deployments:

```yaml
----
jobs:
  deploy:
    environment: production
    runs-on: ubuntu-latest
----

== Use Case and Architecture

=== Use Case Diagram

[plantuml, use-cases, svg]
----
@startuml
!theme plain

left to right direction

actor "Developer" as dev
actor "GitHub Actions" as ga
actor "End User" as user

package "CI/CD System" {
  usecase "Run Linting" as lint
  usecase "Execute Tests" as test
  usecase "Build Package" as build
  usecase "Publish to PyPI" as publish
  usecase "Create Release" as release
}

dev --> lint : Pushes code
dev --> test : Pushes code
dev --> release : Creates release
ga --> build : On release trigger
ga --> publish : After successful build
user --> publish : Downloads package

lint .> test : triggers
test .> build : on success
build .> publish : on success

@enduml
----

=== Deployment Architecture

[plantuml, deployment-architecture, svg]
----
@startuml
!theme plain

cloud "GitHub" {
  [Source Repository]
  [GitHub Actions]
  [Secrets Store]
}

cloud "PyPI" {
  [Package Registry]
  [Distribution Files]
}

node "CI/CD Pipeline" {
  [Linting]
  [Testing]
  [Building]
  [Publishing]
}

[Source Repository] --> [GitHub Actions] : Triggers
[GitHub Actions] --> [Linting]
[Linting] --> [Testing]
[Testing] --> [Building]
[Building] --> [Publishing]
[Publishing] --> [Package Registry] : Uploads
[Secrets Store] --> [Publishing] : Provides token

note as N1
  Secure token-based
  authentication
end note

[Secrets Store] .. N1

@enduml
----

== Monitoring and Debugging

=== Logs and Monitoring

GitHub Actions provides detailed logs for each step. To debug:

1. **Examine the logs**of each step
2. **Enable debug**with`ACTIONS_STEP_DEBUG`
3. **Use artifacts**to save build files

```yaml
----
- name: Upload build artifacts
  uses: actions/upload-artifact@v3
  if: failure()
  with:
    name: build-logs
    path: build/
----

=== Notifications

Add Slack or email notifications:

```yaml
----
- name: Notify on failure
  if: failure()
  uses: 8398a7/action-slack@v3
  with:
    status: ${{ job.status }}
    webhook_url: ${{ secrets.SLACK_WEBHOOK }}
----

== Conclusion

Setting up a robust CI/CD pipeline with GitHub Actions radically transforms the development experience. By automating linting, testing, and deployment, you:

- **Reduce errors**in production - **Accelerate cycles**of development - **Improve confidence**in your releases - **Facilitate collaboration**within the team

This pipeline can be adapted to different types of Python projects by adjusting the linting tools, test frameworks, or deployment destinations.

The initial investment in configuring these workflows is quickly offset by the time saved and the reduction of manual errors during deployments.

== Additional Resources

- https://docs.github.com/en/actions[GitHub Actions Documentation] - https://packaging.python.org/[Python Packaging Guide] - https://docs.pytest.org/[Pytest Documentation] - https://docs.astral.sh/ruff/[Ruff Documentation] - https://twine.readthedocs.io/[Twine Documentation]

✅ Functional pipeline achieved! You now have a simple CI/CD pipeline that allows you to automate your tests and publish your Python package to PyPI directly from GitHub Actions.

However, this pipeline remains intentionally minimalist. It does not yet cover certain aspects essential in a professional context:

Multi-version Python tests,

Automatic security analysis,

Progressive deployment via Test PyPI,

Pipeline monitoring and metrics,

Versioning automation and integration of modern best practices (pyproject.toml).

In the next part, we will move to the next level. You will learn how to transform this basic pipeline into a true industrial deployment chain, robust and secure, ready for production Python projects.