第一部分:为 Python 和 PyPI 设置简单的 CI/CD 流程
Publié le 17 July 2025
Sommaire
介绍
目标 : 引导读者为一个 Python 应用程序构建一个简约但功能完整的 CI/CD 流程,实现自动部署到 PyPI。
开发流程的自动化已经成为现代项目中不可或缺的一部分。一个设计良好的CI/CD流水线不仅能够在开发周期早期检测到回归,还能够完全自动化部署过程。
在本文中,我们将探讨如何为 Python 应用程序使用 GitHub Actions 搭建一个完整的流水线,从持续集成 (CI) 到持续部署 (CD) 发布到 PyPI。
Pipeline架构
我们的流水线由两个不同的工作流组成:
-
CI 流水线在每次 push 和 pull request 时执行
-
持续交付流水线: 仅在 GitHub 发布期间触发
持续集成(CI)管道配置
CI 工作流程结构
CI 工作流程旨在验证每次代码贡献。以下是其完整配置:
name: CI/CD Pipeline
on:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install ruff pytest pytest-mock
- name: Run Linting (Ruff)
run: ruff check .
- name: Run Tests (Pytest)
run: pytest
CI步骤分析
1. 触发器 (on)
----
----
on:
push:
branches:
- main
pull_request:
branches:
- main
----
流水线在以下情况下触发: - 每次推送到该分支`main` - � 每个 pull request 向`main`
这种方法保证主代码保持稳定,且在集成之前验证所有贡献。
==== 2. 运行环境
----
runs-on: ubuntu-latest
Ubuntu Latest 在性能、成本和兼容性之间提供了一个很好的折中方案,适用于大多数 Python 项目。
==== 3. 代码检出
```yaml
----
----
- name: Checkout code
uses: actions/checkout@v4
----
行动`checkout@v4`检索存储库的源代码。v4 版本带来了性能和安全性的改进。
==== 4. Python配置
----
- name: Set up Python uses: actions/setup-python@v5 with: python-version: '3.x'
使用`'3.x'`允许自动使用 Python 3 的最新稳定版本,简化了维护。
==== 5. 安装依赖
```yaml
----
----
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install ruff pytest pytest-mock
----
这一步: - 将 pip 更新到最新版本 - 安装项目依赖 - 添加开发工具(linting 和测试)
==== 6. 使用 Ruff 进行代码检查
----
- name: Run Linting (Ruff) run: ruff check .
**Ruff**它是一个使用Rust编写的超快速Python linter。它结合了多个工具(Flake8、Black、isort)的功能,成为一个高性能的单一工具。
==== 7. 测试执行
```yaml
----
----
- name: Run Tests (Pytest)
run: pytest
----
Pytest 执行整个测试套件,确保修改不会引入回归。
== 部署流水线配置 (CD)
=== CD 工作流的结构
CD 工作流仅在 GitHub 发布时触发,并自动化将包发布到 PyPI:
[source,yaml]
----
name: Publish to PyPI
on:
release:
types:
- published
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install setuptools wheel twine
- name: Build and publish
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
run: |
python setup.py sdist bdist_wheel
twine upload dist/*
----
=== CD步骤分析
==== 1. 触发器释放
----
on: release: types: - published
CD 管道仅在发布 GitHub 版本时触发。此方法确保对部署进行精确控制。
==== 2. 安装构建工具
```yaml
----
----
pip install setuptools wheel twine
----
- **setuptools**Python打包工具 - **轮子**现代的 Python 发布格式 - **绳子**用于安全上传至 PyPI 的工具
==== 3. 身份验证配置
----
env: TWINE_USERNAME: __token__ TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
身份验证采用存储为 GitHub 密钥的 PyPI API 令牌,比传统凭据更安全。
==== 4. 构建和发布
```yaml
----
----
run: |
python setup.py sdist bdist_wheel
twine upload dist/*
----
- `sdist`: 创建源分发 - `bdist_wheel`: 创建一个 wheel (二进制分发) - `twine upload`: 在 PyPI 上发布分发包
== Python包配置
=== setup.py 的结构
为了让管道正常运行,您的项目必须包含一个文件`setup.py` :
[source,python]
----
from setuptools import setup, find_packages
with open("README.adoc", "r", encoding="utf-8") as fh:
long_description = fh.read()
setup(
name="playlist-downloader",
version="1.0.0",
author="Votre Nom",
author_email="[email protected]",
description="CLI tool for managing YouTube playlists",
long_description=long_description,
long_description_content_type="text/plain",
url="https://github.com/cheroliv/playlist-downloader",
packages=find_packages(),
classifiers=[
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
],
python_requires=">=3.8",
install_requires=[
"typer>=0.9.0",
"yt-dlp>=2023.1.6",
"google-api-python-client>=2.70.0",
"google-auth-oauthlib>=0.7.1",
"pymonad>=2.4.0",
"pyyaml>=6.0",
],
entry_points={
"console_scripts": [
"playlist-downloader=cli:app",
],
},
)
----
=== setup.py 的关键点
1. **元数据**名称, 版本, 作者, 描述
2. **依赖**: 所需软件包列表
3. **入口点**:暴露的 CLI 命令
4. **分类器**: 用于 PyPI 的元数据
== 使用 GitHub Secrets 进行安全防护
=== PyPI 令牌配置
1. **在 PyPI 上创建 API 令牌** :
- 登录到 PyPI - 进入 账户设置 > API 令牌 - 创建一个具有适当权限的新令牌
1. **在 GitHub 中添加密钥**:
- 仓库设置 > 密钥和变量 > 操作 - 创建一个新的已命名密钥`PYPI_API_TOKEN` - 粘贴您的 PyPI 令牌
[plantuml, secrets-flow, svg]
----
@startuml
!theme plain
actor Developer as dev
participant "GitHub 仓库" as repo
participant "GitHub Actions" as actions
participant PyPI
dev -> repo : Configure PYPI_API_TOKEN secret
repo -> actions : Trigger CD pipeline on release
actions -> actions : Access secret securely
actions -> PyPI : Authenticate with token
PyPI -> PyPI : Validate and publish package
note over actions, PyPI
Token never exposed in logs
Automatic rotation possible
end note
@enduml
----
== 完整的部署工作流
=== 部署顺序
[plantuml, deployment-sequence, svg]
----
@startuml
!theme plain
actor Developer as dev
participant "本地 Git" as git
participant "GitHub" as github
participant "GitHub Actions" as actions
participant PyPI
participant "最终用户" as users
dev -> git : git tag v1.0.0
dev -> git : git push origin v1.0.0
git -> github : Push tag
dev -> github : Create release from tag
github -> actions : Trigger CD pipeline
actions -> actions : Checkout code
actions -> actions : Setup Python environment
actions -> actions : Install build tools
actions -> actions : Build distributions (sdist + wheel)
actions -> PyPI : Upload to PyPI with token
PyPI -> PyPI : Validate and publish
users -> PyPI : pip install playlist-downloader
note over dev, github
Release creation can be automated
or done manually through GitHub UI
end note
@enduml
----
=== 管道状态
[plantuml, pipeline-states, svg]
----
@startuml
!theme plain
[*] --> Idle
Idle --> CI_Running : Push/PR created
CI_Running --> CI_Success : All checks pass
CI_Running --> CI_Failed : Linting/Tests fail
CI_Success --> Idle : Merge completed
CI_Failed --> Idle : Fix and retry
Idle --> CD_Running : Release published
CD_Running --> CD_Success : Package published
CD_Running --> CD_Failed : Build/Upload error
CD_Success --> Idle : Package available on PyPI
CD_Failed --> Idle : Fix and retry release
note on link #red : Blocks merge
note on link #green : Allows deployment
@enduml
----
== 良好实践与优化
=== 1. 版本管理
使用语义化的 Git 标签:
----
git tag -a v1.2.3 -m "Release version 1.2.3" git push origin v1.2.3
=== 2. 矩阵测试
为了在多个 Python 版本上进行测试:
```yaml
----
----
strategy:
matrix:
python-version: [3.8, 3.9, "3.10", "3.11"]
----
=== 3. 依赖缓存
用缓存加速构建:
----
- name: Cache pip dependencies uses: actions/cache@v3 with: path: ~/.cache/pip key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
=== 4. 部署环境
使用 GitHub 环境进行受控部署 :
```yaml
----
----
jobs:
deploy:
environment: production
runs-on: ubuntu-latest
----
== 用例和架构
=== 用例图
[plantuml, use-cases, svg]
----
@startuml
!theme plain
left to right direction
actor "开发者" as dev
actor "GitHub Actions" as ga
actor "终端用户" as user
package "CI/CD 系统" {
usecase "运行代码检查" as lint
usecase "�执行测试" as test
usecase "构建包" as build
usecase "发布到 PyPI" as publish
usecase "创建版本" as release
}
dev --> lint : Pushes code
dev --> test : Pushes code
dev --> release : Creates release
ga --> build : On release trigger
ga --> publish : After successful build
user --> publish : Downloads package
lint .> test : triggers
test .> build : on success
build .> publish : on success
@enduml
----
=== 部署架构
[plantuml, deployment-architecture, svg]
----
@startuml
!theme plain
cloud "GitHub" {
[Source Repository]
[GitHub Actions]
[Secrets Store]
}
cloud "PyPI" {
[Package Registry]
[Distribution Files]
}
node "CI/CD 流水线" {
[Linting]
[Testing]
[Building]
[Publishing]
}
[Source Repository] --> [GitHub Actions] : Triggers
[GitHub Actions] --> [Linting]
[Linting] --> [Testing]
[Testing] --> [Building]
[Building] --> [Publishing]
[Publishing] --> [Package Registry] : Uploads
[Secrets Store] --> [Publishing] : Provides token
note as N1
Secure token-based
authentication
end note
[Secrets Store] .. N1
@enduml
----
== �监控和调试
=== 日志和监控
GitHub Actions 提供每个步骤的详细日志。为了调试:
1. **�检查日志**每个步骤的
2. **启用调试**与`ACTIONS_STEP_DEBUG`
3. **使用 artifacts**为了保存构建文件
----
- name: Upload build artifacts uses: actions/upload-artifact@v3 if: failure() with: name: build-logs path: build/
=== 通知
�添加 Slack 或电子邮件通知:
```yaml
----
----
- name: Notify on failure
if: failure()
uses: 8398a7/action-slack@v3
with:
status: ${{ job.status }}
webhook_url: ${{ secrets.SLACK_WEBHOOK }}
----
== 结论
使用 GitHub Actions 建立一个强大的 CI/CD 流水线彻底改变了开发体验。通过自动化 linting、测试和部署,您:
- **减少错误**正在生产 - **加速循环**的发展 - **提升信任**在您的发布版 - **�促进协作**作为一个团队
此管道可以通过调整 lint 工具、测试框架或部署目标来适应不同类型的 Python 项目。
对这些工作流程进行初始配置的投资能够很快通过节省时间和减少部署过程中的手动错误来收回成本。
== 补充资源
- https://docs.github.com/en/actions[GitHub Actions 文档] - https://packaging.python.org/[Python 打包指南] - https://docs.pytest.org/[Pytest 文档] - https://docs.astral.sh/ruff/[文档 Ruff] - https://twine.readthedocs.io/[文档 Twine]
✅ 功能管道已达成! 您现在拥有一个简单的 CI/CD 流水线,可以让您自动化测试并直接从 GitHub Actions 将 Python 包发布到 PyPI。
然而,此流水线仍然保持故意的极简主义。它尚未涵盖在专业环境中必不可少的一些方面:
多版本 Python 测试,
自动安全分析,
通过 Test PyPI 进行渐进式部署,
流水线的监控和指标
自动化版本控制并集成现代最佳实践(pyproject.toml)。
在下一部分,我们将进入更高一步。您将学习如何将这个基本的管道转换为真正的工业级部署链——健壮且安全,并准备好用于生产环境的 Python 项目。
----