Helps investigate and modify Apache Beam's automated build and test workflows in GitHub Actions. Explains workflow types, failure logs, targeted reruns, local reproduction, and the cloud credentials needed for workflows that access GCP.
中文简介正在准备中。当前显示英语简介。
Guides understanding and working with Apache Beam's CI/CD system using GitHub Actions. Use when debugging CI failures, understanding test workflows, or modifying CI configuration.
安装前可以查看交给 AI 代理的指令内容。
Apache Beam uses GitHub Actions for CI/CD. Workflows are located in .github/workflows/.
beam_PreCommit_*.ymlbeam_PostCommit_*.ymlnightly-master| Workflow | Description |
|---|---|
beam_PreCommit_Java.yml | Java build and tests |
beam_PreCommit_Python.yml | Python tests |
beam_PreCommit_Go.yml | Go tests |
beam_PreCommit_RAT.yml | License header checks |
beam_PreCommit_Spotless.yml | Code formatting |
| Workflow | Description |
|---|---|
beam_PostCommit_Java.yml | Full Java test suite |
beam_PostCommit_Java_ValidatesRunner_*.yml | Runner validation tests |
beam_PostCommit_Java_Examples_*.yml | Example pipeline tests |
| Workflow | Description |
|---|---|
beam_PostCommit_Python.yml | Full Python test suite |
beam_PostCommit_Python_ValidatesRunner_*.yml | Runner validation |
beam_PostCommit_Python_Examples_*.yml | Examples |
| Workflow | Description |
|---|---|
beam_LoadTests_*.yml | Load testing |
beam_PerformanceTests_*.yml | I/O performance |
Use trigger files to run specific workflows.
Most workflows support manual triggering via GitHub UI.
Workflows requiring GCP access use these secrets:
GCP_PROJECT_ID - Project ID (e.g., apache-beam-testing)GCP_REGION - Region (e.g., us-central1)GCP_TESTING_BUCKET - Temp storage bucketGCP_PYTHON_WHEELS_BUCKET - Python wheels bucketGCP_SA_EMAIL - Service account emailGCP_SA_KEY - Base64-encoded service account keyRequired IAM roles:
beam_*.ymlname: Workflow Name
on:
push:
branches: [master]
pull_request:
branches: [master]
schedule:
- cron: '0 0 * * *'
workflow_dispatch:
jobs:
build:
runs-on: [self-hosted, ...]
steps:
- uses: actions/checkout@v4
- name: Run Gradle
run: ./gradlew :task:name
Check workflow file's run commands:
./gradlew :sdks:java:core:test
./gradlew :sdks:python:test
rm -rf ~/.gradle .gradlerm -rf build| Workflow | Purpose |
|---|---|
cut_release_branch.yml | Create release branch |
build_release_candidate.yml | Build RC |
finalize_release.yml | Finalize release |
publish_github_release_notes.yml | Publish notes |
还没有评价。欢迎分享你的使用感受。
无需登录即可发表。欢迎分享使用感受、提问，或说明在哪些环境下无法运行。
还没有评论。来写第一条吧。
Guides adding metadata, extra information attached to data elements, to Apache Beam's WindowedValue. Maps the changes needed across serialization, storage, processing, and runner interfaces so new fields remain available during pipeline execution.
中文简介正在准备中。当前显示英语简介。
Learn Apache Beam's core concepts through Java and Python examples. Understand distributed datasets, processing steps, pipeline execution, and time windows, with guidance on aggregation, late data, error handling, and sharing processing steps across languages.
中文简介正在准备中。当前显示英语简介。
Navigate Apache Beam contributions from finding an issue to submitting a pull request, a proposed code change, and responding to reviews. Covers environment setup, tests, formatting, licensing requirements, and closer evaluation of changes to core components.
中文简介正在准备中。当前显示英语简介。
Build Apache Beam connectors that read and write external data. This guide covers Java implementation, schema support for structured records, access from Python and Go, Managed API integration, and testing.
中文简介正在准备中。当前显示英语简介。
Build and test Apache Beam with Gradle, the system that manages its project tasks. Find relevant configuration files, inspect library dependencies, run formatting checks, publish locally, and investigate build failures.
中文简介正在准备中。当前显示英语简介。
Work with Apache Beam I/O connectors, components that read and write external data. Get guidance on connector usage, unit and integration tests, troubleshooting data connections, and the basic structure of new connectors.
中文简介正在准备中。当前显示英语简介。