Package an agent for submission

Paths. scripts/… and references/… below are relative to the directory holding this SKILL.md. Your working directory is the user's project, not that directory, so prefix them with its absolute path or the command is not found.

The leaderboard evaluates a docker image, not a repository. make_docker.sh assembles one from four trees plus your code:

$CARLA_ROOT/PythonAPI   ->  /workspace/CARLA/PythonAPI
$SCENARIO_RUNNER_ROOT   ->  /workspace/scenario_runner
$LEADERBOARD_ROOT       ->  /workspace/leaderboard
$TEAM_CODE_ROOT         ->  /workspace/team_code

and bakes PYTHONPATH, TEAM_AGENT, CHALLENGE_TRACK_CODENAME and the routes into the image.

Instructions

Progress:
- [ ] Step 1: Check prerequisites (bash scripts/check_env.sh) — the four roots and the egg layout
- [ ] Step 2: Confirm the agent runs locally first (run-leaderboard-evaluation)
- [ ] Step 3: Build the image
- [ ] Step 4: Verify the image imports the stack and loads the agent
- [ ] Step 5: Submit

Step 1: The egg trap

make_docker.sh does this unconditionally:

cp -fr ${CARLA_ROOT}/PythonAPI .lbtmp
mv .lbtmp/PythonAPI/carla/dist/carla*-py2*.egg .lbtmp/PythonAPI/carla/dist/carla-leaderboard-py2.7.egg
mv .lbtmp/PythonAPI/carla/dist/carla*-py3*.egg .lbtmp/PythonAPI/carla/dist/carla-leaderboard-py3x.egg

so it fails if $CARLA_ROOT/PythonAPI/carla/dist/ has no .egg files — which is the case for a pip/wheel-only install. check_env.sh reports this before the build wastes time. Get a CARLA tree that ships eggs (a release tarball or a source build), or add the wheel to the image yourself.

The py2.7 rename is legacy; a modern release ships only a py3 egg, so the py2 mv fails harmlessly and the script continues because it is not run under set -e.

Step 3: Build

source scripts/env.sh

export TEAM_CODE_ROOT=~/team_code
bash scripts/package_agent.sh build                    # -> leaderboard-user
bash scripts/package_agent.sh build --ros noetic       # -> leaderboard-user:ros-noetic

The wrapper checks the roots, warns about the egg layout and about copying a huge TEAM_CODE_ROOT (model weights inflate the image fast), then calls the repo's make_docker.sh. Supported ROS distros are exactly melodic, noetic, foxy — anything else prints the usage and exits.

Set the agent-specific values in the image by editing the block in scripts/Dockerfile.master:

ENV TEAM_AGENT ${TEAM_CODE_ROOT}/my_agent.py
# ENV TEAM_CONFIG ${TEAM_CODE_ROOT}/config.json
ENV CHALLENGE_TRACK_CODENAME SENSORS

Step 4: Verify before submitting

bash scripts/package_agent.sh verify                   # imports + agent class inside the image
bash scripts/package_agent.sh shell                    # interactive poke around

verify runs the image and checks, inside it: carla, agents, srunner and leaderboard all import; TEAM_AGENT exists and defines the class the evaluator will derive from its filename; and the declared track is valid. That is the whole class of failures that otherwise comes back as a rejected submission hours later.

A full local dry run is better still — same image, your own server:

bash scripts/package_agent.sh run --routes-subset 0

Step 5: Submit

Submission is through the leaderboard website (https://leaderboard.carla.org/submit/): push the image to a registry and provide the tag. Read the current instructions there — the mechanics (registry, quotas, allowed track) change between challenge rounds and are not in the repository.

Examples

Example 1: "build the docker image for my agent"

export TEAM_CODE_ROOT=~/team_code, check_env.sh, edit the Dockerfile's TEAM_AGENT, build, then verify.

Example 2: "make_docker.sh fails on the mv of the egg"

Your CARLA is wheel-only. Use a release tarball for CARLA_ROOT, or copy the wheel into the image and install it in the Dockerfile instead of relying on the egg.

Example 3: "my ROS agent needs the bridge"

export CARLA_ROS_BRIDGE_ROOT=~/ros-bridge (cloned with --recurse-submodules -b leaderboard-2.1), then build --ros noetic, which uses Dockerfile.ros and copies the bridge into the image.

Example 4: "the image builds but the submission is rejected"

Almost always sensors or the agent class. verify catches both; [[write-leaderboard-agent]] validate catches the sensor budget specifically.

Troubleshooting

Problem: Error $CARLA_ROOT is empty. Set $CARLA_ROOT as an environment variable first. Cause: make_docker.sh checks all four roots up front (and CARLA_ROS_BRIDGE_ROOT when -r is given). Solution: export them; source scripts/env.sh does the first three.

Problem: mv: cannot stat '.lbtmp/PythonAPI/carla/dist/carla*-py3*.egg' Cause: no egg in the CARLA tree. Solution: see Step 1.

Problem: the image is tens of gigabytes Cause: TEAM_CODE_ROOT is copied wholesale — datasets, checkpoints, .git. Solution: keep only what the agent needs at runtime; check_env.sh reports the directory size before you build.

Problem: python3: can't open file '/workspace/leaderboard/leaderboard/leaderboard_evaluator.py' Cause: LEADERBOARD_ROOT pointed somewhere that is not a leaderboard checkout, so the copy landed wrong. Solution: check_env.sh validates each root by looking for its marker file.

Problem: imports work locally but not in the image Cause: the image's PYTHONPATH uses carla-leaderboard-py3x.egg, a renamed copy of your egg. If your local setup used a wheel, the image is using a different client. Solution: verify, and make the local and image clients the same.

Problem: the ROS image builds but the stack never starts Cause: ROS_DISTRO mismatch with what your launch files expect, or the bridge was cloned without submodules. Solution: re-clone with --recurse-submodules; build with the matching distro.

Outputs

A docker image (leaderboard-user, or leaderboard-user:ros-<distro>) containing CARLA's PythonAPI, the paired scenario_runner and leaderboard, your team code and the environment the evaluator expects. verify reports whether the image can actually import the stack and load the agent.

Dockerfile contents, the environment it bakes in, and what the submission expects are in references/submission.md.