Skip to content
cccskills
FreePublished on GitHub

aamas-artifact-evaluation

Use when packaging AAMAS multiagent code, environments, opponent and population sets, random seeds, game definitions, and logs as anonymous supplementary evidence or a public post-acceptance release, even without a separate artifact badge, so that game-theory and MARL reviewers can inspect and re-run the interaction claims.

The English summary is being prepared. Showing the original description.

Run score Not tested

See how to install

Files included (1)

  • SKILL.md3.7 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

AAMAS Artifact Evaluation

Use this for evidence packaging around AAMAS. Because the venue is about interaction, an artifact must make a multiagent claim inspectable: the game, the other agents, and the protocol, not just a single trained model.

Artifact plan

  • Decide what a reviewer needs to believe the interaction claim: game or environment code, opponent/population definitions, the training regime, seeds, payoff logs, proofs, or qualitative episode traces.
  • Keep decision-critical evidence in the main paper or appendix; optional bulk runs can live in the supplementary zip.
  • Anonymize repository history, paths, environment names, license headers, cluster paths, and commit authors for the review version.
  • Include a minimal reproduction map: environment build, dependencies, hardware, commands, expected outputs, per-run wall-clock, seeds, and known nondeterminism (especially in self-play).
  • For a deployed or human-subject setting, give enough provenance for credible reproduction without violating data-use terms.
  • After acceptance, replace anonymous archives with a public, licensed, citable artifact.

What AAMAS evidence reviewers open first

The single fact that shapes packaging: a reviewer will re-run a small game far sooner than they will retrain a large policy, so make the strategic core turnkey before polishing anything.

Claim typeFirst artifact inspectedCommon failure caught
Convergence to an equilibriumThe game definition and the learning-rule codeSolution concept named in the paper but not encoded in the evaluation
Emergent cooperation/defectionThe environment and reward specificationResult depends on an undocumented reward-shaping constant
Beats other agentsThe opponent/population set and match protocolOnly self-play reported; no held-out opponents
Mechanism is truthfulThe payment rule plus a strategic-deviation testNo script that lets an agent try to game the mechanism

Worked vignette: packaging a self-play study

A hypothetical submission claims a learning rule that converges to a correlated equilibrium in a repeated congestion game, shown by self-play.

  • Ship the game as one parameterized generator (number of agents, capacity, payoff scale) rather than constants buried in a notebook, so reviewers can vary the interaction.
  • Record the exact seed sequence and replication count behind every convergence plot; an equilibrium-convergence claim without seeds is unfalsifiable.
  • Emit payoff and regret tables directly from logged results so PDF and artifact numbers cannot drift.
  • Include a strategic-deviation harness: a script that drops in a non-conforming agent and measures whether it profits, because that is exactly what a game-theory reviewer will try.

Calibration anchors

  • Supplement inspection at AAMAS is at reviewer discretion; assume only the README and one entry script get opened, and design the top level accordingly.
  • Supplement size and format caps vary by cycle (25 MB single zip in 2026); verify against the current OpenReview form rather than a past year.

Output format

[Artifact role] anonymous supplement / camera-ready release / public archive
[Contents] <game/env/opponents/seeds/proofs/logs>
[Anonymity risks] <paths/metadata/licenses/URLs>
[Reproduction level] turnkey / scripted / descriptive / weak
[Fixes before upload] <ordered list>

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

Use when packaging AAAI code, data, multimedia appendices, technical appendices, reproducibility evidence, and post-acceptance artifact releases without violating double-blind or immutable-supplement rules.

The English summary is being prepared. Showing the original description.

brycewang-stanford/Awesome-Journal-Skills1.3KUpdated Sep 27, 2026

Use when drafting an AAAI author response (rebuttal) under the single short character-limited author-feedback window, the no-URL rule, no-new-results guidance, AI-generated-review handling, and the AAAI two-phase review process where Phase-2 papers receive one feedback round before SPC/AC discussion.

The English summary is being prepared. Showing the original description.

brycewang-stanford/Awesome-Journal-Skills1.3KUpdated Sep 27, 2026

Use when preparing an accepted AAAI paper for camera-ready source submission to AAAI Press, including proceedings page limits, two-column template compliance, copyright transfer, purchased extra technical pages, deanonymization, registration, oral or poster presentation, and final public artifact release.

The English summary is being prepared. Showing the original description.

brycewang-stanford/Awesome-Journal-Skills1.3KUpdated Sep 27, 2026

Use when designing or auditing AAAI experiments for the broad-AI program committee, including baselines, ablations, statistical significance, robustness, human evaluation, AI-for-Social-Impact and alignment/safety evidence, compute and cost reporting, and reproducibility-checklist alignment for Phase-1 survival.

The English summary is being prepared. Showing the original description.

brycewang-stanford/Awesome-Journal-Skills1.3KUpdated Sep 27, 2026

Use when positioning an AAAI paper's novelty against archival work, contemporaneous arXiv or workshop papers, and AAAI/IJCAI/NeurIPS/ICML/ICLR neighbors across the broad AI scope, while staying inside AAAI's dual-submission and AI-as-source policy constraints and writing a related-work section legible to non-specialist reviewers.

The English summary is being prepared. Showing the original description.

brycewang-stanford/Awesome-Journal-Skills1.3KUpdated Sep 27, 2026

Use when strengthening an AAAI paper's reproducibility checklist (placed after references), experimental traceability, seed and hyperparameter reporting, compute and cost disclosure, dataset access and licensing, code/data ZIP readiness, and the claim-to-evidence map that Phase-1 reviewers use to judge rigor across AAAI's broad AI scope.

The English summary is being prepared. Showing the original description.

brycewang-stanford/Awesome-Journal-Skills1.3KUpdated Sep 27, 2026

See all skills by brycewang-stanford

Report a problem with this skill