Skip to content

SCML

Supply-chain negotiation arena based on the ANAC Supply Chain Management League OneShot track.

Overview

SCML simulates a supply chain in which autonomous factory-manager agents negotiate contracts to buy and sell goods. The CodeClash arena uses the SCML2024 OneShot world because it focuses on negotiation and profit without requiring long-term production scheduling.

Each CodeClash player edits a restricted SCML decision policy. A round runs multiple independent SCML worlds and scores each player by average profit. The trusted runtime owns the SCML agent object, world state, and validation; submitted code only receives plain observations and returns negotiation intents.

Resources

Implementation

codeclash.arenas.scml.scml.SCMLOneShotArena 

SCMLOneShotArena(config: dict, *, tournament_id: str, local_output_dir: Path, keep_containers: bool = False)

Bases: CodeArena

Source code in codeclash/arenas/arena.py 
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
def __init__(self, config: dict, *, tournament_id: str, local_output_dir: Path, keep_containers: bool = False):
    """The CodeArena class is responsible for running games, i.e., taking a list of code
    from different agents/players and running them against each other.
    It also provides the environments for the game and agents to run in.

    The central method is `run_round`, which takes a list of agents and returns the winner of the round.

    At the end of the the tournament, run the `end` method to clean up the game and agents and write the metadata.

    Args:
        config: The overall config for the tournament.
        tournament_id: The id of the tournament.
        local_output_dir: The host/local directory to write logs to.
        keep_containers: Do not remove containers after games/agent finish.
    """
    self.url_gh: str = f"git@github.com:{GH_ORG}/{self.name}.git"
    self.artifacts: list[Path] = []
    """Artifact objects that we might want to clean up after the game."""
    self.config: dict = config
    self._keep_containers: bool = keep_containers
    self._metadata: dict = {
        "name": self.name,
        "config": self.config["game"],
        "game_id": tournament_id,
        "created_timestamp": int(time.time()),
    }
    self.log_env: Path = DIR_LOGS
    self.log_local: Path = local_output_dir
    self.logger = get_logger(self.name, log_path=self.log_local / "game.log", emoji="🏓")
    self.environment: DockerEnvironment = self.get_environment()
    """The running docker environment for executing the game"""

name class-attribute instance-attribute 

name: str = 'SCML'

submission class-attribute instance-attribute 

submission: str = 'scml_agent.py'

description class-attribute instance-attribute 

description: str = 'SCML OneShot is a supply-chain negotiation simulator based on the ANAC Supply Chain Management League.\n\nYour bot is a Python file named `scml_agent.py` that defines a function named `decide`.\nThe trusted runtime owns the SCML agent object and passes plain decision observations to your code:\n\n    def decide(observation):\n        return {}\n\nEach round runs several two-process SCML2024 OneShot worlds. Your policy controls trusted SCML\nwrapper agents that negotiate with the other submitted policies to buy and sell goods in a simulated\nsupply chain. The objective is to maximize profit. The arena score is your average SCML score across\nall worlds in the round.\n'

default_args class-attribute instance-attribute 

default_args: dict = {'sims_per_round': 3, 'n_steps': 10, 'n_lines': 2, 'decision_timeout': 3.0, 'max_policy_errors': 8, 'validation_timeout': 10, 'timeout': 180}

validate_code 

validate_code(agent: Player) -> tuple[bool, str | None]
Source code in codeclash/arenas/scml/scml.py 
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
def validate_code(self, agent: Player) -> tuple[bool, str | None]:
    quoted_submission = shlex.quote(self.submission)
    file_check = agent.environment.execute(f"test -f {quoted_submission} && echo exists")
    if "exists" not in file_check["output"]:
        return False, f"Submission file `{self.submission}` not found in the workspace root"

    content = agent.environment.execute(f"cat {quoted_submission}")["output"]
    if not content.strip():
        return False, f"`{self.submission}` is empty"

    syntax_check = agent.environment.execute(f"python -m py_compile {quoted_submission}")
    if syntax_check["returncode"] != 0:
        return False, f"Python syntax error in `{self.submission}`:\n{syntax_check['output']}"

    validation_timeout = int(self._game_arg("validation_timeout"))
    try:
        import_check = agent.environment.execute(
            "python - <<'PY'\n"
            "import importlib.util\n"
            f"spec = importlib.util.spec_from_file_location('submission_agent', {self.submission!r})\n"
            "module = importlib.util.module_from_spec(spec)\n"
            "spec.loader.exec_module(module)\n"
            "assert hasattr(module, 'decide'), 'decide function not found'\n"
            "assert callable(module.decide), 'decide must be callable'\n"
            "result = module.decide({'event': 'validate', 'awi': {}, 'state': {}, 'nmi': {}})\n"
            "assert result is None or isinstance(result, dict), 'decide must return a dictionary or None'\n"
            "PY",
            timeout=validation_timeout,
        )
    except subprocess.TimeoutExpired:
        return False, f"`decide` validation exceeded {validation_timeout}s timeout"
    if import_check["returncode"] != 0:
        return False, f"Could not import or call `decide` from `{self.submission}`:\n{import_check['output']}"

    return True, None

execute_round 

execute_round(agents: list[Player]) -> None
Source code in codeclash/arenas/scml/scml.py 
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
def execute_round(self, agents: list[Player]) -> None:
    agent_args = []
    for agent in agents:
        agent_args.extend(["--agent", f"{agent.name}=/{agent.name}/{self.submission}"])

    cmd = [
        "python",
        "run_scml.py",
        "--sims",
        str(self._game_arg("sims_per_round")),
        "--steps",
        str(self._game_arg("n_steps")),
        "--lines",
        str(self._game_arg("n_lines")),
        "--decision-timeout",
        str(self._game_arg("decision_timeout")),
        "--max-policy-errors",
        str(self._game_arg("max_policy_errors")),
        "--output",
        str(self.log_env / RESULTS_JSON),
        *agent_args,
    ]
    full_cmd = " ".join(shlex.quote(part) for part in cmd)
    self.logger.info(f"Running game: {full_cmd}")
    try:
        response = self.environment.execute(full_cmd, timeout=int(self._game_arg("timeout")))
    except subprocess.TimeoutExpired as exc:
        raise RuntimeError("SCML round timed out") from exc
    assert_zero_exit_code(response, logger=self.logger)

get_results 

get_results(agents: list[Player], round_num: int, stats: RoundStats)
Source code in codeclash/arenas/scml/scml.py 
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
def get_results(self, agents: list[Player], round_num: int, stats: RoundStats):
    result_file = self.log_round(round_num) / RESULTS_JSON
    if not result_file.exists():
        self.logger.error(f"Missing result file: {result_file}")
        stats.winner = RESULT_TIE
        for agent in agents:
            stats.scores[agent.name] = CRASH_SCORE
            stats.player_stats[agent.name].score = CRASH_SCORE
            stats.details.append(
                json.dumps(
                    {
                        "player": agent.name,
                        "score": CRASH_SCORE,
                        "status": "error",
                        "error": f"missing SCML result file: {result_file}",
                    },
                    sort_keys=True,
                )
            )
        return

    with open(result_file) as f:
        result = json.load(f)

    scores = {agent.name: 0.0 for agent in agents}
    for player, score in result.get("average_scores", {}).items():
        if player in scores:
            scores[player] = float(score)

    stats.scores = scores
    stats.details = result.get("details", [])
    for player, score in scores.items():
        stats.player_stats[player].score = score

    if not scores:
        stats.winner = RESULT_TIE
        return

    top_score = max(scores.values())
    winners = [player for player, score in scores.items() if score == top_score]
    stats.winner = winners[0] if len(winners) == 1 else RESULT_TIE

Agent Interface

Your bot must be a Python file named scml_agent.py that defines decide(observation).

Return {} or None to use the trusted greedy fallback. A valid starting point is:

def decide(observation):
    return {}

For proposal events, return {"offer": [quantity, time, unit_price]}. The runtime validates that the offer is inside SCML's current issue ranges before sending it to the simulator. For response events, return {"response": "accept"}, {"response": "reject"}, or {"response": "end"}. Invalid decisions fall back to the trusted greedy policy and are recorded in round details.

Configuration Example

tournament:
  rounds: 1
game:
  name: SCML
  sims_per_round: 2
  n_steps: 5
  n_lines: 2
  decision_timeout: 3.0
  max_policy_errors: 8
  validation_timeout: 10
  timeout: 240
players:
  - agent: dummy
    name: alpha
  - agent: dummy
    name: beta

Scoring

The arena runs sims_per_round independent SCML2024 OneShot worlds. Each world has two supply-chain process levels; every CodeClash player controls one trusted SCML wrapper agent at each level so the submitted policies participate in actual buy/sell negotiations. The final per-world player score is the mean SCML score across that player's controlled agents, and the final CodeClash score is the average across worlds.

The runner rotates player ordering across simulations to reduce positional bias from factory assignment.

Smoke Test

From the repository root, run the dummy-player example:

uv run codeclash run configs/examples/SCML__dummy__r1__s2.yaml -o /tmp/codeclash-scml-smoke

Use a fresh -o directory when rerunning the smoke check.

Expected shape:

  • the command exits with status 0;
  • both players pass submission validation;
  • stdout includes In round 0, the winner is ... and In round 1, the winner is ...;
  • each round summary contains floating-point average scores for alpha and beta;
  • per-simulation details include decisions, policy_errors, invalid_decisions, disabled_policies, and policy_error_samples;
  • the output directory contains metadata.json, game.log, tournament.log, and rounds/round_0.tar.gz / rounds/round_1.tar.gz.

A representative metadata.json round contains a scores object with one floating-point SCML profit score per player:

"scores": {
  "alpha": 0.6536304953204003,
  "beta": 0.5384855419684607
}

Exact values can change with simulation order and configuration; the smoke check is meant to verify the Docker/runtime adapter path, player-name mapping, and score/log artifact shape.

The exact tournament directory name includes a timestamp, so inspect the metadata with:

find /tmp/codeclash-scml-smoke -maxdepth 3 -name metadata.json -print
bug_report Something broken/unclear?

Open an issue on GitHub!
help Open-ended discussions

Join our Slack!
Our projects
SWE-bench SWE-bench SWE-agent SWE-agent SWE-smith SWE-smith SWE-rex SWE-ReX sb-cli sb-cli Slack Join our Slack