Skip to content

SCML

Supply-chain negotiation arena based on the ANAC Supply Chain Management League OneShot track.

Overview

SCML simulates a supply chain in which autonomous factory-manager agents negotiate contracts to buy and sell goods. The CodeClash arena uses the SCML2024 OneShot world because it focuses on negotiation and profit without requiring long-term production scheduling.

Each CodeClash player edits an SCML OneShot agent. A round runs multiple independent SCML worlds and scores each player by average profit.

Resources

Implementation

codeclash.arenas.scml.scml.SCMLOneShotArena 

SCMLOneShotArena(config: dict, *, tournament_id: str, local_output_dir: Path, keep_containers: bool = False)

Bases: CodeArena

Source code in codeclash/arenas/arena.py 
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
def __init__(self, config: dict, *, tournament_id: str, local_output_dir: Path, keep_containers: bool = False):
    """The CodeArena class is responsible for running games, i.e., taking a list of code
    from different agents/players and running them against each other.
    It also provides the environments for the game and agents to run in.

    The central method is `run_round`, which takes a list of agents and returns the winner of the round.

    At the end of the the tournament, run the `end` method to clean up the game and agents and write the metadata.

    Args:
        config: The overall config for the tournament.
        tournament_id: The id of the tournament.
        local_output_dir: The host/local directory to write logs to.
        keep_containers: Do not remove containers after games/agent finish.
    """
    self.url_gh: str = f"git@github.com:{GH_ORG}/{self.name}.git"
    self.artifacts: list[Path] = []
    """Artifact objects that we might want to clean up after the game."""
    self.config: dict = config
    self._keep_containers: bool = keep_containers
    self._metadata: dict = {
        "name": self.name,
        "config": self.config["game"],
        "game_id": tournament_id,
        "created_timestamp": int(time.time()),
    }
    self.log_env: Path = DIR_LOGS
    self.log_local: Path = local_output_dir
    self.logger = get_logger(self.name, log_path=self.log_local / "game.log", emoji="🏓")
    self.environment: DockerEnvironment = self.get_environment()
    """The running docker environment for executing the game"""

name class-attribute instance-attribute 

name: str = 'SCML'

submission class-attribute instance-attribute 

submission: str = 'scml_agent.py'

description class-attribute instance-attribute 

description: str = 'SCML OneShot is a supply-chain negotiation simulator based on the ANAC Supply Chain Management League.\n\nYour bot is a Python file named `scml_agent.py` that defines a class named `MyAgent`.\n`MyAgent` should inherit from an SCML OneShot agent class, for example:\n\n    from scml.oneshot.agents import GreedySyncAgent\n\n    class MyAgent(GreedySyncAgent):\n        ...\n\nEach round runs several SCML2024 OneShot worlds. Your agent negotiates with the other submitted\nagents to buy or sell goods in a simulated supply chain. The objective is to maximize profit. The\narena score is your average SCML score across all worlds in the round.\n'

default_args class-attribute instance-attribute 

default_args: dict = {'sims_per_round': 3, 'n_steps': 10, 'n_lines': 2, 'timeout': 180}

validate_code 

validate_code(agent: Player) -> tuple[bool, str | None]
Source code in codeclash/arenas/scml/scml.py 
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
def validate_code(self, agent: Player) -> tuple[bool, str | None]:
    quoted_submission = shlex.quote(self.submission)
    file_check = agent.environment.execute(f"test -f {quoted_submission} && echo exists")
    if "exists" not in file_check["output"]:
        return False, f"Submission file `{self.submission}` not found in the workspace root"

    content = agent.environment.execute(f"cat {quoted_submission}")["output"]
    if not content.strip():
        return False, f"`{self.submission}` is empty"

    syntax_check = agent.environment.execute(f"python -m py_compile {quoted_submission}")
    if syntax_check["returncode"] != 0:
        return False, f"Python syntax error in `{self.submission}`:\n{syntax_check['output']}"

    import_check = agent.environment.execute(
        "python - <<'PY'\n"
        "import importlib.util\n"
        f"spec = importlib.util.spec_from_file_location('submission_agent', {self.submission!r})\n"
        "module = importlib.util.module_from_spec(spec)\n"
        "spec.loader.exec_module(module)\n"
        "assert hasattr(module, 'MyAgent'), 'MyAgent class not found'\n"
        "from scml.oneshot.agent import OneShotAgent\n"
        "assert issubclass(module.MyAgent, OneShotAgent), 'MyAgent must inherit from an SCML OneShotAgent class'\n"
        "PY"
    )
    if import_check["returncode"] != 0:
        return False, f"Could not import `MyAgent` from `{self.submission}`:\n{import_check['output']}"

    return True, None

execute_round 

execute_round(agents: list[Player]) -> None
Source code in codeclash/arenas/scml/scml.py 
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
def execute_round(self, agents: list[Player]) -> None:
    agent_args = []
    for agent in agents:
        agent_args.extend(["--agent", f"{agent.name}=/{agent.name}/{self.submission}"])

    cmd = [
        "python",
        "run_scml.py",
        "--sims",
        str(self._game_arg("sims_per_round")),
        "--steps",
        str(self._game_arg("n_steps")),
        "--lines",
        str(self._game_arg("n_lines")),
        "--output",
        str(self.log_env / RESULTS_JSON),
        *agent_args,
    ]
    full_cmd = " ".join(shlex.quote(part) for part in cmd)
    self.logger.info(f"Running game: {full_cmd}")
    try:
        response = self.environment.execute(full_cmd, timeout=int(self._game_arg("timeout")))
    except subprocess.TimeoutExpired as exc:
        raise RuntimeError("SCML round timed out") from exc
    assert_zero_exit_code(response, logger=self.logger)

get_results 

get_results(agents: list[Player], round_num: int, stats: RoundStats)
Source code in codeclash/arenas/scml/scml.py 
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
def get_results(self, agents: list[Player], round_num: int, stats: RoundStats):
    result_file = self.log_round(round_num) / RESULTS_JSON
    if not result_file.exists():
        self.logger.error(f"Missing result file: {result_file}")
        stats.winner = RESULT_TIE
        for agent in agents:
            stats.scores[agent.name] = 0.0
            stats.player_stats[agent.name].score = 0.0
        return

    with open(result_file) as f:
        result = json.load(f)

    scores = {agent.name: 0.0 for agent in agents}
    for player, score in result.get("average_scores", {}).items():
        if player in scores:
            scores[player] = float(score)

    stats.scores = scores
    stats.details = result.get("details", [])
    for player, score in scores.items():
        stats.player_stats[player].score = score

    if not scores:
        stats.winner = RESULT_TIE
        return

    top_score = max(scores.values())
    winners = [player for player, score in scores.items() if score == top_score]
    stats.winner = winners[0] if len(winners) == 1 else RESULT_TIE

Agent Interface

Your bot must be a Python file named scml_agent.py that defines MyAgent.

MyAgent must inherit from an SCML OneShot agent class. A valid starting point is:

from scml.oneshot.agents import GreedySyncAgent


class MyAgent(GreedySyncAgent):
    pass

Agents can use the normal SCML OneShot APIs exposed by the upstream scml package. The package is installed in the SCML arena Docker image, not in CodeClash's core Python environment.

Configuration Example

tournament:
  rounds: 1
game:
  name: SCML
  sims_per_round: 2
  n_steps: 5
  n_lines: 2
players:
  - agent: dummy
    name: alpha
  - agent: dummy
    name: beta

Scoring

The arena runs sims_per_round independent SCML2024 OneShot worlds. For each world, it maps SCML agent scores back to CodeClash player names. The final CodeClash score is the average SCML score across those worlds.

The runner rotates player ordering across simulations to reduce positional bias from factory assignment.

bug_report Something broken/unclear?

Open an issue on GitHub!
help Open-ended discussions

Join our Slack!
Our projects
SWE-bench SWE-bench SWE-agent SWE-agent SWE-smith SWE-smith SWE-rex SWE-ReX sb-cli sb-cli Slack Join our Slack