Unit 7: AI vs AI — Train a SoccerTwos Team (MA-POCA + Self-Play)¶
The environment 🎮¶
- SoccerTwos — a 2v2 soccer environment made by the Unity MLAgents Team
- Your team of 2 agents must score goals while defending your own net
The library used 📚¶
Before you start ✅¶
Session options > Accelerator > None(CPU is fine — this env doesn't use GPU)- Internet must be enabled (for cloning the repo and pushing to Hub)
- Add your HuggingFace write token as a Kaggle Secret named
HF_TOKEN- Kaggle sidebar → Add-ons → Secrets → Add New Secret
- Get your token at: https://huggingface.co/settings/tokens
📄 Course page: https://huggingface.co/learn/deep-rl-course/unit7/hands-on
🏆 Leaderboard: https://huggingface.co/spaces/huggingface-projects/AIvsAI-SoccerTwos
🎮 Watch your agent: https://huggingface.co/spaces/unity/ML-Agents-SoccerTwos
Install ML-Agents and the dependencies 🔽¶
We install ML-Agents in a Python 3.10 virtual environment to avoid compatibility issues with Kaggle's default Python 3.12.
In [1]:
%%capture
# Cell 1: Install Python 3.10
!add-apt-repository ppa:deadsnakes/ppa -y
!apt-get update -q
!apt-get install -y python3.10 python3.10-venv python3.10-distutils -q
# Cell 2: Create venv and install with full dependency resolution
!python3.10 -m venv /kaggle/working/mlagents-env
!/kaggle/working/mlagents-env/bin/pip install --upgrade pip -q
!/kaggle/working/mlagents-env/bin/pip install \
"mlagents==1.1.0" \
"mlagents-envs==1.1.0" \
"numpy==1.23.5" \
"onnx==1.15.0" \
"grpcio==1.48.2" \
"protobuf==3.19.6" \
torch --index-url https://download.pytorch.org/whl/cpu
In [2]:
# Cell 3: Verify
!/kaggle/working/mlagents-env/bin/mlagents-learn --help | head -50
usage: mlagents-learn [-h] [--env ENV_PATH] [--resume] [--deterministic]
[--force] [--run-id RUN_ID] [--initialize-from RUN_ID]
[--seed SEED] [--inference] [--base-port BASE_PORT]
[--num-envs NUM_ENVS] [--num-areas NUM_AREAS] [--debug]
[--env-args ...]
[--max-lifetime-restarts MAX_LIFETIME_RESTARTS]
[--restarts-rate-limit-n RESTARTS_RATE_LIMIT_N]
[--restarts-rate-limit-period-s RESTARTS_RATE_LIMIT_PERIOD_S]
[--torch] [--tensorflow] [--results-dir RESULTS_DIR]
[--timeout-wait TIMEOUT_WAIT] [--width WIDTH]
[--height HEIGHT] [--quality-level QUALITY_LEVEL]
[--time-scale TIME_SCALE]
[--target-frame-rate TARGET_FRAME_RATE]
[--capture-frame-rate CAPTURE_FRAME_RATE]
[--no-graphics] [--no-graphics-monitor]
[--torch-device DEVICE]
[trainer_config_path]
positional arguments:
trainer_config_path
options:
-h, --help show this help message and exit
--env ENV_PATH Path to the Unity executable to train (default: None)
--resume Whether to resume training from a checkpoint. Specify
a --run-id to use this option. If set, the training
code loads an already trained model to initialize the
neural network before resuming training. This option
is only valid when the models exist, and have the same
behavior names as the current agents in your scene.
(default: False)
--deterministic Whether to select actions deterministically in policy.
`dist.mean` for continuous action space, and
`dist.argmax` for deterministic action space (default:
False)
--force Whether to force-overwrite this run-id's existing
summary and model data. (Without this flag, attempting
to train a model with a run-id that has been used
before will throw an error. (default: False)
--run-id RUN_ID The identifier for the training run. This identifier
is used to name the subdirectories in which the
trained model and summary statistics are saved as well
as the saved model itself. If you use TensorBoard to
view the training statistics, always set a unique run-
id for each training run. (The statistics for all runs
with the same id are combined as if they were produced
by a the same session.) (default: ppo)
--initialize-from RUN_ID
Specify a previously saved run ID from which to
initialize the model from. This can be used, for
In [ ]:
# # 1. Install Python 3.10
# !add-apt-repository ppa:deadsnakes/ppa -y -q
# !apt-get update -q
# !apt-get install -y python3.10 python3.10-venv python3.10-distutils -q
# # 2. Create venv
# !python3.10 -m venv /kaggle/working/mlagents-env
# # 3. Install everything in one shot with pinned versions
# !/kaggle/working/mlagents-env/bin/pip install -q \
# mlagents==1.1.0 \
# mlagents-envs==1.1.0 \
# numpy==1.23.5 \
# onnx==1.15.0 \
# grpcio==1.48.2 \
# protobuf==3.19.6 \
# cattrs==1.1.2 \
# attrs \
# six \
# typing_extensions \
# cloudpickle \
# gym \
# h5py \
# Pillow \
# pyyaml \
# absl-py \
# tensorboard \
# werkzeug \
# markdown \
# --no-deps
# # 4. Install torch CPU-only (small, no deps)
# # !/kaggle/working/mlagents-env/bin/pip install torch --index-url https://download.pytorch.org/whl/cpu -q --no-deps
# # 5. Verify
# !/kaggle/working/mlagents-env/bin/mlagents-learn --help | head -3
Download and move the SoccerTwos executable 🔽¶
We download the Linux SoccerTwos environment binary from the course's Google Drive and place it in ./training-envs-executables/SoccerTwos/.
The executable contains 8 parallel copies of the environment for faster training.
In [3]:
!mkdir -p ./training-envs-executables/SoccerTwos
In [4]:
%%capture
# Install other required tools
!pip install huggingface_hub gdown -q
# Install virtual display (required for Unity headless on Linux)
!apt-get install -y xvfb -q
# Install git-lfs (required for pushing to HuggingFace Hub)
!apt-get install -y git-lfs -q
!git lfs install
print('✅ Extra dependencies installed!')
In [5]:
# Download Linux SoccerTwos executable from course Google Drive
!gdown "1KuqBKYiXiIcU4kNMqEzhgypuFP5_45CL" -O ./training-envs-executables/SoccerTwos.zip
Downloading... From (original): https://drive.google.com/uc?id=1KuqBKYiXiIcU4kNMqEzhgypuFP5_45CL From (redirected): https://drive.google.com/uc?id=1KuqBKYiXiIcU4kNMqEzhgypuFP5_45CL&confirm=t&uuid=d489bdfc-25df-462b-b168-a6ecb1f0228c To: /kaggle/working/training-envs-executables/SoccerTwos.zip 100%|██████████████████████████████████████| 37.0M/37.0M [00:00<00:00, 76.9MB/s]
In [9]:
%%capture
!unzip -d ./training-envs-executables/SoccerTwos/ ./training-envs-executables/SoccerTwos.zip
In [10]:
# Make sure the executable is accessible
!chmod -R 755 ./training-envs-executables/SoccerTwos/
In [11]:
# Confirm the executable is present
!ls -la ./training-envs-executables/SoccerTwos/
total 33812 drwxr-xr-x 4 root root 4096 Feb 13 20:05 . drwxr-xr-x 3 root root 4096 Feb 13 20:26 .. drwxr-xr-x 3 root root 4096 Jan 31 2023 SoccerTwos_BurstDebugInformation_DoNotShip drwxr-xr-x 7 root root 4096 Feb 13 20:07 SoccerTwos_Data -rwxr-xr-x 1 root root 14720 Jan 31 2023 SoccerTwos.x86_64 -rwxr-xr-x 1 root root 34584808 Jan 31 2023 UnityPlayer.so
Create the SoccerTwos config file¶
We use the MA-POCA (poca) trainer with self-play.
- MA-POCA uses a centralised critic to train cooperative team behaviour
- Self-play pits the team against past versions of itself
max_steps is set to 3,000,000 (~3–4 hrs on Kaggle) which is enough for certification.
The course default is 5,000,000 but that risks timing out.
You can modify the hyperparameters here to try to get a better ELO score:
batch_size,hidden_units,learning_rateaffect learning stabilitysave_steps,swap_stepsinself_playaffect how often opponents are updated
In [12]:
config_yaml = """
behaviors:
SoccerTwos:
trainer_type: poca
hyperparameters:
batch_size: 2048
buffer_size: 20480
learning_rate: 0.0003
beta: 0.005
epsilon: 0.2
lambd: 0.95
num_epoch: 3
learning_rate_schedule: constant
network_settings:
normalize: false
hidden_units: 512
num_layers: 2
vis_encode_type: simple
reward_signals:
extrinsic:
gamma: 0.99
strength: 1.0
keep_checkpoints: 5
max_steps: 3000000
time_horizon: 1000
summary_freq: 10000
self_play:
save_steps: 50000
team_change: 200000
swap_steps: 2000
window: 10
play_against_latest_model_ratio: 0.5
initial_elo: 1200.0
"""
import os
os.makedirs("/kaggle/working/ml-agents/config/poca", exist_ok=True)
with open("/kaggle/working/ml-agents/config/poca/SoccerTwos.yaml", "w") as f:
f.write(config_yaml)
print("Config written to /kaggle/working/ml-agents/config/poca/SoccerTwos.yaml")
Config written to /kaggle/working/ml-agents/config/poca/SoccerTwos.yaml
Train our agents 🚀¶
To train, we launch mlagents-learn with four parameters:
mlagents-learn <config>— path to the hyperparameter config file--env— path to the environment executable--run-id— name for this training run--no-graphics— don't launch visualisation during training
Use --resume to continue a previously interrupted run.
It will fail first time when you use
--resume— try running the block again to bypass the error.
⚠️ Normal behaviour:
- ELO score may drop below 1200 for the first ~2M steps — this is completely normal!
- Agents spend the early steps exploring randomly before learning to play
Training will take approximately 3–4 hours. Go grab a ☕ — you deserve it 🤗
In [13]:
!/kaggle/working/mlagents-env/bin/mlagents-learn ./config/poca/SoccerTwos.yaml \
--env=./training-envs-executables/SoccerTwos/SoccerTwos.x86_64 \
--run-id="SoccerTwos" \
--no-graphics \
--force
┐ ╖
╓╖╬│╡ ││╬╖╖
╓╖╬│││││┘ ╬│││││╬╖
╖╬│││││╬╜ ╙╬│││││╖╖ ╗╗╗
╬╬╬╬╖││╦╖ ╖╬││╗╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╜╜╜ ╟╣╣
╬╬╬╬╬╬╬╬╖│╬╖╖╓╬╪│╓╣╣╣╣╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╒╣╣╖╗╣╣╣╗ ╣╣╣ ╣╣╣╣╣╣ ╟╣╣╖ ╣╣╣
╬╬╬╬┐ ╙╬╬╬╬│╓╣╣╣╝╜ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╣╙ ╙╣╣╣ ╣╣╣ ╙╟╣╣╜╙ ╫╣╣ ╟╣╣
╬╬╬╬┐ ╙╬╬╣╣ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣ ╣╣╣┌╣╣╜
╬╬╬╜ ╬╬╣╣ ╙╝╣╣╬ ╙╣╣╣╗╖╓╗╣╣╣╜ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣╦╓ ╣╣╣╣╣
╙ ╓╦╖ ╬╬╣╣ ╓╗╗╖ ╙╝╣╣╣╣╝╜ ╘╝╝╜ ╝╝╝ ╝╝╝ ╙╣╣╣ ╟╣╣╣
╩╬╬╬╬╬╬╦╦╬╬╣╣╗╣╣╣╣╣╣╣╝ ╫╣╣╣╣
╙╬╬╬╬╬╬╬╣╣╣╣╣╣╝╜
╙╬╬╬╣╣╣╜
╙
Version information:
ml-agents: 1.1.0,
ml-agents-envs: 1.1.0,
Communicator API: 1.5.0,
PyTorch: 2.10.0+cpu
[INFO] Connected to Unity environment with package version 2.3.0-exp.3 and communication version 1.5.0
[INFO] Connected new brain: SoccerTwos?team=1
[INFO] Connected new brain: SoccerTwos?team=0
[WARNING] Deleting TensorBoard data events.out.tfevents.1771013811.a0871a2ab7dc.8306.0 that was left over from a previous run.
[INFO] Hyperparameters for behavior name SoccerTwos:
trainer_type: poca
hyperparameters:
batch_size: 2048
buffer_size: 20480
learning_rate: 0.0003
beta: 0.005
epsilon: 0.2
lambd: 0.95
num_epoch: 3
learning_rate_schedule: constant
beta_schedule: constant
epsilon_schedule: constant
checkpoint_interval: 500000
network_settings:
normalize: False
hidden_units: 512
num_layers: 2
vis_encode_type: simple
memory: None
goal_conditioning_type: hyper
deterministic: False
reward_signals:
extrinsic:
gamma: 0.99
strength: 1.0
network_settings:
normalize: False
hidden_units: 128
num_layers: 2
vis_encode_type: simple
memory: None
goal_conditioning_type: hyper
deterministic: False
init_path: None
keep_checkpoints: 5
even_checkpoints: False
max_steps: 3000000
time_horizon: 1000
summary_freq: 10000
threaded: False
self_play:
save_steps: 50000
team_change: 200000
swap_steps: 2000
window: 10
play_against_latest_model_ratio: 0.5
initial_elo: 1200.0
behavioral_cloning: None
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/utils.py:289: UserWarning: The use of `x.T` on tensors of dimension other than 2 to reverse their shape is deprecated and it will throw an error in a future release. Consider `x.mT` to transpose batches of matrices or `x.permute(*torch.arange(x.ndim - 1, -1, -1))` to reverse the dimensions of a tensor. (Triggered internally at /pytorch/aten/src/ATen/native/TensorShape.cpp:4480.)
torch.nn.functional.one_hot(_act.T, action_size[i]).float()
[INFO] SoccerTwos. Step: 10000. Time Elapsed: 29.528 s. Mean Reward: 0.000. Mean Group Reward: -0.190. Training. ELO: 1199.506.
[INFO] SoccerTwos. Step: 20000. Time Elapsed: 47.911 s. Mean Reward: 0.000. Mean Group Reward: -0.343. Training. ELO: 1199.680.
[INFO] SoccerTwos. Step: 30000. Time Elapsed: 96.386 s. Mean Reward: 0.000. Mean Group Reward: -0.177. Training. ELO: 1199.268.
[INFO] SoccerTwos. Step: 40000. Time Elapsed: 111.823 s. Mean Reward: 0.000. Mean Group Reward: -0.164. Training. ELO: 1198.528.
[INFO] SoccerTwos. Step: 50000. Time Elapsed: 154.765 s. Mean Reward: 0.000. Mean Group Reward: -0.312. Training. ELO: 1199.328.
[INFO] SoccerTwos. Step: 60000. Time Elapsed: 170.551 s. Mean Reward: 0.000. Mean Group Reward: 0.085. Training. ELO: 1200.039.
[INFO] SoccerTwos. Step: 70000. Time Elapsed: 215.376 s. Mean Reward: 0.000. Mean Group Reward: 0.041. Training.
[INFO] SoccerTwos. Step: 80000. Time Elapsed: 235.110 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 90000. Time Elapsed: 271.978 s. Mean Reward: 0.000. Mean Group Reward: 0.140. Training. ELO: 1200.039.
[INFO] SoccerTwos. Step: 100000. Time Elapsed: 292.340 s. Mean Reward: 0.000. Mean Group Reward: -0.046. Training. ELO: 1200.039.
[INFO] SoccerTwos. Step: 110000. Time Elapsed: 329.369 s. Mean Reward: 0.000. Mean Group Reward: -0.016. Training. ELO: 1200.289.
[INFO] SoccerTwos. Step: 120000. Time Elapsed: 350.891 s. Mean Reward: 0.000. Mean Group Reward: -0.578. Training. ELO: 1201.038.
[INFO] SoccerTwos. Step: 130000. Time Elapsed: 399.159 s. Mean Reward: 0.000. Mean Group Reward: 0.081. Training. ELO: 1202.652.
[INFO] SoccerTwos. Step: 140000. Time Elapsed: 417.556 s. Mean Reward: 0.000. Mean Group Reward: 0.120. Training. ELO: 1203.023.
[INFO] SoccerTwos. Step: 150000. Time Elapsed: 432.034 s. Mean Reward: 0.000. Mean Group Reward: 0.270. Training. ELO: 1203.846.
[INFO] SoccerTwos. Step: 160000. Time Elapsed: 473.163 s. Mean Reward: 0.000. Mean Group Reward: -0.339. Training. ELO: 1203.692.
[INFO] SoccerTwos. Step: 170000. Time Elapsed: 489.889 s. Mean Reward: 0.000. Mean Group Reward: -0.167. Training. ELO: 1201.147.
[INFO] SoccerTwos. Step: 180000. Time Elapsed: 532.855 s. Mean Reward: 0.000. Mean Group Reward: -0.308. Training. ELO: 1199.646.
[INFO] SoccerTwos. Step: 190000. Time Elapsed: 547.867 s. Mean Reward: 0.000. Mean Group Reward: -0.094. Training. ELO: 1198.899.
[INFO] SoccerTwos. Step: 200000. Time Elapsed: 587.535 s. Mean Reward: 0.000. Mean Group Reward: 0.241. Training. ELO: 1198.899.
[INFO] SoccerTwos. Step: 210000. Time Elapsed: 618.049 s. Mean Reward: 0.000. Mean Group Reward: -0.218. Training. ELO: 1199.087.
[INFO] SoccerTwos. Step: 220000. Time Elapsed: 652.945 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 230000. Time Elapsed: 673.879 s. Mean Reward: 0.000. Mean Group Reward: 0.153. Training. ELO: 1199.898.
[INFO] SoccerTwos. Step: 240000. Time Elapsed: 705.226 s. Mean Reward: 0.000. Mean Group Reward: -0.027. Training. ELO: 1199.898.
[INFO] SoccerTwos. Step: 250000. Time Elapsed: 723.186 s. Mean Reward: 0.000. Mean Group Reward: -0.240. Training. ELO: 1199.898.
[INFO] SoccerTwos. Step: 260000. Time Elapsed: 762.794 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 270000. Time Elapsed: 779.382 s. Mean Reward: 0.000. Mean Group Reward: 0.118. Training. ELO: 1200.648.
[INFO] SoccerTwos. Step: 280000. Time Elapsed: 819.275 s. Mean Reward: 0.000. Mean Group Reward: -0.054. Training. ELO: 1200.563.
[INFO] SoccerTwos. Step: 290000. Time Elapsed: 838.629 s. Mean Reward: 0.000. Mean Group Reward: 0.128. Training. ELO: 1200.898.
[INFO] SoccerTwos. Step: 300000. Time Elapsed: 851.048 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 310000. Time Elapsed: 893.621 s. Mean Reward: 0.000. Mean Group Reward: 0.008. Training. ELO: 1201.641.
[INFO] SoccerTwos. Step: 320000. Time Elapsed: 911.762 s. Mean Reward: 0.000. Mean Group Reward: -0.172. Training. ELO: 1201.889.
[INFO] SoccerTwos. Step: 330000. Time Elapsed: 950.639 s. Mean Reward: 0.000. Mean Group Reward: -0.076. Training. ELO: 1201.135.
[INFO] SoccerTwos. Step: 340000. Time Elapsed: 971.757 s. Mean Reward: 0.000. Mean Group Reward: -0.188. Training. ELO: 1199.720.
[INFO] SoccerTwos. Step: 350000. Time Elapsed: 1013.490 s. Mean Reward: 0.000. Mean Group Reward: 0.252. Training. ELO: 1200.045.
[INFO] SoccerTwos. Step: 360000. Time Elapsed: 1032.353 s. Mean Reward: 0.000. Mean Group Reward: -0.182. Training. ELO: 1200.877.
[INFO] SoccerTwos. Step: 370000. Time Elapsed: 1077.693 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 380000. Time Elapsed: 1089.226 s. Mean Reward: 0.000. Mean Group Reward: -0.254. Training. ELO: 1200.877.
[INFO] SoccerTwos. Step: 390000. Time Elapsed: 1133.513 s. Mean Reward: 0.000. Mean Group Reward: -0.333. Training. ELO: 1200.903.
[INFO] SoccerTwos. Step: 400000. Time Elapsed: 1149.010 s. Mean Reward: 0.000. Mean Group Reward: -0.013. Training. ELO: 1200.564.
[INFO] SoccerTwos. Step: 410000. Time Elapsed: 1194.670 s. Mean Reward: 0.000. Mean Group Reward: -0.265. Training. ELO: 1200.924.
[INFO] SoccerTwos. Step: 420000. Time Elapsed: 1216.309 s. Mean Reward: 0.000. Mean Group Reward: 0.074. Training. ELO: 1199.685.
[INFO] SoccerTwos. Step: 430000. Time Elapsed: 1257.861 s. Mean Reward: 0.000. Mean Group Reward: -0.141. Training. ELO: 1199.522.
[INFO] SoccerTwos. Step: 440000. Time Elapsed: 1270.938 s. Mean Reward: 0.000. Mean Group Reward: -0.333. Training. ELO: 1199.844.
[INFO] SoccerTwos. Step: 450000. Time Elapsed: 1296.353 s. Mean Reward: 0.000. Mean Group Reward: 0.206. Training. ELO: 1200.917.
[INFO] SoccerTwos. Step: 460000. Time Elapsed: 1332.784 s. Mean Reward: 0.000. Mean Group Reward: 0.130. Training. ELO: 1202.334.
[INFO] SoccerTwos. Step: 470000. Time Elapsed: 1351.167 s. Mean Reward: 0.000. Mean Group Reward: -0.113. Training. ELO: 1201.974.
[INFO] SoccerTwos. Step: 480000. Time Elapsed: 1396.701 s. Mean Reward: 0.000. Mean Group Reward: -0.076. Training. ELO: 1202.314.
[INFO] SoccerTwos. Step: 490000. Time Elapsed: 1404.357 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 500000. Time Elapsed: 1456.888 s. Mean Reward: 0.000. Mean Group Reward: 0.160. Training. ELO: 1202.998.
Traceback (most recent call last):
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer_controller.py", line 175, in start_learning
n_steps = self.advance(env_manager)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents_envs/timers.py", line 305, in wrapped
return func(*args, **kwargs)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer_controller.py", line 250, in advance
trainer.advance()
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/ghost/trainer.py", line 254, in advance
self.trainer.advance()
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer/rl_trainer.py", line 293, in advance
self._process_trajectory(t)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/poca/trainer.py", line 76, in _process_trajectory
super()._process_trajectory(trajectory)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer/rl_trainer.py", line 224, in _process_trajectory
self._maybe_save_model(self.get_step + len(trajectory.steps))
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer/rl_trainer.py", line 264, in _maybe_save_model
self._checkpoint()
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents_envs/timers.py", line 305, in wrapped
return func(*args, **kwargs)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer/rl_trainer.py", line 144, in _checkpoint
export_path, auxillary_paths = self.model_saver.save_checkpoint(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/model_saver/torch_model_saver.py", line 60, in save_checkpoint
self.export(checkpoint_path, behavior_name)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/model_saver/torch_model_saver.py", line 65, in export
self.exporter.export_policy_model(output_filepath)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py", line 164, in export_policy_model
torch.onnx.export(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/__init__.py", line 282, in export
from torch.onnx._internal.exporter import _compat
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_compat.py", line 16, in <module>
from torch.onnx._internal.exporter import (
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_core.py", line 19, in <module>
import onnxscript
ModuleNotFoundError: No module named 'onnxscript'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/kaggle/working/mlagents-env/bin/mlagents-learn", line 8, in <module>
sys.exit(main())
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/learn.py", line 270, in main
run_cli(parse_command_line())
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/learn.py", line 266, in run_cli
run_training(run_seed, options, num_areas)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/learn.py", line 138, in run_training
tc.start_learning(env_manager)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents_envs/timers.py", line 305, in wrapped
return func(*args, **kwargs)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer_controller.py", line 200, in start_learning
self._save_models()
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents_envs/timers.py", line 305, in wrapped
return func(*args, **kwargs)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer_controller.py", line 80, in _save_models
self.trainers[brain_name].save_model()
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/ghost/trainer.py", line 334, in save_model
self.trainer.save_model()
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer/rl_trainer.py", line 172, in save_model
model_checkpoint = self._checkpoint()
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents_envs/timers.py", line 305, in wrapped
return func(*args, **kwargs)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer/rl_trainer.py", line 144, in _checkpoint
export_path, auxillary_paths = self.model_saver.save_checkpoint(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/model_saver/torch_model_saver.py", line 60, in save_checkpoint
self.export(checkpoint_path, behavior_name)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/model_saver/torch_model_saver.py", line 65, in export
self.exporter.export_policy_model(output_filepath)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py", line 164, in export_policy_model
torch.onnx.export(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/__init__.py", line 282, in export
from torch.onnx._internal.exporter import _compat
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_compat.py", line 16, in <module>
from torch.onnx._internal.exporter import (
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_core.py", line 19, in <module>
import onnxscript
ModuleNotFoundError: No module named 'onnxscript'
In [20]:
%%capture
!/kaggle/working/mlagents-env/bin/pip install onnxscript "protobuf==3.20.3"
In [21]:
!/kaggle/working/mlagents-env/bin/python -c "import onnxscript; import google.protobuf; print('OK')"
OK
In [22]:
!/kaggle/working/mlagents-env/bin/mlagents-learn ./config/poca/SoccerTwos.yaml \
--env=./training-envs-executables/SoccerTwos/SoccerTwos.x86_64 \
--run-id="SoccerTwos" \
--no-graphics \
--resume
┐ ╖
╓╖╬│╡ ││╬╖╖
╓╖╬│││││┘ ╬│││││╬╖
╖╬│││││╬╜ ╙╬│││││╖╖ ╗╗╗
╬╬╬╬╖││╦╖ ╖╬││╗╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╜╜╜ ╟╣╣
╬╬╬╬╬╬╬╬╖│╬╖╖╓╬╪│╓╣╣╣╣╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╒╣╣╖╗╣╣╣╗ ╣╣╣ ╣╣╣╣╣╣ ╟╣╣╖ ╣╣╣
╬╬╬╬┐ ╙╬╬╬╬│╓╣╣╣╝╜ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╣╙ ╙╣╣╣ ╣╣╣ ╙╟╣╣╜╙ ╫╣╣ ╟╣╣
╬╬╬╬┐ ╙╬╬╣╣ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣ ╣╣╣┌╣╣╜
╬╬╬╜ ╬╬╣╣ ╙╝╣╣╬ ╙╣╣╣╗╖╓╗╣╣╣╜ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣╦╓ ╣╣╣╣╣
╙ ╓╦╖ ╬╬╣╣ ╓╗╗╖ ╙╝╣╣╣╣╝╜ ╘╝╝╜ ╝╝╝ ╝╝╝ ╙╣╣╣ ╟╣╣╣
╩╬╬╬╬╬╬╦╦╬╬╣╣╗╣╣╣╣╣╣╣╝ ╫╣╣╣╣
╙╬╬╬╬╬╬╬╣╣╣╣╣╣╝╜
╙╬╬╬╣╣╣╜
╙
Version information:
ml-agents: 1.1.0,
ml-agents-envs: 1.1.0,
Communicator API: 1.5.0,
PyTorch: 2.10.0+cpu
[INFO] Connected to Unity environment with package version 2.3.0-exp.3 and communication version 1.5.0
[INFO] Connected new brain: SoccerTwos?team=1
[INFO] Connected new brain: SoccerTwos?team=0
[INFO] Hyperparameters for behavior name SoccerTwos:
trainer_type: poca
hyperparameters:
batch_size: 2048
buffer_size: 20480
learning_rate: 0.0003
beta: 0.005
epsilon: 0.2
lambd: 0.95
num_epoch: 3
learning_rate_schedule: constant
beta_schedule: constant
epsilon_schedule: constant
checkpoint_interval: 500000
network_settings:
normalize: False
hidden_units: 512
num_layers: 2
vis_encode_type: simple
memory: None
goal_conditioning_type: hyper
deterministic: False
reward_signals:
extrinsic:
gamma: 0.99
strength: 1.0
network_settings:
normalize: False
hidden_units: 128
num_layers: 2
vis_encode_type: simple
memory: None
goal_conditioning_type: hyper
deterministic: False
init_path: None
keep_checkpoints: 5
even_checkpoints: False
max_steps: 3000000
time_horizon: 1000
summary_freq: 10000
threaded: False
self_play:
save_steps: 50000
team_change: 200000
swap_steps: 2000
window: 10
play_against_latest_model_ratio: 0.5
initial_elo: 1200.0
behavioral_cloning: None
[INFO] Resuming from results/SoccerTwos/SoccerTwos.
[INFO] Resuming training from step 499868.
[INFO] SoccerTwos. Step: 500000. Time Elapsed: 8.836 s. No episode was completed since last summary. Training.
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py:164: UserWarning: Exporting a model while it is in training mode. Please ensure that this is intended, as it may lead to different behavior during inference. Calling model.eval() before export is recommended.
torch.onnx.export(
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py:164: UserWarning: # 'dynamic_axes' is not recommended when dynamo=True, and may lead to 'torch._dynamo.exc.UserError: Constraints violated.' Supply the 'dynamic_shapes' argument instead if export is unsuccessful.
torch.onnx.export(
W0213 21:20:10.338000 10240 torch/onnx/_internal/exporter/_compat.py:125] Setting ONNX exporter to use operator set version 18 because the requested opset_version 9 is a lower version than we have implementations for. Automatic version conversion will be performed, which may not be successful at converting to the requested version. If version conversion is unsuccessful, the opset version of the exported model will be kept at 18. Please consider setting opset_version >=18 to leverage latest ONNX features
W0213 21:20:11.362000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::nms
W0213 21:20:11.363000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::roi_align
W0213 21:20:11.363000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::roi_pool
[torch.onnx] Obtain model graph for `SimpleActor([...]` with `torch.export.export(..., strict=False)`...
[torch.onnx] Obtain model graph for `SimpleActor([...]` with `torch.export.export(..., strict=False)`... ✅
[torch.onnx] Run decomposition...
/usr/lib/python3.10/copyreg.py:101: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
return cls.__new__(cls, *args)
[torch.onnx] Run decomposition... ✅
[torch.onnx] Translate the graph into ONNX...
[torch.onnx] Translate the graph into ONNX... ✅
/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_onnx_program.py:460: UserWarning: # The axis name: batch will not be used, since it shares the same shape constraints with another axis: batch.
rename_mapping = _dynamic_shapes.create_rename_mapping(
The model version conversion is not supported by the onnxscript version converter and fallback is enabled. The model will be converted using the onnx C API (target version: 9).
Failed to convert the model to the target version 9 using the ONNX C API. The model was not modified
Traceback (most recent call last):
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/__init__.py", line 120, in call
converted_proto = _c_api_utils.call_onnx_api(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/_c_api_utils.py", line 65, in call_onnx_api
result = func(proto)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/__init__.py", line 115, in _partial_convert_version
return onnx.version_converter.convert_version(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnx/version_converter.py", line 38, in convert_version
converted_model_str = C.convert_version(model_str, target_version)
RuntimeError: /github/workspace/onnx/version_converter/BaseConverter.h:73: adapter_lookup: Assertion `false` failed: No Adapter From Version $15 for Shape
Applied 3 of general pattern rewrite rules.
Skipped deduplication of initializer 'version_number' as it is a graph input or output
Skipped deduplication of initializer 'memory_size' as it is a graph input or output
Skipped deduplication of initializer 'discrete_action_output_shape' as it is a graph input or output
[INFO] Exported results/SoccerTwos/SoccerTwos/SoccerTwos-499868.onnx
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/utils.py:289: UserWarning: The use of `x.T` on tensors of dimension other than 2 to reverse their shape is deprecated and it will throw an error in a future release. Consider `x.mT` to transpose batches of matrices or `x.permute(*torch.arange(x.ndim - 1, -1, -1))` to reverse the dimensions of a tensor. (Triggered internally at /pytorch/aten/src/ATen/native/TensorShape.cpp:4480.)
torch.nn.functional.one_hot(_act.T, action_size[i]).float()
[INFO] SoccerTwos. Step: 510000. Time Elapsed: 30.718 s. Mean Reward: 0.000. Mean Group Reward: -0.218. Training. ELO: 1204.305.
[INFO] SoccerTwos. Step: 520000. Time Elapsed: 56.111 s. Mean Reward: 0.000. Mean Group Reward: -0.364. Training. ELO: 1202.935.
[INFO] SoccerTwos. Step: 530000. Time Elapsed: 95.613 s. Mean Reward: 0.000. Mean Group Reward: -0.020. Training. ELO: 1202.811.
[INFO] SoccerTwos. Step: 540000. Time Elapsed: 116.816 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 550000. Time Elapsed: 152.388 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 560000. Time Elapsed: 166.304 s. Mean Reward: 0.000. Mean Group Reward: -0.105. Training. ELO: 1202.811.
[INFO] SoccerTwos. Step: 570000. Time Elapsed: 188.333 s. Mean Reward: 0.000. Mean Group Reward: 0.173. Training. ELO: 1203.189.
[INFO] SoccerTwos. Step: 580000. Time Elapsed: 218.953 s. Mean Reward: 0.000. Mean Group Reward: 0.090. Training. ELO: 1204.571.
[INFO] SoccerTwos. Step: 590000. Time Elapsed: 241.424 s. Mean Reward: 0.000. Mean Group Reward: 0.128. Training. ELO: 1204.822.
[INFO] SoccerTwos. Step: 600000. Time Elapsed: 281.024 s. Mean Reward: 0.000. Mean Group Reward: -0.064. Training. ELO: 1205.009.
[INFO] SoccerTwos. Step: 610000. Time Elapsed: 298.508 s. Mean Reward: 0.000. Mean Group Reward: -0.182. Training. ELO: 1205.069.
[INFO] SoccerTwos. Step: 620000. Time Elapsed: 341.182 s. Mean Reward: 0.000. Mean Group Reward: -0.048. Training. ELO: 1204.233.
[INFO] SoccerTwos. Step: 630000. Time Elapsed: 363.472 s. Mean Reward: 0.000. Mean Group Reward: -0.064. Training. ELO: 1203.735.
[INFO] SoccerTwos. Step: 640000. Time Elapsed: 403.716 s. Mean Reward: 0.000. Mean Group Reward: -0.091. Training. ELO: 1204.819.
[INFO] SoccerTwos. Step: 650000. Time Elapsed: 421.030 s. Mean Reward: 0.000. Mean Group Reward: -0.352. Training. ELO: 1202.788.
[INFO] SoccerTwos. Step: 660000. Time Elapsed: 457.479 s. Mean Reward: 0.000. Mean Group Reward: -0.108. Training. ELO: 1202.610.
[INFO] SoccerTwos. Step: 670000. Time Elapsed: 482.751 s. Mean Reward: 0.000. Mean Group Reward: -0.153. Training. ELO: 1201.995.
[INFO] SoccerTwos. Step: 680000. Time Elapsed: 521.545 s. Mean Reward: 0.000. Mean Group Reward: -0.200. Training. ELO: 1201.132.
[INFO] SoccerTwos. Step: 690000. Time Elapsed: 536.127 s. Mean Reward: 0.000. Mean Group Reward: -0.182. Training. ELO: 1200.887.
[INFO] SoccerTwos. Step: 700000. Time Elapsed: 577.070 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 710000. Time Elapsed: 607.569 s. Mean Reward: 0.000. Mean Group Reward: -0.286. Training. ELO: 1199.408.
[INFO] SoccerTwos. Step: 720000. Time Elapsed: 643.302 s. Mean Reward: 0.000. Mean Group Reward: 0.251. Training. ELO: 1200.184.
[INFO] SoccerTwos. Step: 730000. Time Elapsed: 665.529 s. Mean Reward: 0.000. Mean Group Reward: -0.009. Training. ELO: 1200.948.
[INFO] SoccerTwos. Step: 740000. Time Elapsed: 710.292 s. Mean Reward: 0.000. Mean Group Reward: -0.067. Training. ELO: 1201.458.
[INFO] SoccerTwos. Step: 750000. Time Elapsed: 726.084 s. Mean Reward: 0.000. Mean Group Reward: -0.070. Training. ELO: 1200.090.
[INFO] SoccerTwos. Step: 760000. Time Elapsed: 744.823 s. Mean Reward: 0.000. Mean Group Reward: -0.250. Training. ELO: 1199.599.
[INFO] SoccerTwos. Step: 770000. Time Elapsed: 779.938 s. Mean Reward: 0.000. Mean Group Reward: 0.050. Training. ELO: 1199.354.
[INFO] SoccerTwos. Step: 780000. Time Elapsed: 801.000 s. Mean Reward: 0.000. Mean Group Reward: 0.011. Training. ELO: 1199.284.
[INFO] SoccerTwos. Step: 790000. Time Elapsed: 847.281 s. Mean Reward: 0.000. Mean Group Reward: -0.182. Training. ELO: 1199.310.
[INFO] SoccerTwos. Step: 800000. Time Elapsed: 858.352 s. Mean Reward: 0.000. Mean Group Reward: -0.142. Training. ELO: 1198.576.
[INFO] SoccerTwos. Step: 810000. Time Elapsed: 897.668 s. Mean Reward: 0.000. Mean Group Reward: -0.300. Training. ELO: 1198.721.
[INFO] SoccerTwos. Step: 820000. Time Elapsed: 921.392 s. Mean Reward: 0.000. Mean Group Reward: -0.385. Training. ELO: 1196.881.
[INFO] SoccerTwos. Step: 830000. Time Elapsed: 957.074 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 840000. Time Elapsed: 975.854 s. Mean Reward: 0.000. Mean Group Reward: 0.095. Training. ELO: 1197.388.
[INFO] SoccerTwos. Step: 850000. Time Elapsed: 1013.087 s. Mean Reward: 0.000. Mean Group Reward: -0.318. Training. ELO: 1197.558.
[INFO] SoccerTwos. Step: 860000. Time Elapsed: 1033.931 s. Mean Reward: 0.000. Mean Group Reward: -0.167. Training. ELO: 1196.490.
[INFO] SoccerTwos. Step: 870000. Time Elapsed: 1079.538 s. Mean Reward: 0.000. Mean Group Reward: 0.125. Training. ELO: 1196.244.
[INFO] SoccerTwos. Step: 880000. Time Elapsed: 1091.395 s. Mean Reward: 0.000. Mean Group Reward: -0.179. Training. ELO: 1195.380.
[INFO] SoccerTwos. Step: 890000. Time Elapsed: 1133.172 s. Mean Reward: 0.000. Mean Group Reward: 0.168. Training. ELO: 1195.639.
[INFO] SoccerTwos. Step: 900000. Time Elapsed: 1152.821 s. Mean Reward: 0.000. Mean Group Reward: -0.167. Training. ELO: 1195.544.
[INFO] SoccerTwos. Step: 910000. Time Elapsed: 1180.108 s. Mean Reward: 0.000. Mean Group Reward: -0.053. Training. ELO: 1195.300.
[INFO] SoccerTwos. Step: 920000. Time Elapsed: 1216.848 s. Mean Reward: 0.000. Mean Group Reward: -0.001. Training. ELO: 1195.300.
[INFO] SoccerTwos. Step: 930000. Time Elapsed: 1233.683 s. Mean Reward: 0.000. Mean Group Reward: -0.165. Training. ELO: 1196.212.
[INFO] SoccerTwos. Step: 940000. Time Elapsed: 1275.991 s. Mean Reward: 0.000. Mean Group Reward: -0.333. Training. ELO: 1196.342.
[INFO] SoccerTwos. Step: 950000. Time Elapsed: 1290.070 s. Mean Reward: 0.000. Mean Group Reward: -0.333. Training. ELO: 1195.468.
[INFO] SoccerTwos. Step: 960000. Time Elapsed: 1333.836 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 970000. Time Elapsed: 1346.051 s. Mean Reward: 0.000. Mean Group Reward: -0.182. Training. ELO: 1195.343.
[INFO] SoccerTwos. Step: 980000. Time Elapsed: 1387.751 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 990000. Time Elapsed: 1411.652 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1000000. Time Elapsed: 1445.862 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py:164: UserWarning: Exporting a model while it is in training mode. Please ensure that this is intended, as it may lead to different behavior during inference. Calling model.eval() before export is recommended.
torch.onnx.export(
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py:164: UserWarning: # 'dynamic_axes' is not recommended when dynamo=True, and may lead to 'torch._dynamo.exc.UserError: Constraints violated.' Supply the 'dynamic_shapes' argument instead if export is unsuccessful.
torch.onnx.export(
W0213 21:44:07.151000 10240 torch/onnx/_internal/exporter/_compat.py:125] Setting ONNX exporter to use operator set version 18 because the requested opset_version 9 is a lower version than we have implementations for. Automatic version conversion will be performed, which may not be successful at converting to the requested version. If version conversion is unsuccessful, the opset version of the exported model will be kept at 18. Please consider setting opset_version >=18 to leverage latest ONNX features
W0213 21:44:07.741000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::nms
W0213 21:44:07.742000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::roi_align
W0213 21:44:07.742000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::roi_pool
[torch.onnx] Obtain model graph for `SimpleActor([...]` with `torch.export.export(..., strict=False)`...
[torch.onnx] Obtain model graph for `SimpleActor([...]` with `torch.export.export(..., strict=False)`... ✅
[torch.onnx] Run decomposition...
/usr/lib/python3.10/copyreg.py:101: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
return cls.__new__(cls, *args)
[torch.onnx] Run decomposition... ✅
[torch.onnx] Translate the graph into ONNX...
[torch.onnx] Translate the graph into ONNX... ✅
/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_onnx_program.py:460: UserWarning: # The axis name: batch will not be used, since it shares the same shape constraints with another axis: batch.
rename_mapping = _dynamic_shapes.create_rename_mapping(
The model version conversion is not supported by the onnxscript version converter and fallback is enabled. The model will be converted using the onnx C API (target version: 9).
Failed to convert the model to the target version 9 using the ONNX C API. The model was not modified
Traceback (most recent call last):
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/__init__.py", line 120, in call
converted_proto = _c_api_utils.call_onnx_api(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/_c_api_utils.py", line 65, in call_onnx_api
result = func(proto)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/__init__.py", line 115, in _partial_convert_version
return onnx.version_converter.convert_version(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnx/version_converter.py", line 38, in convert_version
converted_model_str = C.convert_version(model_str, target_version)
RuntimeError: /github/workspace/onnx/version_converter/BaseConverter.h:73: adapter_lookup: Assertion `false` failed: No Adapter From Version $15 for Shape
Applied 3 of general pattern rewrite rules.
Skipped deduplication of initializer 'version_number' as it is a graph input or output
Skipped deduplication of initializer 'memory_size' as it is a graph input or output
Skipped deduplication of initializer 'discrete_action_output_shape' as it is a graph input or output
[INFO] Exported results/SoccerTwos/SoccerTwos/SoccerTwos-999348.onnx
[INFO] SoccerTwos. Step: 1010000. Time Elapsed: 1467.685 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1020000. Time Elapsed: 1511.474 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1030000. Time Elapsed: 1520.440 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1040000. Time Elapsed: 1564.292 s. Mean Reward: 0.000. Mean Group Reward: 0.123. Training. ELO: 1196.092.
[INFO] SoccerTwos. Step: 1050000. Time Elapsed: 1575.381 s. Mean Reward: 0.000. Mean Group Reward: -0.100. Training. ELO: 1195.602.
[INFO] SoccerTwos. Step: 1060000. Time Elapsed: 1592.478 s. Mean Reward: 0.000. Mean Group Reward: 0.056. Training. ELO: 1196.120.
[INFO] SoccerTwos. Step: 1070000. Time Elapsed: 1637.577 s. Mean Reward: 0.000. Mean Group Reward: -0.031. Training. ELO: 1195.899.
[INFO] SoccerTwos. Step: 1080000. Time Elapsed: 1652.561 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1090000. Time Elapsed: 1691.019 s. Mean Reward: 0.000. Mean Group Reward: -0.200. Training. ELO: 1195.649.
[INFO] SoccerTwos. Step: 1100000. Time Elapsed: 1710.144 s. Mean Reward: 0.000. Mean Group Reward: 0.143. Training. ELO: 1196.144.
[INFO] SoccerTwos. Step: 1110000. Time Elapsed: 1760.508 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1120000. Time Elapsed: 1776.293 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1130000. Time Elapsed: 1817.962 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1140000. Time Elapsed: 1836.703 s. Mean Reward: 0.000. Mean Group Reward: -0.182. Training. ELO: 1195.650.
[INFO] SoccerTwos. Step: 1150000. Time Elapsed: 1875.708 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1160000. Time Elapsed: 1896.003 s. Mean Reward: 0.000. Mean Group Reward: 0.143. Training. ELO: 1196.155.
[INFO] SoccerTwos. Step: 1170000. Time Elapsed: 1945.154 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1180000. Time Elapsed: 1958.549 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1190000. Time Elapsed: 1981.186 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1200000. Time Elapsed: 2023.447 s. Mean Reward: 0.000. Mean Group Reward: 0.032. Training. ELO: 1196.405.
[INFO] SoccerTwos. Step: 1210000. Time Elapsed: 2044.020 s. Mean Reward: 0.000. Mean Group Reward: 0.032. Training.
[INFO] SoccerTwos. Step: 1220000. Time Elapsed: 2092.936 s. Mean Reward: 0.000. Mean Group Reward: 0.001. Training. ELO: 1196.405.
[INFO] SoccerTwos. Step: 1230000. Time Elapsed: 2109.914 s. Mean Reward: 0.000. Mean Group Reward: -0.167. Training. ELO: 1196.405.
[INFO] SoccerTwos. Step: 1240000. Time Elapsed: 2156.877 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1250000. Time Elapsed: 2181.593 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1260000. Time Elapsed: 2217.883 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1270000. Time Elapsed: 2244.953 s. Mean Reward: 0.000. Mean Group Reward: -0.200. Training. ELO: 1196.405.
[INFO] SoccerTwos. Step: 1280000. Time Elapsed: 2280.288 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1290000. Time Elapsed: 2303.143 s. Mean Reward: 0.000. Mean Group Reward: 0.132. Training. ELO: 1197.154.
[INFO] SoccerTwos. Step: 1300000. Time Elapsed: 2351.131 s. Mean Reward: 0.000. Mean Group Reward: -0.308. Training. ELO: 1196.146.
[INFO] SoccerTwos. Step: 1310000. Time Elapsed: 2382.839 s. Mean Reward: 0.000. Mean Group Reward: -0.111. Training. ELO: 1195.402.
[INFO] SoccerTwos. Step: 1320000. Time Elapsed: 2425.519 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1330000. Time Elapsed: 2442.854 s. Mean Reward: 0.000. Mean Group Reward: 0.005. Training. ELO: 1196.147.
[INFO] SoccerTwos. Step: 1340000. Time Elapsed: 2491.340 s. Mean Reward: 0.000. Mean Group Reward: 0.079. Training. ELO: 1196.395.
[INFO] SoccerTwos. Step: 1350000. Time Elapsed: 2509.249 s. Mean Reward: 0.000. Mean Group Reward: 0.069. Training. ELO: 1197.142.
[INFO] SoccerTwos. Step: 1360000. Time Elapsed: 2526.666 s. Mean Reward: 0.000. Mean Group Reward: 0.026. Training. ELO: 1198.137.
[INFO] SoccerTwos. Step: 1370000. Time Elapsed: 2564.244 s. Mean Reward: 0.000. Mean Group Reward: -0.126. Training. ELO: 1197.878.
[INFO] SoccerTwos. Step: 1380000. Time Elapsed: 2583.614 s. Mean Reward: 0.000. Mean Group Reward: -0.097. Training. ELO: 1198.033.
[INFO] SoccerTwos. Step: 1390000. Time Elapsed: 2630.892 s. Mean Reward: 0.000. Mean Group Reward: 0.082. Training.
[INFO] SoccerTwos. Step: 1400000. Time Elapsed: 2647.283 s. Mean Reward: 0.000. Mean Group Reward: -0.182. Training. ELO: 1198.363.
[INFO] SoccerTwos. Step: 1410000. Time Elapsed: 2691.025 s. Mean Reward: 0.000. Mean Group Reward: 0.170. Training. ELO: 1199.221.
[INFO] SoccerTwos. Step: 1420000. Time Elapsed: 2717.002 s. Mean Reward: 0.000. Mean Group Reward: -0.100. Training. ELO: 1201.193.
[INFO] SoccerTwos. Step: 1430000. Time Elapsed: 2750.898 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1440000. Time Elapsed: 2775.001 s. Mean Reward: 0.000. Mean Group Reward: -0.005. Training. ELO: 1200.367.
[INFO] SoccerTwos. Step: 1450000. Time Elapsed: 2828.109 s. Mean Reward: 0.000. Mean Group Reward: -0.424. Training. ELO: 1199.691.
[INFO] SoccerTwos. Step: 1460000. Time Elapsed: 2836.876 s. Mean Reward: 0.000. Mean Group Reward: -0.136. Training. ELO: 1198.315.
[INFO] SoccerTwos. Step: 1470000. Time Elapsed: 2886.001 s. Mean Reward: 0.000. Mean Group Reward: 0.164. Training. ELO: 1198.315.
[INFO] SoccerTwos. Step: 1480000. Time Elapsed: 2908.425 s. Mean Reward: 0.000. Mean Group Reward: -0.145. Training. ELO: 1198.636.
[INFO] SoccerTwos. Step: 1490000. Time Elapsed: 2944.004 s. Mean Reward: 0.000. Mean Group Reward: -0.261. Training. ELO: 1197.375.
[INFO] SoccerTwos. Step: 1500000. Time Elapsed: 2966.935 s. Mean Reward: 0.000. Mean Group Reward: -0.318. Training. ELO: 1197.291.
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py:164: UserWarning: Exporting a model while it is in training mode. Please ensure that this is intended, as it may lead to different behavior during inference. Calling model.eval() before export is recommended.
torch.onnx.export(
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py:164: UserWarning: # 'dynamic_axes' is not recommended when dynamo=True, and may lead to 'torch._dynamo.exc.UserError: Constraints violated.' Supply the 'dynamic_shapes' argument instead if export is unsuccessful.
torch.onnx.export(
W0213 22:09:28.220000 10240 torch/onnx/_internal/exporter/_compat.py:125] Setting ONNX exporter to use operator set version 18 because the requested opset_version 9 is a lower version than we have implementations for. Automatic version conversion will be performed, which may not be successful at converting to the requested version. If version conversion is unsuccessful, the opset version of the exported model will be kept at 18. Please consider setting opset_version >=18 to leverage latest ONNX features
W0213 22:09:28.844000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::nms
W0213 22:09:28.844000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::roi_align
W0213 22:09:28.844000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::roi_pool
[torch.onnx] Obtain model graph for `SimpleActor([...]` with `torch.export.export(..., strict=False)`...
[torch.onnx] Obtain model graph for `SimpleActor([...]` with `torch.export.export(..., strict=False)`... ✅
[torch.onnx] Run decomposition...
/usr/lib/python3.10/copyreg.py:101: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
return cls.__new__(cls, *args)
[torch.onnx] Run decomposition... ✅
[torch.onnx] Translate the graph into ONNX...
[torch.onnx] Translate the graph into ONNX... ✅
/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_onnx_program.py:460: UserWarning: # The axis name: batch will not be used, since it shares the same shape constraints with another axis: batch.
rename_mapping = _dynamic_shapes.create_rename_mapping(
The model version conversion is not supported by the onnxscript version converter and fallback is enabled. The model will be converted using the onnx C API (target version: 9).
Failed to convert the model to the target version 9 using the ONNX C API. The model was not modified
Traceback (most recent call last):
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/__init__.py", line 120, in call
converted_proto = _c_api_utils.call_onnx_api(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/_c_api_utils.py", line 65, in call_onnx_api
result = func(proto)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/__init__.py", line 115, in _partial_convert_version
return onnx.version_converter.convert_version(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnx/version_converter.py", line 38, in convert_version
converted_model_str = C.convert_version(model_str, target_version)
RuntimeError: /github/workspace/onnx/version_converter/BaseConverter.h:73: adapter_lookup: Assertion `false` failed: No Adapter From Version $15 for Shape
Applied 3 of general pattern rewrite rules.
Skipped deduplication of initializer 'version_number' as it is a graph input or output
Skipped deduplication of initializer 'memory_size' as it is a graph input or output
Skipped deduplication of initializer 'discrete_action_output_shape' as it is a graph input or output
[INFO] Exported results/SoccerTwos/SoccerTwos/SoccerTwos-1499948.onnx
[INFO] SoccerTwos. Step: 1510000. Time Elapsed: 2997.946 s. Mean Reward: 0.000. Mean Group Reward: -0.300. Training. ELO: 1198.252.
[INFO] SoccerTwos. Step: 1520000. Time Elapsed: 3040.600 s. Mean Reward: 0.000. Mean Group Reward: -0.308. Training. ELO: 1197.910.
[INFO] SoccerTwos. Step: 1530000. Time Elapsed: 3052.576 s. Mean Reward: 0.000. Mean Group Reward: -0.182. Training. ELO: 1197.281.
[INFO] SoccerTwos. Step: 1540000. Time Elapsed: 3095.073 s. Mean Reward: 0.000. Mean Group Reward: -0.089. Training. ELO: 1197.418.
[INFO] SoccerTwos. Step: 1550000. Time Elapsed: 3113.675 s. Mean Reward: 0.000. Mean Group Reward: 0.112. Training. ELO: 1199.001.
[INFO] SoccerTwos. Step: 1560000. Time Elapsed: 3150.671 s. Mean Reward: 0.000. Mean Group Reward: -0.200. Training. ELO: 1198.491.
[INFO] SoccerTwos. Step: 1570000. Time Elapsed: 3166.129 s. Mean Reward: 0.000. Mean Group Reward: -0.166. Training. ELO: 1198.115.
[INFO] SoccerTwos. Step: 1580000. Time Elapsed: 3208.445 s. Mean Reward: 0.000. Mean Group Reward: -0.273. Training. ELO: 1196.494.
[INFO] SoccerTwos. Step: 1590000. Time Elapsed: 3220.387 s. Mean Reward: 0.000. Mean Group Reward: -0.200. Training. ELO: 1195.370.
[INFO] SoccerTwos. Step: 1600000. Time Elapsed: 3258.781 s. Mean Reward: 0.000. Mean Group Reward: -0.187. Training. ELO: 1195.999.
[INFO] SoccerTwos. Step: 1610000. Time Elapsed: 3274.931 s. Mean Reward: 0.000. Mean Group Reward: 0.113. Training. ELO: 1196.249.
[INFO] SoccerTwos. Step: 1620000. Time Elapsed: 3308.028 s. Mean Reward: 0.000. Mean Group Reward: 0.009. Training. ELO: 1197.506.
[INFO] SoccerTwos. Step: 1630000. Time Elapsed: 3322.794 s. Mean Reward: 0.000. Mean Group Reward: -0.238. Training. ELO: 1197.305.
[INFO] SoccerTwos. Step: 1640000. Time Elapsed: 3366.644 s. Mean Reward: 0.000. Mean Group Reward: 0.148. Training. ELO: 1197.275.
[INFO] SoccerTwos. Step: 1650000. Time Elapsed: 3375.529 s. Mean Reward: 0.000. Mean Group Reward: -0.333. Training. ELO: 1196.900.
[INFO] SoccerTwos. Step: 1660000. Time Elapsed: 3414.038 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1670000. Time Elapsed: 3435.768 s. Mean Reward: 0.000. Mean Group Reward: 0.111. Training. ELO: 1197.151.
[INFO] SoccerTwos. Step: 1680000. Time Elapsed: 3462.688 s. Mean Reward: 0.000. Mean Group Reward: 0.004. Training. ELO: 1198.023.
[INFO] SoccerTwos. Step: 1690000. Time Elapsed: 3484.400 s. Mean Reward: 0.000. Mean Group Reward: -0.475. Training. ELO: 1196.649.
[INFO] SoccerTwos. Step: 1700000. Time Elapsed: 3499.237 s. Mean Reward: 0.000. Mean Group Reward: -0.182. Training. ELO: 1195.330.
[INFO] SoccerTwos. Step: 1710000. Time Elapsed: 3547.338 s. Mean Reward: 0.000. Mean Group Reward: -0.048. Training. ELO: 1194.381.
[INFO] SoccerTwos. Step: 1720000. Time Elapsed: 3563.442 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training. ELO: 1193.582.
[INFO] SoccerTwos. Step: 1730000. Time Elapsed: 3606.927 s. Mean Reward: 0.000. Mean Group Reward: -0.009. Training. ELO: 1193.333.
[INFO] SoccerTwos. Step: 1740000. Time Elapsed: 3627.112 s. Mean Reward: 0.000. Mean Group Reward: -0.182. Training. ELO: 1192.592.
[INFO] SoccerTwos. Step: 1750000. Time Elapsed: 3669.166 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1760000. Time Elapsed: 3687.197 s. Mean Reward: 0.000. Mean Group Reward: -0.182. Training. ELO: 1191.605.
[INFO] SoccerTwos. Step: 1770000. Time Elapsed: 3724.930 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1780000. Time Elapsed: 3738.447 s. Mean Reward: 0.000. Mean Group Reward: -0.185. Training. ELO: 1191.486.
[INFO] SoccerTwos. Step: 1790000. Time Elapsed: 3787.265 s. Mean Reward: 0.000. Mean Group Reward: -0.271. Training. ELO: 1189.936.
[INFO] SoccerTwos. Step: 1800000. Time Elapsed: 3797.066 s. Mean Reward: 0.000. Mean Group Reward: 0.139. Training. ELO: 1190.447.
[INFO] SoccerTwos. Step: 1810000. Time Elapsed: 3840.001 s. Mean Reward: 0.000. Mean Group Reward: 0.077. Training. ELO: 1191.218.
[INFO] SoccerTwos. Step: 1820000. Time Elapsed: 3858.332 s. Mean Reward: 0.000. Mean Group Reward: -0.109. Training. ELO: 1190.600.
[INFO] SoccerTwos. Step: 1830000. Time Elapsed: 3897.578 s. Mean Reward: 0.000. Mean Group Reward: 0.213. Training. ELO: 1192.234.
[INFO] SoccerTwos. Step: 1840000. Time Elapsed: 3918.039 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1850000. Time Elapsed: 3928.874 s. Mean Reward: 0.000. Mean Group Reward: 0.226. Training. ELO: 1194.747.
[INFO] SoccerTwos. Step: 1860000. Time Elapsed: 3980.566 s. Mean Reward: 0.000. Mean Group Reward: 0.150. Training. ELO: 1195.503.
[INFO] SoccerTwos. Step: 1870000. Time Elapsed: 4006.048 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 1880000. Time Elapsed: 4040.715 s. Mean Reward: 0.000. Mean Group Reward: 0.013. Training. ELO: 1195.503.
[INFO] SoccerTwos. Step: 1890000. Time Elapsed: 4066.380 s. Mean Reward: 0.000. Mean Group Reward: -0.100. Training. ELO: 1194.759.
[INFO] SoccerTwos. Step: 1900000. Time Elapsed: 4113.390 s. Mean Reward: 0.000. Mean Group Reward: 0.047. Training. ELO: 1195.263.
[INFO] SoccerTwos. Step: 1910000. Time Elapsed: 4151.575 s. Mean Reward: 0.000. Mean Group Reward: -0.134. Training. ELO: 1194.616.
[INFO] SoccerTwos. Step: 1920000. Time Elapsed: 4186.020 s. Mean Reward: 0.000. Mean Group Reward: 0.038. Training. ELO: 1194.534.
[INFO] SoccerTwos. Step: 1930000. Time Elapsed: 4211.073 s. Mean Reward: 0.000. Mean Group Reward: -0.050. Training. ELO: 1194.534.
[INFO] SoccerTwos. Step: 1940000. Time Elapsed: 4257.686 s. Mean Reward: 0.000. Mean Group Reward: -0.060. Training. ELO: 1193.866.
[INFO] SoccerTwos. Step: 1950000. Time Elapsed: 4277.137 s. Mean Reward: 0.000. Mean Group Reward: -0.286. Training. ELO: 1194.525.
[INFO] SoccerTwos. Step: 1960000. Time Elapsed: 4325.382 s. Mean Reward: 0.000. Mean Group Reward: -0.222. Training. ELO: 1193.274.
[INFO] SoccerTwos. Step: 1970000. Time Elapsed: 4336.406 s. Mean Reward: 0.000. Mean Group Reward: -0.200. Training. ELO: 1192.526.
[INFO] SoccerTwos. Step: 1980000. Time Elapsed: 4391.114 s. Mean Reward: 0.000. Mean Group Reward: 0.086. Training. ELO: 1194.419.
[INFO] SoccerTwos. Step: 1990000. Time Elapsed: 4400.893 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 2000000. Time Elapsed: 4427.037 s. Mean Reward: 0.000. Mean Group Reward: -0.076. Training. ELO: 1195.426.
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py:164: UserWarning: Exporting a model while it is in training mode. Please ensure that this is intended, as it may lead to different behavior during inference. Calling model.eval() before export is recommended.
torch.onnx.export(
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py:164: UserWarning: # 'dynamic_axes' is not recommended when dynamo=True, and may lead to 'torch._dynamo.exc.UserError: Constraints violated.' Supply the 'dynamic_shapes' argument instead if export is unsuccessful.
torch.onnx.export(
W0213 22:33:48.332000 10240 torch/onnx/_internal/exporter/_compat.py:125] Setting ONNX exporter to use operator set version 18 because the requested opset_version 9 is a lower version than we have implementations for. Automatic version conversion will be performed, which may not be successful at converting to the requested version. If version conversion is unsuccessful, the opset version of the exported model will be kept at 18. Please consider setting opset_version >=18 to leverage latest ONNX features
W0213 22:33:48.983000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::nms
W0213 22:33:48.983000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::roi_align
W0213 22:33:48.984000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::roi_pool
[torch.onnx] Obtain model graph for `SimpleActor([...]` with `torch.export.export(..., strict=False)`...
[torch.onnx] Obtain model graph for `SimpleActor([...]` with `torch.export.export(..., strict=False)`... ✅
[torch.onnx] Run decomposition...
/usr/lib/python3.10/copyreg.py:101: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
return cls.__new__(cls, *args)
[torch.onnx] Run decomposition... ✅
[torch.onnx] Translate the graph into ONNX...
[torch.onnx] Translate the graph into ONNX... ✅
/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_onnx_program.py:460: UserWarning: # The axis name: batch will not be used, since it shares the same shape constraints with another axis: batch.
rename_mapping = _dynamic_shapes.create_rename_mapping(
The model version conversion is not supported by the onnxscript version converter and fallback is enabled. The model will be converted using the onnx C API (target version: 9).
Failed to convert the model to the target version 9 using the ONNX C API. The model was not modified
Traceback (most recent call last):
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/__init__.py", line 120, in call
converted_proto = _c_api_utils.call_onnx_api(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/_c_api_utils.py", line 65, in call_onnx_api
result = func(proto)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/__init__.py", line 115, in _partial_convert_version
return onnx.version_converter.convert_version(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnx/version_converter.py", line 38, in convert_version
converted_model_str = C.convert_version(model_str, target_version)
RuntimeError: /github/workspace/onnx/version_converter/BaseConverter.h:73: adapter_lookup: Assertion `false` failed: No Adapter From Version $15 for Shape
Applied 3 of general pattern rewrite rules.
Skipped deduplication of initializer 'version_number' as it is a graph input or output
Skipped deduplication of initializer 'memory_size' as it is a graph input or output
Skipped deduplication of initializer 'discrete_action_output_shape' as it is a graph input or output
[INFO] Exported results/SoccerTwos/SoccerTwos/SoccerTwos-1999930.onnx
[INFO] SoccerTwos. Step: 2010000. Time Elapsed: 4475.555 s. Mean Reward: 0.000. Mean Group Reward: -0.018. Training. ELO: 1195.550.
[INFO] SoccerTwos. Step: 2020000. Time Elapsed: 4491.870 s. Mean Reward: 0.000. Mean Group Reward: -0.275. Training. ELO: 1195.550.
[INFO] SoccerTwos. Step: 2030000. Time Elapsed: 4543.318 s. Mean Reward: 0.000. Mean Group Reward: 0.026. Training. ELO: 1195.550.
[INFO] SoccerTwos. Step: 2040000. Time Elapsed: 4554.358 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 2050000. Time Elapsed: 4605.964 s. Mean Reward: 0.000. Mean Group Reward: 0.051. Training. ELO: 1195.211.
[INFO] SoccerTwos. Step: 2060000. Time Elapsed: 4631.418 s. Mean Reward: 0.000. Mean Group Reward: -0.038. Training. ELO: 1195.044.
[INFO] SoccerTwos. Step: 2070000. Time Elapsed: 4665.457 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 2080000. Time Elapsed: 4691.970 s. Mean Reward: 0.000. Mean Group Reward: -0.200. Training. ELO: 1194.786.
[INFO] SoccerTwos. Step: 2090000. Time Elapsed: 4732.100 s. Mean Reward: 0.000. Mean Group Reward: 0.108. Training. ELO: 1194.532.
[INFO] SoccerTwos. Step: 2100000. Time Elapsed: 4754.041 s. Mean Reward: 0.000. Mean Group Reward: 0.097. Training. ELO: 1195.279.
[INFO] SoccerTwos. Step: 2110000. Time Elapsed: 4808.511 s. Mean Reward: 0.000. Mean Group Reward: -0.098. Training. ELO: 1195.528.
[INFO] SoccerTwos. Step: 2120000. Time Elapsed: 4830.627 s. Mean Reward: 0.000. Mean Group Reward: -0.182. Training.
[INFO] SoccerTwos. Step: 2130000. Time Elapsed: 4843.792 s. Mean Reward: 0.000. Mean Group Reward: 0.002. Training. ELO: 1195.528.
[INFO] SoccerTwos. Step: 2140000. Time Elapsed: 4892.978 s. Mean Reward: 0.000. Mean Group Reward: 0.050. Training. ELO: 1196.020.
[INFO] SoccerTwos. Step: 2150000. Time Elapsed: 4914.325 s. Mean Reward: 0.000. Mean Group Reward: -0.200. Training.
[INFO] SoccerTwos. Step: 2160000. Time Elapsed: 4957.129 s. Mean Reward: 0.000. Mean Group Reward: -0.248. Training. ELO: 1195.510.
[INFO] SoccerTwos. Step: 2170000. Time Elapsed: 4978.360 s. Mean Reward: 0.000. Mean Group Reward: -0.119. Training. ELO: 1195.510.
[INFO] SoccerTwos. Step: 2180000. Time Elapsed: 5023.181 s. Mean Reward: 0.000. Mean Group Reward: -0.293. Training. ELO: 1195.257.
[INFO] SoccerTwos. Step: 2190000. Time Elapsed: 5040.233 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 2200000. Time Elapsed: 5089.952 s. Mean Reward: 0.000. Mean Group Reward: -0.167. Training. ELO: 1193.753.
[INFO] SoccerTwos. Step: 2210000. Time Elapsed: 5103.206 s. Mean Reward: 0.000. Mean Group Reward: 0.050. Training. ELO: 1194.256.
[INFO] SoccerTwos. Step: 2220000. Time Elapsed: 5148.378 s. Mean Reward: 0.000. Mean Group Reward: 0.138. Training. ELO: 1195.253.
[INFO] SoccerTwos. Step: 2230000. Time Elapsed: 5169.718 s. Mean Reward: 0.000. Mean Group Reward: -0.462. Training. ELO: 1194.578.
[INFO] SoccerTwos. Step: 2240000. Time Elapsed: 5208.017 s. Mean Reward: 0.000. Mean Group Reward: -0.200. Training. ELO: 1194.494.
[INFO] SoccerTwos. Step: 2250000. Time Elapsed: 5226.148 s. Mean Reward: 0.000. Mean Group Reward: -0.196. Training. ELO: 1192.648.
[INFO] SoccerTwos. Step: 2260000. Time Elapsed: 5275.938 s. Mean Reward: 0.000. Mean Group Reward: 0.126. Training. ELO: 1191.510.
[INFO] SoccerTwos. Step: 2270000. Time Elapsed: 5299.409 s. Mean Reward: 0.000. Mean Group Reward: 0.112. Training. ELO: 1191.510.
[INFO] SoccerTwos. Step: 2280000. Time Elapsed: 5311.868 s. Mean Reward: 0.000. Mean Group Reward: 0.119. Training. ELO: 1192.266.
[INFO] SoccerTwos. Step: 2290000. Time Elapsed: 5360.091 s. Mean Reward: 0.000. Mean Group Reward: -0.462. Training. ELO: 1191.357.
[INFO] SoccerTwos. Step: 2300000. Time Elapsed: 5379.539 s. Mean Reward: 0.000. Mean Group Reward: 0.062. Training. ELO: 1191.288.
[INFO] SoccerTwos. Step: 2310000. Time Elapsed: 5424.478 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training. ELO: 1190.796.
[INFO] SoccerTwos. Step: 2320000. Time Elapsed: 5440.247 s. Mean Reward: 0.000. Mean Group Reward: -0.308. Training. ELO: 1190.549.
[INFO] SoccerTwos. Step: 2330000. Time Elapsed: 5479.669 s. Mean Reward: 0.000. Mean Group Reward: -0.186. Training. ELO: 1190.900.
[INFO] SoccerTwos. Step: 2340000. Time Elapsed: 5496.419 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training.
[INFO] SoccerTwos. Step: 2350000. Time Elapsed: 5535.646 s. Mean Reward: 0.000. Mean Group Reward: -0.333. Training. ELO: 1189.703.
[INFO] SoccerTwos. Step: 2360000. Time Elapsed: 5550.054 s. Mean Reward: 0.000. Mean Group Reward: -0.007. Training. ELO: 1189.579.
[INFO] SoccerTwos. Step: 2370000. Time Elapsed: 5592.199 s. Mean Reward: 0.000. Mean Group Reward: -0.017. Training. ELO: 1189.579.
[INFO] SoccerTwos. Step: 2380000. Time Elapsed: 5601.618 s. Mean Reward: 0.000. Mean Group Reward: -0.200. Training. ELO: 1188.838.
[INFO] SoccerTwos. Step: 2390000. Time Elapsed: 5649.403 s. Mean Reward: 0.000. Mean Group Reward: 0.150. Training. ELO: 1190.813.
[INFO] SoccerTwos. Step: 2400000. Time Elapsed: 5664.792 s. Mean Reward: 0.000. Mean Group Reward: 0.000. Training. ELO: 1192.625.
[INFO] SoccerTwos. Step: 2410000. Time Elapsed: 5677.336 s. Mean Reward: 0.000. Mean Group Reward: 0.095. Training.
[INFO] SoccerTwos. Step: 2420000. Time Elapsed: 5718.563 s. Mean Reward: 0.000. Mean Group Reward: -0.308. Training. ELO: 1192.625.
[INFO] SoccerTwos. Step: 2430000. Time Elapsed: 5729.905 s. Mean Reward: 0.000. Mean Group Reward: -0.250. Training. ELO: 1192.622.
[INFO] SoccerTwos. Step: 2440000. Time Elapsed: 5775.453 s. Mean Reward: 0.000. Mean Group Reward: -0.385. Training. ELO: 1191.388.
[INFO] SoccerTwos. Step: 2450000. Time Elapsed: 5790.669 s. Mean Reward: 0.000. Mean Group Reward: -0.073. Training. ELO: 1191.029.
[INFO] SoccerTwos. Step: 2460000. Time Elapsed: 5828.209 s. Mean Reward: 0.000. Mean Group Reward: 0.270. Training. ELO: 1192.030.
[INFO] SoccerTwos. Step: 2470000. Time Elapsed: 5848.330 s. Mean Reward: 0.000. Mean Group Reward: 0.059. Training. ELO: 1193.168.
[INFO] SoccerTwos. Step: 2480000. Time Elapsed: 5884.775 s. Mean Reward: 0.000. Mean Group Reward: 0.180. Training. ELO: 1194.160.
[INFO] SoccerTwos. Step: 2490000. Time Elapsed: 5910.214 s. Mean Reward: 0.000. Mean Group Reward: 0.039. Training. ELO: 1197.235.
[INFO] SoccerTwos. Step: 2500000. Time Elapsed: 5946.050 s. Mean Reward: 0.000. Mean Group Reward: 0.060. Training. ELO: 1198.824.
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py:164: UserWarning: Exporting a model while it is in training mode. Please ensure that this is intended, as it may lead to different behavior during inference. Calling model.eval() before export is recommended.
torch.onnx.export(
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py:164: UserWarning: # 'dynamic_axes' is not recommended when dynamo=True, and may lead to 'torch._dynamo.exc.UserError: Constraints violated.' Supply the 'dynamic_shapes' argument instead if export is unsuccessful.
torch.onnx.export(
W0213 22:59:07.324000 10240 torch/onnx/_internal/exporter/_compat.py:125] Setting ONNX exporter to use operator set version 18 because the requested opset_version 9 is a lower version than we have implementations for. Automatic version conversion will be performed, which may not be successful at converting to the requested version. If version conversion is unsuccessful, the opset version of the exported model will be kept at 18. Please consider setting opset_version >=18 to leverage latest ONNX features
W0213 22:59:07.880000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::nms
W0213 22:59:07.880000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::roi_align
W0213 22:59:07.880000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::roi_pool
[torch.onnx] Obtain model graph for `SimpleActor([...]` with `torch.export.export(..., strict=False)`...
[torch.onnx] Obtain model graph for `SimpleActor([...]` with `torch.export.export(..., strict=False)`... ✅
[torch.onnx] Run decomposition...
/usr/lib/python3.10/copyreg.py:101: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
return cls.__new__(cls, *args)
[torch.onnx] Run decomposition... ✅
[torch.onnx] Translate the graph into ONNX...
[torch.onnx] Translate the graph into ONNX... ✅
/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_onnx_program.py:460: UserWarning: # The axis name: batch will not be used, since it shares the same shape constraints with another axis: batch.
rename_mapping = _dynamic_shapes.create_rename_mapping(
The model version conversion is not supported by the onnxscript version converter and fallback is enabled. The model will be converted using the onnx C API (target version: 9).
Failed to convert the model to the target version 9 using the ONNX C API. The model was not modified
Traceback (most recent call last):
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/__init__.py", line 120, in call
converted_proto = _c_api_utils.call_onnx_api(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/_c_api_utils.py", line 65, in call_onnx_api
result = func(proto)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/__init__.py", line 115, in _partial_convert_version
return onnx.version_converter.convert_version(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnx/version_converter.py", line 38, in convert_version
converted_model_str = C.convert_version(model_str, target_version)
RuntimeError: /github/workspace/onnx/version_converter/BaseConverter.h:73: adapter_lookup: Assertion `false` failed: No Adapter From Version $15 for Shape
Applied 3 of general pattern rewrite rules.
Skipped deduplication of initializer 'version_number' as it is a graph input or output
Skipped deduplication of initializer 'memory_size' as it is a graph input or output
Skipped deduplication of initializer 'discrete_action_output_shape' as it is a graph input or output
[INFO] Exported results/SoccerTwos/SoccerTwos/SoccerTwos-2499340.onnx
[INFO] SoccerTwos. Step: 2510000. Time Elapsed: 5974.054 s. Mean Reward: 0.000. Mean Group Reward: 0.268. Training. ELO: 1203.143.
[INFO] SoccerTwos. Step: 2520000. Time Elapsed: 6013.930 s. Mean Reward: 0.000. Mean Group Reward: 0.113. Training. ELO: 1206.464.
[INFO] SoccerTwos. Step: 2530000. Time Elapsed: 6030.411 s. Mean Reward: 0.000. Mean Group Reward: 0.247. Training. ELO: 1205.817.
[INFO] SoccerTwos. Step: 2540000. Time Elapsed: 6069.553 s. Mean Reward: 0.000. Mean Group Reward: -0.225. Training. ELO: 1205.143.
[INFO] SoccerTwos. Step: 2550000. Time Elapsed: 6080.882 s. Mean Reward: 0.000. Mean Group Reward: -0.333. Training. ELO: 1203.869.
[INFO] SoccerTwos. Step: 2560000. Time Elapsed: 6121.552 s. Mean Reward: 0.000. Mean Group Reward: 0.165. Training. ELO: 1203.553.
[INFO] SoccerTwos. Step: 2570000. Time Elapsed: 6139.386 s. Mean Reward: 0.000. Mean Group Reward: -0.134. Training. ELO: 1204.059.
[INFO] SoccerTwos. Step: 2580000. Time Elapsed: 6181.637 s. Mean Reward: 0.000. Mean Group Reward: -0.182. Training. ELO: 1203.289.
[INFO] SoccerTwos. Step: 2590000. Time Elapsed: 6192.637 s. Mean Reward: 0.000. Mean Group Reward: -0.008. Training. ELO: 1203.033.
[INFO] SoccerTwos. Step: 2600000. Time Elapsed: 6210.991 s. Mean Reward: 0.000. Mean Group Reward: -0.212. Training. ELO: 1203.616.
[INFO] SoccerTwos. Step: 2610000. Time Elapsed: 6247.396 s. Mean Reward: 0.000. Mean Group Reward: -0.180. Training. ELO: 1203.444.
[INFO] SoccerTwos. Step: 2620000. Time Elapsed: 6264.068 s. Mean Reward: 0.000. Mean Group Reward: 0.286. Training. ELO: 1203.201.
[INFO] SoccerTwos. Step: 2630000. Time Elapsed: 6297.036 s. Mean Reward: 0.000. Mean Group Reward: 0.379. Training. ELO: 1205.369.
[INFO] SoccerTwos. Step: 2640000. Time Elapsed: 6316.568 s. Mean Reward: 0.000. Mean Group Reward: 0.174. Training. ELO: 1207.984.
[INFO] SoccerTwos. Step: 2650000. Time Elapsed: 6353.386 s. Mean Reward: 0.000. Mean Group Reward: 0.232. Training. ELO: 1209.551.
[INFO] SoccerTwos. Step: 2660000. Time Elapsed: 6374.162 s. Mean Reward: 0.000. Mean Group Reward: -0.254. Training. ELO: 1209.551.
[INFO] SoccerTwos. Step: 2670000. Time Elapsed: 6417.928 s. Mean Reward: 0.000. Mean Group Reward: 0.580. Training. ELO: 1213.125.
[INFO] SoccerTwos. Step: 2680000. Time Elapsed: 6435.016 s. Mean Reward: 0.000. Mean Group Reward: 0.403. Training. ELO: 1217.920.
[INFO] SoccerTwos. Step: 2690000. Time Elapsed: 6472.235 s. Mean Reward: 0.000. Mean Group Reward: -0.146. Training. ELO: 1220.356.
[INFO] SoccerTwos. Step: 2700000. Time Elapsed: 6488.340 s. Mean Reward: 0.000. Mean Group Reward: 0.124. Training. ELO: 1223.457.
[INFO] SoccerTwos. Step: 2710000. Time Elapsed: 6543.565 s. Mean Reward: 0.000. Mean Group Reward: -0.070. Training. ELO: 1225.189.
[INFO] SoccerTwos. Step: 2720000. Time Elapsed: 6565.391 s. Mean Reward: 0.000. Mean Group Reward: -0.376. Training. ELO: 1228.011.
[INFO] SoccerTwos. Step: 2730000. Time Elapsed: 6611.561 s. Mean Reward: 0.000. Mean Group Reward: 0.107. Training. ELO: 1229.428.
[INFO] SoccerTwos. Step: 2740000. Time Elapsed: 6625.589 s. Mean Reward: 0.000. Mean Group Reward: -0.097. Training. ELO: 1230.843.
[INFO] SoccerTwos. Step: 2750000. Time Elapsed: 6681.435 s. Mean Reward: 0.000. Mean Group Reward: -0.230. Training. ELO: 1230.591.
[INFO] SoccerTwos. Step: 2760000. Time Elapsed: 6695.447 s. Mean Reward: 0.000. Mean Group Reward: -0.009. Training. ELO: 1229.713.
[INFO] SoccerTwos. Step: 2770000. Time Elapsed: 6752.450 s. Mean Reward: 0.000. Mean Group Reward: -0.004. Training. ELO: 1231.979.
[INFO] SoccerTwos. Step: 2780000. Time Elapsed: 6767.469 s. Mean Reward: 0.000. Mean Group Reward: 0.072. Training. ELO: 1232.281.
[INFO] SoccerTwos. Step: 2790000. Time Elapsed: 6784.660 s. Mean Reward: 0.000. Mean Group Reward: 0.238. Training. ELO: 1233.873.
[INFO] SoccerTwos. Step: 2800000. Time Elapsed: 6829.358 s. Mean Reward: 0.000. Mean Group Reward: 0.175. Training. ELO: 1236.065.
[INFO] SoccerTwos. Step: 2810000. Time Elapsed: 6851.401 s. Mean Reward: 0.000. Mean Group Reward: -0.017. Training. ELO: 1238.278.
[INFO] SoccerTwos. Step: 2820000. Time Elapsed: 6896.798 s. Mean Reward: 0.000. Mean Group Reward: -0.489. Training. ELO: 1242.890.
[INFO] SoccerTwos. Step: 2830000. Time Elapsed: 6915.641 s. Mean Reward: 0.000. Mean Group Reward: 0.054. Training. ELO: 1243.541.
[INFO] SoccerTwos. Step: 2840000. Time Elapsed: 6965.545 s. Mean Reward: 0.000. Mean Group Reward: 0.301. Training. ELO: 1244.889.
[INFO] SoccerTwos. Step: 2850000. Time Elapsed: 6979.418 s. Mean Reward: 0.000. Mean Group Reward: -0.298. Training. ELO: 1250.239.
[INFO] SoccerTwos. Step: 2860000. Time Elapsed: 7025.624 s. Mean Reward: 0.000. Mean Group Reward: 0.190. Training. ELO: 1253.383.
[INFO] SoccerTwos. Step: 2870000. Time Elapsed: 7044.851 s. Mean Reward: 0.000. Mean Group Reward: -0.102. Training. ELO: 1254.130.
[INFO] SoccerTwos. Step: 2880000. Time Elapsed: 7100.792 s. Mean Reward: 0.000. Mean Group Reward: 0.180. Training. ELO: 1254.938.
[INFO] SoccerTwos. Step: 2890000. Time Elapsed: 7119.609 s. Mean Reward: 0.000. Mean Group Reward: -0.280. Training. ELO: 1255.735.
[INFO] SoccerTwos. Step: 2900000. Time Elapsed: 7180.629 s. Mean Reward: 0.000. Mean Group Reward: -0.059. Training. ELO: 1256.031.
[INFO] SoccerTwos. Step: 2910000. Time Elapsed: 7201.814 s. Mean Reward: 0.000. Mean Group Reward: 0.205. Training. ELO: 1264.740.
[INFO] SoccerTwos. Step: 2920000. Time Elapsed: 7270.949 s. Mean Reward: 0.000. Mean Group Reward: 0.194. Training. ELO: 1273.022.
[INFO] SoccerTwos. Step: 2930000. Time Elapsed: 7291.452 s. Mean Reward: 0.000. Mean Group Reward: 0.127. Training. ELO: 1277.094.
[INFO] SoccerTwos. Step: 2940000. Time Elapsed: 7356.019 s. Mean Reward: 0.000. Mean Group Reward: -0.416. Training. ELO: 1278.161.
[INFO] SoccerTwos. Step: 2950000. Time Elapsed: 7372.666 s. Mean Reward: 0.000. Mean Group Reward: 0.114. Training. ELO: 1276.528.
[INFO] SoccerTwos. Step: 2960000. Time Elapsed: 7436.433 s. Mean Reward: 0.000. Mean Group Reward: 0.036. Training. ELO: 1276.130.
[INFO] SoccerTwos. Step: 2970000. Time Elapsed: 7447.685 s. Mean Reward: 0.000. Mean Group Reward: 0.514. Training. ELO: 1286.586.
[INFO] SoccerTwos. Step: 2980000. Time Elapsed: 7507.719 s. Mean Reward: 0.000. Mean Group Reward: -0.108. Training. ELO: 1291.570.
[INFO] SoccerTwos. Step: 2990000. Time Elapsed: 7524.573 s. Mean Reward: 0.000. Mean Group Reward: 0.178. Training. ELO: 1290.603.
[INFO] SoccerTwos. Step: 3000000. Time Elapsed: 7543.765 s. Mean Reward: 0.000. Mean Group Reward: 0.216. Training. ELO: 1299.462.
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py:164: UserWarning: Exporting a model while it is in training mode. Please ensure that this is intended, as it may lead to different behavior during inference. Calling model.eval() before export is recommended.
torch.onnx.export(
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py:164: UserWarning: # 'dynamic_axes' is not recommended when dynamo=True, and may lead to 'torch._dynamo.exc.UserError: Constraints violated.' Supply the 'dynamic_shapes' argument instead if export is unsuccessful.
torch.onnx.export(
W0213 23:25:45.039000 10240 torch/onnx/_internal/exporter/_compat.py:125] Setting ONNX exporter to use operator set version 18 because the requested opset_version 9 is a lower version than we have implementations for. Automatic version conversion will be performed, which may not be successful at converting to the requested version. If version conversion is unsuccessful, the opset version of the exported model will be kept at 18. Please consider setting opset_version >=18 to leverage latest ONNX features
W0213 23:25:45.622000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::nms
W0213 23:25:45.622000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::roi_align
W0213 23:25:45.623000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::roi_pool
[torch.onnx] Obtain model graph for `SimpleActor([...]` with `torch.export.export(..., strict=False)`...
[torch.onnx] Obtain model graph for `SimpleActor([...]` with `torch.export.export(..., strict=False)`... ✅
[torch.onnx] Run decomposition...
/usr/lib/python3.10/copyreg.py:101: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
return cls.__new__(cls, *args)
[torch.onnx] Run decomposition... ✅
[torch.onnx] Translate the graph into ONNX...
[torch.onnx] Translate the graph into ONNX... ✅
/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_onnx_program.py:460: UserWarning: # The axis name: batch will not be used, since it shares the same shape constraints with another axis: batch.
rename_mapping = _dynamic_shapes.create_rename_mapping(
The model version conversion is not supported by the onnxscript version converter and fallback is enabled. The model will be converted using the onnx C API (target version: 9).
Failed to convert the model to the target version 9 using the ONNX C API. The model was not modified
Traceback (most recent call last):
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/__init__.py", line 120, in call
converted_proto = _c_api_utils.call_onnx_api(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/_c_api_utils.py", line 65, in call_onnx_api
result = func(proto)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/__init__.py", line 115, in _partial_convert_version
return onnx.version_converter.convert_version(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnx/version_converter.py", line 38, in convert_version
converted_model_str = C.convert_version(model_str, target_version)
RuntimeError: /github/workspace/onnx/version_converter/BaseConverter.h:73: adapter_lookup: Assertion `false` failed: No Adapter From Version $15 for Shape
Applied 3 of general pattern rewrite rules.
Skipped deduplication of initializer 'version_number' as it is a graph input or output
Skipped deduplication of initializer 'memory_size' as it is a graph input or output
Skipped deduplication of initializer 'discrete_action_output_shape' as it is a graph input or output
[INFO] Exported results/SoccerTwos/SoccerTwos/SoccerTwos-2999870.onnx
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py:164: UserWarning: Exporting a model while it is in training mode. Please ensure that this is intended, as it may lead to different behavior during inference. Calling model.eval() before export is recommended.
torch.onnx.export(
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py:164: UserWarning: # 'dynamic_axes' is not recommended when dynamo=True, and may lead to 'torch._dynamo.exc.UserError: Constraints violated.' Supply the 'dynamic_shapes' argument instead if export is unsuccessful.
torch.onnx.export(
W0213 23:25:46.795000 10240 torch/onnx/_internal/exporter/_compat.py:125] Setting ONNX exporter to use operator set version 18 because the requested opset_version 9 is a lower version than we have implementations for. Automatic version conversion will be performed, which may not be successful at converting to the requested version. If version conversion is unsuccessful, the opset version of the exported model will be kept at 18. Please consider setting opset_version >=18 to leverage latest ONNX features
W0213 23:25:47.616000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::nms
W0213 23:25:47.616000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::roi_align
W0213 23:25:47.617000 10240 torch/onnx/_internal/exporter/_registration.py:110] torchvision is not installed. Skipping torchvision::roi_pool
[torch.onnx] Obtain model graph for `SimpleActor([...]` with `torch.export.export(..., strict=False)`...
[torch.onnx] Obtain model graph for `SimpleActor([...]` with `torch.export.export(..., strict=False)`... ✅
[torch.onnx] Run decomposition...
/usr/lib/python3.10/copyreg.py:101: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
return cls.__new__(cls, *args)
[torch.onnx] Run decomposition... ✅
[torch.onnx] Translate the graph into ONNX...
[torch.onnx] Translate the graph into ONNX... ✅
/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_onnx_program.py:460: UserWarning: # The axis name: batch will not be used, since it shares the same shape constraints with another axis: batch.
rename_mapping = _dynamic_shapes.create_rename_mapping(
The model version conversion is not supported by the onnxscript version converter and fallback is enabled. The model will be converted using the onnx C API (target version: 9).
Failed to convert the model to the target version 9 using the ONNX C API. The model was not modified
Traceback (most recent call last):
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/__init__.py", line 120, in call
converted_proto = _c_api_utils.call_onnx_api(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/_c_api_utils.py", line 65, in call_onnx_api
result = func(proto)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnxscript/version_converter/__init__.py", line 115, in _partial_convert_version
return onnx.version_converter.convert_version(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/onnx/version_converter.py", line 38, in convert_version
converted_model_str = C.convert_version(model_str, target_version)
RuntimeError: /github/workspace/onnx/version_converter/BaseConverter.h:73: adapter_lookup: Assertion `false` failed: No Adapter From Version $15 for Shape
Applied 3 of general pattern rewrite rules.
Skipped deduplication of initializer 'version_number' as it is a graph input or output
Skipped deduplication of initializer 'memory_size' as it is a graph input or output
Skipped deduplication of initializer 'discrete_action_output_shape' as it is a graph input or output
[INFO] Exported results/SoccerTwos/SoccerTwos/SoccerTwos-3000202.onnx
[INFO] Copied results/SoccerTwos/SoccerTwos/SoccerTwos-3000202.onnx to results/SoccerTwos/SoccerTwos.onnx.
Push the agent to the 🤗 Hub¶
Now that training is done, we push our model to the Hub to:
- Complete the certification requirement
- Enter the AI vs AI challenge leaderboard
- Watch it play against others in the browser
We use Kaggle Secrets to safely access the HuggingFace token (no hardcoded tokens!).
Make sure you added HF_TOKEN as a Kaggle Secret before running this section.
In [23]:
# from huggingface_hub import notebook_login
# notebook_login()
In [24]:
!git config --global credential.helper store
In [26]:
import os
from kaggle_secrets import UserSecretsClient
from huggingface_hub import HfApi, login
# Fetch token from Kaggle Secrets (add HF_TOKEN in Kaggle sidebar > Add-ons > Secrets)
user_secrets = UserSecretsClient()
hf_token = user_secrets.get_secret("HF_TOKEN")
# Login
login(token=hf_token, add_to_git_credential=True)
# Verify identity and permissions
api = HfApi()
user_info = api.whoami()
print(f"Successfully logged in as: {user_info['name']}")
print(f"Token has Write access: {user_info['auth']['type'] == 'WRITE'}")
Successfully logged in as: Chiz Token has Write access: False
We define 4 parameters for the push:
--run-id— the name of the training run id (must match what you used above)--local-dir— where the agent was saved:results/<run_id>--repo-id— your HuggingFace repo:<your_username>/<repo_name>(auto-created if it doesn't exist)--commit-message— git commit message
Edit the
--repo-idbelow to use your own HuggingFace username!
In [29]:
!/kaggle/working/mlagents-env/bin/pip install huggingface_hub
Collecting huggingface_hub Using cached huggingface_hub-1.4.1-py3-none-any.whl.metadata (13 kB) Requirement already satisfied: filelock in ./mlagents-env/lib/python3.10/site-packages (from huggingface_hub) (3.21.2) Requirement already satisfied: fsspec>=2023.5.0 in ./mlagents-env/lib/python3.10/site-packages (from huggingface_hub) (2026.2.0) Collecting hf-xet<2.0.0,>=1.2.0 (from huggingface_hub) Downloading hf_xet-1.2.0-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.9 kB) Collecting httpx<1,>=0.23.0 (from huggingface_hub) Downloading httpx-0.28.1-py3-none-any.whl.metadata (7.1 kB) Requirement already satisfied: packaging>=20.9 in ./mlagents-env/lib/python3.10/site-packages (from huggingface_hub) (26.0) Requirement already satisfied: pyyaml>=5.1 in ./mlagents-env/lib/python3.10/site-packages (from huggingface_hub) (6.0.3) Collecting shellingham (from huggingface_hub) Downloading shellingham-1.5.4-py2.py3-none-any.whl.metadata (3.5 kB) Collecting tqdm>=4.42.1 (from huggingface_hub) Downloading tqdm-4.67.3-py3-none-any.whl.metadata (57 kB) Collecting typer-slim (from huggingface_hub) Downloading typer_slim-0.23.1-py3-none-any.whl.metadata (4.2 kB) Requirement already satisfied: typing-extensions>=4.1.0 in ./mlagents-env/lib/python3.10/site-packages (from huggingface_hub) (4.15.0) Collecting anyio (from httpx<1,>=0.23.0->huggingface_hub) Downloading anyio-4.12.1-py3-none-any.whl.metadata (4.3 kB) Collecting certifi (from httpx<1,>=0.23.0->huggingface_hub) Downloading certifi-2026.1.4-py3-none-any.whl.metadata (2.5 kB) Collecting httpcore==1.* (from httpx<1,>=0.23.0->huggingface_hub) Downloading httpcore-1.0.9-py3-none-any.whl.metadata (21 kB) Collecting idna (from httpx<1,>=0.23.0->huggingface_hub) Downloading idna-3.11-py3-none-any.whl.metadata (8.4 kB) Collecting h11>=0.16 (from httpcore==1.*->httpx<1,>=0.23.0->huggingface_hub) Downloading h11-0.16.0-py3-none-any.whl.metadata (8.3 kB) Collecting exceptiongroup>=1.0.2 (from anyio->httpx<1,>=0.23.0->huggingface_hub) Downloading exceptiongroup-1.3.1-py3-none-any.whl.metadata (6.7 kB) Collecting typer>=0.23.1 (from typer-slim->huggingface_hub) Downloading typer-0.23.1-py3-none-any.whl.metadata (16 kB) Collecting click>=8.0.0 (from typer>=0.23.1->typer-slim->huggingface_hub) Downloading click-8.3.1-py3-none-any.whl.metadata (2.6 kB) Collecting rich>=10.11.0 (from typer>=0.23.1->typer-slim->huggingface_hub) Downloading rich-14.3.2-py3-none-any.whl.metadata (18 kB) Collecting annotated-doc>=0.0.2 (from typer>=0.23.1->typer-slim->huggingface_hub) Downloading annotated_doc-0.0.4-py3-none-any.whl.metadata (6.6 kB) Collecting markdown-it-py>=2.2.0 (from rich>=10.11.0->typer>=0.23.1->typer-slim->huggingface_hub) Downloading markdown_it_py-4.0.0-py3-none-any.whl.metadata (7.3 kB) Collecting pygments<3.0.0,>=2.13.0 (from rich>=10.11.0->typer>=0.23.1->typer-slim->huggingface_hub) Downloading pygments-2.19.2-py3-none-any.whl.metadata (2.5 kB) Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich>=10.11.0->typer>=0.23.1->typer-slim->huggingface_hub) Downloading mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB) Downloading huggingface_hub-1.4.1-py3-none-any.whl (553 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 553.3/553.3 kB 10.8 MB/s 0:00:00 Downloading hf_xet-1.2.0-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 62.5 MB/s 0:00:00 Downloading httpx-0.28.1-py3-none-any.whl (73 kB) Downloading httpcore-1.0.9-py3-none-any.whl (78 kB) Downloading h11-0.16.0-py3-none-any.whl (37 kB) Downloading tqdm-4.67.3-py3-none-any.whl (78 kB) Downloading anyio-4.12.1-py3-none-any.whl (113 kB) Downloading exceptiongroup-1.3.1-py3-none-any.whl (16 kB) Downloading idna-3.11-py3-none-any.whl (71 kB) Downloading certifi-2026.1.4-py3-none-any.whl (152 kB) Downloading shellingham-1.5.4-py2.py3-none-any.whl (9.8 kB) Downloading typer_slim-0.23.1-py3-none-any.whl (3.4 kB) Downloading typer-0.23.1-py3-none-any.whl (56 kB) Downloading annotated_doc-0.0.4-py3-none-any.whl (5.3 kB) Downloading click-8.3.1-py3-none-any.whl (108 kB) Downloading rich-14.3.2-py3-none-any.whl (309 kB) Downloading pygments-2.19.2-py3-none-any.whl (1.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 57.3 MB/s 0:00:00 Downloading markdown_it_py-4.0.0-py3-none-any.whl (87 kB) Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB) Installing collected packages: tqdm, shellingham, pygments, mdurl, idna, hf-xet, h11, exceptiongroup, click, certifi, annotated-doc, markdown-it-py, httpcore, anyio, rich, httpx, typer, typer-slim, huggingface_hub ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19/19 [huggingface_hub] [huggingface_hub] ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. mlagents 1.1.0 requires onnx==1.15.0, but you have onnx 1.17.0 which is incompatible. Successfully installed annotated-doc-0.0.4 anyio-4.12.1 certifi-2026.1.4 click-8.3.1 exceptiongroup-1.3.1 h11-0.16.0 hf-xet-1.2.0 httpcore-1.0.9 httpx-0.28.1 huggingface_hub-1.4.1 idna-3.11 markdown-it-py-4.0.0 mdurl-0.1.2 pygments-2.19.2 rich-14.3.2 shellingham-1.5.4 tqdm-4.67.3 typer-0.23.1 typer-slim-0.23.1
In [30]:
# Replace YOUR_HF_USERNAME with your actual HuggingFace username (e.g. "Chiz")
!/kaggle/working/mlagents-env/bin/mlagents-push-to-hf \
--run-id="SoccerTwos" \
--local-dir="./results/SoccerTwos" \
--repo-id="Chiz/poca-SoccerTwos" \
--commit-message="Unit 7 SoccerTwos MA-POCA"
[INFO] This function will create a model card and upload your SoccerTwos into HuggingFace Hub. This is a work in progress: If you encounter a bug, please send open an issue [INFO] Pushing repo SoccerTwos to the Hugging Face Hub Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B ...occerTwos/SoccerTwos-0.pt: 1%|▏ | 14.0B / 1.29kB ...s/SoccerTwos-1499948.onnx: 1%|▏ | 679B / 61.0kB ...cerTwos-1499948.onnx.data: 1%|▏ | 19.6kB / 1.76MB ...rTwos/SoccerTwos-15406.pt: 1%|▏ | 106kB / 9.48MB ...s/SoccerTwos-1999930.onnx: 1%|▏ | 679B / 61.0kB ...cerTwos-1999930.onnx.data: 1%|▏ | 19.6kB / 1.76MB ...s/SoccerTwos-2499340.onnx: 1%|▏ | 679B / 61.0kB ...cerTwos-2499340.onnx.data: 1%|▏ | 19.6kB / 1.76MB ...s/SoccerTwos-2999870.onnx: 1%|▏ | 679B / 61.0kB ...cerTwos-2999870.onnx.data: 1%|▏ | 19.6kB / 1.76MB ...occerTwos/SoccerTwos-0.pt: 1%|▏ | 14.0B / 1.29kB ...s/SoccerTwos-1499948.onnx: 1%|▏ | 679B / 61.0kB ...cerTwos-1499948.onnx.data: 1%|▏ | 19.6kB / 1.76MB ...rTwos/SoccerTwos-15406.pt: 1%|▏ | 106kB / 9.48MB ...s/SoccerTwos-1999930.onnx: 1%|▏ | 679B / 61.0kB ...cerTwos-1999930.onnx.data: 1%|▏ | 19.6kB / 1.76MB ...s/SoccerTwos-2499340.onnx: 1%|▏ | 679B / 61.0kB ...cerTwos-2499340.onnx.data: 1%|▏ | 19.6kB / 1.76MB ...s/SoccerTwos-2999870.onnx: 1%|▏ | 679B / 61.0kB Processing Files (0 / 21) : 0%| | 820kB / 195MB, 1.37MB/s New Data Upload : 1%| | 587kB / 110MB, 979kB/s ...occerTwos/SoccerTwos-0.pt: 13%|█▊ | 172B / 1.29kB ...s/SoccerTwos-1499948.onnx: 13%|█▊ | 8.15kB / 61.0kB ...cerTwos-1499948.onnx.data: 13%|█▊ | 235kB / 1.76MB ...rTwos/SoccerTwos-15406.pt: 13%|█▊ | 1.27MB / 9.48MB ...s/SoccerTwos-1999930.onnx: 13%|█▊ | 8.15kB / 61.0kB ...cerTwos-1999930.onnx.data: 13%|█▊ | 235kB / 1.76MB ...s/SoccerTwos-2499340.onnx: 13%|█▊ | 8.15kB / 61.0kB ...cerTwos-2499340.onnx.data: 13%|█▊ | 235kB / 1.76MB ...s/SoccerTwos-2999870.onnx: 13%|█▊ | 8.15kB / 61.0kB Processing Files (0 / 24) : 8%|█▏ | 15.9MB / 195MB, 19.9MB/s New Data Upload : 8%|█ | 13.1MB / 174MB, 16.4MB/s ...occerTwos/SoccerTwos-0.pt: 70%|█████████▊ | 906B / 1.29kB ...s/SoccerTwos-1499948.onnx: 70%|█████████▊ | 42.8kB / 61.0kB ...cerTwos-1499948.onnx.data: 70%|█████████▊ | 1.23MB / 1.76MB ...rTwos/SoccerTwos-15406.pt: 70%|█████████▊ | 6.65MB / 9.48MB ...s/SoccerTwos-1999930.onnx: 70%|█████████▊ | 42.8kB / 61.0kB ...cerTwos-1999930.onnx.data: 70%|█████████▊ | 1.23MB / 1.76MB ...s/SoccerTwos-2499340.onnx: 70%|█████████▊ | 42.8kB / 61.0kB ...cerTwos-2499340.onnx.data: 70%|█████████▊ | 1.23MB / 1.76MB ...s/SoccerTwos-2999870.onnx: 70%|█████████▊ | 42.8kB / 61.0kB Processing Files (0 / 24) : 46%|██████▍ | 89.3MB / 195MB, 89.3MB/s New Data Upload : 43%|██████ | 74.6MB / 174MB, 74.6MB/s ...occerTwos/SoccerTwos-0.pt: 99%|█████████████▊| 1.28kB / 1.29kB ...s/SoccerTwos-1499948.onnx: 99%|█████████████▉| 60.5kB / 61.0kB ...cerTwos-1499948.onnx.data: 99%|█████████████▉| 1.74MB / 1.76MB ...rTwos/SoccerTwos-15406.pt: 99%|█████████████▉| 9.40MB / 9.48MB ...s/SoccerTwos-1999930.onnx: 99%|█████████████▉| 60.5kB / 61.0kB ...cerTwos-1999930.onnx.data: 99%|█████████████▉| 1.74MB / 1.76MB ...s/SoccerTwos-2499340.onnx: 99%|█████████████▉| 60.5kB / 61.0kB ...cerTwos-2499340.onnx.data: 99%|█████████████▉| 1.74MB / 1.76MB ...s/SoccerTwos-2999870.onnx: 99%|█████████████▉| 60.5kB / 61.0kB Processing Files (0 / 24) : 84%|███████████▊ | 164MB / 195MB, 137MB/s New Data Upload : 82%|███████████▌ | 143MB / 174MB, 119MB/s ...occerTwos/SoccerTwos-0.pt: 99%|█████████████▊| 1.28kB / 1.29kB ...s/SoccerTwos-1499948.onnx: 99%|█████████████▉| 60.5kB / 61.0kB ...cerTwos-1499948.onnx.data: 99%|█████████████▉| 1.74MB / 1.76MB ...rTwos/SoccerTwos-15406.pt: 99%|█████████████▉| 9.40MB / 9.48MB ...s/SoccerTwos-1999930.onnx: 99%|█████████████▉| 60.5kB / 61.0kB ...cerTwos-1999930.onnx.data: 99%|█████████████▉| 1.74MB / 1.76MB ...s/SoccerTwos-2499340.onnx: 99%|█████████████▉| 60.5kB / 61.0kB ...cerTwos-2499340.onnx.data: 99%|█████████████▉| 1.74MB / 1.76MB ...s/SoccerTwos-2999870.onnx: 99%|█████████████▉| 60.5kB / 61.0kB Processing Files (0 / 24) : 97%|█████████████▌| 188MB / 195MB, 135MB/s New Data Upload : 96%|█████████████▍| 168MB / 174MB, 120MB/s ...occerTwos/SoccerTwos-0.pt: 99%|█████████████▊| 1.28kB / 1.29kB ...s/SoccerTwos-1499948.onnx: 99%|█████████████▉| 60.5kB / 61.0kB ...cerTwos-1499948.onnx.data: 99%|█████████████▉| 1.74MB / 1.76MB ...rTwos/SoccerTwos-15406.pt: 99%|█████████████▉| 9.40MB / 9.48MB ...s/SoccerTwos-1999930.onnx: 99%|█████████████▉| 60.5kB / 61.0kB ...cerTwos-1999930.onnx.data: 99%|█████████████▉| 1.74MB / 1.76MB ...s/SoccerTwos-2499340.onnx: 99%|█████████████▉| 60.5kB / 61.0kB ...cerTwos-2499340.onnx.data: 99%|█████████████▉| 1.74MB / 1.76MB ...s/SoccerTwos-2999870.onnx: 99%|█████████████▉| 60.5kB / 61.0kB Processing Files (0 / 24) : 100%|█████████████▉| 194MB / 195MB, 121MB/s New Data Upload : 100%|█████████████▉| 173MB / 174MB, 108MB/s ...occerTwos/SoccerTwos-0.pt: 100%|██████████████| 1.29kB / 1.29kB ...s/SoccerTwos-1499948.onnx: 100%|██████████████| 61.0kB / 61.0kB ...cerTwos-1499948.onnx.data: 100%|██████████████| 1.76MB / 1.76MB ...rTwos/SoccerTwos-15406.pt: 100%|██████████████| 9.48MB / 9.48MB ...s/SoccerTwos-1999930.onnx: 100%|██████████████| 61.0kB / 61.0kB ...cerTwos-1999930.onnx.data: 100%|██████████████| 1.76MB / 1.76MB ...s/SoccerTwos-2499340.onnx: 100%|██████████████| 61.0kB / 61.0kB ...cerTwos-2499340.onnx.data: 100%|██████████████| 1.76MB / 1.76MB ...s/SoccerTwos-2999870.onnx: 100%|██████████████| 61.0kB / 61.0kB Processing Files (19 / 24) : 100%|█████████████▉| 195MB / 195MB, 108MB/s New Data Upload : 100%|█████████████▉| 174MB / 174MB, 96.5MB/s ...wos/SoccerTwos-3000202.pt: 100%|█████████████▉| 28.4MB / 28.4MB ...ccerTwos-999348.onnx.data: 100%|██████████████| 1.76MB / 1.76MB .../SoccerTwos/checkpoint.pt: 100%|█████████████▉| 28.4MB / 28.4MB ...4587.a0871a2ab7dc.10011.0: 100%|██████████████| 393kB / 393kB ...7111.a0871a2ab7dc.10124.0: 100%|██████████████| 1.41kB / 1.41kB ...7601.a0871a2ab7dc.10240.0: 100%|██████████████| 1.75MB / 1.75MB ...occerTwos/SoccerTwos.onnx: 100%|██████████████| 61.0kB / 61.0kB ...wos/SoccerTwos-1499948.pt: 100%|██████████████| 28.4MB / 28.4MB ...wos/SoccerTwos-2499340.pt: 100%|██████████████| 28.4MB / 28.4MB Processing Files (21 / 24) : 100%|█████████████▉| 195MB / 195MB, 97.3MB/s New Data Upload : 100%|█████████████▉| 174MB / 174MB, 86.9MB/s ...wos/SoccerTwos-3000202.pt: 100%|█████████████▉| 28.4MB / 28.4MB ...ccerTwos-999348.onnx.data: 100%|██████████████| 1.76MB / 1.76MB .../SoccerTwos/checkpoint.pt: 100%|█████████████▉| 28.4MB / 28.4MB ...4587.a0871a2ab7dc.10011.0: 100%|██████████████| 393kB / 393kB ...7111.a0871a2ab7dc.10124.0: 100%|██████████████| 1.41kB / 1.41kB ...7601.a0871a2ab7dc.10240.0: 100%|██████████████| 1.75MB / 1.75MB ...occerTwos/SoccerTwos.onnx: 100%|██████████████| 61.0kB / 61.0kB ...wos/SoccerTwos-1499948.pt: 100%|██████████████| 28.4MB / 28.4MB ...wos/SoccerTwos-2499340.pt: 100%|██████████████| 28.4MB / 28.4MB ...wos/SoccerTwos-1999930.pt: 100%|█████████████▉| 28.3MB / 28.4MB ...wos/SoccerTwos-3000202.pt: 100%|██████████████| 28.4MB / 28.4MB ...ccerTwos-999348.onnx.data: 100%|██████████████| 1.76MB / 1.76MB .../SoccerTwos/checkpoint.pt: 100%|██████████████| 28.4MB / 28.4MB ...4587.a0871a2ab7dc.10011.0: 100%|██████████████| 393kB / 393kB ...7111.a0871a2ab7dc.10124.0: 100%|██████████████| 1.41kB / 1.41kB ...7601.a0871a2ab7dc.10240.0: 100%|██████████████| 1.75MB / 1.75MB ...occerTwos/SoccerTwos.onnx: 100%|██████████████| 61.0kB / 61.0kB ...wos/SoccerTwos-1499948.pt: 100%|██████████████| 28.4MB / 28.4MB ...wos/SoccerTwos-2499340.pt: 100%|██████████████| 28.4MB / 28.4MB Processing Files (24 / 24) : 100%|██████████████| 195MB / 195MB, 81.2MB/s New Data Upload : 100%|██████████████| 174MB / 174MB, 72.5MB/s ...wos/SoccerTwos-3000202.pt: 100%|██████████████| 28.4MB / 28.4MB ...ccerTwos-999348.onnx.data: 100%|██████████████| 1.76MB / 1.76MB .../SoccerTwos/checkpoint.pt: 100%|██████████████| 28.4MB / 28.4MB ...4587.a0871a2ab7dc.10011.0: 100%|██████████████| 393kB / 393kB ...7111.a0871a2ab7dc.10124.0: 100%|██████████████| 1.41kB / 1.41kB ...7601.a0871a2ab7dc.10240.0: 100%|██████████████| 1.75MB / 1.75MB ...occerTwos/SoccerTwos.onnx: 100%|██████████████| 61.0kB / 61.0kB ...wos/SoccerTwos-1499948.pt: 100%|██████████████| 28.4MB / 28.4MB ...wos/SoccerTwos-2499340.pt: 100%|██████████████| 28.4MB / 28.4MB ...wos/SoccerTwos-1999930.pt: 100%|██████████████| 28.4MB / 28.4MB ...wos/SoccerTwos-3000202.pt: 100%|██████████████| 28.4MB / 28.4MB ...ccerTwos-999348.onnx.data: 100%|██████████████| 1.76MB / 1.76MB .../SoccerTwos/checkpoint.pt: 100%|██████████████| 28.4MB / 28.4MB ...4587.a0871a2ab7dc.10011.0: 100%|██████████████| 393kB / 393kB ...7111.a0871a2ab7dc.10124.0: 100%|██████████████| 1.41kB / 1.41kB ...7601.a0871a2ab7dc.10240.0: 100%|██████████████| 1.75MB / 1.75MB ...occerTwos/SoccerTwos.onnx: 100%|██████████████| 61.0kB / 61.0kB ...wos/SoccerTwos-1499948.pt: 100%|██████████████| 28.4MB / 28.4MB ...wos/SoccerTwos-2499340.pt: 100%|██████████████| 28.4MB / 28.4MB Processing Files (24 / 24) : 100%|██████████████| 195MB / 195MB, 75.0MB/s New Data Upload : 100%|██████████████| 174MB / 174MB, 66.9MB/s ...wos/SoccerTwos-3000202.pt: 100%|██████████████| 28.4MB / 28.4MB ...ccerTwos-999348.onnx.data: 100%|██████████████| 1.76MB / 1.76MB .../SoccerTwos/checkpoint.pt: 100%|██████████████| 28.4MB / 28.4MB ...4587.a0871a2ab7dc.10011.0: 100%|██████████████| 393kB / 393kB ...7111.a0871a2ab7dc.10124.0: 100%|██████████████| 1.41kB / 1.41kB ...7601.a0871a2ab7dc.10240.0: 100%|██████████████| 1.75MB / 1.75MB ...occerTwos/SoccerTwos.onnx: 100%|██████████████| 61.0kB / 61.0kB ...wos/SoccerTwos-1499948.pt: 100%|██████████████| 28.4MB / 28.4MB ...wos/SoccerTwos-2499340.pt: 100%|██████████████| 28.4MB / 28.4MB ...wos/SoccerTwos-1999930.pt: 100%|██████████████| 28.4MB / 28.4MB [INFO] Your model is pushed to the hub. You can view your model here: https://huggingface.co/Chiz/poca-SoccerTwos
If everything worked you should see at the end:
Your model is pushed to the hub. You can view your model here: https://huggingface.co/YOUR_USERNAME/poca-SoccerTwos
Verify your model is ready for the AI vs AI Challenge ✅¶
After pushing, check your model page for:
- The tag
ML-Agents-SoccerTwos— needed to enter the challenge pool - A
SoccerTwos.onnxfile in the repo
If the tag is missing, edit the model card on HuggingFace and add it manually.
Your model will be picked up in the next AI vs AI matchmaking cycle (~every 4 hours).
In [31]:
from huggingface_hub import HfApi
# Replace with your username
HF_USERNAME = "Chiz"
REPO_NAME = "poca-SoccerTwos"
repo_id = f"{HF_USERNAME}/{REPO_NAME}"
api = HfApi()
try:
info = api.model_info(repo_id)
tags = info.tags or []
files = list(api.list_repo_files(repo_id))
print(f"Model : {repo_id}")
print(f"Tags : {tags}")
print(f"Files : {files}")
if "ML-Agents-SoccerTwos" in tags:
print("\nTag ML-Agents-SoccerTwos present — you will appear on the leaderboard!")
else:
print("\nTag ML-Agents-SoccerTwos MISSING — add it manually on your model page.")
onnx = [f for f in files if f.endswith(".onnx")]
if onnx:
print(f"ONNX file found: {onnx}")
else:
print("No .onnx file found — push may not have completed correctly.")
print(f"\nModel page : https://huggingface.co/{repo_id}")
print(f"Leaderboard : https://huggingface.co/spaces/huggingface-projects/AIvsAI-SoccerTwos")
print(f"Watch match : https://huggingface.co/spaces/unity/ML-Agents-SoccerTwos")
except Exception as e:
print(f"Error: {e}")
print("Make sure the push cell ran successfully and your username is correct.")
Model : Chiz/poca-SoccerTwos Tags : ['ml-agents', 'tensorboard', 'onnx', 'SoccerTwos', 'deep-reinforcement-learning', 'reinforcement-learning', 'ML-Agents-SoccerTwos', 'region:us'] Files : ['.gitattributes', 'README.md', 'SoccerTwos.onnx', 'SoccerTwos/SoccerTwos-0.pt', 'SoccerTwos/SoccerTwos-1499948.onnx', 'SoccerTwos/SoccerTwos-1499948.onnx.data', 'SoccerTwos/SoccerTwos-1499948.pt', 'SoccerTwos/SoccerTwos-15406.pt', 'SoccerTwos/SoccerTwos-1999930.onnx', 'SoccerTwos/SoccerTwos-1999930.onnx.data', 'SoccerTwos/SoccerTwos-1999930.pt', 'SoccerTwos/SoccerTwos-2499340.onnx', 'SoccerTwos/SoccerTwos-2499340.onnx.data', 'SoccerTwos/SoccerTwos-2499340.pt', 'SoccerTwos/SoccerTwos-2999870.onnx', 'SoccerTwos/SoccerTwos-2999870.onnx.data', 'SoccerTwos/SoccerTwos-2999870.pt', 'SoccerTwos/SoccerTwos-3000202.onnx', 'SoccerTwos/SoccerTwos-3000202.onnx.data', 'SoccerTwos/SoccerTwos-3000202.pt', 'SoccerTwos/SoccerTwos-499868.onnx.data', 'SoccerTwos/SoccerTwos-999348.onnx.data', 'SoccerTwos/checkpoint.pt', 'SoccerTwos/events.out.tfevents.1771014587.a0871a2ab7dc.10011.0', 'SoccerTwos/events.out.tfevents.1771017111.a0871a2ab7dc.10124.0', 'SoccerTwos/events.out.tfevents.1771017601.a0871a2ab7dc.10240.0', 'config.json', 'configuration.yaml', 'run_logs/Player-0.log', 'run_logs/timers.json', 'run_logs/training_status.json'] Tag ML-Agents-SoccerTwos present — you will appear on the leaderboard! ONNX file found: ['SoccerTwos.onnx', 'SoccerTwos/SoccerTwos-1499948.onnx', 'SoccerTwos/SoccerTwos-1999930.onnx', 'SoccerTwos/SoccerTwos-2499340.onnx', 'SoccerTwos/SoccerTwos-2999870.onnx', 'SoccerTwos/SoccerTwos-3000202.onnx'] Model page : https://huggingface.co/Chiz/poca-SoccerTwos Leaderboard : https://huggingface.co/spaces/huggingface-projects/AIvsAI-SoccerTwos Watch match : https://huggingface.co/spaces/unity/ML-Agents-SoccerTwos
Watch your agent play 🎮¶
Go to https://huggingface.co/spaces/unity/ML-Agents-SoccerTwos
- Select your model as team blue (or purple)
- Pick an opponent — try the baseline model or whoever is top of the leaderboard
Live matches here aren't counted in your ELO, but it's a great way to see how your agent performs!
Don't forget to share your results in the #ai-vs-ai-challenge channel on Discord 🔥