Unit 5: An Introduction to ML-Agents¶
In this notebook, you'll learn about ML-Agents and train two agents:
- SnowballTarget: Learn to shoot snowballs onto spawning targets
- Pyramids: Press a button to spawn a pyramid, navigate to it, knock it over, and move to the gold brick at the top (uses curiosity)
After training, you'll be able to watch your agents playing directly in your browser.
📄 Course: https://huggingface.co/deep-rl-course/en/unit0/introduction#certification-process
⬇️ Here is what you will achieve at the end of this unit ⬇️
🎮 Environments:¶
- SnowballTarget
- Pyramids
Before you start ✅¶
- Enable Internet in Kaggle session settings
- Add your HuggingFace write token as a Kaggle Secret named
HF_TOKEN- Kaggle sidebar → Add-ons → Secrets → Add New Secret
- Get your token at: https://huggingface.co/settings/tokens
Install ML-Agents 🔽¶
We install ML-Agents in a Python 3.10 virtual environment to avoid compatibility issues with Kaggle's default Python 3.12.
In [2]:
%%capture
!add-apt-repository ppa:deadsnakes/ppa -y
!apt-get update -q
!apt-get install -y python3.10 python3.10-venv python3.10-distutils -q
In [4]:
%%capture
# Create the environment
!python3.10 -m venv /kaggle/working/mlagents-env
!/kaggle/working/mlagents-env/bin/pip install --upgrade pip -q
# Step A: Install PyTorch CPU first from the specific index
!/kaggle/working/mlagents-env/bin/pip install torch --index-url https://download.pytorch.org/whl/cpu
# Step B: Install mlagents and core dependencies from PyPI
!/kaggle/working/mlagents-env/bin/pip install \
"mlagents==1.1.0" \
"mlagents-envs==1.1.0" \
"numpy==1.23.5" \
"onnx==1.15.0" \
"grpcio==1.48.2" \
"protobuf==3.20.3"
In [5]:
# Verify installation
!/kaggle/working/mlagents-env/bin/mlagents-learn --help | head -50
usage: mlagents-learn [-h] [--env ENV_PATH] [--resume] [--deterministic]
[--force] [--run-id RUN_ID] [--initialize-from RUN_ID]
[--seed SEED] [--inference] [--base-port BASE_PORT]
[--num-envs NUM_ENVS] [--num-areas NUM_AREAS] [--debug]
[--env-args ...]
[--max-lifetime-restarts MAX_LIFETIME_RESTARTS]
[--restarts-rate-limit-n RESTARTS_RATE_LIMIT_N]
[--restarts-rate-limit-period-s RESTARTS_RATE_LIMIT_PERIOD_S]
[--torch] [--tensorflow] [--results-dir RESULTS_DIR]
[--timeout-wait TIMEOUT_WAIT] [--width WIDTH]
[--height HEIGHT] [--quality-level QUALITY_LEVEL]
[--time-scale TIME_SCALE]
[--target-frame-rate TARGET_FRAME_RATE]
[--capture-frame-rate CAPTURE_FRAME_RATE]
[--no-graphics] [--no-graphics-monitor]
[--torch-device DEVICE]
[trainer_config_path]
positional arguments:
trainer_config_path
options:
-h, --help show this help message and exit
--env ENV_PATH Path to the Unity executable to train (default: None)
--resume Whether to resume training from a checkpoint. Specify
a --run-id to use this option. If set, the training
code loads an already trained model to initialize the
neural network before resuming training. This option
is only valid when the models exist, and have the same
behavior names as the current agents in your scene.
(default: False)
--deterministic Whether to select actions deterministically in policy.
`dist.mean` for continuous action space, and
`dist.argmax` for deterministic action space (default:
False)
--force Whether to force-overwrite this run-id's existing
summary and model data. (Without this flag, attempting
to train a model with a run-id that has been used
before will throw an error. (default: False)
--run-id RUN_ID The identifier for the training run. This identifier
is used to name the subdirectories in which the
trained model and summary statistics are saved as well
as the saved model itself. If you use TensorBoard to
view the training statistics, always set a unique run-
id for each training run. (The statistics for all runs
with the same id are combined as if they were produced
by a the same session.) (default: ppo)
--initialize-from RUN_ID
Specify a previously saved run ID from which to
initialize the model from. This can be used, for
Download Environment Executables 🔽¶
In [6]:
!mkdir -p ./training-envs-executables/SnowballTarget
!mkdir -p ./training-envs-executables/Pyramids
In [7]:
%%capture
!pip install huggingface_hub gdown -q
In [8]:
# Download SnowballTarget
!wget "https://github.com/huggingface/Snowball-Target/raw/main/SnowballTarget.zip" -O ./training-envs-executables/SnowballTarget.zip
--2026-02-24 19:59:16-- https://github.com/huggingface/Snowball-Target/raw/main/SnowballTarget.zip Resolving github.com (github.com)... 140.82.112.4 Connecting to github.com (github.com)|140.82.112.4|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://media.githubusercontent.com/media/huggingface/Snowball-Target/main/SnowballTarget.zip [following] --2026-02-24 19:59:16-- https://media.githubusercontent.com/media/huggingface/Snowball-Target/main/SnowballTarget.zip Resolving media.githubusercontent.com (media.githubusercontent.com)... 185.199.111.133, 185.199.109.133, 185.199.108.133, ... Connecting to media.githubusercontent.com (media.githubusercontent.com)|185.199.111.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 35134213 (34M) [application/zip] Saving to: ‘./training-envs-executables/SnowballTarget.zip’ ./training-envs-exe 100%[===================>] 33.51M 73.3MB/s in 0.5s 2026-02-24 19:59:17 (73.3 MB/s) - ‘./training-envs-executables/SnowballTarget.zip’ saved [35134213/35134213]
In [9]:
%%capture
!unzip -d ./training-envs-executables/SnowballTarget/ ./training-envs-executables/SnowballTarget.zip
In [10]:
!chmod -R 755 ./training-envs-executables/SnowballTarget/
In [11]:
# Confirm the executable is present
!ls -la ./training-envs-executables/SnowballTarget/
total 12 drwxr-xr-x 3 root root 4096 Feb 24 19:59 . drwxr-xr-x 4 root root 4096 Feb 24 19:59 .. drwxr-xr-x 4 root root 4096 Jan 8 2023 SnowballTarget
In [12]:
!ls -R ./training-envs-executables/SnowballTarget/
./training-envs-executables/SnowballTarget/: SnowballTarget ./training-envs-executables/SnowballTarget/SnowballTarget: SnowballTarget_BurstDebugInformation_DoNotShip SnowballTarget.x86_64 SnowballTarget_Data UnityPlayer.so ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_BurstDebugInformation_DoNotShip: Data ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_BurstDebugInformation_DoNotShip/Data: Plugins ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_BurstDebugInformation_DoNotShip/Data/Plugins: lib_burst_generated.txt ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data: app.info Managed RuntimeInitializeOnLoads.json boot.config MonoBleedingEdge ScriptingAssemblies.json globalgamemanagers Plugins sharedassets0.assets globalgamemanagers.assets Resources sharedassets0.assets.resS level0 resources.assets StreamingAssets level0.resS resources.assets.resS ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/Managed: Assembly-CSharp.dll Google.Protobuf.dll Grpc.Core.dll Mono.Security.dll mscorlib.dll netstandard.dll Newtonsoft.Json.dll System.ComponentModel.Composition.dll System.Configuration.dll System.Core.dll System.Data.DataSetExtensions.dll System.Data.dll System.dll System.Drawing.dll System.EnterpriseServices.dll System.Interactive.Async.dll System.IO.Abstractions.dll System.IO.Abstractions.TestingHelpers.dll System.IO.Compression.dll System.IO.Compression.FileSystem.dll System.Net.Http.dll System.Numerics.dll System.Runtime.dll System.Runtime.Serialization.dll System.Security.dll System.ServiceModel.Internals.dll System.Transactions.dll System.Xml.dll System.Xml.Linq.dll Unity.Barracuda.BurstBLAS.dll Unity.Barracuda.dll Unity.Barracuda.ONNX.dll Unity.Burst.Cecil.dll Unity.Burst.Cecil.Mdb.dll Unity.Burst.Cecil.Pdb.dll Unity.Burst.Cecil.Rocks.dll Unity.Burst.dll Unity.Burst.Unsafe.dll UnityEngine.AccessibilityModule.dll UnityEngine.AIModule.dll UnityEngine.AndroidJNIModule.dll UnityEngine.AnimationModule.dll UnityEngine.AssetBundleModule.dll UnityEngine.AudioModule.dll UnityEngine.ClothModule.dll UnityEngine.ClusterInputModule.dll UnityEngine.ClusterRendererModule.dll UnityEngine.CoreModule.dll UnityEngine.CrashReportingModule.dll UnityEngine.DirectorModule.dll UnityEngine.dll UnityEngine.DSPGraphModule.dll UnityEngine.GameCenterModule.dll UnityEngine.GIModule.dll UnityEngine.GridModule.dll UnityEngine.HotReloadModule.dll UnityEngine.ImageConversionModule.dll UnityEngine.IMGUIModule.dll UnityEngine.InputLegacyModule.dll UnityEngine.InputModule.dll UnityEngine.JSONSerializeModule.dll UnityEngine.LocalizationModule.dll UnityEngine.ParticleSystemModule.dll UnityEngine.PerformanceReportingModule.dll UnityEngine.Physics2DModule.dll UnityEngine.PhysicsModule.dll UnityEngine.ProfilerModule.dll UnityEngine.RuntimeInitializeOnLoadManagerInitializerModule.dll UnityEngine.ScreenCaptureModule.dll UnityEngine.SharedInternalsModule.dll UnityEngine.SpriteMaskModule.dll UnityEngine.SpriteShapeModule.dll UnityEngine.StreamingModule.dll UnityEngine.SubstanceModule.dll UnityEngine.SubsystemsModule.dll UnityEngine.TerrainModule.dll UnityEngine.TerrainPhysicsModule.dll UnityEngine.TextCoreFontEngineModule.dll UnityEngine.TextCoreTextEngineModule.dll UnityEngine.TextRenderingModule.dll UnityEngine.TilemapModule.dll UnityEngine.TLSModule.dll UnityEngine.UI.dll UnityEngine.UIElementsModule.dll UnityEngine.UIElementsNativeModule.dll UnityEngine.UIModule.dll UnityEngine.UmbraModule.dll UnityEngine.UNETModule.dll UnityEngine.UnityAnalyticsCommonModule.dll UnityEngine.UnityAnalyticsModule.dll UnityEngine.UnityConnectModule.dll UnityEngine.UnityCurlModule.dll UnityEngine.UnityTestProtocolModule.dll UnityEngine.UnityWebRequestAssetBundleModule.dll UnityEngine.UnityWebRequestAudioModule.dll UnityEngine.UnityWebRequestModule.dll UnityEngine.UnityWebRequestTextureModule.dll UnityEngine.UnityWebRequestWWWModule.dll UnityEngine.VehiclesModule.dll UnityEngine.VFXModule.dll UnityEngine.VideoModule.dll UnityEngine.VirtualTexturingModule.dll UnityEngine.VRModule.dll UnityEngine.WindModule.dll UnityEngine.XRModule.dll Unity.Mathematics.dll Unity.ML-Agents.CommunicatorObjects.dll Unity.ML-Agents.dll Unity.ProGrids.dll Unity.Services.Core.Analytics.dll Unity.Services.Core.Configuration.dll Unity.Services.Core.Device.dll Unity.Services.Core.dll Unity.Services.Core.Environments.dll Unity.Services.Core.Environments.Internal.dll Unity.Services.Core.Internal.dll Unity.Services.Core.Networking.dll Unity.Services.Core.Registration.dll Unity.Services.Core.Scheduler.dll Unity.Services.Core.Telemetry.dll Unity.Services.Core.Threading.dll Unity.TextMeshPro.dll Unity.Timeline.dll ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/MonoBleedingEdge: etc x86_64 ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/MonoBleedingEdge/etc: config mono ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/MonoBleedingEdge/etc/mono: 2.0 4.0 4.5 browscap.ini config mconfig ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/MonoBleedingEdge/etc/mono/2.0: Browsers machine.config web.config DefaultWsdlHelpGenerator.aspx settings.map ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/MonoBleedingEdge/etc/mono/2.0/Browsers: Compat.browser ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/MonoBleedingEdge/etc/mono/4.0: Browsers machine.config web.config DefaultWsdlHelpGenerator.aspx settings.map ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/MonoBleedingEdge/etc/mono/4.0/Browsers: Compat.browser ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/MonoBleedingEdge/etc/mono/4.5: Browsers machine.config web.config DefaultWsdlHelpGenerator.aspx settings.map ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/MonoBleedingEdge/etc/mono/4.5/Browsers: Compat.browser ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/MonoBleedingEdge/etc/mono/mconfig: config.xml ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/MonoBleedingEdge/x86_64: libmonobdwgc-2.0.so libmono-native.so libMonoPosixHelper.so ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/Plugins: lib_burst_generated.so libgrpc_csharp_ext.x64.so ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/Resources: unity_builtin_extra 'unity default resources' UnityPlayer.png ./training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget_Data/StreamingAssets: UnityServicesProjectConfiguration.json
In [15]:
# Download Pyramids
!wget "https://huggingface.co/spaces/unity/ML-Agents-Pyramids/resolve/main/Pyramids.zip" -O ./training-envs-executables/Pyramids.zip
--2026-02-24 20:07:28-- https://huggingface.co/spaces/unity/ML-Agents-Pyramids/resolve/main/Pyramids.zip Resolving huggingface.co (huggingface.co)... 3.170.185.25, 3.170.185.14, 3.170.185.35, ... Connecting to huggingface.co (huggingface.co)|3.170.185.25|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://us.gcp.cdn.hf.co/xet-bridge-us/62b2c0284ea3eff279c966ab/2f754b6bf0b817493a77fafe582b7dde47fa25a476d3f8e68a24fdd4efcaa282?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27Pyramids.zip%3B+filename%3D%22Pyramids.zip%22%3B&response-content-type=application%2Fzip&Expires=1771967248&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiRXBvY2hUaW1lIjoxNzcxOTY3MjQ4fX0sIlJlc291cmNlIjoiaHR0cHM6Ly91cy5nY3AuY2RuLmhmLmNvL3hldC1icmlkZ2UtdXMvNjJiMmMwMjg0ZWEzZWZmMjc5Yzk2NmFiLzJmNzU0YjZiZjBiODE3NDkzYTc3ZmFmZTU4MmI3ZGRlNDdmYTI1YTQ3NmQzZjhlNjhhMjRmZGQ0ZWZjYWEyODJcXD9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSomcmVzcG9uc2UtY29udGVudC10eXBlPSoifV19&Signature=HtsYW0ZL5VzC9TuIWETRaG3oAXBMtyJImFsvVh6aGDL9fYLhjp2-uyX%7E36HroY9V4MaTmNnBsmwlEEMkzZ27yK-NQHKmQH6WTXf92iLqJx7bkKrjKTShIymjlcep%7ELnCkK2StlggyHnLBsCqM7ATJuzkhSHGmn7FxCoCnqxgc0zRlhMCycEK2Wk8uiMAgM%7ELm979lP2xmZmvRJJ3Dstt1xRzClV6rQwFLRibvdAvnt91iowcxGmDBJS2t2mHGQTzmSNmAlFcb6FFOIROVNrj-G9b2zDUCC89TUsY3V2k5UHzDTS%7En6PGXY%7EnRL%7ELvpaaJPyUnducSlVYO%7EWgJiedfQ__&Key-Pair-Id=KJLH8B0YWU4Y8M [following] --2026-02-24 20:07:28-- https://us.gcp.cdn.hf.co/xet-bridge-us/62b2c0284ea3eff279c966ab/2f754b6bf0b817493a77fafe582b7dde47fa25a476d3f8e68a24fdd4efcaa282?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27Pyramids.zip%3B+filename%3D%22Pyramids.zip%22%3B&response-content-type=application%2Fzip&Expires=1771967248&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiRXBvY2hUaW1lIjoxNzcxOTY3MjQ4fX0sIlJlc291cmNlIjoiaHR0cHM6Ly91cy5nY3AuY2RuLmhmLmNvL3hldC1icmlkZ2UtdXMvNjJiMmMwMjg0ZWEzZWZmMjc5Yzk2NmFiLzJmNzU0YjZiZjBiODE3NDkzYTc3ZmFmZTU4MmI3ZGRlNDdmYTI1YTQ3NmQzZjhlNjhhMjRmZGQ0ZWZjYWEyODJcXD9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSomcmVzcG9uc2UtY29udGVudC10eXBlPSoifV19&Signature=HtsYW0ZL5VzC9TuIWETRaG3oAXBMtyJImFsvVh6aGDL9fYLhjp2-uyX%7E36HroY9V4MaTmNnBsmwlEEMkzZ27yK-NQHKmQH6WTXf92iLqJx7bkKrjKTShIymjlcep%7ELnCkK2StlggyHnLBsCqM7ATJuzkhSHGmn7FxCoCnqxgc0zRlhMCycEK2Wk8uiMAgM%7ELm979lP2xmZmvRJJ3Dstt1xRzClV6rQwFLRibvdAvnt91iowcxGmDBJS2t2mHGQTzmSNmAlFcb6FFOIROVNrj-G9b2zDUCC89TUsY3V2k5UHzDTS%7En6PGXY%7EnRL%7ELvpaaJPyUnducSlVYO%7EWgJiedfQ__&Key-Pair-Id=KJLH8B0YWU4Y8M Resolving us.gcp.cdn.hf.co (us.gcp.cdn.hf.co)... 35.209.178.129 Connecting to us.gcp.cdn.hf.co (us.gcp.cdn.hf.co)|35.209.178.129|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 42907187 (41M) [application/zip] Saving to: ‘./training-envs-executables/Pyramids.zip’ ./training-envs-exe 100%[===================>] 40.92M 1.92MB/s in 13s 2026-02-24 20:07:41 (3.10 MB/s) - ‘./training-envs-executables/Pyramids.zip’ saved [42907187/42907187]
In [16]:
%%capture
!unzip -d ./training-envs-executables/Pyramids/ ./training-envs-executables/Pyramids.zip
In [17]:
!chmod -R 755 ./training-envs-executables/Pyramids/
In [18]:
# Confirm the executable is present
!ls -la ./training-envs-executables/Pyramids/
total 12 drwxr-xr-x 3 root root 4096 Feb 24 20:07 . drwxr-xr-x 4 root root 4096 Feb 24 20:07 .. drwxr-xr-x 4 root root 4096 Jun 23 2022 Pyramids
In [19]:
!ls -R ./training-envs-executables/Pyramids/
./training-envs-executables/Pyramids/: Pyramids ./training-envs-executables/Pyramids/Pyramids: Pyramids Pyramids_Data Pyramids_BurstDebugInformation_DoNotShip UnityPlayer.so ./training-envs-executables/Pyramids/Pyramids/Pyramids_BurstDebugInformation_DoNotShip: Data ./training-envs-executables/Pyramids/Pyramids/Pyramids_BurstDebugInformation_DoNotShip/Data: Plugins ./training-envs-executables/Pyramids/Pyramids/Pyramids_BurstDebugInformation_DoNotShip/Data/Plugins: lib_burst_generated.txt ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data: app.info Managed resources.assets boot.config ML-Agents RuntimeInitializeOnLoads.json globalgamemanagers MonoBleedingEdge ScriptingAssemblies.json globalgamemanagers.assets Plugins sharedassets0.assets level0 Resources sharedassets0.assets.resS ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/Managed: Assembly-CSharp.dll Assembly-CSharp.pdb Google.Protobuf.dll Google.Protobuf.pdb Grpc.Core.dll Mono.Security.dll mscorlib.dll netstandard.dll Newtonsoft.Json.dll Newtonsoft.Json.pdb System.ComponentModel.Composition.dll System.Configuration.dll System.Core.dll System.Data.DataSetExtensions.dll System.Data.dll System.dll System.Drawing.dll System.EnterpriseServices.dll System.Interactive.Async.dll System.IO.Abstractions.dll System.IO.Abstractions.TestingHelpers.dll System.IO.Compression.dll System.IO.Compression.FileSystem.dll System.Net.Http.dll System.Numerics.dll System.Runtime.dll System.Runtime.Serialization.dll System.Security.dll System.ServiceModel.Internals.dll System.Transactions.dll System.Xml.dll System.Xml.Linq.dll Unity.Barracuda.BurstBLAS.dll Unity.Barracuda.BurstBLAS.pdb Unity.Barracuda.dll Unity.Barracuda.ONNX.dll Unity.Barracuda.ONNX.pdb Unity.Barracuda.pdb Unity.Burst.Cecil.dll Unity.Burst.Cecil.Mdb.dll Unity.Burst.Cecil.Pdb.dll Unity.Burst.Cecil.Rocks.dll Unity.Burst.dll Unity.Burst.pdb Unity.Burst.Unsafe.dll UnityEngine.AccessibilityModule.dll UnityEngine.AccessibilityModule.pdb UnityEngine.AIModule.dll UnityEngine.AIModule.pdb UnityEngine.AndroidJNIModule.dll UnityEngine.AndroidJNIModule.pdb UnityEngine.AnimationModule.dll UnityEngine.AnimationModule.pdb UnityEngine.AssetBundleModule.dll UnityEngine.AssetBundleModule.pdb UnityEngine.AudioModule.dll UnityEngine.AudioModule.pdb UnityEngine.ClothModule.dll UnityEngine.ClothModule.pdb UnityEngine.ClusterInputModule.dll UnityEngine.ClusterInputModule.pdb UnityEngine.ClusterRendererModule.dll UnityEngine.ClusterRendererModule.pdb UnityEngine.CoreModule.dll UnityEngine.CoreModule.pdb UnityEngine.CrashReportingModule.dll UnityEngine.CrashReportingModule.pdb UnityEngine.DirectorModule.dll UnityEngine.DirectorModule.pdb UnityEngine.dll UnityEngine.DSPGraphModule.dll UnityEngine.DSPGraphModule.pdb UnityEngine.GameCenterModule.dll UnityEngine.GameCenterModule.pdb UnityEngine.GIModule.dll UnityEngine.GIModule.pdb UnityEngine.GridModule.dll UnityEngine.GridModule.pdb UnityEngine.HotReloadModule.dll UnityEngine.HotReloadModule.pdb UnityEngine.ImageConversionModule.dll UnityEngine.ImageConversionModule.pdb UnityEngine.IMGUIModule.dll UnityEngine.IMGUIModule.pdb UnityEngine.InputLegacyModule.dll UnityEngine.InputLegacyModule.pdb UnityEngine.InputModule.dll UnityEngine.InputModule.pdb UnityEngine.JSONSerializeModule.dll UnityEngine.JSONSerializeModule.pdb UnityEngine.LocalizationModule.dll UnityEngine.LocalizationModule.pdb UnityEngine.ParticleSystemModule.dll UnityEngine.ParticleSystemModule.pdb UnityEngine.pdb UnityEngine.PerformanceReportingModule.dll UnityEngine.PerformanceReportingModule.pdb UnityEngine.Physics2DModule.dll UnityEngine.Physics2DModule.pdb UnityEngine.PhysicsModule.dll UnityEngine.PhysicsModule.pdb UnityEngine.ProfilerModule.dll UnityEngine.ProfilerModule.pdb UnityEngine.RuntimeInitializeOnLoadManagerInitializerModule.dll UnityEngine.RuntimeInitializeOnLoadManagerInitializerModule.pdb UnityEngine.ScreenCaptureModule.dll UnityEngine.ScreenCaptureModule.pdb UnityEngine.SharedInternalsModule.dll UnityEngine.SharedInternalsModule.pdb UnityEngine.SpriteMaskModule.dll UnityEngine.SpriteMaskModule.pdb UnityEngine.SpriteShapeModule.dll UnityEngine.SpriteShapeModule.pdb UnityEngine.StreamingModule.dll UnityEngine.StreamingModule.pdb UnityEngine.SubstanceModule.dll UnityEngine.SubstanceModule.pdb UnityEngine.SubsystemsModule.dll UnityEngine.SubsystemsModule.pdb UnityEngine.TerrainModule.dll UnityEngine.TerrainModule.pdb UnityEngine.TerrainPhysicsModule.dll UnityEngine.TerrainPhysicsModule.pdb UnityEngine.TextCoreFontEngineModule.dll UnityEngine.TextCoreFontEngineModule.pdb UnityEngine.TextCoreTextEngineModule.dll UnityEngine.TextCoreTextEngineModule.pdb UnityEngine.TextRenderingModule.dll UnityEngine.TextRenderingModule.pdb UnityEngine.TilemapModule.dll UnityEngine.TilemapModule.pdb UnityEngine.TLSModule.dll UnityEngine.TLSModule.pdb UnityEngine.UI.dll UnityEngine.UIElementsModule.dll UnityEngine.UIElementsModule.pdb UnityEngine.UIElementsNativeModule.dll UnityEngine.UIElementsNativeModule.pdb UnityEngine.UIModule.dll UnityEngine.UIModule.pdb UnityEngine.UI.pdb UnityEngine.UmbraModule.dll UnityEngine.UmbraModule.pdb UnityEngine.UNETModule.dll UnityEngine.UNETModule.pdb UnityEngine.UnityAnalyticsModule.dll UnityEngine.UnityAnalyticsModule.pdb UnityEngine.UnityConnectModule.dll UnityEngine.UnityConnectModule.pdb UnityEngine.UnityCurlModule.dll UnityEngine.UnityCurlModule.pdb UnityEngine.UnityTestProtocolModule.dll UnityEngine.UnityTestProtocolModule.pdb UnityEngine.UnityWebRequestAssetBundleModule.dll UnityEngine.UnityWebRequestAssetBundleModule.pdb UnityEngine.UnityWebRequestAudioModule.dll UnityEngine.UnityWebRequestAudioModule.pdb UnityEngine.UnityWebRequestModule.dll UnityEngine.UnityWebRequestModule.pdb UnityEngine.UnityWebRequestTextureModule.dll UnityEngine.UnityWebRequestTextureModule.pdb UnityEngine.UnityWebRequestWWWModule.dll UnityEngine.UnityWebRequestWWWModule.pdb UnityEngine.VehiclesModule.dll UnityEngine.VehiclesModule.pdb UnityEngine.VFXModule.dll UnityEngine.VFXModule.pdb UnityEngine.VideoModule.dll UnityEngine.VideoModule.pdb UnityEngine.VirtualTexturingModule.dll UnityEngine.VirtualTexturingModule.pdb UnityEngine.VRModule.dll UnityEngine.VRModule.pdb UnityEngine.WindModule.dll UnityEngine.WindModule.pdb UnityEngine.XRModule.dll UnityEngine.XRModule.pdb Unity.InputSystem.dll Unity.InputSystem.pdb Unity.Mathematics.dll Unity.Mathematics.pdb Unity.ML-Agents.CommunicatorObjects.dll Unity.ML-Agents.CommunicatorObjects.pdb Unity.ML-Agents.dll Unity.ML-Agents.Extensions.dll Unity.ML-Agents.Extensions.Input.dll Unity.ML-Agents.Extensions.Input.pdb Unity.ML-Agents.Extensions.pdb Unity.ML-Agents.pdb ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/ML-Agents: Timers ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/ML-Agents/Timers: Pyramids_timers.json ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/MonoBleedingEdge: etc x86_64 ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/MonoBleedingEdge/etc: config mono ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono: 2.0 4.0 4.5 browscap.ini config mconfig ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/2.0: Browsers machine.config web.config DefaultWsdlHelpGenerator.aspx settings.map ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/2.0/Browsers: Compat.browser ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.0: Browsers machine.config web.config DefaultWsdlHelpGenerator.aspx settings.map ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.0/Browsers: Compat.browser ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.5: Browsers machine.config web.config DefaultWsdlHelpGenerator.aspx settings.map ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.5/Browsers: Compat.browser ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/mconfig: config.xml ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/MonoBleedingEdge/x86_64: libmonobdwgc-2.0.so libmono-native.so libMonoPosixHelper.so ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/Plugins: lib_burst_generated.so libgrpc_csharp_ext.x64.so ./training-envs-executables/Pyramids/Pyramids/Pyramids_Data/Resources: unity_builtin_extra 'unity default resources' UnityPlayer.png
Part 1: SnowballTarget ❄️🎯¶
Train an agent to shoot snowballs at spawning targets.
Create the SnowballTarget config file¶
In [20]:
import os
snowball_config = """
behaviors:
SnowballTarget:
trainer_type: ppo
hyperparameters:
batch_size: 128
buffer_size: 2048
learning_rate: 0.0003
beta: 0.005
epsilon: 0.2
lambd: 0.95
num_epoch: 3
learning_rate_schedule: linear
network_settings:
normalize: false
hidden_units: 256
num_layers: 2
vis_encode_type: simple
reward_signals:
extrinsic:
gamma: 0.99
strength: 1.0
checkpoint_interval: 200000
keep_checkpoints: 5
max_steps: 1000000
time_horizon: 64
summary_freq: 10000
"""
os.makedirs("/kaggle/working/ml-agents/config/ppo", exist_ok=True)
with open("/kaggle/working/ml-agents/config/ppo/SnowballTarget.yaml", "w") as f:
f.write(snowball_config)
print("✅ SnowballTarget config written to: /kaggle/working/ml-agents/config/ppo/SnowballTarget.yaml")
✅ SnowballTarget config written to: /kaggle/working/ml-agents/config/ppo/SnowballTarget.yaml
Train SnowballTarget agent¶
Training takes ~30–45 minutes. The agent learns to aim and shoot snowballs at targets.
In [23]:
!find /kaggle/working/ -name "SnowballTarget.yaml"
/kaggle/working/ml-agents/config/ppo/SnowballTarget.yaml
In [24]:
!/kaggle/working/mlagents-env/bin/mlagents-learn \
/kaggle/working/ml-agents/config/ppo/SnowballTarget.yaml \
--env=/kaggle/working/training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget.x86_64 \
--run-id="SnowballTarget1" \
--no-graphics \
--force
┐ ╖
╓╖╬│╡ ││╬╖╖
╓╖╬│││││┘ ╬│││││╬╖
╖╬│││││╬╜ ╙╬│││││╖╖ ╗╗╗
╬╬╬╬╖││╦╖ ╖╬││╗╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╜╜╜ ╟╣╣
╬╬╬╬╬╬╬╬╖│╬╖╖╓╬╪│╓╣╣╣╣╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╒╣╣╖╗╣╣╣╗ ╣╣╣ ╣╣╣╣╣╣ ╟╣╣╖ ╣╣╣
╬╬╬╬┐ ╙╬╬╬╬│╓╣╣╣╝╜ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╣╙ ╙╣╣╣ ╣╣╣ ╙╟╣╣╜╙ ╫╣╣ ╟╣╣
╬╬╬╬┐ ╙╬╬╣╣ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣ ╣╣╣┌╣╣╜
╬╬╬╜ ╬╬╣╣ ╙╝╣╣╬ ╙╣╣╣╗╖╓╗╣╣╣╜ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣╦╓ ╣╣╣╣╣
╙ ╓╦╖ ╬╬╣╣ ╓╗╗╖ ╙╝╣╣╣╣╝╜ ╘╝╝╜ ╝╝╝ ╝╝╝ ╙╣╣╣ ╟╣╣╣
╩╬╬╬╬╬╬╦╦╬╬╣╣╗╣╣╣╣╣╣╣╝ ╫╣╣╣╣
╙╬╬╬╬╬╬╬╣╣╣╣╣╣╝╜
╙╬╬╬╣╣╣╜
╙
Version information:
ml-agents: 1.1.0,
ml-agents-envs: 1.1.0,
Communicator API: 1.5.0,
PyTorch: 2.10.0+cpu
[INFO] Connected to Unity environment with package version 2.1.0-exp.1 and communication version 1.5.0
[INFO] Connected new brain: SnowballTarget?team=0
[INFO] Hyperparameters for behavior name SnowballTarget:
trainer_type: ppo
hyperparameters:
batch_size: 128
buffer_size: 2048
learning_rate: 0.0003
beta: 0.005
epsilon: 0.2
lambd: 0.95
num_epoch: 3
shared_critic: False
learning_rate_schedule: linear
beta_schedule: linear
epsilon_schedule: linear
checkpoint_interval: 200000
network_settings:
normalize: False
hidden_units: 256
num_layers: 2
vis_encode_type: simple
memory: None
goal_conditioning_type: hyper
deterministic: False
reward_signals:
extrinsic:
gamma: 0.99
strength: 1.0
network_settings:
normalize: False
hidden_units: 128
num_layers: 2
vis_encode_type: simple
memory: None
goal_conditioning_type: hyper
deterministic: False
init_path: None
keep_checkpoints: 5
even_checkpoints: False
max_steps: 1000000
time_horizon: 64
summary_freq: 10000
threaded: False
self_play: None
behavioral_cloning: None
[INFO] SnowballTarget. Step: 10000. Time Elapsed: 19.001 s. Mean Reward: 3.795. Std of Reward: 1.995. Training.
[INFO] SnowballTarget. Step: 20000. Time Elapsed: 37.908 s. Mean Reward: 6.691. Std of Reward: 2.879. Training.
[INFO] SnowballTarget. Step: 30000. Time Elapsed: 55.105 s. Mean Reward: 8.205. Std of Reward: 2.710. Training.
[INFO] SnowballTarget. Step: 40000. Time Elapsed: 73.790 s. Mean Reward: 12.782. Std of Reward: 2.896. Training.
[INFO] SnowballTarget. Step: 50000. Time Elapsed: 91.887 s. Mean Reward: 14.886. Std of Reward: 2.414. Training.
[INFO] SnowballTarget. Step: 60000. Time Elapsed: 109.500 s. Mean Reward: 15.618. Std of Reward: 2.370. Training.
[INFO] SnowballTarget. Step: 70000. Time Elapsed: 126.932 s. Mean Reward: 18.318. Std of Reward: 2.530. Training.
[INFO] SnowballTarget. Step: 80000. Time Elapsed: 145.023 s. Mean Reward: 20.745. Std of Reward: 2.414. Training.
[INFO] SnowballTarget. Step: 90000. Time Elapsed: 161.220 s. Mean Reward: 23.409. Std of Reward: 2.026. Training.
[INFO] SnowballTarget. Step: 100000. Time Elapsed: 180.526 s. Mean Reward: 23.927. Std of Reward: 2.350. Training.
[INFO] SnowballTarget. Step: 110000. Time Elapsed: 198.283 s. Mean Reward: 24.204. Std of Reward: 2.320. Training.
[INFO] SnowballTarget. Step: 120000. Time Elapsed: 215.972 s. Mean Reward: 23.311. Std of Reward: 2.648. Training.
[INFO] SnowballTarget. Step: 130000. Time Elapsed: 235.264 s. Mean Reward: 24.018. Std of Reward: 2.268. Training.
[INFO] SnowballTarget. Step: 140000. Time Elapsed: 252.434 s. Mean Reward: 25.091. Std of Reward: 2.193. Training.
[INFO] SnowballTarget. Step: 150000. Time Elapsed: 271.206 s. Mean Reward: 24.673. Std of Reward: 2.442. Training.
[INFO] SnowballTarget. Step: 160000. Time Elapsed: 289.138 s. Mean Reward: 24.727. Std of Reward: 2.425. Training.
[INFO] SnowballTarget. Step: 170000. Time Elapsed: 307.672 s. Mean Reward: 24.727. Std of Reward: 2.347. Training.
[INFO] SnowballTarget. Step: 180000. Time Elapsed: 325.413 s. Mean Reward: 22.364. Std of Reward: 3.966. Training.
[INFO] SnowballTarget. Step: 190000. Time Elapsed: 343.961 s. Mean Reward: 23.764. Std of Reward: 3.406. Training.
[INFO] SnowballTarget. Step: 200000. Time Elapsed: 360.892 s. Mean Reward: 22.909. Std of Reward: 3.168. Training.
Traceback (most recent call last):
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer_controller.py", line 175, in start_learning
n_steps = self.advance(env_manager)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents_envs/timers.py", line 305, in wrapped
return func(*args, **kwargs)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer_controller.py", line 250, in advance
trainer.advance()
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer/rl_trainer.py", line 293, in advance
self._process_trajectory(t)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/ppo/trainer.py", line 74, in _process_trajectory
super()._process_trajectory(trajectory)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer/rl_trainer.py", line 224, in _process_trajectory
self._maybe_save_model(self.get_step + len(trajectory.steps))
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer/rl_trainer.py", line 264, in _maybe_save_model
self._checkpoint()
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents_envs/timers.py", line 305, in wrapped
return func(*args, **kwargs)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer/rl_trainer.py", line 144, in _checkpoint
export_path, auxillary_paths = self.model_saver.save_checkpoint(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/model_saver/torch_model_saver.py", line 60, in save_checkpoint
self.export(checkpoint_path, behavior_name)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/model_saver/torch_model_saver.py", line 65, in export
self.exporter.export_policy_model(output_filepath)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py", line 164, in export_policy_model
torch.onnx.export(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/__init__.py", line 282, in export
from torch.onnx._internal.exporter import _compat
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_compat.py", line 16, in <module>
from torch.onnx._internal.exporter import (
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_core.py", line 19, in <module>
import onnxscript
ModuleNotFoundError: No module named 'onnxscript'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/kaggle/working/mlagents-env/bin/mlagents-learn", line 6, in <module>
sys.exit(main())
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/learn.py", line 270, in main
run_cli(parse_command_line())
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/learn.py", line 266, in run_cli
run_training(run_seed, options, num_areas)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/learn.py", line 138, in run_training
tc.start_learning(env_manager)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents_envs/timers.py", line 305, in wrapped
return func(*args, **kwargs)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer_controller.py", line 200, in start_learning
self._save_models()
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents_envs/timers.py", line 305, in wrapped
return func(*args, **kwargs)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer_controller.py", line 80, in _save_models
self.trainers[brain_name].save_model()
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer/rl_trainer.py", line 172, in save_model
model_checkpoint = self._checkpoint()
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents_envs/timers.py", line 305, in wrapped
return func(*args, **kwargs)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/trainer/rl_trainer.py", line 144, in _checkpoint
export_path, auxillary_paths = self.model_saver.save_checkpoint(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/model_saver/torch_model_saver.py", line 60, in save_checkpoint
self.export(checkpoint_path, behavior_name)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/model_saver/torch_model_saver.py", line 65, in export
self.exporter.export_policy_model(output_filepath)
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/model_serialization.py", line 164, in export_policy_model
torch.onnx.export(
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/__init__.py", line 282, in export
from torch.onnx._internal.exporter import _compat
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_compat.py", line 16, in <module>
from torch.onnx._internal.exporter import (
File "/kaggle/working/mlagents-env/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_core.py", line 19, in <module>
import onnxscript
ModuleNotFoundError: No module named 'onnxscript'
In [31]:
!/kaggle/working/mlagents-env/bin/pip install "torch==2.0.1" --index-url https://download.pytorch.org/whl/cpu --force-reinstall
!/kaggle/working/mlagents-env/bin/pip install onnxscript onnx==1.15.0
WARNING: Ignoring invalid distribution -orch (/kaggle/working/mlagents-env/lib/python3.10/site-packages) Looking in indexes: https://download.pytorch.org/whl/cpu Collecting torch==2.0.1 Using cached https://download.pytorch.org/whl/cpu/torch-2.0.1%2Bcpu-cp310-cp310-linux_x86_64.whl (195.4 MB) Collecting filelock (from torch==2.0.1) Using cached filelock-3.20.0-py3-none-any.whl.metadata (2.1 kB) Collecting typing-extensions (from torch==2.0.1) Using cached https://download.pytorch.org/whl/typing_extensions-4.15.0-py3-none-any.whl.metadata (3.3 kB) Collecting sympy (from torch==2.0.1) Using cached sympy-1.14.0-py3-none-any.whl.metadata (12 kB) Collecting networkx (from torch==2.0.1) Using cached networkx-3.4.2-py3-none-any.whl.metadata (6.3 kB) Collecting jinja2 (from torch==2.0.1) Using cached https://download.pytorch.org/whl/jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB) Collecting MarkupSafe>=2.0 (from jinja2->torch==2.0.1) Using cached https://download.pytorch.org/whl/MarkupSafe-3.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB) Collecting mpmath<1.4,>=1.1.0 (from sympy->torch==2.0.1) Using cached mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB) Using cached filelock-3.20.0-py3-none-any.whl (16 kB) Using cached https://download.pytorch.org/whl/jinja2-3.1.6-py3-none-any.whl (134 kB) Using cached https://download.pytorch.org/whl/MarkupSafe-3.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (20 kB) Using cached networkx-3.4.2-py3-none-any.whl (1.7 MB) Using cached sympy-1.14.0-py3-none-any.whl (6.3 MB) Using cached mpmath-1.3.0-py3-none-any.whl (536 kB) Using cached https://download.pytorch.org/whl/typing_extensions-4.15.0-py3-none-any.whl (44 kB) WARNING: Ignoring invalid distribution -orch (/kaggle/working/mlagents-env/lib/python3.10/site-packages) Installing collected packages: mpmath, typing-extensions, sympy, networkx, MarkupSafe, filelock, jinja2, torch Attempting uninstall: mpmath Found existing installation: mpmath 1.3.0 Uninstalling mpmath-1.3.0: Successfully uninstalled mpmath-1.3.0 Attempting uninstall: typing-extensions━━━━━━━ 0/8 [mpmath] Found existing installation: typing_extensions 4.15.0 [mpmath] Uninstalling typing_extensions-4.15.0:━━ 0/8 [mpmath] Successfully uninstalled typing_extensions-4.15.0━━━━━━━━━━━ 1/8 [typing-extensions] Attempting uninstall: sympy━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/8 [typing-extensions] Found existing installation: sympy 1.14.0━━━━━━━━━━━━━━━━━ 1/8 [typing-extensions] Uninstalling sympy-1.14.0:0m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2/8 [sympy]ensions] Successfully uninstalled sympy-1.14.0━━━━━━━━━━━━━━━━━━━━━━━ 2/8 [sympy] Attempting uninstall: networkx━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2/8 [sympy] Found existing installation: networkx 3.4.2━━━━━━━━━━━━━━━ 2/8 [sympy] Uninstalling networkx-3.4.2:╺━━━━━━━━━━━━━━━━━━━━━━━━ 3/8 [networkx] Successfully uninstalled networkx-3.4.2━━━━━━━━━━━━━━━━━ 3/8 [networkx] Attempting uninstall: MarkupSafe[0m━━━━━━━━━━━━━━━━━━━━━━━━ 3/8 [networkx] Found existing installation: MarkupSafe 3.0.2━━━━━━━━━━━━━ 3/8 [networkx] Uninstalling MarkupSafe-3.0.2:[90m━━━━━━━━━━━━━━━━━━━━━━━━ 3/8 [networkx] Successfully uninstalled MarkupSafe-3.0.2━━━━━━━━━━━━━━━ 3/8 [networkx] Attempting uninstall: filelockm━━━━━━━━━━━━━━━━━━━━━━━━ 3/8 [networkx] Found existing installation: filelock 3.20.0━━━━━━━━━━━━━━ 3/8 [networkx] Uninstalling filelock-3.20.0:╺━━━━━━━━━━━━━━ 5/8 [filelock] Successfully uninstalled filelock-3.20.00m━━━━━━━━━━━━━━ 5/8 [filelock] Attempting uninstall: jinja2[0m╺━━━━━━━━━━━━━━ 5/8 [filelock] Found existing installation: Jinja2 3.1.690m━━━━━━━━━━━━━━ 5/8 [filelock] Uninstalling Jinja2-3.1.6:[0m╺━━━━━━━━━━━━━━ 5/8 [filelock] Successfully uninstalled Jinja2-3.1.6━━━━━━━━━━━━━━ 5/8 [filelock] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8/8 [torch]32m7/8 [torch] ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. mlagents 1.1.0 requires torch>=2.1.1, but you have torch 2.0.1+cpu which is incompatible. WARNING: Ignoring invalid distribution -orch (/kaggle/working/mlagents-env/lib/python3.10/site-packages) Successfully installed MarkupSafe-3.0.2 filelock-3.20.0 jinja2-3.1.6 mpmath-1.3.0 networkx-3.4.2 sympy-1.14.0 torch-2.0.1+cpu typing-extensions-4.15.0 WARNING: Ignoring invalid distribution -orch (/kaggle/working/mlagents-env/lib/python3.10/site-packages) Collecting onnxscript Using cached onnxscript-0.6.2-py3-none-any.whl.metadata (13 kB) Requirement already satisfied: onnx==1.15.0 in ./mlagents-env/lib/python3.10/site-packages (1.15.0) Requirement already satisfied: numpy in ./mlagents-env/lib/python3.10/site-packages (from onnx==1.15.0) (1.23.5) Requirement already satisfied: protobuf>=3.20.2 in ./mlagents-env/lib/python3.10/site-packages (from onnx==1.15.0) (3.20.3) Collecting ml_dtypes (from onnxscript) Using cached ml_dtypes-0.5.4-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (8.9 kB) Collecting onnx_ir<2,>=0.1.15 (from onnxscript) Using cached onnx_ir-0.2.0-py3-none-any.whl.metadata (3.3 kB) INFO: pip is looking at multiple versions of onnxscript to determine which version is compatible with other requirements. This could take a while. Collecting onnxscript Using cached onnxscript-0.6.1-py3-none-any.whl.metadata (13 kB) Using cached onnxscript-0.6.0-py3-none-any.whl.metadata (13 kB) Using cached onnxscript-0.5.7-py3-none-any.whl.metadata (13 kB) Using cached onnxscript-0.5.6-py3-none-any.whl.metadata (13 kB) Using cached onnxscript-0.5.4-py3-none-any.whl.metadata (13 kB) Using cached onnxscript-0.5.3-py3-none-any.whl.metadata (13 kB) Using cached onnxscript-0.5.2-py3-none-any.whl.metadata (13 kB) INFO: pip is still looking at multiple versions of onnxscript to determine which version is compatible with other requirements. This could take a while. Using cached onnxscript-0.5.1-py3-none-any.whl.metadata (13 kB) Using cached onnxscript-0.5.0-py3-none-any.whl.metadata (13 kB) Using cached onnxscript-0.4.0-py3-none-any.whl.metadata (13 kB) Using cached onnxscript-0.3.2-py3-none-any.whl.metadata (13 kB) Using cached onnxscript-0.3.1-py3-none-any.whl.metadata (13 kB) INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C. Using cached onnxscript-0.3.0-py3-none-any.whl.metadata (14 kB) Using cached onnxscript-0.2.7-py3-none-any.whl.metadata (15 kB) Using cached onnxscript-0.2.6-py3-none-any.whl.metadata (15 kB) Using cached onnxscript-0.2.5-py3-none-any.whl.metadata (15 kB) Using cached onnxscript-0.2.4-py3-none-any.whl.metadata (15 kB) Using cached onnxscript-0.2.3-py3-none-any.whl.metadata (15 kB) Using cached onnxscript-0.2.2-py3-none-any.whl.metadata (15 kB) Using cached onnxscript-0.2.1-py3-none-any.whl.metadata (15 kB) Using cached onnxscript-0.2.0-py3-none-any.whl.metadata (15 kB) Using cached onnxscript-0.1.0-py3-none-any.whl.metadata (15 kB) Collecting onnx==1.15.0 Using cached onnx-1.15.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (15 kB) ERROR: Cannot install onnx==1.15.0, onnxscript==0.1.0, onnxscript==0.2.0, onnxscript==0.2.1, onnxscript==0.2.2, onnxscript==0.2.3, onnxscript==0.2.4, onnxscript==0.2.5, onnxscript==0.2.6, onnxscript==0.2.7, onnxscript==0.3.0, onnxscript==0.3.1, onnxscript==0.3.2, onnxscript==0.4.0, onnxscript==0.5.0, onnxscript==0.5.1, onnxscript==0.5.2, onnxscript==0.5.3, onnxscript==0.5.4, onnxscript==0.5.6, onnxscript==0.5.7, onnxscript==0.6.0, onnxscript==0.6.1 and onnxscript==0.6.2 because these package versions have conflicting dependencies. The conflict is caused by: The user requested onnx==1.15.0 onnxscript 0.6.2 depends on onnx>=1.17 onnxscript 0.6.1 depends on onnx>=1.17 onnxscript 0.6.0 depends on onnx>=1.17 onnxscript 0.5.7 depends on onnx>=1.16 onnxscript 0.5.6 depends on onnx>=1.16 onnxscript 0.5.4 depends on onnx>=1.16 onnxscript 0.5.3 depends on onnx>=1.16 onnxscript 0.5.2 depends on onnx>=1.16 onnxscript 0.5.1 depends on onnx>=1.16 onnxscript 0.5.0 depends on onnx>=1.16 onnxscript 0.4.0 depends on onnx>=1.16 onnxscript 0.3.2 depends on onnx>=1.16 onnxscript 0.3.1 depends on onnx>=1.16 onnxscript 0.3.0 depends on onnx>=1.16 onnxscript 0.2.7 depends on onnx>=1.16 onnxscript 0.2.6 depends on onnx>=1.16 onnxscript 0.2.5 depends on onnx>=1.16 onnxscript 0.2.4 depends on onnx>=1.16 onnxscript 0.2.3 depends on onnx>=1.16 onnxscript 0.2.2 depends on onnx>=1.16 onnxscript 0.2.1 depends on onnx>=1.16 onnxscript 0.2.0 depends on onnx>=1.16 onnxscript 0.1.0 depends on onnx>=1.16 Additionally, some packages in these conflicts have no matching distributions available for your environment: onnx To fix this you could try to: 1. loosen the range of package versions you've specified 2. remove package versions to allow pip to attempt to solve the dependency conflict ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
In [32]:
!/kaggle/working/mlagents-env/bin/mlagents-learn \
/kaggle/working/ml-agents/config/ppo/SnowballTarget.yaml \
--env=/kaggle/working/training-envs-executables/SnowballTarget/SnowballTarget/SnowballTarget.x86_64 \
--run-id="SnowballTarget1" \
--no-graphics \
--resume
┐ ╖
╓╖╬│╡ ││╬╖╖
╓╖╬│││││┘ ╬│││││╬╖
╖╬│││││╬╜ ╙╬│││││╖╖ ╗╗╗
╬╬╬╬╖││╦╖ ╖╬││╗╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╜╜╜ ╟╣╣
╬╬╬╬╬╬╬╬╖│╬╖╖╓╬╪│╓╣╣╣╣╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╒╣╣╖╗╣╣╣╗ ╣╣╣ ╣╣╣╣╣╣ ╟╣╣╖ ╣╣╣
╬╬╬╬┐ ╙╬╬╬╬│╓╣╣╣╝╜ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╣╙ ╙╣╣╣ ╣╣╣ ╙╟╣╣╜╙ ╫╣╣ ╟╣╣
╬╬╬╬┐ ╙╬╬╣╣ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣ ╣╣╣┌╣╣╜
╬╬╬╜ ╬╬╣╣ ╙╝╣╣╬ ╙╣╣╣╗╖╓╗╣╣╣╜ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣╦╓ ╣╣╣╣╣
╙ ╓╦╖ ╬╬╣╣ ╓╗╗╖ ╙╝╣╣╣╣╝╜ ╘╝╝╜ ╝╝╝ ╝╝╝ ╙╣╣╣ ╟╣╣╣
╩╬╬╬╬╬╬╦╦╬╬╣╣╗╣╣╣╣╣╣╣╝ ╫╣╣╣╣
╙╬╬╬╬╬╬╬╣╣╣╣╣╣╝╜
╙╬╬╬╣╣╣╜
╙
Version information:
ml-agents: 1.1.0,
ml-agents-envs: 1.1.0,
Communicator API: 1.5.0,
PyTorch: 2.0.1+cpu
[WARNING] PyTorch checkpoint was saved with a different version of PyTorch. Model may not resume properly.
[INFO] Connected to Unity environment with package version 2.1.0-exp.1 and communication version 1.5.0
[INFO] Connected new brain: SnowballTarget?team=0
[INFO] Hyperparameters for behavior name SnowballTarget:
trainer_type: ppo
hyperparameters:
batch_size: 128
buffer_size: 2048
learning_rate: 0.0003
beta: 0.005
epsilon: 0.2
lambd: 0.95
num_epoch: 3
shared_critic: False
learning_rate_schedule: linear
beta_schedule: linear
epsilon_schedule: linear
checkpoint_interval: 200000
network_settings:
normalize: False
hidden_units: 256
num_layers: 2
vis_encode_type: simple
memory: None
goal_conditioning_type: hyper
deterministic: False
reward_signals:
extrinsic:
gamma: 0.99
strength: 1.0
network_settings:
normalize: False
hidden_units: 128
num_layers: 2
vis_encode_type: simple
memory: None
goal_conditioning_type: hyper
deterministic: False
init_path: None
keep_checkpoints: 5
even_checkpoints: False
max_steps: 1000000
time_horizon: 64
summary_freq: 10000
threaded: False
self_play: None
behavioral_cloning: None
[INFO] Resuming from results/SnowballTarget1/SnowballTarget.
[INFO] Resuming training from step 199984.
[INFO] SnowballTarget. Step: 200000. Time Elapsed: 1.629 s. No episode was completed since last summary. Training.
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results/SnowballTarget1/SnowballTarget/SnowballTarget-199984.onnx
[INFO] SnowballTarget. Step: 210000. Time Elapsed: 19.013 s. Mean Reward: 24.909. Std of Reward: 2.475. Training.
[INFO] SnowballTarget. Step: 220000. Time Elapsed: 37.747 s. Mean Reward: 24.636. Std of Reward: 2.307. Training.
[INFO] SnowballTarget. Step: 230000. Time Elapsed: 55.985 s. Mean Reward: 25.227. Std of Reward: 2.204. Training.
[INFO] SnowballTarget. Step: 240000. Time Elapsed: 74.304 s. Mean Reward: 25.218. Std of Reward: 2.592. Training.
[INFO] SnowballTarget. Step: 250000. Time Elapsed: 92.591 s. Mean Reward: 24.864. Std of Reward: 2.904. Training.
[INFO] SnowballTarget. Step: 260000. Time Elapsed: 111.120 s. Mean Reward: 25.036. Std of Reward: 2.264. Training.
[INFO] SnowballTarget. Step: 270000. Time Elapsed: 129.744 s. Mean Reward: 26.205. Std of Reward: 1.816. Training.
[INFO] SnowballTarget. Step: 280000. Time Elapsed: 148.789 s. Mean Reward: 25.509. Std of Reward: 2.070. Training.
[INFO] SnowballTarget. Step: 290000. Time Elapsed: 165.501 s. Mean Reward: 25.795. Std of Reward: 2.519. Training.
[INFO] SnowballTarget. Step: 300000. Time Elapsed: 184.637 s. Mean Reward: 25.945. Std of Reward: 2.136. Training.
[INFO] SnowballTarget. Step: 310000. Time Elapsed: 203.207 s. Mean Reward: 25.982. Std of Reward: 2.461. Training.
[INFO] SnowballTarget. Step: 320000. Time Elapsed: 220.336 s. Mean Reward: 26.455. Std of Reward: 2.189. Training.
[INFO] SnowballTarget. Step: 330000. Time Elapsed: 238.905 s. Mean Reward: 25.673. Std of Reward: 2.081. Training.
[INFO] SnowballTarget. Step: 340000. Time Elapsed: 256.520 s. Mean Reward: 25.682. Std of Reward: 2.353. Training.
[INFO] SnowballTarget. Step: 350000. Time Elapsed: 274.644 s. Mean Reward: 26.400. Std of Reward: 2.179. Training.
[INFO] SnowballTarget. Step: 360000. Time Elapsed: 292.174 s. Mean Reward: 26.091. Std of Reward: 2.294. Training.
[INFO] SnowballTarget. Step: 370000. Time Elapsed: 310.135 s. Mean Reward: 25.909. Std of Reward: 2.391. Training.
[INFO] SnowballTarget. Step: 380000. Time Elapsed: 327.947 s. Mean Reward: 26.364. Std of Reward: 2.186. Training.
[INFO] SnowballTarget. Step: 390000. Time Elapsed: 347.061 s. Mean Reward: 27.018. Std of Reward: 2.111. Training.
[INFO] SnowballTarget. Step: 400000. Time Elapsed: 364.352 s. Mean Reward: 26.091. Std of Reward: 1.940. Training.
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results/SnowballTarget1/SnowballTarget/SnowballTarget-399968.onnx
[INFO] SnowballTarget. Step: 410000. Time Elapsed: 383.094 s. Mean Reward: 26.727. Std of Reward: 2.178. Training.
[INFO] SnowballTarget. Step: 420000. Time Elapsed: 402.364 s. Mean Reward: 26.582. Std of Reward: 2.349. Training.
[INFO] SnowballTarget. Step: 430000. Time Elapsed: 418.857 s. Mean Reward: 25.886. Std of Reward: 2.525. Training.
[INFO] SnowballTarget. Step: 440000. Time Elapsed: 438.141 s. Mean Reward: 26.236. Std of Reward: 2.140. Training.
[INFO] SnowballTarget. Step: 450000. Time Elapsed: 456.119 s. Mean Reward: 26.523. Std of Reward: 2.221. Training.
[INFO] SnowballTarget. Step: 460000. Time Elapsed: 474.642 s. Mean Reward: 25.945. Std of Reward: 2.611. Training.
[INFO] SnowballTarget. Step: 470000. Time Elapsed: 492.296 s. Mean Reward: 26.682. Std of Reward: 2.353. Training.
[INFO] SnowballTarget. Step: 480000. Time Elapsed: 510.310 s. Mean Reward: 26.600. Std of Reward: 2.023. Training.
[INFO] SnowballTarget. Step: 490000. Time Elapsed: 528.375 s. Mean Reward: 26.250. Std of Reward: 2.861. Training.
[INFO] SnowballTarget. Step: 500000. Time Elapsed: 547.245 s. Mean Reward: 26.855. Std of Reward: 1.566. Training.
[INFO] SnowballTarget. Step: 510000. Time Elapsed: 563.757 s. Mean Reward: 26.614. Std of Reward: 2.025. Training.
[INFO] SnowballTarget. Step: 520000. Time Elapsed: 582.670 s. Mean Reward: 27.236. Std of Reward: 2.191. Training.
[INFO] SnowballTarget. Step: 530000. Time Elapsed: 601.421 s. Mean Reward: 27.018. Std of Reward: 2.136. Training.
[INFO] SnowballTarget. Step: 540000. Time Elapsed: 618.274 s. Mean Reward: 26.705. Std of Reward: 2.360. Training.
[INFO] SnowballTarget. Step: 550000. Time Elapsed: 636.728 s. Mean Reward: 26.945. Std of Reward: 2.084. Training.
[INFO] SnowballTarget. Step: 560000. Time Elapsed: 654.531 s. Mean Reward: 26.977. Std of Reward: 1.631. Training.
[INFO] SnowballTarget. Step: 570000. Time Elapsed: 672.480 s. Mean Reward: 26.800. Std of Reward: 2.022. Training.
[INFO] SnowballTarget. Step: 580000. Time Elapsed: 689.968 s. Mean Reward: 26.795. Std of Reward: 2.191. Training.
[INFO] SnowballTarget. Step: 590000. Time Elapsed: 707.380 s. Mean Reward: 27.491. Std of Reward: 1.934. Training.
[INFO] SnowballTarget. Step: 600000. Time Elapsed: 724.653 s. Mean Reward: 27.500. Std of Reward: 1.865. Training.
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results/SnowballTarget1/SnowballTarget/SnowballTarget-599976.onnx
[INFO] SnowballTarget. Step: 610000. Time Elapsed: 743.710 s. Mean Reward: 27.309. Std of Reward: 2.173. Training.
[INFO] SnowballTarget. Step: 620000. Time Elapsed: 760.940 s. Mean Reward: 26.341. Std of Reward: 2.131. Training.
[INFO] SnowballTarget. Step: 630000. Time Elapsed: 779.676 s. Mean Reward: 26.400. Std of Reward: 2.085. Training.
[INFO] SnowballTarget. Step: 640000. Time Elapsed: 798.535 s. Mean Reward: 27.018. Std of Reward: 1.931. Training.
[INFO] SnowballTarget. Step: 650000. Time Elapsed: 815.239 s. Mean Reward: 27.136. Std of Reward: 2.029. Training.
[INFO] SnowballTarget. Step: 660000. Time Elapsed: 834.252 s. Mean Reward: 27.400. Std of Reward: 1.825. Training.
[INFO] SnowballTarget. Step: 670000. Time Elapsed: 852.775 s. Mean Reward: 27.614. Std of Reward: 1.921. Training.
[INFO] SnowballTarget. Step: 680000. Time Elapsed: 870.797 s. Mean Reward: 27.055. Std of Reward: 2.136. Training.
[INFO] SnowballTarget. Step: 690000. Time Elapsed: 888.840 s. Mean Reward: 27.273. Std of Reward: 1.737. Training.
[INFO] SnowballTarget. Step: 700000. Time Elapsed: 907.005 s. Mean Reward: 26.800. Std of Reward: 1.976. Training.
[INFO] SnowballTarget. Step: 710000. Time Elapsed: 924.374 s. Mean Reward: 27.182. Std of Reward: 1.934. Training.
[INFO] SnowballTarget. Step: 720000. Time Elapsed: 943.753 s. Mean Reward: 26.964. Std of Reward: 1.926. Training.
[INFO] SnowballTarget. Step: 730000. Time Elapsed: 960.548 s. Mean Reward: 27.136. Std of Reward: 2.138. Training.
[INFO] SnowballTarget. Step: 740000. Time Elapsed: 979.223 s. Mean Reward: 27.127. Std of Reward: 1.849. Training.
[INFO] SnowballTarget. Step: 750000. Time Elapsed: 997.664 s. Mean Reward: 27.618. Std of Reward: 2.170. Training.
[INFO] SnowballTarget. Step: 760000. Time Elapsed: 1014.738 s. Mean Reward: 27.295. Std of Reward: 1.890. Training.
[INFO] SnowballTarget. Step: 770000. Time Elapsed: 1033.298 s. Mean Reward: 27.327. Std of Reward: 1.695. Training.
[INFO] SnowballTarget. Step: 780000. Time Elapsed: 1051.449 s. Mean Reward: 27.205. Std of Reward: 2.292. Training.
[INFO] SnowballTarget. Step: 790000. Time Elapsed: 1069.487 s. Mean Reward: 27.418. Std of Reward: 2.051. Training.
[INFO] SnowballTarget. Step: 800000. Time Elapsed: 1087.116 s. Mean Reward: 27.114. Std of Reward: 1.945. Training.
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results/SnowballTarget1/SnowballTarget/SnowballTarget-799984.onnx
[INFO] SnowballTarget. Step: 810000. Time Elapsed: 1105.034 s. Mean Reward: 27.091. Std of Reward: 1.822. Training.
[INFO] SnowballTarget. Step: 820000. Time Elapsed: 1123.226 s. Mean Reward: 27.705. Std of Reward: 1.890. Training.
[INFO] SnowballTarget. Step: 830000. Time Elapsed: 1141.849 s. Mean Reward: 27.127. Std of Reward: 2.081. Training.
[INFO] SnowballTarget. Step: 840000. Time Elapsed: 1158.724 s. Mean Reward: 26.955. Std of Reward: 2.142. Training.
[INFO] SnowballTarget. Step: 850000. Time Elapsed: 1177.338 s. Mean Reward: 27.273. Std of Reward: 1.793. Training.
[INFO] SnowballTarget. Step: 860000. Time Elapsed: 1196.156 s. Mean Reward: 27.945. Std of Reward: 2.004. Training.
[INFO] SnowballTarget. Step: 870000. Time Elapsed: 1212.851 s. Mean Reward: 27.500. Std of Reward: 2.083. Training.
[INFO] SnowballTarget. Step: 880000. Time Elapsed: 1232.091 s. Mean Reward: 27.436. Std of Reward: 1.970. Training.
[INFO] SnowballTarget. Step: 890000. Time Elapsed: 1250.109 s. Mean Reward: 27.841. Std of Reward: 1.770. Training.
[INFO] SnowballTarget. Step: 900000. Time Elapsed: 1268.321 s. Mean Reward: 27.218. Std of Reward: 2.086. Training.
[INFO] SnowballTarget. Step: 910000. Time Elapsed: 1286.310 s. Mean Reward: 27.591. Std of Reward: 1.762. Training.
[INFO] SnowballTarget. Step: 920000. Time Elapsed: 1304.378 s. Mean Reward: 27.473. Std of Reward: 1.953. Training.
[INFO] SnowballTarget. Step: 930000. Time Elapsed: 1322.035 s. Mean Reward: 27.659. Std of Reward: 2.121. Training.
[INFO] SnowballTarget. Step: 940000. Time Elapsed: 1340.872 s. Mean Reward: 27.164. Std of Reward: 1.837. Training.
[INFO] SnowballTarget. Step: 950000. Time Elapsed: 1357.849 s. Mean Reward: 27.318. Std of Reward: 1.427. Training.
[INFO] SnowballTarget. Step: 960000. Time Elapsed: 1376.915 s. Mean Reward: 27.709. Std of Reward: 2.394. Training.
[INFO] SnowballTarget. Step: 970000. Time Elapsed: 1395.783 s. Mean Reward: 27.655. Std of Reward: 2.108. Training.
[INFO] SnowballTarget. Step: 980000. Time Elapsed: 1412.830 s. Mean Reward: 27.750. Std of Reward: 1.734. Training.
[INFO] SnowballTarget. Step: 990000. Time Elapsed: 1431.565 s. Mean Reward: 27.364. Std of Reward: 2.066. Training.
[INFO] SnowballTarget. Step: 1000000. Time Elapsed: 1449.492 s. Mean Reward: 27.614. Std of Reward: 2.134. Training.
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results/SnowballTarget1/SnowballTarget/SnowballTarget-999992.onnx
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results/SnowballTarget1/SnowballTarget/SnowballTarget-1000696.onnx
[INFO] Copied results/SnowballTarget1/SnowballTarget/SnowballTarget-1000696.onnx to results/SnowballTarget1/SnowballTarget.onnx.
Part 2: Pyramids 🏛️¶
Train an agent to press a button, navigate to a pyramid, knock it over, and reach the gold brick using curiosity-driven exploration.
Create the Pyramids config file (with curiosity)¶
In [39]:
pyramids_config = """
behaviors:
Pyramids:
trainer_type: ppo
hyperparameters:
batch_size: 128
buffer_size: 2048
learning_rate: 0.0003
beta: 0.01
epsilon: 0.2
lambd: 0.95
num_epoch: 3
learning_rate_schedule: linear
network_settings:
normalize: false
hidden_units: 256
num_layers: 2
vis_encode_type: simple
reward_signals:
extrinsic:
gamma: 0.99
strength: 1.0
curiosity:
gamma: 0.99
strength: 0.02
encoding_size: 256
learning_rate: 0.0003
checkpoint_interval: 200000
keep_checkpoints: 5
max_steps: 1500000
time_horizon: 128
summary_freq: 10000
"""
with open("/kaggle/working/ml-agents/config/ppo/Pyramids.yaml", "w") as f:
f.write(pyramids_config)
print("✅ Pyramids config written (with curiosity reward) to: /kaggle/working/ml-agents/config/ppo/Pyramids.yaml")
✅ Pyramids config written (with curiosity reward) to: /kaggle/working/ml-agents/config/ppo/Pyramids.yaml
Train Pyramids agent¶
Training takes ~45–60 minutes. Curiosity helps the agent explore and discover the pyramid task.
In [46]:
!/kaggle/working/mlagents-env/bin/mlagents-learn \
/kaggle/working/ml-agents/config/ppo/Pyramids.yaml \
--env=./training-envs-executables/Pyramids/Pyramids/Pyramids \
--run-id="Pyramids1" \
--no-graphics \
--force
[WARNING] 'encoding_size' was deprecated for RewardSignals. Please use network_settings.
┐ ╖
╓╖╬│╡ ││╬╖╖
╓╖╬│││││┘ ╬│││││╬╖
╖╬│││││╬╜ ╙╬│││││╖╖ ╗╗╗
╬╬╬╬╖││╦╖ ╖╬││╗╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╜╜╜ ╟╣╣
╬╬╬╬╬╬╬╬╖│╬╖╖╓╬╪│╓╣╣╣╣╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╒╣╣╖╗╣╣╣╗ ╣╣╣ ╣╣╣╣╣╣ ╟╣╣╖ ╣╣╣
╬╬╬╬┐ ╙╬╬╬╬│╓╣╣╣╝╜ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╣╙ ╙╣╣╣ ╣╣╣ ╙╟╣╣╜╙ ╫╣╣ ╟╣╣
╬╬╬╬┐ ╙╬╬╣╣ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣ ╣╣╣┌╣╣╜
╬╬╬╜ ╬╬╣╣ ╙╝╣╣╬ ╙╣╣╣╗╖╓╗╣╣╣╜ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣╦╓ ╣╣╣╣╣
╙ ╓╦╖ ╬╬╣╣ ╓╗╗╖ ╙╝╣╣╣╣╝╜ ╘╝╝╜ ╝╝╝ ╝╝╝ ╙╣╣╣ ╟╣╣╣
╩╬╬╬╬╬╬╦╦╬╬╣╣╗╣╣╣╣╣╣╣╝ ╫╣╣╣╣
╙╬╬╬╬╬╬╬╣╣╣╣╣╣╝╜
╙╬╬╬╣╣╣╜
╙
Version information:
ml-agents: 1.1.0,
ml-agents-envs: 1.1.0,
Communicator API: 1.5.0,
PyTorch: 2.0.1+cpu
[INFO] Connected to Unity environment with package version 2.2.1-exp.1 and communication version 1.5.0
[INFO] Connected new brain: Pyramids?team=0
[INFO] Hyperparameters for behavior name Pyramids:
trainer_type: ppo
hyperparameters:
batch_size: 128
buffer_size: 2048
learning_rate: 0.0003
beta: 0.01
epsilon: 0.2
lambd: 0.95
num_epoch: 3
shared_critic: False
learning_rate_schedule: linear
beta_schedule: linear
epsilon_schedule: linear
checkpoint_interval: 200000
network_settings:
normalize: False
hidden_units: 256
num_layers: 2
vis_encode_type: simple
memory: None
goal_conditioning_type: hyper
deterministic: False
reward_signals:
extrinsic:
gamma: 0.99
strength: 1.0
network_settings:
normalize: False
hidden_units: 128
num_layers: 2
vis_encode_type: simple
memory: None
goal_conditioning_type: hyper
deterministic: False
curiosity:
gamma: 0.99
strength: 0.02
network_settings:
normalize: False
hidden_units: 256
num_layers: 2
vis_encode_type: simple
memory: None
goal_conditioning_type: hyper
deterministic: False
learning_rate: 0.0003
encoding_size: 256
init_path: None
keep_checkpoints: 5
even_checkpoints: False
max_steps: 1500000
time_horizon: 128
summary_freq: 10000
threaded: False
self_play: None
behavioral_cloning: None
/kaggle/working/mlagents-env/lib/python3.10/site-packages/mlagents/trainers/torch_entities/utils.py:289: UserWarning: The use of `x.T` on tensors of dimension other than 2 to reverse their shape is deprecated and it will throw an error in a future release. Consider `x.mT` to transpose batches of matrices or `x.permute(*torch.arange(x.ndim - 1, -1, -1))` to reverse the dimensions of a tensor. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3571.)
torch.nn.functional.one_hot(_act.T, action_size[i]).float()
[INFO] Pyramids. Step: 10000. Time Elapsed: 17.756 s. No episode was completed since last summary. Training.
[INFO] Pyramids. Step: 20000. Time Elapsed: 33.986 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 30000. Time Elapsed: 53.988 s. No episode was completed since last summary. Training.
[INFO] Pyramids. Step: 40000. Time Elapsed: 72.397 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 50000. Time Elapsed: 94.296 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 60000. Time Elapsed: 113.265 s. No episode was completed since last summary. Training.
[INFO] Pyramids. Step: 70000. Time Elapsed: 134.724 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 80000. Time Elapsed: 154.479 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 90000. Time Elapsed: 177.123 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 100000. Time Elapsed: 196.413 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 110000. Time Elapsed: 217.226 s. No episode was completed since last summary. Training.
[INFO] Pyramids. Step: 120000. Time Elapsed: 236.291 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 130000. Time Elapsed: 258.261 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 140000. Time Elapsed: 279.391 s. Mean Reward: 1.636. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 150000. Time Elapsed: 301.032 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 160000. Time Elapsed: 322.935 s. Mean Reward: -0.859. Std of Reward: 0.527. Training.
[INFO] Pyramids. Step: 170000. Time Elapsed: 345.800 s. Mean Reward: -0.999. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 180000. Time Elapsed: 371.130 s. Mean Reward: -0.845. Std of Reward: 0.579. Training.
[INFO] Pyramids. Step: 190000. Time Elapsed: 390.961 s. Mean Reward: -0.409. Std of Reward: 1.023. Training.
[INFO] Pyramids. Step: 200000. Time Elapsed: 413.524 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results/Pyramids1/Pyramids/Pyramids-199893.onnx
[INFO] Pyramids. Step: 210000. Time Elapsed: 435.773 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 220000. Time Elapsed: 455.510 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 230000. Time Elapsed: 476.366 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 240000. Time Elapsed: 499.116 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 250000. Time Elapsed: 517.418 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 260000. Time Elapsed: 541.526 s. Mean Reward: -0.857. Std of Reward: 0.535. Training.
[INFO] Pyramids. Step: 270000. Time Elapsed: 561.436 s. Mean Reward: -0.362. Std of Reward: 1.104. Training.
[INFO] Pyramids. Step: 280000. Time Elapsed: 582.724 s. Mean Reward: -0.651. Std of Reward: 0.869. Training.
[INFO] Pyramids. Step: 290000. Time Elapsed: 606.796 s. Mean Reward: -0.854. Std of Reward: 0.527. Training.
[INFO] Pyramids. Step: 300000. Time Elapsed: 626.147 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 310000. Time Elapsed: 648.971 s. Mean Reward: -0.830. Std of Reward: 0.590. Training.
[INFO] Pyramids. Step: 320000. Time Elapsed: 670.158 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 330000. Time Elapsed: 689.621 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 340000. Time Elapsed: 712.316 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 350000. Time Elapsed: 733.511 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 360000. Time Elapsed: 756.097 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 370000. Time Elapsed: 777.613 s. Mean Reward: -0.844. Std of Reward: 0.583. Training.
[INFO] Pyramids. Step: 380000. Time Elapsed: 798.685 s. Mean Reward: -0.557. Std of Reward: 0.886. Training.
[INFO] Pyramids. Step: 390000. Time Elapsed: 822.755 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 400000. Time Elapsed: 844.765 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results/Pyramids1/Pyramids/Pyramids-399998.onnx
[INFO] Pyramids. Step: 410000. Time Elapsed: 867.054 s. Mean Reward: -0.129. Std of Reward: 1.238. Training.
[INFO] Pyramids. Step: 420000. Time Elapsed: 889.200 s. Mean Reward: -0.665. Std of Reward: 0.821. Training.
[INFO] Pyramids. Step: 430000. Time Elapsed: 910.580 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 440000. Time Elapsed: 929.504 s. Mean Reward: -0.806. Std of Reward: 0.672. Training.
[INFO] Pyramids. Step: 450000. Time Elapsed: 954.202 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 460000. Time Elapsed: 974.237 s. Mean Reward: -0.580. Std of Reward: 1.029. Training.
[INFO] Pyramids. Step: 470000. Time Elapsed: 997.153 s. Mean Reward: -0.771. Std of Reward: 0.760. Training.
[INFO] Pyramids. Step: 480000. Time Elapsed: 1021.295 s. Mean Reward: -0.803. Std of Reward: 0.681. Training.
[INFO] Pyramids. Step: 490000. Time Elapsed: 1044.781 s. Mean Reward: -0.575. Std of Reward: 0.949. Training.
[INFO] Pyramids. Step: 500000. Time Elapsed: 1071.086 s. Mean Reward: -0.564. Std of Reward: 0.874. Training.
[INFO] Pyramids. Step: 510000. Time Elapsed: 1093.338 s. Mean Reward: -0.523. Std of Reward: 0.954. Training.
[INFO] Pyramids. Step: 520000. Time Elapsed: 1120.895 s. Mean Reward: -0.499. Std of Reward: 0.964. Training.
[INFO] Pyramids. Step: 530000. Time Elapsed: 1146.384 s. Mean Reward: -0.459. Std of Reward: 0.991. Training.
[INFO] Pyramids. Step: 540000. Time Elapsed: 1171.356 s. Mean Reward: -0.399. Std of Reward: 1.042. Training.
[INFO] Pyramids. Step: 550000. Time Elapsed: 1196.973 s. Mean Reward: -0.087. Std of Reward: 1.086. Training.
[INFO] Pyramids. Step: 560000. Time Elapsed: 1222.421 s. Mean Reward: -0.790. Std of Reward: 0.665. Training.
[INFO] Pyramids. Step: 570000. Time Elapsed: 1248.902 s. Mean Reward: -0.496. Std of Reward: 1.069. Training.
[INFO] Pyramids. Step: 580000. Time Elapsed: 1274.425 s. Mean Reward: -0.027. Std of Reward: 1.163. Training.
[INFO] Pyramids. Step: 590000. Time Elapsed: 1299.765 s. Mean Reward: -0.486. Std of Reward: 0.962. Training.
[INFO] Pyramids. Step: 600000. Time Elapsed: 1325.589 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results/Pyramids1/Pyramids/Pyramids-599991.onnx
[INFO] Pyramids. Step: 610000. Time Elapsed: 1347.930 s. Mean Reward: -0.535. Std of Reward: 0.937. Training.
[INFO] Pyramids. Step: 620000. Time Elapsed: 1367.798 s. Mean Reward: -0.472. Std of Reward: 0.998. Training.
[INFO] Pyramids. Step: 630000. Time Elapsed: 1390.923 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 640000. Time Elapsed: 1416.431 s. Mean Reward: -0.602. Std of Reward: 0.977. Training.
[INFO] Pyramids. Step: 650000. Time Elapsed: 1436.191 s. Mean Reward: -0.509. Std of Reward: 1.046. Training.
[INFO] Pyramids. Step: 660000. Time Elapsed: 1459.233 s. Mean Reward: -0.779. Std of Reward: 0.662. Training.
[INFO] Pyramids. Step: 670000. Time Elapsed: 1481.262 s. Mean Reward: -0.203. Std of Reward: 1.136. Training.
[INFO] Pyramids. Step: 680000. Time Elapsed: 1502.378 s. Mean Reward: -0.511. Std of Reward: 0.979. Training.
[INFO] Pyramids. Step: 690000. Time Elapsed: 1526.764 s. Mean Reward: -0.773. Std of Reward: 0.716. Training.
[INFO] Pyramids. Step: 700000. Time Elapsed: 1551.514 s. Mean Reward: -0.546. Std of Reward: 0.909. Training.
[INFO] Pyramids. Step: 710000. Time Elapsed: 1575.423 s. Mean Reward: -0.305. Std of Reward: 1.069. Training.
[INFO] Pyramids. Step: 720000. Time Elapsed: 1596.748 s. Mean Reward: -0.637. Std of Reward: 0.856. Training.
[INFO] Pyramids. Step: 730000. Time Elapsed: 1620.093 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 740000. Time Elapsed: 1644.521 s. Mean Reward: 0.016. Std of Reward: 1.218. Training.
[INFO] Pyramids. Step: 750000. Time Elapsed: 1670.498 s. Mean Reward: -0.518. Std of Reward: 1.031. Training.
[INFO] Pyramids. Step: 760000. Time Elapsed: 1692.517 s. Mean Reward: -0.513. Std of Reward: 0.976. Training.
[INFO] Pyramids. Step: 770000. Time Elapsed: 1717.586 s. Mean Reward: -0.299. Std of Reward: 1.124. Training.
[INFO] Pyramids. Step: 780000. Time Elapsed: 1741.320 s. Mean Reward: -0.451. Std of Reward: 1.042. Training.
[INFO] Pyramids. Step: 790000. Time Elapsed: 1764.947 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.
[INFO] Pyramids. Step: 800000. Time Elapsed: 1790.069 s. Mean Reward: -0.218. Std of Reward: 1.174. Training.
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results/Pyramids1/Pyramids/Pyramids-799915.onnx
[INFO] Pyramids. Step: 810000. Time Elapsed: 1814.915 s. Mean Reward: 0.215. Std of Reward: 1.237. Training.
[INFO] Pyramids. Step: 820000. Time Elapsed: 1837.599 s. Mean Reward: -0.109. Std of Reward: 1.141. Training.
[INFO] Pyramids. Step: 830000. Time Elapsed: 1862.661 s. Mean Reward: -0.284. Std of Reward: 1.076. Training.
[INFO] Pyramids. Step: 840000. Time Elapsed: 1887.899 s. Mean Reward: -0.203. Std of Reward: 1.136. Training.
[INFO] Pyramids. Step: 850000. Time Elapsed: 1910.626 s. Mean Reward: -0.367. Std of Reward: 1.048. Training.
[INFO] Pyramids. Step: 860000. Time Elapsed: 1936.349 s. Mean Reward: -0.400. Std of Reward: 1.047. Training.
[INFO] Pyramids. Step: 870000. Time Elapsed: 1960.875 s. Mean Reward: 0.563. Std of Reward: 1.121. Training.
[INFO] Pyramids. Step: 880000. Time Elapsed: 1985.631 s. Mean Reward: -0.589. Std of Reward: 0.872. Training.
[INFO] Pyramids. Step: 890000. Time Elapsed: 2011.212 s. Mean Reward: -0.325. Std of Reward: 1.105. Training.
[INFO] Pyramids. Step: 900000. Time Elapsed: 2035.333 s. Mean Reward: -0.152. Std of Reward: 1.126. Training.
[INFO] Pyramids. Step: 910000. Time Elapsed: 2057.371 s. Mean Reward: -0.774. Std of Reward: 0.713. Training.
[INFO] Pyramids. Step: 920000. Time Elapsed: 2079.570 s. Mean Reward: -0.107. Std of Reward: 1.191. Training.
[INFO] Pyramids. Step: 930000. Time Elapsed: 2106.338 s. Mean Reward: -0.751. Std of Reward: 0.785. Training.
[INFO] Pyramids. Step: 940000. Time Elapsed: 2133.242 s. Mean Reward: -0.491. Std of Reward: 0.976. Training.
[INFO] Pyramids. Step: 950000. Time Elapsed: 2154.992 s. Mean Reward: 0.402. Std of Reward: 1.155. Training.
[INFO] Pyramids. Step: 960000. Time Elapsed: 2179.480 s. Mean Reward: -0.421. Std of Reward: 1.011. Training.
[INFO] Pyramids. Step: 970000. Time Elapsed: 2205.585 s. Mean Reward: -0.678. Std of Reward: 0.790. Training.
[INFO] Pyramids. Step: 980000. Time Elapsed: 2228.276 s. Mean Reward: 0.160. Std of Reward: 1.278. Training.
[INFO] Pyramids. Step: 990000. Time Elapsed: 2253.409 s. Mean Reward: -0.115. Std of Reward: 1.191. Training.
[INFO] Pyramids. Step: 1000000. Time Elapsed: 2281.976 s. Mean Reward: -0.107. Std of Reward: 1.097. Training.
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results/Pyramids1/Pyramids/Pyramids-999967.onnx
[INFO] Pyramids. Step: 1010000. Time Elapsed: 2304.828 s. Mean Reward: 0.037. Std of Reward: 1.313. Training.
[INFO] Pyramids. Step: 1020000. Time Elapsed: 2327.869 s. Mean Reward: -0.344. Std of Reward: 1.075. Training.
[INFO] Pyramids. Step: 1030000. Time Elapsed: 2353.226 s. Mean Reward: 0.436. Std of Reward: 1.185. Training.
[INFO] Pyramids. Step: 1040000. Time Elapsed: 2376.004 s. Mean Reward: -0.738. Std of Reward: 0.785. Training.
[INFO] Pyramids. Step: 1050000. Time Elapsed: 2398.931 s. Mean Reward: -0.697. Std of Reward: 0.855. Training.
[INFO] Pyramids. Step: 1060000. Time Elapsed: 2422.084 s. Mean Reward: -0.012. Std of Reward: 1.215. Training.
[INFO] Pyramids. Step: 1070000. Time Elapsed: 2447.028 s. Mean Reward: -0.007. Std of Reward: 1.093. Training.
[INFO] Pyramids. Step: 1080000. Time Elapsed: 2469.708 s. Mean Reward: 0.470. Std of Reward: 1.222. Training.
[INFO] Pyramids. Step: 1090000. Time Elapsed: 2496.017 s. Mean Reward: 0.255. Std of Reward: 1.196. Training.
[INFO] Pyramids. Step: 1100000. Time Elapsed: 2522.231 s. Mean Reward: -0.292. Std of Reward: 1.084. Training.
[INFO] Pyramids. Step: 1110000. Time Elapsed: 2546.351 s. Mean Reward: 0.057. Std of Reward: 1.165. Training.
[INFO] Pyramids. Step: 1120000. Time Elapsed: 2572.098 s. Mean Reward: -0.155. Std of Reward: 1.267. Training.
[INFO] Pyramids. Step: 1130000. Time Elapsed: 2600.307 s. Mean Reward: -0.486. Std of Reward: 0.964. Training.
[INFO] Pyramids. Step: 1140000. Time Elapsed: 2628.436 s. Mean Reward: 0.099. Std of Reward: 1.247. Training.
[INFO] Pyramids. Step: 1150000. Time Elapsed: 2652.555 s. Mean Reward: 0.016. Std of Reward: 1.288. Training.
[INFO] Pyramids. Step: 1160000. Time Elapsed: 2677.288 s. Mean Reward: -0.164. Std of Reward: 1.108. Training.
[INFO] Pyramids. Step: 1170000. Time Elapsed: 2702.759 s. Mean Reward: -0.051. Std of Reward: 1.264. Training.
[INFO] Pyramids. Step: 1180000. Time Elapsed: 2727.691 s. Mean Reward: -0.002. Std of Reward: 1.234. Training.
[INFO] Pyramids. Step: 1190000. Time Elapsed: 2751.381 s. Mean Reward: 0.058. Std of Reward: 1.254. Training.
[INFO] Pyramids. Step: 1200000. Time Elapsed: 2775.015 s. Mean Reward: -0.804. Std of Reward: 0.619. Training.
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results/Pyramids1/Pyramids/Pyramids-1199919.onnx
[INFO] Pyramids. Step: 1210000. Time Elapsed: 2799.189 s. Mean Reward: 0.125. Std of Reward: 1.131. Training.
[INFO] Pyramids. Step: 1220000. Time Elapsed: 2820.953 s. Mean Reward: 0.181. Std of Reward: 1.200. Training.
[INFO] Pyramids. Step: 1230000. Time Elapsed: 2846.184 s. Mean Reward: -0.651. Std of Reward: 0.780. Training.
[INFO] Pyramids. Step: 1240000. Time Elapsed: 2870.960 s. Mean Reward: 0.081. Std of Reward: 1.254. Training.
[INFO] Pyramids. Step: 1250000. Time Elapsed: 2894.984 s. Mean Reward: 0.595. Std of Reward: 1.079. Training.
[INFO] Pyramids. Step: 1260000. Time Elapsed: 2919.346 s. Mean Reward: 0.358. Std of Reward: 1.283. Training.
[INFO] Pyramids. Step: 1270000. Time Elapsed: 2943.083 s. Mean Reward: 0.378. Std of Reward: 1.257. Training.
[INFO] Pyramids. Step: 1280000. Time Elapsed: 2967.064 s. Mean Reward: -0.198. Std of Reward: 1.213. Training.
[INFO] Pyramids. Step: 1290000. Time Elapsed: 2991.695 s. Mean Reward: -0.194. Std of Reward: 1.146. Training.
[INFO] Pyramids. Step: 1300000. Time Elapsed: 3012.417 s. Mean Reward: 0.213. Std of Reward: 1.216. Training.
[INFO] Pyramids. Step: 1310000. Time Elapsed: 3037.440 s. Mean Reward: -0.271. Std of Reward: 1.100. Training.
[INFO] Pyramids. Step: 1320000. Time Elapsed: 3060.686 s. Mean Reward: 0.425. Std of Reward: 1.304. Training.
[INFO] Pyramids. Step: 1330000. Time Elapsed: 3085.988 s. Mean Reward: -0.132. Std of Reward: 1.028. Training.
[INFO] Pyramids. Step: 1340000. Time Elapsed: 3108.567 s. Mean Reward: -0.390. Std of Reward: 1.059. Training.
[INFO] Pyramids. Step: 1350000. Time Elapsed: 3134.178 s. Mean Reward: 0.241. Std of Reward: 1.252. Training.
[INFO] Pyramids. Step: 1360000. Time Elapsed: 3157.285 s. Mean Reward: 0.465. Std of Reward: 1.209. Training.
[INFO] Pyramids. Step: 1370000. Time Elapsed: 3178.498 s. Mean Reward: -0.058. Std of Reward: 1.222. Training.
[INFO] Pyramids. Step: 1380000. Time Elapsed: 3201.074 s. Mean Reward: -0.195. Std of Reward: 1.142. Training.
[INFO] Pyramids. Step: 1390000. Time Elapsed: 3226.196 s. Mean Reward: 0.037. Std of Reward: 1.246. Training.
[INFO] Pyramids. Step: 1400000. Time Elapsed: 3249.969 s. Mean Reward: -0.088. Std of Reward: 1.238. Training.
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results/Pyramids1/Pyramids/Pyramids-1399964.onnx
[INFO] Pyramids. Step: 1410000. Time Elapsed: 3272.763 s. Mean Reward: 0.017. Std of Reward: 1.213. Training.
[INFO] Pyramids. Step: 1420000. Time Elapsed: 3296.189 s. Mean Reward: -0.210. Std of Reward: 1.124. Training.
[INFO] Pyramids. Step: 1430000. Time Elapsed: 3321.810 s. Mean Reward: 0.540. Std of Reward: 1.164. Training.
[INFO] Pyramids. Step: 1440000. Time Elapsed: 3348.786 s. Mean Reward: 0.216. Std of Reward: 1.234. Training.
[INFO] Pyramids. Step: 1450000. Time Elapsed: 3373.464 s. Mean Reward: 0.120. Std of Reward: 1.276. Training.
[INFO] Pyramids. Step: 1460000. Time Elapsed: 3397.313 s. Mean Reward: 0.166. Std of Reward: 1.262. Training.
[INFO] Pyramids. Step: 1470000. Time Elapsed: 3420.263 s. Mean Reward: 0.331. Std of Reward: 1.139. Training.
[INFO] Pyramids. Step: 1480000. Time Elapsed: 3442.837 s. Mean Reward: 0.397. Std of Reward: 1.190. Training.
[INFO] Pyramids. Step: 1490000. Time Elapsed: 3467.505 s. Mean Reward: 0.249. Std of Reward: 1.263. Training.
[INFO] Pyramids. Step: 1500000. Time Elapsed: 3490.427 s. Mean Reward: 0.826. Std of Reward: 1.129. Training.
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
[INFO] Exported results/Pyramids1/Pyramids/Pyramids-1500031.onnx
[INFO] Copied results/Pyramids1/Pyramids/Pyramids-1500031.onnx to results/Pyramids1/Pyramids.onnx.
Push agents to the 🤗 Hub¶
We use Kaggle Secrets for the HuggingFace token.
In [34]:
from huggingface_hub import notebook_login
# notebook_login()
In [35]:
!git config --global credential.helper store
In [36]:
import os
from kaggle_secrets import UserSecretsClient
from huggingface_hub import HfApi, login
user_secrets = UserSecretsClient()
hf_token = user_secrets.get_secret("HF_TOKEN")
login(token=hf_token, add_to_git_credential=True)
api = HfApi()
user_info = api.whoami()
print(f"✅ Logged in as: {user_info['name']}")
print(f"✅ Token has Write access: {user_info['auth']['type'] == 'WRITE'}")
✅ Logged in as: Chiz ✅ Token has Write access: False
Push SnowballTarget¶
Edit --repo-id to use your HuggingFace username
In [38]:
!/kaggle/working/mlagents-env/bin/mlagents-push-to-hf \
--run-id="SnowballTarget1" \
--local-dir="./results/SnowballTarget1" \
--repo-id="Chiz/ppo-SnowballTarget" \
--commit-message="Unit 5 SnowballTarget"
[INFO] This function will create a model card and upload your SnowballTarget1 into HuggingFace Hub. This is a work in progress: If you encounter a bug, please send open an issue [INFO] Pushing repo SnowballTarget1 to the Hugging Face Hub Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B ...owballTarget-1000696.onnx: 2%|▎ | 13.6kB / 651kB ...SnowballTarget-1000696.pt: 2%|▎ | 80.7kB / 3.85MB ...nowballTarget-399968.onnx: 2%|▎ | 13.6kB / 651kB .../SnowballTarget-399968.pt: 2%|▎ | 80.7kB / 3.85MB ...nowballTarget-599976.onnx: 2%|▎ | 13.6kB / 651kB .../SnowballTarget-599976.pt: 2%|▎ | 80.7kB / 3.85MB ...nowballTarget-799984.onnx: 2%|▎ | 13.6kB / 651kB .../SnowballTarget-799984.pt: 2%|▎ | 80.7kB / 3.85MB ...nowballTarget-999992.onnx: 2%|▎ | 13.6kB / 651kB .../SnowballTarget-999992.pt: 2%|▎ | 80.7kB / 3.85MB ...owballTarget-1000696.onnx: 2%|▎ | 13.6kB / 651kB ...SnowballTarget-1000696.pt: 2%|▎ | 80.7kB / 3.85MB ...nowballTarget-399968.onnx: 2%|▎ | 13.6kB / 651kB .../SnowballTarget-399968.pt: 2%|▎ | 80.7kB / 3.85MB ...nowballTarget-599976.onnx: 2%|▎ | 13.6kB / 651kB .../SnowballTarget-599976.pt: 2%|▎ | 80.7kB / 3.85MB ...nowballTarget-799984.onnx: 2%|▎ | 13.6kB / 651kB .../SnowballTarget-799984.pt: 2%|▎ | 80.7kB / 3.85MB ...nowballTarget-999992.onnx: 2%|▎ | 13.6kB / 651kB Processing Files (0 / 16) : 2%|▎ | 568kB / 27.1MB, 2.84MB/s New Data Upload : 2%|▎ | 568kB / 27.1MB, 2.84MB/s ...owballTarget-1000696.onnx: 40%|█████▌ | 259kB / 651kB ...SnowballTarget-1000696.pt: 40%|█████▌ | 1.53MB / 3.85MB ...nowballTarget-399968.onnx: 40%|█████▌ | 259kB / 651kB .../SnowballTarget-399968.pt: 40%|█████▌ | 1.53MB / 3.85MB ...nowballTarget-599976.onnx: 40%|█████▌ | 259kB / 651kB .../SnowballTarget-599976.pt: 40%|█████▌ | 1.53MB / 3.85MB ...nowballTarget-799984.onnx: 40%|█████▌ | 259kB / 651kB .../SnowballTarget-799984.pt: 40%|█████▌ | 1.53MB / 3.85MB ...nowballTarget-999992.onnx: 40%|█████▌ | 259kB / 651kB Processing Files (0 / 16) : 40%|█████▌ | 10.8MB / 27.1MB, 27.0MB/s New Data Upload : 40%|█████▌ | 10.8MB / 27.1MB, 27.0MB/s ...owballTarget-1000696.onnx: 99%|█████████████▊| 641kB / 651kB ...SnowballTarget-1000696.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-399968.onnx: 99%|█████████████▊| 641kB / 651kB .../SnowballTarget-399968.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-599976.onnx: 99%|█████████████▊| 641kB / 651kB .../SnowballTarget-599976.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-799984.onnx: 99%|█████████████▊| 641kB / 651kB .../SnowballTarget-799984.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-999992.onnx: 99%|█████████████▊| 641kB / 651kB Processing Files (0 / 16) : 99%|█████████████▊| 26.7MB / 27.1MB, 44.5MB/s New Data Upload : 99%|█████████████▊| 26.7MB / 27.1MB, 44.5MB/s ...owballTarget-1000696.onnx: 99%|█████████████▊| 641kB / 651kB ...SnowballTarget-1000696.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-399968.onnx: 99%|█████████████▊| 641kB / 651kB .../SnowballTarget-399968.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-599976.onnx: 99%|█████████████▊| 641kB / 651kB .../SnowballTarget-599976.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-799984.onnx: 99%|█████████████▊| 641kB / 651kB .../SnowballTarget-799984.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-999992.onnx: 99%|█████████████▊| 641kB / 651kB [+ 6 files] : 99%|█████████████▊| 8.33MB / 8.45MB ...owballTarget-1000696.onnx: 99%|█████████████▊| 641kB / 651kB ...SnowballTarget-1000696.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-399968.onnx: 99%|█████████████▊| 641kB / 651kB .../SnowballTarget-399968.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-599976.onnx: 99%|█████████████▊| 641kB / 651kB .../SnowballTarget-599976.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-799984.onnx: 99%|█████████████▊| 641kB / 651kB .../SnowballTarget-799984.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-999992.onnx: 99%|█████████████▊| 641kB / 651kB [+ 6 files] : 99%|█████████████▊| 8.33MB / 8.45MB ...owballTarget-1000696.onnx: 99%|█████████████▊| 641kB / 651kB ...SnowballTarget-1000696.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-399968.onnx: 99%|█████████████▊| 641kB / 651kB .../SnowballTarget-399968.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-599976.onnx: 99%|█████████████▊| 641kB / 651kB .../SnowballTarget-599976.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-799984.onnx: 99%|█████████████▊| 641kB / 651kB .../SnowballTarget-799984.pt: 99%|█████████████▊| 3.79MB / 3.85MB ...nowballTarget-999992.onnx: 99%|█████████████▊| 641kB / 651kB [+ 6 files] : 99%|█████████████▊| 8.33MB / 8.45MB ...owballTarget-1000696.onnx: 100%|██████████████| 651kB / 651kB ...SnowballTarget-1000696.pt: 100%|██████████████| 3.85MB / 3.85MB ...nowballTarget-399968.onnx: 100%|██████████████| 651kB / 651kB .../SnowballTarget-399968.pt: 100%|██████████████| 3.85MB / 3.85MB ...nowballTarget-599976.onnx: 100%|██████████████| 651kB / 651kB .../SnowballTarget-599976.pt: 100%|██████████████| 3.85MB / 3.85MB ...nowballTarget-799984.onnx: 100%|██████████████| 651kB / 651kB .../SnowballTarget-799984.pt: 100%|██████████████| 3.85MB / 3.85MB ...nowballTarget-999992.onnx: 100%|██████████████| 651kB / 651kB Processing Files (16 / 16) : 100%|██████████████| 27.1MB / 27.1MB, 19.4MB/s New Data Upload : 100%|██████████████| 27.1MB / 27.1MB, 19.4MB/s ...nowballTarget-799984.onnx: 100%|██████████████| 651kB / 651kB .../SnowballTarget-799984.pt: 100%|██████████████| 3.85MB / 3.85MB ...nowballTarget-999992.onnx: 100%|██████████████| 651kB / 651kB .../SnowballTarget-999992.pt: 100%|██████████████| 3.85MB / 3.85MB ...wballTarget/checkpoint.pt: 100%|██████████████| 3.85MB / 3.85MB ...64269.99d11e677bcf.1923.0: 100%|██████████████| 33.4kB / 33.4kB ...64824.99d11e677bcf.2014.0: 100%|██████████████| 1.17kB / 1.17kB ...65162.99d11e677bcf.2086.0: 100%|██████████████| 1.17kB / 1.17kB ...65496.99d11e677bcf.2184.0: 100%|██████████████| 66.7kB / 66.7kB ...rget1/SnowballTarget.onnx: 100%|██████████████| 651kB / 651kB ...nowballTarget-799984.onnx: 100%|██████████████| 651kB / 651kB .../SnowballTarget-799984.pt: 100%|██████████████| 3.85MB / 3.85MB ...nowballTarget-999992.onnx: 100%|██████████████| 651kB / 651kB .../SnowballTarget-999992.pt: 100%|██████████████| 3.85MB / 3.85MB ...wballTarget/checkpoint.pt: 100%|██████████████| 3.85MB / 3.85MB ...64269.99d11e677bcf.1923.0: 100%|██████████████| 33.4kB / 33.4kB ...64824.99d11e677bcf.2014.0: 100%|██████████████| 1.17kB / 1.17kB ...65162.99d11e677bcf.2086.0: 100%|██████████████| 1.17kB / 1.17kB ...65496.99d11e677bcf.2184.0: 100%|██████████████| 66.7kB / 66.7kB Processing Files (16 / 16) : 100%|██████████████| 27.1MB / 27.1MB, 16.9MB/s New Data Upload : 100%|██████████████| 27.1MB / 27.1MB, 16.9MB/s ...nowballTarget-799984.onnx: 100%|██████████████| 651kB / 651kB .../SnowballTarget-799984.pt: 100%|██████████████| 3.85MB / 3.85MB ...nowballTarget-999992.onnx: 100%|██████████████| 651kB / 651kB .../SnowballTarget-999992.pt: 100%|██████████████| 3.85MB / 3.85MB ...wballTarget/checkpoint.pt: 100%|██████████████| 3.85MB / 3.85MB ...64269.99d11e677bcf.1923.0: 100%|██████████████| 33.4kB / 33.4kB ...64824.99d11e677bcf.2014.0: 100%|██████████████| 1.17kB / 1.17kB ...65162.99d11e677bcf.2086.0: 100%|██████████████| 1.17kB / 1.17kB ...65496.99d11e677bcf.2184.0: 100%|██████████████| 66.7kB / 66.7kB ...rget1/SnowballTarget.onnx: 100%|██████████████| 651kB / 651kB [INFO] Your model is pushed to the hub. You can view your model here: https://huggingface.co/Chiz/ppo-SnowballTarget
Push Pyramids¶
Edit --repo-id to use your HuggingFace username
In [47]:
!/kaggle/working/mlagents-env/bin/mlagents-push-to-hf \
--run-id="Pyramids1" \
--local-dir="./results/Pyramids1" \
--repo-id="Chiz/ppo-Pyramids" \
--commit-message="Unit 5 Pyramids with curiosity"
[INFO] This function will create a model card and upload your Pyramids1 into HuggingFace Hub. This is a work in progress: If you encounter a bug, please send open an issue [INFO] Pushing repo Pyramids1 to the Hugging Face Hub Processing Files (0 / 0) : | | 0.00B / 0.00B New Data Upload : | | 0.00B / 0.00B ...ids/Pyramids-1199919.onnx: 2%|▎ | 8.73kB / 451kB ...amids/Pyramids-1199919.pt: 2%|▎ | 81.2kB / 4.19MB ...ids/Pyramids-1399964.onnx: 2%|▎ | 8.73kB / 451kB ...amids/Pyramids-1399964.pt: 2%|▎ | 81.2kB / 4.19MB ...ids/Pyramids-1500031.onnx: 2%|▎ | 8.73kB / 451kB ...amids/Pyramids-1500031.pt: 2%|▎ | 81.2kB / 4.19MB ...mids/Pyramids-799915.onnx: 2%|▎ | 8.73kB / 451kB ...ramids/Pyramids-799915.pt: 2%|▎ | 81.2kB / 4.19MB ...mids/Pyramids-999967.onnx: 2%|▎ | 8.73kB / 451kB ...ramids/Pyramids-999967.pt: 2%|▎ | 81.2kB / 4.19MB ...ids/Pyramids-1199919.onnx: 2%|▎ | 8.73kB / 451kB ...amids/Pyramids-1199919.pt: 2%|▎ | 81.2kB / 4.19MB ...ids/Pyramids-1399964.onnx: 2%|▎ | 8.73kB / 451kB ...amids/Pyramids-1399964.pt: 2%|▎ | 81.2kB / 4.19MB ...ids/Pyramids-1500031.onnx: 2%|▎ | 8.73kB / 451kB ...amids/Pyramids-1500031.pt: 2%|▎ | 81.2kB / 4.19MB ...mids/Pyramids-799915.onnx: 2%|▎ | 8.73kB / 451kB ...ramids/Pyramids-799915.pt: 2%|▎ | 81.2kB / 4.19MB ...mids/Pyramids-999967.onnx: 2%|▎ | 8.73kB / 451kB Processing Files (0 / 13) : 2%|▎ | 563kB / 29.1MB, 2.88MB/s New Data Upload : 2%|▎ | 563kB / 29.1MB, 2.88MB/s ...ids/Pyramids-1199919.onnx: 35%|████▉ | 157kB / 451kB ...amids/Pyramids-1199919.pt: 35%|████▉ | 1.46MB / 4.19MB ...ids/Pyramids-1399964.onnx: 35%|████▉ | 157kB / 451kB ...amids/Pyramids-1399964.pt: 35%|████▉ | 1.46MB / 4.19MB ...ids/Pyramids-1500031.onnx: 35%|████▉ | 157kB / 451kB ...amids/Pyramids-1500031.pt: 35%|████▉ | 1.46MB / 4.19MB ...mids/Pyramids-799915.onnx: 35%|████▉ | 157kB / 451kB ...ramids/Pyramids-799915.pt: 35%|████▉ | 1.46MB / 4.19MB ...mids/Pyramids-999967.onnx: 35%|████▉ | 157kB / 451kB Processing Files (0 / 13) : 35%|████▉ | 10.1MB / 29.1MB, 25.6MB/s New Data Upload : 35%|████▉ | 10.1MB / 29.1MB, 25.6MB/s ...ids/Pyramids-1199919.onnx: 99%|█████████████▊| 445kB / 451kB ...amids/Pyramids-1199919.pt: 99%|█████████████▊| 4.14MB / 4.19MB ...ids/Pyramids-1399964.onnx: 99%|█████████████▊| 445kB / 451kB ...amids/Pyramids-1399964.pt: 99%|█████████████▊| 4.14MB / 4.19MB ...ids/Pyramids-1500031.onnx: 99%|█████████████▊| 445kB / 451kB ...amids/Pyramids-1500031.pt: 99%|█████████████▊| 4.14MB / 4.19MB ...mids/Pyramids-799915.onnx: 99%|█████████████▊| 445kB / 451kB ...ramids/Pyramids-799915.pt: 99%|█████████████▊| 4.14MB / 4.19MB ...mids/Pyramids-999967.onnx: 99%|█████████████▊| 445kB / 451kB Processing Files (0 / 13) : 99%|█████████████▊| 28.7MB / 29.1MB, 48.2MB/s New Data Upload : 99%|█████████████▊| 28.7MB / 29.1MB, 48.2MB/s ...ids/Pyramids-1199919.onnx: 99%|█████████████▊| 445kB / 451kB ...amids/Pyramids-1199919.pt: 99%|█████████████▊| 4.14MB / 4.19MB ...ids/Pyramids-1399964.onnx: 99%|█████████████▊| 445kB / 451kB ...amids/Pyramids-1399964.pt: 99%|█████████████▊| 4.14MB / 4.19MB ...ids/Pyramids-1500031.onnx: 99%|█████████████▊| 445kB / 451kB ...amids/Pyramids-1500031.pt: 99%|█████████████▊| 4.14MB / 4.19MB ...mids/Pyramids-799915.onnx: 99%|█████████████▊| 445kB / 451kB ...ramids/Pyramids-799915.pt: 99%|█████████████▊| 4.14MB / 4.19MB ...mids/Pyramids-999967.onnx: 99%|█████████████▊| 445kB / 451kB [+ 3 files] : 99%|█████████████▊| 9.94MB / 10.1MB ...ids/Pyramids-1199919.onnx: 99%|█████████████▊| 445kB / 451kB ...amids/Pyramids-1199919.pt: 99%|█████████████▊| 4.14MB / 4.19MB ...ids/Pyramids-1399964.onnx: 99%|█████████████▊| 445kB / 451kB ...amids/Pyramids-1399964.pt: 99%|█████████████▊| 4.14MB / 4.19MB ...ids/Pyramids-1500031.onnx: 99%|█████████████▊| 445kB / 451kB ...amids/Pyramids-1500031.pt: 99%|█████████████▊| 4.14MB / 4.19MB ...mids/Pyramids-799915.onnx: 99%|█████████████▊| 445kB / 451kB ...ramids/Pyramids-799915.pt: 99%|█████████████▊| 4.14MB / 4.19MB ...mids/Pyramids-999967.onnx: 99%|█████████████▊| 445kB / 451kB [+ 3 files] : 99%|█████████████▊| 9.94MB / 10.1MB ...ids/Pyramids-1199919.onnx: 100%|██████████████| 451kB / 451kB ...amids/Pyramids-1199919.pt: 100%|██████████████| 4.19MB / 4.19MB ...ids/Pyramids-1399964.onnx: 100%|██████████████| 451kB / 451kB ...amids/Pyramids-1399964.pt: 100%|██████████████| 4.19MB / 4.19MB ...ids/Pyramids-1500031.onnx: 100%|██████████████| 451kB / 451kB ...amids/Pyramids-1500031.pt: 100%|██████████████| 4.19MB / 4.19MB ...mids/Pyramids-799915.onnx: 100%|██████████████| 451kB / 451kB ...ramids/Pyramids-799915.pt: 100%|██████████████| 4.19MB / 4.19MB ...mids/Pyramids-999967.onnx: 100%|██████████████| 451kB / 451kB Processing Files (13 / 13) : 100%|██████████████| 29.1MB / 29.1MB, 24.3MB/s New Data Upload : 100%|██████████████| 29.1MB / 29.1MB, 24.3MB/s ...amids/Pyramids-1399964.pt: 100%|██████████████| 4.19MB / 4.19MB ...ids/Pyramids-1500031.onnx: 100%|██████████████| 451kB / 451kB ...amids/Pyramids-1500031.pt: 100%|██████████████| 4.19MB / 4.19MB ...mids/Pyramids-799915.onnx: 100%|██████████████| 451kB / 451kB ...ramids/Pyramids-799915.pt: 100%|██████████████| 4.19MB / 4.19MB ...mids/Pyramids-999967.onnx: 100%|██████████████| 451kB / 451kB ...ramids/Pyramids-999967.pt: 100%|██████████████| 4.19MB / 4.19MB ...s1/Pyramids/checkpoint.pt: 100%|██████████████| 4.19MB / 4.19MB ...68356.99d11e677bcf.2389.0: 100%|██████████████| 1.22MB / 1.22MB ...s/Pyramids1/Pyramids.onnx: 100%|██████████████| 451kB / 451kB ...amids/Pyramids-1399964.pt: 100%|██████████████| 4.19MB / 4.19MB ...ids/Pyramids-1500031.onnx: 100%|██████████████| 451kB / 451kB ...amids/Pyramids-1500031.pt: 100%|██████████████| 4.19MB / 4.19MB ...mids/Pyramids-799915.onnx: 100%|██████████████| 451kB / 451kB ...ramids/Pyramids-799915.pt: 100%|██████████████| 4.19MB / 4.19MB ...mids/Pyramids-999967.onnx: 100%|██████████████| 451kB / 451kB ...ramids/Pyramids-999967.pt: 100%|██████████████| 4.19MB / 4.19MB ...s1/Pyramids/checkpoint.pt: 100%|██████████████| 4.19MB / 4.19MB ...68356.99d11e677bcf.2389.0: 100%|██████████████| 1.22MB / 1.22MB Processing Files (13 / 13) : 100%|██████████████| 29.1MB / 29.1MB, 20.8MB/s New Data Upload : 100%|██████████████| 29.1MB / 29.1MB, 20.8MB/s ...amids/Pyramids-1399964.pt: 100%|██████████████| 4.19MB / 4.19MB ...ids/Pyramids-1500031.onnx: 100%|██████████████| 451kB / 451kB ...amids/Pyramids-1500031.pt: 100%|██████████████| 4.19MB / 4.19MB ...mids/Pyramids-799915.onnx: 100%|██████████████| 451kB / 451kB ...ramids/Pyramids-799915.pt: 100%|██████████████| 4.19MB / 4.19MB ...mids/Pyramids-999967.onnx: 100%|██████████████| 451kB / 451kB ...ramids/Pyramids-999967.pt: 100%|██████████████| 4.19MB / 4.19MB ...s1/Pyramids/checkpoint.pt: 100%|██████████████| 4.19MB / 4.19MB ...68356.99d11e677bcf.2389.0: 100%|██████████████| 1.22MB / 1.22MB ...s/Pyramids1/Pyramids.onnx: 100%|██████████████| 451kB / 451kB [INFO] Your model is pushed to the hub. You can view your model here: https://huggingface.co/Chiz/ppo-Pyramids
Congrats on finishing Unit 5! 🎉¶
You can now play with your agents in the browser:
- SnowballTarget: https://huggingface.co/spaces/ThomasSimonini/SnowballTarget
- Pyramids: https://huggingface.co/spaces/unity/ML-Agents-Pyramids
Just select your model from the dropdown!