1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

Introduction to ML Agents

Last updated at Posted at 2024-12-24

Introduction

I am a member of Arcanain, a volunteer team working on autonomous driving robots in Japan. Recently, we decided to explore reinforcement learning for motion planning. To do this, we chose Unity ML-Agents as our simulation and training environment. Eventually, we hope to achieve something similar to what is described in this article, but in Unity.

In this post, we’ll focus on how to set up the environment for Unity ML-Agents. We’ll continue to build on this foundation and share updates as we progress toward our goal of autonomous driving simulation in Unity.

Prior Information

Environment

For this tutorial, we'll use either Ubuntu 22.04 or macOS 15.2 with the following key components on the table:

  • Table
name version
Unity Editor 2022.3.55f1
Python 3.10.8
ML-Agents ML-Agents Release 21
python packages

The result of the command $ pip list is bellow:

Packages version
mlagents 0.30.0
mlagents-envs 0.30.0
gym 0.26.2
gym-notices 0.0.8
numpy 1.21.2
protobuf 5.29.2
torch 1.11.0
tensorboard 2.18.0
tensorboard-data-server 0.7.2
PyYAML 6.0.2
attrs 24.3.0
cattrs 1.5.0
grpcio 1.69.0

Note
It’s important that the versions of mlagents and mlagents-envs match the requirements for the specific ML-Agents Unity Toolkit release you’re using. In this article, we’ll install Release 21, so we’ll use mlagents and mlagents-envs at version 0.30.0. For more deteils see this release note.

1. Install Unity Hub

Reference1.
Add the public signing key, run the following command:

$ wget -qO - https://hub.unity3d.com/linux/keys/public | gpg --dearmor | sudo tee /usr/share/keyrings/Unity_Technologies_ApS.gpg > /dev/null

To add the Unity Hub repository, you need an entry in /etc/apt/sources.list.d. Run the following command to add the Unity Hub repository:

$ sudo sh -c 'echo "deb [signed-by=/usr/share/keyrings/Unity_Technologies_ApS.gpg] https://hub.unity3d.com/linux/repos/deb stable main" > /etc/apt/sources.list.d/unityhub.list'

Update the package cache and install the package:

Mac user can install it on this page: Start Your Creative Projects and Download the Unity Hub | Unity

$ sudo apt update
$ sudo apt-get install unityhub

Then, intsallation is complete.
Start Unity Hub by terminal command

$ unityhub

If the installation finished without any problems, you should see a window like this:
image.png

Create Account

If you don't have an Unity account, you have to create new one.

image.png

2. Install Unity Editor

After cretating an account or loggin, you can install the Unity Editor.
The minimum supported Unity version was updated to 2022.3. 2, make sure to select 2022.3 or newer when installing.
image.png

Licence (Optional)

Recently, you don't have to add a license for personal use. A Free Personal License is added by default.

image.png

image.png

image.png

image.png

2. Install ML-Agents

2.1 Install ML-Agents Tool Kit

First, clone the ML-Agents Toolkit from GitHub by running the following command:

$ git clone https://github.com/Unity-Technologies/ml-agents.git

After the cloning finished, checkout the branch to release 21.

$ cd ml-agents
$ git checkout release_21

2.2 Install the mlagents for python

The mlagents Python package provides the core training scripts and interfaces for communicating with the Unity environment.

Install it by running:

$ pip3 install mlagents

When the installation finishes, verify it with the following command:

$ mlagents-learn --help

If you see an output similar to:

usage: mlagents-learn [-h] [--env ENV_PATH] [--resume] [--deterministic]
                      [--force] [--run-id RUN_ID] [--initialize-from RUN_ID]
                      [--seed SEED] [--inference] [--base-port BASE_PORT]
                      [--num-envs NUM_ENVS] [--num-areas NUM_AREAS] [--debug]
                      [--env-args ...]
                      [--max-lifetime-restarts MAX_LIFETIME_RESTARTS]
                      [--restarts-rate-limit-n RESTARTS_RATE_LIMIT_N]
                      [--restarts-rate-limit-period-s RESTARTS_RATE_LIMIT_PERIOD_S]
                .
                .
                .
                .
                

then you have successfully installed the mlagents Python package!

3. Tutorial

Start unity hub with the following command:

$ unityhub

First, you have to create a new Project.
Push [Add] -> Add project from disk.

image.png

In this tutorial, we use cloened repository ml-agents.
Choose the directory which you have cloned before

  • .../ml-agents/Project
    • ml-agents is the cloned directory

image.png

If you can see this window it is working.
image.png

demo

You can see Assets on the bottom of the previous window.

click "「ML-Agents」→「Examples」→「3DBall」→「Scenes」→「3DBall」"
image.png

If you can see these objects, loading has successed.
image.png

run this command:

$ cd /path/to/Unity/ml-agents 
$ mlagents-learn config/ppo/3DBall.yaml --run-id=first3DBallRun

image.png

then, push "start botton" of Unity Editor.

image.png

Reinforcement learning starts!!
After a while the learning will finish and you can see the reslult of learned model.

Acknowledgements

This project is supported by GxP (Growth xPartners) Inc.. They have provided us with valuable advice and essential equipment for our robots. We extend our deepest gratitude for their support.

  1. Install the Unity Hub | Unity Hub | 3.0

  2. https://github.com/Unity-Technologies/ml-agents/releases/tag/release_21

1
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?