ReinforcementLearning
Multi-Agent-System

[Review] An Introduction to Multi-Agent System

Introduction

In my university, there is a lot of interesting modules to take. However, I do not have enough time to attend all of them. So in order to enhance my motivation further and equip myself more, I would like to review the course textbook in advance, and summarise it in this article.

CH1. Introduction

This section is about the motivation to study MA(Multi Agent system). And the author clarified the difference between MA and other algorithmic domain, for instance game theory and economics. Actually he gave us three important keywords to understand the way for MA to describe the human society. The keywords are coordination, cooperation and negotiation. Then he gave two examples to elaborate the definition of each keyword. One is about NASA's space probe controlling system. The other is aircraft monitoring system. quoting these examples, he meant the significance of the importance existing for agents to converse and decide what to do to fulfill the task.

CH2. Intelligent Agents

In this section, the aim is to give us an understanding of what agents are and some of the issues in building them. Actually the agents have to interact with the environment to learn to finish the given task.
This basic concept leads us to the necessity of understanding of environment as well.
Environments are composed of various types, for instance continuous or discrete time steps and deterministic or non-deterministic as below.

  • Accessible or Inaccessible
  • Deterministic or Non-deterministic
  • Static or Dynamic
  • Discrete or Continuous

The importance of the environmental properties is considered when it comes to actual implementation of the agents because it defines the design process of the agents. With regard to the role of the agents, firstly we should consider the question like "What is intelligent?". This is not an easy one though, if we break down the properties of agents, it will become much clearer. Let's list the kinds of capabilities we expect from an agent.
The agent should be able to:

  • Reactivity: perceive their environment and respond it in a timely fashion
  • Proactiveness: exhibit goal-oriented behaviour to satisfy their designed objectives
  • Social Ability: interact with other agents

By now, we are familiar with the concept and the objectives of agents and environments.
But what we have missed more? Please consider the situation that you are asked to create an agent which can finish some specific task and you might be asked the same work again. So what shall you do? You might want some blueprint, don't you?
In fact, for programmer, this concept is already very clear, it's object.
What the author tries to say here is quite simple, generally agents don't arise in nothing, someone has to program it. Long story short, it is written that the agents are basically programmable. However, we, human being have to program it whatever we want. As you assume, where to start? I would like to touch upon the important points for this. So as you might know, objects are conceptual entities that encapsulate some states. And we can expect the agent is also an object composed of many values and methods inside. Using the metaphor of thermostats the author indicates that an agent is a software. Hence, we have to intricately specify tasks and its approaches in advance. In other word, Designing plays an important part in a multi-agent system. By the way, the way to describe the environment, agents and its actions or probabilistic transitions of the state are analogous to the way of Reinforcement Learning, so we can just refer to it.
One thing we have to ensure is the time when this book is written there was not computationally powerful machine. Lastly, through the running example of Gridworld, we can lean the utility function telling agents what/how to do.