Controller

Controller Class

The Controller class is a standalone module that manages the system and environment. It executes high-level tasks such as initialization, instantiation, and synchronization. Similar to the Agent class, custom controllers are developed with FABMLA’s template. This allows FABMLA to handle the underlying sub-processes, while the controller handles behaviors fitted to the simulation.

For example, in a spatial sciences simulation, the controller can declare certain qualities of the solar system, such as the amount of planets, initial distance of planets, gravity strength, and environmental influences and FABMLA manages agent registration, agent-stepping, reward distribution, and episode management.

Usage

When creating a custom, simulation-specific controller, you should inherit this class and design it to fit the needs of a simulation, as below:

public class SpecificController : ABMController {}

Properties

The Agent class includes properties that are required to be defined. These include simulationAgent and agentAmount. Both properties must be defined through Unity3D's GUI. The simulationAgent property is a reference to the Prefab of an agent — it must contain the Agent class. The agentAmount is the number of agents the controller summons.

Agents Reference

The Agent class also includes a reference to all agents in a scene. To access this property use the list agents. For example:

agents.ToList().ForEach(a => {
    a.AddReward(1.0f); // Add 1 reward to *each* agent
});

Methods

The Controller class contains methods intended to be overridden to develop custom controllers.

SetupEpisode()

The SetupEpisode() method is called at the start of each episode. It is intended to manage all items within the simulation except the Agent. To initialize an agent at the start of an episode, see the OnEpisodeBegin() in the Agent class. If using a Logger, it is important to keep base.SetupEpisode(); as the parent function handles connections with the Logger class. The convention to override this class is as follows:

protected override void SetupEpisode() {
    base.SetupEpisode(); // Using logger, must keep this line
    target.transform.localPosition = RandomLocation() ; // Set target location randomly
}

CalculateReward(ABMAgent agent)

The CalculateReward() method is called at the end of each Step and is intended to calculate the reward of each agent. It is important to note that you do not need to use this method, you can calculate rewards in other conventions such as in our 4CoopPuzzle example. The convention to override this method is as follows:

protected override void CalculateReward(ABMAgent agent) {
    agent.AddReward(1f / (Distance(agent, target.transform.localPosition)+0.1f)); // Reward agents based on distance to target
}

EndCase()

The EndCase() method is, similar to the CalculateReward() is called at the end of each Step. This method is responsible for declaring cases in which the episode should end. This method, for example, can be used to end the episode if max steps are reached or an agent goes out of bounds. To end the episode, call the EndAll() method. The convention to use this method is as follows:

protected override void EndCase() {
    bool isEnd = false;
    agents.ToList().ForEach(a => { // For each agent
        if(isEnd) return;
 
        if(a.MaxStep > 0 && a.StepCount >= a.MaxStep) { isEnd = true; EndAll(); } // Max stepcount reached
        if(Distance(a, new Vector3(0, 0, 0)) > boundsRadius) { isEnd = true; EndAll(); } // Out of bounds
    });
}

Log()

The Log() method is Logger specific; if you are not using a Logger, you do not need to override this method. This method returns a String intended to be logged to the console and/or a file. This method is called at the end of each episode. The convention to override the file is as follows:

protected override String Log() {
    return $"{step},{reachedTarget}"; // Log step count and if the target was reached in CSV format
}

Advanced Methods

The methods below are advanced methods, while a custom controller has the ability to override these methods, it is important that altering these methods change the core functionality of FABMLA. Be careful when overriding these methods.

Start()

This method summons agents to the scene. It is called once the scene is set up. Removing base.Start(); removes the agent summoning. The following example is from our 3PongSelfPlay2D example:

protected override void Start() {
    // Removed base.Start();
    // Must create own methods to summon agents
    Spawn(Team.Blue, out blue);
    Spawn(Team.Purple, out purple);

    blue.speed = agentSpeed; purple.speed = agentSpeed; // Set agent speeds

    ball.speed = ballSpeed; // Set ball speed
    ball.controller = this; // Reference controller
    
    SetupEpisode(); // Setup Episode
}

Useful Methods

The Controller class contains a couple of method useful to reference in (or outside of) your custom controller class.

EndAll()

protected void EndAll()

Ends the simulation for all agents within a simulation.

ClearAgents()

protected void ClearAgents()

Clears all agents from a simulation. It deregisters them from the agents list and removes the gameobject from the scene.

ClearAgent(ABMAgent agent)

public void ClearAgent(ABMAgent a)

Clears a specific agent from a simulation. It deregisters them from the agents list and removes the gameobject from the scene.

CheckToEndEpisode()

public void CheckToEndEpisode()

Checks to end the current episode by calling the EndAll()

SetTeamId(ABMAgent a, int id)

public void SetTeamId(ABMAgent a, int id)

Sets the teamId of a specific agent to id.

Source code

To dive deeper into FABMLA, and understand the innerworkings of the Controller class, see the source code.