agent naming environment

In this project we create an agent able to distinguish and name object on the environment

Breakout

Similar to keras article we create a model to parse the input image from atari games and decide how to interact with the environment: i.e. how to move.

breakout using openai gym to train an agent to play breakout

To understand how the agent is learning the enviroment we create a visualization to focus on the main step of learning.

attention_init visualization to monitor the development of the learning process

The visualization consists in:

Based on the reward (bricks crushed by the ball) the agent learns how to behave in such environment

implementation

Compared to the original implementation of the game we performed few differences:

For more references

attention_init velocity and collision as input parameters

The model outputs different classes defined by measurable interaction with the environment, mainly: myself, tool, walls, info, dangers…

attention

The attention map tells a lot about the key challanges of this exercise

Compared to other games breakout has a lot of interesting challanges in the understanding of causalit y.

attention_calss attivation classes

We do the same for the different types of moves

causality

In this game you get the reward the moment the ball hits the bricks but you control only the paddle. In the eyes of the agent I first see a clear correlation:

attention_init evolution of attention over time, the focus on digits drops over time, the agent is then learning to avoid losing life focusing on the bottom.

We insert additional scores as output which will depend directly from an action and study the activation for that specific action: