Autonomous Patterns of Sensorimotor Activity

Overview

Much of our behaviour is habitual. We run or cook or think in a particular way not because it is necessarily the most optimal way to do so, but because that is the way that we have done these things in the past. For a long period of time, habits were seen as the most fundamental concept in the study of the mind. But more recently, enthusiasm for computationalist views of the mind has relegated habits to a relatively unimportant role of "offloading computational load." The idea here is that once you have figured out how to do something, why spend any of "your brain's computational power" figuring it out again?

But what is lost in this simplification of habits?
In this research project, we bring modern tools to study the older, richer concept of habit. Instead of seeing habits as atemporal learned correlations between stimulus and response, we investigate habits as patterns of behaviour that reinforce the mechanism(s) that produce those very patterns of behaviour.

Sensorimotor space and the IDSM

Our approach is at an intermediate "meso-" level, where instead of describing dynamics at a "micro"-level of neural-networks, or the "macro"-level of whole actions, we consider self-reinforcing, patterns of sensorimotor activity. By monitoring trajectories in sensorimotor space (see Figures 1 and 2), our innovated robot controller, the Iterant Deformable Sensorimotor Medium (IDSM) causes robots or virtual agents to repeat patterns of sensorimotor activity that it has previously experienced.

Certain patterns of sensorimotor activity facilitate the repetition of that behaviour more than others. For instance, walking around in a circle until you are where you started sets you up perfectly to repeat that behaviour, where as jumping out of an exploding airplane is an action that does not.

In our first explorations of the IDSM, we were intrigued to notice that even in the absence of a reward mechanism, the patterns of behaviour that form relate to the environment in "meaningful" or goal-like ways. For example, the central pane of Video 1 shows an overhead view of a robot moving around a two-dimensional environment. In the center of this world is a light. Over time, the robot demonstrates a number of repetative, self-reinforcing "habits." ---and interestingly, they all relate in some way to the light, in many cases causing the robot to circle around the light. Other experiments have shown similar results, even when the light is moving around the environment.

After some consideration, it becomes clear that these behaviours must relate to the light in a consistent and repeatable manner, as they are based upon repeated sensorimotor trajectories. The only way for a sensorimotor trajectory to be repeated is if the robots motion causes its sensory state to change in a way that is consistently associated with motor changes. Without any reward or additional selection mechanism, the robot falls into patterns of behaviour that are based upon --in a sense, built out of-- sensorimotor contingencies.

Next steps

There are many appealing avenues for further investigation. Briefly here, I will outline a few projects of interest. If you have other ideas, please get in touch.

Enumerating, categorizing and calculating the relative frequency of the set of possible self-reinforcing sensorimotor trajectories of minimalistic robots in minimalistic environments. Some patterns of activity can be repeated others cannot. Some can be repeated from many more initial conditions than others. Studying the simplest possible environments and robots, we are building a compendium of habits, and calculating the relative frequency of each category of behaviour. By understanding what habits form in the absence of additional constraints, we can develop a better understanding of how to implement additional selective mechanisms that increases the likelihood of target habits forming.

Investigating the coconstruction of language. By coupling the IDSM to a microphone and a sound-synthesizer, we will study what is necessary for the IDSM to create self-sustaining and self-reinforcing patterns of interaction with humans.

How do "hard-wired" reflexive behaviours bias habit formation in the IDSM? Using genetic algorithms, we will evolve reflexive behaviours that bias habit formation such that desired patterns of behaviour are generated.

References