Daniel Marbach - Master Thesis (2004-2005)

Evolution and Online Optimization of Modular Robot Locomotion

The contribution of this project is three-fold:

1) Inspired by Central Pattern Generators (CPGs), i.e. neural networks found in vertebrates that produce complex patterns of oscillatory outputs without oscillatory inputs, I propose a distributed locomotion controller based on coupled nonlinear oscillators.

2) For offline optimization, a genetic algorithm that co-evolves the CPG with the configuration of the modular robot is presented

3) Clearly, the most interesting part of my work is online optimization (we could also say learning or adaptation) of locomotion. I propose a novel method for fast online adaptation of locomotion gaits in the face of module failures or new, previously unknown configurations.

An asynchronous, distributed locomotion controller based on nonlinear oscillators

It is generally acknowledged that modular robot control should be distributed in order to be fault tolerant and to scale with the number of modules used. Using coupled nonlinear oscillators as canonical subsystems to model CPGs it is indeed possible to design distributed, asynchronous controllers that rely only on local communication (the couplings). Global synchronization emerges gradually through local interaction and neighboring modules synchronize smoothly, i.e. there are no abrupt changes in the generated trajectories.

Co-evolution of morphology and control

For offline optimization of locomotion, a genetic algorithm co-evolves the configuration of the modular robot with the CPG. A genetic encoding that restricts the phenotype space to symmetric configurations is used.

The evolved robots are used as a benchmark and testbed for the online optimization.

Some evolved robots

Online Optimization / Adaptation of Locomotion

Adaptive locomotion in unstructured and unpredictable environments is one of the most advertised features of modular robots in the literature. Autonomous modular robots are expected to adapt in the face of a dynamic environment, unexpected tasks and/or module failures. There are two levels of adaptation: Within a static configuration, a chain-type modular robot can adapt its locomotion gait using its many degrees of freedom and the inherent redundancy. In addition, the robot may self-reconfigure to adapt also its morphology. Online optimization of locomotion in a self-organizing manner is mandatory within this context.

I see two major applications for the online optimization (we could also say adaptation or learning) algorithm: First of all, it is possible to quickly optimize the locomotion gait for a new, previously unknown configuration. This may be the case after self-reconfiguration or selfassembly. Secondly, the locomotion gait may be adapted to module failure(s). The algorithm relies on Powell's method for multidimensional function optimization.

Fitness landscapes

Movies

The movies are in mpeg-4 format. The Quicktime version has a very small image, the mpeg movies are much nicer. If you experience problems watching them, this is a good opportunity to get a better player - for example Quicktime or VLC - it's just one click away and installation takes a few seconds.

	Some interesting results from different co-evolutionary runs: mpeg-4 H.264 (H.264 encoding. Best quality, smallest file size) mpeg-4 standard (standard encoding, try only if H.264 doesn't work) DivX 5.2 (standard encoding, try only if H.264 doesn't work) mpeg-1 standard (standard encoding, try only if H.264 doesn't work) Quicktime (Not recommended, very small image!)
	Online optimization of a simple walking robot that had previously been evolved with the genetic algorithm. All control parameters are reset to random values before the online optimization starts: mpeg-4 H.264 (H.264 encoding. Best quality, smallest file size) mpeg-4 standard (standard encoding, try only if H.264 doesn't work) DivX 5.2 (standard encoding, try only if H.264 doesn't work) mpeg-1 standard (standard encoding, try only if H.264 doesn't work) Quicktime (Not recommended, very small image!)
	Online optimization of an eleven degrees of freedom robot. All control parameters are reset to random values before the online optimization starts. This robot is quite complex, nevertheless a very efficient and elegant locomotion gait is found in less than 40 minutes of simulated time:

Peer-reviewed Proceedings

D. Marbach and A.J Ijspeert. Online Optimization of Modular Robot Locomotion. In Proceedings of the IEEE International Conference on Mechatronics & Automation (ICMA 2005), Niagara Falls, Canada, pp.248-253, July 2005.

D. Marbach and A.J. Ijspeert. Co-evolution of configuration and control for homogenous modular robots. In F. Groen et al., editor, Proceedings of the Eighth Conference on Intelligent Autonomous Systems (IAS8), pp.712-719. IOS Press, 2004.

Master Thesis

Master Thesis Report (16.3 MB)

Paper describing the online optimization algorithm (Annex to the report, 1.9 MB)

Slides of the presentation (6.6 MB)

Links

MR hardware at BIRG

MR adaptive locomotion at BIRG

My undergraduate research project on MR locomotion

PhD project:Reverse Engineering of Gene Regulatory Networks

"Robots", the robotics podcast

Maconly

Contact

Daniel Marbach