Robot Brain-

I have been thinking for sometime about the design of a robot brain. The primary ideas that I want to explore are to try to minimize code development and maximize data. I want the system to stand up and walk without dealing with vision or with path finding or route planning. I would like the system to learn to walk entirely on its own. It should lie around and thrash and learn to move its legs and eventually figure out how to crawl, stand, walk, run, jump, skip and do flips.

The primary notion that I have in mind is that the robot’s goals are different from my goals. I want the robot to stand still. I want the robot to walk over there and turn around. However the robot as it is learning has a very different set of goals. The robot is simply trying to map out a relationship between operations it can perform, like activating motors, and effects that those operations have on it’s state.

The robot lives in state space. It has a simple goal, which is to learn how to go from any internal state to any other internal state. It does this in the following manner. At any time the robot is in some state. It performs some action (possibly random) and looks at what state it ends up in. Next time the robot is in that same state, it either performs some new action of the old action. It performs the old action sufficient times to see if it is stable i.e. if actionX takes us from stateA to stateB nearly every time then actionX is a stable path from A to B.

The robot’s goal is to map out state space. It should be thrilled whenever it finds itself in a new state. It should try to get back to that state. Perhaps it can get there, perhaps it cannot. Some regions of state space will be stable. Some others will be chaotic. For example if you are falling, clattering to the ground you may end up in some unpredictable state.

The fundamental thing that the robot does is that it learns to navigate through state space, primarily by traversing stable actions from one place to another. As it learns a section of state space, “Hey, look, when my knee was at this angle and I move this motor, the angle changes in strict proportion every single time!” it becomes bored. The system traverses stable paths a few times to see that they are in fact stable, but thereafter does not spend its time exploring those paths, instead it uses those paths to navigate itself into states that it has not thoroughly explored.

The presumption is that given time the robot will do a reasonable job of mapping its state and will be able to get from any place to any other place. Navigation becomes the simple algorithm of finding a minimal path in the graph of states connected by stable actions.

If it behaves as expected, the system should thrash around learning the map of its knee joints, just rolling around on the floor. Presumably by accident it does something that gets itself sitting up. Cool! Can it do it again? It simply tries to get from wherever it ended up back to the state just before sitting up to see if it can consistently get into that state.

Some things it does while sitting up will cause it to fall over. Other things will not. It maps out where it can get to from sitting up. Can it get into a crouch? From a crouch can it get to a standing position? Can it stay in a standing position? That is all the robot does

After you have let the system learn how to walk around in a clear and empty room you introduce elements that it can trip over. Stable pathways are no longer necessarily stable. Used to be if you swung your leg forward it would get where it was going. Now, occasionally, it runs into something and its hips are suddenly torquing forward. This is a new state. Can it recover from that state? Can it regain its balance?

Notice that the system should be very robust. It maps a path through state space from here to there. It tries to stay on stable pathways. Something happens and it ends up in a state that it did not expect. Fine, just calculate a new path from where you accidentally ended up back to the original goal. The robot should end up making a minor correction if such a thing is possible and just keeps on going.

Notice also that the system does not just get stuck rolling around on the floor like an idiot. Been there done that. It certainly spent some time rolling around on the floor when that was new and interesting. When you ask the system to go from here to there, does it lie down and roll, or does it stand up and walk? If walking is more efficient and you can get from here to there in fewer state-steps than you can by rolling then the system will take that shorter state space path and walk over there. However if there is a low ceiling (let’s ignore for the moment how the mindless robot detected that) so that it can’t get at some of those standing up states, It simply plots a path through its state space (using only legal low down states) lays down and rolls there.

We have of course ignored here any of the considerations about how you want to encode the state space and the stable transitions. I expect that state space will have large essentially linear essentially orthogonal sections. For example, moving this motor moves this knee and moving that motor moves that knee and the dimensions are separable and independent just like Euclidean space. One should not need to code up every single state as a separate data item with a separate bunch of links coming out of it representing the actions. What I mean is that I described the robot brain in terms of state space nodes and action edges for an enormous graph, however it may well be that the state space you are mapping resembles Euclidean 3 space. You can move any continuous amount in X, in Y and in Z and lo and behold you end up at where you were plus delta X, delta Y, and delta Z. You can encode large linear regions much more efficiently than using a ball and stick model of a graph.