Evolutionary Robotics Programming Assignment 10 of 10

CS206: Evolutionary Robotics
Programming Assignment 10 of 10
Description: In this final assignment, you will connect the Python application you developed in
assignments 1 through 3 with the robot simulator you have created in assignments 4 through 9.
When you are done, the two applications will work together as follows:
• The Python hill climber will start up the robot simulator and send it a collection of random
synaptic weights.
• The robot simulator will read in these weights, and use them to set the synaptic weights of
the artificial neural network you created in the last assignment.
• The robot simulator will run for 1000 time steps, and then record the final position of the
robot’s main body. It will save this value to a file and then exit.
• The Python hill climber will read in this value and treat it as the fitness of the synaptic
weights it just sent.
• It will create a copy of these weights, and mutate them slightly. It will then start up the robot
simulator again, send it this new set of weights, and so on.
Let us get started.
1. Create an empty directory called Assignment10.
2. Copy your Assignment3.py file from assignment 3 into this directory and rename it
Assignment10.py.
3. Recall that the Python code implements a hill climber, a simple evolutionary algorithm that
optimizes a neural network to exhibit some desired behavior. Comment out each line of the
main function so that when you run the code it starts and then immediately terminates:
#numGenes = 10;
#numGenerations = 1000;
...
#fits = Evolve(numGenes , ...);
4. Recall that in the function Evolve, a 10 × 10 matrix of synaptic weights was evolved. We
now need to evolve a 4 × 8 matrix of synaptic weights: one synaptic to connect each of the
four sensors to each of the eight motors. So replace the numGenes=10 line with
numSensors = 4;
numMotors = 8;
#numGenerations = 1000;
...
1
5. Now uncomment the Evolve function, and change the parameters sent to it:
fits = Evolve(numSensors , numMotors , ...)
6. Comment out all of the lines in the Evolve function so that your code still runs yet does
not do anything (yet).
7. Now uncomment and modify the first four lines in Evolve accordingly: these lines create
a vector for saving the best fitness as evolution proceeds; they create a random matrix of
synaptic weights; and they evaluate the fitness of those weights:
fits = Matrix_Create(1,numGenerations);
parent = Matrix_Create(numSensors,numMotors);
parent = Matrix_Randomize(parent)*2-1;
parentFit = Fitness3_Get(parent);
8. The new function Fitness3 Get starts up the robot simulator and sends the matrix of
weights to it; it waits for the robot simulator to terminate (after the simulator has saved out
the fitness of those weights); reads in the fitness value; and deletes the files that allow the
two programs to communicate with each other.
def Fitness3_Get(synapses):
weightsFileName = ’weights.dat’;
fitFileName
= ’fit.dat’;
Send_Synapse_Weights_ToFile(synapses,weightsFileName);
Simulate_Robot();
Wait_For_Fitness_File(fitFileName);
fitness = Fitness_Collect_From_File(fitFileName);
Delete_File(weightsFileName);
Delete_File(fitFileName);
return( fitness );
9. Comment out all of the lines below Send Synapse Weights ToFile, and then implement only this function. This function should save all 32 weights to file weights.dat in
the same directory as the python and robot simulator executables. Given that the rest of the
Python code is commented out, when you run your code it should save this file and then quit.
2
10. Let us turn now to your robot simulator code. Make a copy of Assignment 9, and name
the copy Assignment 10. Make sure that the simulator executable is compiled into your
new Assignment10 directory.
11. We need to ensure that the robot simulator, when initiated, runs for 1000 time steps and then
terminates. To do so, we need to define a new variable int timeStep, set it to zero in
initPhysics, and then increment it during each pass through clientMoveAndDisplay.
(Make sure also that the application always starts unpaused now.)
12. Once 1000 timeSteps have elapsed, the application should terminate:
void RagdollDemo::clientMoveAndDisplay() {
...
timeStep++;
...
if ( timeStep==1000 )
exit(0);
}
13. Run the executable several times. You should see the robot move for 1000 time steps and
then see the simulation terminate.
14. You should have code in initPhysics that sets the elements of matrix weights to
random values. Modify this code so that instead the elements are set to the values stored in
weights.dat.
15. Now we need to capture the final position of the robot’s main body. So modify the code in
clientMoveAndDisplay as follows (you will need to make use of Bullet’s getCenter
OfMassPosition function):
void RagdollDemo::clientMoveAndDisplay() {
...
if ( timeStep==1000 ) {
Save_Position(body[0]);
exit(0);
}
}
Within the function, capture the (x,y,z) position of the main body, and print the z-component
of this position to the screen. This component reports how much the robot moves toward the
viewer (negative values) and how much it moves ‘into the screen’ (positive values).
16. You may need to pause execution after you print the value, because the application will
terminate immediately after printing the value. (The following will pause execution until the
user presses ‘Enter’.)
3
printf...
char ch = getchar();
17. Within Save Position, write this value to the file fits.dat immediately after printing
it. Verify that the number in the file and the number printed to the screen are the same.
18. Now run the executable several times. Note that it is reading in the same set of weights each
time, so the robot’s behavior (and its final position) should always be the same. You may see
that the behavior is different from one execution to the next. This is because it is sometimes
difficult to get the physics engine to behave deterministically, even though in theory it should
do so at all times.
19. If you do see that the final positions differ from one run to the next, the culprit often lies in
the detection and resolution of contacts. This sometimes results in the touch sensors only
being updated once every few time steps, and sometimes this updating is not deterministic.
You can fix this by replacing
m_dynamicsWorld->stepSimulation(ms / 1000000.f);
in clientMoveAndDisplay with the following:
while ( (touches[2]==0)
(touches[4]==0)
(touches[6]==0)
(touches[8]==0)
&&
&&
&&
)
m_dynamicsWorld->stepSimulation(ms / 1000000.f);
(This assumes that the third, fifth, seventh and ninth body parts are the four lower legs
respectively.) This new code continuously updates the motion of the robot until at least one
of the touch sensors fires.
20. If you add this code and you are still seeing different final positions from one run to the
next ensure that (1) you are reading in the same 32 synaptic weights in initPhysics
each time; (2) you are running the simulation for exactly 1000 time steps each time; and (3)
all eight motors receive the same desired velocity each time enableAngularMotor is
called. If you are still observing non-deterministic behavior, contact the instructor.
21. Remove the line that prints the main body’s final position and the line that pauses the simulator so that the simulator now runs and then terminates without any user intervention.
22. Now return to your Python code. Immediately after Fitness3 Get sends the weight matrix to a file, it must start the robot simulator. You can use the os package which allows
system commands to be executed from within Python code:
4
import os;
...
def Fitness3_Get(...):
[send weight matrix to file weights.dat]
# If you are using a Windows machine...
os.system(’App_RagdollDemo_vs2010.exe’);
# If you are using a Mac or Linux machine...
#os.system(’./AppRagdollDemo’);
When you run your Python code now, you should see the robot simulator start automatically and then terminate after 1000 time steps. There should be both a weights.dat and
fit.dat file sitting in the directory.
23. We must now instruct the Python application to wait until the fit.dat file appears. First,
make sure to delete fit.dat in the directory by hand and then add the code:
import time;
import sys;
...
def Fitness3...
os.system(...)
while (exists(’fit.dat’) == False):
time.sleep(0.2);
sys.exit(0);
Now, when you run your Python application, you should see it stay open while the robot
simulator runs. When the simulator terminates, so too should the Python application.
24. Remove the sys.exit(0) line, and follow it with code that reads in the single value stored
in fit.dat. This value should be stored in the variable fitness, which is returned when
Fitness3 Get terminates.
25. But before this function terminates it must delete the two files weights.dat and fit.
You can accomplish this using the Python command os.remove(fileName).
26. Now when you run your Python code you should should observe the following: (1) the
Python application stays open while the simulator runs; (2) both applications terminate together; and (3) weights.dat and fit.dat are no longer in the directory.
27. You can now uncomment the remaining lines of Evolve, so that the hill climber runs.
Remember to use the new Fitness3 Get function when evaluating the fitness of the child
set of synaptic weights:
5
Figure 1: Visual display of a high-fitness robot and a running print out from the hill climber that
produced it.
...
parentFit = Fitness3_Get(parent);
for g in range(0,numGenerations):
...
childFit = Fitness3_Get(child);
print g, parentFit, childFit;
if ( childFit > parentFit ):
...
28. You should observe a series of robots now. Over a few minutes, you should observe robots
evolving the ability to locomote increasingly further ‘into the screen’. Capture one such
robot in a screenshot, alongside a print out of your hill climber running. The first value in
each row should report the current generation; the second value reports the furthest any robot
has yet travelled ‘into the screen’ (parentFit); and the third value should report the distance
travelled by the previous robot (childFit). Upload this single screenshot to the Wiki.
6