A couple of advanced data science algorithms. Implemented both for the walking table. ARS is great. We hear a lot about deep learning. This one is shallow learning, and does very well on simpler tasks. Just inputs and outputs, no hidden layers.
It’s similar to the Evolution Strategies algorithm. Generally trying some random stuff out, and slowly changing the model based on what gets you closer to the goal.
After much confusion, I made a table with legs that can move.
So I raised the table to z=1 plane and put the origin of the joints in the right place. THEN you set the legs/links with origin at -0.5 because the joint is at 1, and the leg is 1 long, and presumably it needs 0.5 because that’s the centre of the box.
I did visualisation by changing the filename in this file in the bullet3 folders, and running in python:
So I’ve got the RPi camera images sending to my laptop now, after installing OpenCV4, and running the test code from https://github.com/jeffbass/imagezmq
from board import SCL, SDA
import busio
from adafruit_pca9685 import PCA9685
from adafruit_motor import servo
i2c = busio.I2C(SCL, SDA)
pca = PCA9685(i2c)
pca.frequency = 50
servo2 = servo.Servo(pca.channels[2])
for i in range(90):
servo2.angle = i
for i in range(90):
servo2.angle = 90 - i
pca.deinit()
i changed it to 90 degrees and got rid of the comments. It suggests a min and max for the servo.
I ran it and the servo got angry with me and wouldn’t stop. I had to unplug everything because it was eating up the cables in its madness.
It keeps going if I plug just the power back in. It seems to rotate continuously. So something is f***ed. Rebooting RPi. It’s supposed to be 180 degree rotation. Will need to read up on servo GPIO forums.
Starting Nmap 7.60 ( https://nmap.org ) at 2020-04-05 17:06 UTC Nmap scan report for _gateway (192.168.101.1) Host is up (0.0026s latency). MAC Address: B8:69:F4:1B:D5:0F (Unknown) Nmap scan report for 192.168.101.43 Host is up (0.042s latency). MAC Address: 28:0D:FC:76:BB:3E (Sony Interactive Entertainment) Nmap scan report for 192.168.101.100 Host is up (0.049s latency). MAC Address: 18:F0:E4:E9:AF:E3 (Unknown) Nmap scan report for 192.168.101.101 Host is up (0.015s latency). MAC Address: DC:85:DE:22:AC:5D (AzureWave Technology) Nmap scan report for 192.168.101.103 Host is up (-0.057s latency). MAC Address: 74:C1:4F:31:47:61 (Unknown) Nmap scan report for 192.168.101.105 Host is up (-0.097s latency). MAC Address: B8:27:EB:03:24:B0 (Raspberry Pi Foundation) Nmap scan report for 192.168.101.111 Host is up (-0.087s latency). MAC Address: 00:24:D7:87:78:EC (Intel Corporate) Nmap scan report for 192.168.101.121 Host is up (-0.068s latency). MAC Address: AC:E0:10:C0:84:26 (Liteon Technology) Nmap scan report for 192.168.101.130 Host is up (-0.097s latency). MAC Address: 80:5E:C0:52:7A:27 (Yealink(xiamen) Network Technology) Nmap scan report for 192.168.101.247 Host is up (0.15s latency). MAC Address: DC:4F:22:FB:0B:27 (Unknown) Nmap scan report for chrx (192.168.101.127) Host is up. Nmap done: 256 IP addresses (11 hosts up) scanned in 2.45 seconds
if nmap is not installed,
apt-get install nmap
Connect to whatever IP it is
ssh -vvvv pi@192.168.101.105
Are you sure you want to continue connecting (yes/no)? yes
And I’ll unplug the ethernet and try connect by ssh again
Ah, but it’s moved up to 192.168.1.106 now
nmap -sP 192.168.101.0/24 (I checked again) and now it was ‘Unknown’, but ssh pi@192.168.101.106 worked
(If you can connect to your router, eg. 192.168.0.1 for most D-Link routers, you can go to something like Status -> Wireless, to see connected devices too, and skip the nmap stuff.)
I log in, then to configure some stuff:
sudo raspi-config
Under the interfaces peripheral section, Enable the camera and I2C
worked for setting up wireless access on the sd card.
“Since May 2016, Raspbian has been able to copy wifi details from /boot/wpa_supplicant.conf into /etc/wpa_supplicant/wpa_supplicant.conf to automatically configure wireless network access”
In essence, the default pi user no longer exists, so you have to create it and set its password using either the official Imager tool or by creating a userconf file in the boot partition of your microSD card, which should contain a single line of text: username:hashed-password
I started up a raspberry pi with raspbian installed on it. (I used balena’s etcher to flash an sd card): the light version, basically just Debian OS for rpi: https://www.raspberrypi.org/downloads/raspbian/
Ok but how do we power servos? We can’t run it off the RPi’s 5V 2A. Oh duh, there’s that big DC socket connected to the PCA9685
There is something to be said for running the robot on an Arduino. You get something robotic. You upload someone’s hexapod spider code, and it does a little dance. You can control it remotely. It can interact with sensors.
Arduinos are dirt cheap. So I bet we could have tiny neural networks running in arduinos… shit, do we already have them? Let’s see… ok wow there is like a whole thing. https://blog.arduino.cc/2019/10/15/get-started-with-machine-learning-on-arduino/ ok but 2K of RAM is not much. You would need to be a real demo scene junky to use a 2KB NN, but yeah, you could do something crudely intelligent with it.
Robots on ESP32s are definitely a thing. But ok no, Raspberry Pi for real robot. We need Linux for this.
Ok so I need to wire this up. But I also need a chassis for the robot.
According to some blog bit: https://gitmemory.com/issue/bulletphysics/bullet3/2118/468943185 there might be an issue: “The problem with this demo is the following: It can not optimize since the robots are not properly restored. Their physical settings are not properly restored and thus their performance varies. I think it should only serve as a bullet example that does some simulation. If you can improve it to restore the physics correctly, that is fine, but this might be hard to do if you are new to bullet. It would be best to reimplement it in pybullet, because it features deterministic restoring of robots.”
So will have to keep that in mind.
So starting in the NN3DWalkersTimeWarp code, we quickly run into ERP and CFM http://www.ode.org/ode-latest-userguide.html#sec_3_7_0 – These are error reduction parameter and constraint force mixing and help fix errors caused by joints that aren’t set up perfectly.
The code also loads a number of Constraint Solvers:
btDefaultCollisionConfiguration in a btCollisionDispatcher
a default btDbvtBroadphase (a sweeping style collision detection)
and whichever solver is being used.
The Dynamics world is initiated as a btDiscreteDynamicsWorld
There’s timeWarpSimulation which we override, and the stepSimulation which is in a loop called the ‘canonical game loop’. It updates any variables changed by the UI parameters, then calls our code, and then updates and time variables that need refreshing, and gives you a chance to update some graphics. It gets to the end of the loop, checks the time, to decide whether to render the graphics, or to do the loop again.
That’s sufficient for now, to move on with the CPP file code.
Another interesting project I saw now, using OGRE, which is some other physics engine, I think, is https://github.com/benelot/minemonics – has some evolved life too.
But we are evolving something that looks more like these creepy rainbow spider bombs, that ‘we made’ in MotorDemo.cpp.
These above are the variables for the walker, which is similar to the MotorDemo critter. The code has been upgraded here and there since MotorDemo.cpp. It’s pretty great that people share their hard work so that we can just waltz on in and hack a quick robot together. This guy https://github.com/erwincoumans founded the Bullet project, and now works for Google Brain. (on Google Brain?)
So these are some of the public methods for the WalkersExample
We want to evolve behaviours based on stimulus, and the NNWalkers even have m_touchSensors, which we’ll have a look at later, to wee what they let us do.
Interlude: http://news.mit.edu/2015/algorithm-helps-robots-handle-uncertainty-0602 here says the following, and also has a video with sick drone rock music. This is a good part to remember, that either the robot is pausing, evaluating, pausing, evaluating, to accomplish the behaviours using ‘online’ feedback and offline policy processing.
There’s an offline planning phase where the agents can figure out a policy together that says, ‘If I take this set of actions, given that I’ve made these observations during online execution, and you take these other sets of actions, given that you’ve made these observations, then we can all agree that the whole set of actions that we take is pretty close to optimal,’” says Shayegan Omidshafiei, an MIT graduate student in aeronautics and astronautics and first author on the new paper. “There’s no point during the online phase where the agents stop and say, ‘This is my belief. This is your belief. Let’s come up with a consensus on the best overall belief and replan.’ Each one just does its own thing.
bool fitnessComparator(const NNWalker* a, const NNWalker* b)
{
return a->getFitness() > b->getFitness(); // sort walkers descending
}
So we are maximising fitness, and this is the Comparator function
The ratingEvaluations function orders by fitness, and prints the square root of the best performing individual’s distance fitness. Then we iterate through the walkers, update the time series canvas tick (next generation). Zero the times, and counters.
void NN3DWalkersExample::rateEvaluations()
{
m_walkersInPopulation.quickSort(fitnessComparator); // Sort walkers by fitness
b3Printf("Best performing walker: %f meters", btSqrt(m_walkersInPopulation[0]->getDistanceFitness()));
for (int i = 0; i < NUM_WALKERS; i++)
{
m_timeSeriesCanvas->insertDataAtCurrentTime(btSqrt(m_walkersInPopulation[i]->getDistanceFitness()), 0, true);
}
m_timeSeriesCanvas->nextTick();
for (int i = 0; i < NUM_WALKERS; i++)
{
m_walkersInPopulation[i]->setEvaluationTime(0);
}
m_nextReaped = 0;
}
The reap function uses the REAP_QTY (0.3) to iterate backwards through the worst 30%, set them to reaped.
void NN3DWalkersExample::reap()
{
int reaped = 0;
for (int i = NUM_WALKERS - 1; i >= (NUM_WALKERS - 1) * (1 - REAP_QTY); i--)
{ // reap a certain percentage
m_walkersInPopulation[i]->setReaped(true);
reaped++;
b3Printf("%i Walker(s) reaped.", reaped);
}
}
getRandomElite and getRandomNonElite use SOW_ELITE_QTY (set to 0.2). The functions return one in the first 20% and the last 80%.
The getNextReaped() function checks if we’ve reaped the REAP_QTY percentage yet. If not, increment counter and return the next reaped individual.
NNWalker* NN3DWalkersExample::getNextReaped()
{
if ((NUM_WALKERS - 1) - m_nextReaped >= (NUM_WALKERS - 1) * (1 - REAP_QTY))
{
m_nextReaped++;
}
if (m_walkersInPopulation[(NUM_WALKERS - 1) - m_nextReaped + 1]->isReaped())
{
return m_walkersInPopulation[(NUM_WALKERS - 1) - m_nextReaped + 1];
}
else
{
return NULL; // we asked for too many
}
}
Next is sow()…
SOW_CROSSOVER_QTY is 0.2, SOW_ELITE_PARTNER is 0.8. SOW_MUTATION_QTY is 0.5. MUTATION_RATE is 0.5. SOW_ELITE_QTY is 0.2. So we iterate over 20% of the population, increase sow count, get random elite mother, and 80% of the time, a random elite father (20% non-elite). Grab a reaped individual, and make it into a crossover of the parents.
Then mutations are performed on the population from 20% to 70%, passing in some scalar to the mutate().
Finally, the REAP_QTY – SOW_CROSSOVER_QTY is 10%, who are sown as randomized motor weight individuals.
Ok…
void NN3DWalkersExample::sow()
{
int sow = 0;
for (int i = 0; i < NUM_WALKERS * (SOW_CROSSOVER_QTY); i++)
{ // create number of new crossover creatures
sow++;
b3Printf("%i Walker(s) sown.", sow);
NNWalker* mother = getRandomElite(); // Get elite partner (mother)
NNWalker* father = (SOW_ELITE_PARTNER < rand() / RAND_MAX) ? getRandomElite() : getRandomNonElite(); //Get elite or random partner (father)
NNWalker* offspring = getNextReaped();
crossover(mother, father, offspring);
}
for (int i = NUM_WALKERS * SOW_ELITE_QTY; i < NUM_WALKERS * (SOW_ELITE_QTY + SOW_MUTATION_QTY); i++)
{ // create mutants
mutate(m_walkersInPopulation[i], btScalar(MUTATION_RATE / (NUM_WALKERS * SOW_MUTATION_QTY) * (i - NUM_WALKERS * SOW_ELITE_QTY)));
}
for (int i = 0; i < (NUM_WALKERS - 1) * (REAP_QTY - SOW_CROSSOVER_QTY); i++)
{
sow++;
b3Printf("%i Walker(s) sown.", sow);
NNWalker* reaped = getNextReaped();
reaped->setReaped(false);
reaped->randomizeSensoryMotorWeights();
}
}
Crossover goes through the joints, and sets half of the motor weights to the mom and half to the dad, randomly.
void NN3DWalkersExample::crossover(NNWalker* mother, NNWalker* father, NNWalker* child)
{
for (int i = 0; i < BODYPART_COUNT * JOINT_COUNT; i++)
{
btScalar random = ((double)rand() / (RAND_MAX));
if (random >= 0.5f)
{
child->getSensoryMotorWeights()[i] = mother->getSensoryMotorWeights()[i];
}
else
{
child->getSensoryMotorWeights()[i] = father->getSensoryMotorWeights()[i];
}
}
}
mutate() takes a mutation rate, and randomizes motor weights at that rate.
void NN3DWalkersExample::mutate(NNWalker* mutant, btScalar mutationRate)
{
for (int i = 0; i < BODYPART_COUNT * JOINT_COUNT; i++)
{
btScalar random = ((double)rand() / (RAND_MAX));
if (random >= mutationRate)
{
mutant->getSensoryMotorWeights()[i] = ((double)rand() / (RAND_MAX)) * 2.0f - 1.0f;
}
}
}