Odometry on a Mecanum Robot Using an Optical Flow Sensor

While I’ve lately been mostly focused on my KR01 robot, I’ve also been planning a Mecanum-wheeled robot to be called the KRZ02. I long ago decided my robots would incorporate odometry, counting the rotations of each motor in order to track the robot’s location. This is accomplished by either an optical or Hall-effect encoder connected to the left and right motor shafts. On the KR01, since I know the gear ratio of the motor, the size of the wheel (and that it travels 218mm per rotation), and that the encoder sends 494 steps per rotation of the wheel, I can calculate there’ll be 2262 steps per meter.

To be honest, I didn’t actually come up with such a formula but simply measured this by repeatedly having the robot move exactly one meter forward. In science this is called an observation. Observing is generally easier than calculating, but is still a valid means of finding things out. You can make calculations without having a robot, but if your calculation (the model) is flawed or incomplete it won’t be accurate, whereas careful observation of the robot while in the actual working environment can be quite accurate. My guess is that if I had gone to the trouble to develop an odometric formula I wouldn’t have ended up with a value of 2262 steps per meter, as somewhere in the mix a concrete, physical system often introduces variables that haven’t been accounted for in the model.

But back to the point. With knowledge of how many steps the motor encoders return I can calculate both the velocity and distance traveled for each motor, and therefore with a reasonable degree of accuracy where the robot is located from its starting location. David Anderson of the DPRG has made odometry into a fine art, and his robots can travel great distances in both complex indoor environments and even across difficult terrain up in the mountains and return to within inches of their starting locations. I find David’s work very inspiring.

But all this clever odometry falls apart when using Mecanum wheels, which have a series of rubber rollers that each have an axis of rotation at 45° to the wheel plane and at 45° to the axle line.

A Mecanum wheel with micro metal gearmotor and built-in Hall-effect encoder.

Whereas a traditional wheel translates its energy to the ground in a rather predictable way, the rotation of a Mecanum wheel interacts in complex ways with the other Mecanum wheels. I’m sure there’s a mathematical formula for this, one that would involve the direction and rotational velocity of each wheel, the total weight and weight distribution of the robot, the hardness of and therefore how much of each roller is in contact with the ground, the traction and rolling friction of the wheel rollers, the friction due to contact with the ground, roller slippage, and probably another half dozen unknowns. Call me lazy but my life is too short to even consider trying to create that formula and develop sensors for the various parameters of that equation. And we’re back to that abstract model versus concrete observation issue. If the goal is accurately tracking the robot’s movement over the ground (odometry), then we need to come up with another method.

Flying drones can’t do motor-based odometry but instead use a specialised camera called an optical flow sensor, something we might call a camera-as-sensor. An optical flow sensor’s camera looks down at the ground, generating a series of image frames, calculating the distance the drone has moved across the landscape by tracking the differences in position between image frames. This is returned as an x,y value at the frame rate of the camera. With both a GPS unit and a LiDAR providing a distance to ground measurement it’s possible to accurately perform odometry in mid-air.

PMW3901MB-TXQT
2x actual size

Now, my robot lives on the ground but there’s no reason we can’t try a similar trick. The actual sensor I’m using is from PixArt Imaging, designated the PMW3901MB-TXQT, and is all of 6 x 6 x 3 mm in size, using 6 milliamps of current. That’s including the camera and all of the electronics. This reminds me of the sensor used in the VL531LX Time of Flight sensor, which is a LiDAR the size of a grain of rice.

These sensors are generally sold on a carrier board so that they can be integrated into a commercial or hobbiest application. The carrier board I’m using is from Pimoroni, called the PMW3901 Optical Flow Sensor Breakout, part of their Breakout Garden series of sensors. The cost is about what we pay for two meals at a local Indian restaurant. You can solder a 7 pin header to the carrier board or simply plug it into an SPI socket on a Breakout Garden board.

The Pimoroni PMW3901 Optical Flow Sensor SPI Breakout

The PMW3901 has a frame rate of 121 frames per second and a minimum range of about 80mm. I’m hoping to mount it looking down from the underside of the robot’s upper board, which will be just above that 80mm minimum. The problem is twofold: how to provide the sensor’s camera with a clear view of the ground, and how to mount it in the center of the robot. This is necessary so that when the robot rotates around its center the sensor won’t register any absolute movement, just a rotational theta.

On the KRZ02 I’m using 48mm Mecanum wheels made by Nexus Robot. They’re a high quality steel framed wheel with some rather solid brass hubs and a load capacity of 3kg. When mounted on a Pololu Micro Metal Gearmotor the bottom of the robot’s 3mm thick Delrin plastic lower board is 30mm from the ground. This means the PMW3901 needs to be at least 47mm above that board.

I built a test rig frame out of a cheap nylon chopping board. A photo of the rig can be found at the top of this post. This holds the PMW3901 at a distance of 90mm from the ground, looking through a 50mm hole cut into a lower board. 50mm is the biggest hole cutter I have:

I used a Raspberry Pi Zero W and wrote a Python library and test file. In the test, the PMW3901 sensor returns positive and negative x and y values as it senses movement. Based on those values I’m setting the RGB LEDs to red for a movement to port, green for starboard, cyan for forward and yellow for aft/reverse.

The 5×5 RGB Matrix used to indicate direction.

My initial tests quickly proved faulty. The white plastic of the cutting board was providing all sorts of reflections of ambient light as well as the PMW3901’s illumination LEDs, and this confused the sensor to no end. Even at rest it would be indicating what seemed to be random motion. I taped some black matte paper to the board and this almost entirely eliminated the problem. I won’t be using white nylon on the robot but rather black Delrin plastic, which I can sand to a dull matte finish, so hopefully this will be sufficient. If not, there’s a rubberised black fabric used in photography that reflects almost no light.

The following video shows the test rig in operation.

As I noted in the video, the results indicate the test is a success, i.e., mounting a PMW3901 Optical Flow Sensor at about 90mm from the ground can provide odometry information for a ground-based robot.

With this important test out of the way I can now finish the plans and begin building the KRZ02 robot.

Credit where credit’s due: the beautiful fabric you can see in the photos and video is actually a reusable grocery bag designed by one of my favourite New Zealand artists, Michel Tuffery, as part of the Paper Rain Project.

We’re Goin’ Mecanum!

Not that I’ve spent much time with my KRZ01 robot. I feel almost bad that I haven’t let the project develop much before making some significant changes. Like that high school girlfriend with braces.

Mecanum wheel with micro metal gearmotor and built-in rotary encoder

It’s just that my main project, the KR01 robot, is where I’ve been devoting most of my time and energy, and the KRZ01 isn’t frankly that different, despite being rather petite: about 1/4th the size and 6% of the weight (160g with its battery). Both are wheeled robots intended to operate using the Robot Operating System (ROS) I’m writing in Python.

So what made the KRZ01 the target of a redesign was the purchase of four Mecanum wheels. This post describes the beginning of this project — I’m only at the design stages right now.

What are Mecanum Wheels?

There’s a lot of descriptions (e.g., the YouTube video below) and demonstrations of Mecanum wheels on the web already (such as a Turkish Mecanum forklift!) so I won’t go into much detail here, suffice it to say that they allow a robot to travel in any direction without changing its compass heading. Well… not up or down. But crab travel, sure.

Just to mess with your head I’ll quote the description of how they work from the Wikipedia page on Mecanum wheels:

  • Running all four wheels in the same direction at the same speed will result in a forward/backward movement, as the longitudinal force vectors add up but the tranverse vectors cancel each other out;
  • Running (all at the same speed) both wheels on one side in one direction while the other side in the opposite direction, will result in a stationary rotation of the vehicle, as the transverse vectors cancel out but the longitudinal vectors couple to generate a torque around the central vertical axis of the vehicle;
  • Running (all at the same speed) the diagonal wheels in one direction while the other diagnoal in the opposite direction will result in a sideway movement, as the transverse vectors add up but the longitudinal vectors cancel out.

That kind of talk totally does my head in, but the concept obviously works, so I’m on board. When I get to the point of programming the motor controller for this I’m sure I’ll need a couple shots of good bourbon to focus my mind appropriately to the task.

The “Before” Photo (click to enlarge)

There are some design considerations regarding Mecanum wheels, and both David Anderson’s advice and my own experience with the KR01 suggest that I want the robot as balanced as possible, both in terms of weight and the position of the wheels relative to the center of the robot. With Mecanum wheels, weight distribution is even more critical than on a normal wheeled robot. Having too much weight on one wheel would significantly alter its behaviour, and not in a good way.

The Plans

So I have been planning this out. I can’t actually build anything just yet because I stupidly only bought two of the brass wheel hubs rather than four (“there were two in the photo” he says in his defense), so I’m waiting on another shipment from Canada.

Because it will be a fundamentally different robot the redesign will be called the KRZ02.

My plan for a Mecanum-equipped KRZ01 robot (click for larger view)

Update as of 2020-05-12: That first plan had a glaring error, in that while the centers of the wheels were on the circumference of a circle, that hardly meant they were equidistant, meaning their centers would fit on the four corners of a square. I’d drawn the diagram wrong. I tried a second time, this time also moving the motors as close towards the center of the robot as I dared, and including the positions of the Raspberry Pi Zero W, the two Picon Zero motor controllers, the Pimoroni Black Hat Hack3r expansion board, and the Breakout Garden Mini to hold one SPI and two I²C sensors . The robot got bigger but also more symmetrical. Witness version 2:

KRZ01 Mecanum Plan Plan 2

One thing seems pretty clear at the outset: the current KRZ01 has two “moon buggy” wheels and a ball caster, and its physical extent (i.e., how much space it takes up) is a circle 128mm in diameter — it’s a small robot. Plan 1 expands that to 210mm. Plan 2 above expands that extent to 227mm, almost double in size. It looks like it’d be a much larger robot than the KRZ01. Plan 1 used a 75mm chassis width, which is the current width of the KRZ01. Plan 2 has the motors as close together as seems reasonable but by fixing my design “bug” the robot is almost as big as David Anderson’s SR04 at 11″ (280mm). Not a small robot anymore.

I discussed the issue of symmetry with the guys at the DPRG and it seems that weight balance is critically more important than symmetry. I’m not happy with the Plan 2 being such a big robot, and from the plans there seems to be a fair bit of wasted space (i.e., it’s a lot longer than is strictly necessary) so I think I might try a third, shorter design.

More later…

From Failure, Success

KRZ-01 Robot

There’s been another mishap around here. I guess building robots has its ups and downs and last week was no different.

I’m kinda ashamed to say that while I was working on the KR01 robot I’ve now managed to burn out two Thunderborg motor controllers and one Ultraborg servo controller. Well, not quite “burn out”. The motor controller parts of the Thunderborgs still work but the RGB LED used to display the battery level has somehow gotten fried on both units, and the Ultraborg (which is used for sonar and servo control) seems to have died during the first Thunderborg catastrophe (sympathetic death). I have no idea really how this has happened, but of course the only real possibility is that I’ve done something wrong. I mean, I’ve been very careful with checking my wiring before applying the power, but at some point I must have got my wires crossed. The PiBorg folks who make these boards have been quite helpful and I’m sending them back to the UK to see if they can figure it out. But that will take awhile.

King Ghidorah anatomy by Shoji Phtomo
Not to be confused with Monster Zero 1

This means that for at least a few weeks I would be without a robot (the horror)! I really can’t have this happening just as I’m getting the robot operating system up and running. So last weekend I went ahead and built out one of my design prototypes, which I’ve been calling the KRZ-01 (Kiwi Robot Zero), as it’s based on a Raspberry Pi Zero W. It uses a Picon Zero for a motor controller, a Pimoroni Breakout Garden to mount some of its sensors, and a trio of infrared detectors rather than a front bumper.

Happily, the build posed only a few problems and I had it up and running rather quickly. I rewrote the Python modules that had been used to control the KR01’s motors to instead use the Picon Zero and I had it dancing around on the carpet today for the first time:

KRZ-01 Motor Control Demo

The KRZ01 is meant to be small and relatively cheap, but still have the ability to carry some impressive sensors. It actually isn’t a whole lot less capable than its larger sibling, the KR01. Without including shipping the parts come out around NZ$250, so it’s not the cheapest robot you could build but it’s got a lot of functionality2.

Side View of KRZ-01
Side View of KRZ-01

It’s based on a Raspberry Pi Zero W, which has 500MB of memory and supports both WiFi and Bluetooth. The OS is Raspbian Linux. The Picon Zero motor controller and a Breakout Garden Mini are both mounted on a Mini Black HAT Hack3r breakout board. This is an extremely compact setup. You can see this on the side view photo.

The sensors include: three Sharp infrared detectors; a VL53L1X Time of Flight (ToF) distance sensor mounted on a micro servo, which can measure distance up to about 4m with a 25mm accuracy (this is the same sensor I used on my night light); and two 298:1 ratio micro gear motors with encoders so we can measure how far we’ve travelled.

Bottom view of KRZ01
Bottom View Showing the Motors and Motor Encoders

There’s still two free I²C Breakout Garden sockets so additional sensors can be swapped in and out without any soldering. I added a couple of 11×7 LED Matrix boards as status displays but they’re hardly necessary. The whole thing runs on a common USB battery. The chassis is made out of 3mm Delrin plastic. For locomotion it uses a pair of Moon Buggy wheels, a lightweight plastic ball caster in the front, a heavier stainless ball in the back (so its balance is towards the back caster).

Since the robot supports WiFi I connect to it remotely using ssh, which is how I’ve been installing and loading its software, starting and stopping programs, and shutting it down 3. Remarkably, the Raspberry Pi W includes a tiny HDMI connector so I could plug it into a monitor, but that hardly seems necessary. This seems like a command line robot.

The chassis is 75mm wide and 120mm long. Without a battery the whole thing weighs 120 grams. For comparison, that’s 17 grams less than my iPhone 5. I have a 5200mAh battery that weighs 136 grams and a 4400mAh battery that only weighs 40 grams, so unless battery life is an issue I’ll probably use the smaller battery. I have a 10000mAh battery (200g) that would last many hours but I can’t imagine leaving the robot alone that long. What kind of trouble could it get into?

For more information about the KRZ01 Robot, visit its NZPRG wiki page.

Note: as of today the NZPRG has its own YouTube Channel.

Edit: after some back in forth in email and finally posting the boards back to PiBorg in the UK, I learned from them that what seemed to have happened was that the UltraBorg tested as faulty, and that was apparently what was burning out the LEDs on the ThunderBorgs. They’ve since sent me replacements for both and all is working well now. A well-deserved thank you to PiBorg for their patience and help!