Nikhil Murty

Background

I have always been interested in how our hands work. It is insane to me that we can do anything from working in a construction site to performing brain surgery, all with the same hands. This interest is actually why I majored in Biomedical Engineering, I wanted to learn how the human body works and how we can use that understanding to improve people's lives. During my time at Johns Hopkins, I worked a lot on devices to understand how our hands felt things as well as how to improve hand function. I have posted some of these in my projects section. Unfortunately, there wasn't much of a career in prosthetics when I graduated, so I started working in my second field of interest: surgical robotics.

However, with the dawn of humanoid robotics and AI recently, I started thinking about robotic hands again. It seems like every one of these humanoid robots has some very dextrous hands and I decided to learn how they work by trying to make one of my own (to a much simpler degree of course). As for controlling it, I decided I wanted to map my own hand gestures to control the robot hand.

DISCLAIMER: This project is still a work in progress, but I have made some decent steps forward so I thought I'd share it here. I will continue to update this as I make more progress!

---

The Hand

I decided to go with a tension driven mechanism for the fingers because it seemed cheaper and less complex. The downside is that I won't have a full range of motion for the fingers but at least they will be able to grab things. It would be cool to have individual motors for each segment, but that would be incredibly expensive. It would be fun to try though...

V1

V1 started with the idea of linking the finger segments together using a snap-fit interlocking pin joint, similar to how an IGUS e-chain carrier works for its linkages. This was designed for an easy build since it didn't require any additional screws, pins, or fasteners.

Unfortunately, this snap fit joint introduced a lot of friction to the system. The motor could barely move it and motion wasn't as smooth as I hoped. Also, pressing the linkages together sometimes led to the plastic cracking and/or snapping. The prints also would sometimes come out a little warped, so the mating features would not fit together well, leading to the joints being weak and harder to move.

V2

I went back to the drawing board for V2, purchasing some small bearings and pins for this iteration. Although working with small bearings can be tedious, a pack of 10 was inexpensive on Amazon. I replaced the snap-fit joint with holes designed to slot a 5mm dowel pin through and made cutouts to press-fit a small 8mm OD bearing.

I also decided to add hardstops to this version, which were placed at each linkage point. This would allow the fingers to go back to their straight position when we add the spring tensioner. Speaking of which, I added two M3 through holes at the tips of the finger. One hole was for the driving mechanism, which would allow for the tendon to loop around it. When the tendon was pulled, this would allow for the finger to close. The other hole was another M3 through hole but this time, it was meant for the spring + tendon mechanism that would pull the fingers back to the open finger position. These all then attached to a mount with a Dynamixel XL330 servo with a spool attached to it.

Almost immediately, the motion on this version felt smoother. Pressing the small bearings into their receptacles was a pain, but once I got the hang of it, it got better.

---

The Controller

I originally wanted to use a Myo armband that I had lying around to control the robot hand. In college, there was a lab where I got to control a virtual hand using the Myo and it was really cool! However, the Myo I owned had been collecting dust for too long, and when I tried to boot it up, the battery was dead and I was missing the USB dongle to wirelessly control it. To make things worse, I tried to take it apart as a final Hail Mary to see if I could replace the battery, but I ended up breaking the communication cable. Since the company has been defunct for some time, that was the end of my Myo band :(

However, this led me to the idea of using gesture controls with a video camera. I've worked with OpenCV a bunch and I've seen how good hand models are at estimating hand and finger positions, so I thought it would be cool to use the camera to control the hand. I decided to go with Google's MediaPipe for the hand tracking because it was fairly lightweight and very responsive (around 15ms per frame).