Smart thumb sensor provides cost-effective accuracy

Using computer vision and a deep neural network to infer haptic contact information inside a thumb-shaped sensor (see figure), researchers at the Max Planck Institute for Intelligent Systems (MPI-IS; Stuttgart, Germany) are adding a desirable layer of sensitivity to robots.

Known as Insight, the sensor is constructed around an internal monocular camera, with only a single layer of elastomer over-molded onto a stiff frame to guarantee sensitivity, robustness, and soft contact. This design allows it to continually provide a directional force-distribution map over its entire conical sensing surface.

Insight is also the first system to combine photometric stereo and structured light using a collimator to detect 3D deformation of its flexible and replaceable skin. It generates a 3D force map when the thumb is touched, which estimates where objects meet the sensor and the amount of force applied.

Much like biological systems, robots need touch-sensitive skins to provide practical benefits. Previous touch sensors are mostly in the concept-designing phases, such as creating functional materials and testing flat-sheet-shaped sensors. These designs only offer small sensing areas, are delicate and hard to fabricate, and cannot often feel contact parallel to the skin.

“In our research, we invented a soft and durable sensor shaped like a 3D cone with an extremely large contact surface; it can feel force distributions of multiple contacts with different force magnitudes and directions,” says primary researcher Huanbo Sun. “The research could significantly break the bottleneck of current haptic perception in robotic applications.” Insight is surprisingly straightforward to manufacture and quite modularized, explains Sun. Plus, all the components are low cost with online accessibility or could be easily 3D-printed. The sensor is also robust and durable for long-term use—two years so far without visible performance drop.

The team also designed the sensor with the imaging and sensing surface as separate systems. “If the sensing surface is damaged by collision and extremely high force impact, it’s effortless to replace it by unscrewing the old one at the cost of around 2 to 5 Euro,” he says. “And the sensor is data-driven in its current state. We’ve found that our machine-learning algorithm is very general and robust to use.” Flexibility is key as this technology develops. While Insight is designed to be very sensitive, robust, and high resolution, its shape can be changed; the sensor size can be rescaled; the data processing method can be quickly adapted or modified according to application requirements; and the sensor can also work using a raw camera image. “We hope every robotic lab can access our sensor at a meager cost with our open-access research philosophy,” says Sun. Of course, like human skin, multiple types of cells (mechanoreceptors) are responsible for contact with different speeds and temperatures. The current sensor cannot measure temperature and runs at 10 frames per second (fps), but Sun is currently working on improving the speed and integrating temperature-sensing modularity.

“Different components are involved in our sensor design, such as materials, cameras, lighting systems, 3D printing techniques, data collection procedures, data processing methods (machine-learning algorithms),” says Sun. “We’re exploring ways to optimize the design from the above-listed aspects and integrate more functionalities in our sensor.”