In 2016, we at Carunda24 decided to start building a wrist-mounted gesture recognition system based wearable sensors and research in soft body dynamics (SBD). With the emerging trends in augmented and virtual reality, one of the biggest challenges we’ll face in society and tech in the coming years will be how to interact with immersive technologies. In the past we’ve followed a pattern of establishing some input methods to a computer (historically with a keyboard and mouse), and then modifying our behavior to the machine interface. There has also been a desire for Natural User Interaction (NUI), and the development of technologies which allow us to interact with computers in the ways we naturally interact with people and the world.
NUI Drone Development
While researching soft body dynamics and gesture recognition for various applications, it seemed for us a natural and interesting challenge to start with the NUI control of a drone. As part of the development journey we thought of which connected devices we would like to interact with, and flying a drone was one of our first inspirations. We were looking for something we could play with, and a small Parrot drone already almost feels like a pet bird or dog rather than a flying computer. So we set out to connect our wearable sensor to the Parrot Mini Cargo drone.
Wearable Sensor Technology
At the core of the smart strap is a Soft Condensed Matter Sensor (SCMS), essentially a conductive polymer composite material developed at Empa in Switzerland. The SCMS allows us to measure the strain on the skin continuously, so we can actually detect the moment of tendons as the hand creates different gestures. It’s based on a high tech combination of nano-textured conductive carbon particles mixed with a highly elastic thermoplastic elastomer. Research has shown the SCMS can measure strain related to hand gestures and is even sensitive enough to measure the pulse-wave of blood (Piezoresistive Soft Condensed Matter Sensor for Body-Mounted Vital Function Applications).
For us the Parrot drone was just the beginning, in the future we will be further developing our gesture recognition smart strap technology and also integrating with different robotic and AR/VR systems.