Thanks to a few patent filings that have surfaced over the last two years, we’ve had an idea that Sony’s been tinkering with finger-tracking technology for a line of new VR motion controllers. It’s still unclear whether these controllers are intended to be a successor to PlayStation Move or part of the kit for the yet-unannounced PlayStation VR 2 or something else entirely, but at least now we can see them in action thanks to a new video from Sony researchers.
As spotted by UploadVR earlier this week, Kazuyuki Arimatsu and Hideki Mori with Sony Interactive Entertainment, the division of Sony behind the PlayStation’s development, recently published a research paper entitled “Evaluation of Machine Learning Techniques for Hand Pose Estimation on Handheld Device with Proximity Sensor.” In it, researchers detail how they’ve equipped a prototype for the controller with capacitive proximity sensors to determine how the player’s hands are positioned, information that is then replicated in realtime in virtual reality. The team developed training datasets by using an optical tracking system to record a variety of different hand poses from 12 people with varying hand sizes.
A demonstration video accompanying the paper shows their work in motion. You can see a researcher stacking blocks in virtual reality with the prototype controllers strapped to the back of his hands, which seem to function in a similar way to Valve’s Index Controllers (they even look like a pared-down version, too). The team also highlights that this technology could be used for “non-verbal communication” in the virtual world as well, though it doesn’t go into more detail. I imagine whatever sensors are responsible for the middle finger will get a lot of use. Just a hunch.
You can check out the full clip below, courtesy of UploadVR.
As shown in the video, researchers are still struggling to replicate certain hand poses, especially when the player’s fingers are further away from the surface of the controller. However, it still maintains its accuracy even if the player’s hands start sweating or if they are wearing rings, watches, or other jewelry.
However Sony ends up deploying these controllers, don’t expect them to roll out with the PlayStation 5’s launch this winter. Sony’s already confirmed the PS5 will be compatible with existing PSVR tech, and, as I mentioned before, a successor hasn’t even been officially announced yet.
As for what we do know: Last year, the company licensed advanced haptics tech for VR controllers that’s capable of simulating “sensations of pushing, pulling, grasping, and pulsing,” per the tech’s patent holder, Immersion Corp. Sony’s also publically announced that it’s currently researching next-generation VR headsets, likely to be paired with these new finger-tracking controllers.
