News: Emteqs Facial Tracking For VR Reads Your Muscle Impulses And More
10-27-2016 10:45 PM
As one of the top 20 universities in the UK, The University of Sussex offers undergraduate courses that could be pivotal to the future of VR. Its teachings stretch across neuro, cognitive and biomedical sciences, among other areas. We could well see a new generation of VR recruits come out of its campus. But Im not visiting the grounds to talk about that; Im here to talk about tech.
Housed away in a quiet corner of the campus is the Sussex Innovation Centre. Quieter still is the office that facial recognition company Emteq occupies inside that building. CEO Graeme Cox tells me that the small team has the Friday feeling as he closes the office door behind me, though its difficult to imagine the friendly engineers and developers that greet me being in any other frame of mind. For a company thats working in one of VRs most vital areas, Emteq makes for a remarkably calm environment.
Perhaps thats a reflection of the confidence the team has in faceteq, the platform its building for emotion tracking in VR headsets and other devices. Rather than using cameras to monitor and replicate your facial expressions, faceteq uses a sensor-based system that lines the area of a headset that touches your face. The system reads electrical muscle activity, heart-rate, skin response and other indicators, then translates them back into the relevant expression.
This is an important area of VR, especially as the industry begins to shift toward social experiences. Facebook was on-stage at Oculus Connect 3 earlier in the month to demonstrate cartoonish avatars with animated faces that try to replicate the users emotions and reactions. Its also an area in very early stages; an hour after that demo Oculus Chief Scientist Michael Abrash told the audience that perfect virtual humans, of which emotion tracking will play a big part in, were a long way out.
And Emteqs work is indeed early; the current iteration of the tech is just over a year old but shows plenty of promise, though a much improved version is right around the corner.
Understanding human context, emotional context, and physical context and how theyre formed [and having] VR systems, augmented reality systems, AI systems that can do it with the human is really important, Cox says. Moving through VR in AR and having that emotional response built into it, thats our mission.
Today the company has a rough demo to show me, both hands-on and off. The team is showing the system with the Oculus Rift DK2, though, behind me is Electrical Engineer Dr Mohsen Fatoorechis desk where hes currently piecing together a prototype for CV1 that will be completed very soon.
Once its finished it will just be a plug-in insert, Cox explains.
He also plans a Vive version that will be a simple lining that replaces the velcro-attached foam one that comes with the kit. The company is also looking into Daydream.
I first watch the faceteq-modified DK2 used with an avatar thats been pre-calibrated to Fatoorechis expressions. At first he sits in a basic environment. To the right a debug menu of sorts is giving realtime feedback on the sensors readings. To the left is a large set of facial expressions that the user selects to individually calibrate. The process involves performing a specific action such as blinking with the right or left eye three times over so that the system can memorise the feedback with machine learning.
Each expression takes about 30 seconds to calibrate, but the team is working to streamline the process in the future.
Fatoorechi is using winks to select options on a menu, an action that does away with the seconds spent staring at options when using gaze controls. In the first demonstration, he sits in front of a woman that mirrors his expressions. Its not the most refined showcase; every time the woman blinks her jagged face looks like shes in agony, but thats not the point of this demonstration at this stage. Though buggy, the system responds well to Fatoorechis reactions, at least as far as I can tell when much of his face is obscured by a headset.
The two other demos I see present some unique use cases for facial tracking that I hadnt yet thought of. One is a cinema screening room. Here Chief Scientific Officer Charles Nduka gets me to imagine a Hollywood preview screening where movie studios might gather feedback data from the sensors, which is one example of what Cox labels objective emotional analytics, and could also be used in research projects or game testing. Or you could watch content with your friends and see how they react. Imagine laughing together at a comedy or both jolting in shock at a jump scare.
The other demo is a little more unique. Emteq has created its own Facebook wall in which, just by looking at posts, you can assign emojis based on your expression. Facebook has been heading in the direction of more specific emotional responses for a while, and this system presents a more honest and immediate way of relaying that information, even if that might not always be ideal for maintaining friendships.
Following that demo, I get to try the tech for myself. Its important to note that I have long hair which provided a little interference in this early prototype. I first calibrate a right wink, which is a fairly smooth and pain-free process, though the system gave me a little trouble with a shocked expression with eyebrows raised, and I had to abandon calibrating an open-mouthed smile as it simply wouldnt register. Again, this was early prototype tech.
With what I had calibrated, I sampled the demo with the woman. Impressively, she quickly replicated my winks and the position of my eyebrows and forehead wrinkles to intimidate some expressions. The system was remembering the impulses I had given off making certain expressions during calibrating, and then producing the relevant emotion or action.
Ideally, faceteq would reach a stage where it recognizes impulses and replicates expressions on the fly without the calibration process. Cox points out that the companys method of tracking can pick up on minute muscle movements that camera tracking cant recognize, but if you need to calibrate expressions so small then where do you stop? What number of variations of muscle impulses and reactions is enough for a flawless virtual human?
Hopefully Emteq can refine the process enough that this isnt an issue, which is what its working toward.
Its a system built on data, Nduka says. The more data it gets, the more intelligent it becomes.
Im looking forward to seeing the system improve in the coming months. The group is also working on a form of eye-tracking using electrooculography, which, according to Cox, isnt a high-precision solution. It wont be enough to enable essential processes like foveated rendering, but will be able to read small and large movements and the general direction of your eyes. That could add another compelling layer to whats already here.
But Emteq is already making use of what it has now. The companys tech is being used in a treatment app for sufferers of a type of paralysis known as facial palsy. And faceteq can also be paired with glasses to give you key information in every day scenarios, like registering your weariness as you drive low on sleep.
Currently, the company is in process of raising seed money, though Cox says its pretty well funded from a research standpoint. For a possible consumer release of its VR tech, it sees a range of options, including the aforementioned inserts for headsets.
Weve got an exciting road ahead, Cox says. Theres still plenty to do, obviously, but were able to demonstrate some pretty interesting stuff already and by the time we get Q1 next year, were going to have a fully working system that can do some really quite surprising stuff.
Source: Emteq's Facial Tracking For VR Reads Your Muscle Impulses And More10-27-2016 06:15 PMLike 0
- Through the Lenses
- VR News & Discussion
News: Emteqs Facial Tracking For VR Reads Your Muscle Impulses And More
- By miyanc in forum Samsung Gear VRReplies: 6Last Post: 10-31-2016, 12:49 PM
- By VirtuaTyKing in forum VR News & DiscussionReplies: 2Last Post: 10-30-2016, 03:49 AM
- By VirtuaTyKing in forum VR News & DiscussionReplies: 1Last Post: 10-28-2016, 11:08 AM
- By VirtuaTyKing in forum VR News & DiscussionReplies: 1Last Post: 10-27-2016, 11:39 PM
- By VirtuaTyKing in forum VR GamesReplies: 1Last Post: 10-27-2016, 12:20 PM
LINK TO POST COPIED TO CLIPBOARD