Mark Zuckerberg’s metaverse push appears to be taking a different and this time it seems for the better. After spending nearly $40 billion in the metaverse, Meta’s efforts to onboard its massive user base into the 3D world appeared to have failed after Reality Labs, the division driving its metaverse campaign recorded $13.72 billion in losses last year.
All this coupled with mockery of its cartoonish VR avatars and the perception that the company was changing its focus to AI, seemed to spell an end to Zuckerberg’s metaverse dream.
However, following last week’s unveiling of a new tech stack, which Zuckerberg hopes will re-ignite his company’s metaverse push, the Meta CEO also demonstrated the company’s latest version of VR avatars, which contrary to their predecessors, wowed former critics.
Zuckerberg unveiled his new photorealistic VR avatars in an interview with podcaster Lex Fridman, with their avatars appearing almost like real copies of themselves.
According to the Meta co-founder, the avatars are part of a research program and required both Fridman and Zuckerberg to sit in a studio for hours having their faces scanned using multiple cameras while making different facial expressions.
Those motion movements were then captured to build motion-controllable avatar heads. So instead of transmitting real-time 3D images of Fridman and Zuckerberg to the 3D world, the data is transferred and used to reconstruct their faces, which according to Zuckerberg is more efficient from a computational perspective.
“In addition to being photo-realistic, it’s also much more bandwidth efficient than transmitting a full video or 3D immersive video of a scene like this,” he said.
This movement data, plus audio from the headset’s microphones, comprises a compact data stream that can be sent cross-country with near-zero latency, to drive a copy of each person’s avatar pre-downloaded to their conversation partner’s headset, Zuckerberg explained.
“Most communication, when people are speaking, is not actually the words, it’s their expressions. There’s a certain realism that comes with delivering this photo-realistic experience that is really magical. This gets to the core of what the vision around virtual and augmented reality is, delivering a sense of presence as if you’re there together,” added Zuckerberg.
Although the metaverse is widely viewed as an entertainment-oriented innovation, some companies like Accenture see it more as a platform where multinational corporations can hold virtual meetings in 3D avatars.
And Zuckerberg thinks his photorealistic avatar technology can be used to facilitate such conversations.
“In the future, once you have mixed reality and augmented reality, we could have codec avatars like this and go into a meeting and have some people physically there and have some people show up in this photo-realistic form, superimposed on the physical environment. Stuff like that is going to be super powerful.”
