Web XR Technology

The following are some of the Web XR technologies that have been applied to MetaBell:

Motion capture technology is a technology that records and processes body movement data of humans or other objects and generates motion animations of the corresponding virtual assets. Motion capture technology is used to make the animation look more realistic and natural, which greatly improves the quality of virtual 3D content. The main techniques used in MetaBell are body and face capture.

Body capture technology uses an inertial motion capture system to capture human body data from moving joints and other parts of the body to restore the body's movement by building a skeletal model of the human body. Facial capture mainly uses 3D camera to read facial dot patterns using structured light principle. The product combines the advantages and features of single camera, markerless optical facial expression capture technology and inertial motion capture technology. It uses markerless optical motion capture technology to capture real human facial expressions and inertial motion capture technology to capture real human movement information to build a realistic virtual character.

Facial capture technology only requires a single camera-based, markerless facial expression capture system to capture human data of real people's fine facial expressions to create stunning animations, supporting wireless data transmission and real-time preview of expression animations, capturing fine effects, with user-friendly software features designed. This technology is generally used in film and television special effects and AAA game production.

*Body data consists of Voxel. The stereo containing Voxel can be represented by stereo rendering or by extracting the polygonal equivalent surface of the threshold contour. One is the smallest unit of digital data on the 3D spatial partition. Conceptually similar to pixels, the smallest unit of two-dimensional space, pixels are used for image data in two-dimensional computer images, while Voxel is used in three-dimensional imaging, scientific data and medical imaging. Usually, these data follow certain rules such as one cut per millimeter, and usually have a certain number of image pixels.

Core Algorithm:

Local feature-based method. Local features are extracted on the basis of key points or local surfaces, including: key point detection, key point feature description, feature matching and other steps.

Our application is based on the face feature point method (face feature points defined according to the facial organs, such as the contour points of the eyes, nose, mouth, eyebrows and other parts) to automatically and accurately locate. We manually selected 25 key points from the face based on the features of the face structure, and then calculate the geodesic distance of these points, based on which a set of scale coefficients are defined as the features of the face.