Lithuanian biometrics firm Neurotechnology, a provider of high-precision biometric, robotics, AI and computer vision technologies, has introduced the SentiMask Software Development Kit (SDK) for real-time face tracking and masking, 3D digital character control and other applications. The SentiMask algorithm can detect and track the user’s face, in real time from a regular video stream, such as a webcam, or in a video file, and it does not require depth sensors or any other special hardware.
The output from SentiMask is in the form of 2D and 3D coordinates that represent certain facial feature points, also known as landmarks. These landmarks can be used to construct a face mesh and various textures can be applied onto that mesh. When used as an animated avatar or mask, for example, the mesh and textures respond to – and move in concert with – the user’s facial expressions.
“For many years Neurotechnology has been known as one of the top companies in the person identification market for its high quality, precision biometric algorithms,” said Dr. Vilius Matiukas, SentiMask project lead for Neurotechnology. “Our decades of experience in biometric facial recognition has enabled us to create our new SentiMask SDK product, which uses our latest facial image processing technologies in a non-biometric way. With SentiMask, our customers can easily incorporate face detection and tracking capabilities into their own solutions, such as entertainment or interactive marketing applications,” Matiukas added.
With SentiMask SDK, users can not only map textures onto a person’s face but also control digital characters’ facial expressions. Using 3D facial landmark estimates, the SentiMask SDK can be used to control facial bones in 3D modeling applications and even estimate certain facial expressions and convert them into events. Such events can be used to animate digital avatars as well, using blend shapes (or shape keys in some applications like Blender 3D). SentiMask SDK works with regular hardware, such as webcams, and no depth sensors or special cameras are required. It can be integrated with 3D modeling software and 3D game engines and provides easy integration for custom projects.
SentiMask SDK features include: real-time face detection and tracking; 2D & 3D facial landmark estimation; 3D facial pose, shape and expression estimation; 3D facial mesh output, and; facial expression analysis.
SentiMask SDK supports development environments under Windows and Linux PC platforms and the mobile Android platform.