This was a fun project to work on and had many facets.
First, we needed to create high-quality meshes for a multitude of expressions. For each expression, there are two matte images for the left and right expression (not all expressions happen on both sides of the face) After that we created a tracking target for Molly our test subject. This required capturing multiple expressions in various lighting. Smart body was our next adventure, and after meeting with Ari Shapiro we knew that Smart body would be able to handle this project. After learning a bit of the API, we were able to attach all 10 (20 Left and Right sides) morph targets to our mesh. We turned on smartbody’s built-in facetrack port, and there we had our model being tracked in realtime.