Omote / Real-Time Face Tracking & Projection Mapping

 

by Nobumichi Asai, Hiroto Kuwahara and Paul Lacroix

Project Omote is a collaboration between Japanese director Nobumichi Asai, makeup artist Hiroto Kuwahara and French digital image engineer Paul Lacroix.

WHAT IS OMOTE?

 

OMOTE is a Japanese word for face or mask. Face is considered as a MIRROR that reflects human soul, a separation between Omote (exterior) and Ura (interior), and in Nogaku, Japan’s classical musical plays, performers use Omote masks to express multitude of dramatic emotions.

 

 

As we spent more time on the project, we became more aware of its similarities with Nogaku’s Omote masks, and explored further possibilities through integration of latest technology and classical Japanese art.

 

SPREADING JAPANESE BEAUTY ALL OVER THE WORLD

 

One of the intention and source of expression of Hiroto Kuwahara was to let the world feel original Japanese culture and make-up: “I wanted to express Japanese sense of beauty and philosophy. Not the classical one, but more the 80’s Japonesque movement, with artists such as Sayoko Yamaguchi".

There is also a strong influence of Science Fiction Movies, Manga and Animations.

 

WHAT TRIGGERED PROJECT OMOTE?

 

Initially, Omote was supposed to be a business project for a cosmetic company but the project never began. We felt the strong potential of the concept, so we decided to go forward as a personal project. After three months, we did the first tests and then realized how promising it was. Then we decided to invest more time in order to do something new, well-polished, that nobody has seen before.

 

TECHNOLOGY

 

The overall system was designed and developed by Paul Lacroix. It is technically very difficult to track and then project in real-time. Nobody has done it before to that level because the process is quite complex: capturing face markers with sensors, processing the marker data, estimating face position and orientation, rendering face CG model with animated texture upon, and finally send the image through the projector.

 

 

For detecting the markers, Paul decided to use the motion capture system OptiTrack. It provides a swift response. He also created original facial projection mapping software called Live Mapper. In order to reduce the latency and reach real-time, the whole face tracking and projection mapping processes have been integrated into one single program.

 

ART CONCEPT: FUSION OF BEAUTY, MAKE UP AND TECHNOLOGY

 

Face projection mapping allows expressing effects and gimmicks that are not expressible with physical make-up: moving, shining, changing…

But an important point for us was also to let the people to perceive the beauty and fashion of women behind the technology.

 

MAKE UP ART WORK

 

Creating too perfect make up looks unnatural and artificial; it does not look like human being at all. Practically, pictures of Yuka Sekimizu were taken from different angles then projected on the 3D model of her face. Then Hiroto Kuwahara retouched the elements of the digital make-up using Photoshop. The important was to keep as much information as possible from original pictures in order to make the result look realistic.

 

 

Technique of projection mapping consists in projecting colors (adding light). On the other other side, Make-up consists in painting colors (removing light). In that sense, they are perfectly opposite to each other, additive color mixture and subtractive color mixture.

 

NOBUMICHI ASAI

 

Graduated from Tohoku University, Department of Science. He belongs to WOW(www.w0w.co.jp) and works on music videos, CM, and projection mapping as a planner and producer and technical director. He is noted for combining his science background and artistic skills to create original and innovative works. He has also been active in mapping projects since creating Seikei 3D Projection Mapping, the first large-scale projection-mapping project in Japan. His work has been awarded many prizes and has been shown worldwide.

 

HIROTO KUWAHARA  (webpage link)

 

After graduating from the Harada Gen Make-up Academy, worked as a makeup artist at the Harada Gen Make-up Center. Produced color recipe for the “Custom Blend Foundation” (Japanese Shade) for Estée Lauder Companies, Inc. “PRESCRIPTIVE” artists. In 2003, formed an art unit "Kaya" to express Japanese aesthetics globally and began his career as an Art Director. His work expresses emotional nuances that one feels from the expression and skin tone of models, which cannot be put into words. Knowing his models by touch, Hiroto Kuwahara’s makeup is a way of producing a person’s spirit and the way of life. He continues to search ways to penetrate deep into the audience’s psyche. Hiroto creates beautiful world of uniquely Japanese spirituality.

 

PAUL LACROIX  (webpage link)

 

After graduating from University Paul Sabatier of Toulouse in France with a Master's Degree in Digital Image Engineering, he came to Japan where he has been living for the last 10 years. He works as a Technical Director and System Architect at Transit Digital Works Inc. Specialized in image processing, computer vision and real-time rendering, he brings his expertise to 3D movies (Ghost in the Shell 2.0), TV series (Tron: Uprising), projection mappings, interactive contents and also develops original systems (clothes AR-fitting, live stage 3D character). More recently, he created the Real-time Face Tracking and Projection Mapping software that has been used in "Omote" and "Face Hacking".

 

DETAILED CREDITS

 

PROJECT MEMBERS

NOBUMICHI ASAI (PRODUCER / DIRECTOR / TECHNICAL PRODUCER)

HIROTO KUWAHARA (ART DIRECTOR & MAKEUP)

PAUL LACROIX (TECHNICAL DIRECTOR / PROGRAMMER - TRANSIT DIGITAL WORKS)

JIN HASEGAWA (CG DESIGNER - SPADE)

TAKASHI ISHIBASHI (CG DESIGNER - SPADE)

 

SPECIAL THANKS

AYAKA MOTOYOSHI (PRODUCTION MANAGER - P.I.C.S.)

KOJI KOJIMA (PRODUCER)

AYA KUMAKURA (PRODUCTION MANAGER)

KAZUHIRO NAKAMURA (COLORIST - McRAY)

KENJI NAKAZONO (PHOTOGRAPHER - CREATIVE STUDIO WORKS)

KIMIHIRO MORIKAWA (PHOTOGRAPHER - SHOOTING & LIGHTING)

RHEA TOR’S INC.

DAN (STYLIST)

K.FURUMOTO (HAIR - &´S MANAGEMENT)

YUKA SEKIMIZU (MODEL - SATORU JAPAN)

SPICE (OPTITRACK MOTION CAPTURE SENSORS)

HIDEAKI TAKAHASHI (MUSIC - mjuc)

HIROSHI SATO (PROJECTOR INSTRUMENTS - AMSA)

Webpage Copyright (C) 2015-2016 by Hiroto Kuwahara and Paul Lacroix

Project Omote video is Copyright (C) 2014 by Nobumichi Asai, Hiroto Kuwahara and Paul Lacroix