Augmented reality snapchat filter

This augmented reality snapchat filter works by recognizing the mural painted on the wall (pictured left) and attaching a 3D animation to it. Once the mural is recognized and the animation is attached, the user can walk around the space and view the augmented mural from any angle.


Process: modeling and animation

The eyes were modeled and animated using Cinema4D, a popular platform for 3D modeling and animation. The first image in the gallery below shows how the eyelid shape was created. The second image shows an initial attempt at texturing. I wanted the eyes to be cohesive with the mural so it was important for the eye to have a striped texture. Adjusting existing textures wasn’t giving me the effect I wanted so I ended up painting my own custom texture. The third image shows what it looked like inside of Cinema4D when I animated the eyes. I placed all the eyes in the correct position and adjusted their sizes to match the mural. Then I individually animated each eye. The video pictured right shows a first attempt at animating an eye moving in coordination with the eyelids blinking. Unfortunately the blinking motion of the eye did not translate well into the snapchat filter software used and so it was not included in the final version of the filter.


process: developing the snapchat filter

Snapchat offers a free piece of software called Lens Studio that anyone can use to develop augmented reality experiences, this is what I used to create this augmented reality snapchat filter. The software lets you import your own artwork and then to decide how users can interact with it through their snapchat apps. In this case, I used Lens Studio’s image tracking functionality to track an image of the mural and to play my animation on top of that mural on a loop. The video to the left shows a first attempt at getting the image recognition to work inside of Lens Studio. Lens Studio has tools like webcam integration and connection to your smartphone and snapchat account that let you test your ideas and projects before you publish them.


Next steps

The snapchat filter is now published through the app and anyone with a snapchat account can find the filter through searching or through using the snapcode pictured right. While the filter is meant to work with a specific mural at the ATLAS Institute at University of Colorado Boulder, it will also work with photos of the mural such as the one pictured right.

Next, I would like to improve the animation and attempt to make the eye blinking mechanism work inside of Lens Studio. Eventually I would like to make filters for more of the murals inside of the ATLAS building.