The joy of a hack day is the time constraint, the 24-hour crunch that forces coders to get creative and make hard decisions about how much they can really accomplish. At Aviary’s photo hack day yesterday, a very well-attended event that produced over 40 hacks and oodles of prizes, the most used API was from Face.com, an Israeli start-up working on facial recognition.
Connecting to their API allowed a hack to identify Facebook friends and even chart their faces across five emotions (happy, sad, angry surprised, neutral), adding emotional and social depth to projects built on short notice.
The winner of the event was Photobot, a service that analyzed users pictures and gave the suggestions for how to improve. Honey Badger, which took second place, used the Face.com API. But the most interesting projects to come out of the event were Emotional Breakdown and Facialytics, which the judges passed over because they were suspicious something so robust could have been built in 24 hours.
Emotional Breakdown used the face.com API to scan through news feeds from sites like The Guardian and break down the emotions in photographs. While the overall paper was a mix of all emotions for example, looking at coverage from of the London riots showed most people were angry and sad, with almost no one happy. While that example seems pretty obvious, the technology could certainly be used to find some interesting patterns if used at scale and over time.
Facialytics had clearest path to a real business. The team took infrared footage of audiences watching films (mostly grabbed off YouTube) and spliced them up into single frames. Then they ran that through the face.com API to get an emotional register. “We found, surprisingly, the API was actually more effective at capturing emotion in infrared,” said Jonathan Bensamoun, a senior associate at PricewaterhouseCoopers and one of the hack’s four creators.
“It was incredible how quickly Face.com could process the images and that they could capture emotion for every face in the crowd,” added Mr. Bensamoun. Facialytics was then able to chart the average mood of the audience throughout the film and find high points for pleasure, surprise or sadness. “I was thinking it would be tough to get cameras installed in every theater to do this,” he went on to say. “But someone told me that most movie theaters already have infrared cameras to watch the audience for piracy or lewd behavior.” It’s easy to see how the big film studios would pay for analytics that showed exactly what parts of their films were resonating with the audience.
Gil Hirsch, Face.com CEO, was visiting from Tel Aviv. “We were just blown away by the incredible uses people found for our API. What’s going on in New York right now is very impressive.” Face.com isn’t looking for revenue right now, said Mr. Hirsch, but rather trying to get their technology in front of as many people as possible. “We saw a bunch of projects tonight that I think have the potential to be big.”