Say Cheese

NYU ITP Student Builds a Camera That Prints Descriptions Instead of Photos

The "Descriptive Camera" uses Amazon's Mechanical Turk service to describe a scene instead of capturing a photo.
 NYU ITP Student Builds a Camera That Prints Descriptions Instead of Photos

Mr. Richardson (itp.nyu.edu)

The Descriptive Camera hit the front page of Reddit’s r/technology subreddit today, and for good reason. The project, built by NYU ITP student Matt Richardson, uses the Amazon Mechanical Turk Service to turn a photo into a textual description of the captured scene.

“I’ve been thinking a lot about how cameras store a lot of metadata about a photo,” said Mr. Richardson by phone. “It’s limited to longitude, latitude, the time of day, the settings, and make and model of the camera. I thought, ‘Hey, wouldn’t it be neat if it also recorded searchable text about what is going on in the picture?’ We have some methods like that: tagging faces algorithmically and finding out who’s who in the picture, but I wanted something more than that.”

If you’re unfamiliar with Amazon’s Mechanical Turk service, it works like this: the service takes tasks that are still too difficult for computers to accomplish and farms them out to willing workers. The Descriptive Camera is a USB web cam hooked up to a thermal printer, as well as an electrical board that runs software Mr. Richardson developed. The camera snaps a photo and sends it off to the Turk service, where it takes three to six minutes for users to drum up a description of the photo, which is then printed out on paper.

Mr. Richardson said that computers are very close to completing these tasks themselves, but for now the Descriptive Camera has to rely on Amazon’s army of workers.

“People think it’s neat and interesting, but there’s a lot of confusion, with people saying, ‘Why would you ever want something like this?'” said Mr. Richardson. “It’s a question I never ask when I make something. I make what I want to make and I don’t ask why.”

Mr. Richardson is no stranger to viral tech projects. He’s a contributor to Make Magazine, where he develops DIY technology products. One such project is called “Enough Already,” which hooks up to your TV and monitors closed captioning for specific words. When it catches terms like “Kardashian,” it does you a huge solid and automatically mutes your TV.

Mr. Richardson is only in his first year at ITP, and intends to continue work on the Descriptive Camera, particularly focusing on making it wireless. “It’s not really a camera if you can’t walk around with it and take pictures,” he added.

Follow Jessica Roy on Twitter or via RSS. jroy@observer.com