We all have a soft spot for Prisma, the app that turns smartphone photos into stylized artwork. But the reverse process — transforming artwork into pictures — is no less fascinating. And it’s not far from becoming real, researchers in the Netherlands said.
A team of four neuroscientists at Radboud University is working on a model for inverting face sketches to synthesize photorealistic face images by using deep neural networks. The results of the study (Convolutional Sketch Inversion) were first made available in the online archive arXiv and have recently been accepted at the European Conference on Computer Vision in Amsterdam.
Scientists said applications of their model could include fine arts, for turning self-portraits into something more akin to a photo, but also in forensics, for turning sketches based on eyewitness accounts into something a photo-recognition tool could use, for example.
“We were inspired by the recent work on neural style transfer, an algorithm to reimagine photos in the style of artworks,” Yağmur Güçlütürk, 29 and Umut Güçlü, 30, the two PhD students in cognitive neuroscience who developed the study together with Marcel van Gerven and Rob van Lier, wrote in an email to TechCrunch.
The article Yağmur and Umut referred to described a technique to reimagine a photo of the city of Tübingen, Germany, in the style of The Starry Night, the oil on canvas painting by Vincent van Gogh. “This example got us thinking about its inverse problem. That is, what the artworks of Vincent van Gogh would look like as photos,” Güçlütürk and Güçlü wrote.
Here is how Güçlütürk and Güçlü describe how their software, which uses an artificial neural network, works:
“Let’s say that I, scientist, want to teach you, artificial neural network, how to convert sketches (inputs) to photos (outputs).
