Image-To-Image Demo

Set of interactive browser-based machine learning experiments put together by Christopher Hesse turns doodles into images with various neural network trained image data. For example, in ‘edges2cats’, your simple doodle will be recreated with image data from a dataset of cat pictures:

Recently, I made a Tensorflow port of pix2pix by Isola et al., covered in the article Image-to-Image Translation in Tensorflow. I’ve taken a few pre-trained models and made an interactive web thing for trying them out. Chrome is recommended.The pix2pix model works by training on pairs of images such as building facade labels to building facades, and then attempts to generate the corresponding output image from any input image you give it. The idea is straight from the pix2pix paper, which is a good read. …

[On edges2cats:] Trained on about 2k stock cat photos and edges automatically generated from those photos. Generates cat-colored objects, some with nightmare faces. The best one I’ve seen yet was a cat-beholder.Some of the pictures look especially creepy, I think because it’s easier to notice when an animal looks wrong, especially around the eyes. The auto-detected edges are not very good and in many cases didn’t detect the cat’s eyes, making it a bit worse for training the image translation model. 

You can also try doodles in datasets comprising of shoes, handbags, and building facades.

You can play around with the neural doodle experiments here