Songwei Ge, Vedanuj Goswami, Larry Zitnick, Devi Parikh

pasted image 0

Prateeks (2020). We call this piece Prateeks. Prateek means symbol in Sanskrit. Symbols — be they characters in a fictional native language of the Na’vi, the humanoid inhabitants of the moon Pandora in the movie Avatar (2009), or hieroglyphs in the ancient Egyptian writing system — only hold meaning to the extent that the interlocutors have shared context. This piece symbolizes a future where humans and machines share context and collaborate seamlessly towards enhanced creativity and expression. 

These are generations from a Generative Adversarial Network (StyleGAN2) trained on 10,000 creative sketches of creatures drawn by humans. People were shown a random initial stroke on a canvas on a web interface. They were asked to draw an eye on the canvas wherever they liked. They were then asked to step back, and imagine what a creature would look like that incorporates both the random initial stroke and the eye they placed — making imagination and creativity central to the sketching exercise. They were then asked to draw the creature they imagined, one part at a time.

The generations from the model are rasterized images represented as a grid of 64 x 64 pixels. They were converted to vector graphics. This gives them a visually appealing aesthetic, as well as an arbitrary resolution.

Overall, these generations are a consequence of 

  • human creativity (captured in the human-drawn creative sketches of creatures fed as training data to the model),
  • neural machine creativity (captured in the GAN generating novel samples that mimic the distribution but are not duplicates of the training data), and 
  • procedural machine creativity (captured in the aesthetic introduced by the conversion of the rasterized image to vector graphics). 

Style-mixing experiments demonstrate that we can systematically control the shape of the characters as well as the “writing styles”.