Deep Learning for Design

pix2code: a design mockup translated directly into working markup.

Recently there have been some exciting developments in machine learning to advance the practice of design in multiple fields:

• Pix2Pix
• pix2code
• Sketching Interfaces
• Sketch to Code in Keras
• Logo Synthesis
• Building Information Design Synthesis

What is new and exciting about these deep learning methods, as compared to methods of generative design in the past, is that they enable automated translation between multiple representations. Today, translating between representations can be very difficult and time-intensive. For instance, engineering a custom website requires that a domain expert in web development understand a design specification and then write code in multiple languages to describe complex interactions, data synchronization, and appearance. These early examples show promise for encoding a target representation as a series of patterns and decoding these complex functions from simple and fungible representations.

Turning design mockups into code#

The interactions afforded by these new methods will enable new possibilities for designers of the built environment. Conditional GAN (Generative Adversarial Network) systems show that constraints may be respected in the decoding of a trained neural network. The implication of this is that a site may be imported as an initial context with a few high-level requirements, and an entire building model would be rendered to respect the boundary conditions and goals described by the user.

The field of generative design that has been maturing over decades will experience a major impact as parametric design and optimization tools become more accessible to non-experts.