Art and Computing

By Clarissa Littler / Tech in Real Life (IRL) / October 2021 Issue

Join us as we explore how computers and people can create art differently!

Can computers create art? It’s a question that’s been asked for as long as we’ve been making computers but one that’s become more intense as we’ve entered an age where computers seem to be crafting stories, creating realistic paintings, and making songs in the style of any musician who’s ever been recorded.

It’s a question that’s about the nature of computers and what intelligence and creativity are. It’s also a question that gets at the heart of what art even is.

To start, it’s kind of a funny question when you think about it: can a computer do anything? I mean, a computer is just a thing you run code on. It doesn’t make decisions or choices. So, instead, let’s reframe the question as can a person write a program that can automate creating art? By automate, we mean that once the program is started it produces creative works like what we can make without further human intervention.

The answer to that question, it comes down to can someone write a program that acts creatively.

Now we get to a really interesting question: what on Earth is creativity?

Thankfully, your humble author doesn’t need to come up with an answer to that all by themselves. There’s an entire field of research called computational creativity that’s about this question. In fact, much of the rest of this article is going to be borrowing ideas invented by Margaret Boden—one of the founders and giants of the field.

Dr. Boden is one of the people who has tried to break down the different kinds of creativity that exist. There’s combinatorial creativity, which is like remixing existing ideas in different ways than they were given to you, riffing off of things you’ve seen before. If you’ve ever come up with a game idea that’s it’s like ____ meets ____, then you’ve played with combinatorial creativity.

There’s exploratory creativity, that’s about trying things within a set of well-established rules of an art form. Exploratory creativity is when you’re doing something like writing a poem in a form with well-established rules.

Finally, there’s transformational creativity. This is where you make up new rules for the art form. This is a really hard kind of creativity, one that often involves major breakthroughs in a field: new forms, new techniques, radically new ideas. The first people to film movies as movies instead of as recorded plays were involved in transformational creativity. As a side note, transformational creativity frequently involves being able to look at something with very fresh eyes and to take apart the assumptions that go into what makes a form of art good or bad or interesting.

So of these three forms of creativity listed, the first two are ones that a computer can do reasonably well. Exploring a well-defined space is what a lot of machine learning techniques do. They’re trained on millions and millions of examples in order to copy the patterns of what they’ve learned, to make new objects that look like the ones they’ve already seen. Similarly, mashups also show up in research on machine learning and procedural generation. That’s the kind of creativity explored when you have a program that can take a song and play it in the style of a different musician or like when a program takes a picture and animates the mouth to be singing along with a song.

But transformational creativity is hard for computers. By definition, it requires a lot more intentional thought about restrictions and definitions of what an artform even means. I’m not saying we’ll never ever figure out how to automate this kind of creativity, but right now we have no idea how.

But this brings us to the punchline of this article: computers are good at certain kinds of creativity but you and I are good at a different kind altogether. I can come up with new rules and strange new ideas to explore, but what I can’t do is look at a hundred thousand paintings and replicate the style of them.

So rather than asking the question can computers create art? I think we should be asking the question what kind of art can people and computers create together?, and that’s what I call art in partnership.

Learn More

Clarissa Littler

Clarissa has worked in mathematics, physics, and computer science research but spends much of her time now trying to make computer science education accessible to a broader audience.

View all posts