OpenAI’s GPT-4 Scores in the Top 1% of Creative Thinking On Benchmark

Of all the forms of human intellect that one might expect artificial intelligence to emulate, few people would likely place creativity at the top of their list. Creativity is wonderfully mysterious-and frustratingly fleeting. The use of AI for creative endeavors is now growing.

New AI tools like DALL-E and Midjourney are increasingly part of creative production, and some have started to win awards for their creative output.

The growing impact is both social and economic-as just one example, the potential of AI to generate new, creative content is a defining flashpoint behind the Hollywood writers strike.

A Blend of Novelty and Utility When people are at their most creative, they’re responding to a need, goal, or problem by generating something new-a product or solution that didn’t previously exist.

Quite often, the result of creative thinking is also surprising, leading to something the creator did not-and perhaps could not-foresee.

As a researcher of creative thinking, I immediately noticed something interesting about the content generated by the latest versions of AI, including GPT-4.

Putting AI to the Test With researchers in creativity and entrepreneurship Christian Byrge and Christian Gilde, I decided to put AI’s creative abilities to the test by having it take the Torrance Tests of Creative Thinking, or TTCT. The TTCT prompts the test-taker to engage in the kinds of creativity required for real-life tasks: asking questions, how to be more resourceful or efficient, guessing cause and effect, or improving a product.

The tests are not designed to measure historical creativity, which is what some researchers use to describe the transformative brilliance of figures like Mozart and Einstein.

Rather, it assesses the general creative abilities of individuals, often referred to as psychological or personal creativity. Other researchers are arriving at similar conclusions in their research of AI and creativity.

Yes, Creativity Can Be Evaluated The emerging creative ability of AI is surprising for a number of reasons. For one, many outside of the research community continue to believe that creativity cannot be defined, let alone scored.

Creative work has been defined and scored in fields like psychology since at least the 1950s.

The “Person, product, process, and press” model of creativity, which researcher Mel Rhodes introduced in 1961, was an attempt to categorize the myriad ways in which creativity had been understood and evaluated until that point.

The understanding of creativity has only grown. Still others are surprised that the term “Creativity” might be applied to nonhuman entities like computers.

On this point, we tend to agree with cognitive scientist Margaret Boden, who has argued that the question of whether the term creativity should be applied to AI is a philosophical rather than scientific question.

AI’s Founders Foresaw Its Creative Abilities It’s worth noting that we studied only the output of AI in our research.

We didn’t study its creative process, which is likely very different from human thinking processes, or the environment in which the ideas were generated.

Had we defined creativity as requiring a human person, then we would have had to conclude, by definition, that AI cannot possibly be creative. Regardless of the debate over definitions of creativity and the creative process, the products generated by the latest versions of AI are novel and useful.

We believe this satisfies the definition of creativity that is now dominant in the fields of psychology and science.

The creative abilities of AI’s current iterations are not entirely unexpected. In their now famous proposal for the 1956 Dartmouth Summer Research Project on Artificial Intelligence, the founders of AI highlighted their desire to simulate “Every aspect of learning or any other feature of intelligence”-including creativity.

In this same proposal, computer scientist Nathaniel Rochester revealed his motivation: “How can I make a machine which will exhibit originality in its solution of problems?” Apparently, AI’s founders believed that creativity, including the originality of ideas, was among the specific forms of human intelligence that machines could emulate.

The surprising creativity scores of GPT-4 and other AI models highlight a more pressing concern: Within US schools, very few official programs and curricula have been implemented to date that specifically target human creativity and cultivate its development.

In this sense, the creative abilities now realized by AI may provide a “Sputnik moment” for educators and others interested in furthering human creative abilities, including those who see creativity as an essential condition of individual, social, and economic growth.