Researchers from The Wharton School conducted a study aiming to enhance the creative potential of an artificial intelligence model known as GPT-4 during brainstorming sessions. They found that while GPT-4 was adept at generating ideas, the produced ideas appeared to have minimal diversity, making them similar to already existing concepts instead of presenting unique, out-of-the-box ideas.
The research team hypothesized that the use of innovative prompting strategies could enhance GPT-4’s creative potential. To test this, they designed an exercise where GPT-4 was assigned to generate ideas for a new consumer product, meant for college students and priced under $50. The generated ideas from the AI were then compared with the ideas provided by groups of MBA students for diversity and novelty.
Through varying prompts like minimal prompting, asking GPT-4 to imitate different personas, employing creative thinking methods from current literature, and utilizing Chain of Thought (CoT) techniques, researchers discovered that the use of longer prompts resulted in greater diversity in the ideas that GPT-4 produced. Among all the strategies, the implementation of CoT yielded the most diverse ideas, coming close to the level of diversity exhibited by the ideas generated by human subjects.
The diversity of the ideas was evaluated using a Cosine similarity metric, a text analysis tool that measures the similarity of concepts. It was found that GPT-4, without special prompting, generated ideas less diverse than their human counterparts. However, with the use of more complex prompts, the diversity increased.
One interesting observation was that when GPT-4 was prompted to take on different personas, the results fluctuated with no notable pattern. For example, prompting the AI to emulate Steve Jobs resulted in more diverse ideas than doing the same with Elon Musk.
Considering these findings, it seems evident that if AI tools like GPT-4 are to be employed in brainstorming exercises, using longer prompts and CoT techniques, as well as prompting them to take on different personas can yield the most diverse results. However, even with these improvements, a group of MBA students outperformed GPT-4, suggesting that while AI can aid idea generation, the human imagination still holds the upper hand.
