A team from University of Montana took responses from ChatGPT (with GPT-4 engine) and a control group of 24 University of Montana students that they gave to Torrence Test of Creative Thinking. Then, they were compared with 2700 college students who took the test in 2016.
The scoring was made by Scholastic Testing Service which didn't know AI was involved in a portion of the responses and AI wasn't able to cheat as it didn't have previous access to information about the TTCT test.
The analysis was made in three fields: fluency (ability to generate large volumes of ideas), originality (ability to come up with different ideas) and flexibility (the ability to generate different types and categories of ideas).
The researchers were expecting ChatGPT to score in the top percent in fluency as its job was to generate large volumes of ideas. However, when the ChatGPT scored in the top 1% in originality, that was a novel and surprising result.
“For ChatGPT and GPT-4, we showed for the first time that it performs in the top 1% for originality,” the lead-researcher said. “That was new.”
Even, ChatGPT outscored majority of university students that were subjected to the test.
The researchers showed their work in May 2023 at the Southern Oregon University Creativity Conference.
“At the conference, we learned of previous research on GPT-3 that was done a year ago,” the lead-researcher said. “At that time, ChatGPT did not score as well as humans on tasks that involved original thinking. Now with the more advanced GPT-4, it’s in the top 1% of all human responses.”
These findings raises the question of whether AI can surpass human qualities some day as creativity is a hallmark of human capability and AI now seems to have taken a huge step into approaching this.
Resource: https://scitechdaily.com/chatgpt-tests-into-top-1-for-original-creative-thinking/#google_vignette
Edited by: İdil Ada Aydos
Comments