By Ankita Chakravarti: A new study revealed that ChatGPT, an AI language model, can solve problems as well as or even better than college students. The researchers tested GPT-3, the current model of ChatGPT, on reasoning problems found in intelligence tests and exams like the SAT, the Guardian reports. Notably, SAT is a standardized test that is used for college admissions in the United States. It is a multiple-choice, pencil-and-paper test that measures a student’s reading, writing, and math skills. The SAT is scored on a scale of 400 to 1600, with an average score of 1060.
There are many different universities that accept SAT scores. Some of the most selective universities, such as Harvard, Yale, and Princeton, require SAT scores for admission. However, there are also many other universities that accept SAT scores, even if they are not as selective.
As per The Guardian, to check its abilities, the researchers asked GPT-3 to predict the next image in a complex shape arrangement, which they turned into text so the model could understand. The researchers also ensured that GPT-3 had never seen these questions before.
Surprisingly, GPT-3 performed impressively well, solving 80 per cent of the problems correctly. This outperformed the average score of human participants, who were 40 college students from UCLA.
Additionally, the researchers challenged the AI model with some SAT analogy questions, where it had to link pairs of words. They believed these questions were not available on the internet, making it a tougher test. Despite that, GPT-3 still did better than the average score of human college applicants.
However, there were areas where GPT-3 struggled. One test asked GPT-3 and student volunteers to match a passage of prose with a different short story conveying the same meaning. GPT-3 did not perform as well as the students, but its improved version, GPT-4, showed better results.
According to the study’s lead author, Taylor Webb, it was made clear that ChatGPT’s AI did not reach the level of human-level intelligence or artificial general intelligence.It faced difficulties with social interactions, math reasoning, and problems that required understanding physical space.
While GPT-3 demonstrated a strong capacity for recognizing patterns and making inferences, the researchers couldn’t fully understand how its reasoning abilities worked. They couldn’t determine if GPT-3 was thinking like a human or showcasing a new form of intelligence.
Keith Holyoak, a psychology professor at UCLA, added that GPT-3’s training method was different from how humans learn, making its reasoning process intriguing. They aim to uncover if it represents a real form of artificial intelligence, which would be a groundbreaking discovery.
Overall, the study highlights GPT-3’s exceptional problem-solving abilities, but it also emphasizes the need to explore its inner workings to better understand its unique reasoning capabilities.