María Tajadura I Jaén, (EFE).- A group of teachers from the ‘Virgen del Carmen de Jaén’ Secondary School (IES) have tested the capacity of Artificial Intelligence ChatGPT by submitting it to the Evaluation for Access to Information University (EVAU), in which it has obtained an average score of 8.36 out of 14.
Juan de Dios Marín, professor of Marketing at the IES ‘Virgen del Carmen’ and promoter of the study, explained to EFE that this idea arose from the democratization of the use of this artificial intelligence, to verify its usefulness and the sources from which it is used. to elaborate a piece of information before the students begin to experiment with it on their own.
The group of professors decided to carry out this research to evaluate the ability of this artificial intelligence to solve EVAU university entrance exams.
The ChatGPT has had to answer real and close questions in time and in the presence of the teachers who participated in the court of the last EVAU on six different subjects: English, Applied Mathematics, Spanish Language, History of Spain, Business Economics and Philosophy.
Good at letters, bad at math
With the cut-off grade obtained, an 8.36 out of 14, you could access careers such as Physics, Chemistry, Biology, Law or Business Administration and Management (EADE).
Professor Marín explained that the results obtained have followed the evaluation criteria of the 2022 EVAU exams of the University of Jaén, to which the AI did not have access, “since ChatGPT is trained until 2021”.
The ChatGPT has passed the EVAU, “with its lights and its shadows”, with a score of 9 in English, “which indicates that it has a solid knowledge of the grammar and syntax of the language”, the teachers say.
However, in the other subjects, the scores were much lower, with a grade of 2.5 in Applied Mathematics; 2.75 in Chemistry; 6 in Spanish Language; 4 in History of Spain; 5.5 in Business Economics; and 5.5 in Philosophy, while in Biology it has received 8.8.
Marín considers that “as the chat is designed to develop a conversation with a person, the grades have been better in matters of letters”, but on the other hand, “it is not prepared to make graphic representations, which is why it has failed Mathematics.
An AI lacking depth
Despite the fact that ChatGPT demonstrated a good understanding of grammar and syntax, the teachers who have participated in the correction concluded that the AI responses were superficial, lacking in depth and without citing their sources.
ChatGPT has not used technical language and has treated the topics in a very general way, without showing critical thinking or personal opinion, in some cases, even seemed to fabricate results to make their answers more coherent.
“It is important to highlight that ChatGPT has only been provided with the questions, without any type of context or additional information, this may have influenced its ability to answer in depth and, therefore, its results” has pointed out Marín, who He believes that if he had been given a context, for example telling him that he is taking a Chemistry exam, he would have found more information.
The professors understand that the research shows that, despite the great advancement of artificial intelligence, “there is still a long way to go before machines can compete with humans in terms of knowledge and critical thinking.” EFE
