Try asking an artificial intelligence system for a quote by Leopardi or a maxim by Nietzsche. The answer will come in a few seconds: fluid, credible, well constructed. Nevertheless, in almost one case in five, it will be false. Plausible, yes, but not true: never written or declared.
This is the central point of the study conducted by Let’s booka culture and literary dissemination portal, which analyzed almost 1,500 interactions with the main generative artificial intelligence systems over the course of about a year. A work designed to simulate the real use of the tool, the daily use of students, readers and disseminators.
The analysis of artificial intelligence and its results
Seven categories were investigated in the field of culture: literature, poetry, books, author quotations, philosophical maxims, works of art and, as a term of comparison, Italian grammar. Each category was meticulously analyzed over 200 interactions, obviously with one systematic verification against the original sources.
The data that emerged tells an alarming and reassuring story at the same time. Alarming in the systematic imprecision demonstrated (also in light of the lightness with which AI is often used by users). Reassuring because it means that man’s contribution is still essential for correct, quality information. The results showed that, on average, only 38 percent of the answers were correct and verifiable. 44 percent fall into the gray area of ”plausible” content: coherent, well written, but not traceable in the original texts. The remaining 19 percent, by contrast, is simply wrongwith false attributions or obvious errors.
What saves the average is the grammar: here the correctness reaches between 85 and 90 percent. In the humanities, however, the picture deteriorates rapidly. In author quotations, plausible contents reach 62 percent, in philosophy 65. Correctness stops at 15 and 13 percent respectively.
The problem of the verisimilitude of the answers
The most insidious point, in reality, is not the obviously wrong answer. No, it’s much more dangerous the one that sounds good, that seems right, that no one goes to check. AI sometimes generates content that works communicatively, even when it is not authentic.
The risk, over time, is the construction of what researchers of Let’s book they call an artificial cultural profile: Nietzsche reduced to a symbol of individual strength, Seneca to a time management manual, Dante to a metaphor for the inner journey. A chilling cultural limitation in which the author is no longer studied, but recognized through a function. Complexity, curiosity and research would no longer guide knowledgebut rather recognizability.
The decalogue for using it well
It would be a serious mistake to give up the tool for fear of its more negative consequences. Artificial intelligence remains an extremely powerful resource for expanding access to knowledge. It represents an essential support for the present and future of humanity, and there is no going back. We need to look forward, fighting the worrying superficiality with which it is often used, especially out of laziness.
Because of this, Let’s book has developed a ten-point decalogue:
1. Don’t replace thought
Artificial intelligence must not replace reasoning, but support it. The answers he provides are starting points, not conclusions.
2. Always check your sources
Content is reliable only if it can be traced back to a text, an author and a precise context. Without verification, there is no knowledge.
3. Distinguish between plausible and authentic
Not everything that appears coherent is true. Plausibility is a form of credibility, not a guarantee of authenticity.
4. Recognize reinterpretations
Many contents are reworkings of real concepts. They can help understanding, but should not be confused with original texts.
5. Don’t delegate learning
Knowledge requires time, comparison and construction. AI can facilitate access, but it cannot replace the cognitive process.
6. Use AI to deepen, not to simplify
Synthesis is useful, but it cannot replace the complexity of thought and works.
7. Always reconstruct the context
Each content must be placed in its historical, cultural and theoretical context. Without context, knowledge becomes meaningless.
8. Avoid cultural standardization
If the same authors and the same interpretations are always proposed, it is necessary to broaden our gaze and look for alternatives.
9. Integrate human comparison
Books, teachers and direct sources remain fundamental. AI does not replace dialogue and discussion.
10. Develop critical expertise
Understanding how artificial intelligence works and what its limits are is today an integral part of cultural literacy.
Ten rules that have nothing technical. They are, simply, good thinking practices. Because it is not artificial intelligence that is the problem, on the contrary. If anything, it is the man who stops thinking.
