Questions to artificial intelligence: what and why

The paper shows that a dialogue with the artificial intelligence today is becoming part of the everyday human practice. In addition to those users pursuing their practical goals, there are many enthusiasts and researchers, who purposefully seek to ask questions that artificial intelligence would answer incorrectly. This is quite a difficult problem, and resembles the test questions that are present in the cultural tradition since the ancient times. A candidate for the status or benefit is required to solve a riddle. The paper analyzes posts and articles presenting erroneous answers from the large language models. It identifies the following categories of questions, to which artificial intelligence gives incorrect answers: questions based on a language game, requiring metacommunicative skills, provoking the “artificial intelligence hallucinations”, concerning controversial and ethically ambiguous topics, as well as the logical and mathematical problems. The paper notes that at the current stage of the human-machine interaction, humanity is actively testing the artificial intelligence, seeks to prove its imperfection, thus learns it, and enriches the technology.

EDN QYHMVL

References
[1] Bender E.M. et al. On the dangers of stochastic parrots: can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. New York, Association for Computing Machinery, 2021, pp. 610–623. https://doi.org/10.1145/3442188.3445922
[2] Derrida J. Signature, event, context. Limited Inc., Evanston, Northwestern University Press, 1988, pp. 1–24.
[3] Derrida J. Of Grammatology. Baltimore, the John Hopkins University Press, 1976, 397 p.
[4] Perez L.R. Do language models communicate? Communicative intent and referent from a Derridean perspective. Technology and Language, 2024, vol. 5, no. 2, pp. 40–56. https://doi.org/10.48417/technolang.2024.02.04
[5] Katz D.M., Bommarito M.J., Gao S., Arredondo P. GPT-4 passes the bar exam. SSRN Scholarly Paper, 2023. https://doi.org/10.2139/ssrn.4389233
[6] Lalova T.I., Simonova S.V. Ispolzovanie iskusstvennogo intellekta pri obuchenii inostrannomu yazyku studentov inzhenernykh spetsialnostey [Use of artificial intelligence in teaching a foreign language to engineering students]. Sovremennoe pedagogicheskoe obrazovanie — Modern Pedagogical Education, 2023, no. 12, pp. 107–111.
[7] Morozova A.A., Popova S.N. Fenomen iskustvennogo intellekta v sovremennoy nauke: ponyatie, vektory i problemy primeneniya v sfere massmediya [The phenomenon of artificial intelligence in modern science: concept, vectors, and problems of application in the sphere of mass media]. Znak: problemnoe pole mediaobrazovaniya, 2021, no. 4, pp. 41–52.
[8] Lallee A., Muco N. Hello! How can I assist you today? An Analysis of GPT Technology in Supporting International Entrepreneurship. Smaland, Linneuniversitet, 2023, 108 p.
[9] Olasik M. “Good morning, ChatGPT, can we become friends?”: an interdisciplinary scholar’s experience of ‘getting acquainted’ with the OpenAI’s Chat GPT: an auto ethnographical report. European Research Studies Journal, 2023, vol. 26, no. 2, pp. 190–205. https://doi.org/10.35808/ersj/3168
[10] Chen L., Zaharia M., Zou J. How is ChatGPT’s behavior changing over time? Harvard Data Science Review, 2024, no. 6. https://doi.org/10.1162/99608f92.5317da47
[11] Zaitsev D.V. Pochemu bolshie yazykovye modeli ne (vsegda) rassuzhdayut kak lyudi? [Why don’t big language models (always) reason like humans?]. Vestnik Moskovskogo universiteta. Ser.7 Filosofiya — Lomonosov Philosophy Journal, 2024, no. 1, pp. 76–93.
[12] Shalak V.I. Izbavlenie ot illyuziy II na primere ChatGPT [Exposing illusions – the limits of AI by the example of ChatGPT]. Technology and Language, 2024, no. 5, pp. 26–39. https://doi.org/10.48417/technolang.2024.02.03
[13] Zashikhina I.M. Podgotovka nauchnoy statyi: spravitsya li ChatGPT? [Scientific article writing: will ChatGPT help?]. Vysshee obrazovanie v Rossii — Higher Education in Russia, 2023, no. 32, pp. 24–47.
[14] Giannakidou A., Mari A. The human and the mechanical: logos, truthfulness, and ChatGPT. arXiv, 2024. https://doi.org/10.48550/arXiv.2402.01267
[15] Kuo R. Noam Chomsky: the false promise of ChatGPT. The New York Times. Available at: https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html (accessed January 15, 2025).