Researchers from Skoltech and Sberbank’s Center for Practical Artificial Intelligence have proposed a new method, TOHA, for detecting hallucinations in large language models operating in retrieval-augmented generation (RAG) systems. The approach analyzes the topological structure of a model’s attention maps and makes it possible to identify responses that are not supported by the provided context. The method does not require training additional models and uses only a small amount of annotated data for configuration.

The paper was published in the proceedings of the Association for Computational Linguistics — ACL 2026, an A*-ranked conference. One of the authors is Aleksey Zaytsev, Associate Professor at Skoltech and Head of the joint Skoltech–Sberbank Laboratory (LARSS).

Large language models are increasingly becoming the foundation of AI assistants and autonomous agents. To improve the accuracy of their responses, modern systems often use RAG technology: before generating an answer, the model receives additional information from corporate knowledge bases, documents, or other external sources. However, even in this setting, it may produce plausible responses that are not supported by the provided context.

Such errors can be identified using specialized hallucination-detection methods, but many existing approaches require either large amounts of annotated data or additional computation — for example, generating multiple responses and subsequently comparing the results.

The TOHA method — TOpology-based HAllucination detector — proposed by the authors takes a different approach. The researchers analyze the language model’s attention matrices and represent them as graphs. They then calculate a topological characteristic called MTop-Div, which captures differences between the structures of the subgraphs corresponding to the original context and the generated response.

The study showed that, for certain attention heads, higher values of this topological divergence are consistently associated with responses that are not supported by the original context. This makes it possible to use internal signals from the language model itself to assess the reliability of its response. The authors evaluated the approach on question-answering and text-summarization tasks. Across several benchmarks, TOHA achieved results comparable to or better than existing methods while requiring relatively little annotated data and computational resources.

One of the advantages of the method is that there is no need to train a separate classifier. A small set of examples is sufficient for configuration, after which the selected informative attention heads are used to evaluate new responses. According to the authors, TOHA achieves results comparable to the substantially more computationally intensive SelfCheckGPT method, which relies on generating multiple responses.

“When language models are used in applied systems, it is important not only to obtain a convincing-sounding answer but also to understand whether it is actually based on the information that was provided to the model,” said Aleksey Zaytsev. “In this work, we show that such a signal can be extracted from the model’s internal representation of the text if it is analyzed in the right way. The activation of specific topological features signals potential hallucinations without requiring the training of an additional detector model or repeated response generation.”

“The fight against AI hallucinations is entering a new stage,” explained Sergey Ryabov, Senior Managing Director and Director of AI Transformation at Sberbank. “The TOHA method, developed jointly with Skoltech, demonstrates that reliability does not necessarily require high costs. Instead of large-scale data annotation, we use targeted calibration of the model’s attention, making it adhere closely to the context and independently correct inaccuracies. The technology has already been integrated into the open-source SIRIN library, making it available for the development of corporate assistants that work with knowledge bases. This sets a new standard of trust in artificial intelligence responses.”

A practical implementation of the approach has already been incorporated into SIRIN, an open-source library developed by researchers at Sberbank’s Center for Practical Artificial Intelligence to detect and analyze contextual inconsistencies in systems based on large language models. This makes it possible to use the method when developing AI assistants and other solutions that work with corporate knowledge bases and documents.

For companies, the tool may be particularly useful in areas where errors made by AI systems can create financial, legal, or reputational risks — for example, in banking, insurance, and legal services. For researchers, the study also offers a new way to investigate the internal mechanisms of language models through the topological properties of attention graphs.

The study was conducted as part of joint research activities between Sber and Skoltech.