Retrieval-Augmented Large Language Models for Robust Domain-Specific Question Answering: A Benchmark and Error Taxonomy
Keywords:
question answering, retrieval augmentation, knowledge grounding, error taxonomy, naural language processingAbstract
The rapid advancement of large language models has fundamentally transformed natural language processing, yet their application in highly specialized domains remains constrained by inherent limitations such as hallucination, lack of verifiable sourcing, and temporal knowledge degradation. Retrieval-augmented generation has emerged as a promising paradigm to address these vulnerabilities by grounding model responses in external, continuously updated knowledge bases. However, evaluating the efficacy and robustness of these systems in specialized fields such as medicine, law, and finance requires comprehensive benchmarks and a nuanced understanding of failure modes. This paper introduces a novel, domain-specific question answering benchmark designed explicitly to stress-test retrieval-augmented architectures. Furthermore, we propose a comprehensive error taxonomy that systematically categorizes failures into distinct retrieval, generation, and reasoning inadequacies. Through extensive empirical analysis of multiple state-of-the-art models, we reveal that while retrieval augmentation significantly reduces factual hallucination, it introduces new classes of errors, particularly related to context fragmentation and semantic misalignment. The proposed taxonomy provides a foundational framework for diagnosing system bottlenecks, thereby guiding future research toward more reliable, transparent, and accurate domain-specific question answering systems.
References
1. Dong, J., Yang, L., Wang, Y., Xie, X., & Lai, J. (2023). Toward intrinsic adversarial robustness through probabilistic training. IEEE Transactions on Image Processing, 32, 3862-3872.
2. Greengard, L., & Rokhlin, V. (1987). A fast algorithm for particle simulations. Journal of Computational Physics, 73(2), 325–348.
3. Dong, J., Koniusz, P., Chen, J., Wang, Z. J., & Ong, Y. S. (2024). Robust distillation via untargeted and targeted intermediate adversarial samples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 28432-28442).
4. Xie, Z., Ren, X., Zheng, T., Bai, J., Fan, W., Xu, B., ... & Song, Y. A Survey on AI Agent Harness.
5. Kong, R., Li, Y., Feng, Q., Wang, W., Ye, X., Ouyang, Y., ... & Liu, Y. (2024, August). SwapMoE: Serving off-the-shelf MoE-based large language models with tunable memory budget. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 6710-6720).
6. Zhang, W., Zhang, C., Gu, C., Kou, J., Yuan, H., Fang, X., ... & Fang, Y. (2024, October). Hallucination in Large Language Models: From Mechanistic Understanding to Novel Control Frameworks. In 2024 7th International Conference on Universal Village (UV) (pp. 1-36). IEEE.
7. Xue, X., Wang, S., Yao, H. D., Davidson, L., & Coveney, P. V. (2024). Physics informed data-driven near-wall modelling for lattice Boltzmann simulation of high Reynolds number turbulent flows. Communications Physics, 7(1), 338.
8. Zhang, Y., Carvalho, D., & Freitas, A. (2024, August). Learning disentangled semantic spaces of explanations via invertible neural networks. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 2113-2134).
9. Guo, J., Chen, X., Xia, Q., Wang, Z., Ou, J., Qin, L., ... & Tian, W. (2025, July). HASH-RAG: bridging deep hashing with retriever for efficient, fine retrieval and augmented generation. In Findings of the Association for Computational Linguistics: ACL 2025 (pp. 26847-26858).
10. Ou, J., Guo, J., Jiang, S., Wang, Z., Chen, Y., Xue, R., & Tian, W. (2025, June). Low-Rank Decomposition Assisted Quantization and Inference Compensation for Quality Large Language Model Inference. In 2025 International Joint Conference on Neural Networks (IJCNN) (pp. 1-9). IEEE.
11. Yang, Tianyue, and Xiao Xue. "Meno: Meanflow-enhanced neural operators for dynamical systems." arXiv preprint arXiv:2604.06881 (2026).
12. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., et al. (2020). Language models are few-shot learners. In Advances in Neural Information Processing Systems.
13. Xu, X., Tu, W., & Yang, Y. (2024). Adaptive selection of local and non-local attention mechanisms for speech enhancement. Neural Networks, 174, 106236.
14. Xue, Xiao, et al. "Fast-Forward Lattice Boltzmann: Learning Kinetic Behaviour with Physics-Informed Neural Operators." arXiv preprint arXiv:2509.22411 (2025).
15. Li, J., Shuang, K., Guo, J., Shi, Z., & Wang, H. (2023). Enhancing semantic relation classification with shortest dependency path reasoning. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31, 1550-1560.
16. Huang, Jinbang, et al. "One Demo Is All It Takes: Planning Domain Derivation with LLMs from A Single Demonstration." arXiv preprint arXiv:2505.18382 (2025).
17. Li, Weixian Waylon, et al. "Time is Not a Label: Continuous Phase Rotation for Temporal Knowledge Graphs and Agentic Memory." arXiv preprint arXiv:2604.11544 (2026).
18. Dong, J., Koniusz, P., Chen, J., & Ong, Y. S. (2024, September). Adversarially robust distillation by reducing the student-teacher variance gap. In European Conference on Computer Vision (pp. 92-111). Cham: Springer Nature Switzerland.
19. Li, K., Shuang, K., & Guo, J. (2026). Bring order to the jumbled input: Layered attention for Large Language Model based Event Argument Extraction. Engineering Applications of Artificial Intelligence, 165, 113399.
20. Wang, S., Yu, Y., Feldt, R., & Parthasarathy, D. (2025). Automating a complete software test process using llms: An automotive case study. arXiv preprint arXiv:2502.04008.
21. Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems.
22. Huang, Z., Wang, J., Chen, L., Xiao, B., Cai, L., Zeng, Y., & Xu, J. (2025, October). MVISU-Bench: Benchmarking Mobile Agents for Real-World Tasks by Multi-App, Vague, Interactive, Single-App and Unethical Instructions. In Proceedings of the 33rd ACM International Conference on Multimedia (pp. 8797-8805).
23. Tang, Y., Zhang, G., Liu, J. K., & Qin, R. (2025). Weakly supervised land-cover classification of high-resolution images with low-resolution labels through optimized label refinement. International Journal of Remote Sensing, 46(5), 1913-1937.
24. Tu, P., Huang, Y., Zheng, F., He, Z., Cao, L., & Shao, L. (2022, June). Guidedmix-net: Semi-supervised semantic segmentation by using labeled images as reference. In Proceedings of the AAAI conference on artificial intelligence (Vol. 36, No. 2, pp. 2379-2387).
25. Li, Weixian Waylon, et al. "Spectral attention steering for prompt highlighting." arXiv preprint arXiv:2603.01281 (2026).
26. Zhang, Y., Valentino, M., Carvalho, D., Pratt-Hartmann, I., & Freitas, A. (2024, June). Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoders. In Findings of the Association for Computational Linguistics: NAACL 2024 (pp. 474-489).
27. Chen, L. (2026). Beyond external constraints: The missing dimension of ai governance. Available at SSRN 6449738.
28. Wang J, Fan L, Li B, et al. A Dynamic Factor Gating Architecture with Market Regime Awareness for Stock Return Forecasting[J]. 2026.
29. Yang, Huan, et al. "Kvshare: An llm service system with efficient and effective multi-tenant kv cache reuse." arXiv preprint arXiv:2503.16525 (2025).
30. Xu, X., Zhang, Y., Li, J., Yang, Y., Luo, Y., & Tu, W. (2025). FIRING-Net: A filtered feature recycling network for speech enhancement. In The Thirteenth International Conference on Learning Representations.
31. Zhang, Jiaquan, et al. "Learning global hypothesis space for enhancing synergistic reasoning chain." arXiv preprint arXiv:2602.09794 (2026).
32. Lo, S. C., Zingaro, A., McCullough, J. W., Xue, X., Gonzalez-Martin, P., Joo, B., ... & Coveney, P. V. (2025). A multi-component, multi-physics computational model for solving coupled cardiac electromechanics and vascular haemodynamics. Computer Methods in Applied Mechanics and Engineering, 446, 118185.
33. Vuruma, S. K. R., Wu, D., Gupta, S. S., Aust, L., Lookingbill, V., Henry, C., ... & Huang, M. (2024). Utilizing large language models to identify reddit users considering vaping cessation for digital interventions. arXiv preprint arXiv:2404.17607.
34. Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. In Proceedings of NAACL-HLT.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Lauriane Denis, Camila Russo, Haruto Sato

This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles are published under the Creative Commons Attribution 4.0 (CC BY 4.0) License. Users may share, copy, adapt, and distribute the work for any purpose, provided proper credit is given to the original author(s) and source. Authors retain copyright.
