Three key findings from the study:
- They point out that RAG systems are vulnerable to minor but frequent textual errors within the documents.
- An attack method called GARAG is proposed, based on an algorithm searching for adversarial documents.
- RAG systems are susceptible to noisy documents in real-world databases.
The reader’s ability to accurately ground information significantly depends on the retriever’s capability of sourcing query-relevant documents.
GARAG assesses the holistic robustness of a RAG system against minor textual errors, offering insights into the system’s resilience through iterative adversarial refinement.
This new study contains three main contributions…
- Highlighting a vulnerability in RAG systems pertaining to frequent minor textual errors within documents. This evaluation focuses on the retriever and reader components’ functionality.
- Introducing GARAG, a straightforward & potent attack strategy leveraging a genetic algorithm to craft adversarial documents capable of exploiting weaknesses in both components of RAG simultaneously.
- Through experimentation, demonstrating the detrimental impact of noisy documents on the RAG system within real-world databases.
The results show that typos seriously harm the RAG system, making it work much worse. Even though the retriever helps protect the reader, it can still be affected by small disruptions.
⭐️ Follow me on LinkedIn for updates on Large Language Models ⭐️
I’m currently the Chief Evangelist @ Kore AI. I explore & write about all things at the intersection of AI & language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces & more.
