AttentionRetriever: Attention Layers are Secretly Long Document Retrievers

Retrieval Augmented Generation (RAG) is crucial for Large Language Models (LLMs) in processing long documents, but current retrieval models are inadequate for this task due to challenges like context-awareness, causal dependence, and retrieval scope. This paper proposes AttentionRetriever, a novel long document retrieval model that leverages the attention mechanism and entity-based retrieval to create context-aware embeddings and define retrieval scope. Extensive experiments show that AttentionRetriever significantly outperforms existing retrieval models on long document datasets while maintaining efficiency comparable to dense retrieval models.