What is a parent-document retriever and why is it needed?

Parent-document retriever is a component in Retrieval-Augmented Generation (RAG) systems and large language models (LLMs) that is responsible for searching and extracting parent documents or contexts related to a specific piece of information.

Why it is needed:

• In RAG models, relevant documents are first retrieved from a knowledge base.
• Sometimes, it is necessary to obtain not only a specific fragment but also its "parent" document to provide full context for answer generation.
• The parent-document retriever helps connect individual pieces of information with their sources, improving the quality and accuracy of answers.

For example, if a paragraph is extracted from an article, the parent-document retriever will find the entire article to which it belongs, so that the LLM can consider a broader context when forming a response.