What is DoRA and how does it differ from LoRA?

DoRA (Domain-oriented Retrieval Augmentation) and LoRA (Low-Rank Adaptation) are methods of adapting large language models, but with different approaches.

• LoRA is a fine-tuning method that adds low-rank matrices to the model's weights, allowing efficient adaptation without changing all parameters.

• DoRA is a method that focuses on improving the model by integrating relevant information from external sources (retrieval), targeting a specific domain. This enables the model to use current data for generating responses.

Thus, LoRA is a parameter adaptation technique, while DoRA is an approach that extends the model with external knowledge tailored to a specific domain.