What techniques increase the accuracy of small LLMs (Llama 3 8B, Mistral 7B) on complex tasks?

To improve the accuracy of small LLMs (e.g., Llama 3 8B, Mistral 7B) on complex tasks, the following techniques are often used:

• Chain-of-Thought prompting — encouraging the model to generate intermediate reasoning, which helps better understand and solve the task.
• Few-shot learning — providing several examples in the prompt to guide the model.
• Self-consistency — generating multiple answers and selecting the most frequent or consistent one.
• Using external tools — such as calculators or knowledge bases to verify and refine answers.
• Decomposition of the task — breaking down a complex task into simpler subtasks.

Example of Chain-of-Thought:

Question: If I have 3 apples and buy 2 more, how many apples do I have in total?
Answer (Chain-of-Thought): I had 3 apples. I bought 2 more apples. So, I have 3 + 2 = 5 apples.