How is request routing between scenarios implemented — in what form does the LLM give an answer and how is the scenario chosen based on it?

The typical approach for routing requests between scenarios using LLM (Large Language Models) is as follows:

1. LLM Response Format: The LLM provides a structured response that includes not only text but also metadata or key tags indicating the intended scenario. For example, the response could be in JSON format with fields like "intent" (intention), "entities" (entities), "response" (text).

2. Scenario Identification: Based on the "intent" field or other metadata, the system selects the appropriate processing scenario. This could be:

   • Direct matching by keyword or tag.
   • Using a classifier that determines the scenario based on the LLM response.

3. Routing: After identifying the scenario, the request is forwarded to the corresponding module or handler, which performs further logic.

Example:

LLM returns:

{
  "intent": "order_status",
  "entities": {"order_id": "12345"},
  "response": "Your order №12345 is on the way."
}

The system reads "intent": "order_status" and directs the request to the order status processing scenario.

Thus, the key to routing is a structured LLM response with an explicit indication of intent, which the system uses to select the appropriate scenario.