Effective load management of a machine learning service

Create a Python service that internally calls an ML model. The service is deployed on a server with limited computational resources, so it is necessary to consider that:

• More powerful hardware accelerates the model's performance;
• Other services run on the same server, so resources cannot be fully monopolized. It is required to design a mechanism for launching model computations so that:
• The model uses available resources with maximum efficiency;
• Its operation does not degrade the performance of other processes;
• The system automatically adapts to current load (for example, through priorities, queues, thread limits, CPU/GPU load monitoring, etc.).