Scheduling in on-demand GPU-as-a-service system: a review

In recent years, the use of graphics processing resources has increased due to the ability to run tasks in parallel. Also, due to the increase in the use of systems based on machine learning and deep learning and the ability to execute these types of requests in parallel, graphics processors are often used to train this category of computational models in order to increase performance. The use of graphics processors (GPGPU) aims to parallelize the execution of tasks, which is possible in deep learning tasks. Most service systems, such as cloud services that receive requests with parallelization capabilities, tend to use Graphics Processing Unit (GPU) servers. The unit time price of GPU-based virtual machines is ۵ to ۸ times higher than that of CPU-based virtual machines. In this regard, the execution speed in GPU processors is much higher than the execution speed in the CPU. For this reason and in line with the optimal use of graphics processing resources available in this type of server, the issue of scheduling requests is a challenge. Usually, scheduling is used to balance loads on the system. Scheduling also ensures that a computer system is able to respond to most requests. The main goal in the mentioned schedule is to increase the acceptance rate of received requests, reduce the user's cost, and also increase the profitability of the resource provider. Many methods have been proposed for scheduling requests in GPU-based systems. Some solutions consider the user's budget as well as the priority of resources or tasks. This paper intends to examine the available methods for scheduling GPU-based tasks and mention their advantages and disadvantages. Also, the open problems in this field will also be stated at the end.