FastAPI is a poppolar framework to build API service. It is easy to use, fast and support async requests. FastAPI support multiple workers and parallel high volume (middle level) requests. But when you quickly develop an API using the default setting, and deploy into production, you may find the response is slow or even some of requests failed due to time of out in case of high traffic (thousands of ) requests. You google and read documents, all saysing thousands of requests should be ok. What you do to solve the problem?
First, you try increase computing resources, e.g. from 2-cores increasing 10-cores. Then test and still failed, many portions of requests are failed.
But the main reason may be in the fastapi service configure. Normally, you start your app service like
#assume your api code in api.py and app is instance.
uvicorn.run("api:app", host="0.0.0.0", port = 5012)Almost all samples code are written like this. It is ok as a simple demo code. By default, the above sample code only start 1 worker, which means 1 processs to serve the requests. It is not as your expects that as more as possible workers auto-start.
If you want to start multiple workers, you need explicitly set in the arguments, e.g.
#assume your api code in api.py and app is instance.
uvicorn.run("api:app", host="0.0.0.0", port = 5012, workers = 2)In the case, 2 workers are started.
How do you know your API server actually start the number of workers you specified? Simply, you monitor the logging information when api starts.
Here are example logging info in different configures.
- Default configure (1 worker): only 1 process start
2. workers = 2
2 process start (process id: 20441, 20442)
ps -aF to check the running process. you will find 4 process-id.
In the stage of API server starting, if multiple process start (process number is worker number, if worker number is less than core)
After you set workers correctly, you will find requests processing is fast, and you enjoy fastapi power.
