Pfthb

I have an ASP.NET Core Web API project. That has one controller with a method called GetLocations

GetLocations connects to 5 other web services on the internet. Gathers some info and return a collection via json. In this method I am caching the data every 5 mins using In Memory caching.

If the cache expires, it tries to connect to all 5 services and get the info and so on.

My problem is:

I have a lot of users requesting this data constantly, 50 requests a second to this API.

When the cache expires I believe there is some kind of thread locking. I have limited visibility into the project at the moment but I suspect that all these requests are calling the method and reaching out to the 5 dependent services until one of them gets a completed response from all 5.

Is my assumption right? If so how can I go about fixing this? Will I need to make each call to the web services async? Will that help this scenario? I am not 100% sure because the requests are what triggers the method call.

You should definitely make the calls to the external services use Async / Await.

That's just a given - as the best practice is to always use async for I/O heavy operations (such as calling a third-party service).

Now, you should also create a class that manages these calls. You can add it as a Singleton in your IoCConfig. In that class, make sure you're "locking" to avoid the issue you just described and not call the underlying services numerous times while the cache is being built.

Check here:
https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/lock-statement

You are facing this issue because of following reason.

1. You are using Cache and it will expire at some definite time.


2. After Cache expire you call external web service method to collect the data. Now at this point of time it might happen that other request in queue get choose for execution.


3. Once that another request is chosen for execution it also end up checking cache and it now data in cache then execute external service and so on for other request.

Solution to this.

1. First Check cache contains data or not.


2. If Not create lock so following section only be executed by single thread.


3. Now in that lock section again check for cache and if cache contains data then simply return but it does not contains then call external service.


4. At this point of time if another thread get selected for execution then it has to wait for execlsive section to complete its works.


5. Once that section get completed it store data in cache and so after if any queued or new request is there it choose data from cache.

Note : It should something like this.

 public List<string> GetData()
 
 if(Cache[key] == null)
 
 lock(obj) // obj should be static
 
 if(Cache[key] == null)
 
 // Load data from service
 Cache[key] == data;
 
 
 
 return (List<string>)Cache[Key];