Office 365 SharePoint Online enforces throttling limits to prevent users from over-consuming resources. When a user runs CSOM or REST code that exceeds usage limits, SharePoint Online throttles any further request from the user for a brief period.
Find out about throttling in SharePoint Online, and learn how to avoid being throttled or blocked. Includes sample CSOM code you can use to make your life easier.
What happens when you get throttled in SharePoint Online?
When a user exceeds usage limits, SharePoint Online throttles any further requests from that user account for a short period. All user actions are throttled while the throttle is in effect.
- For requests that a user performs directly in the browser, SharePoint Online redirects you to the throttling information page, and the requests fail.
- For all other requests, including CSOM or REST calls, SharePoint Online returns HTTP status code 429 (“Too many requests”), and the requests fail.
Why throttling is tricky subject
Throttling is depending on the overall traffic at given point in time. This cannot be predicted, especially when the SharePoint farm is shared with multiple customers.
What is the best practice to handle throttling
- Decorate your calls with user agent info. This will give priority for the calls over other normal calls
using (ClientContext ctx= new ClientContext(sitename)){
SecureString passWord = new SecureString();
foreach (char c in KEY.ToCharArray()) passWord.AppendChar(c);
clientContext.Credentials = new SharePointOnlineCredentials(UserId, passWord);
Web web = ctx.Web;
ctx.ExecutingWebRequest += delegate (object sender, WebRequestEventArgs e)
{
e.WebRequestExecutor.WebRequest.UserAgent = "NONISV|<company>|app>/1.0";
};
}
- Implement A Resource Count Semaphore in the program so that it wouldn’t allow more than 30 calls concurrently
- Implement A Back off algorithm is implemented to detect the throttling and retry after the prescribed time
public static class ClientContextExtension
{
private static SemaphoreSlim _sem=new SemaphoreSlim(30);
public static void ExecuteQueryWithIncrementalRetry(this ClientContext clientContext,int retryCount,int backoffInterval)
{
try
{
_sem.Wait(); //block the call if more than 30 concurrent calls
int retryAttempts = 0;
int retryAfterInterval = 0;
bool retry = false;
ClientRequestWrapper wrapper = null;
// Do while retry attempt is less than retry count
while (retryAttempts < retryCount)
{
try
{
if (!retry)
{
clientContext.ExecuteQuery();
return;
}
else
{
// retry the previous request
if (wrapper != null && wrapper.Value != null)
{
clientContext.RetryQuery(wrapper.Value);
return;
}
}
}
catch (WebException ex)
{
var response = ex.Response as HttpWebResponse;
// Check if request was throttled - http status code 429
// Check is request failed due to server unavailable
//- http status code 503
if (response != null && (response.StatusCode == (HttpStatusCode)429
|| response.StatusCode == (HttpStatusCode)503))
{
wrapper = (ClientRequestWrapper)ex.Data["ClientRequest"];
if(wrapper == null)
{
logger.Info("CSOM Wrapper is null");
throw;
}
retry = true;
// Determine the retry after value -
//use the retry-after header when available
string retryAfterHeader = response.GetResponseHeader("Retry-After");
if (!string.IsNullOrEmpty(retryAfterHeader))
{
if (!Int32.TryParse(retryAfterHeader, out retryAfterInterval))
{
retryAfterInterval = backoffInterval;
}
}
else
{
retryAfterInterval = backoffInterval;
}
// Delay for the requested seconds
Thread.Sleep(retryAfterInterval * 1000);
// Increase counters
retryAttempts++;
backoffInterval = backoffInterval * 2;
}
else
{
throw;
}
}
}
throw new MaximumRetryAttemptedException(
$"Maximum retry attempts {retryCount}, has be attempted."
);
}
catch
{
throw;
}
finally
{
_sem.Release(); //release the semaphore
}
}
public class MaximumRetryAttemptedException : Exception
{
public MaximumRetryAttemptedException(string message)
: base(message)
{
}
}
}
Now call the extension method as below in your CSOM code.
clientContext.ExecuteQueryWithIncrementalRetry(3,5000);
This is the first example of this code I’ve seen that limits the number of concurrent calls. I’m interested in where the limit of 30 came from? Is it a number that seems to work for you or is there some specific documentation or guidance somewhere that suggests that number?
LikeLike
We have performed quite bit of trail runs using a load testing program. Throttling is depending on the overall traffic at given point in time. When we set the concurrent limit to 30, the throttling rate were very low. Recently we have upgraded the algorithm with multiple service account used in a round robin fashion to increase the limit to 30 per each service account. Our current deployment uses 10 service account and we are able to top 300 concurrent request split between 10 service accounts.
LikeLike