Commit 8c36d5b8 authored by Marc Gravell's avatar Marc Gravell

- try to auto-detect `LegacyAspNetSynchronizationContext` and enable the feature-flag

- additional guidance on thread-theft
- use UQUWI for continuations
- remove handling from simple (non-task) results; no risk of thread-theft
parent d37289b2
......@@ -45,7 +45,15 @@ configure ASP.NET with:
<add key="aspnet:UseTaskFriendlySynchronizationContext" value="false" />
```
In this scenario, we would once again end up with the reader being stolen and used for
or
```
<httpRuntime targetFramework="4.5" />
```
([citation](https://devblogs.microsoft.com/aspnet/all-about-httpruntime-targetframework))
In these scenarios, we would once again end up with the reader being stolen and used for
processing your application logic. This can doom any further `await`s to timeouts,
either temporarily (until the application logic chooses to release the thread), or permanently
(essentially deadlocking yourself).
......
......@@ -3,6 +3,11 @@
Verify what's the maximum bandwidth supported on your client and on the server where redis-server is hosted. If there are requests that are getting bound by bandwidth, it will take longer for them to complete and thereby can cause timeouts.
Similarly, verify you are not getting CPU bound on client or on the server box which would cause requests to be waiting for CPU time and thereby have timeouts.
Are you experiencing "thread theft" of the reader?
---
The parameter “`qs`” in the error message tells you the state of the reader; if this is frquently reporting `CompletePendingMessage`,
it is possible that the reader loop has been hijacked; see [Thread Theft](ThreadTheft) for specific guidance.
Are there commands taking long time to process on the redis-server?
---------------
There can be commands that are taking long time to process on the redis-server causing the request to timeout. Few examples of long running commands are mget with large number of keys, keys * or poorly written lua script. You can run the SlowLog command to see if there are requests taking longer than expected. More details regarding the command can be found [here](https://redis.io/commands/slowlog).
......
......@@ -45,6 +45,18 @@ public static void SetFeatureFlag(string flag, bool enabled)
}
}
static ConnectionMultiplexer()
{
bool value = false;
try
{ // attempt to detect a known problem scenario
value = SynchronizationContext.Current?.GetType()?.Name
== "LegacyAspNetSynchronizationContext";
}
catch { }
SetFeatureFlag(nameof(FeatureFlags.PreventThreadTheft), value);
}
/// <summary>
/// Returns the state of a feature flag; this should only be used under support guidance
/// </summary>
......
......@@ -27,15 +27,7 @@ internal abstract class SimpleResultBox : IResultBox
void IResultBox.SetException(Exception exception) => _exception = exception ?? CancelledException;
void IResultBox.Cancel() => _exception = CancelledException;
static readonly WaitCallback s_ActivateContinuations = state => ((SimpleResultBox)state).ActivateContinuationsImpl();
void IResultBox.ActivateContinuations()
{
if (ConnectionMultiplexer.PreventThreadTheft)
ThreadPool.QueueUserWorkItem(s_ActivateContinuations, this);
else
ActivateContinuationsImpl();
}
private void ActivateContinuationsImpl()
{
lock (this)
{ // tell the waiting thread that we're done
......@@ -119,8 +111,8 @@ T IResultBox<T>.GetResult(out Exception ex, bool _)
static readonly WaitCallback s_ActivateContinuations = state => ((TaskResultBox<T>)state).ActivateContinuationsImpl();
void IResultBox.ActivateContinuations()
{
if (ConnectionMultiplexer.PreventThreadTheft)
ThreadPool.QueueUserWorkItem(s_ActivateContinuations, this);
if ((Task.CreationOptions & TaskCreationOptions.RunContinuationsAsynchronously) == 0)
ThreadPool.UnsafeQueueUserWorkItem(s_ActivateContinuations, this);
else
ActivateContinuationsImpl();
}
......@@ -145,15 +137,14 @@ private void ActivateContinuationsImpl()
public static IResultBox<T> Create(out TaskCompletionSource<T> source, object asyncState)
{
// since 2.0, we only support platforms where this is correctly implemented
const TaskCreationOptions CreationOptions = TaskCreationOptions.RunContinuationsAsynchronously;
// it might look a little odd to return the same object as two different things,
// but that's because it is serving two purposes, and I want to make it clear
// how it is being used in those 2 different ways; also, the *fact* that they
// are the same underlying object is an implementation detail that the rest of
// the code doesn't need to know about
var obj = new TaskResultBox<T>(asyncState, CreationOptions);
var obj = new TaskResultBox<T>(asyncState, ConnectionMultiplexer.PreventThreadTheft
? TaskCreationOptions.None // if we don't trust the TPL/sync-context, avoid a double QUWI dispatch
: TaskCreationOptions.RunContinuationsAsynchronously);
source = obj;
return obj;
}
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment