Commit 45477d9b authored by Marc Gravell's avatar Marc Gravell

Keys and Values documentation; try to trap hearbeat fail

parent 9d3de2ae
...@@ -46,6 +46,9 @@ ...@@ -46,6 +46,9 @@
</None> </None>
<None Include="Transactions.md" /> <None Include="Transactions.md" />
</ItemGroup> </ItemGroup>
<ItemGroup>
<Content Include="Keys and Values.txt" />
</ItemGroup>
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" /> <Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
<!-- To modify your build process, add your task inside one of the targets below and uncomment it. <!-- To modify your build process, add your task inside one of the targets below and uncomment it.
Other similar extension points exist, see Microsoft.Common.targets. Other similar extension points exist, see Microsoft.Common.targets.
......
Keys, Values and Channels
===
In dealing with redis, there is quite an important distinction between *keys* and *everything else*. A key is the unique name of a piece of data (which could a String, a List, Hash, or any of the other [redis data types](http://redis.io/topics/data-types)) within a database. Keys are never interpreted as... well, anything: they are simply inert names. Further - when dealing with clustered or sharded systems, it is the key that defines the node (or nodes if there are slaves) that contain this data - so keys are crucial for routing commands.
This contrasts with *values*; values are the *things that you store* against keys - either individually (for String data) or as groups. Values do not affect command routing <small>(caveat: except for [the `SORT` command](http://redis.io/commands/sort) when `BY` or `GET` is specified, but that is *really* complicated to explain)</small>. Likewise, values are often *interpreted* by redis for the purposes of an operation:
- `incr` (and the various similar commands) interpret String values as numeric data
- sorting can interpret values using either numeric or unicode rules
- and many others
The key point is that the API needs to understand what is a key and what is a value. This is reflected in the StackExchange.Redis API, but the good news is that **most of the time** you don't need to know about this at all.
When using pub/sub, we are dealing with *channels*; channels do not affect routing (so they are not keys), but are quite distinct from regular values, so are considered separately.
Keys
---
StackExchange.Redis represents keys by the `RedisKey` type. The good news, though, is that this has implicit conversions to and from both `string` and `byte[]`, allowing both text and binary keys to be used without any complication. For example, the `StringIncrement` method takes a `RedisKey` as the first parameter, but *you don't need to know that*; for example:
string key = ...
db.StringIncrement(key);
or
byte[] key = ...
db.StringIncrement(key);
Likewise, there are operations that *return* keys as `RedisKey` - and again, it simply works:
string someKey = db.RandomKey();
Values
---
StackExchange.Redis represents values by the `RedisValue` type. As with `RedisKey`, there are implicit conversions in place which mean that most of the time you never see this type, for example:
db.StringSet("mykey", "myvalue");
However, in addition to text and binary contents, values can also need to represent typed primitive data - most commonly (in .NET terms) `Int32`, `Int64`, `Double` or `Boolean`. Because of this, `RedisValue` provides a lot more conversion support than `RedisKey`:
db.StringSet("mykey", 123); // this is still a RedisKey and RedisValue
...
int i = (int)db.StringGet("mykey");
Note that while the conversions from primitives to `RedisValue` are implicit, many of the conversions from `RedisValue` to primitives are explicit: this is because it is very possible that these conversions will fail if the data does not have an appropriate value.
Note additionally that *when treated numericically*, redis treats a non-existent key as zero; for consistency with this, nil responses are treated as zero:
db.KeyDelete("abc");
int i = (int)db.StringGet("abc"); // this is ZERO
If you need to detect the nil condition, then you can check for that:
db.KeyDelete("abc");
var value = db.StringGet("abc");
bool isNil = value.IsNull; // this is true
or perhaps more simply, just use the provided `Nullable<T>` support:
db.KeyDelete("abc");
var value = (int?)db.StringGet("abc"); // behaves as you would expect
Hashes
---
Since the field names in hashes do not affect command routing, they are not keys, but can take both text and binary names; thus they are treates as values for the purposes of the API.
Channels
---
Channel names for pub/sub are represented by the `RedisChannel` type; this is largely identical to `RedisKey`, but is handled independently.
Scripting
---
[Lua scripting in redis](http://redis.io/commands/EVAL) has two notable features:
- the inputs must keep keys and values separate (which inside the script become `KEYS` and `ARGV`, respectively)
- the return format is not defined in advance: it is specific to your script
Because of this, the `ScriptEvaluate` method accepts two separate input arrays: one `RedisKey[]` for the keys, one `RedisValue[]` for the values (both are optional, and are assumed to be empty if omitted). This is probably one of the few times that you'll actually need to type `RedisKey` or `RedisValue` in your code, and that is just because of array variance rules:
var result = db.ScriptEvaluate(TransferScript,
new RedisKey[] { from, to }, new RedisValue[] { quantity });
(where `TransferScript` is some `string` containing Lua, not shown for this example)
The response uses the `RedisResult` type (this is unique to scripting; usually the API tries to represent the response as directly and clearly as possible). As before, `RedisResult` offers a range of conversion operations - more, in fact than `RedisValue`, because in addition to being interpreted as text, binary, primitives and nullable-primitives, the response can *also* be interpreted as *arrays* of such, for example:
string[] items = db.ScriptEvaluate(...);
Conclusion
---
The types used in the API are very deliberately chosen to distinguish redis *keys* from *values*. However, in virtually all cases you will not need to directly refer to the unerlying types involved, as conversion operations are provided.
\ No newline at end of file
...@@ -27,5 +27,44 @@ public void TestScan() ...@@ -27,5 +27,44 @@ public void TestScan()
Assert.AreEqual(Count, count); Assert.AreEqual(Count, count);
} }
} }
[Test]
public void RandomKey()
{
using(var conn = Create(allowAdmin: true))
{
var db = conn.GetDatabase();
conn.GetServer(PrimaryServer, PrimaryPort).FlushDatabase();
string anyKey = db.RandomKey();
Assert.IsNull(anyKey);
db.StringSet("abc", "def");
byte[] keyBytes = db.RandomKey();
Assert.AreEqual("abc", Encoding.UTF8.GetString(keyBytes));
}
}
[Test]
public void Zeros()
{
using(var conn = Create())
{
var db = conn.GetDatabase();
db.KeyDelete("abc");
db.StringSet("abc", 123);
int k = (int)db.StringGet("abc");
Assert.AreEqual(123, k);
db.KeyDelete("abc");
int i = (int)db.StringGet("abc");
Assert.AreEqual(0, i);
Assert.IsTrue(db.StringGet("abc").IsNull);
int? value = (int?)db.StringGet("abc");
Assert.IsFalse(value.HasValue);
}
}
} }
} }
...@@ -295,7 +295,7 @@ internal void OnDisconnected(ConnectionFailureType failureType, PhysicalConnecti ...@@ -295,7 +295,7 @@ internal void OnDisconnected(ConnectionFailureType failureType, PhysicalConnecti
CompleteSyncOrAsync(ping); CompleteSyncOrAsync(ping);
} }
if (isCurrent = physical == connection) if (isCurrent = (physical == connection))
{ {
Trace("Bridge noting disconnect from active connection" + (isDisposed ? " (disposed)" : "")); Trace("Bridge noting disconnect from active connection" + (isDisposed ? " (disposed)" : ""));
ChangeState(State.Disconnected); ChangeState(State.Disconnected);
...@@ -355,6 +355,7 @@ internal void OnHeartbeat() ...@@ -355,6 +355,7 @@ internal void OnHeartbeat()
var tmp = physical; var tmp = physical;
if (tmp != null) if (tmp != null)
{ {
tmp.Heartbeat();
int writeEverySeconds = serverEndPoint.WriteEverySeconds; int writeEverySeconds = serverEndPoint.WriteEverySeconds;
if (writeEverySeconds > 0 && tmp.LastWriteSecondsAgo >= writeEverySeconds) if (writeEverySeconds > 0 && tmp.LastWriteSecondsAgo >= writeEverySeconds)
{ {
......
...@@ -71,7 +71,7 @@ private static readonly Message ...@@ -71,7 +71,7 @@ private static readonly Message
public PhysicalConnection(PhysicalBridge bridge) public PhysicalConnection(PhysicalBridge bridge)
{ {
lastWriteTickCount = lastReadTickCount = Environment.TickCount; lastWriteTickCount = lastReadTickCount = lastBeatTickCount = Environment.TickCount;
this.connectionType = bridge.ConnectionType; this.connectionType = bridge.ConnectionType;
this.multiplexer = bridge.Multiplexer; this.multiplexer = bridge.Multiplexer;
this.ChannelPrefix = multiplexer.RawConfig.ChannelPrefix; this.ChannelPrefix = multiplexer.RawConfig.ChannelPrefix;
...@@ -132,13 +132,18 @@ public void Dispose() ...@@ -132,13 +132,18 @@ public void Dispose()
} }
OnCloseEcho(); OnCloseEcho();
} }
long lastWriteTickCount, lastReadTickCount; long lastWriteTickCount, lastReadTickCount, lastBeatTickCount;
public void Flush() public void Flush()
{ {
outStream.Flush(); outStream.Flush();
Interlocked.Exchange(ref lastWriteTickCount, Environment.TickCount); Interlocked.Exchange(ref lastWriteTickCount, Environment.TickCount);
} }
int failureReported; int failureReported;
internal void Heartbeat()
{
Interlocked.Exchange(ref lastBeatTickCount, Environment.TickCount);
}
public void RecordConnectionFailed(ConnectionFailureType failureType, Exception innerException = null) public void RecordConnectionFailed(ConnectionFailureType failureType, Exception innerException = null)
{ {
IdentifyFailureType(innerException, ref failureType); IdentifyFailureType(innerException, ref failureType);
...@@ -152,12 +157,13 @@ public void RecordConnectionFailed(ConnectionFailureType failureType, Exception ...@@ -152,12 +157,13 @@ public void RecordConnectionFailed(ConnectionFailureType failureType, Exception
{ {
try try
{ {
long now = Environment.TickCount, lastRead = Interlocked.Read(ref lastReadTickCount), lastWrite = Interlocked.Read(ref lastWriteTickCount); long now = Environment.TickCount, lastRead = Interlocked.Read(ref lastReadTickCount), lastWrite = Interlocked.Read(ref lastWriteTickCount),
lastBeat = Interlocked.Read(ref lastBeatTickCount);
string message = failureType + " on " + Format.ToString(bridge.ServerEndPoint.EndPoint) + "/" + connectionType string message = failureType + " on " + Format.ToString(bridge.ServerEndPoint.EndPoint) + "/" + connectionType
+ ", input-buffer: " + ioBufferBytes + ", outstanding: " + GetOutstandingCount() + ", input-buffer: " + ioBufferBytes + ", outstanding: " + GetOutstandingCount()
+ ", last-read: " + unchecked(now - lastRead) / 1000 + "s ago, last-write: " + unchecked(now - lastWrite) / 1000 + "s ago, keep-alive: " + bridge.ServerEndPoint.WriteEverySeconds + "s, pending: " + ", last-read: " + unchecked(now - lastRead) / 1000 + "s ago, last-write: " + unchecked(now - lastWrite) / 1000 + "s ago, keep-alive: " + bridge.ServerEndPoint.WriteEverySeconds + "s, pending: "
+ bridge.GetPendingCount(); + bridge.GetPendingCount() + ", last-heartbeat: " + unchecked(now - lastBeat) / 1000 + "s ago";
var ex = innerException == null var ex = innerException == null
? new RedisConnectionException(failureType, message) ? new RedisConnectionException(failureType, message)
......
...@@ -121,6 +121,10 @@ internal static RedisResult TryCreate(RawResult result) ...@@ -121,6 +121,10 @@ internal static RedisResult TryCreate(RawResult result)
/// Interprets the result as an array of RedisKey /// Interprets the result as an array of RedisKey
/// </summary> /// </summary>
public static explicit operator RedisKey[] (RedisResult result) { return result.AsRedisKeyArray(); } public static explicit operator RedisKey[] (RedisResult result) { return result.AsRedisKeyArray(); }
/// <summary>
/// Interprets the result as an array of RedisResult
/// </summary>
public static explicit operator RedisResult[] (RedisResult result) { return result.AsRedisResultArray(); }
internal abstract bool AsBoolean(); internal abstract bool AsBoolean();
...@@ -158,6 +162,8 @@ internal static RedisResult TryCreate(RawResult result) ...@@ -158,6 +162,8 @@ internal static RedisResult TryCreate(RawResult result)
internal abstract RedisValue[] AsRedisValueArray(); internal abstract RedisValue[] AsRedisValueArray();
internal abstract RedisResult[] AsRedisResultArray();
internal abstract string AsString(); internal abstract string AsString();
internal abstract string[] AsStringArray(); internal abstract string[] AsStringArray();
private sealed class ArrayRedisResult : RedisResult private sealed class ArrayRedisResult : RedisResult
...@@ -257,6 +263,8 @@ internal override string AsString() ...@@ -257,6 +263,8 @@ internal override string AsString()
throw new InvalidCastException(); throw new InvalidCastException();
} }
internal override string[] AsStringArray() { return Array.ConvertAll(value, x => x.AsString()); } internal override string[] AsStringArray() { return Array.ConvertAll(value, x => x.AsString()); }
internal override RedisResult[] AsRedisResultArray() { return value; }
} }
private sealed class ErrorRedisResult : RedisResult private sealed class ErrorRedisResult : RedisResult
...@@ -306,6 +314,7 @@ public ErrorRedisResult(string value) ...@@ -306,6 +314,7 @@ public ErrorRedisResult(string value)
internal override string AsString() { throw new RedisServerException(value); } internal override string AsString() { throw new RedisServerException(value); }
internal override string[] AsStringArray() { throw new RedisServerException(value); } internal override string[] AsStringArray() { throw new RedisServerException(value); }
internal override RedisResult[] AsRedisResultArray() { throw new RedisServerException(value); }
} }
private sealed class SingleRedisResult : RedisResult private sealed class SingleRedisResult : RedisResult
...@@ -354,6 +363,7 @@ public SingleRedisResult(RedisValue value) ...@@ -354,6 +363,7 @@ public SingleRedisResult(RedisValue value)
internal override string AsString() { return (string)value; } internal override string AsString() { return (string)value; }
internal override string[] AsStringArray() { return new[] { AsString() }; } internal override string[] AsStringArray() { return new[] { AsString() }; }
internal override RedisResult[] AsRedisResultArray() { throw new InvalidCastException(); }
} }
} }
} }
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment