Commit 544325a7 authored by Savorboard's avatar Savorboard Committed by GitHub

Release 2.1 (#55)

* add dashboard branch.

* add dashboard

* add helper methods.

* Add data model.

* add options.

* add empty implement

* add dashbaord

* add dashboard

* Fixed spelling error

* rename file.

* add dashbaord feature

* impl dashboard storage

* update nuget reference.

* add pages.

* deleted unused fiels.

* modify resource.

* update resource.

* rename

* update dashboard

* rename

* impl monitoring api interface.

* update solution files.

* update samples.

* dashboard.

* add jsonview resource files.

* add json dispatcher.

* modify published pages.

* add routes.

* update pages.

* update resources.

* update dashboard.

* add dashboard of sql server storage impl

* nesting files.

* fixed query bug.

* update resource.

* remove some api.

* add subscriber page.

* update sample

* add resource.

* remove api.

* add SubscriberPage

* add resource

* generate cshtml.

* modify html two table to one table.

* update resource.

* update css

* update subscriber page.

* refactor.

* cleanup.

* cleanup code.

* impl history monitoring api.

* add home page recevied message real-time

* add legend styles.

* update js.

* modify axis color.

* add resource.

* update dashboard home page.

* update css.

* update resource.

* modify DefaultSucceedMessageExpirationAfter to 24 hours.

* add resx,

* add consul discovery.

* remove unused file.

* add node page.

* add node page

* node discovery

* add kafka sqlserver sample.

* update sample.

* add okstats.

* refactor.

* fixed kafka client bugs.

* modify node and subscriber pages

* refactor.

* add Gateway middleware

* remove unused files.

* update resource.

* update gateway.

* refactor.

* update samples.

* remove base middleware

* add node switch click event.

* add NodeId config to options.

* upgrade dependent version.

* add PathMatch configuration

* update NodePage.cshtml

* remove session

* remove matchPath

* refactor

* refactor dashboard middleware

* gateway proxy middleware function maturation

* remove cookie exp

* refactor and remove files.

* add CapCache to cache server nodes.

* refactor.

* renamed message dto.

* add extended interface of IContentSerializer and JsonContentSerializer

* modify unit test

* check the requirement when CAP start.

* correct spelling

* cleanup code.

* add resources.

* processing pages will contains  Scheduled  and Enqueued messages.

* processing pages will contains  Scheduled  and Enqueued messages.

* ignore NU1701 Warning.

* renamed file.

* refactor

* implements dashboard interface.

* rename reference class.

* add mysql monitoring api impl

* fix bug of connection driver.

* remove cap.UseDashboard.  It's will be automatically enabled by registerd services.

* fix sql bug

* cleanup code and fix spelling

* fix postgre sql bug.

* fix mysql sql bug.

* when storage a received message raising an eception, we will reject the message to queue.

* fix spelling mistake

* add dashboard instructions to readme

* modify error log content.

* fix postger sql bug.

* fix consul discovery bug.

* add dashboard introduction to readme.md

* update english resource.

* Update README.md

* renamed files.

* fix postgre sql bug.

* cleanup code.

* update tests.

* rename file.

* update samples.

* Improved query performance without lock table. (#36)

* update sample.

* update samples.

* fix data reader uncolsed bug.

* update add jsonproperty

* refactor

* revert FetchNextMessageAsync sql

* add helper method.

* rafactor subscriber handler.

* add FailedRetryCount options.

* rafactor publisher excutor.

* add IPublishExecutor

* add failed message processor.

* inject failed message processor.

* refactor sql storage.

* fixed unit tests.

* fixed unit test.

* fixed postgresql tests.

* Update .travis.yml

* Update .travis.yml

* Update .travis.yml

* Update .travis.yml

* Update .travis.yml

* Update .travis.yml

* Update .travis.yml

* Update .travis.yml

* Update .travis.yml

* Update .travis.yml

* Update .travis.yml

* add LAN ip to LocalRequestsOnlyAuthorizationFilter

* add and update resource.

* add current node name to layout page if user enabled node discovery

* modify unit tests.

* add callback message sende tests.

* add deserlizer by type to  IContentSerializer

* refactor unit tests

* add summary comment.

* refactor async method.

* add comment and fixed spell error.

* refactor.

* add custom content serializer extension to CapBuilder

* add IMessagePacker

* refactor.

* add connection pool for kafka producer.

* add determines whether the query is null.

* add connection pool size config to KafkaOption.

* fixed json JObject bug

* add custom message wapper interface

* remove unused code.

* fixed callback topic send error bug.

* refactor.

* update unit tests.

* update samples.

* upgrade dependent package.

* remove some class from Abstraction namespce to Internal.

* optimize consumer related code

* add ICallbackMessageSender to DI with singleton.

* add and fixed some unit tests.

* refactor namespace.

* modify class protected level

* assemblies internal class are visible to test project

* add DeSerialize method to IContentSerializer with type deseralize

* refactoring

* Fix the phone style dispaly problem

* add logs

* refactoring

* upgrading `Confluent.Kafka` package

* Optimize message queue error message prompt.

* reorganize error message prompts.

* modify error message prompt

* add summary comments.

* modify dependent

* disabled print connection closed log. see: https://github.com/edenhill/librdkafka/issues/516

* Update README.md

* Update README.md

* Update README.zh-cn.md

* Update README.zh-cn.md

* fix dashboard not config discovery throw exceptions bug.

* Update README.md

* update readme

* Fixed serialized the message type bug. (#53)

* Update README.md

* Update README.md

* refactoring

* refactoring

* update readme.

* update readme.

* add summary comment.

* refactoring

* optimizing publisher interface

* update readme

* update readme.

* add summary comment.

* add summary comment.

* add summary comment.

* add summary comment.

* upgrading package

* add summary comment.

* optimize the RabbitMQ connection pool

* fix the producer connection returned

* dispose resource when connection pool is full
parent 8744532f
language: csharp language: cpp
sudo: required sudo: required
mono: none dist: trusty
matrix: matrix:
include: include:
- os: linux - os: linux
dist: trusty # Ubuntu 14.04 dist: trusty # Ubuntu 14.04
dotnet: 2.0.0
mono: none
env: DOTNETCORE=1
sudo: required sudo: required
- os: osx - os: osx
osx_image: xcode8.3 # macOS 10.12 osx_image: xcode8.3 # macOS 10.12
dotnet: 2.0.0
mono: none env:
env: DOTNETCORE=1 global:
- DOTNET_SKIP_FIRST_TIME_EXPERIENCE: true
before_install: - DOTNET_CLI_TELEMETRY_OPTOUT: 1
- chmod a+x ./build.sh - CLI_VERSION=2.0.0
- if [ "$TRAVIS_OS_NAME" == "osx" ]; then ulimit -n 1024 ; fi
addons:
apt:
packages:
- gettext
- libcurl4-openssl-dev
- libicu-dev
- libssl-dev
- libunwind8
- zlib1g
# Make sure build dependencies are installed.
before_install:
- if test "$TRAVIS_OS_NAME" == "osx"; then ln -s /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib /usr/local/lib/; ln -s /usr/local/opt/openssl/lib/libssl.1.0.0.dylib /usr/local/lib/; fi
- export DOTNET_INSTALL_DIR="$PWD/.dotnetcli"
- export PATH="$DOTNET_INSTALL_DIR:$PATH"
install: install:
- export DOTNET_CLI_TELEMETRY_OPTOUT=1 - travis_retry curl -sSL https://dot.net/v1/dotnet-install.sh | bash /dev/stdin --channel 2.0 --version "$CLI_VERSION" --install-dir "$DOTNET_INSTALL_DIR"
- if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then rvm get stable; brew update; brew install openssl; fi
# Run the build script
script: script:
- ./build.sh - dotnet --info
\ No newline at end of file - dotnet restore
- dotnet test test/DotNetCore.CAP.Test/DotNetCore.CAP.Test.csproj -f netcoreapp2.0
 
Microsoft Visual Studio Solution File, Format Version 12.00 Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 15 # Visual Studio 15
VisualStudioVersion = 15.0.26730.3 VisualStudioVersion = 15.0.26730.15
MinimumVisualStudioVersion = 10.0.40219.1 MinimumVisualStudioVersion = 10.0.40219.1
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "src", "src", "{9B2AE124-6636-4DE9-83A3-70360DABD0C4}" Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "src", "src", "{9B2AE124-6636-4DE9-83A3-70360DABD0C4}"
EndProject EndProject
...@@ -60,7 +60,9 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DotNetCore.CAP.PostgreSql", ...@@ -60,7 +60,9 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DotNetCore.CAP.PostgreSql",
EndProject EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Sample.RabbitMQ.PostgreSql", "samples\Sample.RabbitMQ.PostgreSql\Sample.RabbitMQ.PostgreSql.csproj", "{A17E8E72-DFFC-4822-BB38-73D59A8B264E}" Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Sample.RabbitMQ.PostgreSql", "samples\Sample.RabbitMQ.PostgreSql\Sample.RabbitMQ.PostgreSql.csproj", "{A17E8E72-DFFC-4822-BB38-73D59A8B264E}"
EndProject EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "DotNetCore.CAP.PostgreSql.Test", "test\DotNetCore.CAP.PostgreSql.Test\DotNetCore.CAP.PostgreSql.Test.csproj", "{7CA3625D-1817-4695-881D-7E79A1E1DED2}" Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "DotNetCore.CAP.PostgreSql.Test", "test\DotNetCore.CAP.PostgreSql.Test\DotNetCore.CAP.PostgreSql.Test.csproj", "{7CA3625D-1817-4695-881D-7E79A1E1DED2}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Sample.Kafka.SqlServer", "samples\Sample.Kafka.SqlServer\Sample.Kafka.SqlServer.csproj", "{573B4D39-5489-48B3-9B6C-5234249CB980}"
EndProject EndProject
Global Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution GlobalSection(SolutionConfigurationPlatforms) = preSolution
...@@ -119,6 +121,10 @@ Global ...@@ -119,6 +121,10 @@ Global
{7CA3625D-1817-4695-881D-7E79A1E1DED2}.Debug|Any CPU.Build.0 = Debug|Any CPU {7CA3625D-1817-4695-881D-7E79A1E1DED2}.Debug|Any CPU.Build.0 = Debug|Any CPU
{7CA3625D-1817-4695-881D-7E79A1E1DED2}.Release|Any CPU.ActiveCfg = Release|Any CPU {7CA3625D-1817-4695-881D-7E79A1E1DED2}.Release|Any CPU.ActiveCfg = Release|Any CPU
{7CA3625D-1817-4695-881D-7E79A1E1DED2}.Release|Any CPU.Build.0 = Release|Any CPU {7CA3625D-1817-4695-881D-7E79A1E1DED2}.Release|Any CPU.Build.0 = Release|Any CPU
{573B4D39-5489-48B3-9B6C-5234249CB980}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{573B4D39-5489-48B3-9B6C-5234249CB980}.Debug|Any CPU.Build.0 = Debug|Any CPU
{573B4D39-5489-48B3-9B6C-5234249CB980}.Release|Any CPU.ActiveCfg = Release|Any CPU
{573B4D39-5489-48B3-9B6C-5234249CB980}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection EndGlobalSection
GlobalSection(SolutionProperties) = preSolution GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE HideSolutionNode = FALSE
...@@ -137,6 +143,7 @@ Global ...@@ -137,6 +143,7 @@ Global
{82C403AB-ED68-4084-9A1D-11334F9F08F9} = {9B2AE124-6636-4DE9-83A3-70360DABD0C4} {82C403AB-ED68-4084-9A1D-11334F9F08F9} = {9B2AE124-6636-4DE9-83A3-70360DABD0C4}
{A17E8E72-DFFC-4822-BB38-73D59A8B264E} = {3A6B6931-A123-477A-9469-8B468B5385AF} {A17E8E72-DFFC-4822-BB38-73D59A8B264E} = {3A6B6931-A123-477A-9469-8B468B5385AF}
{7CA3625D-1817-4695-881D-7E79A1E1DED2} = {C09CDAB0-6DD4-46E9-B7F3-3EF2A4741EA0} {7CA3625D-1817-4695-881D-7E79A1E1DED2} = {C09CDAB0-6DD4-46E9-B7F3-3EF2A4741EA0}
{573B4D39-5489-48B3-9B6C-5234249CB980} = {3A6B6931-A123-477A-9469-8B468B5385AF}
EndGlobalSection EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {2E70565D-94CF-40B4-BFE1-AC18D5F736AB} SolutionGuid = {2E70565D-94CF-40B4-BFE1-AC18D5F736AB}
......
This diff is collapsed.
...@@ -6,17 +6,16 @@ ...@@ -6,17 +6,16 @@
[![Member project of .NET China Foundation](https://img.shields.io/badge/member_project_of-.NET_CHINA-red.svg?style=flat&colorB=9E20C8)](https://github.com/dotnetcore) [![Member project of .NET China Foundation](https://img.shields.io/badge/member_project_of-.NET_CHINA-red.svg?style=flat&colorB=9E20C8)](https://github.com/dotnetcore)
[![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](https://raw.githubusercontent.com/dotnetcore/CAP/master/LICENSE.txt) [![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](https://raw.githubusercontent.com/dotnetcore/CAP/master/LICENSE.txt)
CAP 是一个在分布式系统(SOA、MicroService)中实现最终一致性的库,它具有轻量级、易使用、高性能等特点。 CAP 是一个基于 .NET Standard 的 C# 库,它是一种处理分布式事务的解决方案,同样具有 EventBus 的功能,它具有轻量级、易使用、高性能等特点。
你可以在这里[CAP Wiki](https://github.com/dotnetcore/CAP/wiki)看到更多详细资料。 你可以在这里[CAP Wiki](https://github.com/dotnetcore/CAP/wiki)看到更多详细资料。
## 预览(OverView) ## 预览(OverView)
CAP 是在一个 ASP.NET Core 项目中使用的库,当然他可以用于 ASP.NET Core On .NET Framework 中。 在我们构建 SOA 或者 微服务系统的过程中,我们通常需要使用事件来对各个服务进行集成,在这过程中简单的使用消息队列并不能保证数据的最终一致性,
CAP 采用的是和当前数据库集成的本地消息表的方案来解决在分布式系统互相调用的各个环节可能出现的异常,它能够保证任何情况下事件消息都是不会丢失的。
你可以把 CAP 看成是一个 EventBus,因为它具有 EventBus 的所有功能,并且 CAP 提供了更加简化的方式来处理 EventBus 中的发布和订阅。 你同样可以把 CAP 当做 EventBus 来使用,CAP提供了一种更加简单的方式来实现事件消息的发布和订阅,在订阅以及发布的过程中,你不需要继承或实现任何接口。
CAP 具有消息持久化的功能,当你的服务进行重启或者宕机时它可以保证消息的可靠性。CAP提供了基于Microsoft DI 的 Publisher Service 服务,它可以和你的业务服务进行无缝结合,并且支持强一致性的事务。
这是CAP集在ASP.NET Core 微服务架构中的一个示意图: 这是CAP集在ASP.NET Core 微服务架构中的一个示意图:
...@@ -59,33 +58,33 @@ PM> Install-Package DotNetCore.CAP.PostgreSql ...@@ -59,33 +58,33 @@ PM> Install-Package DotNetCore.CAP.PostgreSql
首先配置CAP到 Startup.cs 文件中,如下: 首先配置CAP到 Startup.cs 文件中,如下:
```cs ```c#
public void ConfigureServices(IServiceCollection services) public void ConfigureServices(IServiceCollection services)
{ {
...... ......
services.AddDbContext<AppDbContext>(); services.AddDbContext<AppDbContext>();
services.AddCap(x => services.AddCap(x =>
{ {
// 如果你的 SqlServer 使用的 EF 进行数据操作,你需要添加如下配置: // 如果你的 SqlServer 使用的 EF 进行数据操作,你需要添加如下配置:
// 注意: 你不需要再次配置 x.UseSqlServer(""") // 注意: 你不需要再次配置 x.UseSqlServer(""")
x.UseEntityFramework<AppDbContext>(); x.UseEntityFramework<AppDbContext>();
// 如果你使用的Dapper,你需要添加如下配置: // 如果你使用的Dapper,你需要添加如下配置:
x.UseSqlServer("数据库连接字符串"); x.UseSqlServer("数据库连接字符串");
// 如果你使用的 RabbitMQ 作为MQ,你需要添加如下配置: // 如果你使用的 RabbitMQ 作为MQ,你需要添加如下配置:
x.UseRabbitMQ("localhost"); x.UseRabbitMQ("localhost");
//如果你使用的 Kafka 作为MQ,你需要添加如下配置: //如果你使用的 Kafka 作为MQ,你需要添加如下配置:
x.UseKafka("localhost"); x.UseKafka("localhost");
}); });
} }
public void Configure(IApplicationBuilder app) public void Configure(IApplicationBuilder app)
{ {
..... .....
app.UseCap(); app.UseCap();
} }
...@@ -96,37 +95,43 @@ public void Configure(IApplicationBuilder app) ...@@ -96,37 +95,43 @@ public void Configure(IApplicationBuilder app)
在 Controller 中注入 `ICapPublisher` 然后使用 `ICapPublisher` 进行消息发送 在 Controller 中注入 `ICapPublisher` 然后使用 `ICapPublisher` 进行消息发送
```cs ```c#
public class PublishController : Controller public class PublishController : Controller
{ {
private readonly ICapPublisher _publisher; [Route("~/checkAccountWithTrans")]
public async Task<IActionResult> PublishMessageWithTransaction([FromServices]AppDbContext dbContext, [FromServices]ICapPublisher publisher)
public PublishController(ICapPublisher publisher) {
{ using (var trans = dbContext.Database.BeginTransaction())
_publisher = publisher; {
} // 此处填写你的业务代码
//如果你使用的是EF,CAP会自动发现当前环境中的事务,所以你不必显式传递事务参数。
[Route("~/checkAccount")] //由于本地事务, 当前数据库的业务操作和发布事件日志之间将实现原子性。
public async Task<IActionResult> PublishMessage() await publisher.PublishAsync("xxx.services.account.check", new Person { Name = "Foo", Age = 11 });
{
//指定发送的消息头和内容 trans.Commit();
await _publisher.PublishAsync("xxx.services.account.check", new Person { Name = "Foo", Age = 11 }); }
return Ok();
return Ok(); }
}
[Route("~/publishWithTransactionUsingAdonet")]
[Route("~/checkAccountWithTrans")] public async Task<IActionResult> PublishMessageWithTransactionUsingAdonet([FromServices]ICapPublisher publisher)
public async Task<IActionResult> PublishMessageWithTransaction([FromServices]AppDbContext dbContext) {
{ var connectionString = "";
using (var trans = dbContext.Database.BeginTransaction()) using (var sqlConnection = new SqlConnection(connectionString))
{ {
await _publisher.PublishAsync("xxx.services.account.check", new Person { Name = "Foo", Age = 11 }); sqlConnection.Open();
using (var sqlTransaction = sqlConnection.BeginTransaction())
trans.Commit(); {
} // 此处填写你的业务代码,通常情况下,你可以将业务代码使用一个委托传递进来进行封装该区域代码。
return Ok();
} publisher.Publish("xxx.services.account.check", new Person { Name = "Foo", Age = 11 }, sqlTransaction);
sqlTransaction.Commit();
}
}
return Ok();
}
} }
``` ```
...@@ -137,25 +142,16 @@ public class PublishController : Controller ...@@ -137,25 +142,16 @@ public class PublishController : Controller
在 Action 上添加 CapSubscribeAttribute 来订阅相关消息。 在 Action 上添加 CapSubscribeAttribute 来订阅相关消息。
```cs ```c#
public class PublishController : Controller public class PublishController : Controller
{ {
private readonly ICapPublisher _publisher; [CapSubscribe("xxx.services.account.check")]
public async Task CheckReceivedMessage(Person person)
public PublishController(ICapPublisher publisher) {
{ Console.WriteLine(person.Name);
_publisher = publisher; Console.WriteLine(person.Age);
} return Task.CompletedTask;
}
[NoAction]
[CapSubscribe("xxx.services.account.check")]
public async Task CheckReceivedMessage(Person person)
{
Console.WriteLine(person.Name);
Console.WriteLine(person.Age);
return Task.CompletedTask;
}
} }
``` ```
...@@ -164,37 +160,72 @@ public class PublishController : Controller ...@@ -164,37 +160,72 @@ public class PublishController : Controller
如果你的订阅方法没有位于 Controller 中,则你订阅的类需要继承 `ICapSubscribe` 如果你的订阅方法没有位于 Controller 中,则你订阅的类需要继承 `ICapSubscribe`
```cs ```c#
namespace xxx.Service namespace xxx.Service
{ {
public interface ISubscriberService public interface ISubscriberService
{ {
public void CheckReceivedMessage(Person person); public void CheckReceivedMessage(Person person);
} }
public class SubscriberService: ISubscriberService, ICapSubscribe public class SubscriberService: ISubscriberService, ICapSubscribe
{ {
[CapSubscribe("xxx.services.account.check")] [CapSubscribe("xxx.services.account.check")]
public void CheckReceivedMessage(Person person) public void CheckReceivedMessage(Person person)
{ {
} }
} }
} }
``` ```
然后在 Startup.cs 中的 `ConfigureServices()` 中注入你的 `ISubscriberService` 然后在 Startup.cs 中的 `ConfigureServices()` 中注入你的 `ISubscriberService`
```cs ```c#
public void ConfigureServices(IServiceCollection services) public void ConfigureServices(IServiceCollection services)
{ {
services.AddTransient<ISubscriberService,SubscriberService>(); services.AddTransient<ISubscriberService,SubscriberService>();
} }
``` ```
### Dashboard
CAP 2.1+ 以上版本中提供了仪表盘(Dashboard)功能,你可以很方便的查看发出和接收到的消息。除此之外,你还可以在仪表盘中实时查看发送或者接收到的消息。
在分布式环境中,仪表盘内置集成了 [Consul](http://consul.io) 作为节点的注册发现,同时实现了网关代理功能,你同样可以方便的查看本节点或者其他节点的数据,它就像你访问本地资源一样。
```c#
services.AddCap(x =>
{
//...
// 注册 Dashboard
x.UseDashboard();
// 注册节点到 Consul
x.UseDiscovery(d =>
{
d.DiscoveryServerHostName = "localhost";
d.DiscoveryServerPort = 8500;
d.CurrentNodeHostName = "localhost";
d.CurrentNodePort = 5800;
d.NodeId = 1;
d.NodeName = "CAP No.1 Node";
});
});
```
![dashboard](http://images2017.cnblogs.com/blog/250417/201710/250417-20171004220827302-189215107.png)
![received](http://images2017.cnblogs.com/blog/250417/201710/250417-20171004220934115-1107747665.png)
![subscibers](http://images2017.cnblogs.com/blog/250417/201710/250417-20171004220949193-884674167.png)
![nodes](http://images2017.cnblogs.com/blog/250417/201710/250417-20171004221001880-1162918362.png)
## 贡献 ## 贡献
贡献的最简单的方法之一就是是参与讨论和讨论问题(issue)。你也可以通过提交的 Pull Request 代码变更作出贡献。 贡献的最简单的方法之一就是是参与讨论和讨论问题(issue)。你也可以通过提交的 Pull Request 代码变更作出贡献。
......
...@@ -4,7 +4,7 @@ environment: ...@@ -4,7 +4,7 @@ environment:
BUILDING_ON_PLATFORM: win BUILDING_ON_PLATFORM: win
BuildEnvironment: appveyor BuildEnvironment: appveyor
Cap_SqlServer_ConnectionStringTemplate: Server=(local)\SQL2014;Database={0};User ID=sa;Password=Password12! Cap_SqlServer_ConnectionStringTemplate: Server=(local)\SQL2014;Database={0};User ID=sa;Password=Password12!
Cap_MySql_ConnectionStringTemplate: Server=localhost;Database={0};Uid=root;Pwd=Password12! Cap_MySql_ConnectionStringTemplate: Server=localhost;Database={0};Uid=root;Pwd=Password12!;Allow User Variables=True
Cap_PostgreSql_ConnectionStringTemplate: Server=localhost;Database={0};UserId=postgres;Password=Password12! Cap_PostgreSql_ConnectionStringTemplate: Server=localhost;Database={0};UserId=postgres;Password=Password12!
services: services:
- mssql2014 - mssql2014
......
<Project> <Project>
<PropertyGroup> <PropertyGroup>
<VersionMajor>2</VersionMajor> <VersionMajor>2</VersionMajor>
<VersionMinor>0</VersionMinor> <VersionMinor>1</VersionMinor>
<VersionPatch>2</VersionPatch> <VersionPatch>0</VersionPatch>
<VersionQuality></VersionQuality> <VersionQuality></VersionQuality>
<VersionPrefix>$(VersionMajor).$(VersionMinor).$(VersionPatch)</VersionPrefix> <VersionPrefix>$(VersionMajor).$(VersionMinor).$(VersionPatch)</VersionPrefix>
</PropertyGroup> </PropertyGroup>
......
using Microsoft.EntityFrameworkCore;
namespace Sample.Kafka.SqlServer
{
public class AppDbContext : DbContext
{
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
//optionsBuilder.UseSqlServer("Server=192.168.2.206;Initial Catalog=Sample.Kafka.SqlServer;User Id=cmswuliu;Password=h7xY81agBn*Veiu3;MultipleActiveResultSets=True");
optionsBuilder.UseSqlServer("Server=192.168.2.206;Initial Catalog=TestCap;User Id=cmswuliu;Password=h7xY81agBn*Veiu3;MultipleActiveResultSets=True");
}
}
}
using System;
using DotNetCore.CAP.Abstractions;
using DotNetCore.CAP.Models;
using Newtonsoft.Json;
namespace Sample.RabbitMQ.SqlServer
{
public class MessageContent : CapMessage
{
[JsonProperty("id")]
public override string Id { get; set; }
[JsonProperty("createdTime")]
public override DateTime Timestamp { get; set; }
[JsonProperty("msgBody")]
public override string Content { get; set; }
[JsonProperty("callbackTopicName")]
public override string CallbackName { get; set; }
}
public class MyMessagePacker : IMessagePacker
{
private readonly IContentSerializer _serializer;
public MyMessagePacker(IContentSerializer serializer)
{
_serializer = serializer;
}
public string Pack(CapMessage obj)
{
var content = new MessageContent
{
Id = obj.Id,
Content = obj.Content,
CallbackName = obj.CallbackName,
Timestamp = obj.Timestamp
};
return _serializer.Serialize(content);
}
public CapMessage UnPack(string packingMessage)
{
return _serializer.DeSerialize<MessageContent>(packingMessage);
}
}
}
using System;
using System.Diagnostics;
using System.Threading.Tasks;
using DotNetCore.CAP;
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using Newtonsoft.Json;
namespace Sample.Kafka.SqlServer.Controllers
{
public class Person
{
[JsonProperty("id")]
public string Id { get; set; }
[JsonProperty("uname")]
public string Name { get; set; }
public HAHA Haha { get; set; }
public override string ToString()
{
return "Name:" + Name + ";Id:" + Id + "Haha:" + Haha?.ToString();
}
}
public class HAHA
{
[JsonProperty("id")]
public string Id { get; set; }
[JsonProperty("uname")]
public string Name { get; set; }
public override string ToString()
{
return "Name:" + Name + ";Id:" + Id;
}
}
[Route("api/[controller]")]
public class ValuesController : Controller, ICapSubscribe
{
private readonly ICapPublisher _capBus;
private readonly AppDbContext _dbContext;
public ValuesController(ICapPublisher producer, AppDbContext dbContext)
{
_capBus = producer;
_dbContext = dbContext;
}
[Route("~/publish")]
public IActionResult PublishMessage()
{
var p = new Person
{
Id = Guid.NewGuid().ToString(),
Name = "杨晓东",
Haha = new HAHA
{
Id = Guid.NewGuid().ToString(),
Name = "1-1杨晓东",
}
};
_capBus.Publish("wl.yxd.test", p, "wl.yxd.test.callback");
//_capBus.Publish("wl.cj.test", p);
return Ok();
}
[CapSubscribe("wl.yxd.test.callback")]
public void KafkaTestCallback(Person p)
{
Console.WriteLine("回调内容:" + p);
}
[CapSubscribe("wl.cj.test")]
public string KafkaTestReceived(Person person)
{
Console.WriteLine(person);
Debug.WriteLine(person);
return "this is callback message";
}
[Route("~/publishWithTrans")]
public async Task<IActionResult> PublishMessageWithTransaction()
{
using (var trans = await _dbContext.Database.BeginTransactionAsync())
{
await _capBus.PublishAsync("sample.rabbitmq.mysql", "");
trans.Commit();
}
return Ok();
}
[CapSubscribe("sample.rabbitmq.mysql33333", Group = "Test.Group")]
public void KafkaTest22(Person person)
{
var aa = _dbContext.Database;
_dbContext.Dispose();
Console.WriteLine("[sample.kafka.sqlserver] message received " + person.ToString());
Debug.WriteLine("[sample.kafka.sqlserver] message received " + person.ToString());
}
//[CapSubscribe("sample.rabbitmq.mysql22222")]
//public void KafkaTest22(DateTime time)
//{
// Console.WriteLine("[sample.kafka.sqlserver] message received " + time.ToString());
// Debug.WriteLine("[sample.kafka.sqlserver] message received " + time.ToString());
//}
[CapSubscribe("sample.rabbitmq.mysql22222")]
public async Task<DateTime> KafkaTest33(DateTime time)
{
Console.WriteLine("[sample.kafka.sqlserver] message received " + time.ToString());
Debug.WriteLine("[sample.kafka.sqlserver] message received " + time.ToString());
return await Task.FromResult(time);
}
[NonAction]
[CapSubscribe("sample.kafka.sqlserver3")]
[CapSubscribe("sample.kafka.sqlserver4")]
public void KafkaTest()
{
Console.WriteLine("[sample.kafka.sqlserver] message received");
Debug.WriteLine("[sample.kafka.sqlserver] message received");
}
}
}
\ No newline at end of file
using System.IO;
using Microsoft.AspNetCore;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
namespace Sample.Kafka.SqlServer
{
public class Program
{
public static void Main(string[] args)
{
BuildWebHost(args).Run();
}
public static IWebHost BuildWebHost(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.UseStartup<Startup>()
.Build();
}
}
\ No newline at end of file
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>netcoreapp2.0</TargetFramework>
<AssemblyName>Sample.Kafka.SqlServer</AssemblyName>
<WarningsAsErrors>NU1701</WarningsAsErrors>
<NoWarn>NU1701</NoWarn>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.All" Version="2.0.0" />
</ItemGroup>
<ItemGroup>
<DotNetCliToolReference Include="Microsoft.EntityFrameworkCore.Tools.DotNet" Version="2.0.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\src\DotNetCore.CAP.Kafka\DotNetCore.CAP.Kafka.csproj" />
<ProjectReference Include="..\..\src\DotNetCore.CAP.SqlServer\DotNetCore.CAP.SqlServer.csproj" />
<ProjectReference Include="..\..\src\DotNetCore.CAP\DotNetCore.CAP.csproj" />
</ItemGroup>
</Project>
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Sample.RabbitMQ.SqlServer;
namespace Sample.Kafka.SqlServer
{
public class Startup
{
public void ConfigureServices(IServiceCollection services)
{
services.AddDbContext<AppDbContext>();
services.AddCap(x =>
{
x.UseEntityFramework<AppDbContext>();
x.UseKafka("192.168.2.215:9092");
x.UseDashboard();
//x.UseDiscovery(d =>
//{
// d.DiscoveryServerHostName = "localhost";
// d.DiscoveryServerPort = 8500;
// d.CurrentNodeHostName = "localhost";
// d.CurrentNodePort = 5820;
// d.NodeName = "CAP 2号节点";
//});
}).AddMessagePacker<MyMessagePacker>();
services.AddMvc();
}
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
app.UseMvc();
app.UseCap();
}
}
}
\ No newline at end of file
...@@ -10,8 +10,8 @@ namespace Sample.RabbitMQ.MySql ...@@ -10,8 +10,8 @@ namespace Sample.RabbitMQ.MySql
{ {
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder) protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{ {
//optionsBuilder.UseMySql("Server=localhost;Database=Sample.RabbitMQ.MySql;UserId=root;Password=123123;"); optionsBuilder.UseMySql("Server=localhost;Database=Sample.RabbitMQ.MySql;UserId=root;Password=123123;Allow User Variables=True");
optionsBuilder.UseMySql("Server=192.168.2.206;Database=Sample.RabbitMQ.MySql;UserId=root;Password=123123;"); //optionsBuilder.UseMySql("Server=192.168.2.206;Database=Sample.RabbitMQ.MySql;UserId=root;Password=123123;");
} }
} }
} }
...@@ -24,7 +24,7 @@ namespace Sample.RabbitMQ.MySql.Controllers ...@@ -24,7 +24,7 @@ namespace Sample.RabbitMQ.MySql.Controllers
public IActionResult PublishMessage() public IActionResult PublishMessage()
{ {
_capBus.Publish("sample.rabbitmq.mysql", DateTime.Now); _capBus.Publish("sample.rabbitmq.mysql", DateTime.Now);
return Ok(); return Ok();
} }
......
...@@ -3,6 +3,7 @@ using System.Collections.Generic; ...@@ -3,6 +3,7 @@ using System.Collections.Generic;
using System.IO; using System.IO;
using System.Linq; using System.Linq;
using System.Threading.Tasks; using System.Threading.Tasks;
using Microsoft.AspNetCore;
using Microsoft.AspNetCore.Builder; using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting; using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration; using Microsoft.Extensions.Configuration;
...@@ -13,20 +14,12 @@ namespace Sample.RabbitMQ.MySql ...@@ -13,20 +14,12 @@ namespace Sample.RabbitMQ.MySql
{ {
public static void Main(string[] args) public static void Main(string[] args)
{ {
var config = new ConfigurationBuilder() BuildWebHost(args).Run();
.AddCommandLine(args) }
.AddEnvironmentVariables("ASPNETCORE_")
.Build();
var host = new WebHostBuilder() public static IWebHost BuildWebHost(string[] args) =>
.UseConfiguration(config) WebHost.CreateDefaultBuilder(args)
.UseKestrel()
.UseContentRoot(Directory.GetCurrentDirectory())
.UseIISIntegration()
.UseStartup<Startup>() .UseStartup<Startup>()
.Build(); .Build();
host.Run();
}
} }
} }
...@@ -10,14 +10,8 @@ ...@@ -10,14 +10,8 @@
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<PackageReference Include="Microsoft.AspNetCore" Version="2.0.0" /> <PackageReference Include="Microsoft.AspNetCore.All" Version="2.0.0" />
<PackageReference Include="Microsoft.AspNetCore.Mvc" Version="2.0.0" /> <PackageReference Include="Pomelo.EntityFrameworkCore.MySql" Version="2.0.0" />
<PackageReference Include="Microsoft.EntityFrameworkCore" Version="2.0.0" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="2.0.0" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="2.0.0" />
<PackageReference Include="Microsoft.Extensions.Configuration.CommandLine" Version="2.0.0" />
<PackageReference Include="Microsoft.Extensions.Logging.Debug" Version="2.0.0" />
<PackageReference Include="Pomelo.EntityFrameworkCore.MySql" Version="2.0.0-rtm-10056" />
</ItemGroup> </ItemGroup>
<ItemGroup> <ItemGroup>
<DotNetCliToolReference Include="Microsoft.EntityFrameworkCore.Tools.DotNet" Version="2.0.0" /> <DotNetCliToolReference Include="Microsoft.EntityFrameworkCore.Tools.DotNet" Version="2.0.0" />
......
...@@ -18,11 +18,8 @@ namespace Sample.RabbitMQ.MySql ...@@ -18,11 +18,8 @@ namespace Sample.RabbitMQ.MySql
services.AddCap(x => services.AddCap(x =>
{ {
x.UseEntityFramework<AppDbContext>(); x.UseEntityFramework<AppDbContext>();
x.UseRabbitMQ(y => { x.UseRabbitMQ("localhost");
y.HostName = "192.168.2.206"; x.UseDashboard();
y.UserName = "admin";
y.Password = "123123";
});
}); });
services.AddMvc(); services.AddMvc();
......
...@@ -10,7 +10,7 @@ namespace Sample.RabbitMQ.PostgreSql ...@@ -10,7 +10,7 @@ namespace Sample.RabbitMQ.PostgreSql
{ {
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder) protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{ {
optionsBuilder.UseNpgsql("Server=localhost;Database=Sample.RabbitMQ.PostgreSql;UserId=postgre;Password=123123;"); optionsBuilder.UseNpgsql("Server=localhost;Database=Sample.RabbitMQ.PostgreSql;UserId=postgres;Password=123123;");
} }
} }
} }
...@@ -4,10 +4,7 @@ using System.Linq; ...@@ -4,10 +4,7 @@ using System.Linq;
using System.Threading.Tasks; using System.Threading.Tasks;
using Microsoft.AspNetCore.Builder; using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting; using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace Sample.RabbitMQ.PostgreSql namespace Sample.RabbitMQ.PostgreSql
{ {
...@@ -16,7 +13,21 @@ namespace Sample.RabbitMQ.PostgreSql ...@@ -16,7 +13,21 @@ namespace Sample.RabbitMQ.PostgreSql
// This method gets called by the runtime. Use this method to add services to the container. // This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services) public void ConfigureServices(IServiceCollection services)
{ {
services.AddDbContext<AppDbContext>();
services.AddCap(x =>
{
x.UseEntityFramework<AppDbContext>();
x.UseRabbitMQ("localhost");
x.UseDashboard();
x.UseDiscovery(d =>
{
d.DiscoveryServerHostName = "localhost";
d.DiscoveryServerPort = 8500;
d.CurrentNodeHostName = "localhost";
d.CurrentNodePort = 5800;
d.NodeName = "CAP 一号节点";
});
});
services.AddMvc(); services.AddMvc();
} }
...@@ -24,6 +35,8 @@ namespace Sample.RabbitMQ.PostgreSql ...@@ -24,6 +35,8 @@ namespace Sample.RabbitMQ.PostgreSql
public void Configure(IApplicationBuilder app, IHostingEnvironment env) public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{ {
app.UseMvc(); app.UseMvc();
app.UseCap();
} }
} }
} }
using Microsoft.EntityFrameworkCore; using Microsoft.EntityFrameworkCore;
using Sample.RabbitMQ.SqlServer.Controllers;
namespace Sample.RabbitMQ.SqlServer namespace Sample.RabbitMQ.SqlServer
{ {
public class AppDbContext : DbContext public class AppDbContext : DbContext
{ {
public DbSet<Person> Persons { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder) protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{ {
optionsBuilder.UseSqlServer("Server=192.168.2.206;Initial Catalog=TestCap;User Id=cmswuliu;Password=h7xY81agBn*Veiu3;MultipleActiveResultSets=True"); //optionsBuilder.UseSqlServer("Server=192.168.2.206;Initial Catalog=TestCap;User Id=cmswuliu;Password=h7xY81agBn*Veiu3;MultipleActiveResultSets=True");
//optionsBuilder.UseSqlServer("Server=DESKTOP-M9R8T31;Initial Catalog=Sample.Kafka.SqlServer;User Id=sa;Password=P@ssw0rd;MultipleActiveResultSets=True"); optionsBuilder.UseSqlServer("Server=DESKTOP-M9R8T31;Initial Catalog=Sample.Kafka.SqlServer;User Id=sa;Password=P@ssw0rd;MultipleActiveResultSets=True");
} }
} }
} }
...@@ -2,12 +2,14 @@ ...@@ -2,12 +2,14 @@
using System.Diagnostics; using System.Diagnostics;
using System.Threading.Tasks; using System.Threading.Tasks;
using DotNetCore.CAP; using DotNetCore.CAP;
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc; using Microsoft.AspNetCore.Mvc;
namespace Sample.RabbitMQ.SqlServer.Controllers namespace Sample.RabbitMQ.SqlServer.Controllers
{ {
public class Person public class Person
{ {
public int Id { get; set; }
public string Name { get; set; } public string Name { get; set; }
public int Age { get; set; } public int Age { get; set; }
...@@ -33,12 +35,17 @@ namespace Sample.RabbitMQ.SqlServer.Controllers ...@@ -33,12 +35,17 @@ namespace Sample.RabbitMQ.SqlServer.Controllers
[Route("~/publish")] [Route("~/publish")]
public IActionResult PublishMessage() public IActionResult PublishMessage()
{ {
using(var trans = _dbContext.Database.BeginTransaction())
{ _capBus.Publish("sample.rabbitmq.sqlserver.order.check", DateTime.Now);
//_capBus.Publish("sample.rabbitmq.mysql22222", DateTime.Now);
_capBus.Publish("sample.rabbitmq.mysql33333", new Person { Name = "宜兴", Age = 11 }); //var person = new Person
trans.Commit(); //{
} // Name = "杨晓东",
// Age = 11,
// Id = 23
//};
//_capBus.Publish("sample.rabbitmq.mysql33333", person);
return Ok(); return Ok();
} }
...@@ -48,13 +55,13 @@ namespace Sample.RabbitMQ.SqlServer.Controllers ...@@ -48,13 +55,13 @@ namespace Sample.RabbitMQ.SqlServer.Controllers
using (var trans = await _dbContext.Database.BeginTransactionAsync()) using (var trans = await _dbContext.Database.BeginTransactionAsync())
{ {
await _capBus.PublishAsync("sample.rabbitmq.mysql", ""); await _capBus.PublishAsync("sample.rabbitmq.mysql", "");
trans.Commit(); trans.Commit();
} }
return Ok(); return Ok();
} }
[CapSubscribe("sample.rabbitmq.mysql33333")] [CapSubscribe("sample.rabbitmq.mysql33333",Group ="Test.Group")]
public void KafkaTest22(Person person) public void KafkaTest22(Person person)
{ {
var aa = _dbContext.Database; var aa = _dbContext.Database;
......
// <auto-generated />
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage;
using Microsoft.EntityFrameworkCore.Storage.Internal;
using Sample.RabbitMQ.SqlServer;
using System;
namespace Sample.RabbitMQ.SqlServer.Migrations
{
[DbContext(typeof(AppDbContext))]
[Migration("20170824130007_AddPersons")]
partial class AddPersons
{
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasAnnotation("ProductVersion", "2.0.0-rtm-26452")
.HasAnnotation("SqlServer:ValueGenerationStrategy", SqlServerValueGenerationStrategy.IdentityColumn);
modelBuilder.Entity("Sample.RabbitMQ.SqlServer.Controllers.Person", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd();
b.Property<int>("Age");
b.Property<string>("Name");
b.HasKey("Id");
b.ToTable("Persons");
});
#pragma warning restore 612, 618
}
}
}
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using System;
using System.Collections.Generic;
namespace Sample.RabbitMQ.SqlServer.Migrations
{
public partial class AddPersons : Migration
{
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable(
name: "Persons",
columns: table => new
{
Id = table.Column<int>(type: "int", nullable: false)
.Annotation("SqlServer:ValueGenerationStrategy", SqlServerValueGenerationStrategy.IdentityColumn),
Age = table.Column<int>(type: "int", nullable: false),
Name = table.Column<string>(type: "nvarchar(max)", nullable: true)
},
constraints: table =>
{
table.PrimaryKey("PK_Persons", x => x.Id);
});
}
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "Persons");
}
}
}
// <auto-generated />
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage;
using Microsoft.EntityFrameworkCore.Storage.Internal;
using Sample.RabbitMQ.SqlServer;
using System;
namespace Sample.RabbitMQ.SqlServer.Migrations
{
[DbContext(typeof(AppDbContext))]
partial class AppDbContextModelSnapshot : ModelSnapshot
{
protected override void BuildModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder
.HasAnnotation("ProductVersion", "2.0.0-rtm-26452")
.HasAnnotation("SqlServer:ValueGenerationStrategy", SqlServerValueGenerationStrategy.IdentityColumn);
modelBuilder.Entity("Sample.RabbitMQ.SqlServer.Controllers.Person", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd();
b.Property<int>("Age");
b.Property<string>("Name");
b.HasKey("Id");
b.ToTable("Persons");
});
#pragma warning restore 612, 618
}
}
}
...@@ -8,28 +8,15 @@ namespace Sample.RabbitMQ.SqlServer ...@@ -8,28 +8,15 @@ namespace Sample.RabbitMQ.SqlServer
{ {
public class Program public class Program
{ {
//var config = new ConfigurationBuilder()
// .AddCommandLine(args)
// .AddEnvironmentVariables("ASPNETCORE_")
// .Build();
//var host = new WebHostBuilder()
// .UseConfiguration(config)
// .UseKestrel()
// .UseContentRoot(Directory.GetCurrentDirectory())
// .UseIISIntegration()
// .UseStartup<Startup>()
// .Build();
//host.Run();
public static void Main(string[] args) public static void Main(string[] args)
{ {
BuildWebHost(args).Run(); BuildWebHost(args).Run();
} }
public static IWebHost BuildWebHost(string[] args) => public static IWebHost BuildWebHost(string[] args) =>
WebHost.CreateDefaultBuilder(args) WebHost.CreateDefaultBuilder(args)
.UseUrls("http://*:5800")
.UseStartup<Startup>() .UseStartup<Startup>()
.Build(); .Build();
......
...@@ -6,14 +6,7 @@ ...@@ -6,14 +6,7 @@
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<PackageReference Include="Microsoft.AspNetCore" Version="2.0.0" /> <PackageReference Include="Microsoft.AspNetCore.All" Version="2.0.0" />
<PackageReference Include="Microsoft.AspNetCore.Mvc" Version="2.0.0" />
<PackageReference Include="Microsoft.EntityFrameworkCore" Version="2.0.0" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="2.0.0" />
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="2.0.0" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="2.0.0" />
<PackageReference Include="Microsoft.Extensions.Configuration.CommandLine" Version="2.0.0" />
<PackageReference Include="Microsoft.Extensions.Logging.Debug" Version="2.0.0" />
</ItemGroup> </ItemGroup>
<ItemGroup> <ItemGroup>
<DotNetCliToolReference Include="Microsoft.EntityFrameworkCore.Tools.DotNet" Version="2.0.0" /> <DotNetCliToolReference Include="Microsoft.EntityFrameworkCore.Tools.DotNet" Version="2.0.0" />
......
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using DotNetCore.CAP;
namespace Sample.RabbitMQ.SqlServer.Services
{
public interface ICmsService
{
void Add();
}
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
namespace Sample.RabbitMQ.SqlServer.Services
{
public interface IOrderService
{
void Check();
}
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using DotNetCore.CAP;
namespace Sample.RabbitMQ.SqlServer.Services.Impl
{
public class CmsService : ICmsService, ICapSubscribe
{
public void Add()
{
throw new NotImplementedException();
}
}
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using DotNetCore.CAP;
namespace Sample.RabbitMQ.SqlServer.Services.Impl
{
public class OrderService : IOrderService, ICapSubscribe
{
[CapSubscribe("sample.rabbitmq.sqlserver.order.check")]
public void Check()
{
Console.WriteLine("out");
}
}
}
...@@ -2,6 +2,8 @@ ...@@ -2,6 +2,8 @@
using Microsoft.AspNetCore.Hosting; using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging; using Microsoft.Extensions.Logging;
using Sample.RabbitMQ.SqlServer.Services;
using Sample.RabbitMQ.SqlServer.Services.Impl;
namespace Sample.RabbitMQ.SqlServer namespace Sample.RabbitMQ.SqlServer
{ {
...@@ -11,13 +13,23 @@ namespace Sample.RabbitMQ.SqlServer ...@@ -11,13 +13,23 @@ namespace Sample.RabbitMQ.SqlServer
{ {
services.AddDbContext<AppDbContext>(); services.AddDbContext<AppDbContext>();
services.AddScoped<IOrderService, OrderService>();
services.AddTransient<ICmsService, CmsService>();
services.AddCap(x => services.AddCap(x =>
{ {
x.UseEntityFramework<AppDbContext>(); x.UseEntityFramework<AppDbContext>();
x.UseRabbitMQ(y=> { x.UseRabbitMQ("localhost");
y.HostName = "192.168.2.206"; x.UseDashboard();
y.UserName = "admin"; x.UseDiscovery(d =>
y.Password = "123123"; {
d.DiscoveryServerHostName = "localhost";
d.DiscoveryServerPort = 8500;
d.CurrentNodeHostName = "192.168.1.11";
d.CurrentNodePort = 5800;
d.NodeName = "CAP Node Windows";
d.NodeId = 1;
}); });
}); });
......
...@@ -16,12 +16,16 @@ namespace DotNetCore.CAP ...@@ -16,12 +16,16 @@ namespace DotNetCore.CAP
public void AddServices(IServiceCollection services) public void AddServices(IServiceCollection services)
{ {
services.AddSingleton<CapMessageQueueMakerService>();
var kafkaOptions = new KafkaOptions(); var kafkaOptions = new KafkaOptions();
_configure?.Invoke(kafkaOptions); _configure?.Invoke(kafkaOptions);
services.AddSingleton(kafkaOptions); services.AddSingleton(kafkaOptions);
services.AddSingleton<IConsumerClientFactory, KafkaConsumerClientFactory>(); services.AddSingleton<IConsumerClientFactory, KafkaConsumerClientFactory>();
services.AddSingleton<IQueueExecutor, PublishQueueExecutor>(); services.AddSingleton<IQueueExecutor, PublishQueueExecutor>();
services.AddSingleton<IPublishExecutor, PublishQueueExecutor>();
services.AddSingleton<ConnectionPool>();
} }
} }
} }
\ No newline at end of file
using System; using System;
using System.Collections.Concurrent;
using System.Collections.Generic; using System.Collections.Generic;
using System.Linq; using System.Linq;
...@@ -10,46 +11,51 @@ namespace DotNetCore.CAP ...@@ -10,46 +11,51 @@ namespace DotNetCore.CAP
/// </summary> /// </summary>
public class KafkaOptions public class KafkaOptions
{ {
public KafkaOptions()
{
MainConfig = new Dictionary<string, object>();
}
/// <summary> /// <summary>
/// librdkafka configuration parameters (refer to https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md). /// librdkafka configuration parameters (refer to https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md).
/// <para> /// <para>
/// Topic configuration parameters are specified via the "default.topic.config" sub-dictionary config parameter. /// Topic configuration parameters are specified via the "default.topic.config" sub-dictionary config parameter.
/// </para> /// </para>
/// </summary> /// </summary>
public readonly IDictionary<string, object> MainConfig; public readonly ConcurrentDictionary<string, object> MainConfig;
private IEnumerable<KeyValuePair<string, object>> _kafkaConfig;
public KafkaOptions()
{
MainConfig = new ConcurrentDictionary<string, object>();
}
/// <summary> /// <summary>
/// The `bootstrap.servers` item config of <see cref="MainConfig"/>. /// Producer connection pool size, default is 10
/// </summary>
public int ConnectionPoolSize { get; set; } = 10;
/// <summary>
/// The `bootstrap.servers` item config of <see cref="MainConfig" />.
/// <para> /// <para>
/// Initial list of brokers as a CSV list of broker host or host:port. /// Initial list of brokers as a CSV list of broker host or host:port.
/// </para> /// </para>
/// </summary> /// </summary>
public string Servers { get; set; } public string Servers { get; set; }
internal IEnumerable<KeyValuePair<string, object>> AskafkaConfig() internal IEnumerable<KeyValuePair<string, object>> AsKafkaConfig()
{ {
if (MainConfig.ContainsKey("bootstrap.servers")) if (_kafkaConfig == null)
{ {
return MainConfig.AsEnumerable(); if (string.IsNullOrWhiteSpace(Servers))
throw new ArgumentNullException(nameof(Servers));
MainConfig["bootstrap.servers"] = Servers;
MainConfig["queue.buffering.max.ms"] = "10";
MainConfig["socket.blocking.max.ms"] = "10";
MainConfig["enable.auto.commit"] = "false";
MainConfig["log.connection.close"] = "false";
_kafkaConfig = MainConfig.AsEnumerable();
} }
return _kafkaConfig;
if (string.IsNullOrWhiteSpace(Servers))
{
throw new ArgumentNullException(nameof(Servers));
}
MainConfig.Add("bootstrap.servers", Servers);
MainConfig["queue.buffering.max.ms"] = "10";
MainConfig["socket.blocking.max.ms"] = "10";
MainConfig["enable.auto.commit"] = "false";
return MainConfig.AsEnumerable();
} }
} }
} }
\ No newline at end of file
...@@ -9,18 +9,17 @@ namespace Microsoft.Extensions.DependencyInjection ...@@ -9,18 +9,17 @@ namespace Microsoft.Extensions.DependencyInjection
/// <summary> /// <summary>
/// Configuration to use kafka in CAP. /// Configuration to use kafka in CAP.
/// </summary> /// </summary>
/// <param name="options">CAP configuration options</param>
/// <param name="bootstrapServers">Kafka bootstrap server urls.</param> /// <param name="bootstrapServers">Kafka bootstrap server urls.</param>
public static CapOptions UseKafka(this CapOptions options, string bootstrapServers) public static CapOptions UseKafka(this CapOptions options, string bootstrapServers)
{ {
return options.UseKafka(opt => return options.UseKafka(opt => { opt.Servers = bootstrapServers; });
{
opt.Servers = bootstrapServers;
});
} }
/// <summary> /// <summary>
/// Configuration to use kafka in CAP. /// Configuration to use kafka in CAP.
/// </summary> /// </summary>
/// <param name="options">CAP configuration options</param>
/// <param name="configure">Provides programmatic configuration for the kafka .</param> /// <param name="configure">Provides programmatic configuration for the kafka .</param>
/// <returns></returns> /// <returns></returns>
public static CapOptions UseKafka(this CapOptions options, Action<KafkaOptions> configure) public static CapOptions UseKafka(this CapOptions options, Action<KafkaOptions> configure)
......
using System;
using System.Collections.Concurrent;
using System.Diagnostics;
using System.Threading;
using Confluent.Kafka;
namespace DotNetCore.CAP.Kafka
{
public class ConnectionPool : IConnectionPool, IDisposable
{
private readonly Func<Producer> _activator;
private readonly ConcurrentQueue<Producer> _pool = new ConcurrentQueue<Producer>();
private int _count;
private int _maxSize;
public ConnectionPool(KafkaOptions options)
{
_maxSize = options.ConnectionPoolSize;
_activator = CreateActivator(options);
}
Producer IConnectionPool.Rent()
{
return Rent();
}
bool IConnectionPool.Return(Producer connection)
{
return Return(connection);
}
public void Dispose()
{
_maxSize = 0;
while (_pool.TryDequeue(out var context))
context.Dispose();
}
private static Func<Producer> CreateActivator(KafkaOptions options)
{
return () => new Producer(options.AsKafkaConfig());
}
public virtual Producer Rent()
{
if (_pool.TryDequeue(out var connection))
{
Interlocked.Decrement(ref _count);
Debug.Assert(_count >= 0);
return connection;
}
connection = _activator();
return connection;
}
public virtual bool Return(Producer connection)
{
if (Interlocked.Increment(ref _count) <= _maxSize)
{
_pool.Enqueue(connection);
return true;
}
Interlocked.Decrement(ref _count);
Debug.Assert(_maxSize == 0 || _pool.Count <= _maxSize);
return false;
}
}
}
\ No newline at end of file
...@@ -9,12 +9,12 @@ ...@@ -9,12 +9,12 @@
</PropertyGroup> </PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
<WarningsAsErrors>NU1605</WarningsAsErrors> <WarningsAsErrors>NU1605;NU1701</WarningsAsErrors>
<NoWarn>NU1701</NoWarn> <NoWarn>NU1701</NoWarn>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<PackageReference Include="Confluent.Kafka" Version="0.11.0" /> <PackageReference Include="Confluent.Kafka" Version="0.11.2" />
</ItemGroup> </ItemGroup>
<ItemGroup> <ItemGroup>
......

using Confluent.Kafka;
namespace DotNetCore.CAP.Kafka
{
public interface IConnectionPool
{
Producer Rent();
bool Return(Producer context);
}
}
\ No newline at end of file
...@@ -13,12 +13,6 @@ namespace DotNetCore.CAP.Kafka ...@@ -13,12 +13,6 @@ namespace DotNetCore.CAP.Kafka
private readonly KafkaOptions _kafkaOptions; private readonly KafkaOptions _kafkaOptions;
private Consumer<Null, string> _consumerClient; private Consumer<Null, string> _consumerClient;
public event EventHandler<MessageContext> OnMessageReceieved;
public event EventHandler<string> OnError;
public IDeserializer<string> StringDeserializer { get; set; }
public KafkaConsumerClient(string groupId, KafkaOptions options) public KafkaConsumerClient(string groupId, KafkaOptions options)
{ {
_groupId = groupId; _groupId = groupId;
...@@ -26,15 +20,19 @@ namespace DotNetCore.CAP.Kafka ...@@ -26,15 +20,19 @@ namespace DotNetCore.CAP.Kafka
StringDeserializer = new StringDeserializer(Encoding.UTF8); StringDeserializer = new StringDeserializer(Encoding.UTF8);
} }
public IDeserializer<string> StringDeserializer { get; set; }
public event EventHandler<MessageContext> OnMessageReceived;
public event EventHandler<string> OnError;
public void Subscribe(IEnumerable<string> topics) public void Subscribe(IEnumerable<string> topics)
{ {
if (topics == null) if (topics == null)
throw new ArgumentNullException(nameof(topics)); throw new ArgumentNullException(nameof(topics));
if (_consumerClient == null) if (_consumerClient == null)
{
InitKafkaClient(); InitKafkaClient();
}
//_consumerClient.Assign(topics.Select(x=> new TopicPartition(x, 0))); //_consumerClient.Assign(topics.Select(x=> new TopicPartition(x, 0)));
_consumerClient.Subscribe(topics); _consumerClient.Subscribe(topics);
...@@ -47,6 +45,7 @@ namespace DotNetCore.CAP.Kafka ...@@ -47,6 +45,7 @@ namespace DotNetCore.CAP.Kafka
cancellationToken.ThrowIfCancellationRequested(); cancellationToken.ThrowIfCancellationRequested();
_consumerClient.Poll(timeout); _consumerClient.Poll(timeout);
} }
// ReSharper disable once FunctionNeverReturns
} }
public void Commit() public void Commit()
...@@ -54,6 +53,11 @@ namespace DotNetCore.CAP.Kafka ...@@ -54,6 +53,11 @@ namespace DotNetCore.CAP.Kafka
_consumerClient.CommitAsync(); _consumerClient.CommitAsync();
} }
public void Reject()
{
// Ignore, Kafka will not commit offset when not commit.
}
public void Dispose() public void Dispose()
{ {
_consumerClient.Dispose(); _consumerClient.Dispose();
...@@ -63,15 +67,23 @@ namespace DotNetCore.CAP.Kafka ...@@ -63,15 +67,23 @@ namespace DotNetCore.CAP.Kafka
private void InitKafkaClient() private void InitKafkaClient()
{ {
_kafkaOptions.MainConfig.Add("group.id", _groupId); _kafkaOptions.MainConfig["group.id"] = _groupId;
var config = _kafkaOptions.AskafkaConfig(); var config = _kafkaOptions.AsKafkaConfig();
_consumerClient = new Consumer<Null, string>(config, null, StringDeserializer); _consumerClient = new Consumer<Null, string>(config, null, StringDeserializer);
_consumerClient.OnConsumeError += ConsumerClient_OnConsumeError;
_consumerClient.OnMessage += ConsumerClient_OnMessage; _consumerClient.OnMessage += ConsumerClient_OnMessage;
_consumerClient.OnError += ConsumerClient_OnError; _consumerClient.OnError += ConsumerClient_OnError;
} }
private void ConsumerClient_OnConsumeError(object sender, Message e)
{
var message = e.Deserialize<Null, string>(null, StringDeserializer);
OnError?.Invoke(sender, $"An error occurred during consume the message; Topic:'{e.Topic}'," +
$"Message:'{message.Value}', Reason:'{e.Error}'.");
}
private void ConsumerClient_OnMessage(object sender, Message<Null, string> e) private void ConsumerClient_OnMessage(object sender, Message<Null, string> e)
{ {
var message = new MessageContext var message = new MessageContext
...@@ -81,12 +93,12 @@ namespace DotNetCore.CAP.Kafka ...@@ -81,12 +93,12 @@ namespace DotNetCore.CAP.Kafka
Content = e.Value Content = e.Value
}; };
OnMessageReceieved?.Invoke(sender, message); OnMessageReceived?.Invoke(sender, message);
} }
private void ConsumerClient_OnError(object sender, Error e) private void ConsumerClient_OnError(object sender, Error e)
{ {
OnError?.Invoke(sender, e.Reason); OnError?.Invoke(sender, e.ToString());
} }
#endregion private methods #endregion private methods
......
using System; using System;
using System.Text; using System.Text;
using System.Threading.Tasks; using System.Threading.Tasks;
using Confluent.Kafka;
using DotNetCore.CAP.Processor.States; using DotNetCore.CAP.Processor.States;
using Microsoft.Extensions.Logging; using Microsoft.Extensions.Logging;
...@@ -9,51 +8,54 @@ namespace DotNetCore.CAP.Kafka ...@@ -9,51 +8,54 @@ namespace DotNetCore.CAP.Kafka
{ {
internal class PublishQueueExecutor : BasePublishQueueExecutor internal class PublishQueueExecutor : BasePublishQueueExecutor
{ {
private readonly ConnectionPool _connectionPool;
private readonly ILogger _logger; private readonly ILogger _logger;
private readonly KafkaOptions _kafkaOptions;
public PublishQueueExecutor( public PublishQueueExecutor(
CapOptions options, CapOptions options,
IStateChanger stateChanger, IStateChanger stateChanger,
KafkaOptions kafkaOptions, ConnectionPool connectionPool,
ILogger<PublishQueueExecutor> logger) ILogger<PublishQueueExecutor> logger)
: base(options, stateChanger, logger) : base(options, stateChanger, logger)
{ {
_logger = logger; _logger = logger;
_kafkaOptions = kafkaOptions; _connectionPool = connectionPool;
} }
public override Task<OperateResult> PublishAsync(string keyName, string content) public override async Task<OperateResult> PublishAsync(string keyName, string content)
{ {
var producer = _connectionPool.Rent();
try try
{ {
var config = _kafkaOptions.AskafkaConfig();
var contentBytes = Encoding.UTF8.GetBytes(content); var contentBytes = Encoding.UTF8.GetBytes(content);
using (var producer = new Producer(config))
var message = await producer.ProduceAsync(keyName, null, contentBytes);
if (!message.Error.HasError)
{ {
var message = producer.ProduceAsync(keyName, null, contentBytes).Result; _logger.LogDebug($"kafka topic message [{keyName}] has been published.");
if (!message.Error.HasError) return OperateResult.Success;
{
_logger.LogDebug($"kafka topic message [{keyName}] has been published.");
return Task.FromResult(OperateResult.Success);
}
else
{
return Task.FromResult(OperateResult.Failed(new OperateError
{
Code = message.Error.Code.ToString(),
Description = message.Error.Reason
}));
}
} }
return OperateResult.Failed(new OperateError
{
Code = message.Error.Code.ToString(),
Description = message.Error.Reason
});
} }
catch (Exception ex) catch (Exception ex)
{ {
_logger.LogError($"kafka topic message [{keyName}] has benn raised an exception of sending. the exception is: {ex.Message}"); _logger.LogError(ex,
$"An error occurred during sending the topic message to kafka. Topic:[{keyName}], Exception: {ex.Message}");
return Task.FromResult(OperateResult.Failed(ex)); return OperateResult.Failed(ex);
}
finally
{
var returned = _connectionPool.Return(producer);
if (!returned)
producer.Dispose();
} }
} }
} }
......
...@@ -6,7 +6,7 @@ namespace DotNetCore.CAP ...@@ -6,7 +6,7 @@ namespace DotNetCore.CAP
public class EFOptions public class EFOptions
{ {
/// <summary> /// <summary>
/// EF dbcontext type. /// EF db context type.
/// </summary> /// </summary>
internal Type DbContextType { get; set; } internal Type DbContextType { get; set; }
} }
......
...@@ -18,32 +18,29 @@ namespace DotNetCore.CAP ...@@ -18,32 +18,29 @@ namespace DotNetCore.CAP
public void AddServices(IServiceCollection services) public void AddServices(IServiceCollection services)
{ {
services.AddSingleton<CapDatabaseStorageMarkerService>();
services.AddSingleton<IStorage, MySqlStorage>(); services.AddSingleton<IStorage, MySqlStorage>();
services.AddScoped<IStorageConnection, MySqlStorageConnection>(); services.AddSingleton<IStorageConnection, MySqlStorageConnection>();
services.AddScoped<ICapPublisher, CapPublisher>(); services.AddScoped<ICapPublisher, CapPublisher>();
services.AddTransient<ICallbackPublisher, CapPublisher>(); services.AddScoped<ICallbackPublisher, CapPublisher>();
services.AddTransient<IAdditionalProcessor, DefaultAdditionalProcessor>(); services.AddTransient<IAdditionalProcessor, DefaultAdditionalProcessor>();
var mysqlOptions = new MySqlOptions(); var mysqlOptions = new MySqlOptions();
_configure(mysqlOptions); _configure(mysqlOptions);
if (mysqlOptions.DbContextType != null) if (mysqlOptions.DbContextType != null)
{
services.AddSingleton(x => services.AddSingleton(x =>
{ {
using (var scope = x.CreateScope()) using (var scope = x.CreateScope())
{ {
var provider = scope.ServiceProvider; var provider = scope.ServiceProvider;
var dbContext = (DbContext)provider.GetService(mysqlOptions.DbContextType); var dbContext = (DbContext) provider.GetService(mysqlOptions.DbContextType);
mysqlOptions.ConnectionString = dbContext.Database.GetDbConnection().ConnectionString; mysqlOptions.ConnectionString = dbContext.Database.GetDbConnection().ConnectionString;
return mysqlOptions; return mysqlOptions;
} }
}); });
}
else else
{
services.AddSingleton(mysqlOptions); services.AddSingleton(mysqlOptions);
}
} }
} }
} }
\ No newline at end of file
// ReSharper disable once CheckNamespace // ReSharper disable once CheckNamespace
namespace DotNetCore.CAP namespace DotNetCore.CAP
{ {
public class MySqlOptions : EFOptions public class MySqlOptions : EFOptions
......
...@@ -9,10 +9,7 @@ namespace Microsoft.Extensions.DependencyInjection ...@@ -9,10 +9,7 @@ namespace Microsoft.Extensions.DependencyInjection
{ {
public static CapOptions UseMySql(this CapOptions options, string connectionString) public static CapOptions UseMySql(this CapOptions options, string connectionString)
{ {
return options.UseMySql(opt => return options.UseMySql(opt => { opt.ConnectionString = connectionString; });
{
opt.ConnectionString = connectionString;
});
} }
public static CapOptions UseMySql(this CapOptions options, Action<MySqlOptions> configure) public static CapOptions UseMySql(this CapOptions options, Action<MySqlOptions> configure)
...@@ -27,10 +24,7 @@ namespace Microsoft.Extensions.DependencyInjection ...@@ -27,10 +24,7 @@ namespace Microsoft.Extensions.DependencyInjection
public static CapOptions UseEntityFramework<TContext>(this CapOptions options) public static CapOptions UseEntityFramework<TContext>(this CapOptions options)
where TContext : DbContext where TContext : DbContext
{ {
return options.UseEntityFramework<TContext>(opt => return options.UseEntityFramework<TContext>(opt => { opt.DbContextType = typeof(TContext); });
{
opt.DbContextType = typeof(TContext);
});
} }
public static CapOptions UseEntityFramework<TContext>(this CapOptions options, Action<EFOptions> configure) public static CapOptions UseEntityFramework<TContext>(this CapOptions options, Action<EFOptions> configure)
...@@ -38,7 +32,7 @@ namespace Microsoft.Extensions.DependencyInjection ...@@ -38,7 +32,7 @@ namespace Microsoft.Extensions.DependencyInjection
{ {
if (configure == null) throw new ArgumentNullException(nameof(configure)); if (configure == null) throw new ArgumentNullException(nameof(configure));
var efOptions = new EFOptions { DbContextType = typeof(TContext) }; var efOptions = new EFOptions {DbContextType = typeof(TContext)};
configure(efOptions); configure(efOptions);
options.RegisterExtension(new MySqlCapOptionsExtension(configure)); options.RegisterExtension(new MySqlCapOptionsExtension(configure));
......
...@@ -13,9 +13,9 @@ namespace DotNetCore.CAP.MySql ...@@ -13,9 +13,9 @@ namespace DotNetCore.CAP.MySql
{ {
public class CapPublisher : CapPublisherBase, ICallbackPublisher public class CapPublisher : CapPublisherBase, ICallbackPublisher
{ {
private readonly DbContext _dbContext;
private readonly ILogger _logger; private readonly ILogger _logger;
private readonly MySqlOptions _options; private readonly MySqlOptions _options;
private readonly DbContext _dbContext;
public CapPublisher(IServiceProvider provider, public CapPublisher(IServiceProvider provider,
ILogger<CapPublisher> logger, ILogger<CapPublisher> logger,
...@@ -25,10 +25,16 @@ namespace DotNetCore.CAP.MySql ...@@ -25,10 +25,16 @@ namespace DotNetCore.CAP.MySql
_options = options; _options = options;
_logger = logger; _logger = logger;
if (_options.DbContextType != null) if (_options.DbContextType == null) return;
IsUsingEF = true;
_dbContext = (DbContext) ServiceProvider.GetService(_options.DbContextType);
}
public async Task PublishAsync(CapPublishedMessage message)
{
using (var conn = new MySqlConnection(_options.ConnectionString))
{ {
IsUsingEF = true; await conn.ExecuteAsync(PrepareSql(), message);
_dbContext = (DbContext)ServiceProvider.GetService(_options.DbContextType);
} }
} }
...@@ -45,36 +51,33 @@ namespace DotNetCore.CAP.MySql ...@@ -45,36 +51,33 @@ namespace DotNetCore.CAP.MySql
dbContextTransaction = _dbContext.Database.BeginTransaction(IsolationLevel.ReadCommitted); dbContextTransaction = _dbContext.Database.BeginTransaction(IsolationLevel.ReadCommitted);
dbTrans = dbContextTransaction.GetDbTransaction(); dbTrans = dbContextTransaction.GetDbTransaction();
} }
DbTranasaction = dbTrans; DbTransaction = dbTrans;
} }
protected override void Execute(IDbConnection dbConnection, IDbTransaction dbTransaction, CapPublishedMessage message) protected override void Execute(IDbConnection dbConnection, IDbTransaction dbTransaction,
CapPublishedMessage message)
{ {
dbConnection.Execute(PrepareSql(), message, dbTransaction); dbConnection.Execute(PrepareSql(), message, dbTransaction);
_logger.LogInformation("Published Message has been persisted in the database. name:" + message.ToString()); _logger.LogInformation("Published Message has been persisted in the database. name:" + message);
} }
protected override async Task ExecuteAsync(IDbConnection dbConnection, IDbTransaction dbTransaction, CapPublishedMessage message) protected override Task ExecuteAsync(IDbConnection dbConnection, IDbTransaction dbTransaction,
CapPublishedMessage message)
{ {
await dbConnection.ExecuteAsync(PrepareSql(), message, dbTransaction); dbConnection.ExecuteAsync(PrepareSql(), message, dbTransaction);
_logger.LogInformation("Published Message has been persisted in the database. name:" + message.ToString()); _logger.LogInformation("Published Message has been persisted in the database. name:" + message);
}
public async Task PublishAsync(CapPublishedMessage message) return Task.CompletedTask;
{
using (var conn = new MySqlConnection(_options.ConnectionString))
{
await conn.ExecuteAsync(PrepareSql(), message);
}
} }
#region private methods #region private methods
private string PrepareSql() private string PrepareSql()
{ {
return $"INSERT INTO `{_options.TableNamePrefix}.published` (`Name`,`Content`,`Retries`,`Added`,`ExpiresAt`,`StatusName`)VALUES(@Name,@Content,@Retries,@Added,@ExpiresAt,@StatusName)"; return
$"INSERT INTO `{_options.TableNamePrefix}.published` (`Name`,`Content`,`Retries`,`Added`,`ExpiresAt`,`StatusName`)VALUES(@Name,@Content,@Retries,@Added,@ExpiresAt,@StatusName)";
} }
#endregion private methods #endregion private methods
......
...@@ -12,7 +12,7 @@ ...@@ -12,7 +12,7 @@
<PackageReference Include="Dapper" Version="1.50.2" /> <PackageReference Include="Dapper" Version="1.50.2" />
<PackageReference Include="Microsoft.EntityFrameworkCore" Version="2.0.0" /> <PackageReference Include="Microsoft.EntityFrameworkCore" Version="2.0.0" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Relational" Version="2.0.0" /> <PackageReference Include="Microsoft.EntityFrameworkCore.Relational" Version="2.0.0" />
<PackageReference Include="MySqlConnector" Version="0.25.1" /> <PackageReference Include="MySqlConnector" Version="0.28.2" />
</ItemGroup> </ItemGroup>
<ItemGroup> <ItemGroup>
......
...@@ -9,21 +9,16 @@ namespace DotNetCore.CAP.MySql ...@@ -9,21 +9,16 @@ namespace DotNetCore.CAP.MySql
{ {
internal class DefaultAdditionalProcessor : IAdditionalProcessor internal class DefaultAdditionalProcessor : IAdditionalProcessor
{ {
private readonly IServiceProvider _provider;
private readonly ILogger _logger;
private readonly MySqlOptions _options;
private const int MaxBatch = 1000; private const int MaxBatch = 1000;
private readonly TimeSpan _delay = TimeSpan.FromSeconds(1); private readonly TimeSpan _delay = TimeSpan.FromSeconds(1);
private readonly ILogger _logger;
private readonly MySqlOptions _options;
private readonly TimeSpan _waitingInterval = TimeSpan.FromMinutes(5); private readonly TimeSpan _waitingInterval = TimeSpan.FromMinutes(5);
public DefaultAdditionalProcessor( public DefaultAdditionalProcessor(ILogger<DefaultAdditionalProcessor> logger,
IServiceProvider provider,
ILogger<DefaultAdditionalProcessor> logger,
MySqlOptions mysqlOptions) MySqlOptions mysqlOptions)
{ {
_logger = logger; _logger = logger;
_provider = provider;
_options = mysqlOptions; _options = mysqlOptions;
} }
...@@ -31,20 +26,22 @@ namespace DotNetCore.CAP.MySql ...@@ -31,20 +26,22 @@ namespace DotNetCore.CAP.MySql
{ {
_logger.LogDebug("Collecting expired entities."); _logger.LogDebug("Collecting expired entities.");
var tables = new string[]{ var tables = new[]
{
$"{_options.TableNamePrefix}.published", $"{_options.TableNamePrefix}.published",
$"{_options.TableNamePrefix}.received" $"{_options.TableNamePrefix}.received"
}; };
foreach (var table in tables) foreach (var table in tables)
{ {
var removedCount = 0; int removedCount;
do do
{ {
using (var connection = new MySqlConnection(_options.ConnectionString)) using (var connection = new MySqlConnection(_options.ConnectionString))
{ {
removedCount = await connection.ExecuteAsync($@"DELETE FROM `{table}` WHERE ExpiresAt < @now limit @count;", removedCount = await connection.ExecuteAsync(
new { now = DateTime.Now, count = MaxBatch }); $@"DELETE FROM `{table}` WHERE ExpiresAt < @now limit @count;",
new {now = DateTime.Now, count = MaxBatch});
} }
if (removedCount != 0) if (removedCount != 0)
......
...@@ -8,11 +8,11 @@ namespace DotNetCore.CAP.MySql ...@@ -8,11 +8,11 @@ namespace DotNetCore.CAP.MySql
{ {
public class MySqlFetchedMessage : IFetchedMessage public class MySqlFetchedMessage : IFetchedMessage
{ {
private readonly IDbConnection _connection;
private readonly IDbTransaction _transaction;
private readonly Timer _timer;
private static readonly TimeSpan KeepAliveInterval = TimeSpan.FromMinutes(1); private static readonly TimeSpan KeepAliveInterval = TimeSpan.FromMinutes(1);
private readonly IDbConnection _connection;
private readonly object _lockObject = new object(); private readonly object _lockObject = new object();
private readonly Timer _timer;
private readonly IDbTransaction _transaction;
public MySqlFetchedMessage(int messageId, public MySqlFetchedMessage(int messageId,
MessageType type, MessageType type,
......
using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using Dapper;
using DotNetCore.CAP.Dashboard;
using DotNetCore.CAP.Dashboard.Monitoring;
using DotNetCore.CAP.Infrastructure;
using DotNetCore.CAP.Models;
namespace DotNetCore.CAP.MySql
{
internal class MySqlMonitoringApi : IMonitoringApi
{
private readonly string _prefix;
private readonly MySqlStorage _storage;
public MySqlMonitoringApi(IStorage storage, MySqlOptions options)
{
_storage = storage as MySqlStorage ?? throw new ArgumentNullException(nameof(storage));
_prefix = options?.TableNamePrefix ?? throw new ArgumentNullException(nameof(options));
}
public StatisticsDto GetStatistics()
{
var sql = string.Format(@"
set transaction isolation level read committed;
select count(Id) from `{0}.published` where StatusName = N'Succeeded';
select count(Id) from `{0}.received` where StatusName = N'Succeeded';
select count(Id) from `{0}.published` where StatusName = N'Failed';
select count(Id) from `{0}.received` where StatusName = N'Failed';
select count(Id) from `{0}.published` where StatusName in (N'Processing',N'Scheduled',N'Enqueued');
select count(Id) from `{0}.received` where StatusName in (N'Processing',N'Scheduled',N'Enqueued');", _prefix);
var statistics = UseConnection(connection =>
{
var stats = new StatisticsDto();
using (var multi = connection.QueryMultiple(sql))
{
stats.PublishedSucceeded = multi.ReadSingle<int>();
stats.ReceivedSucceeded = multi.ReadSingle<int>();
stats.PublishedFailed = multi.ReadSingle<int>();
stats.ReceivedFailed = multi.ReadSingle<int>();
stats.PublishedProcessing = multi.ReadSingle<int>();
stats.ReceivedProcessing = multi.ReadSingle<int>();
}
return stats;
});
return statistics;
}
public IDictionary<DateTime, int> HourlyFailedJobs(MessageType type)
{
var tableName = type == MessageType.Publish ? "published" : "received";
return UseConnection(connection =>
GetHourlyTimelineStats(connection, tableName, StatusName.Failed));
}
public IDictionary<DateTime, int> HourlySucceededJobs(MessageType type)
{
var tableName = type == MessageType.Publish ? "published" : "received";
return UseConnection(connection =>
GetHourlyTimelineStats(connection, tableName, StatusName.Succeeded));
}
public IList<MessageDto> Messages(MessageQueryDto queryDto)
{
var tableName = queryDto.MessageType == MessageType.Publish ? "published" : "received";
var where = string.Empty;
if (!string.IsNullOrEmpty(queryDto.StatusName))
if (string.Equals(queryDto.StatusName, StatusName.Processing,
StringComparison.CurrentCultureIgnoreCase))
where += " and StatusName in (N'Processing',N'Scheduled',N'Enqueued')";
else
where += " and StatusName=@StatusName";
if (!string.IsNullOrEmpty(queryDto.Name))
where += " and Name=@Name";
if (!string.IsNullOrEmpty(queryDto.Group))
where += " and Group=@Group";
if (!string.IsNullOrEmpty(queryDto.Content))
where += " and Content like '%@Content%'";
var sqlQuery =
$"select * from `{_prefix}.{tableName}` where 1=1 {where} order by Added desc limit @Limit offset @Offset";
return UseConnection(conn => conn.Query<MessageDto>(sqlQuery, new
{
queryDto.StatusName,
queryDto.Group,
queryDto.Name,
queryDto.Content,
Offset = queryDto.CurrentPage * queryDto.PageSize,
Limit = queryDto.PageSize
}).ToList());
}
public int PublishedFailedCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "published", StatusName.Failed));
}
public int PublishedProcessingCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "published", StatusName.Processing));
}
public int PublishedSucceededCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "published", StatusName.Succeeded));
}
public int ReceivedFailedCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "received", StatusName.Failed));
}
public int ReceivedProcessingCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "received", StatusName.Processing));
}
public int ReceivedSucceededCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "received", StatusName.Succeeded));
}
private int GetNumberOfMessage(IDbConnection connection, string tableName, string statusName)
{
var sqlQuery = statusName == StatusName.Processing
? $"select count(Id) from `{_prefix}.{tableName}` where StatusName in (N'Processing',N'Scheduled',N'Enqueued')"
: $"select count(Id) from `{_prefix}.{tableName}` where StatusName = @state";
var count = connection.ExecuteScalar<int>(sqlQuery, new {state = statusName});
return count;
}
private T UseConnection<T>(Func<IDbConnection, T> action)
{
return _storage.UseConnection(action);
}
private Dictionary<DateTime, int> GetHourlyTimelineStats(IDbConnection connection, string tableName,
string statusName)
{
var endDate = DateTime.Now;
var dates = new List<DateTime>();
for (var i = 0; i < 24; i++)
{
dates.Add(endDate);
endDate = endDate.AddHours(-1);
}
var keyMaps = dates.ToDictionary(x => x.ToString("yyyy-MM-dd-HH"), x => x);
return GetTimelineStats(connection, tableName, statusName, keyMaps);
}
private Dictionary<DateTime, int> GetTimelineStats(
IDbConnection connection,
string tableName,
string statusName,
IDictionary<string, DateTime> keyMaps)
{
var sqlQuery =
$@"
select aggr.* from (
select date_format(`Added`,'%Y-%m-%d-%H') as `Key`,
count(id) `Count`
from `{_prefix}.{tableName}`
where StatusName = @statusName
group by date_format(`Added`,'%Y-%m-%d-%H')
) aggr where `Key` in @keys;";
var valuesMap = connection.Query(
sqlQuery,
new {keys = keyMaps.Keys, statusName})
.ToDictionary(x => (string) x.Key, x => (int) x.Count);
foreach (var key in keyMaps.Keys)
if (!valuesMap.ContainsKey(key)) valuesMap.Add(key, 0);
var result = new Dictionary<DateTime, int>();
for (var i = 0; i < keyMaps.Count; i++)
{
var value = valuesMap[keyMaps.ElementAt(i).Key];
result.Add(keyMaps.ElementAt(i).Value, value);
}
return result;
}
}
}
\ No newline at end of file
using System;
using System.Data;
using System.Threading; using System.Threading;
using System.Threading.Tasks; using System.Threading.Tasks;
using Dapper; using Dapper;
using DotNetCore.CAP.Dashboard;
using Microsoft.Extensions.Logging; using Microsoft.Extensions.Logging;
using MySql.Data.MySqlClient; using MySql.Data.MySqlClient;
...@@ -8,15 +11,30 @@ namespace DotNetCore.CAP.MySql ...@@ -8,15 +11,30 @@ namespace DotNetCore.CAP.MySql
{ {
public class MySqlStorage : IStorage public class MySqlStorage : IStorage
{ {
private readonly MySqlOptions _options; private readonly IDbConnection _existingConnection = null;
private readonly ILogger _logger; private readonly ILogger _logger;
private readonly MySqlOptions _options;
private readonly CapOptions _capOptions;
public MySqlStorage(ILogger<MySqlStorage> logger, MySqlOptions options) public MySqlStorage(ILogger<MySqlStorage> logger,
MySqlOptions options,
CapOptions capOptions)
{ {
_options = options; _options = options;
_capOptions = capOptions;
_logger = logger; _logger = logger;
} }
public IStorageConnection GetConnection()
{
return new MySqlStorageConnection(_options, _capOptions);
}
public IMonitoringApi GetMonitoringApi()
{
return new MySqlMonitoringApi(this, _options);
}
public async Task InitializeAsync(CancellationToken cancellationToken) public async Task InitializeAsync(CancellationToken cancellationToken)
{ {
if (cancellationToken.IsCancellationRequested) return; if (cancellationToken.IsCancellationRequested) return;
...@@ -32,7 +50,7 @@ namespace DotNetCore.CAP.MySql ...@@ -32,7 +50,7 @@ namespace DotNetCore.CAP.MySql
protected virtual string CreateDbTablesScript(string prefix) protected virtual string CreateDbTablesScript(string prefix)
{ {
var batchSql = var batchSql =
$@" $@"
CREATE TABLE IF NOT EXISTS `{prefix}.queue` ( CREATE TABLE IF NOT EXISTS `{prefix}.queue` (
`MessageId` int(11) NOT NULL, `MessageId` int(11) NOT NULL,
`MessageType` tinyint(4) NOT NULL `MessageType` tinyint(4) NOT NULL
...@@ -62,5 +80,41 @@ CREATE TABLE IF NOT EXISTS `{prefix}.published` ( ...@@ -62,5 +80,41 @@ CREATE TABLE IF NOT EXISTS `{prefix}.published` (
) ENGINE=InnoDB DEFAULT CHARSET=utf8;"; ) ENGINE=InnoDB DEFAULT CHARSET=utf8;";
return batchSql; return batchSql;
} }
internal T UseConnection<T>(Func<IDbConnection, T> func)
{
IDbConnection connection = null;
try
{
connection = CreateAndOpenConnection();
return func(connection);
}
finally
{
ReleaseConnection(connection);
}
}
internal IDbConnection CreateAndOpenConnection()
{
var connection = _existingConnection ?? new MySqlConnection(_options.ConnectionString);
if (connection.State == ConnectionState.Closed)
connection.Open();
return connection;
}
internal bool IsExistingConnection(IDbConnection connection)
{
return connection != null && ReferenceEquals(connection, _existingConnection);
}
internal void ReleaseConnection(IDbConnection connection)
{
if (connection != null && !IsExistingConnection(connection))
connection.Dispose();
}
} }
} }
\ No newline at end of file
...@@ -11,16 +11,19 @@ namespace DotNetCore.CAP.MySql ...@@ -11,16 +11,19 @@ namespace DotNetCore.CAP.MySql
{ {
public class MySqlStorageConnection : IStorageConnection public class MySqlStorageConnection : IStorageConnection
{ {
private readonly MySqlOptions _options; private readonly CapOptions _capOptions;
private readonly string _prefix; private readonly string _prefix;
public MySqlStorageConnection(MySqlOptions options) private const string DateTimeMaxValue = "9999-12-31 23:59:59";
public MySqlStorageConnection(MySqlOptions options, CapOptions capOptions)
{ {
_options = options; _capOptions = capOptions;
_prefix = _options.TableNamePrefix; Options = options;
_prefix = Options.TableNamePrefix;
} }
public MySqlOptions Options => _options; public MySqlOptions Options { get; }
public IStorageTransaction CreateTransaction() public IStorageTransaction CreateTransaction()
{ {
...@@ -31,7 +34,7 @@ namespace DotNetCore.CAP.MySql ...@@ -31,7 +34,7 @@ namespace DotNetCore.CAP.MySql
{ {
var sql = $@"SELECT * FROM `{_prefix}.published` WHERE `Id`={id};"; var sql = $@"SELECT * FROM `{_prefix}.published` WHERE `Id`={id};";
using (var connection = new MySqlConnection(_options.ConnectionString)) using (var connection = new MySqlConnection(Options.ConnectionString))
{ {
return await connection.QueryFirstOrDefaultAsync<CapPublishedMessage>(sql); return await connection.QueryFirstOrDefaultAsync<CapPublishedMessage>(sql);
} }
...@@ -39,26 +42,23 @@ namespace DotNetCore.CAP.MySql ...@@ -39,26 +42,23 @@ namespace DotNetCore.CAP.MySql
public Task<IFetchedMessage> FetchNextMessageAsync() public Task<IFetchedMessage> FetchNextMessageAsync()
{ {
//Last execute statement(FOR UPDATE to fix dirty read) :
//SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
//START TRANSACTION;
//SELECT MessageId,MessageType FROM `{_prefix}.queue` LIMIT 1 FOR UPDATE;
//DELETE FROM `{_prefix}.queue` LIMIT 1;
//COMMIT;
var sql = $@" var sql = $@"
SELECT `MessageId`,`MessageType` FROM `{_prefix}.queue` LIMIT 1 FOR UPDATE; SELECT `MessageId`,`MessageType` FROM `{_prefix}.queue` LIMIT 1 FOR UPDATE;
DELETE FROM `{_prefix}.queue` LIMIT 1;"; DELETE FROM `{_prefix}.queue` LIMIT 1;";
// var sql = $@"
//SELECT @MId:=`MessageId` as MessageId, @MType:=`MessageType` as MessageType FROM `{_prefix}.queue` LIMIT 1;
//DELETE FROM `{_prefix}.queue` where `MessageId` = @MId AND `MessageType`=@MType;";
return FetchNextMessageCoreAsync(sql); return FetchNextMessageCoreAsync(sql);
} }
public async Task<CapPublishedMessage> GetNextPublishedMessageToBeEnqueuedAsync() public async Task<CapPublishedMessage> GetNextPublishedMessageToBeEnqueuedAsync()
{ {
var sql = $"SELECT * FROM `{_prefix}.published` WHERE `StatusName` = '{StatusName.Scheduled}' LIMIT 1;"; var sql = $@"
UPDATE `{_prefix}.published` SET Id=LAST_INSERT_ID(Id),ExpiresAt='{DateTimeMaxValue}' WHERE ExpiresAt IS NULL AND `StatusName` = '{StatusName.Scheduled}' LIMIT 1;
SELECT * FROM `{_prefix}.published` WHERE Id=LAST_INSERT_ID();";
using (var connection = new MySqlConnection(_options.ConnectionString)) using (var connection = new MySqlConnection(Options.ConnectionString))
{ {
return await connection.QueryFirstOrDefaultAsync<CapPublishedMessage>(sql); return await connection.QueryFirstOrDefaultAsync<CapPublishedMessage>(sql);
} }
...@@ -66,16 +66,14 @@ DELETE FROM `{_prefix}.queue` LIMIT 1;"; ...@@ -66,16 +66,14 @@ DELETE FROM `{_prefix}.queue` LIMIT 1;";
public async Task<IEnumerable<CapPublishedMessage>> GetFailedPublishedMessages() public async Task<IEnumerable<CapPublishedMessage>> GetFailedPublishedMessages()
{ {
var sql = $"SELECT * FROM `{_prefix}.published` WHERE `StatusName` = '{StatusName.Failed}';"; var sql = $"SELECT * FROM `{_prefix}.published` WHERE `Retries`<{_capOptions.FailedRetryCount} AND `StatusName` = '{StatusName.Failed}' LIMIT 200;";
using (var connection = new MySqlConnection(_options.ConnectionString)) using (var connection = new MySqlConnection(Options.ConnectionString))
{ {
return await connection.QueryAsync<CapPublishedMessage>(sql); return await connection.QueryAsync<CapPublishedMessage>(sql);
} }
} }
// CapReceviedMessage
public async Task StoreReceivedMessageAsync(CapReceivedMessage message) public async Task StoreReceivedMessageAsync(CapReceivedMessage message)
{ {
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
...@@ -84,7 +82,7 @@ DELETE FROM `{_prefix}.queue` LIMIT 1;"; ...@@ -84,7 +82,7 @@ DELETE FROM `{_prefix}.queue` LIMIT 1;";
INSERT INTO `{_prefix}.received`(`Name`,`Group`,`Content`,`Retries`,`Added`,`ExpiresAt`,`StatusName`) INSERT INTO `{_prefix}.received`(`Name`,`Group`,`Content`,`Retries`,`Added`,`ExpiresAt`,`StatusName`)
VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);"; VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);";
using (var connection = new MySqlConnection(_options.ConnectionString)) using (var connection = new MySqlConnection(Options.ConnectionString))
{ {
await connection.ExecuteAsync(sql, message); await connection.ExecuteAsync(sql, message);
} }
...@@ -93,44 +91,82 @@ VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);"; ...@@ -93,44 +91,82 @@ VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);";
public async Task<CapReceivedMessage> GetReceivedMessageAsync(int id) public async Task<CapReceivedMessage> GetReceivedMessageAsync(int id)
{ {
var sql = $@"SELECT * FROM `{_prefix}.received` WHERE Id={id};"; var sql = $@"SELECT * FROM `{_prefix}.received` WHERE Id={id};";
using (var connection = new MySqlConnection(_options.ConnectionString)) using (var connection = new MySqlConnection(Options.ConnectionString))
{ {
return await connection.QueryFirstOrDefaultAsync<CapReceivedMessage>(sql); return await connection.QueryFirstOrDefaultAsync<CapReceivedMessage>(sql);
} }
} }
public async Task<CapReceivedMessage> GetNextReceviedMessageToBeEnqueuedAsync() public async Task<CapReceivedMessage> GetNextReceivedMessageToBeEnqueuedAsync()
{ {
var sql = $"SELECT * FROM `{_prefix}.received` WHERE `StatusName` = '{StatusName.Scheduled}' LIMIT 1;"; var sql = $@"
using (var connection = new MySqlConnection(_options.ConnectionString)) UPDATE `{_prefix}.received` SET Id=LAST_INSERT_ID(Id),ExpiresAt='{DateTimeMaxValue}' WHERE ExpiresAt IS NULL AND `StatusName` = '{StatusName.Scheduled}' LIMIT 1;
SELECT * FROM `{_prefix}.received` WHERE Id=LAST_INSERT_ID();";
using (var connection = new MySqlConnection(Options.ConnectionString))
{ {
return await connection.QueryFirstOrDefaultAsync<CapReceivedMessage>(sql); return await connection.QueryFirstOrDefaultAsync<CapReceivedMessage>(sql);
} }
} }
public async Task<IEnumerable<CapReceivedMessage>> GetFailedReceviedMessages() public async Task<IEnumerable<CapReceivedMessage>> GetFailedReceivedMessages()
{ {
var sql = $"SELECT * FROM `{_prefix}.received` WHERE `StatusName` = '{StatusName.Failed}';"; var sql = $"SELECT * FROM `{_prefix}.received` WHERE `Retries`<{_capOptions.FailedRetryCount} AND `StatusName` = '{StatusName.Failed}' LIMIT 200;";
using (var connection = new MySqlConnection(_options.ConnectionString)) using (var connection = new MySqlConnection(Options.ConnectionString))
{ {
return await connection.QueryAsync<CapReceivedMessage>(sql); return await connection.QueryAsync<CapReceivedMessage>(sql);
} }
} }
public void Dispose() public void Dispose()
{ {
} }
public bool ChangePublishedState(int messageId, string state)
{
var sql =
$"UPDATE `{_prefix}.published` SET `Retries`=`Retries`+1,`StatusName` = '{state}' WHERE `Id`={messageId}";
using (var connection = new MySqlConnection(Options.ConnectionString))
{
return connection.Execute(sql) > 0;
}
}
public bool ChangeReceivedState(int messageId, string state)
{
var sql =
$"UPDATE `{_prefix}.received` SET `Retries`=`Retries`+1,`StatusName` = '{state}' WHERE `Id`={messageId}";
using (var connection = new MySqlConnection(Options.ConnectionString))
{
return connection.Execute(sql) > 0;
}
}
private async Task<IFetchedMessage> FetchNextMessageCoreAsync(string sql, object args = null) private async Task<IFetchedMessage> FetchNextMessageCoreAsync(string sql, object args = null)
{ {
//here don't use `using` to dispose //here don't use `using` to dispose
var connection = new MySqlConnection(_options.ConnectionString); var connection = new MySqlConnection(Options.ConnectionString);
await connection.OpenAsync(); await connection.OpenAsync();
var transaction = connection.BeginTransaction(IsolationLevel.ReadCommitted); var transaction = connection.BeginTransaction(IsolationLevel.ReadCommitted);
FetchedMessage fetchedMessage = null; FetchedMessage fetchedMessage = null;
try try
{ {
fetchedMessage = await connection.QueryFirstOrDefaultAsync<FetchedMessage>(sql, args, transaction); //fetchedMessage = await connection.QuerySingleOrDefaultAsync<FetchedMessage>(sql, args, transaction);
// An anomaly with unknown causes, sometimes QuerySingleOrDefaultAsync can't return expected result.
using (var reader = connection.ExecuteReader(sql, args, transaction))
{
while (reader.Read())
{
fetchedMessage = new FetchedMessage
{
MessageId = (int)reader.GetInt64(0),
MessageType = (MessageType)reader.GetInt64(1)
};
}
}
} }
catch (MySqlException) catch (MySqlException)
{ {
...@@ -146,7 +182,8 @@ VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);"; ...@@ -146,7 +182,8 @@ VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);";
return null; return null;
} }
return new MySqlFetchedMessage(fetchedMessage.MessageId, fetchedMessage.MessageType, connection, transaction); return new MySqlFetchedMessage(fetchedMessage.MessageId, fetchedMessage.MessageType, connection,
transaction);
} }
} }
} }
\ No newline at end of file
...@@ -7,12 +7,12 @@ using MySql.Data.MySqlClient; ...@@ -7,12 +7,12 @@ using MySql.Data.MySqlClient;
namespace DotNetCore.CAP.MySql namespace DotNetCore.CAP.MySql
{ {
public class MySqlStorageTransaction : IStorageTransaction, IDisposable public class MySqlStorageTransaction : IStorageTransaction
{ {
private readonly string _prefix; private readonly IDbConnection _dbConnection;
private readonly IDbTransaction _dbTransaction; private readonly IDbTransaction _dbTransaction;
private readonly IDbConnection _dbConnection; private readonly string _prefix;
public MySqlStorageTransaction(MySqlStorageConnection connection) public MySqlStorageTransaction(MySqlStorageConnection connection)
{ {
...@@ -28,7 +28,8 @@ namespace DotNetCore.CAP.MySql ...@@ -28,7 +28,8 @@ namespace DotNetCore.CAP.MySql
{ {
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
var sql = $"UPDATE `{_prefix}.published` SET `Retries` = @Retries,`ExpiresAt` = @ExpiresAt,`StatusName`=@StatusName WHERE `Id`=@Id;"; var sql =
$"UPDATE `{_prefix}.published` SET `Retries` = @Retries,`Content`= @Content,`ExpiresAt` = @ExpiresAt,`StatusName`=@StatusName WHERE `Id`=@Id;";
_dbConnection.Execute(sql, message, _dbTransaction); _dbConnection.Execute(sql, message, _dbTransaction);
} }
...@@ -36,7 +37,8 @@ namespace DotNetCore.CAP.MySql ...@@ -36,7 +37,8 @@ namespace DotNetCore.CAP.MySql
{ {
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
var sql = $"UPDATE `{_prefix}.received` SET `Retries` = @Retries,`ExpiresAt` = @ExpiresAt,`StatusName`=@StatusName WHERE `Id`=@Id;"; var sql =
$"UPDATE `{_prefix}.received` SET `Retries` = @Retries,`Content`= @Content,`ExpiresAt` = @ExpiresAt,`StatusName`=@StatusName WHERE `Id`=@Id;";
_dbConnection.Execute(sql, message, _dbTransaction); _dbConnection.Execute(sql, message, _dbTransaction);
} }
...@@ -45,7 +47,8 @@ namespace DotNetCore.CAP.MySql ...@@ -45,7 +47,8 @@ namespace DotNetCore.CAP.MySql
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
var sql = $"INSERT INTO `{_prefix}.queue` values(@MessageId,@MessageType);"; var sql = $"INSERT INTO `{_prefix}.queue` values(@MessageId,@MessageType);";
_dbConnection.Execute(sql, new CapQueue { MessageId = message.Id, MessageType = MessageType.Publish }, _dbTransaction); _dbConnection.Execute(sql, new CapQueue {MessageId = message.Id, MessageType = MessageType.Publish},
_dbTransaction);
} }
public void EnqueueMessage(CapReceivedMessage message) public void EnqueueMessage(CapReceivedMessage message)
...@@ -53,7 +56,8 @@ namespace DotNetCore.CAP.MySql ...@@ -53,7 +56,8 @@ namespace DotNetCore.CAP.MySql
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
var sql = $"INSERT INTO `{_prefix}.queue` values(@MessageId,@MessageType);"; var sql = $"INSERT INTO `{_prefix}.queue` values(@MessageId,@MessageType);";
_dbConnection.Execute(sql, new CapQueue { MessageId = message.Id, MessageType = MessageType.Subscribe }, _dbTransaction); _dbConnection.Execute(sql, new CapQueue {MessageId = message.Id, MessageType = MessageType.Subscribe},
_dbTransaction);
} }
public Task CommitAsync() public Task CommitAsync()
......
...@@ -9,7 +9,7 @@ namespace DotNetCore.CAP ...@@ -9,7 +9,7 @@ namespace DotNetCore.CAP
/// <summary> /// <summary>
/// Gets or sets the schema to use when creating database objects. /// Gets or sets the schema to use when creating database objects.
/// Default is <see cref="DefaultSchema"/>. /// Default is <see cref="DefaultSchema" />.
/// </summary> /// </summary>
public string Schema { get; set; } = DefaultSchema; public string Schema { get; set; } = DefaultSchema;
......
...@@ -9,10 +9,7 @@ namespace Microsoft.Extensions.DependencyInjection ...@@ -9,10 +9,7 @@ namespace Microsoft.Extensions.DependencyInjection
{ {
public static CapOptions UsePostgreSql(this CapOptions options, string connectionString) public static CapOptions UsePostgreSql(this CapOptions options, string connectionString)
{ {
return options.UsePostgreSql(opt => return options.UsePostgreSql(opt => { opt.ConnectionString = connectionString; });
{
opt.ConnectionString = connectionString;
});
} }
public static CapOptions UsePostgreSql(this CapOptions options, Action<PostgreSqlOptions> configure) public static CapOptions UsePostgreSql(this CapOptions options, Action<PostgreSqlOptions> configure)
...@@ -27,10 +24,7 @@ namespace Microsoft.Extensions.DependencyInjection ...@@ -27,10 +24,7 @@ namespace Microsoft.Extensions.DependencyInjection
public static CapOptions UseEntityFramework<TContext>(this CapOptions options) public static CapOptions UseEntityFramework<TContext>(this CapOptions options)
where TContext : DbContext where TContext : DbContext
{ {
return options.UseEntityFramework<TContext>(opt => return options.UseEntityFramework<TContext>(opt => { opt.DbContextType = typeof(TContext); });
{
opt.DbContextType = typeof(TContext);
});
} }
public static CapOptions UseEntityFramework<TContext>(this CapOptions options, Action<EFOptions> configure) public static CapOptions UseEntityFramework<TContext>(this CapOptions options, Action<EFOptions> configure)
...@@ -38,7 +32,7 @@ namespace Microsoft.Extensions.DependencyInjection ...@@ -38,7 +32,7 @@ namespace Microsoft.Extensions.DependencyInjection
{ {
if (configure == null) throw new ArgumentNullException(nameof(configure)); if (configure == null) throw new ArgumentNullException(nameof(configure));
var efOptions = new EFOptions { DbContextType = typeof(TContext) }; var efOptions = new EFOptions {DbContextType = typeof(TContext)};
configure(efOptions); configure(efOptions);
options.RegisterExtension(new PostgreSqlCapOptionsExtension(configure)); options.RegisterExtension(new PostgreSqlCapOptionsExtension(configure));
......
...@@ -18,17 +18,17 @@ namespace DotNetCore.CAP ...@@ -18,17 +18,17 @@ namespace DotNetCore.CAP
public void AddServices(IServiceCollection services) public void AddServices(IServiceCollection services)
{ {
services.AddSingleton<CapDatabaseStorageMarkerService>();
services.AddSingleton<IStorage, PostgreSqlStorage>(); services.AddSingleton<IStorage, PostgreSqlStorage>();
services.AddScoped<IStorageConnection, PostgreSqlStorageConnection>(); services.AddSingleton<IStorageConnection, PostgreSqlStorageConnection>();
services.AddScoped<ICapPublisher, CapPublisher>(); services.AddScoped<ICapPublisher, CapPublisher>();
services.AddTransient<ICallbackPublisher, CapPublisher>(); services.AddScoped<ICallbackPublisher, CapPublisher>();
services.AddTransient<IAdditionalProcessor, DefaultAdditionalProcessor>(); services.AddTransient<IAdditionalProcessor, DefaultAdditionalProcessor>();
var postgreSqlOptions = new PostgreSqlOptions(); var postgreSqlOptions = new PostgreSqlOptions();
_configure(postgreSqlOptions); _configure(postgreSqlOptions);
if (postgreSqlOptions.DbContextType != null) if (postgreSqlOptions.DbContextType != null)
{
services.AddSingleton(x => services.AddSingleton(x =>
{ {
using (var scope = x.CreateScope()) using (var scope = x.CreateScope())
...@@ -39,11 +39,8 @@ namespace DotNetCore.CAP ...@@ -39,11 +39,8 @@ namespace DotNetCore.CAP
return postgreSqlOptions; return postgreSqlOptions;
} }
}); });
}
else else
{
services.AddSingleton(postgreSqlOptions); services.AddSingleton(postgreSqlOptions);
}
} }
} }
} }
\ No newline at end of file
// ReSharper disable once CheckNamespace // ReSharper disable once CheckNamespace
namespace DotNetCore.CAP namespace DotNetCore.CAP
{ {
public class PostgreSqlOptions : EFOptions public class PostgreSqlOptions : EFOptions
......
...@@ -13,9 +13,9 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -13,9 +13,9 @@ namespace DotNetCore.CAP.PostgreSql
{ {
public class CapPublisher : CapPublisherBase, ICallbackPublisher public class CapPublisher : CapPublisherBase, ICallbackPublisher
{ {
private readonly DbContext _dbContext;
private readonly ILogger _logger; private readonly ILogger _logger;
private readonly PostgreSqlOptions _options; private readonly PostgreSqlOptions _options;
private readonly DbContext _dbContext;
public CapPublisher(IServiceProvider provider, public CapPublisher(IServiceProvider provider,
ILogger<CapPublisher> logger, ILogger<CapPublisher> logger,
...@@ -28,7 +28,15 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -28,7 +28,15 @@ namespace DotNetCore.CAP.PostgreSql
if (_options.DbContextType != null) if (_options.DbContextType != null)
{ {
IsUsingEF = true; IsUsingEF = true;
_dbContext = (DbContext)ServiceProvider.GetService(_options.DbContextType); _dbContext = (DbContext) ServiceProvider.GetService(_options.DbContextType);
}
}
public async Task PublishAsync(CapPublishedMessage message)
{
using (var conn = new NpgsqlConnection(_options.ConnectionString))
{
await conn.ExecuteAsync(PrepareSql(), message);
} }
} }
...@@ -45,36 +53,33 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -45,36 +53,33 @@ namespace DotNetCore.CAP.PostgreSql
dbContextTransaction = _dbContext.Database.BeginTransaction(IsolationLevel.ReadCommitted); dbContextTransaction = _dbContext.Database.BeginTransaction(IsolationLevel.ReadCommitted);
dbTrans = dbContextTransaction.GetDbTransaction(); dbTrans = dbContextTransaction.GetDbTransaction();
} }
DbTranasaction = dbTrans; DbTransaction = dbTrans;
} }
protected override void Execute(IDbConnection dbConnection, IDbTransaction dbTransaction, CapPublishedMessage message) protected override void Execute(IDbConnection dbConnection, IDbTransaction dbTransaction,
CapPublishedMessage message)
{ {
dbConnection.Execute(PrepareSql(), message, dbTransaction); dbConnection.Execute(PrepareSql(), message, dbTransaction);
_logger.LogInformation("Published Message has been persisted in the database. name:" + message.ToString()); _logger.LogInformation("Published Message has been persisted in the database. name:" + message);
} }
protected override async Task ExecuteAsync(IDbConnection dbConnection, IDbTransaction dbTransaction, CapPublishedMessage message) protected override Task ExecuteAsync(IDbConnection dbConnection, IDbTransaction dbTransaction,
CapPublishedMessage message)
{ {
await dbConnection.ExecuteAsync(PrepareSql(), message, dbTransaction); dbConnection.ExecuteAsync(PrepareSql(), message, dbTransaction);
_logger.LogInformation("Published Message has been persisted in the database. name:" + message.ToString()); _logger.LogInformation("Published Message has been persisted in the database. name:" + message);
}
public async Task PublishAsync(CapPublishedMessage message) return Task.CompletedTask;
{
using (var conn = new NpgsqlConnection(_options.ConnectionString))
{
await conn.ExecuteAsync(PrepareSql(), message);
}
} }
#region private methods #region private methods
private string PrepareSql() private string PrepareSql()
{ {
return $"INSERT INTO \"{_options.Schema}\".\"published\" (\"Name\",\"Content\",\"Retries\",\"Added\",\"ExpiresAt\",\"StatusName\")VALUES(@Name,@Content,@Retries,@Added,@ExpiresAt,@StatusName)"; return
$"INSERT INTO \"{_options.Schema}\".\"published\" (\"Name\",\"Content\",\"Retries\",\"Added\",\"ExpiresAt\",\"StatusName\")VALUES(@Name,@Content,@Retries,@Added,@ExpiresAt,@StatusName)";
} }
#endregion private methods #endregion private methods
......
...@@ -9,26 +9,22 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -9,26 +9,22 @@ namespace DotNetCore.CAP.PostgreSql
{ {
internal class DefaultAdditionalProcessor : IAdditionalProcessor internal class DefaultAdditionalProcessor : IAdditionalProcessor
{ {
private readonly IServiceProvider _provider;
private readonly ILogger _logger;
private readonly PostgreSqlOptions _options;
private const int MaxBatch = 1000; private const int MaxBatch = 1000;
private readonly TimeSpan _delay = TimeSpan.FromSeconds(1);
private readonly TimeSpan _waitingInterval = TimeSpan.FromMinutes(5);
private static readonly string[] Tables = private static readonly string[] Tables =
{ {
"published","received" "published", "received"
}; };
public DefaultAdditionalProcessor( private readonly TimeSpan _delay = TimeSpan.FromSeconds(1);
IServiceProvider provider, private readonly ILogger _logger;
ILogger<DefaultAdditionalProcessor> logger, private readonly PostgreSqlOptions _options;
private readonly TimeSpan _waitingInterval = TimeSpan.FromMinutes(5);
public DefaultAdditionalProcessor(ILogger<DefaultAdditionalProcessor> logger,
PostgreSqlOptions sqlServerOptions) PostgreSqlOptions sqlServerOptions)
{ {
_logger = logger; _logger = logger;
_provider = provider;
_options = sqlServerOptions; _options = sqlServerOptions;
} }
...@@ -43,8 +39,9 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -43,8 +39,9 @@ namespace DotNetCore.CAP.PostgreSql
{ {
using (var connection = new NpgsqlConnection(_options.ConnectionString)) using (var connection = new NpgsqlConnection(_options.ConnectionString))
{ {
removedCount = await connection.ExecuteAsync($"DELETE FROM \"{_options.Schema}\".\"{table}\" WHERE \"ExpiresAt\" < @now AND \"Id\" IN (SELECT \"Id\" FROM \"{_options.Schema}\".\"{table}\" LIMIT @count);", removedCount = await connection.ExecuteAsync(
new { now = DateTime.Now, count = MaxBatch }); $"DELETE FROM \"{_options.Schema}\".\"{table}\" WHERE \"ExpiresAt\" < @now AND \"Id\" IN (SELECT \"Id\" FROM \"{_options.Schema}\".\"{table}\" LIMIT @count);",
new {now = DateTime.Now, count = MaxBatch});
} }
if (removedCount != 0) if (removedCount != 0)
......
...@@ -8,11 +8,11 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -8,11 +8,11 @@ namespace DotNetCore.CAP.PostgreSql
{ {
public class PostgreSqlFetchedMessage : IFetchedMessage public class PostgreSqlFetchedMessage : IFetchedMessage
{ {
private readonly IDbConnection _connection;
private readonly IDbTransaction _transaction;
private readonly Timer _timer;
private static readonly TimeSpan KeepAliveInterval = TimeSpan.FromMinutes(1); private static readonly TimeSpan KeepAliveInterval = TimeSpan.FromMinutes(1);
private readonly IDbConnection _connection;
private readonly object _lockObject = new object(); private readonly object _lockObject = new object();
private readonly Timer _timer;
private readonly IDbTransaction _transaction;
public PostgreSqlFetchedMessage(int messageId, public PostgreSqlFetchedMessage(int messageId,
MessageType type, MessageType type,
......
using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using Dapper;
using DotNetCore.CAP.Dashboard;
using DotNetCore.CAP.Dashboard.Monitoring;
using DotNetCore.CAP.Infrastructure;
using DotNetCore.CAP.Models;
namespace DotNetCore.CAP.PostgreSql
{
public class PostgreSqlMonitoringApi : IMonitoringApi
{
private readonly PostgreSqlOptions _options;
private readonly PostgreSqlStorage _storage;
public PostgreSqlMonitoringApi(IStorage storage, PostgreSqlOptions options)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
_storage = storage as PostgreSqlStorage ?? throw new ArgumentNullException(nameof(storage));
}
public StatisticsDto GetStatistics()
{
var sql = string.Format(@"
select count(""Id"") from ""{0}"".""published"" where ""StatusName"" = N'Succeeded';
select count(""Id"") from ""{0}"".""received"" where ""StatusName"" = N'Succeeded';
select count(""Id"") from ""{0}"".""published"" where ""StatusName"" = N'Failed';
select count(""Id"") from ""{0}"".""received"" where ""StatusName"" = N'Failed';
select count(""Id"") from ""{0}"".""published"" where ""StatusName"" in (N'Processing',N'Scheduled',N'Enqueued');
select count(""Id"") from ""{0}"".""received"" where ""StatusName"" in (N'Processing',N'Scheduled',N'Enqueued');",
_options.Schema);
var statistics = UseConnection(connection =>
{
var stats = new StatisticsDto();
using (var multi = connection.QueryMultiple(sql))
{
stats.PublishedSucceeded = multi.ReadSingle<int>();
stats.ReceivedSucceeded = multi.ReadSingle<int>();
stats.PublishedFailed = multi.ReadSingle<int>();
stats.ReceivedFailed = multi.ReadSingle<int>();
stats.PublishedProcessing = multi.ReadSingle<int>();
stats.ReceivedProcessing = multi.ReadSingle<int>();
}
return stats;
});
return statistics;
}
public IList<MessageDto> Messages(MessageQueryDto queryDto)
{
var tableName = queryDto.MessageType == MessageType.Publish ? "published" : "received";
var where = string.Empty;
if (!string.IsNullOrEmpty(queryDto.StatusName))
if (string.Equals(queryDto.StatusName, StatusName.Processing,
StringComparison.CurrentCultureIgnoreCase))
where += " and \"StatusName\" in (N'Processing',N'Scheduled',N'Enqueued')";
else
where += " and Lower(\"StatusName\") = Lower(@StatusName)";
if (!string.IsNullOrEmpty(queryDto.Name))
where += " and Lower(\"Name\") = Lower(@Name)";
if (!string.IsNullOrEmpty(queryDto.Group))
where += " and Lower(\"Group\") = Lower(@Group)";
if (!string.IsNullOrEmpty(queryDto.Content))
where += " and \"Content\" ILike '%@Content%'";
var sqlQuery =
$"select * from \"{_options.Schema}\".\"{tableName}\" where 1=1 {where} order by \"Added\" desc offset @Offset limit @Limit";
return UseConnection(conn => conn.Query<MessageDto>(sqlQuery, new
{
queryDto.StatusName,
queryDto.Group,
queryDto.Name,
queryDto.Content,
Offset = queryDto.CurrentPage * queryDto.PageSize,
Limit = queryDto.PageSize
}).ToList());
}
public int PublishedFailedCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "published", StatusName.Failed));
}
public int PublishedProcessingCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "published", StatusName.Processing));
}
public int PublishedSucceededCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "published", StatusName.Succeeded));
}
public int ReceivedFailedCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "received", StatusName.Failed));
}
public int ReceivedProcessingCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "received", StatusName.Processing));
}
public int ReceivedSucceededCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "received", StatusName.Succeeded));
}
public IDictionary<DateTime, int> HourlySucceededJobs(MessageType type)
{
var tableName = type == MessageType.Publish ? "published" : "received";
return UseConnection(connection =>
GetHourlyTimelineStats(connection, tableName, StatusName.Succeeded));
}
public IDictionary<DateTime, int> HourlyFailedJobs(MessageType type)
{
var tableName = type == MessageType.Publish ? "published" : "received";
return UseConnection(connection =>
GetHourlyTimelineStats(connection, tableName, StatusName.Failed));
}
private int GetNumberOfMessage(IDbConnection connection, string tableName, string statusName)
{
var sqlQuery = statusName == StatusName.Processing
? $"select count(\"Id\") from \"{_options.Schema}\".\"{tableName}\" where \"StatusName\" in (N'Processing',N'Scheduled',N'Enqueued')"
: $"select count(\"Id\") from \"{_options.Schema}\".\"{tableName}\" where Lower(\"StatusName\") = Lower(@state)";
var count = connection.ExecuteScalar<int>(sqlQuery, new { state = statusName });
return count;
}
private T UseConnection<T>(Func<IDbConnection, T> action)
{
return _storage.UseConnection(action);
}
private Dictionary<DateTime, int> GetHourlyTimelineStats(IDbConnection connection, string tableName,
string statusName)
{
var endDate = DateTime.Now;
var dates = new List<DateTime>();
for (var i = 0; i < 24; i++)
{
dates.Add(endDate);
endDate = endDate.AddHours(-1);
}
var keyMaps = dates.ToDictionary(x => x.ToString("yyyy-MM-dd-HH"), x => x);
return GetTimelineStats(connection, tableName, statusName, keyMaps);
}
private Dictionary<DateTime, int> GetTimelineStats(
IDbConnection connection,
string tableName,
string statusName,
IDictionary<string, DateTime> keyMaps)
{
var sqlQuery =
$@"
with aggr as (
select to_char(""Added"",'yyyy-MM-dd-HH') as ""Key"",
count(""Id"") as ""Count""
from ""{_options.Schema}"".""{tableName}""
where ""StatusName"" = @statusName
group by to_char(""Added"", 'yyyy-MM-dd-HH')
)
select ""Key"",""Count"" from aggr where ""Key""= Any(@keys);";
var valuesMap = connection.Query(sqlQuery,new { keys = keyMaps.Keys.ToList(), statusName })
.ToList()
.ToDictionary(x => (string)x.Key, x => (int)x.Count);
foreach (var key in keyMaps.Keys)
if (!valuesMap.ContainsKey(key)) valuesMap.Add(key, 0);
var result = new Dictionary<DateTime, int>();
for (var i = 0; i < keyMaps.Count; i++)
{
var value = valuesMap[keyMaps.ElementAt(i).Key];
result.Add(keyMaps.ElementAt(i).Value, value);
}
return result;
}
}
}
\ No newline at end of file
using System;
using System.Data;
using System.Threading; using System.Threading;
using System.Threading.Tasks; using System.Threading.Tasks;
using Dapper; using Dapper;
using Microsoft.EntityFrameworkCore; using DotNetCore.CAP.Dashboard;
using Microsoft.Extensions.Logging; using Microsoft.Extensions.Logging;
using Npgsql; using Npgsql;
...@@ -9,13 +11,28 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -9,13 +11,28 @@ namespace DotNetCore.CAP.PostgreSql
{ {
public class PostgreSqlStorage : IStorage public class PostgreSqlStorage : IStorage
{ {
private readonly PostgreSqlOptions _options; private readonly IDbConnection _existingConnection = null;
private readonly ILogger _logger; private readonly ILogger _logger;
private readonly CapOptions _capOptions;
private readonly PostgreSqlOptions _options;
public PostgreSqlStorage(ILogger<PostgreSqlStorage> logger, PostgreSqlOptions options) public PostgreSqlStorage(ILogger<PostgreSqlStorage> logger,
CapOptions capOptions,
PostgreSqlOptions options)
{ {
_options = options; _options = options;
_logger = logger; _logger = logger;
_capOptions = capOptions;
}
public IStorageConnection GetConnection()
{
return new PostgreSqlStorageConnection(_options, _capOptions);
}
public IMonitoringApi GetMonitoringApi()
{
return new PostgreSqlMonitoringApi(this, _options);
} }
public async Task InitializeAsync(CancellationToken cancellationToken) public async Task InitializeAsync(CancellationToken cancellationToken)
...@@ -31,6 +48,42 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -31,6 +48,42 @@ namespace DotNetCore.CAP.PostgreSql
_logger.LogDebug("Ensuring all create database tables script are applied."); _logger.LogDebug("Ensuring all create database tables script are applied.");
} }
internal T UseConnection<T>(Func<IDbConnection, T> func)
{
IDbConnection connection = null;
try
{
connection = CreateAndOpenConnection();
return func(connection);
}
finally
{
ReleaseConnection(connection);
}
}
internal IDbConnection CreateAndOpenConnection()
{
var connection = _existingConnection ?? new NpgsqlConnection(_options.ConnectionString);
if (connection.State == ConnectionState.Closed)
connection.Open();
return connection;
}
internal bool IsExistingConnection(IDbConnection connection)
{
return connection != null && ReferenceEquals(connection, _existingConnection);
}
internal void ReleaseConnection(IDbConnection connection)
{
if (connection != null && !IsExistingConnection(connection))
connection.Dispose();
}
protected virtual string CreateDbTablesScript(string schema) protected virtual string CreateDbTablesScript(string schema)
{ {
var batchSql = $@" var batchSql = $@"
......
...@@ -11,14 +11,15 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -11,14 +11,15 @@ namespace DotNetCore.CAP.PostgreSql
{ {
public class PostgreSqlStorageConnection : IStorageConnection public class PostgreSqlStorageConnection : IStorageConnection
{ {
private readonly PostgreSqlOptions _options; private readonly CapOptions _capOptions;
public PostgreSqlStorageConnection(PostgreSqlOptions options) public PostgreSqlStorageConnection(PostgreSqlOptions options, CapOptions capOptions)
{ {
_options = options; _capOptions = capOptions;
Options = options;
} }
public PostgreSqlOptions Options => _options; public PostgreSqlOptions Options { get; }
public IStorageTransaction CreateTransaction() public IStorageTransaction CreateTransaction()
{ {
...@@ -27,9 +28,9 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -27,9 +28,9 @@ namespace DotNetCore.CAP.PostgreSql
public async Task<CapPublishedMessage> GetPublishedMessageAsync(int id) public async Task<CapPublishedMessage> GetPublishedMessageAsync(int id)
{ {
var sql = $"SELECT * FROM \"{_options.Schema}\".\"published\" WHERE \"Id\"={id}"; var sql = $"SELECT * FROM \"{Options.Schema}\".\"published\" WHERE \"Id\"={id} FOR UPDATE SKIP LOCKED";
using (var connection = new NpgsqlConnection(_options.ConnectionString)) using (var connection = new NpgsqlConnection(Options.ConnectionString))
{ {
return await connection.QueryFirstOrDefaultAsync<CapPublishedMessage>(sql); return await connection.QueryFirstOrDefaultAsync<CapPublishedMessage>(sql);
} }
...@@ -37,15 +38,16 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -37,15 +38,16 @@ namespace DotNetCore.CAP.PostgreSql
public Task<IFetchedMessage> FetchNextMessageAsync() public Task<IFetchedMessage> FetchNextMessageAsync()
{ {
var sql = $@"DELETE FROM ""{_options.Schema}"".""queue"" WHERE ""MessageId"" = (SELECT ""MessageId"" FROM ""{_options.Schema}"".""queue"" FOR UPDATE SKIP LOCKED LIMIT 1) RETURNING *;"; var sql = $@"DELETE FROM ""{Options.Schema}"".""queue"" WHERE ""MessageId"" = (SELECT ""MessageId"" FROM ""{Options.Schema}"".""queue"" FOR UPDATE SKIP LOCKED LIMIT 1) RETURNING *;";
return FetchNextMessageCoreAsync(sql); return FetchNextMessageCoreAsync(sql);
} }
public async Task<CapPublishedMessage> GetNextPublishedMessageToBeEnqueuedAsync() public async Task<CapPublishedMessage> GetNextPublishedMessageToBeEnqueuedAsync()
{ {
var sql = $"SELECT * FROM \"{_options.Schema}\".\"published\" WHERE \"StatusName\" = '{StatusName.Scheduled}' FOR UPDATE SKIP LOCKED LIMIT 1;"; var sql =
$"SELECT * FROM \"{Options.Schema}\".\"published\" WHERE \"StatusName\" = '{StatusName.Scheduled}' FOR UPDATE SKIP LOCKED LIMIT 1;";
using (var connection = new NpgsqlConnection(_options.ConnectionString)) using (var connection = new NpgsqlConnection(Options.ConnectionString))
{ {
return await connection.QueryFirstOrDefaultAsync<CapPublishedMessage>(sql); return await connection.QueryFirstOrDefaultAsync<CapPublishedMessage>(sql);
} }
...@@ -53,23 +55,23 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -53,23 +55,23 @@ namespace DotNetCore.CAP.PostgreSql
public async Task<IEnumerable<CapPublishedMessage>> GetFailedPublishedMessages() public async Task<IEnumerable<CapPublishedMessage>> GetFailedPublishedMessages()
{ {
var sql = $"SELECT * FROM \"{_options.Schema}\".\"published\" WHERE \"StatusName\"='{StatusName.Failed}' LIMIT 1000;"; var sql =
$"SELECT * FROM \"{Options.Schema}\".\"published\" WHERE \"Retries\"<{_capOptions.FailedRetryCount} AND \"StatusName\"='{StatusName.Failed}' LIMIT 200;";
using (var connection = new NpgsqlConnection(_options.ConnectionString)) using (var connection = new NpgsqlConnection(Options.ConnectionString))
{ {
return await connection.QueryAsync<CapPublishedMessage>(sql); return await connection.QueryAsync<CapPublishedMessage>(sql);
} }
} }
// CapReceviedMessage
public async Task StoreReceivedMessageAsync(CapReceivedMessage message) public async Task StoreReceivedMessageAsync(CapReceivedMessage message)
{ {
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
var sql = $"INSERT INTO \"{_options.Schema}\".\"received\"(\"Name\",\"Group\",\"Content\",\"Retries\",\"Added\",\"ExpiresAt\",\"StatusName\")VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);"; var sql =
$"INSERT INTO \"{Options.Schema}\".\"received\"(\"Name\",\"Group\",\"Content\",\"Retries\",\"Added\",\"ExpiresAt\",\"StatusName\")VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);";
using (var connection = new NpgsqlConnection(_options.ConnectionString)) using (var connection = new NpgsqlConnection(Options.ConnectionString))
{ {
await connection.ExecuteAsync(sql, message); await connection.ExecuteAsync(sql, message);
} }
...@@ -77,26 +79,28 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -77,26 +79,28 @@ namespace DotNetCore.CAP.PostgreSql
public async Task<CapReceivedMessage> GetReceivedMessageAsync(int id) public async Task<CapReceivedMessage> GetReceivedMessageAsync(int id)
{ {
var sql = $"SELECT * FROM \"{_options.Schema}\".\"received\" WHERE \"Id\"={id}"; var sql = $"SELECT * FROM \"{Options.Schema}\".\"received\" WHERE \"Id\"={id} FOR UPDATE SKIP LOCKED";
using (var connection = new NpgsqlConnection(_options.ConnectionString)) using (var connection = new NpgsqlConnection(Options.ConnectionString))
{ {
return await connection.QueryFirstOrDefaultAsync<CapReceivedMessage>(sql); return await connection.QueryFirstOrDefaultAsync<CapReceivedMessage>(sql);
} }
} }
public async Task<CapReceivedMessage> GetNextReceviedMessageToBeEnqueuedAsync() public async Task<CapReceivedMessage> GetNextReceivedMessageToBeEnqueuedAsync()
{ {
var sql = $"SELECT * FROM \"{_options.Schema}\".\"received\" WHERE \"StatusName\" = '{StatusName.Scheduled}' FOR UPDATE SKIP LOCKED LIMIT 1;"; var sql =
using (var connection = new NpgsqlConnection(_options.ConnectionString)) $"SELECT * FROM \"{Options.Schema}\".\"received\" WHERE \"StatusName\" = '{StatusName.Scheduled}' FOR UPDATE SKIP LOCKED LIMIT 1;";
using (var connection = new NpgsqlConnection(Options.ConnectionString))
{ {
return await connection.QueryFirstOrDefaultAsync<CapReceivedMessage>(sql); return await connection.QueryFirstOrDefaultAsync<CapReceivedMessage>(sql);
} }
} }
public async Task<IEnumerable<CapReceivedMessage>> GetFailedReceviedMessages() public async Task<IEnumerable<CapReceivedMessage>> GetFailedReceivedMessages()
{ {
var sql = $"SELECT * FROM \"{_options.Schema}\".\"received\" WHERE \"StatusName\"='{StatusName.Failed}' LIMIT 1000;"; var sql =
using (var connection = new NpgsqlConnection(_options.ConnectionString)) $"SELECT * FROM \"{Options.Schema}\".\"received\" WHERE \"Retries\"<{_capOptions.FailedRetryCount} AND \"StatusName\"='{StatusName.Failed}' LIMIT 200;";
using (var connection = new NpgsqlConnection(Options.ConnectionString))
{ {
return await connection.QueryAsync<CapReceivedMessage>(sql); return await connection.QueryAsync<CapReceivedMessage>(sql);
} }
...@@ -106,13 +110,35 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -106,13 +110,35 @@ namespace DotNetCore.CAP.PostgreSql
{ {
} }
public bool ChangePublishedState(int messageId, string state)
{
var sql =
$"UPDATE \"{Options.Schema}\".\"published\" SET \"Retries\"=\"Retries\"+1,\"StatusName\" = '{state}' WHERE \"Id\"={messageId}";
using (var connection = new NpgsqlConnection(Options.ConnectionString))
{
return connection.Execute(sql) > 0;
}
}
public bool ChangeReceivedState(int messageId, string state)
{
var sql =
$"UPDATE \"{Options.Schema}\".\"received\" SET \"Retries\"=\"Retries\"+1,\"StatusName\" = '{state}' WHERE \"Id\"={messageId}";
using (var connection = new NpgsqlConnection(Options.ConnectionString))
{
return connection.Execute(sql) > 0;
}
}
private async Task<IFetchedMessage> FetchNextMessageCoreAsync(string sql, object args = null) private async Task<IFetchedMessage> FetchNextMessageCoreAsync(string sql, object args = null)
{ {
//here don't use `using` to dispose //here don't use `using` to dispose
var connection = new NpgsqlConnection(_options.ConnectionString); var connection = new NpgsqlConnection(Options.ConnectionString);
await connection.OpenAsync(); await connection.OpenAsync();
var transaction = connection.BeginTransaction(IsolationLevel.ReadCommitted); var transaction = connection.BeginTransaction(IsolationLevel.ReadCommitted);
FetchedMessage fetchedMessage = null; FetchedMessage fetchedMessage;
try try
{ {
fetchedMessage = await connection.QueryFirstOrDefaultAsync<FetchedMessage>(sql, args, transaction); fetchedMessage = await connection.QueryFirstOrDefaultAsync<FetchedMessage>(sql, args, transaction);
...@@ -131,7 +157,8 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -131,7 +157,8 @@ namespace DotNetCore.CAP.PostgreSql
return null; return null;
} }
return new PostgreSqlFetchedMessage(fetchedMessage.MessageId, fetchedMessage.MessageType, connection, transaction); return new PostgreSqlFetchedMessage(fetchedMessage.MessageId, fetchedMessage.MessageType, connection,
transaction);
} }
} }
} }
\ No newline at end of file
...@@ -7,12 +7,12 @@ using Npgsql; ...@@ -7,12 +7,12 @@ using Npgsql;
namespace DotNetCore.CAP.PostgreSql namespace DotNetCore.CAP.PostgreSql
{ {
public class PostgreSqlStorageTransaction : IStorageTransaction, IDisposable public class PostgreSqlStorageTransaction : IStorageTransaction
{ {
private readonly string _schema; private readonly IDbConnection _dbConnection;
private readonly IDbTransaction _dbTransaction; private readonly IDbTransaction _dbTransaction;
private readonly IDbConnection _dbConnection; private readonly string _schema;
public PostgreSqlStorageTransaction(PostgreSqlStorageConnection connection) public PostgreSqlStorageTransaction(PostgreSqlStorageConnection connection)
{ {
...@@ -28,7 +28,10 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -28,7 +28,10 @@ namespace DotNetCore.CAP.PostgreSql
{ {
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
var sql = $@"UPDATE ""{_schema}"".""published"" SET ""Retries""=@Retries,""ExpiresAt""=@ExpiresAt,""StatusName""=@StatusName WHERE ""Id""=@Id;"; var sql =
$@"UPDATE ""{
_schema
}"".""published"" SET ""Retries""=@Retries,""Content""= @Content,""ExpiresAt""=@ExpiresAt,""StatusName""=@StatusName WHERE ""Id""=@Id;";
_dbConnection.Execute(sql, message, _dbTransaction); _dbConnection.Execute(sql, message, _dbTransaction);
} }
...@@ -36,7 +39,10 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -36,7 +39,10 @@ namespace DotNetCore.CAP.PostgreSql
{ {
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
var sql = $@"UPDATE ""{_schema}"".""received"" SET ""Retries""=@Retries,""ExpiresAt""=@ExpiresAt,""StatusName""=@StatusName WHERE ""Id""=@Id;"; var sql =
$@"UPDATE ""{
_schema
}"".""received"" SET ""Retries""=@Retries,""Content""= @Content,""ExpiresAt""=@ExpiresAt,""StatusName""=@StatusName WHERE ""Id""=@Id;";
_dbConnection.Execute(sql, message, _dbTransaction); _dbConnection.Execute(sql, message, _dbTransaction);
} }
...@@ -45,7 +51,8 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -45,7 +51,8 @@ namespace DotNetCore.CAP.PostgreSql
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
var sql = $@"INSERT INTO ""{_schema}"".""queue"" values(@MessageId,@MessageType);"; var sql = $@"INSERT INTO ""{_schema}"".""queue"" values(@MessageId,@MessageType);";
_dbConnection.Execute(sql, new CapQueue { MessageId = message.Id, MessageType = MessageType.Publish }, _dbTransaction); _dbConnection.Execute(sql, new CapQueue {MessageId = message.Id, MessageType = MessageType.Publish},
_dbTransaction);
} }
public void EnqueueMessage(CapReceivedMessage message) public void EnqueueMessage(CapReceivedMessage message)
...@@ -53,7 +60,8 @@ namespace DotNetCore.CAP.PostgreSql ...@@ -53,7 +60,8 @@ namespace DotNetCore.CAP.PostgreSql
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
var sql = $@"INSERT INTO ""{_schema}"".""queue"" values(@MessageId,@MessageType);"; var sql = $@"INSERT INTO ""{_schema}"".""queue"" values(@MessageId,@MessageType);";
_dbConnection.Execute(sql, new CapQueue { MessageId = message.Id, MessageType = MessageType.Subscribe }, _dbTransaction); _dbConnection.Execute(sql, new CapQueue {MessageId = message.Id, MessageType = MessageType.Subscribe},
_dbTransaction);
} }
public Task CommitAsync() public Task CommitAsync()
......
...@@ -8,10 +8,7 @@ namespace Microsoft.Extensions.DependencyInjection ...@@ -8,10 +8,7 @@ namespace Microsoft.Extensions.DependencyInjection
{ {
public static CapOptions UseRabbitMQ(this CapOptions options, string hostName) public static CapOptions UseRabbitMQ(this CapOptions options, string hostName)
{ {
return options.UseRabbitMQ(opt => return options.UseRabbitMQ(opt => { opt.HostName = hostName; });
{
opt.HostName = hostName;
});
} }
public static CapOptions UseRabbitMQ(this CapOptions options, Action<RabbitMQOptions> configure) public static CapOptions UseRabbitMQ(this CapOptions options, Action<RabbitMQOptions> configure)
......
using System; // ReSharper disable once CheckNamespace
// ReSharper disable once CheckNamespace
namespace DotNetCore.CAP namespace DotNetCore.CAP
{ {
public class RabbitMQOptions public class RabbitMQOptions
......
...@@ -16,15 +16,16 @@ namespace DotNetCore.CAP ...@@ -16,15 +16,16 @@ namespace DotNetCore.CAP
public void AddServices(IServiceCollection services) public void AddServices(IServiceCollection services)
{ {
services.AddSingleton<CapMessageQueueMakerService>();
var options = new RabbitMQOptions(); var options = new RabbitMQOptions();
_configure?.Invoke(options); _configure?.Invoke(options);
services.AddSingleton(options); services.AddSingleton(options);
services.AddSingleton<IConsumerClientFactory, RabbitMQConsumerClientFactory>(); services.AddSingleton<IConsumerClientFactory, RabbitMQConsumerClientFactory>();
services.AddSingleton<IConnectionChannelPool, ConnectionChannelPool>();
services.AddSingleton<ConnectionPool>();
services.AddSingleton<IQueueExecutor, PublishQueueExecutor>(); services.AddSingleton<IQueueExecutor, PublishQueueExecutor>();
services.AddSingleton<IPublishExecutor, PublishQueueExecutor>();
} }
} }
} }
\ No newline at end of file
...@@ -2,31 +2,61 @@ ...@@ -2,31 +2,61 @@
using System.Collections.Concurrent; using System.Collections.Concurrent;
using System.Diagnostics; using System.Diagnostics;
using System.Threading; using System.Threading;
using Microsoft.Extensions.Logging;
using RabbitMQ.Client; using RabbitMQ.Client;
namespace DotNetCore.CAP.RabbitMQ namespace DotNetCore.CAP.RabbitMQ
{ {
public class ConnectionPool : IConnectionPool, IDisposable public class ConnectionChannelPool : IConnectionChannelPool, IDisposable
{ {
private const int DefaultPoolSize = 15; private const int DefaultPoolSize = 15;
private readonly Func<IConnection> _connectionActivator;
private readonly ILogger<ConnectionChannelPool> _logger;
private readonly ConcurrentQueue<IModel> _pool = new ConcurrentQueue<IModel>();
private IConnection _connection;
private readonly ConcurrentQueue<IConnection> _pool = new ConcurrentQueue<IConnection>();
private readonly Func<IConnection> _activator;
private int _maxSize;
private int _count; private int _count;
private int _maxSize;
public ConnectionPool(RabbitMQOptions options) public ConnectionChannelPool(ILogger<ConnectionChannelPool> logger,
RabbitMQOptions options)
{ {
_logger = logger;
_maxSize = DefaultPoolSize; _maxSize = DefaultPoolSize;
_activator = CreateActivator(options); _connectionActivator = CreateConnection(options);
}
IModel IConnectionChannelPool.Rent()
{
return Rent();
}
bool IConnectionChannelPool.Return(IModel connection)
{
return Return(connection);
}
public IConnection GetConnection()
{
if (_connection != null && _connection.IsOpen)
return _connection;
_connection = _connectionActivator();
_connection.ConnectionShutdown += RabbitMQ_ConnectionShutdown;
return _connection;
}
public void Dispose()
{
_maxSize = 0;
while (_pool.TryDequeue(out var context))
context.Dispose();
} }
private static Func<IConnection> CreateActivator(RabbitMQOptions options) private static Func<IConnection> CreateConnection(RabbitMQOptions options)
{ {
var factory = new ConnectionFactory() var factory = new ConnectionFactory
{ {
HostName = options.HostName, HostName = options.HostName,
UserName = options.UserName, UserName = options.UserName,
...@@ -41,23 +71,28 @@ namespace DotNetCore.CAP.RabbitMQ ...@@ -41,23 +71,28 @@ namespace DotNetCore.CAP.RabbitMQ
return () => factory.CreateConnection(); return () => factory.CreateConnection();
} }
public virtual IConnection Rent() private void RabbitMQ_ConnectionShutdown(object sender, ShutdownEventArgs e)
{ {
if (_pool.TryDequeue(out IConnection connection)) _logger.LogWarning($"RabbitMQ client connection closed! {e}");
}
public virtual IModel Rent()
{
if (_pool.TryDequeue(out var model))
{ {
Interlocked.Decrement(ref _count); Interlocked.Decrement(ref _count);
Debug.Assert(_count >= 0); Debug.Assert(_count >= 0);
return connection; return model;
} }
connection = _activator(); model = GetConnection().CreateModel();
return connection; return model;
} }
public virtual bool Return(IConnection connection) public virtual bool Return(IModel connection)
{ {
if (Interlocked.Increment(ref _count) <= _maxSize) if (Interlocked.Increment(ref _count) <= _maxSize)
{ {
...@@ -72,20 +107,5 @@ namespace DotNetCore.CAP.RabbitMQ ...@@ -72,20 +107,5 @@ namespace DotNetCore.CAP.RabbitMQ
return false; return false;
} }
IConnection IConnectionPool.Rent() => Rent();
bool IConnectionPool.Return(IConnection connection) => Return(connection);
public void Dispose()
{
_maxSize = 0;
IConnection context;
while (_pool.TryDequeue(out context))
{
context.Dispose();
}
}
} }
} }
\ No newline at end of file
using RabbitMQ.Client;
namespace DotNetCore.CAP.RabbitMQ
{
public interface IConnectionChannelPool
{
IConnection GetConnection();
IModel Rent();
bool Return(IModel context);
}
}
\ No newline at end of file
using System;
using System.Collections.Generic;
using System.Text;
using RabbitMQ.Client;
namespace DotNetCore.CAP.RabbitMQ
{
public interface IConnectionPool
{
IConnection Rent();
bool Return(IConnection context);
}
}
...@@ -9,49 +9,43 @@ namespace DotNetCore.CAP.RabbitMQ ...@@ -9,49 +9,43 @@ namespace DotNetCore.CAP.RabbitMQ
{ {
internal sealed class PublishQueueExecutor : BasePublishQueueExecutor internal sealed class PublishQueueExecutor : BasePublishQueueExecutor
{ {
private readonly IConnectionChannelPool _connectionChannelPool;
private readonly ILogger _logger; private readonly ILogger _logger;
private readonly ConnectionPool _connectionPool;
private readonly RabbitMQOptions _rabbitMQOptions; private readonly RabbitMQOptions _rabbitMQOptions;
public PublishQueueExecutor( public PublishQueueExecutor(ILogger<PublishQueueExecutor> logger, CapOptions options,
CapOptions options, RabbitMQOptions rabbitMQOptions, IConnectionChannelPool connectionChannelPool, IStateChanger stateChanger)
IStateChanger stateChanger,
ConnectionPool connectionPool,
RabbitMQOptions rabbitMQOptions,
ILogger<PublishQueueExecutor> logger)
: base(options, stateChanger, logger) : base(options, stateChanger, logger)
{ {
_logger = logger; _logger = logger;
_connectionPool = connectionPool; _connectionChannelPool = connectionChannelPool;
_rabbitMQOptions = rabbitMQOptions; _rabbitMQOptions = rabbitMQOptions;
} }
public override Task<OperateResult> PublishAsync(string keyName, string content) public override Task<OperateResult> PublishAsync(string keyName, string content)
{ {
var connection = _connectionPool.Rent(); var channel = _connectionChannelPool.Rent();
try try
{ {
using (var channel = connection.CreateModel()) var body = Encoding.UTF8.GetBytes(content);
{
var body = Encoding.UTF8.GetBytes(content); channel.ExchangeDeclare(_rabbitMQOptions.TopicExchangeName, RabbitMQOptions.ExchangeType, true);
channel.BasicPublish(_rabbitMQOptions.TopicExchangeName,
keyName,
null,
body);
channel.ExchangeDeclare(_rabbitMQOptions.TopicExchangeName, RabbitMQOptions.ExchangeType, durable: true); _logger.LogDebug($"RabbitMQ topic message [{keyName}] has been published.");
channel.BasicPublish(exchange: _rabbitMQOptions.TopicExchangeName,
routingKey: keyName,
basicProperties: null,
body: body);
_logger.LogDebug($"rabbitmq topic message [{keyName}] has been published.");
}
return Task.FromResult(OperateResult.Success); return Task.FromResult(OperateResult.Success);
} }
catch (Exception ex) catch (Exception ex)
{ {
_logger.LogError($"rabbitmq topic message [{keyName}] has benn raised an exception of sending. the exception is: {ex.Message}"); _logger.LogError(
$"RabbitMQ topic message [{keyName}] has been raised an exception of sending. the exception is: {ex.Message}");
return Task.FromResult(OperateResult.Failed(ex, return Task.FromResult(OperateResult.Failed(ex,
new OperateError() new OperateError
{ {
Code = ex.HResult.ToString(), Code = ex.HResult.ToString(),
Description = ex.Message Description = ex.Message
...@@ -59,7 +53,9 @@ namespace DotNetCore.CAP.RabbitMQ ...@@ -59,7 +53,9 @@ namespace DotNetCore.CAP.RabbitMQ
} }
finally finally
{ {
_connectionPool.Return(connection); var returned = _connectionChannelPool.Return(channel);
if (!returned)
channel.Dispose();
} }
} }
} }
......
...@@ -10,59 +10,36 @@ namespace DotNetCore.CAP.RabbitMQ ...@@ -10,59 +10,36 @@ namespace DotNetCore.CAP.RabbitMQ
{ {
internal sealed class RabbitMQConsumerClient : IConsumerClient internal sealed class RabbitMQConsumerClient : IConsumerClient
{ {
private readonly IConnectionChannelPool _connectionChannelPool;
private readonly string _exchageName; private readonly string _exchageName;
private readonly string _queueName; private readonly string _queueName;
private readonly RabbitMQOptions _rabbitMQOptions; private readonly RabbitMQOptions _rabbitMQOptions;
private ConnectionPool _connectionPool;
private IModel _channel; private IModel _channel;
private ulong _deliveryTag; private ulong _deliveryTag;
public event EventHandler<MessageContext> OnMessageReceieved;
public event EventHandler<string> OnError;
public RabbitMQConsumerClient(string queueName, public RabbitMQConsumerClient(string queueName,
ConnectionPool connectionPool, IConnectionChannelPool connectionChannelPool,
RabbitMQOptions options) RabbitMQOptions options)
{ {
_queueName = queueName; _queueName = queueName;
_connectionPool = connectionPool; _connectionChannelPool = connectionChannelPool;
_rabbitMQOptions = options; _rabbitMQOptions = options;
_exchageName = options.TopicExchangeName; _exchageName = options.TopicExchangeName;
InitClient(); InitClient();
} }
private void InitClient() public event EventHandler<MessageContext> OnMessageReceived;
{
var connection = _connectionPool.Rent();
_channel = connection.CreateModel();
_channel.ExchangeDeclare( public event EventHandler<string> OnError;
exchange: _exchageName,
type: RabbitMQOptions.ExchangeType,
durable: true);
var arguments = new Dictionary<string, object> { { "x-message-ttl", (int)_rabbitMQOptions.QueueMessageExpires } };
_channel.QueueDeclare(_queueName,
durable: true,
exclusive: false,
autoDelete: false,
arguments: arguments);
_connectionPool.Return(connection);
}
public void Subscribe(IEnumerable<string> topics) public void Subscribe(IEnumerable<string> topics)
{ {
if (topics == null) throw new ArgumentNullException(nameof(topics)); if (topics == null) throw new ArgumentNullException(nameof(topics));
foreach (var topic in topics) foreach (var topic in topics)
{
_channel.QueueBind(_queueName, _exchageName, topic); _channel.QueueBind(_queueName, _exchageName, topic);
}
} }
public void Listening(TimeSpan timeout, CancellationToken cancellationToken) public void Listening(TimeSpan timeout, CancellationToken cancellationToken)
...@@ -72,9 +49,7 @@ namespace DotNetCore.CAP.RabbitMQ ...@@ -72,9 +49,7 @@ namespace DotNetCore.CAP.RabbitMQ
consumer.Shutdown += OnConsumerShutdown; consumer.Shutdown += OnConsumerShutdown;
_channel.BasicConsume(_queueName, false, consumer); _channel.BasicConsume(_queueName, false, consumer);
while (true) while (true)
{
Task.Delay(timeout, cancellationToken).GetAwaiter().GetResult(); Task.Delay(timeout, cancellationToken).GetAwaiter().GetResult();
}
} }
public void Commit() public void Commit()
...@@ -82,11 +57,33 @@ namespace DotNetCore.CAP.RabbitMQ ...@@ -82,11 +57,33 @@ namespace DotNetCore.CAP.RabbitMQ
_channel.BasicAck(_deliveryTag, false); _channel.BasicAck(_deliveryTag, false);
} }
public void Reject()
{
_channel.BasicReject(_deliveryTag, true);
}
public void Dispose() public void Dispose()
{ {
_channel.Dispose(); _channel.Dispose();
} }
private void InitClient()
{
var connection = _connectionChannelPool.GetConnection();
_channel = connection.CreateModel();
_channel.ExchangeDeclare(
_exchageName,
RabbitMQOptions.ExchangeType,
true);
var arguments = new Dictionary<string, object> {
{ "x-message-ttl", _rabbitMQOptions.QueueMessageExpires }
};
_channel.QueueDeclare(_queueName, true, false, false, arguments);
}
private void OnConsumerReceived(object sender, BasicDeliverEventArgs e) private void OnConsumerReceived(object sender, BasicDeliverEventArgs e)
{ {
_deliveryTag = e.DeliveryTag; _deliveryTag = e.DeliveryTag;
...@@ -96,7 +93,7 @@ namespace DotNetCore.CAP.RabbitMQ ...@@ -96,7 +93,7 @@ namespace DotNetCore.CAP.RabbitMQ
Name = e.RoutingKey, Name = e.RoutingKey,
Content = Encoding.UTF8.GetString(e.Body) Content = Encoding.UTF8.GetString(e.Body)
}; };
OnMessageReceieved?.Invoke(sender, message); OnMessageReceived?.Invoke(sender, message);
} }
private void OnConsumerShutdown(object sender, ShutdownEventArgs e) private void OnConsumerShutdown(object sender, ShutdownEventArgs e)
......
using Microsoft.Extensions.Options; namespace DotNetCore.CAP.RabbitMQ
using RabbitMQ.Client;
namespace DotNetCore.CAP.RabbitMQ
{ {
internal sealed class RabbitMQConsumerClientFactory : IConsumerClientFactory internal sealed class RabbitMQConsumerClientFactory : IConsumerClientFactory
{ {
private readonly IConnectionChannelPool _connectionChannelPool;
private readonly RabbitMQOptions _rabbitMQOptions; private readonly RabbitMQOptions _rabbitMQOptions;
private readonly ConnectionPool _connectionPool;
public RabbitMQConsumerClientFactory(RabbitMQOptions rabbitMQOptions, ConnectionPool pool) public RabbitMQConsumerClientFactory(RabbitMQOptions rabbitMQOptions, IConnectionChannelPool channelPool)
{ {
_rabbitMQOptions = rabbitMQOptions; _rabbitMQOptions = rabbitMQOptions;
_connectionPool = pool; _connectionChannelPool = channelPool;
} }
public IConsumerClient Create(string groupId) public IConsumerClient Create(string groupId)
{ {
return new RabbitMQConsumerClient(groupId, _connectionPool, _rabbitMQOptions); return new RabbitMQConsumerClient(groupId, _connectionChannelPool, _rabbitMQOptions);
} }
} }
} }
\ No newline at end of file
...@@ -9,7 +9,7 @@ namespace DotNetCore.CAP ...@@ -9,7 +9,7 @@ namespace DotNetCore.CAP
/// <summary> /// <summary>
/// Gets or sets the schema to use when creating database objects. /// Gets or sets the schema to use when creating database objects.
/// Default is <see cref="DefaultSchema"/>. /// Default is <see cref="DefaultSchema" />.
/// </summary> /// </summary>
public string Schema { get; set; } = DefaultSchema; public string Schema { get; set; } = DefaultSchema;
......
...@@ -9,10 +9,7 @@ namespace Microsoft.Extensions.DependencyInjection ...@@ -9,10 +9,7 @@ namespace Microsoft.Extensions.DependencyInjection
{ {
public static CapOptions UseSqlServer(this CapOptions options, string connectionString) public static CapOptions UseSqlServer(this CapOptions options, string connectionString)
{ {
return options.UseSqlServer(opt => return options.UseSqlServer(opt => { opt.ConnectionString = connectionString; });
{
opt.ConnectionString = connectionString;
});
} }
public static CapOptions UseSqlServer(this CapOptions options, Action<SqlServerOptions> configure) public static CapOptions UseSqlServer(this CapOptions options, Action<SqlServerOptions> configure)
...@@ -27,10 +24,7 @@ namespace Microsoft.Extensions.DependencyInjection ...@@ -27,10 +24,7 @@ namespace Microsoft.Extensions.DependencyInjection
public static CapOptions UseEntityFramework<TContext>(this CapOptions options) public static CapOptions UseEntityFramework<TContext>(this CapOptions options)
where TContext : DbContext where TContext : DbContext
{ {
return options.UseEntityFramework<TContext>(opt => return options.UseEntityFramework<TContext>(opt => { opt.DbContextType = typeof(TContext); });
{
opt.DbContextType = typeof(TContext);
});
} }
public static CapOptions UseEntityFramework<TContext>(this CapOptions options, Action<EFOptions> configure) public static CapOptions UseEntityFramework<TContext>(this CapOptions options, Action<EFOptions> configure)
...@@ -38,7 +32,7 @@ namespace Microsoft.Extensions.DependencyInjection ...@@ -38,7 +32,7 @@ namespace Microsoft.Extensions.DependencyInjection
{ {
if (configure == null) throw new ArgumentNullException(nameof(configure)); if (configure == null) throw new ArgumentNullException(nameof(configure));
var efOptions = new EFOptions { DbContextType = typeof(TContext) }; var efOptions = new EFOptions {DbContextType = typeof(TContext)};
configure(efOptions); configure(efOptions);
options.RegisterExtension(new SqlServerCapOptionsExtension(configure)); options.RegisterExtension(new SqlServerCapOptionsExtension(configure));
......
...@@ -18,10 +18,11 @@ namespace DotNetCore.CAP ...@@ -18,10 +18,11 @@ namespace DotNetCore.CAP
public void AddServices(IServiceCollection services) public void AddServices(IServiceCollection services)
{ {
services.AddSingleton<CapDatabaseStorageMarkerService>();
services.AddSingleton<IStorage, SqlServerStorage>(); services.AddSingleton<IStorage, SqlServerStorage>();
services.AddSingleton<IStorageConnection, SqlServerStorageConnection>(); services.AddSingleton<IStorageConnection, SqlServerStorageConnection>();
services.AddTransient<ICapPublisher, CapPublisher>(); services.AddScoped<ICapPublisher, CapPublisher>();
services.AddTransient<ICallbackPublisher, CapPublisher>(); services.AddScoped<ICallbackPublisher, CapPublisher>();
services.AddTransient<IAdditionalProcessor, DefaultAdditionalProcessor>(); services.AddTransient<IAdditionalProcessor, DefaultAdditionalProcessor>();
AddSqlServerOptions(services); AddSqlServerOptions(services);
} }
...@@ -33,22 +34,18 @@ namespace DotNetCore.CAP ...@@ -33,22 +34,18 @@ namespace DotNetCore.CAP
_configure(sqlServerOptions); _configure(sqlServerOptions);
if (sqlServerOptions.DbContextType != null) if (sqlServerOptions.DbContextType != null)
{
services.AddSingleton(x => services.AddSingleton(x =>
{ {
using (var scope = x.CreateScope()) using (var scope = x.CreateScope())
{ {
var provider = scope.ServiceProvider; var provider = scope.ServiceProvider;
var dbContext = (DbContext)provider.GetService(sqlServerOptions.DbContextType); var dbContext = (DbContext) provider.GetService(sqlServerOptions.DbContextType);
sqlServerOptions.ConnectionString = dbContext.Database.GetDbConnection().ConnectionString; sqlServerOptions.ConnectionString = dbContext.Database.GetDbConnection().ConnectionString;
return sqlServerOptions; return sqlServerOptions;
} }
}); });
}
else else
{
services.AddSingleton(sqlServerOptions); services.AddSingleton(sqlServerOptions);
}
} }
} }
} }
\ No newline at end of file
// ReSharper disable once CheckNamespace // ReSharper disable once CheckNamespace
namespace DotNetCore.CAP namespace DotNetCore.CAP
{ {
public class SqlServerOptions : EFOptions public class SqlServerOptions : EFOptions
......
...@@ -13,9 +13,9 @@ namespace DotNetCore.CAP.SqlServer ...@@ -13,9 +13,9 @@ namespace DotNetCore.CAP.SqlServer
{ {
public class CapPublisher : CapPublisherBase, ICallbackPublisher public class CapPublisher : CapPublisherBase, ICallbackPublisher
{ {
private readonly DbContext _dbContext;
private readonly ILogger _logger; private readonly ILogger _logger;
private readonly SqlServerOptions _options; private readonly SqlServerOptions _options;
private readonly DbContext _dbContext;
public CapPublisher(IServiceProvider provider, public CapPublisher(IServiceProvider provider,
ILogger<CapPublisher> logger, ILogger<CapPublisher> logger,
...@@ -25,10 +25,17 @@ namespace DotNetCore.CAP.SqlServer ...@@ -25,10 +25,17 @@ namespace DotNetCore.CAP.SqlServer
_logger = logger; _logger = logger;
_options = options; _options = options;
if (_options.DbContextType != null) if (_options.DbContextType == null) return;
IsUsingEF = true;
_dbContext = (DbContext)ServiceProvider.GetService(_options.DbContextType);
}
public async Task PublishAsync(CapPublishedMessage message)
{
using (var conn = new SqlConnection(_options.ConnectionString))
{ {
IsUsingEF = true; await conn.ExecuteAsync(PrepareSql(), message);
_dbContext = (DbContext)ServiceProvider.GetService(_options.DbContextType);
} }
} }
...@@ -45,36 +52,33 @@ namespace DotNetCore.CAP.SqlServer ...@@ -45,36 +52,33 @@ namespace DotNetCore.CAP.SqlServer
dbContextTransaction = _dbContext.Database.BeginTransaction(IsolationLevel.ReadCommitted); dbContextTransaction = _dbContext.Database.BeginTransaction(IsolationLevel.ReadCommitted);
dbTrans = dbContextTransaction.GetDbTransaction(); dbTrans = dbContextTransaction.GetDbTransaction();
} }
DbTranasaction = dbTrans; DbTransaction = dbTrans;
} }
protected override void Execute(IDbConnection dbConnection, IDbTransaction dbTransaction, CapPublishedMessage message) protected override void Execute(IDbConnection dbConnection, IDbTransaction dbTransaction,
CapPublishedMessage message)
{ {
dbConnection.Execute(PrepareSql(), message, dbTransaction); dbConnection.Execute(PrepareSql(), message, dbTransaction);
_logger.LogInformation("Published Message has been persisted in the database. name:" + message.ToString()); _logger.LogInformation("Published Message has been persisted in the database. name:" + message);
} }
protected override async Task ExecuteAsync(IDbConnection dbConnection, IDbTransaction dbTransaction, CapPublishedMessage message) protected override Task ExecuteAsync(IDbConnection dbConnection, IDbTransaction dbTransaction,
CapPublishedMessage message)
{ {
await dbConnection.ExecuteAsync(PrepareSql(), message, dbTransaction); dbConnection.ExecuteAsync(PrepareSql(), message, dbTransaction);
_logger.LogInformation("Published Message has been persisted in the database. name:" + message.ToString()); _logger.LogInformation("Published Message has been persisted in the database. name:" + message);
}
public async Task PublishAsync(CapPublishedMessage message) return Task.CompletedTask;
{
using (var conn = new SqlConnection(_options.ConnectionString))
{
await conn.ExecuteAsync(PrepareSql(), message);
}
} }
#region private methods #region private methods
private string PrepareSql() private string PrepareSql()
{ {
return $"INSERT INTO {_options.Schema}.[Published] ([Name],[Content],[Retries],[Added],[ExpiresAt],[StatusName])VALUES(@Name,@Content,@Retries,@Added,@ExpiresAt,@StatusName)"; return
$"INSERT INTO {_options.Schema}.[Published] ([Name],[Content],[Retries],[Added],[ExpiresAt],[StatusName])VALUES(@Name,@Content,@Retries,@Added,@ExpiresAt,@StatusName)";
} }
#endregion private methods #endregion private methods
......
...@@ -9,26 +9,22 @@ namespace DotNetCore.CAP.SqlServer ...@@ -9,26 +9,22 @@ namespace DotNetCore.CAP.SqlServer
{ {
public class DefaultAdditionalProcessor : IAdditionalProcessor public class DefaultAdditionalProcessor : IAdditionalProcessor
{ {
private readonly IServiceProvider _provider;
private readonly ILogger _logger;
private readonly SqlServerOptions _options;
private const int MaxBatch = 1000; private const int MaxBatch = 1000;
private readonly TimeSpan _delay = TimeSpan.FromSeconds(1);
private readonly TimeSpan _waitingInterval = TimeSpan.FromMinutes(5);
private static readonly string[] Tables = private static readonly string[] Tables =
{ {
"Published","Received" "Published", "Received"
}; };
public DefaultAdditionalProcessor( private readonly TimeSpan _delay = TimeSpan.FromSeconds(1);
IServiceProvider provider, private readonly ILogger _logger;
ILogger<DefaultAdditionalProcessor> logger, private readonly SqlServerOptions _options;
private readonly TimeSpan _waitingInterval = TimeSpan.FromMinutes(5);
public DefaultAdditionalProcessor(ILogger<DefaultAdditionalProcessor> logger,
SqlServerOptions sqlServerOptions) SqlServerOptions sqlServerOptions)
{ {
_logger = logger; _logger = logger;
_provider = provider;
_options = sqlServerOptions; _options = sqlServerOptions;
} }
...@@ -38,7 +34,7 @@ namespace DotNetCore.CAP.SqlServer ...@@ -38,7 +34,7 @@ namespace DotNetCore.CAP.SqlServer
foreach (var table in Tables) foreach (var table in Tables)
{ {
var removedCount = 0; int removedCount;
do do
{ {
using (var connection = new SqlConnection(_options.ConnectionString)) using (var connection = new SqlConnection(_options.ConnectionString))
...@@ -46,7 +42,7 @@ namespace DotNetCore.CAP.SqlServer ...@@ -46,7 +42,7 @@ namespace DotNetCore.CAP.SqlServer
removedCount = await connection.ExecuteAsync($@" removedCount = await connection.ExecuteAsync($@"
DELETE TOP (@count) DELETE TOP (@count)
FROM [{_options.Schema}].[{table}] WITH (readpast) FROM [{_options.Schema}].[{table}] WITH (readpast)
WHERE ExpiresAt < @now;", new { now = DateTime.Now, count = MaxBatch }); WHERE ExpiresAt < @now;", new {now = DateTime.Now, count = MaxBatch});
} }
if (removedCount != 0) if (removedCount != 0)
......
...@@ -8,11 +8,11 @@ namespace DotNetCore.CAP.SqlServer ...@@ -8,11 +8,11 @@ namespace DotNetCore.CAP.SqlServer
{ {
public class SqlServerFetchedMessage : IFetchedMessage public class SqlServerFetchedMessage : IFetchedMessage
{ {
private readonly IDbConnection _connection;
private readonly IDbTransaction _transaction;
private readonly Timer _timer;
private static readonly TimeSpan KeepAliveInterval = TimeSpan.FromMinutes(1); private static readonly TimeSpan KeepAliveInterval = TimeSpan.FromMinutes(1);
private readonly IDbConnection _connection;
private readonly object _lockObject = new object(); private readonly object _lockObject = new object();
private readonly Timer _timer;
private readonly IDbTransaction _transaction;
public SqlServerFetchedMessage(int messageId, public SqlServerFetchedMessage(int messageId,
MessageType type, MessageType type,
......
using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using Dapper;
using DotNetCore.CAP.Dashboard;
using DotNetCore.CAP.Dashboard.Monitoring;
using DotNetCore.CAP.Infrastructure;
using DotNetCore.CAP.Models;
namespace DotNetCore.CAP.SqlServer
{
internal class SqlServerMonitoringApi : IMonitoringApi
{
private readonly SqlServerOptions _options;
private readonly SqlServerStorage _storage;
public SqlServerMonitoringApi(IStorage storage, SqlServerOptions options)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
_storage = storage as SqlServerStorage ?? throw new ArgumentNullException(nameof(storage));
}
public StatisticsDto GetStatistics()
{
var sql = string.Format(@"
set transaction isolation level read committed;
select count(Id) from [{0}].Published with (nolock) where StatusName = N'Succeeded';
select count(Id) from [{0}].Received with (nolock) where StatusName = N'Succeeded';
select count(Id) from [{0}].Published with (nolock) where StatusName = N'Failed';
select count(Id) from [{0}].Received with (nolock) where StatusName = N'Failed';
select count(Id) from [{0}].Published with (nolock) where StatusName in (N'Processing',N'Scheduled',N'Enqueued');
select count(Id) from [{0}].Received with (nolock) where StatusName in (N'Processing',N'Scheduled',N'Enqueued');",
_options.Schema);
var statistics = UseConnection(connection =>
{
var stats = new StatisticsDto();
using (var multi = connection.QueryMultiple(sql))
{
stats.PublishedSucceeded = multi.ReadSingle<int>();
stats.ReceivedSucceeded = multi.ReadSingle<int>();
stats.PublishedFailed = multi.ReadSingle<int>();
stats.ReceivedFailed = multi.ReadSingle<int>();
stats.PublishedProcessing = multi.ReadSingle<int>();
stats.ReceivedProcessing = multi.ReadSingle<int>();
}
return stats;
});
return statistics;
}
public IDictionary<DateTime, int> HourlyFailedJobs(MessageType type)
{
var tableName = type == MessageType.Publish ? "Published" : "Received";
return UseConnection(connection =>
GetHourlyTimelineStats(connection, tableName, StatusName.Failed));
}
public IDictionary<DateTime, int> HourlySucceededJobs(MessageType type)
{
var tableName = type == MessageType.Publish ? "Published" : "Received";
return UseConnection(connection =>
GetHourlyTimelineStats(connection, tableName, StatusName.Succeeded));
}
public IList<MessageDto> Messages(MessageQueryDto queryDto)
{
var tableName = queryDto.MessageType == MessageType.Publish ? "Published" : "Received";
var where = string.Empty;
if (!string.IsNullOrEmpty(queryDto.StatusName))
if (string.Equals(queryDto.StatusName, StatusName.Processing,
StringComparison.CurrentCultureIgnoreCase))
where += " and statusname in (N'Processing',N'Scheduled',N'Enqueued')";
else
where += " and statusname=@StatusName";
if (!string.IsNullOrEmpty(queryDto.Name))
where += " and name=@Name";
if (!string.IsNullOrEmpty(queryDto.Group))
where += " and group=@Group";
if (!string.IsNullOrEmpty(queryDto.Content))
where += " and content like '%@Content%'";
var sqlQuery =
$"select * from [{_options.Schema}].{tableName} where 1=1 {where} order by Added desc offset @Offset rows fetch next @Limit rows only";
return UseConnection(conn => conn.Query<MessageDto>(sqlQuery, new
{
queryDto.StatusName,
queryDto.Group,
queryDto.Name,
queryDto.Content,
Offset = queryDto.CurrentPage * queryDto.PageSize,
Limit = queryDto.PageSize
}).ToList());
}
public int PublishedFailedCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "Published", StatusName.Failed));
}
public int PublishedProcessingCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "Published", StatusName.Processing));
}
public int PublishedSucceededCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "Published", StatusName.Succeeded));
}
public int ReceivedFailedCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "Received", StatusName.Failed));
}
public int ReceivedProcessingCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "Received", StatusName.Processing));
}
public int ReceivedSucceededCount()
{
return UseConnection(conn => GetNumberOfMessage(conn, "Received", StatusName.Succeeded));
}
private int GetNumberOfMessage(IDbConnection connection, string tableName, string statusName)
{
var sqlQuery = statusName == StatusName.Processing
? $"select count(Id) from [{_options.Schema}].{tableName} with (nolock) where StatusName in (N'Processing',N'Scheduled',N'Enqueued')"
: $"select count(Id) from [{_options.Schema}].{tableName} with (nolock) where StatusName = @state";
var count = connection.ExecuteScalar<int>(sqlQuery, new {state = statusName});
return count;
}
private T UseConnection<T>(Func<IDbConnection, T> action)
{
return _storage.UseConnection(action);
}
private Dictionary<DateTime, int> GetHourlyTimelineStats(IDbConnection connection, string tableName,
string statusName)
{
var endDate = DateTime.Now;
var dates = new List<DateTime>();
for (var i = 0; i < 24; i++)
{
dates.Add(endDate);
endDate = endDate.AddHours(-1);
}
var keyMaps = dates.ToDictionary(x => x.ToString("yyyy-MM-dd-HH"), x => x);
return GetTimelineStats(connection, tableName, statusName, keyMaps);
}
private Dictionary<DateTime, int> GetTimelineStats(
IDbConnection connection,
string tableName,
string statusName,
IDictionary<string, DateTime> keyMaps)
{
//SQL Server 2012+
var sqlQuery =
$@"
with aggr as (
select FORMAT(Added,'yyyy-MM-dd-HH') as [Key],
count(id) [Count]
from [{_options.Schema}].{tableName}
where StatusName = @statusName
group by FORMAT(Added,'yyyy-MM-dd-HH')
)
select [Key], [Count] from aggr with (nolock) where [Key] in @keys;";
var valuesMap = connection.Query(
sqlQuery,
new {keys = keyMaps.Keys, statusName})
.ToDictionary(x => (string) x.Key, x => (int) x.Count);
foreach (var key in keyMaps.Keys)
if (!valuesMap.ContainsKey(key)) valuesMap.Add(key, 0);
var result = new Dictionary<DateTime, int>();
for (var i = 0; i < keyMaps.Count; i++)
{
var value = valuesMap[keyMaps.ElementAt(i).Key];
result.Add(keyMaps.ElementAt(i).Value, value);
}
return result;
}
}
}
\ No newline at end of file
using System;
using System.Data;
using System.Data.SqlClient; using System.Data.SqlClient;
using System.Threading; using System.Threading;
using System.Threading.Tasks; using System.Threading.Tasks;
using Dapper; using Dapper;
using DotNetCore.CAP.Dashboard;
using Microsoft.Extensions.Logging; using Microsoft.Extensions.Logging;
namespace DotNetCore.CAP.SqlServer namespace DotNetCore.CAP.SqlServer
{ {
public class SqlServerStorage : IStorage public class SqlServerStorage : IStorage
{ {
private readonly SqlServerOptions _options; private readonly IDbConnection _existingConnection = null;
private readonly ILogger _logger; private readonly ILogger _logger;
private readonly CapOptions _capOptions;
private readonly SqlServerOptions _options;
public SqlServerStorage(ILogger<SqlServerStorage> logger, SqlServerOptions options) public SqlServerStorage(ILogger<SqlServerStorage> logger,
CapOptions capOptions,
SqlServerOptions options)
{ {
_options = options; _options = options;
_logger = logger; _logger = logger;
_capOptions = capOptions;
}
public IStorageConnection GetConnection()
{
return new SqlServerStorageConnection(_options, _capOptions);
}
public IMonitoringApi GetMonitoringApi()
{
return new SqlServerMonitoringApi(this, _options);
} }
public async Task InitializeAsync(CancellationToken cancellationToken) public async Task InitializeAsync(CancellationToken cancellationToken)
...@@ -33,7 +51,7 @@ namespace DotNetCore.CAP.SqlServer ...@@ -33,7 +51,7 @@ namespace DotNetCore.CAP.SqlServer
protected virtual string CreateDbTablesScript(string schema) protected virtual string CreateDbTablesScript(string schema)
{ {
var batchSql = var batchSql =
$@" $@"
IF NOT EXISTS (SELECT * FROM sys.schemas WHERE name = '{schema}') IF NOT EXISTS (SELECT * FROM sys.schemas WHERE name = '{schema}')
BEGIN BEGIN
EXEC('CREATE SCHEMA {schema}') EXEC('CREATE SCHEMA {schema}')
...@@ -83,5 +101,41 @@ CREATE TABLE [{schema}].[Published]( ...@@ -83,5 +101,41 @@ CREATE TABLE [{schema}].[Published](
END;"; END;";
return batchSql; return batchSql;
} }
internal T UseConnection<T>(Func<IDbConnection, T> func)
{
IDbConnection connection = null;
try
{
connection = CreateAndOpenConnection();
return func(connection);
}
finally
{
ReleaseConnection(connection);
}
}
internal IDbConnection CreateAndOpenConnection()
{
var connection = _existingConnection ?? new SqlConnection(_options.ConnectionString);
if (connection.State == ConnectionState.Closed)
connection.Open();
return connection;
}
internal bool IsExistingConnection(IDbConnection connection)
{
return connection != null && ReferenceEquals(connection, _existingConnection);
}
internal void ReleaseConnection(IDbConnection connection)
{
if (connection != null && !IsExistingConnection(connection))
connection.Dispose();
}
} }
} }
\ No newline at end of file
...@@ -11,14 +11,15 @@ namespace DotNetCore.CAP.SqlServer ...@@ -11,14 +11,15 @@ namespace DotNetCore.CAP.SqlServer
{ {
public class SqlServerStorageConnection : IStorageConnection public class SqlServerStorageConnection : IStorageConnection
{ {
private readonly SqlServerOptions _options; private readonly CapOptions _capOptions;
public SqlServerStorageConnection(SqlServerOptions options) public SqlServerStorageConnection(SqlServerOptions options, CapOptions capOptions)
{ {
_options = options; _capOptions = capOptions;
Options = options;
} }
public SqlServerOptions Options => _options; public SqlServerOptions Options { get; }
public IStorageTransaction CreateTransaction() public IStorageTransaction CreateTransaction()
{ {
...@@ -27,9 +28,9 @@ namespace DotNetCore.CAP.SqlServer ...@@ -27,9 +28,9 @@ namespace DotNetCore.CAP.SqlServer
public async Task<CapPublishedMessage> GetPublishedMessageAsync(int id) public async Task<CapPublishedMessage> GetPublishedMessageAsync(int id)
{ {
var sql = $@"SELECT * FROM [{_options.Schema}].[Published] WITH (readpast) WHERE Id={id}"; var sql = $@"SELECT * FROM [{Options.Schema}].[Published] WITH (readpast) WHERE Id={id}";
using (var connection = new SqlConnection(_options.ConnectionString)) using (var connection = new SqlConnection(Options.ConnectionString))
{ {
return await connection.QueryFirstOrDefaultAsync<CapPublishedMessage>(sql); return await connection.QueryFirstOrDefaultAsync<CapPublishedMessage>(sql);
} }
...@@ -39,7 +40,7 @@ namespace DotNetCore.CAP.SqlServer ...@@ -39,7 +40,7 @@ namespace DotNetCore.CAP.SqlServer
{ {
var sql = $@" var sql = $@"
DELETE TOP (1) DELETE TOP (1)
FROM [{_options.Schema}].[Queue] WITH (readpast, updlock, rowlock) FROM [{Options.Schema}].[Queue] WITH (readpast, updlock, rowlock)
OUTPUT DELETED.MessageId,DELETED.[MessageType];"; OUTPUT DELETED.MessageId,DELETED.[MessageType];";
return FetchNextMessageCoreAsync(sql); return FetchNextMessageCoreAsync(sql);
...@@ -47,9 +48,10 @@ OUTPUT DELETED.MessageId,DELETED.[MessageType];"; ...@@ -47,9 +48,10 @@ OUTPUT DELETED.MessageId,DELETED.[MessageType];";
public async Task<CapPublishedMessage> GetNextPublishedMessageToBeEnqueuedAsync() public async Task<CapPublishedMessage> GetNextPublishedMessageToBeEnqueuedAsync()
{ {
var sql = $"SELECT TOP (1) * FROM [{_options.Schema}].[Published] WITH (readpast) WHERE StatusName = '{StatusName.Scheduled}'"; var sql =
$"SELECT TOP (1) * FROM [{Options.Schema}].[Published] WITH (readpast) WHERE StatusName = '{StatusName.Scheduled}'";
using (var connection = new SqlConnection(_options.ConnectionString)) using (var connection = new SqlConnection(Options.ConnectionString))
{ {
return await connection.QueryFirstOrDefaultAsync<CapPublishedMessage>(sql); return await connection.QueryFirstOrDefaultAsync<CapPublishedMessage>(sql);
} }
...@@ -57,25 +59,37 @@ OUTPUT DELETED.MessageId,DELETED.[MessageType];"; ...@@ -57,25 +59,37 @@ OUTPUT DELETED.MessageId,DELETED.[MessageType];";
public async Task<IEnumerable<CapPublishedMessage>> GetFailedPublishedMessages() public async Task<IEnumerable<CapPublishedMessage>> GetFailedPublishedMessages()
{ {
var sql = $"SELECT * FROM [{_options.Schema}].[Published] WITH (readpast) WHERE StatusName = '{StatusName.Failed}'"; var sql =
$"SELECT TOP (200) * FROM [{Options.Schema}].[Published] WITH (readpast) WHERE Retries<{_capOptions.FailedRetryCount} AND StatusName = '{StatusName.Failed}'";
using (var connection = new SqlConnection(_options.ConnectionString)) using (var connection = new SqlConnection(Options.ConnectionString))
{ {
return await connection.QueryAsync<CapPublishedMessage>(sql); return await connection.QueryAsync<CapPublishedMessage>(sql);
} }
} }
// CapReceviedMessage public bool ChangePublishedState(int messageId, string state)
{
var sql =
$"UPDATE [{Options.Schema}].[Published] SET Retries=Retries+1,StatusName = '{state}' WHERE Id={messageId}";
using (var connection = new SqlConnection(Options.ConnectionString))
{
return connection.Execute(sql) > 0;
}
}
// CapReceivedMessage
public async Task StoreReceivedMessageAsync(CapReceivedMessage message) public async Task StoreReceivedMessageAsync(CapReceivedMessage message)
{ {
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
var sql = $@" var sql = $@"
INSERT INTO [{_options.Schema}].[Received]([Name],[Group],[Content],[Retries],[Added],[ExpiresAt],[StatusName]) INSERT INTO [{Options.Schema}].[Received]([Name],[Group],[Content],[Retries],[Added],[ExpiresAt],[StatusName])
VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);"; VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);";
using (var connection = new SqlConnection(_options.ConnectionString)) using (var connection = new SqlConnection(Options.ConnectionString))
{ {
await connection.ExecuteAsync(sql, message); await connection.ExecuteAsync(sql, message);
} }
...@@ -83,31 +97,44 @@ VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);"; ...@@ -83,31 +97,44 @@ VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);";
public async Task<CapReceivedMessage> GetReceivedMessageAsync(int id) public async Task<CapReceivedMessage> GetReceivedMessageAsync(int id)
{ {
var sql = $@"SELECT * FROM [{_options.Schema}].[Received] WITH (readpast) WHERE Id={id}"; var sql = $@"SELECT * FROM [{Options.Schema}].[Received] WITH (readpast) WHERE Id={id}";
using (var connection = new SqlConnection(_options.ConnectionString)) using (var connection = new SqlConnection(Options.ConnectionString))
{ {
return await connection.QueryFirstOrDefaultAsync<CapReceivedMessage>(sql); return await connection.QueryFirstOrDefaultAsync<CapReceivedMessage>(sql);
} }
} }
public async Task<CapReceivedMessage> GetNextReceviedMessageToBeEnqueuedAsync() public async Task<CapReceivedMessage> GetNextReceivedMessageToBeEnqueuedAsync()
{ {
var sql = $"SELECT TOP (1) * FROM [{_options.Schema}].[Received] WITH (readpast) WHERE StatusName = '{StatusName.Scheduled}'"; var sql =
using (var connection = new SqlConnection(_options.ConnectionString)) $"SELECT TOP (1) * FROM [{Options.Schema}].[Received] WITH (readpast) WHERE StatusName = '{StatusName.Scheduled}'";
using (var connection = new SqlConnection(Options.ConnectionString))
{ {
return await connection.QueryFirstOrDefaultAsync<CapReceivedMessage>(sql); return await connection.QueryFirstOrDefaultAsync<CapReceivedMessage>(sql);
} }
} }
public async Task<IEnumerable<CapReceivedMessage>> GetFailedReceviedMessages() public async Task<IEnumerable<CapReceivedMessage>> GetFailedReceivedMessages()
{ {
var sql = $"SELECT * FROM [{_options.Schema}].[Received] WITH (readpast) WHERE StatusName = '{StatusName.Failed}'"; var sql =
using (var connection = new SqlConnection(_options.ConnectionString)) $"SELECT TOP (200) * FROM [{Options.Schema}].[Received] WITH (readpast) WHERE Retries<{_capOptions.FailedRetryCount} AND StatusName = '{StatusName.Failed}'";
using (var connection = new SqlConnection(Options.ConnectionString))
{ {
return await connection.QueryAsync<CapReceivedMessage>(sql); return await connection.QueryAsync<CapReceivedMessage>(sql);
} }
} }
public bool ChangeReceivedState(int messageId, string state)
{
var sql =
$"UPDATE [{Options.Schema}].[Received] SET Retries=Retries+1,StatusName = '{state}' WHERE Id={messageId}";
using (var connection = new SqlConnection(Options.ConnectionString))
{
return connection.Execute(sql) > 0;
}
}
public void Dispose() public void Dispose()
{ {
} }
...@@ -115,10 +142,10 @@ VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);"; ...@@ -115,10 +142,10 @@ VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);";
private async Task<IFetchedMessage> FetchNextMessageCoreAsync(string sql, object args = null) private async Task<IFetchedMessage> FetchNextMessageCoreAsync(string sql, object args = null)
{ {
//here don't use `using` to dispose //here don't use `using` to dispose
var connection = new SqlConnection(_options.ConnectionString); var connection = new SqlConnection(Options.ConnectionString);
await connection.OpenAsync(); await connection.OpenAsync();
var transaction = connection.BeginTransaction(IsolationLevel.ReadCommitted); var transaction = connection.BeginTransaction(IsolationLevel.ReadCommitted);
FetchedMessage fetchedMessage = null; FetchedMessage fetchedMessage;
try try
{ {
fetchedMessage = await connection.QueryFirstOrDefaultAsync<FetchedMessage>(sql, args, transaction); fetchedMessage = await connection.QueryFirstOrDefaultAsync<FetchedMessage>(sql, args, transaction);
...@@ -137,7 +164,8 @@ VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);"; ...@@ -137,7 +164,8 @@ VALUES(@Name,@Group,@Content,@Retries,@Added,@ExpiresAt,@StatusName);";
return null; return null;
} }
return new SqlServerFetchedMessage(fetchedMessage.MessageId, fetchedMessage.MessageType, connection, transaction); return new SqlServerFetchedMessage(fetchedMessage.MessageId, fetchedMessage.MessageType, connection,
transaction);
} }
} }
} }
\ No newline at end of file
...@@ -7,12 +7,12 @@ using DotNetCore.CAP.Models; ...@@ -7,12 +7,12 @@ using DotNetCore.CAP.Models;
namespace DotNetCore.CAP.SqlServer namespace DotNetCore.CAP.SqlServer
{ {
public class SqlServerStorageTransaction : IStorageTransaction, IDisposable public class SqlServerStorageTransaction : IStorageTransaction
{ {
private readonly string _schema; private readonly IDbConnection _dbConnection;
private readonly IDbTransaction _dbTransaction; private readonly IDbTransaction _dbTransaction;
private readonly IDbConnection _dbConnection; private readonly string _schema;
public SqlServerStorageTransaction(SqlServerStorageConnection connection) public SqlServerStorageTransaction(SqlServerStorageConnection connection)
{ {
...@@ -28,7 +28,8 @@ namespace DotNetCore.CAP.SqlServer ...@@ -28,7 +28,8 @@ namespace DotNetCore.CAP.SqlServer
{ {
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
var sql = $"UPDATE [{_schema}].[Published] SET [Retries] = @Retries,[ExpiresAt] = @ExpiresAt,[StatusName]=@StatusName WHERE Id=@Id;"; var sql =
$"UPDATE [{_schema}].[Published] SET [Retries] = @Retries,[Content] = @Content,[ExpiresAt] = @ExpiresAt,[StatusName]=@StatusName WHERE Id=@Id;";
_dbConnection.Execute(sql, message, _dbTransaction); _dbConnection.Execute(sql, message, _dbTransaction);
} }
...@@ -36,7 +37,8 @@ namespace DotNetCore.CAP.SqlServer ...@@ -36,7 +37,8 @@ namespace DotNetCore.CAP.SqlServer
{ {
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
var sql = $"UPDATE [{_schema}].[Received] SET [Retries] = @Retries,[ExpiresAt] = @ExpiresAt,[StatusName]=@StatusName WHERE Id=@Id;"; var sql =
$"UPDATE [{_schema}].[Received] SET [Retries] = @Retries,[Content] = @Content,[ExpiresAt] = @ExpiresAt,[StatusName]=@StatusName WHERE Id=@Id;";
_dbConnection.Execute(sql, message, _dbTransaction); _dbConnection.Execute(sql, message, _dbTransaction);
} }
...@@ -45,7 +47,8 @@ namespace DotNetCore.CAP.SqlServer ...@@ -45,7 +47,8 @@ namespace DotNetCore.CAP.SqlServer
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
var sql = $"INSERT INTO [{_schema}].[Queue] values(@MessageId,@MessageType);"; var sql = $"INSERT INTO [{_schema}].[Queue] values(@MessageId,@MessageType);";
_dbConnection.Execute(sql, new CapQueue { MessageId = message.Id, MessageType = MessageType.Publish }, _dbTransaction); _dbConnection.Execute(sql, new CapQueue {MessageId = message.Id, MessageType = MessageType.Publish},
_dbTransaction);
} }
public void EnqueueMessage(CapReceivedMessage message) public void EnqueueMessage(CapReceivedMessage message)
...@@ -53,7 +56,8 @@ namespace DotNetCore.CAP.SqlServer ...@@ -53,7 +56,8 @@ namespace DotNetCore.CAP.SqlServer
if (message == null) throw new ArgumentNullException(nameof(message)); if (message == null) throw new ArgumentNullException(nameof(message));
var sql = $"INSERT INTO [{_schema}].[Queue] values(@MessageId,@MessageType);"; var sql = $"INSERT INTO [{_schema}].[Queue] values(@MessageId,@MessageType);";
_dbConnection.Execute(sql, new CapQueue { MessageId = message.Id, MessageType = MessageType.Subscribe }, _dbTransaction); _dbConnection.Execute(sql, new CapQueue {MessageId = message.Id, MessageType = MessageType.Subscribe},
_dbTransaction);
} }
public Task CommitAsync() public Task CommitAsync()
......
...@@ -10,7 +10,7 @@ namespace DotNetCore.CAP.Abstractions ...@@ -10,7 +10,7 @@ namespace DotNetCore.CAP.Abstractions
public abstract class CapPublisherBase : ICapPublisher, IDisposable public abstract class CapPublisherBase : ICapPublisher, IDisposable
{ {
protected IDbConnection DbConnection { get; set; } protected IDbConnection DbConnection { get; set; }
protected IDbTransaction DbTranasaction { get; set; } protected IDbTransaction DbTransaction { get; set; }
protected bool IsCapOpenedTrans { get; set; } protected bool IsCapOpenedTrans { get; set; }
protected bool IsCapOpenedConn { get; set; } protected bool IsCapOpenedConn { get; set; }
protected bool IsUsingEF { get; set; } protected bool IsUsingEF { get; set; }
...@@ -36,22 +36,20 @@ namespace DotNetCore.CAP.Abstractions ...@@ -36,22 +36,20 @@ namespace DotNetCore.CAP.Abstractions
return PublishWithTransAsync(name, content); return PublishWithTransAsync(name, content);
} }
public void Publish<T>(string name, T contentObj, IDbConnection dbConnection, public void Publish<T>(string name, T contentObj, IDbTransaction dbTransaction, string callbackName = null)
string callbackName = null, IDbTransaction dbTransaction = null)
{ {
CheckIsAdoNet(name); CheckIsAdoNet(name);
PrepareConnectionForAdo(dbConnection, dbTransaction); PrepareConnectionForAdo(dbTransaction);
var content = Serialize(contentObj, callbackName); var content = Serialize(contentObj, callbackName);
PublishWithTrans(name, content); PublishWithTrans(name, content);
} }
public Task PublishAsync<T>(string name, T contentObj, IDbConnection dbConnection, public Task PublishAsync<T>(string name, T contentObj, IDbTransaction dbTransaction, string callbackName = null)
string callbackName = null, IDbTransaction dbTransaction = null)
{ {
CheckIsAdoNet(name); CheckIsAdoNet(name);
PrepareConnectionForAdo(dbConnection, dbTransaction); PrepareConnectionForAdo(dbTransaction);
var content = Serialize(contentObj, callbackName); var content = Serialize(contentObj, callbackName);
...@@ -60,54 +58,72 @@ namespace DotNetCore.CAP.Abstractions ...@@ -60,54 +58,72 @@ namespace DotNetCore.CAP.Abstractions
protected abstract void PrepareConnectionForEF(); protected abstract void PrepareConnectionForEF();
protected abstract void Execute(IDbConnection dbConnection, IDbTransaction dbTransaction, CapPublishedMessage message); protected abstract void Execute(IDbConnection dbConnection, IDbTransaction dbTransaction,
CapPublishedMessage message);
protected abstract Task ExecuteAsync(IDbConnection dbConnection, IDbTransaction dbTransaction, CapPublishedMessage message); protected abstract Task ExecuteAsync(IDbConnection dbConnection, IDbTransaction dbTransaction,
CapPublishedMessage message);
#region private methods protected virtual string Serialize<T>(T obj, string callbackName = null)
private string Serialize<T>(T obj, string callbackName = null)
{ {
var message = new Message(obj) var packer = (IMessagePacker)ServiceProvider.GetService(typeof(IMessagePacker));
string content;
if (obj != null)
{
if (Helper.IsComplexType(obj.GetType()))
{
var serializer = (IContentSerializer)ServiceProvider.GetService(typeof(IContentSerializer));
content = serializer.Serialize(obj);
}
else
{
content = obj.ToString();
}
}
else
{
content = string.Empty;
}
var message = new CapMessageDto(content)
{ {
CallbackName = callbackName CallbackName = callbackName
}; };
return Helper.ToJson(message); return packer.Pack(message);
} }
private void PrepareConnectionForAdo(IDbConnection dbConnection, IDbTransaction dbTransaction) #region private methods
private void PrepareConnectionForAdo(IDbTransaction dbTransaction)
{ {
DbConnection = dbConnection ?? throw new ArgumentNullException(nameof(dbConnection)); DbTransaction = dbTransaction ?? throw new ArgumentNullException(nameof(dbTransaction));
DbConnection = DbTransaction.Connection;
if (DbConnection.State != ConnectionState.Open) if (DbConnection.State != ConnectionState.Open)
{ {
IsCapOpenedConn = true; IsCapOpenedConn = true;
DbConnection.Open(); DbConnection.Open();
} }
DbTranasaction = dbTransaction;
if (DbTranasaction == null)
{
IsCapOpenedTrans = true;
DbTranasaction = dbConnection.BeginTransaction(IsolationLevel.ReadCommitted);
}
} }
private void CheckIsUsingEF(string name) private void CheckIsUsingEF(string name)
{ {
if (name == null) throw new ArgumentNullException(nameof(name)); if (name == null) throw new ArgumentNullException(nameof(name));
if (!IsUsingEF) if (!IsUsingEF)
throw new InvalidOperationException("If you are using the EntityFramework, you need to configure the DbContextType first." + throw new InvalidOperationException(
" otherwise you need to use overloaded method with IDbConnection and IDbTransaction."); "If you are using the EntityFramework, you need to configure the DbContextType first." +
" otherwise you need to use overloaded method with IDbConnection and IDbTransaction.");
} }
private void CheckIsAdoNet(string name) private void CheckIsAdoNet(string name)
{ {
if (name == null) throw new ArgumentNullException(nameof(name)); if (name == null) throw new ArgumentNullException(nameof(name));
if (IsUsingEF) if (IsUsingEF)
throw new InvalidOperationException("If you are using the EntityFramework, you do not need to use this overloaded."); throw new InvalidOperationException(
"If you are using the EntityFramework, you do not need to use this overloaded.");
} }
private async Task PublishWithTransAsync(string name, string content) private Task PublishWithTransAsync(string name, string content)
{ {
var message = new CapPublishedMessage var message = new CapPublishedMessage
{ {
...@@ -116,11 +132,13 @@ namespace DotNetCore.CAP.Abstractions ...@@ -116,11 +132,13 @@ namespace DotNetCore.CAP.Abstractions
StatusName = StatusName.Scheduled StatusName = StatusName.Scheduled
}; };
await ExecuteAsync(DbConnection, DbTranasaction, message); ExecuteAsync(DbConnection, DbTransaction, message);
ClosedCap(); ClosedCap();
PublishQueuer.PulseEvent.Set(); PublishQueuer.PulseEvent.Set();
return Task.CompletedTask;
} }
private void PublishWithTrans(string name, string content) private void PublishWithTrans(string name, string content)
...@@ -132,7 +150,7 @@ namespace DotNetCore.CAP.Abstractions ...@@ -132,7 +150,7 @@ namespace DotNetCore.CAP.Abstractions
StatusName = StatusName.Scheduled StatusName = StatusName.Scheduled
}; };
Execute(DbConnection, DbTranasaction, message); Execute(DbConnection, DbTransaction, message);
ClosedCap(); ClosedCap();
...@@ -143,18 +161,16 @@ namespace DotNetCore.CAP.Abstractions ...@@ -143,18 +161,16 @@ namespace DotNetCore.CAP.Abstractions
{ {
if (IsCapOpenedTrans) if (IsCapOpenedTrans)
{ {
DbTranasaction.Commit(); DbTransaction.Commit();
DbTranasaction.Dispose(); DbTransaction.Dispose();
} }
if (IsCapOpenedConn) if (IsCapOpenedConn)
{
DbConnection.Dispose(); DbConnection.Dispose();
}
} }
public void Dispose() public void Dispose()
{ {
DbTranasaction?.Dispose(); DbTransaction?.Dispose();
DbConnection?.Dispose(); DbConnection?.Dispose();
} }
......
using System;
namespace DotNetCore.CAP.Abstractions
{
/// <summary>
/// a context of consumer invoker.
/// </summary>
public class ConsumerInvokerContext
{
public ConsumerInvokerContext(ConsumerContext consumerContext)
{
ConsumerContext = consumerContext ??
throw new ArgumentNullException(nameof(consumerContext));
}
public ConsumerContext ConsumerContext { get; set; }
public IConsumerInvoker Result { get; set; }
}
}
\ No newline at end of file
using System;
using DotNetCore.CAP.Models;
namespace DotNetCore.CAP.Abstractions
{
/// <summary>
/// Message content serializer.
/// <para>By default, CAP will use Json as a serializer, and you can customize this interface to achieve serialization of other methods.</para>
/// </summary>
public interface IContentSerializer
{
/// <summary>
/// Serializes the specified object to a string.
/// </summary>
/// <typeparam name="T"> The type of the value being serialized.</typeparam>
/// <param name="value">The object to serialize.</param>
/// <returns>A string representation of the object.</returns>
string Serialize<T>(T value);
/// <summary>
/// Deserializes the string to the specified .NET type.
/// </summary>
/// <typeparam name="T">The type of the object to deserialize to.</typeparam>
/// <param name="value">The content string to deserialize.</param>
/// <returns>The deserialized object from the string.</returns>
T DeSerialize<T>(string value);
/// <summary>
/// Deserializes the string to the specified .NET type.
/// </summary>
/// <param name="value">The string to deserialize.</param>
/// <param name="type">The type of the object to deserialize to.</param>
/// <returns>The deserialized object from the string.</returns>
object DeSerialize(string value, Type type);
}
/// <summary>
/// CAP message content wapper.
/// <para>You can customize the message body filed name of the wrapper or add fields that you interested.</para>
/// </summary>
/// <remarks>
/// We use the wrapper to provide some additional information for the message content,which is important for CAP。
/// Typically, we may need to customize the field display name of the message,
/// which includes interacting with other message components, which can be adapted in this manner
/// </remarks>
public interface IMessagePacker
{
/// <summary>
/// Package a message object
/// </summary>
/// <param name="obj">The obj message to be packed.</param>
string Pack(CapMessage obj);
/// <summary>
/// Unpack a message strings to <see cref="CapMessage"/> object.
/// </summary>
/// <param name="packingMessage">The string of packed message.</param>
CapMessage UnPack(string packingMessage);
}
}
\ No newline at end of file
using System.Reflection; using System.Reflection;
using DotNetCore.CAP.Abstractions.ModelBinding; using DotNetCore.CAP.Abstractions.ModelBinding;
namespace DotNetCore.CAP.Internal namespace DotNetCore.CAP.Abstractions
{ {
/// <summary>
/// Model binder factory.
/// </summary>
public interface IModelBinderFactory public interface IModelBinderFactory
{ {
/// <summary>
/// Create a model binder by parameter.
/// </summary>
/// <param name="parameter">The method parameter info</param>
/// <returns>A model binder instance.</returns>
IModelBinder CreateBinder(ParameterInfo parameter); IModelBinder CreateBinder(ParameterInfo parameter);
} }
} }
\ No newline at end of file
using System.Threading.Tasks;
using DotNetCore.CAP.Models;
namespace DotNetCore.CAP.Abstractions
{
/// <summary>
/// Consumer method executor.
/// </summary>
public interface ISubscriberExecutor
{
/// <summary>
/// Execute the consumer method.
/// </summary>
/// <param name="receivedMessage">The received message.</param>
Task<OperateResult> ExecuteAsync(CapReceivedMessage receivedMessage);
}
}
...@@ -8,22 +8,22 @@ namespace DotNetCore.CAP.Abstractions.ModelBinding ...@@ -8,22 +8,22 @@ namespace DotNetCore.CAP.Abstractions.ModelBinding
public struct ModelBindingResult public struct ModelBindingResult
{ {
/// <summary> /// <summary>
/// Creates a <see cref="ModelBindingResult"/> representing a failed model binding operation. /// Creates a <see cref="ModelBindingResult" /> representing a failed model binding operation.
/// </summary> /// </summary>
/// <returns>A <see cref="ModelBindingResult"/> representing a failed model binding operation.</returns> /// <returns>A <see cref="ModelBindingResult" /> representing a failed model binding operation.</returns>
public static ModelBindingResult Failed() public static ModelBindingResult Failed()
{ {
return new ModelBindingResult(model: null, isSuccess: false); return new ModelBindingResult(null, false);
} }
/// <summary> /// <summary>
/// Creates a <see cref="ModelBindingResult"/> representing a successful model binding operation. /// Creates a <see cref="ModelBindingResult" /> representing a successful model binding operation.
/// </summary> /// </summary>
/// <param name="model">The model value. May be <c>null.</c></param> /// <param name="model">The model value. May be <c>null.</c></param>
/// <returns>A <see cref="ModelBindingResult"/> representing a successful model bind.</returns> /// <returns>A <see cref="ModelBindingResult" /> representing a successful model bind.</returns>
public static ModelBindingResult Success(object model) public static ModelBindingResult Success(object model)
{ {
return new ModelBindingResult(model, isSuccess: true); return new ModelBindingResult(model, true);
} }
private ModelBindingResult(object model, bool isSuccess) private ModelBindingResult(object model, bool isSuccess)
...@@ -42,26 +42,16 @@ namespace DotNetCore.CAP.Abstractions.ModelBinding ...@@ -42,26 +42,16 @@ namespace DotNetCore.CAP.Abstractions.ModelBinding
public override string ToString() public override string ToString()
{ {
if (IsSuccess) if (IsSuccess)
{
return $"Success '{Model}'"; return $"Success '{Model}'";
} return "Failed";
else
{
return $"Failed";
}
} }
public override bool Equals(object obj) public override bool Equals(object obj)
{ {
var other = obj as ModelBindingResult?; var other = obj as ModelBindingResult?;
if (other == null) if (other == null)
{
return false; return false;
} return Equals(other.Value);
else
{
return Equals(other.Value);
}
} }
public override int GetHashCode() public override int GetHashCode()
...@@ -77,14 +67,14 @@ namespace DotNetCore.CAP.Abstractions.ModelBinding ...@@ -77,14 +67,14 @@ namespace DotNetCore.CAP.Abstractions.ModelBinding
{ {
return return
IsSuccess == other.IsSuccess && IsSuccess == other.IsSuccess &&
object.Equals(Model, other.Model); Equals(Model, other.Model);
} }
/// <summary> /// <summary>
/// Compares <see cref="ModelBindingResult"/> objects for equality. /// Compares <see cref="ModelBindingResult" /> objects for equality.
/// </summary> /// </summary>
/// <param name="x">A <see cref="ModelBindingResult"/>.</param> /// <param name="x">A <see cref="ModelBindingResult" />.</param>
/// <param name="y">A <see cref="ModelBindingResult"/>.</param> /// <param name="y">A <see cref="ModelBindingResult" />.</param>
/// <returns><c>true</c> if the objects are equal, otherwise <c>false</c>.</returns> /// <returns><c>true</c> if the objects are equal, otherwise <c>false</c>.</returns>
public static bool operator ==(ModelBindingResult x, ModelBindingResult y) public static bool operator ==(ModelBindingResult x, ModelBindingResult y)
{ {
...@@ -92,10 +82,10 @@ namespace DotNetCore.CAP.Abstractions.ModelBinding ...@@ -92,10 +82,10 @@ namespace DotNetCore.CAP.Abstractions.ModelBinding
} }
/// <summary> /// <summary>
/// Compares <see cref="ModelBindingResult"/> objects for inequality. /// Compares <see cref="ModelBindingResult" /> objects for inequality.
/// </summary> /// </summary>
/// <param name="x">A <see cref="ModelBindingResult"/>.</param> /// <param name="x">A <see cref="ModelBindingResult" />.</param>
/// <param name="y">A <see cref="ModelBindingResult"/>.</param> /// <param name="y">A <see cref="ModelBindingResult" />.</param>
/// <returns><c>true</c> if the objects are not equal, otherwise <c>false</c>.</returns> /// <returns><c>true</c> if the objects are not equal, otherwise <c>false</c>.</returns>
public static bool operator !=(ModelBindingResult x, ModelBindingResult y) public static bool operator !=(ModelBindingResult x, ModelBindingResult y)
{ {
......
...@@ -2,8 +2,9 @@ ...@@ -2,8 +2,9 @@
namespace DotNetCore.CAP.Abstractions namespace DotNetCore.CAP.Abstractions
{ {
/// <inheritdoc />
/// <summary> /// <summary>
/// An abstract attribute that for kafka attribute or rabbitmq attribute /// An abstract attribute that for kafka attribute or rabbit mq attribute
/// </summary> /// </summary>
[AttributeUsage(AttributeTargets.Method | AttributeTargets.Class, AllowMultiple = true)] [AttributeUsage(AttributeTargets.Method | AttributeTargets.Class, AllowMultiple = true)]
public abstract class TopicAttribute : Attribute public abstract class TopicAttribute : Attribute
...@@ -14,19 +15,14 @@ namespace DotNetCore.CAP.Abstractions ...@@ -14,19 +15,14 @@ namespace DotNetCore.CAP.Abstractions
} }
/// <summary> /// <summary>
/// topic or exchange route key name. /// Topic or exchange route key name.
/// </summary> /// </summary>
public string Name { get; } public string Name { get; }
/// <summary> /// <summary>
/// kafak --> groups.id /// kafka --> groups.id
/// rabbitmq --> queue.name /// rabbit MQ --> queue.name
/// </summary> /// </summary>
public string Group { get; set; } = "cap.default.group"; public string Group { get; set; } = "cap.default.group";
/// <summary>
/// unused now
/// </summary>
public bool IsOneWay { get; set; }
} }
} }
\ No newline at end of file
using System; using System;
using DotNetCore.CAP; using DotNetCore.CAP;
using DotNetCore.CAP.Dashboard.GatewayProxy;
using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.DependencyInjection;
// ReSharper disable once CheckNamespace
namespace Microsoft.AspNetCore.Builder namespace Microsoft.AspNetCore.Builder
{ {
/// <summary> /// <summary>
/// app extensions for <see cref="IApplicationBuilder"/> /// app extensions for <see cref="IApplicationBuilder" />
/// </summary> /// </summary>
public static class AppBuilderExtensions public static class AppBuilderExtensions
{ {
///<summary> /// <summary>
/// Enables cap for the current application /// Enables cap for the current application
/// </summary> /// </summary>
/// <param name="app">The <see cref="IApplicationBuilder"/> instance this method extends.</param> /// <param name="app">The <see cref="IApplicationBuilder" /> instance this method extends.</param>
/// <returns>The <see cref="IApplicationBuilder"/> instance this method extends.</returns> /// <returns>The <see cref="IApplicationBuilder" /> instance this method extends.</returns>
public static IApplicationBuilder UseCap(this IApplicationBuilder app) public static IApplicationBuilder UseCap(this IApplicationBuilder app)
{ {
if (app == null) if (app == null)
{
throw new ArgumentNullException(nameof(app)); throw new ArgumentNullException(nameof(app));
}
var marker = app.ApplicationServices.GetService<CapMarkerService>(); CheckRequirement(app);
if (marker == null)
{
throw new InvalidOperationException("Add Cap must be called on the service collection.");
}
var provider = app.ApplicationServices; var provider = app.ApplicationServices;
var bootstrapper = provider.GetRequiredService<IBootstrapper>(); var bootstrapper = provider.GetRequiredService<IBootstrapper>();
bootstrapper.BootstrapAsync(); bootstrapper.BootstrapAsync();
if (provider.GetService<DashboardOptions>() != null)
{
if (provider.GetService<DiscoveryOptions>() != null)
app.UseMiddleware<GatewayProxyMiddleware>();
app.UseMiddleware<DashboardMiddleware>();
}
return app; return app;
} }
private static void CheckRequirement(IApplicationBuilder app)
{
var marker = app.ApplicationServices.GetService<CapMarkerService>();
if (marker == null)
throw new InvalidOperationException(
"AddCap() must be called on the service collection. eg: services.AddCap(...)");
var messageQueueMarker = app.ApplicationServices.GetService<CapMessageQueueMakerService>();
if (messageQueueMarker == null)
throw new InvalidOperationException(
"You must be config used message queue provider at AddCap() options! eg: services.AddCap(options=>{ options.UseKafka(...) })");
var databaseMarker = app.ApplicationServices.GetService<CapDatabaseStorageMarkerService>();
if (databaseMarker == null)
throw new InvalidOperationException(
"You must be config used database provider at AddCap() options! eg: services.AddCap(options=>{ options.UseSqlServer(...) })");
}
} }
} }
\ No newline at end of file
using System; using System;
using DotNetCore.CAP.Abstractions;
using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.DependencyInjection;
namespace DotNetCore.CAP namespace DotNetCore.CAP
...@@ -10,10 +11,24 @@ namespace DotNetCore.CAP ...@@ -10,10 +11,24 @@ namespace DotNetCore.CAP
{ {
} }
/// <summary>
/// Used to verify cap database storage extension was added on a ServiceCollection
/// </summary>
public class CapDatabaseStorageMarkerService
{
}
/// <summary>
/// Used to verify cap message queue extension was added on a ServiceCollection
/// </summary>
public class CapMessageQueueMakerService
{
}
/// <summary> /// <summary>
/// Allows fine grained configuration of CAP services. /// Allows fine grained configuration of CAP services.
/// </summary> /// </summary>
public class CapBuilder public sealed class CapBuilder
{ {
public CapBuilder(IServiceCollection services) public CapBuilder(IServiceCollection services)
{ {
...@@ -21,9 +36,39 @@ namespace DotNetCore.CAP ...@@ -21,9 +36,39 @@ namespace DotNetCore.CAP
} }
/// <summary> /// <summary>
/// Gets the <see cref="IServiceCollection"/> where MVC services are configured. /// Gets the <see cref="IServiceCollection" /> where MVC services are configured.
/// </summary>
public IServiceCollection Services { get; }
/// <summary>
/// Add an <see cref="ICapPublisher" />.
/// </summary>
/// <typeparam name="T">The type of the service.</typeparam>
public CapBuilder AddProducerService<T>()
where T : class, ICapPublisher
{
return AddScoped(typeof(ICapPublisher), typeof(T));
}
/// <summary>
/// Add a custom content serializer
/// </summary> /// </summary>
public IServiceCollection Services { get; private set; } /// <typeparam name="T">The type of the service.</typeparam>
public CapBuilder AddContentSerializer<T>()
where T : class, IContentSerializer
{
return AddSingleton(typeof(IContentSerializer), typeof(T));
}
/// <summary>
/// Add a custom message wapper
/// </summary>
/// <typeparam name="T">The type of the service.</typeparam>
public CapBuilder AddMessagePacker<T>()
where T : class, IMessagePacker
{
return AddSingleton(typeof(IMessagePacker), typeof(T));
}
/// <summary> /// <summary>
/// Adds a scoped service of the type specified in serviceType with an implementation /// Adds a scoped service of the type specified in serviceType with an implementation
...@@ -35,13 +80,12 @@ namespace DotNetCore.CAP ...@@ -35,13 +80,12 @@ namespace DotNetCore.CAP
} }
/// <summary> /// <summary>
/// Add an <see cref="ICapPublisher"/>. /// Adds a singleton service of the type specified in serviceType with an implementation
/// </summary> /// </summary>
/// <typeparam name="T">The type of the service.</typeparam> private CapBuilder AddSingleton(Type serviceType, Type concreteType)
public virtual CapBuilder AddProducerService<T>()
where T : class, ICapPublisher
{ {
return AddScoped(typeof(ICapPublisher), typeof(T)); Services.AddSingleton(serviceType, concreteType);
return this;
} }
} }
} }
\ No newline at end of file
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This source diff could not be displayed because it is too large. You can view the blob instead.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment