v3 preview available!

[mgravell]

The first v3.0 preview is now up on NuGet

What is v3.0?

v3.0 is an overhaul of the "read" half of the IO mechanism.

Goals:

• improve stability (there is a strongly suspected memory issue in OOM scenarios, caused by some internals)
  • THIS IS THE PRIMARY DELIVERABLE of v3.0
• improve performance, in particular for large/deep replies (hgetall for large hashes, for example)
• provide a basis for the future 3.1 and 3.2 plans

Non-goals:

• there are no relevant public API changes in this release; you don't need to do anything other than update the package

Details:

• remove the internal DOM that was used for parsing inbound RESP, switching to a new reader API (think XmlDocument to XmlReader, but: not xml)
• remove "arenas", the API behind our DOM implementation
• remove "Pipelines.Sockets.Unofficial" (which provided "arenas" and some other bits)
• add RespReader from RESPite

That's actually a lot of work, but large parts (such as the new codegen tooling) have already been pre-merged into v2 to support improved testing via the "toy server", which is fully using the new RESP parser.

Status can be seen here; there's a few more things to close down before we get a full 3.0 preview. but: it should be stable and usable as-is.

Known gaps:

• if using .NET Framework and a mixture of "sync" and "async", the spiral-of-death may recur under high load, since we're currently using the inbuilt SemaphoreSlim; the next preview release will mitigate this
• some planned work on the write path is incomplete, so has been left using a slightly sub-optimal (but reliable) Pipe usage

What is RESPite?

RESPite is a new package that contains all of the RESP-specific parsing (3.0) and formatting (3.1) implementation details, completely detached from the SE.Redis core. This makes it usable for any low-level RESP tools, such as resp-cli and resp-benchmark (which are currently built from an old fork of RESPite, and will need updating), but also from other RESP-related tools. This code shares a license with SE.Redis.

Currently RESPite exists in both the v2 and v3 branches; however, this is not desirable long-term; when 3.0 is GA, it will be removed from the v2 branch, and v2 will use a <PackageReference> instead of a <ProjectReference>.

How will v2 and v3 co-exist?

Initially, v3 will be available but not mandated; we will aim for the planned major pieces ("active:active", "smart-client-handoffs", etc) to function identically in both. At some point v2 will essentially become a maintenance-only branch (i.e. critical fixes only). That point is not "today".

---

Upcoming:

Planned for 3.1

• Overhaul the "write" half of the IO mechanism to use RESPite
• Switch the internal execution pipeline to focus on front-loading serialization
• Switch most of the internal execution pipeline to use the new codegen (in place of Message)
• Improve the handoff of the "read" mechanism

Planned for 3.2

• Expose a new public API for the new read/write pipelines for external module publishers (or ad-hoc command consumers) - think "NRedisStack"