rdp-proxy/docs/architecture/RDP_SERVICE_CSHARP_TARGET.md

RDP Service C# Target Architecture

Status

Superseded.

The active direction is now documented in:

• docs/architecture/RDP_SERVICE_CPP_PERFORMANCE_TARGET.md

The C++ worker remains the primary RDP runtime. This C# document is retained only as historical/research context and must not be used for implementation unless explicitly re-approved.

Problem Statement

The current RDP MVP proved the platform lifecycle, but its rendering model is not production-grade:

• FreeRDP connects to the target RDP server.
• The worker reads the GDI framebuffer.
• The worker publishes full or cropped BGRA frames through RAP direct WSS.
• The Windows client renders those frames as a custom viewer.

This is not how high-performance RDP clients work. On a fast LAN, the network is not the main bottleneck. The bottleneck is that the service is repeatedly copying and publishing screen images instead of consuming the RDP graphics protocol as a graphics protocol.

Observable symptoms:

• delayed visual feedback after input
• unreliable first-click behavior
• poor hover behavior
• high CPU/memory pressure from framebuffer copies
• unnecessary 1280x720 BGRA full-frame payloads
• fragile coupling between input, render snapshots, and UI timing

External Reference Model

Microsoft RDP performance is based on graphics protocol features rather than screen scraping:

• RDP Graphics Pipeline Extension (MS-RDPEGFX) uses a dynamic virtual channel for graphics pipeline updates.
• RDP supports adaptive graphics, delta detection, caching, mixed-mode encoding, RemoteFX Progressive, H.264/AVC, AVC444, and HEVC in modern environments.
• FreeRDP documentation describes the RDP GFX Pipeline (rdpgfx) and codecs such as RemoteFX Progressive, H.264 AVC420/AVC444, ClearCodec, and ZGFX.

References:

• https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-rdpegfx/da5c75f9-cd99-450c-98c4-014a496942b0
• https://learn.microsoft.com/en-us/azure/virtual-desktop/graphics-encoding
• https://freerdp-freerdp.mintlify.app/concepts/codecs

Target Decision

Replace the internal RDP engine with a C# implementation owned by this project.

The new service:

• is a RAP RDP service adapter, not a generic local RDP client UI
• speaks standard RDP to the target Windows RDP server
• keeps RDP protocol details inside the RDP service boundary
• preserves current backend and cluster data-plane contracts
• does not use FreeRDP as the runtime RDP engine
• does not require the local Windows desktop client to become mstsc

The local Windows client remains a RAP client. It receives RAP display/input/ clipboard/file messages over the existing direct worker WSS data-plane.

What Must Not Change

The following are outside this rewrite:

• backend organization/auth/session lifecycle
• PostgreSQL source-of-truth model
• Redis live coordination model
• worker registration and lease semantics
• data_plane_token model
• direct_worker_wss transport contract
• backend gateway fallback
• clipboard/file policy semantics
• file upload policy semantics
• session attach/detach/reattach/takeover/terminate semantics

Only the RDP service adapter internals change.

Service Boundary

flowchart LR
    Client["Windows RAP Client"]
    Backend["Backend Control Plane"]
    Worker["RDP Service Node"]
    Engine["C# RDP Protocol Engine"]
    Target["Target Windows RDP Server"]

    Client <--> |"direct_worker_wss RAP channels"| Worker
    Backend <--> |"assignments, leases, audit"| Worker
    Worker --> Engine
    Engine <--> |"standard RDP"| Target

The RDP service owns:

• RDP negotiation and transport
• NLA/CredSSP/TLS integration
• input translation to RDP fast-path input
• graphics channel parsing
• virtual channel handling for clipboard and future file features
• conversion from RDP graphics units to RAP render messages
• session runtime ownership and reconnect/takeover binding

The data-plane layer owns:

• data_plane_token validation
• direct WSS connection binding
• logical channel priority
• reliable/droppable semantics
• fallback compatibility

New RDP Service Components

Rap.Rdp.Service

Host process.

Responsibilities:

• load worker/RDP service configuration
• register worker capabilities with existing coordination layer later
• expose the existing direct WSS endpoint later
• create and supervise RDP sessions
• keep the current C++ worker active until cutover

Rap.Rdp.Core

Pure C# protocol and runtime boundaries.

Responsibilities:

• define RDP session lifecycle interfaces
• define protocol engine interfaces
• define graphics/input/clipboard/file abstractions
• avoid any dependency on WPF or backend repositories

Rap.Rdp.Protocol

Future implementation module.

Responsibilities:

• implement RDP connection sequence from Microsoft Open Specifications
• implement security/NLA/CredSSP/TLS
• implement core channels and fast-path input
• implement graphics pipeline negotiation
• implement virtual channel framing

This module must not depend on the Windows desktop UI.

Rap.Rdp.DataPlane

Future adapter module.

Responsibilities:

• map RAP direct WSS JSON/binary envelopes to the protocol engine
• keep input highest priority
• keep render latest-frame or latest-update droppable
• keep clipboard/file reliable and policy-gated

Graphics Strategy

The new render path must not use framebuffer screen scraping as the primary production path.

Priority order:

1. RDPGFX graphics pipeline channel.
2. Surface/dirty-region updates.
3. Encoded graphics payloads where available.
4. Raw bitmap fallback only for compatibility/debug.

Target RAP render message classes:

• surface.create
• surface.delete
• surface.map
• surface.region
• surface.codec_frame
• cursor.update
• frame.ack

The first usable implementation may still decode some graphics to BGRA, but only as a controlled fallback. It must not become the permanent production model.

Input Strategy

Input must be independent from render.

Rules:

• mouse down/up, wheel, and keyboard down/up are reliable and ordered
• pointer move is coalesced latest-only
• pointer position is explicitly sent before button-down when needed
• input never waits behind render
• no UI focus event may be inserted into the same ordered sequence in a way that consumes the first remote click

The current double-click regression is treated as a bug caused by the RAP-side focus/input sequencing, not as a normal RDP behavior.

Clipboard And File Strategy

Existing policy semantics remain:

• clipboard modes stay enforced in backend, gateway/data-plane, and RDP service
• file transfer modes stay enforced in backend, gateway/data-plane, and RDP service
• text clipboard maps to RDP clipboard virtual channel
• restricted drive visibility remains a separate policy-controlled feature

The C# rewrite must not expand clipboard/file scope while replacing render/input.

Staged Migration Plan

RDP-C#-0: Documentation And Skeleton

Create a buildable C# RDP service skeleton with interfaces only.

No runtime cutover.

RDP-C#-1: Control-Plane Compatible Worker Shell

Implement worker registration, heartbeats, lease renewal, assignment consumption, and direct WSS token validation in C# using existing contracts.

The C++ worker remains default.

RDP-C#-2: RDP Handshake Probe

Implement a non-viewing RDP connection probe:

• TCP/TLS
• basic RDP negotiation
• NLA/CredSSP if required
• connect/disconnect lifecycle
• failure reporting

No rendering yet.

RDP-C#-3: Input-Only Protocol Path

After a connected session, send fast-path keyboard/mouse input to the RDP server.

Use diagnostic-only graphics or no graphics.

RDP-C#-4: Basic Graphics Protocol Path

Implement the simplest RDP graphics path needed to display a desktop without FreeRDP.

Allowed as temporary fallback:

• raw bitmap updates
• dirty-region bitmap updates

Not acceptable as final production:

• repeated full-frame screenshot capture

RDP-C#-5: RDPGFX Foundation

Implement RDPGFX channel negotiation and surface update handling.

RDP-C#-6: Codec Path

Implement or relay supported encoded graphics modes:

• RemoteFX Progressive where practical
• H.264/AVC420/AVC444 where negotiated
• client-side decode through platform APIs where possible

RDP-C#-7: Runtime Cutover

Enable the C# RDP service per worker/resource via feature flag.

Rollback must switch back to the current C++ worker without changing backend contracts.

Performance Requirements

Target for LAN:

• first frame under 2 seconds after successful RDP login
• click to visible response under 150 ms for normal UI
• keypress to visible response under 150 ms for text input
• pointer hover response under 100 ms where the target OS emits hover changes
• no unbounded frame queue
• no render work on UI thread except final apply
• no full-frame publish loop for static desktops

Risks

• Implementing RDP from specs is substantial.
• NLA/CredSSP correctness is security-sensitive.
• Graphics codecs are complex.
• Some target servers may negotiate older bitmap paths.
• AVC/AVC444 decode support differs by client platform.
• A partial RDP engine must not be switched into production before smoke proof.

Recommended Immediate Next Step

Proceed with RDP-C#-0 only.

Goal: Create a buildable C# RDP service skeleton and protocol boundaries, without switching runtime traffic away from the current worker.

Strict rules:

• do not change backend contracts
• do not change cluster transport
• do not remove C++ worker
• do not use FreeRDP in the new C# service
• do not use third-party RDP libraries
• do not claim the C# engine is runtime-ready

Deliver:

• buildable workers/rdp-service-csharp
• interfaces for protocol engine, data-plane bridge, graphics sink, input source
• README with migration stages
• docs update marking current C++/FreeRDP path as legacy MVP runtime