rdp-proxy/docs/architecture/RDP_ADAPTER_RUNTIME.md

RDP Adapter Runtime

Paused/archival note: this document remains useful for RDP adapter internals, but it is not the current source of truth for transport/runtime architecture. Fabric transport is now QUIC-only between nodes. For active transport, recovery, and routing behavior, see docs/architecture/DISTRIBUTED_FABRIC_NODE_PROTOCOL_PLAN.md, docs/architecture/FABRIC_FIRST_TRANSPORT_AND_STRESS_PLAN.md, and docs/architecture/SECURE_ACCESS_FABRIC_TARGET.md.

Status: active implementation plan for the new C++ RDP Adapter internals.

Current implementation status:

• RDP-A1 is build-proven: the common Service Adapter channel model and probe exist.
• RDP-A2 is live-smoke-proven on the test Docker environment as of 2026-04-26: SessionRuntime depends on RdpAdapterRuntime, not directly on FreeRDP runtime types, and a real RDP session still connects through the existing direct data plane.
• RDP-Perf-2 is live-smoke-proven on the test Docker environment as of 2026-04-26: the current FreeRDP substrate now logs callback source/timing, capture source, and input-to-first-graphics-callback timing.
• RDP-Perf-3 / RDP-A3 region-first BGRA fallback is live-smoke-proven on the test Docker environment as of 2026-04-26: direct binary region frames render in the Windows client and backend gateway fallback remains compatible.
• RDP-Perf-4 / RDP-A6 gated RDPGFX foundation is build-proven and default-path smoke-proven on the test Docker environment as of 2026-04-26. RDPGFX stays disabled by default because the current live RDP target resets the connection when graphics pipeline support is advertised.
• RDP-A4 CursorAdapter is live-smoke-proven on the test Docker environment as of 2026-04-26: FreeRDP pointer callbacks are normalized into latest-only cursor.update events, direct worker WSS sends them separately from display frames, and backend gateway fallback remains compatible.
• RDP-Perf-5A is build-proven and smoke-proven on the test Docker environment as of 2026-04-26: classic GDI region/interactive frames use a 33 ms publish cadence, hot-loop lease renewal is removed, and direct/fallback paths remain compatible.
• RDP-Perf-6 direct dirty-region binary contract is build/probe/live-smoke-proven on the test Docker environment as of 2026-04-26: direct RAP2 frames now distinguish render.frame.full from render.frame.region, include region payload diagnostics, and the Windows presenter keeps a session framebuffer for region patching. Runtime proof used P3.3 Secret RDP Resource; observed dirty-region savings ranged from 82.22% to 99.56% versus the 3,686,400 byte full frame.
• Current accepted baseline is rap-rdp-worker:rdp-p1-region-order2: ordered dirty-region delivery is preserved through SessionRuntime, worker direct WSS, Windows transport, and WPF presenter queues. Manual visual smoke accepted idle repaint, Start menu/hover, mouse, keyboard, and session close on 2026-04-26.
• Remaining visual limitation is quality/performance rather than correctness: window drag behaves like older/slow-link RDP clients by showing a drag frame, and repaint after releasing a moved window is usable but not yet polished.
• FreeRDP is still present as the current internal substrate behind the RDP Adapter boundary. It must not be removed until the adapter path is live-proven and replacement layers are ready.

This does not change the current cluster/control-plane contracts. The current backend gateway fallback remains available until each data-plane stage is proven.

1. Goal

The RDP Adapter must translate Microsoft RDP into the platform session/data-plane protocol.

Access Client
  <-> platform session/data-plane protocol
RDP Adapter
  <-> FreeRDP / project-owned RDP internals
RDP Server

The adapter must process events from both sides:

• Access Client events: input, clipboard, file upload/download, control.
• RDP Server events: graphics updates, cursor updates, clipboard changes, device/drive events, disconnects, errors.

The adapter must not depend on mouse/keyboard input to discover screen changes.

2. External References And Lessons

FreeRDP exposes an event-driven client model through:

• freerdp_get_event_handles / freerdp_check_event_handles for event dispatch.
• rdpUpdate callbacks such as BeginPaint, EndPaint, BitmapUpdate, RefreshRect, SurfaceBits, SurfaceFrameMarker, and SurfaceFrameBits.
• client channel modules such as cliprdr, rdpdr, and rdpgfx.

Apache Guacamole uses the same architectural principle at a higher level: protocol-specific plugins translate RDP/VNC/SSH into a common client protocol so the client does not implement those protocols directly.

Design implication for this project:

• FreeRDP callbacks/channels are adapter-origin event sources.
• The platform Access Client receives normalized display/cursor/clipboard/file/control events.
• Full-frame polling is only fallback/debug, not the target render mechanism.

3. Runtime Components

SessionRuntime
  owns lifecycle/assignment/policy/lease boundary
  owns RDP Adapter Runtime

RDP Adapter Runtime
  RdpEventPump
  InputAdapter
  DisplayAdapter
  CursorAdapter
  ClipboardAdapter
  FileTransferAdapter
  QualityController
  AdapterEventRouter

DataPlane Sinks
  direct worker WSS
  backend gateway fallback

RdpEventPump

Responsibilities:

• own the FreeRDP event loop
• wait on FreeRDP event handles
• dispatch FreeRDP callbacks promptly
• never sleep instead of processing available server events
• report disconnect/error state

InputAdapter

Responsibilities:

• accept normalized platform input
• preserve keyboard down/up ordering
• preserve mouse button/wheel ordering
• coalesce pointer move to latest
• send focus/move/button/key through FreeRDP input API
• never trigger full-frame capture loops as the main render mechanism

DisplayAdapter

Responsibilities:

• consume FreeRDP update callbacks
• generate platform display events
• prefer dirty regions/surface updates over full frames
• send baseline full frame only on connect/resize/attach/recovery/fallback
• keep a full framebuffer only where needed for compatibility
• never block input on render work

Required event sources:

• BitmapUpdate
• RefreshRect
• SurfaceBits
• SurfaceFrameMarker
• SurfaceFrameBits
• EndPaint
• RDPGFX channel events when enabled and stable
• periodic fallback change detection only as a safety net

CursorAdapter

Responsibilities:

• handle FreeRDP pointer callbacks
• publish cursor position/visibility/shape independently from display frames
• keep cursor events latest-only

ClipboardAdapter

Responsibilities:

• use cliprdr
• preserve existing clipboard_mode
• text-only until explicitly expanded
• enforce max size and lifecycle state
• prevent loops using sequence/origin/hash

FileTransferAdapter

Responsibilities:

• preserve existing file_transfer_mode
• keep upload/download reliable and chunked
• enforce session/controller/policy/state
• keep restricted drive mapping isolated to per-session visible directory
• never expose arbitrary worker filesystem paths

QualityController

Responsibilities:

• choose color mode / FPS / dirty-region threshold
• degrade render before input
• keep file transfer and future VPN-like bulk traffic from starving interactive channels

4. Data-Plane Streams

The target adapter uses independent scheduling classes even if they share one WSS connection in DP-1:

Stream	Channel	Scheduling
Critical input	input	first, ordered, bounded
Control	control	reliable, bounded
Cursor	cursor	latest-only, bypass display cadence
Display	display	droppable, latest region/frame
Clipboard	clipboard	reliable, policy-gated
File transfer	file_transfer	reliable chunked, bandwidth-limited
Telemetry	telemetry	sampled/droppable

Future transports may split streams physically:

• control/input WSS
• display binary WSS or QUIC-like transport
• file transfer chunk stream
• audio/video adaptive stream

DP-1 must keep current direct WSS/fallback intact while enforcing scheduling semantics internally.

5. Display Contract

Display event types:

• display.baseline_full_bgra
• display.region_bgra
• display.surface_create
• display.surface_delete
• display.surface_bits
• display.encoded_frame
• display.resize
• display.sync

Rules:

• Access Client owns the visible framebuffer.
• Region updates patch the existing full-size framebuffer.
• Adapter must send a baseline frame before region-only updates after connect/attach/resize.
• Stale display updates may be dropped.
• Cursor updates must not wait for display frames.
• Full-frame BGRA is fallback, not production target.
• Direct binary display messages use the existing RAP2 frame header: render.frame.full for baseline/recovery frames and render.frame.region for BGRA32 dirty-region payloads.

6. FreeRDP Usage Rules

Default stable mode:

• GDI/primary framebuffer fallback
• update callbacks installed
• cliprdr enabled only when policy permits
• rdpdr restricted drive only when file transfer policy permits

Experimental/next modes:

• RDPGFX dynamic channel behind explicit capability flag
• surface/event parsing before enabling by default
• encoded graphics payloads only when client capability and server support are proven

Do not enable unstable graphics paths globally. Each capability must be gated, logged, and fallback-safe.

7. Migration Plan

RDP-A1: Contract And Scaffolding

Deliver:

• common Service Adapter protocol document
• RDP Adapter runtime document
• compile-safe adapter channel model
• no runtime behavior switch

Status: completed and build-proven.

RDP-A2: Event Router Boundary

Deliver:

• route FreeRDP notifications through AdapterEventRouter
• preserve existing WorkerEvent output
• prove server-origin display events flow without client input

Status: completed and live-smoke-proven on the test Docker environment as of 2026-04-26.

Current code boundary:

• SessionRuntime owns RdpAdapterRuntime.
• RdpAdapterRuntime owns the current FreeRDP substrate.
• AdapterEventRouter normalizes substrate notifications into adapter event descriptors.
• Existing worker events and data-plane contracts are preserved.

Smoke command:

pwsh -ExecutionPolicy Bypass -File scripts/windows-smoke/desktop-smoke.ps1 `
  -PreferDirectDataPlane:$true `
  -AllowInsecureDirectDataPlaneTlsForSmoke:$true `
  -DirectDataPlaneConnectTimeoutMs 2500 `
  -DirectDataPlaneColorMode full_color `
  -SkipOrgSwitchAndTokenRefresh

Smoke evidence:

• worker image: rap-rdp-worker:rdp-adapter-a2
• session id: c835e211-a105-4165-9ed2-885ddf876b84
• worker log: rdp_adapter.runtime_start substrate=freerdp
• worker log: adapter_event channel=display type=display.baseline_full_bgra
• worker log: data_plane_bind_success ... render_transport=binary_v1
• client log: data_plane.transport selected=direct_worker_wss
• client log: SessionWindow rendered frame
• smoke result: login/resource/start/input/detach/attach/takeover/taken_over/logout passed
• runtime creation count: one started new runtime for session entry across start/reattach/takeover

RDP-A3: DisplayAdapter Region-First

Deliver:

• baseline full frame on connect/attach/resize
• region updates as default normal UI path
• client framebuffer patch proof
• full-frame fallback retained

Status: completed and live-smoke-proven on the test Docker environment as of 2026-04-26.

Proof summary:

• BitmapUpdate dirty regions are deferred and flushed once at EndPaint.
• Region payloads are sent over direct binary WSS as message_type=render.frame.region with frame_update_kind=region.
• Windows client renders region frames into the existing framebuffer.
• Backend gateway fallback remains available and smoke-proven.
• Report: artifacts/rdp-perf3-report.md

Prerequisite proof:

• RDP-Perf-2 showed active BitmapUpdate, BeginPaint, and EndPaint callbacks in stable GDI mode.
• RDP-Perf-2 did not observe RefreshRect, SurfaceBits, SurfaceFrameMarker, SurfaceFrameBits, or pointer callbacks in the live smoke.
• The next implementation should prefer BitmapUpdate dirty regions and treat EndPaint as a flush/safety marker instead of producing duplicate captures.

RDP-A4: CursorAdapter

Deliver:

• cursor position/shape/visibility channel
• cursor updates independent from render cadence

Status: completed and live-smoke-proven on the test Docker environment as of 2026-04-26.

Implementation notes:

• CursorAdapter produces normalized cursor position, visibility, shape, cache, hotspot, and mask metadata.
• The FreeRDP substrate invokes original pointer callbacks first, then publishes platform cursor events.
• session_cursor_updated is routed as the adapter event cursor.update.
• Direct worker WSS keeps cursor as latest-only/droppable and does not block it behind render frames.
• The Windows client consumes cursor.update without changing session lifecycle or UI layout.

Proof:

• direct smoke session id: 549806aa-c9db-48a9-917e-cf817cf236b5
• fallback smoke session id: dee3a856-bee1-4eba-9c10-f62edaf56547
• worker image: rap-rdp-worker:rdp-a4-cursor-adapter
• report: artifacts/rdp-a4-cursor-adapter-report.md

RDP-A4.1 / RDP-Perf-5A: GDI Repaint Cadence Hardening

Deliver:

• bounded immediate FreeRDP event-handle drain after signaled event checks
• rate-limited no-change detector logs
• no Redis lease renewal in the hot render/input loop
• 33 ms region/interactive render publish cadence
• 100 ms full-frame fallback cadence retained
• direct worker WSS and backend gateway fallback compatibility

Status: completed and smoke-proven on the test Docker environment as of 2026-04-26.

Proof summary:

• direct smoke session id: 0cca4974-2a82-48dc-a0f6-1036ea8e98f0
• fallback smoke session id: 16deb09e-1c44-4e9d-8448-93b42ac66ed0
• worker image: rap-rdp-worker:rdp-perf5a-repaint-cadence
• direct worker WSS selected in direct smoke
• backend gateway selected in fallback smoke
• direct render stayed binary and skipped JSON/base64 compatibility frame building
• backend gateway fallback still built JSON/base64 compatibility frames
• render queues stayed bounded in observed direct smoke
• report: artifacts/rdp-perf5a-report.md

Follow-up manual validation:

• keyboard behavior reached a usable level
• mouse movement/click behavior became acceptable for the MVP baseline
• remote idle updates such as Task Manager percentages now repaint without local mouse movement
• small redraw artifacts remain and require a focused visual correctness pass

RDP-A4.2: Direct Attach Baseline And Region-Loss Repair

Deliver:

• request a full-frame baseline when a direct client attaches without a cached full frame
• queue direct attach baseline frames as non-droppable reliable events
• preserve region-first rendering for normal updates
• capture throttled full-frame repair when region loss/drop can leave persistent artifacts
• keep input, clipboard, file upload, session lifecycle, direct worker WSS, and backend gateway fallback unchanged

Status: previous accepted baseline, superseded by P1 ordered-region delivery on 2026-04-26.

Proof summary:

• worker image: rap-rdp-worker:rdp-region-repair
• worker probes pass for graphics adapter, cursor adapter, service adapter protocol, and direct data-plane bind validation
• direct attach no longer starts from a black-only framebuffer when no cached full frame is available
• server-origin idle updates are visible without local input
• remaining issue is small redraw artifacts during some region update sequences

Current code boundaries:

• SessionRuntime::PublishDirectAttachBaselineIfRequested
• SessionRuntime::DrainAndPublishRenderNotifications
• RdpAdapterRuntime::CaptureFullFrameNotification
• RdpRuntime::CaptureFullFrameNotification
• DirectWssEventSink::EnqueueEvent

Next hardening target:

• add region sequence/gap diagnostics
• identify whether remaining artifacts come from dropped regions, stale ordering, wrong client patching, missed callbacks, or repair timing
• apply the smallest fix without returning to full-frame polling as the normal render path

RDP-A4.3 / P1: Ordered Region Delivery Candidate

Root cause addressed:

• Region frames were passing through latest-frame-only queues in the direct worker writer, Windows transport, and WPF presenter.
• A second ordered-delivery gap was found in SessionRuntime, where frame notifications were still coalesced before reaching the direct event sink.
• Latest-frame-only behavior is correct for full frames and cursor updates, but it is unsafe for dirty-region patches because dropping an intermediate region can leave stale pixels on the client framebuffer.

Deliver:

• preserve ordered dirty-region frames through the worker direct WSS writer
• preserve ordered dirty-region frames inside SessionRuntime before the direct event sink
• preserve ordered dirty-region frames through the Windows direct transport
• preserve ordered dirty-region application in the WPF session presenter
• keep full frames able to supersede pending region queues
• request a throttled full-frame repair if the worker direct region queue overflows
• add client diagnostics for frame sequence gaps and regions received before a baseline
• keep input, cursor, clipboard, file upload, session lifecycle, direct worker WSS, and backend gateway fallback unchanged

Status: accepted baseline on the test Docker environment as of 2026-04-26.

Proof summary:

• worker image: rap-rdp-worker:rdp-p1-region-order2
• live test container: rap_worker_smoke
• backend go test ./...: PASS
• Windows solution build: PASS
• worker graphics adapter probe: PASS
• worker cursor adapter probe: PASS
• worker service adapter protocol probe: PASS
• worker direct data-plane bind valid probe: PASS
• worker Redis registration: worker:registration:rdp-worker-1 reports status=online
• manual visual smoke: PASS for idle Task Manager updates without local input, Start menu/hover without persistent artifacts, window drag usability, mouse, keyboard, and session close
• known limitation: drag uses old-client frame-only movement and release repaint is not polished

Current code boundaries:

• SessionRuntime::RequestDirectFullFrameRepair
• SessionRuntime::DrainAndPublishRenderNotifications
• DirectWssEventSink::EnqueueEvent
• SessionGatewayClient::QueueFrameEnvelope
• SessionWindow::QueueFrameForPresentation
• SessionWindowViewModel::ApplyFramePayload

Manual acceptance result:

• Start menu/menu hover does not leave persistent stale regions.
• Task Manager graph/percent updates continue without local input.
• Mouse and keyboard responsiveness did not regress.
• Session close works normally.
• Window drag is workable but uses frame-only movement and non-perfect repaint after release; this belongs to the next performance/quality layer.

RDP-Perf-6: Dirty-Region Direct Binary Contract

Goal:

• make dirty-region direct render explicit at the RAP2 binary contract level
• keep full-frame binary support as baseline/recovery fallback
• keep backend gateway JSON/base64 fallback unchanged
• avoid routing high-rate binary regions through Redis/backend

Status: implemented and build/probe/live-smoke-proven on the test Docker environment as of 2026-04-26 using direct worker WSS and rap-rdp-worker:rdp-perf6-dirty-region.

Implementation:

• Worker direct WSS emits render.frame.full for first frame, attach/reattach, resize, region-loss repair, invalid region fallback, and debug fallback.
• Worker direct WSS emits render.frame.region for BGRA32 dirty-region payloads from the current classic GDI region-first path.
• Region metadata includes full desktop dimensions, region coordinates, region_stride, region_format=BGRA32, payload length, sequence, and capture/input timing fields.
• Worker diagnostics include full_frame_sent, region_frame_sent, region_bytes, full_frame_bytes, region_savings_percent, diff_time_ms, render_update_reason, and fallback_to_full_frame_reason.
• Windows direct transport accepts render.frame.full, render.frame.region, and compat session.frame binary messages.
• Windows presenter keeps a per-session framebuffer and patches region bytes into it before presenting the updated WPF surface.
• Smoke proof showed baseline render.frame.full at 3,686,400 bytes and dirty-region render.frame.region payloads such as 16,384, 163,840, 327,680, and 655,360 bytes, with observed savings up to 99.56%.

Boundaries preserved:

• no backend session lifecycle changes
• no organization/auth/policy changes
• no data_plane_token contract changes
• no clipboard or file-transfer semantic changes
• no RDPGFX default enablement
• no mesh/VPN/relay/QUIC/WebRTC work
• backend gateway fallback remains available

RDP-A5: Clipboard/File Adapters

Deliver:

• move current cliprdr/file logic behind adapter boundaries
• no behavior change
• policy enforcement unchanged

RDP-A6: RDPGFX Foundation

Deliver:

• gated RDPGFX surface event support
• fallback to GDI region updates
• no default enable until stable

Status: build-proven and default-path smoke-proven on the test Docker environment as of 2026-04-26.

Notes:

• RDP_WORKER_RDPGFX_ENABLED=true is the explicit gated switch.
• The default runtime path remains classic GDI region-first.
• The current test RDP target failed gated RDPGFX with a connection reset before rdp.gfx channel_connected, so no RDPGFX surface lifecycle proof is available for that target yet.
• Report: artifacts/rdp-perf4-report.md

RDP-A7: Encoded/Adaptive Render

Deliver:

• encoded display payloads where negotiated
• adaptive quality profiles
• weak-channel policy

8. Acceptance Criteria For New RDP Adapter

• idle remote screen changes are visible without local mouse/keyboard input
• first click acts on remote UI, not only focus
• pointer hover updates are visible
• keyboard does not lose characters
• detach/reattach/takeover do not recreate remote session
• worker death marks session failed/recoverable correctly
• clipboard and file transfer remain policy-enforced
• direct worker WSS is preferred and fallback remains working
• input latency is not affected by render/file/telemetry pressure