Initial project snapshot

2026-04-28 22:29:50 +03:00
commit 8ba0561f4f
365 changed files with 91832 additions and 0 deletions
@@ -0,0 +1,559 @@
+# RDP Adapter Runtime
+
+Status: active implementation plan for the new C++ RDP Adapter internals.
+
+Current implementation status:
+
+- RDP-A1 is build-proven: the common Service Adapter channel model and probe exist.
+- RDP-A2 is live-smoke-proven on the test Docker environment as of 2026-04-26: `SessionRuntime` depends on `RdpAdapterRuntime`, not directly on FreeRDP runtime types, and a real RDP session still connects through the existing direct data plane.
+- RDP-Perf-2 is live-smoke-proven on the test Docker environment as of 2026-04-26: the current FreeRDP substrate now logs callback source/timing, capture source, and input-to-first-graphics-callback timing.
+- RDP-Perf-3 / RDP-A3 region-first BGRA fallback is live-smoke-proven on the test Docker environment as of 2026-04-26: direct binary region frames render in the Windows client and backend gateway fallback remains compatible.
+- RDP-Perf-4 / RDP-A6 gated RDPGFX foundation is build-proven and default-path smoke-proven on the test Docker environment as of 2026-04-26. RDPGFX stays disabled by default because the current live RDP target resets the connection when graphics pipeline support is advertised.
+- RDP-A4 CursorAdapter is live-smoke-proven on the test Docker environment as of 2026-04-26: FreeRDP pointer callbacks are normalized into latest-only `cursor.update` events, direct worker WSS sends them separately from display frames, and backend gateway fallback remains compatible.
+- RDP-Perf-5A is build-proven and smoke-proven on the test Docker environment as of 2026-04-26: classic GDI region/interactive frames use a 33 ms publish cadence, hot-loop lease renewal is removed, and direct/fallback paths remain compatible.
+- RDP-Perf-6 direct dirty-region binary contract is build/probe/live-smoke-proven on the test Docker environment as of 2026-04-26: direct `RAP2` frames now distinguish `render.frame.full` from `render.frame.region`, include region payload diagnostics, and the Windows presenter keeps a session framebuffer for region patching. Runtime proof used `P3.3 Secret RDP Resource`; observed dirty-region savings ranged from `82.22%` to `99.56%` versus the `3,686,400` byte full frame.
+- Current accepted baseline is `rap-rdp-worker:rdp-p1-region-order2`: ordered dirty-region delivery is preserved through `SessionRuntime`, worker direct WSS, Windows transport, and WPF presenter queues. Manual visual smoke accepted idle repaint, Start menu/hover, mouse, keyboard, and session close on 2026-04-26.
+- Remaining visual limitation is quality/performance rather than correctness: window drag behaves like older/slow-link RDP clients by showing a drag frame, and repaint after releasing a moved window is usable but not yet polished.
+- FreeRDP is still present as the current internal substrate behind the RDP Adapter boundary. It must not be removed until the adapter path is live-proven and replacement layers are ready.
+
+This does not change the current cluster/control-plane contracts. The current backend gateway fallback remains available until each data-plane stage is proven.
+
+## 1. Goal
+
+The RDP Adapter must translate Microsoft RDP into the platform session/data-plane protocol.
+
+```text
+Access Client
+  <-> platform session/data-plane protocol
+RDP Adapter
+  <-> FreeRDP / project-owned RDP internals
+RDP Server
+```
+
+The adapter must process events from both sides:
+
+- Access Client events: input, clipboard, file upload/download, control.
+- RDP Server events: graphics updates, cursor updates, clipboard changes, device/drive events, disconnects, errors.
+
+The adapter must not depend on mouse/keyboard input to discover screen changes.
+
+## 2. External References And Lessons
+
+FreeRDP exposes an event-driven client model through:
+
+- `freerdp_get_event_handles` / `freerdp_check_event_handles` for event dispatch.
+- `rdpUpdate` callbacks such as `BeginPaint`, `EndPaint`, `BitmapUpdate`, `RefreshRect`, `SurfaceBits`, `SurfaceFrameMarker`, and `SurfaceFrameBits`.
+- client channel modules such as `cliprdr`, `rdpdr`, and `rdpgfx`.
+
+Apache Guacamole uses the same architectural principle at a higher level: protocol-specific plugins translate RDP/VNC/SSH into a common client protocol so the client does not implement those protocols directly.
+
+Design implication for this project:
+
+- FreeRDP callbacks/channels are adapter-origin event sources.
+- The platform Access Client receives normalized display/cursor/clipboard/file/control events.
+- Full-frame polling is only fallback/debug, not the target render mechanism.
+
+## 3. Runtime Components
+
+```text
+SessionRuntime
+  owns lifecycle/assignment/policy/lease boundary
+  owns RDP Adapter Runtime
+
+RDP Adapter Runtime
+  RdpEventPump
+  InputAdapter
+  DisplayAdapter
+  CursorAdapter
+  ClipboardAdapter
+  FileTransferAdapter
+  QualityController
+  AdapterEventRouter
+
+DataPlane Sinks
+  direct worker WSS
+  backend gateway fallback
+```
+
+### RdpEventPump
+
+Responsibilities:
+
+- own the FreeRDP event loop
+- wait on FreeRDP event handles
+- dispatch FreeRDP callbacks promptly
+- never sleep instead of processing available server events
+- report disconnect/error state
+
+### InputAdapter
+
+Responsibilities:
+
+- accept normalized platform input
+- preserve keyboard down/up ordering
+- preserve mouse button/wheel ordering
+- coalesce pointer move to latest
+- send focus/move/button/key through FreeRDP input API
+- never trigger full-frame capture loops as the main render mechanism
+
+### DisplayAdapter
+
+Responsibilities:
+
+- consume FreeRDP update callbacks
+- generate platform display events
+- prefer dirty regions/surface updates over full frames
+- send baseline full frame only on connect/resize/attach/recovery/fallback
+- keep a full framebuffer only where needed for compatibility
+- never block input on render work
+
+Required event sources:
+
+- `BitmapUpdate`
+- `RefreshRect`
+- `SurfaceBits`
+- `SurfaceFrameMarker`
+- `SurfaceFrameBits`
+- `EndPaint`
+- RDPGFX channel events when enabled and stable
+- periodic fallback change detection only as a safety net
+
+### CursorAdapter
+
+Responsibilities:
+
+- handle FreeRDP pointer callbacks
+- publish cursor position/visibility/shape independently from display frames
+- keep cursor events latest-only
+
+### ClipboardAdapter
+
+Responsibilities:
+
+- use `cliprdr`
+- preserve existing `clipboard_mode`
+- text-only until explicitly expanded
+- enforce max size and lifecycle state
+- prevent loops using sequence/origin/hash
+
+### FileTransferAdapter
+
+Responsibilities:
+
+- preserve existing `file_transfer_mode`
+- keep upload/download reliable and chunked
+- enforce session/controller/policy/state
+- keep restricted drive mapping isolated to per-session visible directory
+- never expose arbitrary worker filesystem paths
+
+### QualityController
+
+Responsibilities:
+
+- choose color mode / FPS / dirty-region threshold
+- degrade render before input
+- keep file transfer and future VPN-like bulk traffic from starving interactive channels
+
+## 4. Data-Plane Streams
+
+The target adapter uses independent scheduling classes even if they share one WSS connection in DP-1:
+
+| Stream | Channel | Scheduling |
+| --- | --- | --- |
+| Critical input | `input` | first, ordered, bounded |
+| Control | `control` | reliable, bounded |
+| Cursor | `cursor` | latest-only, bypass display cadence |
+| Display | `display` | droppable, latest region/frame |
+| Clipboard | `clipboard` | reliable, policy-gated |
+| File transfer | `file_transfer` | reliable chunked, bandwidth-limited |
+| Telemetry | `telemetry` | sampled/droppable |
+
+Future transports may split streams physically:
+
+- control/input WSS
+- display binary WSS or QUIC-like transport
+- file transfer chunk stream
+- audio/video adaptive stream
+
+DP-1 must keep current direct WSS/fallback intact while enforcing scheduling semantics internally.
+
+## 5. Display Contract
+
+Display event types:
+
+- `display.baseline_full_bgra`
+- `display.region_bgra`
+- `display.surface_create`
+- `display.surface_delete`
+- `display.surface_bits`
+- `display.encoded_frame`
+- `display.resize`
+- `display.sync`
+
+Rules:
+
+- Access Client owns the visible framebuffer.
+- Region updates patch the existing full-size framebuffer.
+- Adapter must send a baseline frame before region-only updates after connect/attach/resize.
+- Stale display updates may be dropped.
+- Cursor updates must not wait for display frames.
+- Full-frame BGRA is fallback, not production target.
+- Direct binary display messages use the existing `RAP2` frame header:
+  `render.frame.full` for baseline/recovery frames and `render.frame.region`
+  for BGRA32 dirty-region payloads.
+
+## 6. FreeRDP Usage Rules
+
+Default stable mode:
+
+- GDI/primary framebuffer fallback
+- update callbacks installed
+- cliprdr enabled only when policy permits
+- rdpdr restricted drive only when file transfer policy permits
+
+Experimental/next modes:
+
+- RDPGFX dynamic channel behind explicit capability flag
+- surface/event parsing before enabling by default
+- encoded graphics payloads only when client capability and server support are proven
+
+Do not enable unstable graphics paths globally. Each capability must be gated, logged, and fallback-safe.
+
+## 7. Migration Plan
+
+### RDP-A1: Contract And Scaffolding
+
+Deliver:
+
+- common Service Adapter protocol document
+- RDP Adapter runtime document
+- compile-safe adapter channel model
+- no runtime behavior switch
+
+Status: completed and build-proven.
+
+### RDP-A2: Event Router Boundary
+
+Deliver:
+
+- route FreeRDP notifications through `AdapterEventRouter`
+- preserve existing `WorkerEvent` output
+- prove server-origin display events flow without client input
+
+Status: completed and live-smoke-proven on the test Docker environment as of 2026-04-26.
+
+Current code boundary:
+
+- `SessionRuntime` owns `RdpAdapterRuntime`.
+- `RdpAdapterRuntime` owns the current FreeRDP substrate.
+- `AdapterEventRouter` normalizes substrate notifications into adapter event descriptors.
+- Existing worker events and data-plane contracts are preserved.
+
+Smoke command:
+
+```powershell
+pwsh -ExecutionPolicy Bypass -File scripts/windows-smoke/desktop-smoke.ps1 `
+  -PreferDirectDataPlane:$true `
+  -AllowInsecureDirectDataPlaneTlsForSmoke:$true `
+  -DirectDataPlaneConnectTimeoutMs 2500 `
+  -DirectDataPlaneColorMode full_color `
+  -SkipOrgSwitchAndTokenRefresh
+```
+
+Smoke evidence:
+
+- worker image: `rap-rdp-worker:rdp-adapter-a2`
+- session id: `c835e211-a105-4165-9ed2-885ddf876b84`
+- worker log: `rdp_adapter.runtime_start substrate=freerdp`
+- worker log: `adapter_event channel=display type=display.baseline_full_bgra`
+- worker log: `data_plane_bind_success ... render_transport=binary_v1`
+- client log: `data_plane.transport selected=direct_worker_wss`
+- client log: `SessionWindow rendered frame`
+- smoke result: login/resource/start/input/detach/attach/takeover/taken_over/logout passed
+- runtime creation count: one `started new runtime for session` entry across start/reattach/takeover
+
+### RDP-A3: DisplayAdapter Region-First
+
+Deliver:
+
+- baseline full frame on connect/attach/resize
+- region updates as default normal UI path
+- client framebuffer patch proof
+- full-frame fallback retained
+
+Status: completed and live-smoke-proven on the test Docker environment as of 2026-04-26.
+
+Proof summary:
+
+- `BitmapUpdate` dirty regions are deferred and flushed once at `EndPaint`.
+- Region payloads are sent over direct binary WSS as `message_type=render.frame.region` with `frame_update_kind=region`.
+- Windows client renders region frames into the existing framebuffer.
+- Backend gateway fallback remains available and smoke-proven.
+- Report: `artifacts/rdp-perf3-report.md`
+
+Prerequisite proof:
+
+- RDP-Perf-2 showed active `BitmapUpdate`, `BeginPaint`, and `EndPaint` callbacks in stable GDI mode.
+- RDP-Perf-2 did not observe `RefreshRect`, `SurfaceBits`, `SurfaceFrameMarker`, `SurfaceFrameBits`, or pointer callbacks in the live smoke.
+- The next implementation should prefer `BitmapUpdate` dirty regions and treat `EndPaint` as a flush/safety marker instead of producing duplicate captures.
+
+### RDP-A4: CursorAdapter
+
+Deliver:
+
+- cursor position/shape/visibility channel
+- cursor updates independent from render cadence
+
+Status: completed and live-smoke-proven on the test Docker environment as of
+2026-04-26.
+
+Implementation notes:
+
+- `CursorAdapter` produces normalized cursor position, visibility, shape, cache,
+  hotspot, and mask metadata.
+- The FreeRDP substrate invokes original pointer callbacks first, then publishes
+  platform cursor events.
+- `session_cursor_updated` is routed as the adapter event `cursor.update`.
+- Direct worker WSS keeps cursor as latest-only/droppable and does not block it
+  behind render frames.
+- The Windows client consumes `cursor.update` without changing session lifecycle
+  or UI layout.
+
+Proof:
+
+- direct smoke session id: `549806aa-c9db-48a9-917e-cf817cf236b5`
+- fallback smoke session id: `dee3a856-bee1-4eba-9c10-f62edaf56547`
+- worker image: `rap-rdp-worker:rdp-a4-cursor-adapter`
+- report: `artifacts/rdp-a4-cursor-adapter-report.md`
+
+### RDP-A4.1 / RDP-Perf-5A: GDI Repaint Cadence Hardening
+
+Deliver:
+
+- bounded immediate FreeRDP event-handle drain after signaled event checks
+- rate-limited no-change detector logs
+- no Redis lease renewal in the hot render/input loop
+- 33 ms region/interactive render publish cadence
+- 100 ms full-frame fallback cadence retained
+- direct worker WSS and backend gateway fallback compatibility
+
+Status: completed and smoke-proven on the test Docker environment as of
+2026-04-26.
+
+Proof summary:
+
+- direct smoke session id: `0cca4974-2a82-48dc-a0f6-1036ea8e98f0`
+- fallback smoke session id: `16deb09e-1c44-4e9d-8448-93b42ac66ed0`
+- worker image: `rap-rdp-worker:rdp-perf5a-repaint-cadence`
+- direct worker WSS selected in direct smoke
+- backend gateway selected in fallback smoke
+- direct render stayed binary and skipped JSON/base64 compatibility frame
+  building
+- backend gateway fallback still built JSON/base64 compatibility frames
+- render queues stayed bounded in observed direct smoke
+- report: `artifacts/rdp-perf5a-report.md`
+
+Follow-up manual validation:
+
+- keyboard behavior reached a usable level
+- mouse movement/click behavior became acceptable for the MVP baseline
+- remote idle updates such as Task Manager percentages now repaint without local
+  mouse movement
+- small redraw artifacts remain and require a focused visual correctness pass
+
+### RDP-A4.2: Direct Attach Baseline And Region-Loss Repair
+
+Deliver:
+
+- request a full-frame baseline when a direct client attaches without a cached
+  full frame
+- queue direct attach baseline frames as non-droppable reliable events
+- preserve region-first rendering for normal updates
+- capture throttled full-frame repair when region loss/drop can leave persistent
+  artifacts
+- keep input, clipboard, file upload, session lifecycle, direct worker WSS, and
+  backend gateway fallback unchanged
+
+Status: previous accepted baseline, superseded by P1 ordered-region delivery on
+2026-04-26.
+
+Proof summary:
+
+- worker image: `rap-rdp-worker:rdp-region-repair`
+- worker probes pass for graphics adapter, cursor adapter, service adapter
+  protocol, and direct data-plane bind validation
+- direct attach no longer starts from a black-only framebuffer when no cached
+  full frame is available
+- server-origin idle updates are visible without local input
+- remaining issue is small redraw artifacts during some region update
+  sequences
+
+Current code boundaries:
+
+- `SessionRuntime::PublishDirectAttachBaselineIfRequested`
+- `SessionRuntime::DrainAndPublishRenderNotifications`
+- `RdpAdapterRuntime::CaptureFullFrameNotification`
+- `RdpRuntime::CaptureFullFrameNotification`
+- `DirectWssEventSink::EnqueueEvent`
+
+Next hardening target:
+
+- add region sequence/gap diagnostics
+- identify whether remaining artifacts come from dropped regions, stale ordering,
+  wrong client patching, missed callbacks, or repair timing
+- apply the smallest fix without returning to full-frame polling as the normal
+  render path
+
+### RDP-A4.3 / P1: Ordered Region Delivery Candidate
+
+Root cause addressed:
+
+- Region frames were passing through latest-frame-only queues in the direct
+  worker writer, Windows transport, and WPF presenter.
+- A second ordered-delivery gap was found in `SessionRuntime`, where frame
+  notifications were still coalesced before reaching the direct event sink.
+- Latest-frame-only behavior is correct for full frames and cursor updates, but
+  it is unsafe for dirty-region patches because dropping an intermediate region
+  can leave stale pixels on the client framebuffer.
+
+Deliver:
+
+- preserve ordered dirty-region frames through the worker direct WSS writer
+- preserve ordered dirty-region frames inside `SessionRuntime` before the direct
+  event sink
+- preserve ordered dirty-region frames through the Windows direct transport
+- preserve ordered dirty-region application in the WPF session presenter
+- keep full frames able to supersede pending region queues
+- request a throttled full-frame repair if the worker direct region queue
+  overflows
+- add client diagnostics for frame sequence gaps and regions received before a
+  baseline
+- keep input, cursor, clipboard, file upload, session lifecycle, direct worker
+  WSS, and backend gateway fallback unchanged
+
+Status: accepted baseline on the test Docker environment as of 2026-04-26.
+
+Proof summary:
+
+- worker image: `rap-rdp-worker:rdp-p1-region-order2`
+- live test container: `rap_worker_smoke`
+- backend `go test ./...`: PASS
+- Windows solution build: PASS
+- worker graphics adapter probe: PASS
+- worker cursor adapter probe: PASS
+- worker service adapter protocol probe: PASS
+- worker direct data-plane bind valid probe: PASS
+- worker Redis registration: `worker:registration:rdp-worker-1` reports
+  `status=online`
+- manual visual smoke: PASS for idle Task Manager updates without local input,
+  Start menu/hover without persistent artifacts, window drag usability, mouse,
+  keyboard, and session close
+- known limitation: drag uses old-client frame-only movement and release repaint
+  is not polished
+
+Current code boundaries:
+
+- `SessionRuntime::RequestDirectFullFrameRepair`
+- `SessionRuntime::DrainAndPublishRenderNotifications`
+- `DirectWssEventSink::EnqueueEvent`
+- `SessionGatewayClient::QueueFrameEnvelope`
+- `SessionWindow::QueueFrameForPresentation`
+- `SessionWindowViewModel::ApplyFramePayload`
+
+Manual acceptance result:
+
+- Start menu/menu hover does not leave persistent stale regions.
+- Task Manager graph/percent updates continue without local input.
+- Mouse and keyboard responsiveness did not regress.
+- Session close works normally.
+- Window drag is workable but uses frame-only movement and non-perfect repaint
+  after release; this belongs to the next performance/quality layer.
+
+### RDP-Perf-6: Dirty-Region Direct Binary Contract
+
+Goal:
+
+- make dirty-region direct render explicit at the `RAP2` binary contract level
+- keep full-frame binary support as baseline/recovery fallback
+- keep backend gateway JSON/base64 fallback unchanged
+- avoid routing high-rate binary regions through Redis/backend
+
+Status: implemented and build/probe/live-smoke-proven on the test Docker
+environment as of 2026-04-26 using direct worker WSS and
+`rap-rdp-worker:rdp-perf6-dirty-region`.
+
+Implementation:
+
+- Worker direct WSS emits `render.frame.full` for first frame, attach/reattach,
+  resize, region-loss repair, invalid region fallback, and debug fallback.
+- Worker direct WSS emits `render.frame.region` for BGRA32 dirty-region
+  payloads from the current classic GDI region-first path.
+- Region metadata includes full desktop dimensions, region coordinates,
+  `region_stride`, `region_format=BGRA32`, payload length, sequence, and
+  capture/input timing fields.
+- Worker diagnostics include `full_frame_sent`, `region_frame_sent`,
+  `region_bytes`, `full_frame_bytes`, `region_savings_percent`,
+  `diff_time_ms`, `render_update_reason`, and
+  `fallback_to_full_frame_reason`.
+- Windows direct transport accepts `render.frame.full`,
+  `render.frame.region`, and legacy `session.frame` binary messages.
+- Windows presenter keeps a per-session framebuffer and patches region bytes
+  into it before presenting the updated WPF surface.
+- Smoke proof showed baseline `render.frame.full` at `3,686,400` bytes and
+  dirty-region `render.frame.region` payloads such as `16,384`, `163,840`,
+  `327,680`, and `655,360` bytes, with observed savings up to `99.56%`.
+
+Boundaries preserved:
+
+- no backend session lifecycle changes
+- no organization/auth/policy changes
+- no `data_plane_token` contract changes
+- no clipboard or file-transfer semantic changes
+- no RDPGFX default enablement
+- no mesh/VPN/relay/QUIC/WebRTC work
+- backend gateway fallback remains available
+
+### RDP-A5: Clipboard/File Adapters
+
+Deliver:
+
+- move current cliprdr/file logic behind adapter boundaries
+- no behavior change
+- policy enforcement unchanged
+
+### RDP-A6: RDPGFX Foundation
+
+Deliver:
+
+- gated RDPGFX surface event support
+- fallback to GDI region updates
+- no default enable until stable
+
+Status: build-proven and default-path smoke-proven on the test Docker environment as of 2026-04-26.
+
+Notes:
+
+- `RDP_WORKER_RDPGFX_ENABLED=true` is the explicit gated switch.
+- The default runtime path remains classic GDI region-first.
+- The current test RDP target failed gated RDPGFX with a connection reset before `rdp.gfx channel_connected`, so no RDPGFX surface lifecycle proof is available for that target yet.
+- Report: `artifacts/rdp-perf4-report.md`
+
+### RDP-A7: Encoded/Adaptive Render
+
+Deliver:
+
+- encoded display payloads where negotiated
+- adaptive quality profiles
+- weak-channel policy
+
+## 8. Acceptance Criteria For New RDP Adapter
+
+- idle remote screen changes are visible without local mouse/keyboard input
+- first click acts on remote UI, not only focus
+- pointer hover updates are visible
+- keyboard does not lose characters
+- detach/reattach/takeover do not recreate remote session
+- worker death marks session failed/recoverable correctly
+- clipboard and file transfer remain policy-enforced
+- direct worker WSS is preferred and fallback remains working
+- input latency is not affected by render/file/telemetry pressure