Initial SQL-only 1C adapter baseline

2026-07-22 03:03:47 +03:00
commit e2503b77e7
545 changed files with 184711 additions and 0 deletions
@@ -0,0 +1,337 @@
+# 1C MOXCEL Discovery
+
+This runbook describes the read-only discovery loop for tabular document
+MOXCEL payloads.
+
+## Current Artifacts
+
+- `reports/1c-template-baselines/Primer3_moxel_schema_discovery.json`
+  contains inferred decoder rules from marker matrices and history diffs.
+- `reports/1c-template-baselines/Primer3_moxel_property_experiments.json`
+  contains one-property experiment analysis and the next probe plan.
+- `reports/1c-template-baselines/Primer3_moxel_property_experiments.manifest.json`
+  maps existing before/after summaries to property experiment labels.
+
+## Schema Discovery
+
+Run marker/history discovery:
+
+```powershell
+python scripts/discover_1c_moxel_schema.py `
+  --marker-matrix reports/1c-template-baselines/Primer3_marker_matrix_2026-06-27_latest.json `
+  --history-matrix reports/1c-template-baselines/Primer3_history_matrix.json `
+  --property-candidates reports/1c-template-baselines/Primer3_property_candidates.json `
+  --output-json reports/1c-template-baselines/Primer3_moxel_schema_discovery.json `
+  --output-markdown reports/1c-template-baselines/Primer3_moxel_schema_discovery.md
+```
+
+The current strongest rule is:
+
+```text
+moxel.inline_text_cell.column:
+one_based_column = int(last_numeric(preceding_scalars)) + 1
+```
+
+Named range coordinates are partially proven from controlled moves:
+
+```text
+left/right: raw scalar indexes 2 and 4
+top/bottom: raw scalar indexes 3 and 5
+one_based = raw + 1
+```
+
+Use multi-cell and rectangular named ranges to split `left` from `right` and
+`top` from `bottom`.
+
+Build the stable registry after discovery:
+
+```powershell
+python scripts/build_1c_moxel_schema_registry.py `
+  --discovery reports/1c-template-probes/upo_test_auto_moxel_schema_discovery.json `
+  --discovery reports/1c-template-baselines/Primer3_moxel_schema_discovery.json `
+  --output-json plugins/1c/metadata/moxel-schema-registry.json `
+  --output-markdown reports/1c-template-baselines/moxel-schema-registry.md
+```
+
+Validate the registry safety contract:
+
+```powershell
+python scripts/check_1c_moxel_schema_registry.py `
+  --registry plugins/1c/metadata/moxel-schema-registry.json `
+  --output reports/1c-template-baselines/moxel-schema-registry-check.json
+```
+
+The registry allows read-side decoder rules only. Write-side MOXCEL mutation is
+kept blocked until a disposable-base round-trip proves the exact scalar path.
+
+Verify registry rules against concrete probe snapshots:
+
+```powershell
+python scripts/verify_1c_moxel_schema_registry.py `
+  --registry plugins/1c/metadata/moxel-schema-registry.json `
+  --probe reports/1c-template-probes/upo_test_auto_20260627T142419Z_670780b4.json `
+  --output reports/1c-template-baselines/moxel-schema-registry-verification.json
+```
+
+This check is data-backed: `verified_read` rules must pass on the supplied
+probe snapshots, while `candidate_read` rules are reported as diagnostic
+evidence and are not promoted automatically.
+
+## Full Pipeline
+
+After a new probe or one-property experiment has been captured, refresh all
+MOXCEL discovery artifacts with one command:
+
+```powershell
+python scripts/run_1c_moxel_discovery_pipeline.py
+```
+
+To capture the latest live MOXCEL probe first and then refresh all artifacts:
+
+```powershell
+python scripts/run_1c_moxel_discovery_pipeline.py --capture-live --label pipeline-live
+```
+
+The pipeline runs schema discovery, named-range rule analysis, property
+experiment analysis, registry build, registry safety check, registry
+verification, and the next experiment plan. It writes:
+
+- `reports/1c-template-baselines/moxel-discovery-pipeline.json`
+- `reports/1c-template-baselines/moxel-discovery-pipeline.md`
+- `reports/1c-template-probes/latest-live-probe.json` when `--capture-live` is used
+- `reports/1c-template-baselines/moxel-named-range-rules.md`
+- `reports/1c-template-baselines/moxel-next-experiments.md`
+- `reports/1c-template-baselines/moxel-next-action.json`
+- `reports/1c-template-baselines/moxel-next-action.md`
+- `reports/1c-template-baselines/moxel-next-action-check.json`
+- `reports/1c-template-baselines/moxel-status.md`
+
+When `moxel-named-range-rules.json` contains high-confidence rectangular range
+evidence, the registry build step automatically promotes the corresponding
+named-range read rules from `candidate_read` to `verified_read`. Write status
+still remains blocked until a separate round-trip proof exists.
+
+## Property Experiments
+
+Generate or refresh the property probe plan:
+
+```powershell
+python scripts/analyze_1c_moxel_property_experiments.py `
+  --emit-default-plan `
+  --output-json reports/1c-template-baselines/Primer3_moxel_property_experiments.json `
+  --output-markdown reports/1c-template-baselines/Primer3_moxel_property_experiments.md
+```
+
+Analyze existing before/after experiments:
+
+```powershell
+python scripts/analyze_1c_moxel_property_experiments.py `
+  --manifest reports/1c-template-baselines/Primer3_moxel_property_experiments.manifest.json `
+  --emit-default-plan `
+  --output-json reports/1c-template-baselines/Primer3_moxel_property_experiments.json `
+  --output-markdown reports/1c-template-baselines/Primer3_moxel_property_experiments.md
+```
+
+Each new experiment should change exactly one property on the same tracked cell
+or range, then capture a fresh `templates.map`/summary snapshot.
+
+To capture the next manual one-property save automatically, start the watcher
+before changing and saving the template in 1C:
+
+```powershell
+python scripts/watch_1c_moxel_property_experiment.py `
+  --property ВертикальноеПоложение `
+  --target-text "Ячейка 7 - 2" `
+  --target-name R7C2_TEST `
+  --timeout-seconds 600 `
+  --run-pipeline-after
+```
+
+The watcher captures a `before` snapshot, waits for a new latest MOXCEL payload,
+captures `after`, writes Markdown/JSON snapshots to
+`reports/1c-template-probes`, and appends the experiment to
+`reports/1c-template-baselines/Primer3_moxel_property_experiments.manifest.json`.
+With `--run-pipeline-after`, it also refreshes schema discovery, property
+analysis, registry build/check/verify, and the next experiment plan.
+
+Prioritize:
+
+- `ГоризонтальноеПоложение`
+- `ВертикальноеПоложение`
+- `ЦветТекста`
+- `ЦветФона`
+- `Шрифт.Имя`
+- `Шрифт.Размер`
+- borders
+- `Защита`
+- `Гиперссылка`
+- wrapping
+- column width
+- row height
+- merge ranges
+
+Only promote a property path into write support after a disposable-base
+round-trip proves that changing that scalar affects only the intended property.
+
+## XML-Assisted Fixture Plan
+
+Use XML exports only as analysis fixtures. The adapter runtime stays SQL-only:
+it reads `Config`/`ConfigCAS` payloads, while exported `Ext/Template.xml`
+files are used to label and verify decoder hypotheses.
+
+Create a small extension with controlled templates and export it to XML after
+each controlled save:
+
+- one tabular document with sparse text cells, direct parameters, placeholders,
+  and empty formatted cells;
+- one tabular document with horizontal, vertical, and rectangular merged cells;
+- one tabular document with named areas and named ranges, including duplicate
+  names in different positions;
+- one tabular document dedicated to format changes: column width, row height,
+  horizontal/vertical alignment, border, font, text color, background color,
+  protection, wrapping;
+- one fixture per non-tabular template type where possible: text document,
+  binary data, HTML document, graphical/geographical schema, data composition
+  schema, data composition appearance template, external component.
+
+The concrete merge fixture checklist is stored in:
+
+- `reports/1c-template-baselines/moxel-controlled-merge-fixtures.json`
+- `reports/1c-template-baselines/moxel-controlled-merge-fixtures.md`
+
+Start with the merge fixtures `MOXEL_Merge_None_Grid`,
+`MOXEL_Merge_H_R5C18_W3`, `MOXEL_Merge_H_R6C2_W15`,
+`MOXEL_Merge_V_R5C2_H3`, `MOXEL_Merge_Rect_R5C2_R7C4`, and
+`MOXEL_Merge_Mixed_4Ranges`. They are designed to split column edges, row
+edges, width/height, and merge-record ordering without relying on the large
+production print forms.
+
+For MOXCEL discovery, change exactly one property per save, capture the SQL
+payload, then compare it with the exported XML shape. Promote read rules only
+when the SQL decoder can reproduce XML-visible facts from SQL bytes alone.
+
+Use `scripts/analyze_1c_template_xml_profiles.py` to build XML fixture
+profiles without feeding XML into the adapter runtime. Example:
+
+```powershell
+python scripts\analyze_1c_template_xml_profiles.py `
+  --root "Z:\codex\1C\XML\UPO\Структура базы 1с\Конфигурация\Documents\АвансовыйОтчет" `
+  --output-json reports\1c-template-baselines\xml-template-profiles-avansovy-otchet.json `
+  --output-markdown reports\1c-template-baselines\xml-template-profiles-avansovy-otchet.md
+```
+
+Current `АвансовыйОтчет` XML fixture facts:
+
+- `ПФ_MXL_АвансовыйОтчет`: capacity/used `75x26`, `961` cells,
+  `196` text values, `77` parameters, `108` merges, `233` distinct format
+  indexes.
+- `ПФ_MXL_АвансовыйОтчетВВалюте`: capacity/used `72x26`, `938` cells,
+  `188` text values, `75` parameters, `101` merges, `233` distinct format
+  indexes.
+
+The SQL decoder should eventually reproduce these facts from SQL payloads:
+`capacity_dimensions`/`used_dimensions`, coordinate-bound cell text and
+parameters, authoritative `merged_ranges`, and format indexes/styles. XML
+profiles are evidence for decoder hypotheses, not an input source for adapter
+answers.
+
+Compare the SQL-decoded baseline with the XML fixture profile after decoder
+changes:
+
+```powershell
+python scripts\profile_1c_tabular_templates.py `
+  --adapter-url http://docker-gpu.cin.su:8011 `
+  --base-id upo_test `
+  --inventory-json reports\1c-template-baselines\upo_test_configuration_tabular_templates.json `
+  --output-json reports\1c-template-baselines\upo_test_tabular_template_profiles.json `
+  --output-markdown reports\1c-template-baselines\upo_test_tabular_template_profiles.md
+```
+
+```powershell
+python scripts\compare_1c_template_sql_xml_profiles.py `
+  --sql-profile reports\1c-template-baselines\upo_test_tabular_template_profiles.json `
+  --xml-profile reports\1c-template-baselines\xml-template-profiles-avansovy-otchet.json `
+  --output-json reports\1c-template-baselines\sql-xml-template-profile-compare-avansovy-otchet.json `
+  --output-markdown reports\1c-template-baselines\sql-xml-template-profile-compare-avansovy-otchet.md
+```
+
+The current comparison intentionally reports gaps for both `АвансовыйОтчет`
+templates. SQL capacity still reports the MOXCEL allocation `128x72`, while XML
+spreadsheet dimensions are `75x26`/`72x26`. With `template_part_moxel_v8`,
+hint-aware SQL `used_dimensions` improved to `74x25` and `71x25`; the remaining
+edge likely depends on merge/format records. Limited cell/parameter counts,
+missing authoritative `merged_ranges`, and missing format index coverage remain
+open. Use these gaps as the next decoder scorecard; a gap should only disappear
+when SQL bytes alone reproduce the XML-visible fact.
+The compare report also shows `Progress signals`. A non-zero
+`cell_coordinate_hints_available` signal means the SQL decoder recovered
+coordinate evidence from hints, but the `cells_missing_or_limited` gap stays
+open until authoritative cell rows/columns are decoded.
+
+For merge-block row/size reverse engineering, regenerate the row-band report:
+
+```powershell
+python scripts\analyze_1c_moxel_merge_row_bands.py `
+  --template ПФ_MXL_АвансовыйОтчет `
+  --template ПФ_MXL_АвансовыйОтчетВВалюте `
+  --output-json reports\1c-template-baselines\moxel-merge-row-band-analysis-avansovy-otchet.json `
+  --output-markdown reports\1c-template-baselines\moxel-merge-row-band-analysis-avansovy-otchet.md
+```
+
+This report compares XML merge rows with SQL MOXCEL small-scalar bands and
+packed `scalar/32` column-edge evidence. Treat `row_or_size_hints` as low
+confidence until a controlled merge fixture proves which scalar positions are
+row indexes versus widths, heights, or flags.
+
+`merge_record_block_candidates[].evidence.record_analysis` exposes a normalized
+SQL-only view of the candidate block: per-record shape/head, packed
+`scalar/32` values, small scalars, scalar slot summaries, value-to-record runs,
+and sample records. Use it for slot-formula discovery; it is diagnostic
+evidence and does not make `merged_ranges` authoritative by itself.
+
+To score candidate numeric slots against XML-visible merge fields, run:
+
+```powershell
+python scripts\analyze_1c_moxel_merge_slot_candidates.py `
+  --template ПФ_MXL_АвансовыйОтчет `
+  --template ПФ_MXL_АвансовыйОтчетВВалюте `
+  --output-json reports\1c-template-baselines\moxel-merge-slot-candidates-avansovy-otchet.json `
+  --output-markdown reports\1c-template-baselines\moxel-merge-slot-candidates-avansovy-otchet.md
+```
+
+The current large-form slot report is intentionally hypothesis-only. It shows
+that simple value-set and ordered-offset matching are dominated by low-entropy
+height/flag values and do not prove a top/left/bottom/right formula. Use the
+controlled merge fixtures before promoting any SQL rule into authoritative
+`merged_ranges`.
+
+Inline text style candidates expose `coordinate_hints` for the column using the
+discovered rule
+`inline_text_column_from_last_preceding_scalar_plus_one`. Treat this as a
+high-confidence hint for analysis and controlled experiments, not as an
+authoritative cell coordinate until row and merge rules are proven by SQL/XML
+round trips.
+
+The adapter and SQL profile reports include `cell_style_coordinate_hints` in
+counts. Track this count separately from `cells`: it measures how much
+coordinate evidence was recovered from inline style records, while `cells`
+remains reserved for decoded row/cell runs with authoritative coordinates.
+
+`cell_coordinate_hints` combines the style-derived column hint with a matched
+decoded cell row when the text/cell id can be linked. Use it to inspect
+text-to-coordinate evidence during decoder discovery. Do not count it as
+authoritative `cells` coverage until the row rule and merge interactions are
+validated against SQL/XML fixtures.
+
+For compact inspection, request only coordinate evidence from `templates.map`
+or `templates.analyze` with `sections=coordinate_hints`. This returns
+`cell_coordinate_hints` and `cell_style_coordinate_hints` from both the
+structure and analysis layers without dumping all cells/styles.
+
+Coordinate hints, hint-aware `used_dimensions`, merge-record block candidates,
+SQL-only merge-block column-edge hints, low-confidence row/size scalar hints,
+and merge-block `record_analysis` changed the decoded artifact schema, so
+MOXCEL template cache uses `template_part_moxel_v9`. Refresh SQL baselines after
+deploying the new adapter; old `v2`/`v3`/`v4` decoded artifacts will not contain
+the current `cell_coordinate_hints`, merge-block evidence, and used-dimension
+semantics.