Initial SQL-only 1C adapter baseline

This commit is contained in:
2026-07-22 03:03:47 +03:00
commit e2503b77e7
545 changed files with 184711 additions and 0 deletions
+85
View File
@@ -0,0 +1,85 @@
# Secrets
.env
.env.*
!.env.example
*.key
*.pem
*.pfx
*.p12
*token*
*secret*
# Python
__pycache__/
.pytest_cache/
.ruff_cache/
.mypy_cache/
.venv/
venv/
# Runtime state
data/*.db
data/*.sqlite
data/*.sqlite3
onec-repository-locks.json
# Node
node_modules/
dist/
build/
# Local model and dataset artifacts
models/
artifacts/
checkpoints/
outputs/
runs/
wandb/
reports/
*.safetensors
*.gguf
*.bin
*.pt
*.pth
*.onnx
*.ckpt
*.arrow
*.parquet
# Large/private datasets are stored outside git.
datasets/raw/*
datasets/prepared/*
plugins/*/datasets/raw/*
plugins/*/datasets/prepared/*
plugins/1c/rag/sources/*
plugins/1c/rag/official-docs/raw/*
plugins/1c/rag/official-docs/normalized/*
plugins/1c/rag/official-docs/media/*
plugins/1c/rag/official-docs/static/*
plugins/1c/rag/official-docs/.local/*
plugins/1c/metadata/snapshots/*
plugins/1c/training/raw/*
plugins/1c/training/prepared/*
# Keep directory placeholders.
!datasets/raw/.gitkeep
!datasets/prepared/.gitkeep
!plugins/*/datasets/.gitkeep
!plugins/*/datasets/raw/.gitkeep
!plugins/*/datasets/prepared/.gitkeep
!plugins/*/datasets/README.md
!plugins/1c/rag/sources/.gitkeep
!plugins/1c/rag/official-docs/raw/.gitkeep
!plugins/1c/rag/official-docs/normalized/.gitkeep
!plugins/1c/rag/official-docs/media/.gitkeep
!plugins/1c/rag/official-docs/static/.gitkeep
!plugins/1c/rag/official-docs/.local/.gitkeep
!plugins/1c/metadata/snapshots/.gitkeep
!plugins/1c/training/raw/.gitkeep
!plugins/1c/training/prepared/.gitkeep
# OS/editor
.DS_Store
Thumbs.db
.idea/
.vscode/
+34
View File
@@ -0,0 +1,34 @@
## Shared Test Docker Host
- Use SSH alias `test-docker` / `docker-test` for the shared test Docker host.
- Host: `docker-test.cin.su` (`192.168.200.61`)
- SSH user: `test`
- Preferred Docker endpoint when Docker CLI is available: `ssh://test-docker`
- Portainer: `http://docker-test.cin.su:9000/`, user `admin`
- Do not store the password in repositories or project files; use an SSH key for persistent access.
## GPU Docker Host
- This project works with local LLM models and uses GPU resources.
- Use `upo_test` as the default 1C test database `base_id` for adapter checks in this project.
- Use `docker-gpu.cin.su` as the deployment target for GPU workloads.
- Prefer GPU-capable Docker deployments on `docker-gpu.cin.su` when running or serving local models.
- For training/download containers on `docker-gpu`, sync the current repo into `Z:\LLM\model-chat-app` first. These containers should read code from the synced app directory, not directly from `Z:\codex\LLM`.
- Do not store credentials, tokens, model secrets, or host passwords in repositories or project files.
## Test-system security profile
- This project currently runs as an isolated test system; use the minimum security profile unless the user explicitly requests production hardening.
- Network-level access control is sufficient for test web interfaces. `ONEC_ADAPTER_SERVICE_TOKEN` is optional when `ONEC_ADAPTER_ALLOW_UNAUTHENTICATED_ADMIN=true` is explicitly set.
- Do not block test deployment only because a service token is absent when the explicit unauthenticated-admin flag is enabled.
- Even in the minimum profile, never commit credentials or passwords, never return stored passwords through APIs, and keep runtime credential files outside git.
## Architecture
- Use the `core + plugins` architecture.
- Put shared platform capabilities in `core`: model registry, inference, storage, training, evals, deployment, and monitoring.
- Put task-specific logic in `plugins`: text, translation, audio, video, and 1C.
- Design every plugin as a future standalone service: keep its API, pipelines, datasets, evals, and configuration inside the plugin folder.
- Keep model binaries and large datasets out of git. Store only manifests, model cards, metadata, scripts, and reproducible deployment configuration.
- For 1C, start with RAG and tool integrations before fine-tuning. Use LoRA or adapter-based fine-tuning when enough curated examples are collected.
- For 1C adapter work, treat object names as the primary selector. When solving tasks, start from names or public refs such as `РегистрСведений.Имя` or `InformationRegister.Name`; if the implementation needs GUIDs, SQL numbers, or internal codes, resolve them internally from the provided names instead of requiring callers to know storage identifiers.
+245
View File
@@ -0,0 +1,245 @@
# Local LLM Platform
Локальная платформа для работы с моделями под разные задачи: текст, перевод, аудио, видео и 1С.
Архитектура проекта: `core + plugins`.
- `core` содержит общую платформу: реестр моделей, инференс, обучение, eval-тесты, деплой на GPU, хранилище и мониторинг.
- `plugins` содержит прикладные направления: текст, перевод, аудио, видео, фото, 1С.
- Каждый плагин проектируется как будущий отдельный сервис, но на старте живет в одном репозитории.
## Deployment Target
GPU-нагрузки разворачиваются на `docker-gpu.cin.su`.
Секреты, токены, пароли, ключи доступа и приватные датасеты не хранятся в репозитории.
Если `docker-gpu.cin.su` является Windows-хостом с контейнерами на диске `Z:`, используйте runbook `docs/runbooks/windows-docker-gpu-host.md`.
## Structure
```text
LLM/
core/
registry/
inference/
training/
evals/
deploy/
storage/
monitoring/
registry/
model-cards/
templates/
datasets/
raw/
prepared/
plugins/
text/
translation/
audio/
video/
image/
1c/
docs/
```
## Development Stages
1. Описать реестр моделей и формат model card.
2. Поднять базовый inference API для текстовых моделей.
3. Добавить плагины задач и их минимальные пайплайны.
4. Для 1С сначала сделать RAG и инструменты работы с метаданными.
5. После накопления качественных примеров добавить LoRA/adapter fine-tuning для 1С.
6. Тяжелые плагины постепенно выносить в отдельные контейнеры или сервисы.
## First Inference
Шаблон первого GPU inference-сервиса находится в `core/deploy/docker-gpu/vllm`.
Runbook: `docs/runbooks/first-vllm-inference.md`.
Deployment runbook: `docs/runbooks/deploy-vllm.md`.
GPU preflight: `docs/runbooks/gpu-host-preflight.md`.
Полный GPU-контур после настройки SSH-доступа:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_gpu_stack.ps1 -Pull
```
Посмотреть план без запуска:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_gpu_stack.ps1 -PlanOnly -Pull
```
Первая базовая текстовая модель: `Qwen/Qwen3-4B-Instruct-2507`.
Первая GGUF-модель для 1С/code экспериментов: `bartowski/mistralai_Devstral-Small-2-24B-Instruct-2512-GGUF`, quant `Q4_K_M`.
Исследование следующих кандидатов под 1С: `docs/research/1c-model-candidates.md`.
Набор моделей по одному кандидату на каждый плагин: `plugins/model-bundle.yaml`.
Загрузка набора:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/download_plugin_model_bundle.ps1
```
Проверка model cards:
```powershell
python scripts/validate_model_cards.py
```
Все базовые локальные проверки:
```powershell
python scripts/check_all.py
```
Проверка локального хранилища моделей:
```powershell
python scripts/check_model_storage.py --print
```
Сводный статус платформы:
```powershell
python scripts/collect_platform_status.py --print
```
Список моделей:
```powershell
python scripts/list_model_cards.py
```
Загрузка модели в локальное хранилище:
```powershell
python scripts/download_hf_model.py qwen3-4b-instruct-2507 --dry-run
```
Runbook: `docs/runbooks/download-model.md`.
Чат для ручной проверки моделей:
```powershell
python scripts/model_chat_server.py
```
Runbook: `docs/runbooks/model-chat-testbench.md`.
## 1C RAG
Стартовый контур 1С находится в `plugins/1c`.
Подготовка RAG-корпуса:
```powershell
python scripts/build_1c_knowledge_base.py
```
Отдельные шаги, если нужно управлять ими вручную:
```powershell
python scripts/prepare_1c_rag_corpus.py
```
Построение локального lexical index:
```powershell
python scripts/build_1c_rag_index.py
```
Построение локального SQLite vector index:
```powershell
python scripts/build_1c_rag_vector_index.py
python scripts/search_1c_rag_hybrid.py "реквизиты справочника номенклатура" --profile metadata --limit 5
```
Сборка 1С RAG prompt:
```powershell
python scripts/ask_1c_rag.py "Какие реквизиты есть у справочника Номенклатура?" --print-prompt
```
Runbook: `docs/runbooks/1c-rag.md`.
Проверка 1С-плагина:
```powershell
python scripts/check_1c_plugin.py --print
```
Контур безопасного взаимодействия с живыми базами 1С: `docs/runbooks/1c-live-interaction.md`.
Проверка offline/static контура верификации 1С-адаптера:
```powershell
python scripts/check_1c_adapter_verification_stack.py --base-id upo_test
```
Для нестандартных endpoint'ов добавьте `--rest-adapter-url ... --mcp-url ...`;
по умолчанию проверяются текущие стендовые URL. Несколько баз можно передать
после одного `--base-id`. Чтобы принимать только свежие persisted-отчеты,
добавьте `--max-report-age-seconds <seconds>`.
Persisted validation also checks
`reports/1c-sql/<base-id>/saved-state-copy-plan.json`: the plan must be for
the requested base, be `plan_ready`, contain active `Config`/`ConfigCAS` source
rows, and show no `ConfigSave`/`ConfigCASSave` target collisions.
PowerShell deploy/verify defaults saved-state preparation to `ConfigCASSave`;
pass `-SavedStateTable ConfigSave` when checking the base save layer instead.
Сохранить машинно-читаемый итог этой проверки:
```powershell
python scripts/check_1c_adapter_verification_stack.py --base-id upo_test --json --report reports/1c-sql/upo_test/adapter-verification-stack-check.json
```
Metadata snapshots:
```powershell
python scripts/validate_1c_metadata_snapshot.py plugins/1c/metadata/examples/metadata.example.json
python scripts/convert_1c_metadata_to_rag.py --input plugins/1c/metadata/examples/metadata.example.json --output plugins/1c/rag/sources/metadata.example.generated.md
```
Runbook: `docs/runbooks/1c-metadata-snapshot.md`.
## 1C Training Data
Проверка и подготовка синтетического training example:
```powershell
python scripts/validate_1c_training_data.py plugins/1c/training/examples/instruction.examples.jsonl
python scripts/prepare_1c_training_data.py --input plugins/1c/training/examples/instruction.examples.jsonl --output plugins/1c/training/prepared/train.chat.jsonl
```
Runbook: `docs/runbooks/1c-training-data.md`.
LoRA training:
```powershell
python scripts/preflight_1c_training.py
```
Runbook: `docs/runbooks/1c-lora-training.md`.
## Evals
Проверка eval-наборов:
```powershell
python scripts/validate_evals.py
python scripts/run_1c_smoke_eval.py --print
```
Runbook: `docs/runbooks/evals.md`.
@@ -0,0 +1,12 @@
{
"schema": "onec_extension_runner_config.v1",
"runner_id": "manual-disposable-1c-validation",
"runner_kind": "manual",
"platform_version": "8.3.x",
"disposable_base_ref": "test-disposable-upo-copy",
"disposable_base_kind": "file-or-server-copy",
"disposable_base_confirmed": true,
"validation_mode": "manual",
"evidence_root": "reports/1c-sql/upo/extension-validation-evidence",
"notes": "Do not put users, passwords, tokens, or production base references in this file."
}
+19
View File
@@ -0,0 +1,19 @@
{
"upo_test": {
"server": "sql-host",
"database": "infobase-database",
"user": "readonly-sql-user",
"password_env": "ONEC_SQL_PASSWORD_UPO_TEST",
"repository": {
"backend": "karman_bridge",
"runtime_version": "8.3.x",
"bridge_id": "configured-bridge-id",
"layer": "base",
"runner": {
"kind": "http",
"url": "http://host.docker.internal:8121",
"token_env": "ONEC_REPOSITORY_RUNNER_TOKEN"
}
}
}
}
@@ -0,0 +1,22 @@
{
"upo_test": {
"repository": {
"backend": "karman_bridge",
"runner": {
"kind": "local"
},
"designer_path": "C:\\Program Files\\1cv8\\8.3.x.x\\bin\\1cv8.exe",
"runtime_version": "8.3.x",
"endpoint": "tcp://repository-relay.example:15420/repository-name",
"bridge_id": "configured-bridge-id",
"layer": "base",
"infobase": {
"server": "onec-server/infobase"
},
"infobase_user": "designer-user",
"infobase_password_env": "ONEC_INFOBASE_PASSWORD_UPO_TEST",
"repository_user": "repository-user",
"repository_password_env": "ONEC_REPOSITORY_PASSWORD_UPO_TEST"
}
}
}
+83
View File
@@ -0,0 +1,83 @@
{
"default": {
"id": "default",
"label": "Default text/audio",
"command": "powershell -ExecutionPolicy Bypass -File scripts\\switch_gpu_profile.ps1 -Profile default",
"starts": ["vllm-text", "translation-api", "audio-api", "model-chat-ui"],
"stops": ["video-api", "image-api", "llama-gguf"],
"wait": [
{"name": "vLLM", "url": "http://192.168.220.91:8000/v1/models"},
{"name": "translation", "url": "http://192.168.220.91:8010/health"},
{"name": "audio", "url": "http://192.168.220.91:8020/health"},
{"name": "UI", "url": "http://192.168.220.91:8765/api/health"}
],
"notes": "Текст, 1С, перевод и аудио; video/image выключены, чтобы не занимать VRAM."
},
"text": {
"id": "text",
"label": "Text and 1C",
"command": "powershell -ExecutionPolicy Bypass -File scripts\\switch_gpu_profile.ps1 -Profile text",
"starts": ["vllm-text", "translation-api", "audio-api", "model-chat-ui"],
"stops": ["video-api", "image-api", "llama-gguf"],
"wait": [
{"name": "vLLM", "url": "http://192.168.220.91:8000/v1/models"},
{"name": "translation", "url": "http://192.168.220.91:8010/health"},
{"name": "audio", "url": "http://192.168.220.91:8020/health"},
{"name": "UI", "url": "http://192.168.220.91:8765/api/health"}
],
"notes": "Основной режим для текстовых чатов, RAG 1С и переводов."
},
"audio": {
"id": "audio",
"label": "Audio checks",
"command": "powershell -ExecutionPolicy Bypass -File scripts\\switch_gpu_profile.ps1 -Profile audio",
"starts": ["audio-api", "translation-api", "vllm-text", "model-chat-ui"],
"stops": ["video-api", "image-api", "llama-gguf"],
"wait": [
{"name": "audio", "url": "http://192.168.220.91:8020/health"},
{"name": "translation", "url": "http://192.168.220.91:8010/health"},
{"name": "vLLM", "url": "http://192.168.220.91:8000/v1/models"},
{"name": "UI", "url": "http://192.168.220.91:8765/api/health"}
],
"notes": "Распознавание речи плюс текстовый endpoint; video выключен."
},
"video": {
"id": "video",
"label": "Vision/video",
"command": "powershell -ExecutionPolicy Bypass -File scripts\\switch_gpu_profile.ps1 -Profile video",
"starts": ["translation-api", "video-api", "model-chat-ui"],
"stops": ["vllm-text", "audio-api", "image-api", "llama-gguf"],
"wait": [
{"name": "translation", "url": "http://192.168.220.91:8010/health"},
{"name": "video", "url": "http://192.168.220.91:8030/health"},
{"name": "UI", "url": "http://192.168.220.91:8765/api/health"}
],
"notes": "Освобождает VRAM под Qwen2.5-VL; первый анализ изображения может грузиться несколько минут."
},
"image": {
"id": "image",
"label": "Image generation",
"command": "powershell -ExecutionPolicy Bypass -File scripts\\switch_gpu_profile.ps1 -Profile image",
"starts": ["translation-api", "image-api", "model-chat-ui"],
"stops": ["vllm-text", "audio-api", "video-api", "llama-gguf"],
"wait": [
{"name": "translation", "url": "http://192.168.220.91:8010/health"},
{"name": "image", "url": "http://192.168.220.91:8040/health"},
{"name": "UI", "url": "http://192.168.220.91:8765/api/health"}
],
"notes": "Освобождает VRAM под SDXL generation/inpainting; первая генерация загружает diffusers pipeline."
},
"gguf-1c": {
"id": "gguf-1c",
"label": "GGUF 1C",
"command": "powershell -ExecutionPolicy Bypass -File scripts\\switch_gpu_profile.ps1 -Profile gguf-1c",
"starts": ["llama-gguf", "translation-api", "model-chat-ui"],
"stops": ["vllm-text", "audio-api", "video-api", "image-api"],
"wait": [
{"name": "llama.cpp", "url": "http://192.168.220.91:8080/v1/models"},
{"name": "translation", "url": "http://192.168.220.91:8010/health"},
{"name": "UI", "url": "http://192.168.220.91:8765/api/health"}
],
"notes": "Режим для llama.cpp / GGUF 1С после появления готового адаптера или GGUF-сборки."
}
}
+66
View File
@@ -0,0 +1,66 @@
{
"default": "gpu-fast",
"profiles": {
"gpu-fast": {
"id": "gpu-fast",
"label": "GPU fast",
"host": "docker-gpu.cin.su",
"docker_endpoint": "ssh://docker-gpu",
"role": "interactive",
"notes": "Main interactive RTX 4090 host for chat, 1C, code, image, audio and video tasks.",
"endpoints": {
"text": "http://docker-gpu.cin.su:8000",
"1c": "http://docker-gpu.cin.su:8081",
"translation": "http://docker-gpu.cin.su:8010",
"audio": "http://docker-gpu.cin.su:8020",
"video": "http://docker-gpu.cin.su:8030",
"image": "http://docker-gpu.cin.su:8040"
},
"model_overrides": {
"qwen3-coder-30b-a3b-instruct-q6_k": {
"base_url": "http://docker-gpu.cin.su:8081",
"served_model_name": "qwen3-coder-1c-q6",
"container_name": "llm-llama-qwen3-coder-q6-test",
"start_hint": "powershell -ExecutionPolicy Bypass -File scripts\\manage_gpu_q6_service.ps1 -Action start"
},
"qwen3-coder-30b-a3b-instruct-q4_k_m": {
"base_url": "http://docker-gpu.cin.su:8080",
"served_model_name": "qwen3-coder-1c-q4"
},
"devstral-small-2-24b-instruct-2512-q4_k_m": {
"base_url": "http://docker-gpu.cin.su:8080",
"served_model_name": "devstral-1c-q4"
}
}
},
"cpu-test": {
"id": "cpu-test",
"label": "CPU test",
"host": "docker-test.cin.su",
"docker_endpoint": "ssh://test-docker",
"role": "benchmark",
"notes": "Shared CPU Docker host for benchmark and fallback runs. Start heavyweight models manually before selecting this profile.",
"endpoints": {
"text": "http://docker-test.cin.su:18086",
"1c": "http://docker-test.cin.su:18086"
},
"model_overrides": {
"qwen3-coder-30b-a3b-instruct-q6_k": {
"base_url": "http://docker-test.cin.su:18086",
"served_model_name": "qwen3-coder-1c-q6-cpu",
"container_name": "llm-qwen3-coder-q6-cpu-test",
"start_hint": "powershell -ExecutionPolicy Bypass -File scripts\\manage_cpu_q6_service.ps1 -Action start"
}
}
},
"background": {
"id": "background",
"label": "Background",
"host": "docker-test.cin.su",
"docker_endpoint": "ssh://test-docker",
"role": "batch",
"notes": "Use for model downloads, RAG indexing, conversions and other long CPU jobs. Not selected automatically for chat.",
"endpoints": {}
}
}
}
+48
View File
@@ -0,0 +1,48 @@
{
"schema": "onec_access_critical_roles.v1",
"base_id": "upo_test",
"roles": [
{
"role": "администратор",
"user_threshold": 5,
"severity": "high",
"reason": "Полные административные права должны быть у минимального числа пользователей."
},
{
"role": "запись изменение номенклатура поставщиков",
"user_threshold": 30,
"severity": "medium",
"reason": "Изменение поставщиков номенклатуры влияет на закупки и НСИ."
},
{
"role": "добавление изменение номенклатуры",
"user_threshold": 30,
"severity": "medium",
"reason": "Изменение номенклатуры влияет на документы, цены и учет."
},
{
"role": "добавление изменение контрагентов",
"user_threshold": 30,
"severity": "medium",
"reason": "Изменение контрагентов влияет на договоры, расчеты и первичные документы."
},
{
"role": "изменение цен",
"user_threshold": 20,
"severity": "high",
"reason": "Изменение цен напрямую влияет на продажи, закупки и маржинальность."
},
{
"role": "банк касса",
"user_threshold": 15,
"severity": "high",
"reason": "Доступ к банковским и кассовым операциям требует отдельного контроля."
},
{
"role": "проведение документов",
"user_threshold": 50,
"severity": "medium",
"reason": "Проведение документов меняет учетные движения."
}
]
}
+15
View File
@@ -0,0 +1,15 @@
# Core
Общее ядро LLM-платформы.
`core` отвечает за переиспользуемые возможности:
- реестр моделей;
- инференс;
- обучение и дообучение;
- eval-тесты;
- GPU deployment;
- хранение;
- мониторинг.
Прикладная логика задач должна находиться в `plugins`.
+1
View File
@@ -0,0 +1 @@
"""Shared core platform modules."""
+45
View File
@@ -0,0 +1,45 @@
# Core Deploy
Общие правила развертывания.
GPU workload target: `docker-gpu.cin.su`.
Infrastructure/lightweight proxy target: `docker.cin.su`.
В репозитории храним только воспроизводимую конфигурацию и документацию. Секреты, токены и пароли не коммитим.
Локальные модели и RAG-артефакты проверяются отдельно перед переносом или
Docker-запуском: `docs/runbooks/artifact-portability.md`.
Локальная консоль управления запускается командой:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/run_management_console.ps1
```
Runbook: `docs/runbooks/management-console.md`.
1C MCP proxy:
```text
docs/runbooks/adapter-1c-mcp.md
core/deploy/docker/adapter-1c-mcp/compose.yaml
```
1C REST adapter on GPU host:
```text
core/deploy/docker-gpu/adapter-1c/compose.yaml
```
1C agent service (подпроект):
```text
core/deploy/docker/1c-agent/compose.yaml
docs/runbooks/1c-agent.md
```
The current container is read-first and route-index backed. It serves
`http://docker-gpu.cin.su:8011`, keeps the route index in the
`adapter-1c_adapter-1c-data` Docker volume, and is used by
`adapter-1c-mcp` through `ONEC_ADAPTER_URL`.
+7
View File
@@ -0,0 +1,7 @@
# Docker GPU Deployment
Целевой хост для GPU-развертываний: `docker-gpu.cin.su`.
Здесь будут находиться compose-файлы, env-шаблоны и инструкции для запуска GPU-сервисов.
Секреты должны передаваться через окружение, секрет-хранилище или настройки хоста, но не через git.
@@ -0,0 +1,11 @@
ADAPTER_1C_IMAGE=adapter-1c-rest:latest
ADAPTER_1C_CONTAINER_NAME=adapter-1c-rest
ADAPTER_1C_HOST_PORT=8011
# Live SQL connections. Keep real credentials outside git.
# Example:
# ONEC_SQL_BASES_JSON={"upo_test":{"server":"sql-host","database":"upo_test","user":"configured_login","password_env":"ONEC_SQL_PASSWORD_UPO_TEST"}}
# ONEC_SQL_PASSWORD_UPO_TEST=put-this-only-in-a-real-non-committed-env-file
ONEC_SQL_BASES_JSON=
# Optional path inside the container to a JSON file with the same shape as ONEC_SQL_BASES_JSON.
ONEC_SQL_BASES_JSON_FILE=/data/onec-sql-bases.json
@@ -0,0 +1,32 @@
name: adapter-1c
services:
adapter-1c-rest:
build:
context: ../../../../plugins/1c
dockerfile: connector/Dockerfile
image: ${ADAPTER_1C_IMAGE:-adapter-1c-rest:latest}
container_name: ${ADAPTER_1C_CONTAINER_NAME:-adapter-1c-rest}
restart: unless-stopped
ports:
- "${ADAPTER_1C_HOST_PORT:-8011}:8011"
volumes:
- adapter-1c-data:/data
environment:
ONEC_ADAPTER_HOST: 0.0.0.0
ONEC_ADAPTER_PORT: 8011
ONEC_ADAPTER_SERVICE_TOKEN: ${ONEC_ADAPTER_SERVICE_TOKEN:-}
ONEC_ADAPTER_ALLOW_UNAUTHENTICATED_ADMIN: ${ONEC_ADAPTER_ALLOW_UNAUTHENTICATED_ADMIN:-true}
ONEC_SQL_BASES_JSON: ${ONEC_SQL_BASES_JSON:-}
ONEC_SQL_BASES_JSON_FILE: ${ONEC_SQL_BASES_JSON_FILE:-/data/onec-sql-bases.json}
ONEC_INFOBASE_USER_ADMIN_BASES_JSON: ${ONEC_INFOBASE_USER_ADMIN_BASES_JSON:-}
ONEC_INFOBASE_USER_ADMIN_BASES_JSON_FILE: ${ONEC_INFOBASE_USER_ADMIN_BASES_JSON_FILE:-/data/onec-infobase-user-admin.json}
ONEC_INFOBASE_USER_ADMIN_TOKEN_UPO_TEST: ${ONEC_INFOBASE_USER_ADMIN_TOKEN_UPO_TEST:-}
ONEC_INFOBASE_USER_ADMIN_ALLOW_UNAUTHENTICATED: ${ONEC_INFOBASE_USER_ADMIN_ALLOW_UNAUTHENTICATED:-false}
ONEC_REPOSITORY_RUNNER_TOKEN: ${ONEC_REPOSITORY_RUNNER_TOKEN:-}
ONEC_ADAPTER_ENABLE_EXTERNAL_1C: ${ONEC_ADAPTER_ENABLE_EXTERNAL_1C:-false}
ONEC_REPOSITORY_REQUEST_TTL_SECONDS: ${ONEC_REPOSITORY_REQUEST_TTL_SECONDS:-86400}
ONEC_REPOSITORY_CONFIRMATION_TTL_SECONDS: ${ONEC_REPOSITORY_CONFIRMATION_TTL_SECONDS:-7200}
volumes:
adapter-1c-data:
@@ -0,0 +1,17 @@
# Copy to .env on the deployment host and adjust values there.
# Do not commit real tokens, private paths, or credentials.
LLAMA_CONTAINER_NAME=llm-llama-devstral-1c
LLAMA_IMAGE=ghcr.io/ggml-org/llama.cpp:server-cuda
LLAMA_HOST_PORT=8080
LLAMA_MODEL_PATH=/models/gguf/1c/devstral-small-2-24b-instruct-2512-q4_k_m/mistralai_Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf
LLAMA_SERVED_MODEL_NAME=devstral-1c-q4
LLAMA_CTX_SIZE=32768
LLAMA_GPU_LAYERS=999
LLAMA_THREADS=8
LLAMA_PARALLEL=1
LLAMA_FIT=off
LLAMA_REASONING=off
LLAMA_REASONING_FORMAT=none
LLAMA_CHAT_TEMPLATE=mistral-v7-tekken
HOST_MODELS_DIR=/models
@@ -0,0 +1,43 @@
services:
llama-cpp:
image: ${LLAMA_IMAGE:-ghcr.io/ggml-org/llama.cpp:server-cuda}
container_name: ${LLAMA_CONTAINER_NAME:-llm-llama-devstral-1c}
restart: unless-stopped
ports:
- "${LLAMA_HOST_PORT:-8080}:8080"
volumes:
- ${HOST_MODELS_DIR:-/models}:/models
command:
- --host
- 0.0.0.0
- --port
- "8080"
- --model
- ${LLAMA_MODEL_PATH}
- --alias
- ${LLAMA_SERVED_MODEL_NAME:-devstral-1c-q4}
- --ctx-size
- ${LLAMA_CTX_SIZE:-32768}
- --n-gpu-layers
- ${LLAMA_GPU_LAYERS:-999}
- --threads
- ${LLAMA_THREADS:-8}
- --parallel
- ${LLAMA_PARALLEL:-1}
- --fit
- ${LLAMA_FIT:-off}
- --reasoning
- ${LLAMA_REASONING:-off}
- --reasoning-format
- ${LLAMA_REASONING_FORMAT:-none}
- --chat-template
- ${LLAMA_CHAT_TEMPLATE:-mistral-v7-tekken}
- --skip-chat-parsing
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities:
- gpu
@@ -0,0 +1,19 @@
# Qwen3-Coder Q6 GPU llama.cpp settings for docker-gpu.cin.su.
# This model is too large to keep online together with SDXL/vLLM on a 24 GiB RTX 4090.
LLAMA_CONTAINER_NAME=llm-llama-qwen3-coder-q6-test
LLAMA_IMAGE=ghcr.io/ggml-org/llama.cpp:server-cuda
LLAMA_HOST_PORT=8081
LLAMA_MODEL_PATH=/models/gguf/1c/qwen3-coder-30b-a3b-instruct-q6_k/Qwen3-Coder-30B-A3B-Instruct-Q6_K.gguf
LLAMA_SERVED_MODEL_NAME=qwen3-coder-1c-q6
LLAMA_CTX_SIZE=8192
LLAMA_GPU_LAYERS=auto
LLAMA_THREADS=12
LLAMA_PARALLEL=1
LLAMA_FIT=on
LLAMA_REASONING=auto
LLAMA_REASONING_FORMAT=none
LLAMA_CHAT_TEMPLATE=chatml
# Optional future LoRA path after converting the trained adapter to GGUF for llama.cpp:
# LLAMA_LORA_PATH=/models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1.gguf
HOST_MODELS_DIR=Z:/LLM/models
@@ -0,0 +1,24 @@
services:
model-chat-ui:
image: ${MODEL_CHAT_IMAGE:-vllm/vllm-openai:v0.10.2}
container_name: ${MODEL_CHAT_CONTAINER_NAME:-llm-model-chat-ui}
restart: unless-stopped
ports:
- "${MODEL_CHAT_HOST_PORT:-8765}:8765"
volumes:
- ${HOST_APP_DIR:-Z:/LLM/model-chat-app}:/app
- ${HOST_MODELS_DIR:-Z:/LLM/models}:/models:ro
- ${HOST_REPORTS_DIR:-Z:/LLM/reports}:/reports
environment:
MODEL_CHAT_REPORT_ROOT: /reports
working_dir: /app
entrypoint:
- python3
command:
- scripts/model_chat_server.py
- --host
- 0.0.0.0
- --port
- "8765"
- --static-dir
- /app
@@ -0,0 +1,25 @@
services:
train-1c-lora:
image: ${TRAINING_IMAGE:-pytorch/pytorch:2.5.1-cuda12.4-cudnn9-runtime}
container_name: llm-train-1c-lora
working_dir: /workspace
shm_size: 16gb
volumes:
- ${HOST_WORKSPACE_DIR:-/workspace/LLM}:/workspace
- ${HOST_MODELS_DIR:-/models}:/models
command:
- bash
- -lc
- |
pip install -r requirements-training.txt &&
python scripts/preflight_1c_training.py --config ${TRAINING_CONFIG:-/workspace/plugins/1c/training/configs/qwen3-coder-30b-a3b-lora.yaml} &&
if [ "${PREFLIGHT_ONLY:-0}" = "1" ]; then exit 0; fi &&
python scripts/train_1c_lora.py --config ${TRAINING_CONFIG:-/workspace/plugins/1c/training/configs/qwen3-coder-30b-a3b-lora.yaml}
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities:
- gpu
@@ -0,0 +1,4 @@
TRAINING_IMAGE=pytorch/pytorch:2.5.1-cuda12.4-cudnn9-runtime
HOST_WORKSPACE_DIR=Z:/LLM/model-chat-app
HOST_MODELS_DIR=Z:/LLM/models
TRAINING_CONFIG=/workspace/plugins/1c/training/configs/qwen3-coder-30b-a3b-lora.yaml
@@ -0,0 +1,38 @@
name: llm-transformers-audio
services:
audio-api:
image: ${TRANSFORMERS_IMAGE:-vllm/vllm-openai:v0.10.2}
container_name: ${TRANSFORMERS_CONTAINER_NAME:-llm-transformers-audio}
restart: unless-stopped
ipc: host
ports:
- "${TRANSFORMERS_HOST_PORT:-8020}:8020"
environment:
HF_HOME: /root/.cache/huggingface
MODEL_PATH: ${MODEL_PATH:-/models/audio/whisper-large-v3-turbo}
SERVED_MODEL_NAME: ${SERVED_MODEL_NAME:-whisper-large-v3-turbo}
PORT: "8020"
LOAD_ON_START: ${LOAD_ON_START:-0}
volumes:
- ${HOST_APP_DIR:-Z:/LLM/model-chat-app}:/app
- ${HOST_MODELS_DIR:-Z:/LLM/models}:/models
- ${HOST_HF_CACHE_DIR:-Z:/LLM/models/cache/huggingface}:/root/.cache/huggingface
working_dir: /app
entrypoint: python3
command:
- scripts/transformers_plugin_server.py
- --plugin
- audio
- --host
- 0.0.0.0
- --port
- "8020"
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities:
- gpu
@@ -0,0 +1,9 @@
HOST_APP_DIR=Z:/LLM/model-chat-app
HOST_MODELS_DIR=Z:/LLM/models
HOST_HF_CACHE_DIR=Z:/LLM/models/cache/huggingface
TRANSFORMERS_IMAGE=vllm/vllm-openai:v0.10.2
TRANSFORMERS_CONTAINER_NAME=llm-transformers-audio
TRANSFORMERS_HOST_PORT=8020
MODEL_PATH=/models/audio/whisper-large-v3-turbo
SERVED_MODEL_NAME=whisper-large-v3-turbo
LOAD_ON_START=0
@@ -0,0 +1,10 @@
FROM vllm/vllm-openai:v0.10.2
RUN python3 -m pip install --no-cache-dir \
"diffusers>=0.35.0" \
"transformers>=4.51.0" \
"accelerate>=1.0.0" \
safetensors \
pillow
WORKDIR /app
@@ -0,0 +1,42 @@
name: llm-transformers-image
services:
image-api:
image: ${TRANSFORMERS_IMAGE:-llm-transformers-image:latest}
build:
context: ../../../../
dockerfile: core/deploy/docker-gpu/transformers/image.Dockerfile
container_name: ${TRANSFORMERS_CONTAINER_NAME:-llm-transformers-image}
restart: unless-stopped
ipc: host
ports:
- "${TRANSFORMERS_HOST_PORT:-8040}:8040"
environment:
HF_HOME: /root/.cache/huggingface
MODEL_PATH: ${MODEL_PATH:-/models/image/sdxl-base-1.0}
EDIT_MODEL_PATH: ${EDIT_MODEL_PATH:-/models/image/sdxl-inpainting-1.0}
SERVED_MODEL_NAME: ${SERVED_MODEL_NAME:-sdxl-image}
PORT: "8040"
LOAD_ON_START: ${LOAD_ON_START:-1}
BACKGROUND_LOAD_ON_START: ${BACKGROUND_LOAD_ON_START:-1}
volumes:
- ${HOST_APP_DIR:-Z:/LLM/model-chat-app}:/app
- ${HOST_MODELS_DIR:-Z:/LLM/models}:/models
- ${HOST_HF_CACHE_DIR:-Z:/LLM/models/cache/huggingface}:/root/.cache/huggingface
working_dir: /app
entrypoint: /bin/sh
command:
- -lc
- >
exec python3 scripts/transformers_plugin_server.py
--plugin image
--host 0.0.0.0
--port 8040
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities:
- gpu
@@ -0,0 +1,11 @@
HOST_APP_DIR=Z:/LLM/model-chat-app
HOST_MODELS_DIR=Z:/LLM/models
HOST_HF_CACHE_DIR=Z:/LLM/models/cache/huggingface
TRANSFORMERS_IMAGE=llm-transformers-image:latest
TRANSFORMERS_CONTAINER_NAME=llm-transformers-image
TRANSFORMERS_HOST_PORT=8040
MODEL_PATH=/models/image/sdxl-base-1.0
EDIT_MODEL_PATH=/models/image/sdxl-inpainting-1.0
SERVED_MODEL_NAME=sdxl-image
LOAD_ON_START=1
BACKGROUND_LOAD_ON_START=1
@@ -0,0 +1,11 @@
HOST_APP_DIR=Z:/LLM/model-chat-app
HOST_MODELS_DIR=Z:/LLM/models
HOST_HF_CACHE_DIR=Z:/LLM/models/cache/huggingface
TRANSFORMERS_IMAGE=llm-transformers-image:latest
TRANSFORMERS_CONTAINER_NAME=llm-transformers-image
TRANSFORMERS_HOST_PORT=8040
MODEL_PATH=/models/image/qwen-image-edit
EDIT_MODEL_PATH=/models/image/qwen-image-edit
SERVED_MODEL_NAME=qwen-image-edit
LOAD_ON_START=0
BACKGROUND_LOAD_ON_START=0
@@ -0,0 +1,38 @@
name: llm-transformers-translation
services:
translation-api:
image: ${TRANSFORMERS_IMAGE:-vllm/vllm-openai:v0.10.2}
container_name: ${TRANSFORMERS_CONTAINER_NAME:-llm-transformers-translation}
restart: unless-stopped
ipc: host
ports:
- "${TRANSFORMERS_HOST_PORT:-8010}:8010"
environment:
HF_HOME: /root/.cache/huggingface
MODEL_PATH: ${MODEL_PATH:-/models/translation/lmt-60-4b}
SERVED_MODEL_NAME: ${SERVED_MODEL_NAME:-lmt-60-4b}
PORT: "8010"
LOAD_ON_START: ${LOAD_ON_START:-0}
volumes:
- ${HOST_APP_DIR:-Z:/LLM/model-chat-app}:/app
- ${HOST_MODELS_DIR:-Z:/LLM/models}:/models
- ${HOST_HF_CACHE_DIR:-Z:/LLM/models/cache/huggingface}:/root/.cache/huggingface
working_dir: /app
entrypoint: python3
command:
- scripts/transformers_plugin_server.py
- --plugin
- translation
- --host
- 0.0.0.0
- --port
- "8010"
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities:
- gpu
@@ -0,0 +1,9 @@
HOST_APP_DIR=Z:/LLM/model-chat-app
HOST_MODELS_DIR=Z:/LLM/models
HOST_HF_CACHE_DIR=Z:/LLM/models/cache/huggingface
TRANSFORMERS_IMAGE=vllm/vllm-openai:v0.10.2
TRANSFORMERS_CONTAINER_NAME=llm-transformers-translation
TRANSFORMERS_HOST_PORT=8010
MODEL_PATH=/models/translation/lmt-60-4b
SERVED_MODEL_NAME=lmt-60-4b
LOAD_ON_START=0
@@ -0,0 +1,38 @@
name: llm-transformers-video
services:
video-api:
image: ${TRANSFORMERS_IMAGE:-vllm/vllm-openai:v0.10.2}
container_name: ${TRANSFORMERS_CONTAINER_NAME:-llm-transformers-video}
restart: unless-stopped
ipc: host
ports:
- "${TRANSFORMERS_HOST_PORT:-8030}:8030"
environment:
HF_HOME: /root/.cache/huggingface
MODEL_PATH: ${MODEL_PATH:-/models/video/qwen2.5-vl-7b-instruct}
SERVED_MODEL_NAME: ${SERVED_MODEL_NAME:-qwen2.5-vl-7b-instruct}
PORT: "8030"
LOAD_ON_START: ${LOAD_ON_START:-0}
volumes:
- ${HOST_APP_DIR:-Z:/LLM/model-chat-app}:/app
- ${HOST_MODELS_DIR:-Z:/LLM/models}:/models
- ${HOST_HF_CACHE_DIR:-Z:/LLM/models/cache/huggingface}:/root/.cache/huggingface
working_dir: /app
entrypoint: python3
command:
- scripts/transformers_plugin_server.py
- --plugin
- video
- --host
- 0.0.0.0
- --port
- "8030"
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities:
- gpu
@@ -0,0 +1,9 @@
HOST_APP_DIR=Z:/LLM/model-chat-app
HOST_MODELS_DIR=Z:/LLM/models
HOST_HF_CACHE_DIR=Z:/LLM/models/cache/huggingface
TRANSFORMERS_IMAGE=vllm/vllm-openai:v0.10.2
TRANSFORMERS_CONTAINER_NAME=llm-transformers-video
TRANSFORMERS_HOST_PORT=8030
MODEL_PATH=/models/video/qwen2.5-vl-7b-instruct
SERVED_MODEL_NAME=qwen2.5-vl-7b-instruct
LOAD_ON_START=0
+20
View File
@@ -0,0 +1,20 @@
# Copy to .env on the deployment host and adjust values there.
# Do not commit real tokens, private paths, or credentials.
VLLM_CONTAINER_NAME=llm-vllm-text
VLLM_IMAGE=vllm/vllm-openai:latest
VLLM_MODEL_ID=Qwen/Qwen3-4B-Instruct-2507
VLLM_SERVED_MODEL_NAME=qwen3-4b-instruct
VLLM_LORA_MODULES=qwen3-4b-1c=/models/adapters/1c/qwen3-4b-1c-lora-v1
VLLM_MAX_LORAS=1
VLLM_HOST_PORT=8000
VLLM_GPU_MEMORY_UTILIZATION=0.90
VLLM_MAX_MODEL_LEN=32768
VLLM_DTYPE=auto
# Local model/cache paths on docker-gpu.cin.su.
HOST_MODELS_DIR=/models
HOST_HF_CACHE_DIR=/models/cache/huggingface
# Set on the host only if the model source requires it.
HF_TOKEN=
+42
View File
@@ -0,0 +1,42 @@
services:
vllm:
image: ${VLLM_IMAGE:-vllm/vllm-openai:latest}
container_name: ${VLLM_CONTAINER_NAME:-llm-vllm-text}
restart: unless-stopped
ipc: host
ports:
- "${VLLM_HOST_PORT:-8000}:8000"
environment:
HF_HOME: /root/.cache/huggingface
HUGGING_FACE_HUB_TOKEN: ${HF_TOKEN:-}
volumes:
- ${HOST_MODELS_DIR:-/models}:/models
- ${HOST_HF_CACHE_DIR:-/models/cache/huggingface}:/root/.cache/huggingface
command:
- --model
- ${VLLM_MODEL_ID:-Qwen/Qwen3-4B-Instruct-2507}
- --served-model-name
- ${VLLM_SERVED_MODEL_NAME:-qwen3-4b-instruct}
- --enable-lora
- --max-loras
- ${VLLM_MAX_LORAS:-1}
- --lora-modules
- ${VLLM_LORA_MODULES:-qwen3-4b-1c=/models/adapters/1c/qwen3-4b-1c-lora-v1}
- --host
- 0.0.0.0
- --port
- "8000"
- --gpu-memory-utilization
- ${VLLM_GPU_MEMORY_UTILIZATION:-0.90}
- --max-model-len
- ${VLLM_MAX_MODEL_LEN:-32768}
- --dtype
- ${VLLM_DTYPE:-auto}
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities:
- gpu
@@ -0,0 +1,24 @@
ONEC_AGENT_IMAGE=onec-agent:latest
ONEC_AGENT_CONTAINER_NAME=onec-agent
ONEC_AGENT_HOST_PORT=8090
ONEC_AGENT_DEFAULT_BASE_URL=http://docker-gpu.cin.su:8000
ONEC_AGENT_DEFAULT_MODEL=qwen3-4b-instruct-2507
ONEC_ADAPTER_URL=http://docker-gpu.cin.su:8011
# Для интеграции с несколькими ИИ провайдерскими конечными точками
# Формат JSON:
# {
# "default": {
# "type": "openai-compatible",
# "base_url": "http://docker-gpu.cin.su:8000",
# "model": "qwen3-4b-instruct-2507"
# },
# "openrouter": {
# "type": "openai-compatible",
# "base_url": "https://openrouter.ai/api/v1",
# "model": "some-openai-compatible-id",
# "api_key_env": "OPENROUTER_API_KEY"
# }
# }
ONEC_AGENT_PROVIDERS=
OPENROUTER_API_KEY=
+26
View File
@@ -0,0 +1,26 @@
name: onec-agent
services:
onec-agent:
build:
context: ../../../../
dockerfile: plugins/1c/agent/Dockerfile
image: ${ONEC_AGENT_IMAGE:-onec-agent:latest}
container_name: ${ONEC_AGENT_CONTAINER_NAME:-onec-agent}
restart: unless-stopped
ports:
- "${ONEC_AGENT_HOST_PORT:-8090}:8090"
volumes:
- onec-agent-data:/app/data
environment:
ONEC_AGENT_HOST: 0.0.0.0
ONEC_AGENT_PORT: 8090
ONEC_AGENT_DB_PATH: /app/data/onec-agent.db
ONEC_AGENT_DEFAULT_BASE_URL: ${ONEC_AGENT_DEFAULT_BASE_URL:-http://docker-gpu.cin.su:8000}
ONEC_AGENT_DEFAULT_MODEL: ${ONEC_AGENT_DEFAULT_MODEL:-qwen3-4b-instruct-2507}
ONEC_ADAPTER_URL: ${ONEC_ADAPTER_URL:-http://docker-gpu.cin.su:8011}
ONEC_ADAPTER_TOKEN: ${ONEC_ADAPTER_TOKEN:-}
ONEC_AGENT_PROVIDERS: ${ONEC_AGENT_PROVIDERS:-}
volumes:
onec-agent-data:
@@ -0,0 +1,11 @@
ADAPTER_1C_MCP_IMAGE=adapter-1c-mcp:latest
ADAPTER_1C_MCP_CONTAINER_NAME=adapter-1c-mcp
ADAPTER_1C_MCP_HOST_PORT=8021
# REST 1C adapter endpoint. The MCP proxy forwards onec_request(method,payload)
# to this service. Change it when the real adapter container is deployed.
ONEC_ADAPTER_URL=http://docker-gpu.cin.su:8011
ONEC_ADAPTER_TIMEOUT_SECONDS=120
# Optional bearer token for the REST adapter. Do not commit real secrets.
ONEC_ADAPTER_TOKEN=
@@ -0,0 +1,17 @@
name: adapter-1c-mcp
services:
adapter-1c-mcp:
build:
context: ../../../../plugins/1c/mcp
dockerfile: Dockerfile
image: ${ADAPTER_1C_MCP_IMAGE:-adapter-1c-mcp:latest}
container_name: ${ADAPTER_1C_MCP_CONTAINER_NAME:-adapter-1c-mcp}
restart: unless-stopped
ports:
- "${ADAPTER_1C_MCP_HOST_PORT:-8021}:8021"
environment:
PORT: "8021"
ONEC_ADAPTER_URL: ${ONEC_ADAPTER_URL:-http://docker-gpu.cin.su:8011}
ONEC_ADAPTER_TOKEN: ${ONEC_ADAPTER_TOKEN:-}
ONEC_ADAPTER_TIMEOUT_SECONDS: ${ONEC_ADAPTER_TIMEOUT_SECONDS:-240}
+12
View File
@@ -0,0 +1,12 @@
# Core Evals
Общие правила оценки качества моделей.
Eval-наборы должны позволять сравнить:
- базовую модель;
- модель с RAG;
- модель с адаптером;
- разные версии адаптеров.
Для каждого плагина могут быть собственные eval-наборы.
+12
View File
@@ -0,0 +1,12 @@
# Core Inference
Общий слой инференса.
Цель: дать единый интерфейс для запуска моделей разных типов.
Планируемые режимы:
- OpenAI-compatible API для текстовых моделей;
- batch inference;
- локальный inference для eval-тестов;
- подключение LoRA/adapters поверх базовых моделей.
+10
View File
@@ -0,0 +1,10 @@
# Core Monitoring
Мониторинг должен покрывать:
- использование GPU и VRAM;
- время ответа;
- ошибки инференса;
- количество запросов;
- версии используемых моделей;
- результаты eval-прогонов.
+14
View File
@@ -0,0 +1,14 @@
"""Shared observability helpers for services and plugins."""
from .redaction import sanitize_for_logging
from .store import JsonlAuditStore, resolve_audit_root
from .trace import next_trace_id, resolve_request_id, resolve_trace_id
__all__ = [
"JsonlAuditStore",
"next_trace_id",
"resolve_audit_root",
"resolve_request_id",
"resolve_trace_id",
"sanitize_for_logging",
]
+48
View File
@@ -0,0 +1,48 @@
from __future__ import annotations
import re
from typing import Any
REDACTED = "[REDACTED]"
SENSITIVE_KEY_PARTS = (
"authorization",
"api_key",
"apikey",
"access_token",
"refresh_token",
"token",
"password",
"secret",
"cookie",
"set-cookie",
)
BEARER_RE = re.compile(r"(?i)\bBearer\s+[A-Za-z0-9._~+/=-]+")
def _looks_sensitive_key(key: str) -> bool:
lowered = key.strip().lower()
return any(part in lowered for part in SENSITIVE_KEY_PARTS)
def _sanitize_string(value: str) -> str:
return BEARER_RE.sub("Bearer " + REDACTED, value)
def sanitize_for_logging(value: Any) -> Any:
if isinstance(value, dict):
cleaned: dict[str, Any] = {}
for key, item in value.items():
key_text = str(key)
if _looks_sensitive_key(key_text):
cleaned[key_text] = REDACTED
else:
cleaned[key_text] = sanitize_for_logging(item)
return cleaned
if isinstance(value, list):
return [sanitize_for_logging(item) for item in value]
if isinstance(value, tuple):
return [sanitize_for_logging(item) for item in value]
if isinstance(value, str):
return _sanitize_string(value)
return value
+47
View File
@@ -0,0 +1,47 @@
from __future__ import annotations
import json
import threading
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
def utc_now_iso() -> str:
return datetime.now(timezone.utc).isoformat()
def resolve_audit_root(root: Path, *, service: str, env_var: str | None = None) -> Path:
import os
configured = os.environ.get(env_var or "", "").strip() if env_var else ""
if configured:
return Path(configured)
return root / "reports" / "observability" / service
class JsonlAuditStore:
def __init__(self, root: Path, *, service: str) -> None:
self.root = root
self.service = service
self.root.mkdir(parents=True, exist_ok=True)
self._lock = threading.Lock()
def _path_for(self, event_type: str) -> Path:
stamp = datetime.now(timezone.utc).strftime("%Y%m%d")
return self.root / event_type / f"{stamp}.jsonl"
def write_event(self, event_type: str, payload: dict[str, Any]) -> dict[str, Any]:
record = {
"event_type": event_type,
"service": self.service,
"logged_at": utc_now_iso(),
**payload,
}
path = self._path_for(event_type)
path.parent.mkdir(parents=True, exist_ok=True)
line = json.dumps(record, ensure_ascii=False, separators=(",", ":")) + "\n"
with self._lock:
with path.open("a", encoding="utf-8") as handle:
handle.write(line)
return record
+26
View File
@@ -0,0 +1,26 @@
from __future__ import annotations
import uuid
from typing import Mapping
def next_trace_id() -> str:
return uuid.uuid4().hex
def _header_value(headers: Mapping[str, str], *names: str) -> str:
for name in names:
value = headers.get(name)
if value:
return str(value).strip()
return ""
def resolve_trace_id(headers: Mapping[str, str]) -> str:
trace_id = _header_value(headers, "x-trace-id", "X-Trace-Id", "x-request-id", "X-Request-Id")
return trace_id or next_trace_id()
def resolve_request_id(headers: Mapping[str, str], *, fallback_trace_id: str) -> str:
request_id = _header_value(headers, "x-request-id", "X-Request-Id")
return request_id or fallback_trace_id
+10
View File
@@ -0,0 +1,10 @@
# Core Registry
Общий слой работы с реестром моделей.
Планируемые функции:
- проверка `model-card.yaml`;
- поиск моделей по задаче, языку и требованиям VRAM;
- учет базовых моделей и адаптеров;
- контроль статусов `draft`, `staging`, `production`, `archived`.
+24
View File
@@ -0,0 +1,24 @@
# Core Storage
Правила хранения моделей, датасетов и артефактов.
Рекомендуемая внешняя структура:
```text
/models
/base
/adapters
/embeddings
/audio
/video
/translation
/datasets
/raw
/prepared
/evals
/artifacts
```
В git храним только метаданные и инструкции.
+7
View File
@@ -0,0 +1,7 @@
# Core Training
Общие пайплайны подготовки данных и дообучения.
Основной подход для доменных моделей: adapter-based fine-tuning, например LoRA/QLoRA.
Полное дообучение базовых моделей не используем как первый вариант из-за стоимости, сложности хранения и риска ухудшения общего качества.
File diff suppressed because one or more lines are too long
+308
View File
@@ -0,0 +1,308 @@
{
"captured_at": "2026-06-25T01:38:41.221196+00:00",
"base_id": "upo_test",
"table": "ConfigCASSave",
"file_name": "f96a0c45-3eff-11f1-8287-005056b0d483__25c39fbf-35a4-4b43-8e3f-cd1f91082c88.0",
"storage": {
"sha1": "e241f69d71022ffb3dd74922474b5f22f6c65e16",
"bytes": 4336
},
"command": {
"name": "КомандаПример1",
"id": "2",
"title": "Пример1",
"path": "5.3",
"title_path": "5.3.3.2.1",
"id_path": "5.3.1.0",
"name_path": null
},
"command_parameters": [
{
"index": 0,
"presentation": "Маркер записи",
"value": "11",
"value_kind": "number",
"position": {
"indices": [
5,
3,
0
]
}
},
{
"index": 1,
"presentation": "Идентификатор",
"value": "",
"value_kind": "string",
"position": {
"indices": [
5,
3,
1
]
}
},
{
"index": 2,
"presentation": "Имя",
"value": "КомандаПример1",
"value_kind": "string",
"position": {
"indices": [
5,
3,
2
]
}
},
{
"index": 3,
"presentation": "Заголовок",
"value": "",
"value_kind": "string",
"position": {
"indices": [
5,
3,
3
]
}
},
{
"index": 4,
"presentation": "Параметр 4",
"value": "",
"value_kind": "string",
"position": {
"indices": [
5,
3,
4
]
}
},
{
"index": 5,
"presentation": "Параметр 5",
"value": "",
"value_kind": "string",
"position": {
"indices": [
5,
3,
5
]
}
},
{
"index": 6,
"presentation": "Имя",
"value": "",
"value_kind": "string",
"position": {
"indices": [
5,
3,
6
]
}
},
{
"index": 7,
"presentation": "Параметр 7",
"value": "",
"value_kind": "string",
"position": {
"indices": [
5,
3,
7
]
}
},
{
"index": 8,
"presentation": "Параметр 8",
"value": "КомандаПример1",
"value_kind": "string",
"position": {
"indices": [
5,
3,
8
]
}
},
{
"index": 9,
"presentation": "Параметр 9",
"value": "3",
"value_kind": "number",
"position": {
"indices": [
5,
3,
9
]
}
},
{
"index": 10,
"presentation": "Параметр 10",
"value": "0",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
10
]
}
},
{
"index": 11,
"presentation": "Параметр 11",
"value": "0",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
11
]
}
},
{
"index": 12,
"presentation": "Параметр 12",
"value": "",
"value_kind": "string",
"position": {
"indices": [
5,
3,
12
]
}
},
{
"index": 13,
"presentation": "Параметр 13",
"value": "1",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
13
]
}
},
{
"index": 14,
"presentation": "Параметр 14",
"value": "0",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
14
]
}
},
{
"index": 15,
"presentation": "Параметр 15",
"value": "1",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
15
]
}
},
{
"index": 16,
"presentation": "Параметр 16",
"value": "0",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
16
]
}
},
{
"index": 17,
"presentation": "Параметр 17",
"value": "0",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
17
]
}
},
{
"index": 18,
"presentation": "Параметр 18",
"value": "2",
"value_kind": "number",
"position": {
"indices": [
5,
3,
18
]
}
},
{
"index": 19,
"presentation": "Параметр 19",
"value": "0",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
19
]
}
},
{
"index": 20,
"presentation": "Параметр 20",
"value": "0",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
20
]
}
}
],
"decode_counts": {
"items": 33,
"items_total": 33,
"focused_elements": null,
"attributes": 6,
"attributes_total": 6,
"commands": 3,
"commands_total": 3,
"events": 0,
"handler_links": 0,
"resolved_handlers": 0,
"missing_handlers": 0,
"button_command_links": 0
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because one or more lines are too long
+308
View File
@@ -0,0 +1,308 @@
{
"captured_at": "2026-06-25T01:36:16.633769+00:00",
"base_id": "upo_test",
"table": "ConfigCASSave",
"file_name": "f96a0c45-3eff-11f1-8287-005056b0d483__25c39fbf-35a4-4b43-8e3f-cd1f91082c88.0",
"storage": {
"sha1": "73dccf9d0270d3571a9d8e8fe48266c0148ba20f",
"bytes": 4322
},
"command": {
"name": "КомандаПример1",
"id": "2",
"title": "Пример1",
"path": "5.3",
"title_path": "5.3.3.2.1",
"id_path": "5.3.1.0",
"name_path": null
},
"command_parameters": [
{
"index": 0,
"presentation": "Маркер записи",
"value": "11",
"value_kind": "number",
"position": {
"indices": [
5,
3,
0
]
}
},
{
"index": 1,
"presentation": "Идентификатор",
"value": "",
"value_kind": "string",
"position": {
"indices": [
5,
3,
1
]
}
},
{
"index": 2,
"presentation": "Имя",
"value": "КомандаПример1",
"value_kind": "string",
"position": {
"indices": [
5,
3,
2
]
}
},
{
"index": 3,
"presentation": "Заголовок",
"value": "",
"value_kind": "string",
"position": {
"indices": [
5,
3,
3
]
}
},
{
"index": 4,
"presentation": "Параметр 4",
"value": "",
"value_kind": "string",
"position": {
"indices": [
5,
3,
4
]
}
},
{
"index": 5,
"presentation": "Параметр 5",
"value": "",
"value_kind": "string",
"position": {
"indices": [
5,
3,
5
]
}
},
{
"index": 6,
"presentation": "Имя",
"value": "",
"value_kind": "string",
"position": {
"indices": [
5,
3,
6
]
}
},
{
"index": 7,
"presentation": "Параметр 7",
"value": "",
"value_kind": "string",
"position": {
"indices": [
5,
3,
7
]
}
},
{
"index": 8,
"presentation": "Параметр 8",
"value": "КомандаПример1",
"value_kind": "string",
"position": {
"indices": [
5,
3,
8
]
}
},
{
"index": 9,
"presentation": "Параметр 9",
"value": "3",
"value_kind": "number",
"position": {
"indices": [
5,
3,
9
]
}
},
{
"index": 10,
"presentation": "Параметр 10",
"value": "0",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
10
]
}
},
{
"index": 11,
"presentation": "Параметр 11",
"value": "0",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
11
]
}
},
{
"index": 12,
"presentation": "Параметр 12",
"value": "",
"value_kind": "string",
"position": {
"indices": [
5,
3,
12
]
}
},
{
"index": 13,
"presentation": "Параметр 13",
"value": "1",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
13
]
}
},
{
"index": 14,
"presentation": "Параметр 14",
"value": "0",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
14
]
}
},
{
"index": 15,
"presentation": "Параметр 15",
"value": "1",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
15
]
}
},
{
"index": 16,
"presentation": "Параметр 16",
"value": "0",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
16
]
}
},
{
"index": 17,
"presentation": "Параметр 17",
"value": "0",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
17
]
}
},
{
"index": 18,
"presentation": "Параметр 18",
"value": "2",
"value_kind": "number",
"position": {
"indices": [
5,
3,
18
]
}
},
{
"index": 19,
"presentation": "Параметр 19",
"value": "0",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
19
]
}
},
{
"index": 20,
"presentation": "Параметр 20",
"value": "0",
"value_kind": "boolean_or_number",
"position": {
"indices": [
5,
3,
20
]
}
}
],
"decode_counts": {
"items": 33,
"items_total": 33,
"focused_elements": null,
"attributes": 6,
"attributes_total": 6,
"commands": 3,
"commands_total": 3,
"events": 0,
"handler_links": 0,
"resolved_handlers": 0,
"missing_handlers": 0,
"button_command_links": 0
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+29
View File
@@ -0,0 +1,29 @@
import difflib
import hashlib
import json
from pathlib import Path
ROOT = Path('data/1c-write-learning')
before = (ROOT / 'baseline.bin').read_bytes()
after_path = ROOT / 'after.bin'
if not after_path.exists():
raise SystemExit('after.bin is missing; capture after state first')
after = after_path.read_bytes()
def ranges(a: bytes, b: bytes):
sm = difflib.SequenceMatcher(None, a, b, autojunk=False)
out = []
for tag, i1, i2, j1, j2 in sm.get_opcodes():
if tag != 'equal':
out.append({'tag': tag, 'before': [i1, i2], 'after': [j1, j2], 'before_hex': a[i1:i2].hex(), 'after_hex': b[j1:j2].hex()})
return out
report = {
'before': {'bytes': len(before), 'sha1': hashlib.sha1(before).hexdigest()},
'after': {'bytes': len(after), 'sha1': hashlib.sha1(after).hexdigest()},
'delta_bytes': len(after) - len(before),
'ranges': ranges(before, after),
}
(ROOT / 'byte_diff.json').write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding='utf-8')
print(json.dumps({**report, 'ranges': report['ranges'][:20], 'range_count': len(report['ranges'])}, ensure_ascii=False, indent=2))
+240
View File
@@ -0,0 +1,240 @@
{
"before": {
"raw_bytes": 4322,
"payload_bytes": 33820,
"compression": "raw_deflate",
"encoding": "utf-8-sig",
"payload_sha1": "361daeb3bb7ed2f858433ea79f8e92e76d8fe2bd"
},
"after": {
"raw_bytes": 4336,
"payload_bytes": 33826,
"compression": "raw_deflate",
"encoding": "utf-8-sig",
"payload_sha1": "426ce14f4c3f764edb91b855a9b8aac2fabce110"
},
"payload_delta_bytes": 6,
"text_range_count": 14,
"text_ranges": [
{
"tag": "replace",
"before": [
3240,
3242
],
"after": [
3240,
3243
],
"before_text": "97",
"after_text": "108"
},
{
"tag": "insert",
"before": [
4797,
4797
],
"after": [
4798,
4800
],
"before_text": "",
"after_text": "10"
},
{
"tag": "delete",
"before": [
4798,
4799
],
"after": [
4801,
4801
],
"before_text": "8",
"after_text": ""
},
{
"tag": "replace",
"before": [
6100,
6102
],
"after": [
6102,
6105
],
"before_text": "99",
"after_text": "110"
},
{
"tag": "replace",
"before": [
9557,
9559
],
"after": [
9560,
9562
],
"before_text": "00",
"after_text": "11"
},
{
"tag": "delete",
"before": [
11136,
11137
],
"after": [
11139,
11139
],
"before_text": "0",
"after_text": ""
},
{
"tag": "insert",
"before": [
11138,
11138
],
"after": [
11140,
11141
],
"before_text": "",
"after_text": "2"
},
{
"tag": "replace",
"before": [
12739,
12741
],
"after": [
12742,
12744
],
"before_text": "02",
"after_text": "13"
},
{
"tag": "replace",
"before": [
15705,
15707
],
"after": [
15708,
15710
],
"before_text": "04",
"after_text": "15"
},
{
"tag": "replace",
"before": [
15912,
15914
],
"after": [
15915,
15917
],
"before_text": "03",
"after_text": "14"
},
{
"tag": "replace",
"before": [
16113,
16115
],
"after": [
16116,
16118
],
"before_text": "05",
"after_text": "16"
},
{
"tag": "replace",
"before": [
16946,
16948
],
"after": [
16949,
16951
],
"before_text": "06",
"after_text": "17"
},
{
"tag": "replace",
"before": [
22408,
22410
],
"after": [
22411,
22413
],
"before_text": "07",
"after_text": "18"
},
{
"tag": "replace",
"before": [
30327,
30333
],
"after": [
30330,
30337
],
"before_text": "ример2",
"after_text": "РОВЕРКА"
}
],
"path_values": [
{
"path": "5.3",
"before": "",
"after": ""
},
{
"path": "5.3.1.0",
"before": "2",
"after": "2"
},
{
"path": "5.3.2",
"before": "КомандаПример1",
"after": "КомандаПример1"
},
{
"path": "5.3.3",
"before": "",
"after": ""
},
{
"path": "5.3.3.2.1",
"before": "Пример1",
"after": "Пример1"
},
{
"path": "5.3.8",
"before": "КомандаПример1",
"after": "КомандаПример1"
}
],
"strings_added": [
"ПРОВЕРКА"
],
"strings_removed": [
"Пример2"
]
}
@@ -0,0 +1,240 @@
[
{
"range": {
"tag": "replace",
"before": [
3240,
3242
],
"after": [
3240,
3243
],
"before_text": "97",
"after_text": "108"
},
"before_context": "e}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,1,1,\n\n{22,\n\n{97,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"АПанельДействийВыделенныхСтрок\"",
"after_context": "e}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,1,1,\n\n{22,\n\n{108,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"АПанельДействийВыделенныхСтрок\""
},
{
"range": {
"tag": "insert",
"before": [
4797,
4797
],
"after": [
4798,
4800
],
"before_text": "",
"after_text": "10"
},
"before_context": "e}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,1,1,\n\n{22,\n\n{98,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"БПанельДействийВыделенныхСтро",
"after_context": "e}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,1,1,\n\n{22,\n\n{109,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"БПанельДействийВыделенныхСтрок"
},
{
"range": {
"tag": "delete",
"before": [
4798,
4799
],
"after": [
4801,
4801
],
"before_text": "8",
"after_text": ""
},
"before_context": "}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,1,1,\n\n{22,\n\n{98,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"БПанельДействийВыделенныхСтрок\"",
"after_context": "\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,1,1,\n\n{22,\n\n{109,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"БПанельДействийВыделенныхСтрок\""
},
{
"range": {
"tag": "replace",
"before": [
6100,
6102
],
"after": [
6102,
6105
],
"before_text": "99",
"after_text": "110"
},
"before_context": "e}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,1,1,\n\n{22,\n\n{99,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"ХочуКрасненькогоПанельДействийВ",
"after_context": "e}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,1,1,\n\n{22,\n\n{110,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"ХочуКрасненькогоПанельДействийВ"
},
{
"range": {
"tag": "replace",
"before": [
9557,
9559
],
"after": [
9560,
9562
],
"before_text": "00",
"after_text": "11"
},
"before_context": "}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,3,1,\n\n{22,\n\n{100,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"ТЗК1ПанельДействийВыделенныхСтр",
"after_context": "}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,3,1,\n\n{22,\n\n{111,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"ТЗК1ПанельДействийВыделенныхСтр"
},
{
"range": {
"tag": "delete",
"before": [
11136,
11137
],
"after": [
11139,
11139
],
"before_text": "0",
"after_text": ""
},
"before_context": "}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,3,1,\n\n{22,\n\n{101,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"ТЗК2ПанельДействийВыделенныхСт",
"after_context": "}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,3,1,\n\n{22,\n\n{112,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"ТЗК2ПанельДействийВыделенныхС"
},
{
"range": {
"tag": "insert",
"before": [
11138,
11138
],
"after": [
11140,
11141
],
"before_text": "",
"after_text": "2"
},
"before_context": "\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,3,1,\n\n{22,\n\n{101,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"ТЗК2ПанельДействийВыделенныхСтр",
"after_context": "\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,3,1,\n\n{22,\n\n{112,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"ТЗК2ПанельДействийВыделенныхСтр"
},
{
"range": {
"tag": "replace",
"before": [
12739,
12741
],
"after": [
12742,
12744
],
"before_text": "02",
"after_text": "13"
},
"before_context": "}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,3,1,\n\n{22,\n\n{102,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"ТЗПримечаниеПанельДействийВыдел",
"after_context": "}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,3,1,\n\n{22,\n\n{113,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"ТЗПримечаниеПанельДействийВыдел"
},
{
"range": {
"tag": "replace",
"before": [
15705,
15707
],
"after": [
15708,
15710
],
"before_text": "04",
"after_text": "15"
},
"before_context": "\n{20,2},0,3,3,0,\"\"},0,1,0,0,1,0,3,3,0,1,0,0,0,0,1,0,2,2,0,0,\"\",\"\",0,1,\n\n{22,\n\n{104,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"ТЗПанельДействийВыделенныхСтрок",
"after_context": "\n{20,2},0,3,3,0,\"\"},0,1,0,0,1,0,3,3,0,1,0,0,0,0,1,0,2,2,0,0,\"\",\"\",0,1,\n\n{22,\n\n{115,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"ТЗПанельДействийВыделенныхСтрок"
},
{
"range": {
"tag": "replace",
"before": [
15912,
15914
],
"after": [
15915,
15917
],
"before_text": "03",
"after_text": "14"
},
"before_context": "4},\n\n{8,3,0,1,100},\n\n{0,0,0},1,\n\n{0,1,0,1},0,1,0,0,0,3,3,0},0,0,0,2,1,\n\n{22,\n\n{103,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,11,\"ТЗДействияСтроки\",\n\n{1,0},\n\n{1,",
"after_context": "4},\n\n{8,3,0,1,100},\n\n{0,0,0},1,\n\n{0,1,0,1},0,1,0,0,0,3,3,0},0,0,0,2,1,\n\n{22,\n\n{114,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,11,\"ТЗДействияСтроки\",\n\n{1,0},\n\n{1,"
},
{
"range": {
"tag": "replace",
"before": [
16113,
16115
],
"after": [
16116,
16118
],
"before_text": "05",
"after_text": "16"
},
"before_context": ",0,1,100},\n\n{0,0,0},1,\n\n{0,1,0,1},0,1,0,0,0,3,3,0},2,0,0,1,0,0,0,0,0,1,\n\n{6,\n\n{105,02023637-7868-4a5f-8576-835a76e0c9ba},0,0,0,0,\"ТЗПанельДействийВыделенныхСтрокС",
"after_context": ",0,1,100},\n\n{0,0,0},1,\n\n{0,1,0,1},0,1,0,0,0,3,3,0},2,0,0,1,0,0,0,0,0,1,\n\n{6,\n\n{116,02023637-7868-4a5f-8576-835a76e0c9ba},0,0,0,0,\"ТЗПанельДействийВыделенныхСтрокС"
},
{
"range": {
"tag": "replace",
"before": [
16946,
16948
],
"after": [
16949,
16951
],
"before_text": "06",
"after_text": "17"
},
"before_context": "{1,0},0},0,0,1,0,0,1,0,3,3,0,0},2,\n\n{20,0},0,3,3,0,\"ТЗСтрокаПоиска\"},1,\n\n{6,\n\n{106,02023637-7868-4a5f-8576-835a76e0c9ba},0,0,0,2,\"ТЗПанельДействийВыделенныхСтрокУ",
"after_context": "{1,0},0},0,0,1,0,0,1,0,3,3,0,0},2,\n\n{20,0},0,3,3,0,\"ТЗСтрокаПоиска\"},1,\n\n{6,\n\n{117,02023637-7868-4a5f-8576-835a76e0c9ba},0,0,0,2,\"ТЗПанельДействийВыделенныхСтрокУ"
},
{
"range": {
"tag": "replace",
"before": [
22408,
22410
],
"after": [
22411,
22413
],
"before_text": "07",
"after_text": "18"
},
"before_context": "}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,1,1,\n\n{22,\n\n{107,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"КодПрограммыПанельДействийВыдел",
"after_context": "}\n\n},0,1,2,\n\n{1,\n\n{1,0},0},0,0,1,0,0,1,0,3,3,0,0},3,3,0,0,0,0,2,0,1,1,\n\n{22,\n\n{118,02023637-7868-4a5f-8576-835a76e0c9ba},1,0,0,10,\"КодПрограммыПанельДействийВыдел"
},
{
"range": {
"tag": "replace",
"before": [
30327,
30333
],
"after": [
30330,
30337
],
"before_text": "ример2",
"after_text": "РОВЕРКА"
},
"before_context": "11,\n\n{3,409b9a53-7f7e-4178-86c1-33176c7c7a7a},\"КомандаПример2\",\n\n{1,1,\n\n{\"ru\",\"Пример2\"}\n\n},\n\n{1,1,\n\n{\"ru\",\"Команда пример1\"}\n\n},\n\n{0,\n\n{0,\n\n{\"B\",1},0}\n\n},\n\n{0,0,0},\n",
"after_context": "11,\n\n{3,409b9a53-7f7e-4178-86c1-33176c7c7a7a},\"КомандаПример2\",\n\n{1,1,\n\n{\"ru\",\"ПРОВЕРКА\"}\n\n},\n\n{1,1,\n\n{\"ru\",\"Команда пример1\"}\n\n},\n\n{0,\n\n{0,\n\n{\"B\",1},0}\n\n},\n\n{0,0,0},\n"
}
]
+1
View File
@@ -0,0 +1 @@
+1
View File
@@ -0,0 +1 @@
File diff suppressed because it is too large Load Diff
+297
View File
@@ -0,0 +1,297 @@
# 1C Extension Layer Plan
Status: active work plan.
This plan defines how the 1C adapter and agent must handle extensions, full
1C paths, effective reads, provenance, and safe writes.
## Core Principles
The adapter has two responsibilities that must not be merged:
- return the effective picture that the 1C runtime and Configurator see after
applying active extensions;
- keep editor provenance so every change can be routed to the correct layer:
base configuration, a concrete extension, or a saved-state working copy.
Agent-facing APIs use 1C names and full paths. GUIDs, SQL table names, CAS
keys, and payload offsets are storage evidence, not the everyday language of
the agent.
## Addressing Model
The default address format is a full semantic 1C path:
```text
<ObjectKind>.<ObjectName>[.<Section>.<Member>...]
```
Examples:
```text
Справочник.Контрагенты
Справочник.Контрагенты.Наименование
Документ.РеализацияТоваровУслуг.Товары.Номенклатура
РегистрСведений.ЦеныНоменклатуры.Измерения.Номенклатура
РегистрСведений.ЦеныНоменклатуры.Ресурсы.Цена
Документ.РеализацияТоваровУслуг.Форма.ФормаДокумента.Товары
ОбщийМодуль.ИнтеграцияСCRM.ОтправитьКонтрагента
```
Short names are input conveniences only. Before analysis or writing, the agent
must normalize them to a single full path or return candidate paths when the
name is ambiguous.
There are three allowed address levels:
- `canonical_path`: full 1C path, safe for reads and write planning;
- `context_path`: shortened path valid only inside a known object, form, module,
tabular section, or routine;
- `local_name`: local BSL symbol or local form item name, never enough for a
cross-object write by itself.
The adapter must keep metadata path resolution separate from code symbol
resolution. In BSL, `Номенклатура.ЕдИзмерения.Код` may start from a variable,
form attribute, object attribute, tabular-section column, query field, or
procedure parameter. It must not be silently treated as
`Справочник.Номенклатура`.
## Layer Views
Every read API that can be affected by extensions should support these views:
- `base`: only the main configuration;
- `extension`: only one selected extension;
- `effective`: the runtime/configurator picture after applying active
extensions;
- `origin`: provenance of effective members and code fragments;
- `diff`: semantic differences between base and one or more extensions.
The effective view is the default for agent investigation. It is incomplete for
writing unless paired with origin and write-target evidence.
## Extension Provenance
For every metadata object, member, form, module, routine, command, and event
handler, responses should expose:
- `canonical_path`;
- `presentation` and synonym when known;
- `created_in`;
- `modified_by`;
- `effective_owner`;
- `active_extensions`;
- `extension_order`;
- `conflicts`;
- `storage_evidence` for debug/expert mode.
Extension changes must be classified semantically:
- object added by extension;
- base object adopted/extended by extension;
- attribute/tabular section/form/command/event added;
- property changed;
- module added;
- routine added;
- code inserted before;
- code inserted after;
- code replaced;
- code replaced with control.
For `replace_with_control`, the adapter must expose the controlled base
fragment and report whether it still matches the current base/effective source.
## Write Planning
Writes must start with a plan. The agent may read effective text, but must not
write effective text directly.
The write planner must answer:
- what full path or symbol is being changed;
- what layer owns the current effective element;
- which layer is the correct write target;
- whether the target is base saved state, extension saved state, generated
extension source, or read-only reference evidence;
- which operation is allowed: add, property change, insert before, insert after,
replace, replace with control, append routine, upsert routine, move form item;
- which guards are required: sha1, controlled fragment, expected old text,
syntax check, extension order, conflict scan;
- which follow-up reads and validation steps must be run.
The first production-safe write path remains extension-first:
```text
generate extension source
-> validate in 1C tooling
-> package extension
-> load into disposable base
-> run smoke tests
-> produce human approval diff
```
Direct active configuration writes remain forbidden.
## Agent Workflow
For concrete 1C tasks, the agent should follow this sequence:
1. Parse the user request and extract likely full paths, short names, and local
symbols.
2. Resolve all metadata paths through the adapter.
3. Read effective context for the selected objects.
4. Read origin/layer evidence before drawing conclusions about ownership.
5. For BSL expressions, resolve local symbols inside the concrete module,
routine, form, or query context.
6. If a change is needed, build a write plan before generating or applying a
patch.
7. Apply only through allowed saved-state, patch-workspace, or extension-source
routes.
8. Re-read effective and origin views.
9. Run semantic diff, BSL syntax checks, saved-state checks, and extension
conflict checks.
10. Report results to the user in full 1C paths, hiding storage ids unless the
user asks for expert evidence.
## Work Plan
1. Finalize the full-path contract in adapter docs and schemas.
2. Extend the resolver model to return `canonical_path`, `context_path`,
ambiguity candidates, and path-kind diagnostics.
3. Add a code symbol resolver that works inside a selected module/routine/form
context and distinguishes variables from metadata paths.
4. Extend object, form, and module reads with consistent origin fields.
5. Add an extension layer inventory that reports active extensions, order,
adopted base objects, added objects, and conflicts.
6. Add routine-level extension action evidence: before, after, replace, and
replace with control.
7. Add a read-only write planner that chooses base saved state, extension saved
state, or generated extension source without applying changes.
8. Wire the planner into `metadata.write` so writes without a resolved
full-path target and layer decision are rejected.
9. Add smoke tests for ambiguous short names, extension-added attributes,
extension-overridden routines, and replace-with-control drift.
10. Add eval cases so the agent learns to answer with full paths and to refuse
unsafe direct effective-text writes.
## First Implementation Slice
The first executable slice should be read-only:
- update `resolve_1c_fact.py` output to include canonical path metadata;
- add path ambiguity tests for object member paths;
- add origin fields to module/routine reads where extension overlays already
exist;
- document a proposed `metadata.write.plan` shape before implementing apply
behavior.
Current progress:
- `resolve_1c_fact.py` returns `canonical_path`, `path_kind`, and tabular
section `context_path` for verified object/member facts.
- `metadata.write.plan` validates full 1C paths and concrete saved-state/module
references without applying changes.
- `metadata.write.plan` infers form/module write intent from canonical path
sections such as `Форма.<FormName>` and `ОбщийМодуль.<Name>.<Routine>`.
- `metadata.write.plan` can call `metadata.definition.find` for compact
`origin_lookup` evidence.
- `metadata.write.plan` reports `ambiguous_origin_matches` when a full path
still resolves to multiple definitions, including same-layer ambiguity such
as object member versus form member.
- `metadata.write.plan` recommends `base_saved_state`, `extension_saved_state`,
`saved_state`, `blocked_unknown`, or `blocked_conflict` from origin evidence.
- `metadata.write.plan` normalizes `preferred_layer` and reports
`preferred_layer_conflict` when a requested base/extension target disagrees
with the origin-derived write recommendation.
- `metadata.write.plan` accepts `preferred_extension` and reports
`preferred_extension_conflict` when the requested extension name/GUID differs
from the resolved extension owner.
- `metadata.write.plan` validates module-code preconditions for
`replace_with_control`, `replace`, and `insert_before`/`insert_after` before
allowing even saved-state plans.
- `metadata.write.plan` normalizes Russian and English operation names into
stable `operation_class` values such as `insert_before`, `insert_after`,
`replace`, and `replace_with_control`.
- `metadata.write.plan` returns `route.apply_payload_hint` for concrete
saved-state module/form routes, so an agent can carry the validated guards
into `metadata.module.write_apply` or form write planning without inventing
field names.
- `metadata.write.plan` also carries selector fields inferred from
`canonical_path` into `route.apply_payload_hint`, including `kind`, `name`,
`form`, `element`, and `routine_name` when known.
- `metadata.write.plan` marks selector-only module/form hints as
`ready_for_apply_method=false` and returns `next_resolution` pointing to the
required saved-state target resolver before apply methods can be called.
- `metadata.write` blocks direct writes to effective `canonical_path` targets
without a concrete saved-state/module route.
- Blocked `metadata.write` responses now surface the plan's `apply_payload_hint`
and `next_resolution` for parsed form/module paths, keeping the next safe
resolver visible without allowing direct writes to the effective view.
- When a concrete saved-state form file or module stream is later provided with
that same full path, `metadata.write` merges missing selector fields from the
plan hint into the low-level apply payload, such as `form`, `element`, and
`routine_name`.
- Concrete references now keep their field identity in write planning and
hints: `module_ref`, `module_id`, `file_name`, and `form_guid` are not
collapsed into each other.
- Write planning rejects incompatible concrete references with
`concrete_reference_kind_mismatch`, so a form selector cannot be used as a
module route or the other way around.
- `resolve_1c_bsl_symbol.py` adds the first conservative code-symbol resolver:
full paths such as `Справочник.Номенклатура.Артикул` resolve as metadata,
context-proven object-module fields resolve as metadata members, but routine
parameters, local variables, and short object names stay BSL symbols until
proven otherwise.
- The live adapter now exposes the same conservative resolver as
`code.symbol.resolve`, backed by `modules.read`, `metadata.definition.find`,
and `metadata.object.attributes`.
- `check_1c_code_symbol_contract.py` verifies the public adapter method through
`call_method`, including that `canonical_path`, `context_path`, and
`safe_as_metadata_path` survive public-result sanitizing.
- `modules.read` now returns fallback public `origin` evidence from the storage
table for direct module references: applied configuration, saved state, or
unresolved CAS reference that requires owner resolution before write planning.
- `check_1c_module_origin_contract.py` verifies that public `modules.read`
and `code.read` responses keep this origin evidence visible without
`include_storage=true`.
- `code.search` items now preserve `origin` from `modules.search`, so the
agent can see provenance before selecting a `read_selector` for `code.read`.
- `metadata.write.plan` now accepts `target.origin`/`origin` from those prior
read/search results and converts it to `provided_origin_evidence`, preserving
layer recommendation while still requiring a concrete write route before
apply.
- `metadata.resolve_overrides` now exposes `extension_action` per routine link:
known evidence is normalized to `insert_before`, `insert_after`, `replace`,
or `replace_with_control`; unresolved extension routine actions are reported
as `unknown_extension_action` so the agent does not silently treat them as a
plain replace.
- `metadata.write.plan` now accepts that `extension_action` evidence directly:
known actions can infer the module operation when the user did not specify
one, unknown actions block planning, and explicit operation/action mismatches
are rejected before any saved-state apply route can run.
- Multi-item `extension_actions` from an override chain now block write
planning as `extension_action_ambiguous`; the agent must narrow to one
extension/module action before generating a patch.
- `metadata.resolve_overrides` now returns `write_plan_evidence`, a ready
`metadata.write.plan` fragment carrying the routine name, object selector
fields, `target.extension_action`, and `next_resolution.params` for
`metadata.saved_state.modules.search`; saved-state module search now resolves
public `object_type`/`object_name` selectors to owner GUIDs when possible.
This reduces agent-side field translation while still requiring a concrete
saved-state module route.
- `metadata.saved_state.modules.search` now returns
`streams[].write_plan_target`, a concrete module target with `module_ref`,
`file_name`, `stream_index`, and `expected_sha1` for the next
`metadata.write.plan` call.
- The OpenAPI contract now exposes the same write-plan response fields:
concrete reference identity, apply payload hints, next resolution, and
blocked high-level write responses.
- `metadata.write` now refuses to call low-level apply methods when the
corresponding read-only plan is blocked, for example
`replace_with_control` without a control fragment.
- Direct saved-state form/module apply methods now run the same
`metadata.write.plan` gate before SQL apply, so low-level callers cannot
bypass blocked guard checks.
- `metadata.write.plan` blocks `replace_with_control` as
`control_fragment_drift` when optional current-source evidence is supplied
and the controlled fragment no longer matches that source.
+250
View File
@@ -0,0 +1,250 @@
# 1C Saved State Object Compare
Status: draft, read-only.
This adapter view compares saved-but-not-applied SQL state with the active
configuration state and returns the result in 1C configurator terms.
Command:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/compare_1c_saved_state_objects.ps1 `
-Server <SqlServer> `
-Database <SqlDatabase> `
-User <SqlUser> `
-Password <SqlPassword> `
-Output <SavedStateObjectComparisonJson>
```
Output schema:
```text
onec_saved_state_object_comparison.v1
```
Detail command:
```powershell
python scripts/analyze_1c_saved_state_object_details.py `
--comparison <SavedStateObjectComparisonJson> `
--config-save-dir <ConfigSaveExportDir> `
--config-dir <ConfigActiveExportDir> `
--config-cas-save-dir <ConfigCASSaveExportDir> `
--config-cas-dir <ConfigCASActiveExportDir> `
--extension-manifest-summary <ExtensionManifestSummaryJson> `
--config-cas-all-dir <ConfigCASAllDir> `
--output <SavedStateObjectDetailJson>
```
Detail output schema:
```text
onec_saved_state_object_detail.v1
```
Report command:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/build_1c_saved_state_object_report.ps1 `
-Server <SqlServer> `
-Database <SqlDatabase> `
-User <SqlUser> `
-Password <SqlPassword> `
-OutputDir <SavedStateReportDir>
```
Report output schema:
```text
onec_saved_state_object_report.v1
```
The report command is the normal agent entry point. It runs the object
comparison, exports only the required active/saved SQL payload evidence, and
runs detail analysis into one output directory. By default it also writes
`saved-state-object-report.md`; use `-SkipMarkdown` when only machine-readable
JSON is needed. The JSON report includes `agent_summary`, a compact machine
view with changed 1C object names, changed payload parts, payload roles,
active-missing part counts, and short added/removed semantic term hints.
Watch once command:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/watch_1c_saved_state_once.ps1 `
-Server <SqlServer> `
-Database <SqlDatabase> `
-User <SqlUser> `
-Password <SqlPassword> `
-OutputRoot <SavedStateWatchRoot>
```
```powershell
python scripts/list_1c_saved_state_watch_runs.py `
--root <SavedStateWatchRoot> `
--limit <N> `
--only-with-delta `
--output <SavedStateWatchRunListJson>
```
```powershell
python scripts/get_1c_saved_state_latest_watch_run.py `
--root <SavedStateWatchRoot> `
--require-delta `
--output <SavedStateLatestWatchRunJson>
```
When `--output` is provided, latest watch lookup writes Markdown next to the
JSON by default. Use `--skip-markdown` to disable this or `--markdown-output`
to set an explicit Markdown path.
```powershell
python scripts/check_1c_saved_state_latest_watch_run.py `
--latest <SavedStateLatestWatchRunJson> `
--output <SavedStateLatestWatchRunCheckJson>
```
```powershell
python scripts/render_1c_saved_state_latest_watch_run_markdown.py `
--latest <SavedStateLatestWatchRunJson> `
--output <SavedStateLatestWatchRunMarkdown>
```
When `--output` is provided, the watch-run list command writes Markdown next to
the JSON by default. Use `--skip-markdown` to disable this or
`--markdown-output` to set an explicit Markdown path. It also writes
`*-check.json` and runs a contract check by default. Use `--skip-check` to
disable this.
```powershell
python scripts/check_1c_saved_state_watch_run_list.py `
--list <SavedStateWatchRunListJson> `
--output <SavedStateWatchRunListCheckJson>
```
```powershell
python scripts/render_1c_saved_state_watch_run_list_markdown.py `
--list <SavedStateWatchRunListJson> `
--output <SavedStateWatchRunListMarkdown>
```
The watch command creates a timestamped report directory. If a previous
timestamped report exists under the same root, it also writes a delta from the
previous observation to the new observation. By default it writes
`saved-state-watch-run.md`; use `-SkipMarkdown` to disable Markdown rendering.
Markdown command:
```powershell
python scripts/render_1c_saved_state_object_report_markdown.py `
--report <SavedStateObjectReportJson> `
--output <SavedStateObjectReportMarkdown>
```
Check command:
```powershell
python scripts/check_1c_saved_state_object_report.py `
--report <SavedStateObjectReportJson> `
--output <SavedStateObjectReportCheckJson>
```
```powershell
python scripts/check_1c_saved_state_watch_once.py `
--manifest <SavedStateWatchRunJson> `
--output <SavedStateWatchRunCheckJson>
```
```powershell
python scripts/render_1c_saved_state_watch_once_markdown.py `
--manifest <SavedStateWatchRunJson> `
--output <SavedStateWatchRunMarkdown>
```
Report-to-report delta:
```powershell
python scripts/compare_1c_saved_state_object_reports.py `
--before <PreviousSavedStateObjectReportJson> `
--after <CurrentSavedStateObjectReportJson> `
--output <SavedStateObjectReportDeltaJson>
```
When `--output` is provided, the delta command writes Markdown next to the JSON
by default. Use `--skip-markdown` to disable this or `--markdown-output` to set
an explicit Markdown path. It also writes `*-check.json` and runs the delta
contract check by default. Use `--skip-check` to disable this.
```powershell
python scripts/check_1c_saved_state_object_report_delta.py `
--delta <SavedStateObjectReportDeltaJson> `
--output <SavedStateObjectReportDeltaCheckJson>
```
```powershell
python scripts/render_1c_saved_state_object_report_delta_markdown.py `
--delta <SavedStateObjectReportDeltaJson> `
--output <SavedStateObjectReportDeltaMarkdown>
```
Object change lookup:
```powershell
python scripts/list_1c_saved_state_object_changes.py `
--report <SavedStateObjectReportJson> `
--payload-role <PayloadRole> `
--active-missing true `
--output <SavedStateObjectChangeListJson>
```
```powershell
python scripts/get_1c_saved_state_object_change.py `
--report <SavedStateObjectReportJson> `
--name <ConfiguratorObjectName> `
--output <SavedStateObjectChangeJson>
```
Use `--name-b64` instead of `--name` when a shell cannot pass Unicode safely.
Rules:
- read-only SQL access only;
- compare `ConfigSave` with `Config`;
- compare `ConfigCASSave` with `ConfigCAS`;
- expose public changes as 1C objects, for example
`ОбщийМодуль.HttpBridgeКлиент`;
- keep `FileName`, byte sizes, and hashes under `storage` evidence;
- keep `root`, `versions`, and `configinfo` under `system_changes`.
- optional detail analysis can show text deltas, added/removed words, and
saved form/module string samples inside each changed 1C object.
- when extension manifest summary and active `ConfigCAS` export are provided,
detail analysis resolves `ConfigCASSave` object parts to active CAS keys and
compares saved extension payloads with active extension payloads.
- Markdown rendering is read-only and should keep 1C configurator object names
first, with SQL file names only as storage evidence.
- Agents should read `agent_summary` first, then open `detail` only when a
concrete changed payload needs inspection.
- Detail parts include `payload_role` hints such as `bsl_module_text`,
`form_descriptor`, `form_body`, `primary_payload`, or `metadata_payload`.
- Detail payloads may include `semantic_hints.added_terms` and
`semantic_hints.removed_terms`, filtered from raw word diffs to remove
technical/base64-like tokens.
- The report command runs the check command as a final contract gate and writes
`saved-state-object-report-check.json`.
- Report-to-report delta compares two observations by 1C object names and
stable payload-part fingerprints. Use it when the user continues editing and
asks what changed since the previous check.
- Delta reports should pass `onec_saved_state_object_report_delta_check.v1`
before being used as reliable agent input.
- Watch manifests should pass `onec_saved_state_watch_once_check.v1`; they
verify linked report/delta checks and summary counts.
- Watch run lists expose the latest observation, linked artifact paths, check
statuses, and delta counts without rereading SQL.
- Watch run lists should pass `onec_saved_state_watch_run_list_check.v1`
before being used as reliable agent input.
- Latest watch lookup returns the newest matching run, or `found=false` when no
run satisfies `--require-delta` or `--require-changed`. It writes a check JSON
by default when `--output` is used; use `--skip-check` to disable this.
- Object change lookup returns `matched`, `not_found`, or `ambiguous`; it must
return candidates instead of guessing when multiple objects match.
- Object change list supports compact filters by layer, kind, extension,
payload role, text diff, and active-missing state.
File diff suppressed because it is too large Load Diff
+585
View File
@@ -0,0 +1,585 @@
# 1C SQL Knowledge Adapter Notes
Date: 2026-06-20
This document records verified facts and next engineering steps for a 1C
metadata adapter that reads SQL storage transparently instead of guessing
metadata object names from physical table names.
## Current Position
The adapter must be specification- and evidence-driven:
- Read platform-maintained structure files from SQL tables such as `Params`,
`Config`, `ConfigSave`, `ConfigCAS`, and `ConfigCASSave`.
- Treat `Params/DBNames*` as the authoritative map between GUIDs, storage roles,
and numeric SQL suffixes.
- Preserve storage-role labels exactly as the platform stores them: `Document`,
`Reference`, `Fld`, `VT`, `LineNo`, `InfoRg`, and so on.
- Use XML dumps as a validation oracle and name source, not as the only runtime
source.
- Avoid semantic guesses such as "physical table name looks like document".
## UPO SQL Facts Verified
SQL inventory for the UPO database showed:
- `Config`: 59,134 rows.
- `ConfigSave`: 0 rows.
- `ConfigCAS`: 4,711 rows.
- `ConfigCASSave`: 0 rows.
- `Params`: 51 rows.
- `_ExtensionsInfo`: 18 rows.
Exported `Params/DBNames*` files:
- 12 `DBNames` files.
- 61,011 records in the main `DBNames`.
- 141 distinct storage roles.
- 52,668 distinct SQL GUIDs.
Important observed role counts:
- `Fld`: 46,396.
- `VT`: 2,081.
- `LineNo`: 2,081.
- `InfoRg`: 1,266.
- `Enum`: 1,244.
- `Reference`: 794.
- `Document`: 374.
Sample validation against physical SQL schema showed that roles such as
`Document`, `Reference`, `Enum`, `InfoRg`, and `AccumRg` can correspond to
physical tables, while roles such as `Fld`, `VT`, and `LineNo` are structural
parts and not standalone tables.
## UPO XML Cross-Check
Built a top-level XML GUID index from:
`Z:\codex\1C\XML\UPO\Структура базы 1с\Конфигурация`
The first-pass index intentionally scans only top-level metadata XML files:
- `Configuration.xml`
- `Category/ObjectName.xml`, for example `Documents/АвансовыйОтчет.xml`
It does not yet scan deep nested files such as forms, help, templates, and
layouts.
Index result:
- XML files scanned: 17,711.
- Top metadata objects found: 17,702.
- GUIDs found: 138,262.
- Parse/read errors: 0.
SQL/XML comparison result:
- SQL GUIDs: 52,668.
- XML GUIDs: 138,262.
- Matched SQL GUIDs: 51,609.
- Top-object matched SQL GUIDs: 5,227.
- Occurrence-only matched SQL GUIDs: 46,382.
- Unmatched SQL GUIDs: 1,059.
A deeper XML index also exists and scans nested XML files:
- XML files scanned: 47,577.
- Top objects found: 29,651.
- GUIDs found: 164,824.
- SQL GUIDs matched: 51,639.
- Top-object matched SQL GUIDs: 5,257.
- Occurrence-only matched SQL GUIDs: 46,382.
- Unmatched SQL GUIDs: 1,029.
The deeper pass only reduced unmatched SQL GUIDs by 30, so most remaining
unmatched records are likely extension records, service records, or objects
whose matching XML dump is not in the UPO configuration XML root.
Unmatched SQL GUIDs by source file after the deep comparison:
- `DBNames-Ext-93951a6d-3f96-11f0-9fe9-005056b59abc`: 676.
- `DBNames-Ext-1fa5d8e5-2a28-11ee-9fc3-0050569db20c`: 190.
- `DBNames-Ext-3653438a-eee5-11ef-9fe7-005056b59abc`: 115.
- `DBNames-Ext-7177af37-e5a7-11ed-9fba-0050569db20c`: 38.
- Other `DBNames-Ext-*` files: 10 or fewer each.
The unmatched distribution strongly points to extension metadata, not to
missing base configuration object XML.
After adding the XML root for UPO extensions:
`Z:\codex\1C\XML\UPO\Структура базы 1с\Расширения`
Extension XML index result:
- XML files scanned: 1,500.
- Top objects found: 1,206.
- GUIDs found: 7,776.
Combined base-configuration + extension XML comparison:
- Combined XML files: 49,077.
- Combined XML GUIDs: 171,630.
- Matched SQL GUIDs: 52,560.
- Top-object matched SQL GUIDs: 5,314.
- Occurrence-only matched SQL GUIDs: 47,246.
- Unmatched SQL GUIDs: 108.
The remaining unmatched SQL GUIDs are no longer a general metadata problem.
They are concentrated in extension records without matching XML dumps:
- `КонтурEDI`: 100 GUIDs.
- `РазвитиеФункционалаДР`: 7 GUIDs.
- `MCP1C`: 1 GUID.
The folder `ДатаМобайл_Онлайн_УНФ` has two XML candidates:
- `ДатаМобайл_Онлайн_УНФ`: 323 XML files.
- `ДатаМобайл_Онлайн_УНФ prod`: 328 XML files.
Both are treated as source variants for the same SQL extension name. The
adapter must preserve the XML source path and not collapse variants by
extension name.
## Config Object Files
Direct export from SQL `Config` by `FileName = <metadata GUID>` works for
sample metadata objects:
- `84e4c0c3-2a21-4aba-a7b0-f92b3f2878ec`: `Document.АвансовыйОтчет`.
- `40045984-a54c-42bd-8ea8-1c10672f40ec`:
`Catalog.АвансовыйОтчетПрисоединенныеФайлы`.
- `00035364-b591-4e6a-9219-e27dac18f687`:
`InformationRegister.СостоянияКонтрагентовБЭД`.
Observed storage format for these files:
- SQL table: `Config`.
- Key column: `FileName`.
- Payload column: `BinaryData`.
- Compression: raw deflate.
- Text encoding after decompression: UTF-8 with optional BOM.
- Parsed form: brace-based 1C serialized value tree.
The parsed payload contains object names, synonyms, generated type GUIDs,
references, and serialized property structures. This confirms that the adapter
can retrieve a selected metadata object directly from SQL without dumping the
whole configuration.
## Object Context Prototype
This section describes an obsolete prototype. It has been replaced by the
current resolver/context API:
- `scripts/resolve_1c_object.py`
- `scripts/get_1c_object_brief_context.py`
- `scripts/get_1c_object_metadata.py`
- `scripts/get_1c_object_artifacts.py`
- `scripts/get_1c_object_code_context.py`
- `scripts/get_1c_form_context.py`
- `scripts/get_1c_module.py`
The prototype proved that a focused context for one object by GUID or name can
include:
- DBNames source files and storage roles.
- SQL table/column candidates derived from DBNames roles.
- Extension identity when the object comes from `DBNames-Ext-*`.
- XML source candidates and source variants.
- Config file summary when that GUID has already been exported from SQL.
Validated cases:
- `84e4c0c3-2a21-4aba-a7b0-f92b3f2878ec`
(`Document.АвансовыйОтчет`): base DBNames records + Config summary.
- `00035364-b591-4e6a-9219-e27dac18f687`
(`InformationRegister.СостоянияКонтрагентовБЭД`): base DBNames records +
Config summary.
- `198392a0-d153-472e-9d97-711a09dd29e0`
(`ExchangePlan.ДатаМобайл_СписокТСД`): `DBNames-Ext-93951a6d...`,
extension `ДатаМобайл_Онлайн_УНФ`, storage role `Node`, and two XML source
variants.
- `00a81908-5d20-49bd-bdea-c8e0ba641f32`
(`Document.АвансовыйОтчет` inside `ДатаМобайл_Онлайн_УНФ` XML): XML source
variants exist, but there are no DBNames records. This is likely an extension
modification/borrowed object rather than a new SQL storage object.
This distinction is important for the final adapter: extension objects can be
new storage objects, or they can be changes to existing configuration objects.
The old live flow resolved an object, exported `Config` and `ConfigCAS` rows by
GUID, and then rebuilt the focused context. Current scripts keep the same
read-only rule and use SQL credentials only from parameters/environment
variables.
Validated live SQL export:
- `00b28285-400e-44c2-99af-3557299cfd8b`
(`Document.ЗаявкаОтпускКабинетСотрудника`): exported from `Config`, not found
in `ConfigCAS`; parsed as raw-deflate UTF-8 brace tree.
- `198392a0-d153-472e-9d97-711a09dd29e0`
(`ExchangePlan.ДатаМобайл_СписокТСД` from `ДатаМобайл_Онлайн_УНФ`): found in
`DBNames-Ext-*` and XML, but not found as a direct `Config`/`ConfigCAS`
`FileName`. Extension object payload storage needs a separate SQL decoding
path.
## ConfigCAS Content Search
Full `ConfigCAS` export is feasible for UPO:
- Files: 4,711.
- Size: 31.65 MB.
- File names: 40-character hex content-addressed keys, not metadata GUIDs.
- Compression: mostly raw deflate; a few gzip files.
Added `scripts/index_1c_sql_cas.py`.
For `ExchangePlan.ДатаМобайл_СписокТСД`
(`198392a0-d153-472e-9d97-711a09dd29e0`), direct SQL export by `FileName`
returns no `Config`/`ConfigCAS` file, but content search across `ConfigCAS`
finds 18 files containing the object GUID.
Examples:
- `ce8bf17f61c08afbb29d565cc87b000f3f4cacee`: contains object-level strings
`ДатаМобайл_СписокТСД` and `DataMobile: Список ТСД`; brace root begins with
`1`, similar to direct `Config` object payloads.
- `283eb5f07ccc27fad05c347fff1bebe70a562c8e`: large form/module payload with
handlers such as `ПриОткрытии` and `ПриСозданииНаСервере`; brace root begins
with `4`.
- Several files with brace root `10` appear to be relationship/index/reference
structures around the object.
The current CAS indexing path is handled by `scripts/index_1c_sql_cas.py` and
the unified route index builders. CAS hits are internal evidence, not a normal
agent-facing object API.
## Extension CAS Manifest
`_ExtensionsInfo._ExtensionZippedInfo` contains a direct pointer to the root
CAS file:
- First 4 bytes: observed marker.
- Next 20 bytes: raw SHA1 bytes of the extension root `ConfigCAS` file.
- Remaining bytes: small extension metadata, including a UTF-16LE text
fragment with synonym/version-like information.
`ConfigCAS.FileName` equals `SHA1(BinaryData)` for the stored compressed bytes.
This was verified against sampled `ConfigCAS` rows.
Added scripts:
- `scripts/parse_1c_extension_zipped_info.py`
- `scripts/extract_1c_extension_cas_manifest.py`
- `scripts/build_1c_extension_manifests.py`
All 18 UPO extensions have root-CAS keys, and all root-CAS files exist in
`ConfigCAS-all`.
The root-CAS file contains an object manifest:
- A root package/header block.
- A package payload block.
- A manifest list of `object-id -> base64(20-byte sha1)` pairs.
The base64 value decodes to the CAS file key. Object IDs can be plain GUIDs or
GUIDs with suffixes such as `.0`, `.1`, `.3`.
Examples:
- `ДатаМобайл_Онлайн_УНФ` root CAS:
`517f2fb723e10d966684e73dd515c798bdb4a66f`.
- `ExchangePlan.ДатаМобайл_СписокТСД`
(`198392a0-d153-472e-9d97-711a09dd29e0`) maps directly to:
- `ce8bf17f61c08afbb29d565cc87b000f3f4cacee`
- `2563bd2a2cd7df17ec86fd885c64ebfedba91b20`
- `065f05c9a874233dc0dc481213aa8241e2a564fe`
- `4233c26d2552c53727396656e432fcbe198923db`
- `КонтурEDI` root CAS:
`0739041b1cc0c8b4240f6688aa319ff74551fc74`.
- A `КонтурEDI` object
(`687571a4-0a25-4d2c-b4cb-b7317ba8de2a`) maps directly to:
- `8e9ce770a0622acc5c0dee519e4aa0acff3dce00`
- `a5847e560d1795162d5502d5a783591ffa2911e8`
This means the adapter does not need to scan all `ConfigCAS` content to fetch an
extension object. The direct path is:
`_ExtensionsInfo -> _ExtensionZippedInfo -> root_cas_key -> root manifest -> object_id -> cas_key -> ConfigCAS.BinaryData`
## Extension Object Part Evidence
Added:
- `scripts/classify_1c_manifest_payloads.py`
- `scripts/analyze_1c_manifest_object_parts.py`
Important rule: manifest suffixes such as `.0`, `.1`, and `.3` are not accepted
as semantic labels by themselves. A suffix is only an object part key. The
adapter must identify the part from the payload grammar and direct evidence:
root shape, embedded block markers, declared stream lengths, hashes, and
matches against XML/BSL/HTML exports.
The broad structural profile across all UPO extension manifests found:
- 3,832 manifest entries.
- 9 suffix groups.
- Empty suffix: 2,174 entries.
- `.0`: 1,397 entries.
- `.1`: 65 entries.
- `.2`: 175 entries.
- `.3`: 11 entries.
The same suffix can carry different evidence, so a rule like ".0 means module"
is invalid. For example, `.0` entries include base64 blocks, BSL text, HTML, and
payloads with no simple textual evidence.
Object-level comparison was validated for `ExchangePlan.ДатаМобайл_СписокТСД`
(`198392a0-d153-472e-9d97-711a09dd29e0`) from extension
`ДатаМобайл_Онлайн_УНФ`.
Manifest entries:
- `198392a0-d153-472e-9d97-711a09dd29e0` ->
`ce8bf17f61c08afbb29d565cc87b000f3f4cacee`
- `198392a0-d153-472e-9d97-711a09dd29e0.0` ->
`2563bd2a2cd7df17ec86fd885c64ebfedba91b20`
- `198392a0-d153-472e-9d97-711a09dd29e0.1` ->
`065f05c9a874233dc0dc481213aa8241e2a564fe`
- `198392a0-d153-472e-9d97-711a09dd29e0.3` ->
`4233c26d2552c53727396656e432fcbe198923db`
Verified evidence:
- The base object part has brace root marker `1`, root length `8`, and contains
object-level strings and GUIDs that match
`ExchangePlans/ДатаМобайл_СписокТСД.xml`.
- The `.0` part has brace root marker `5`, root length `5`, and contains one
`#base64` block. After UTF-8 decoding and line-ending normalization, that
block equals `Ext/Help/ru.html`.
- The `.1` part has brace root marker `2`, root length `71`, and contains GUID
pairs/flags. In this object-level XML comparison those GUIDs did not match
the local XML files, so its role is still structural evidence only.
- The `.3` part is not a single brace text. It contains stream headers matching
the observed byte pattern `hhhhhhhh hhhhhhhh 7fffffff`, where the hexadecimal
values declare stream sizes. One stream has declared size `00000de3`
(`3555` bytes) and matches `Ext/ManagerModule.bsl` by SHA1 exactly.
Broader extension scan:
- `scripts/summarize_1c_extension_manifest_xml_parts.py`
- `reports/1c-sql/upo/extension-manifest-xml-part-summary.json`
The scan covers all 18 extension manifests and available XML dumps. It confirms
that the same structural patterns appear across multiple extensions:
- `root=1, len=8`: top-level metadata object descriptions, matching the same
shape as base `Config` objects such as documents and catalogs.
- `root=1, len=3`: child metadata objects such as forms/templates represented
by their own GUIDs in the extension manifest.
- `root=4, len=10` and `root=4, len=11`: form payload structures.
- `root=8` with `MOXCEL`: spreadsheet/template payload structures.
- `stream_headers`: container payloads with declared byte lengths; one verified
case matches a BSL module by SHA1.
- `#base64` blocks: embedded textual payloads; one verified case matches HTML
help after UTF-8 decoding and line-ending normalization.
Available XML/manifest coverage examples:
- `ДатаМобайл_Онлайн_УНФ`: 358 manifest entries, 2 XML source variants,
2,907 XML GUIDs, 265 XML GUIDs matched to manifest object IDs.
- `ДоработкаРарус`: 461 manifest entries, 405 XML files, 315 XML GUIDs matched.
- `РасширениеДляВыгрузкиВоФронты`: 68 manifest entries, 51 XML GUIDs matched.
- `РасширениеДляЗагрузкиИзФронтов`: 97 manifest entries, 81 XML GUIDs matched.
- `фс_ДоработкиОбщее`: 120 manifest entries, 95 XML GUIDs matched.
- `фс_Отчеты`: 137 manifest entries, 98 XML GUIDs matched.
Extensions without XML dumps can still be decoded through the root manifest and
CAS files, but their semantic mapping must be proven from payload structure,
DBNames roles, and eventually generated/exported XML:
- `КонтурEDI`
- `РазвитиеФункционалаДР`
- `MCP1C`
- `ИнструментыРазработчикаTormozit`
- `Модуль1СEDI`
## Base Config Object Evidence
Added:
- `scripts/analyze_1c_config_object_xml.py`
The base configuration `Config` table uses the same object-description grammar
observed for base extension object parts:
- SQL table: `Config`.
- Key: `FileName = metadata GUID`.
- Payload: raw-deflate data.
- Text: UTF-8 with BOM.
- Parsed structure: brace tree.
Validated base objects:
- `Document.АвансовыйОтчет`
(`84e4c0c3-2a21-4aba-a7b0-f92b3f2878ec`): `root=1`, `len=8`,
strings/GUIDs match `Documents/АвансовыйОтчет.xml`.
- `Catalog.АвансовыйОтчетПрисоединенныеФайлы`
(`40045984-a54c-42bd-8ea8-1c10672f40ec`): `root=1`, `len=8`,
strings/GUIDs match `Catalogs/АвансовыйОтчетПрисоединенныеФайлы.xml`.
- `InformationRegister.СостоянияКонтрагентовБЭД`
(`00035364-b591-4e6a-9219-e27dac18f687`): `root=1`, `len=9`,
strings/GUIDs match `InformationRegisters/СостоянияКонтрагентовБЭД.xml`.
- `Document.ЗаявкаОтпускКабинетСотрудника`
(`00b28285-400e-44c2-99af-3557299cfd8b`): `root=1`, `len=8`,
strings/GUIDs match `Documents/ЗаявкаОтпускКабинетСотрудника.xml`.
This supports the working model: extension manifests add a CAS-addressed
package layer, but the actual metadata object description payload largely uses
the same grammar as base `Config` object files.
## Unified Route Index
Added:
- `scripts/build_1c_unified_object_route_index.py`
- `scripts/query_1c_unified_object_route.py`
The route index is the first adapter-shaped artifact. For each GUID it records
only deterministic routes and observed payload signatures:
- DBNames storage records.
- XML top-object records and occurrence count.
- Direct base `Config` routes when a `Config` payload file is available.
- Extension manifest/CAS routes with `object_id`, suffix, CAS key, root CAS
file, and payload signature.
Generated report:
- `reports/1c-sql/upo/unified-object-route-index.json`
Result:
- Indexed GUID nodes: 173,034.
- GUIDs with DBNames storage: 52,668.
- GUIDs with XML top-object records: 30,587.
- GUIDs with extension manifest/CAS routes: 2,253.
- GUIDs with currently exported direct base Config routes: 4.
Control routes:
- `Document.АвансовыйОтчет`
(`84e4c0c3-2a21-4aba-a7b0-f92b3f2878ec`): route kinds
`base_config_direct`, `dbnames_storage`, `xml_top_object`.
- `ExchangePlan.ДатаМобайл_СписокТСД`
(`198392a0-d153-472e-9d97-711a09dd29e0`): route kinds
`extension_manifest_cas`, `dbnames_storage`, `xml_top_object`.
- `Form.ФормаВыбора`
(`db752e57-6327-4bac-a97a-d5aef0004302`): route kinds
`extension_manifest_cas`, `xml_top_object`; no DBNames storage record because
it is a metadata child object rather than a separate SQL data table.
- `Template.Бейджи`
(`1ea5e231-9d10-4316-9239-6f72882aeb4a`): route kinds
`extension_manifest_cas`, `xml_top_object`; `.0` payload marker is `MOXCEL`.
- `КонтурEDI` object
(`687571a4-0a25-4d2c-b4cb-b7317ba8de2a`): route kinds
`extension_manifest_cas`, `dbnames_storage`; XML is absent, but the object is
still directly reachable through SQL/CAS.
This is the desired runtime direction: the agent asks the adapter for a route,
then fetches only the required payload instead of scanning the whole
configuration.
Top matched XML kinds:
- `Enum`: 1,238.
- `InformationRegister`: 1,237.
- `Constant`: 1,126.
- `Catalog`: 779.
- `Document`: 371.
- `ScheduledJob`: 212.
- `AccumulationRegister`: 142.
This confirms the main approach: SQL `DBNames` GUIDs can be connected to real
configuration objects and Russian names through XML GUIDs without deriving
meaning from `_DocumentNN` or `_ReferenceNN` table names.
## Current Artifacts
This file is now historical research background. The active API contract is
`docs/1c-adapter-api-contract.md`; the active tool list is
`plugins/1c/tools/README.md`.
Current source/evidence builders:
- `scripts/export_1c_sql_files.ps1`
- `scripts/export_1c_sql_extensions_info.ps1`
- `scripts/inspect_1c_sql_files.py`
- `scripts/extract_1c_dbnames.py`
- `scripts/build_1c_xml_guid_index.py`
- `scripts/compare_1c_sql_xml_guids.py`
- `scripts/index_1c_sql_cas.py`
- `scripts/parse_1c_extension_zipped_info.py`
- `scripts/extract_1c_extension_cas_manifest.py`
- `scripts/build_1c_extension_inventory.py`
- `scripts/build_1c_extension_manifests.py`
- `scripts/build_1c_unified_object_route_index.py`
Current agent-facing adapter scripts:
- `scripts/resolve_1c_object.py`
- `scripts/get_1c_object_brief_context.py`
- `scripts/get_1c_object_metadata.py`
- `scripts/search_1c_object_context.py`
- `scripts/get_1c_object_artifacts.py`
- `scripts/get_1c_object_code_context.py`
- `scripts/get_1c_module.py`
- `scripts/get_1c_form_context.py`
- `scripts/read_1c_object_view.py`
- `scripts/plan_1c_task_context.py`
- `scripts/build_1c_task_evidence.py`
- `scripts/propose_1c_task_changes.py`
- `scripts/create_1c_patch_workspace.py`
- `scripts/check_1c_patch_preflight.py`
Current core reports:
- `reports/1c-sql/upo/dbnames.json`
- `reports/1c-sql/upo/xml-guid-index-combined.json`
- `reports/1c-sql/upo/sql-xml-guid-compare-combined.json`
- `reports/1c-sql/upo/unified-object-route-index.json`
- `reports/1c-sql/upo/structured-metadata-all-kinds-dbnames-summary.json`
- `reports/1c-sql/upo/predicted-column-validation.json`
- `reports/1c-sql/upo/enum-presentation-map.json`
Current task sample reports:
- `reports/1c-sql/upo/object-brief-context-prihodnaya-nakladnaya-effective.json`
- `reports/1c-sql/upo/object-metadata-prihodnaya-nakladnaya-effective.json`
- `reports/1c-sql/upo/form-context-prihodnaya-nakladnaya-effective.json`
- `reports/1c-sql/upo/module-prihodnaya-nakladnaya-object-effective.json`
- `reports/1c-sql/upo/task-plan-prihodnaya-nakladnaya-ceny-srok.json`
- `reports/1c-sql/upo/task-evidence-prihodnaya-nakladnaya-ceny-srok.json`
- `reports/1c-sql/upo/task-change-proposal-prihodnaya-nakladnaya-ceny-srok.json`
- `reports/1c-sql/upo/patch-workspaces/prihodnaya-nakladnaya-ceny-srok/manifest.json`
- `reports/1c-sql/upo/patch-preflight-prihodnaya-nakladnaya-ceny-srok.json`
## Open Risks
- Some extension XML exports may be absent; SQL/CAS evidence must then fill the
gap without guessing from physical table names.
- Direct SQL writes to configuration storage are not accepted as safe. The
supported write path is an extension patch workspace plus human/apply gates.
- More object kinds need read-view smoke coverage, but the adapter API must
remain kind-neutral.
## Next Steps
1. Keep the current adapter surface stable: resolve by 1C name, then read
metadata, form, module, data view, and patch workspace evidence.
2. Expand SQL/CAS-backed form/module extraction where XML export evidence is
missing.
3. Add regression tests for all major metadata object kinds through the same
`effective/base/extension` view model.
+688
View File
@@ -0,0 +1,688 @@
# 1C Write Path Safety
Status: active configuration writes are blocked by design. Saved-state form
writes are allowed only through the explicit proposal/apply gates described
below.
Layer and addressing work plan: `docs/1c-extension-layer-plan.md`.
The adapter may read SQL, Config, ConfigCAS, XML exports, and extension
manifests. It must not write active SQL metadata/data tables or active Config
payloads until the following gates are implemented and tested. The only current
write exception is saved-state form payload editing in `ConfigSave` and
`ConfigCASSave`.
## Required Gates
1. Backup gate
Every write run must record database, configuration state, extension state,
SQL transaction boundary, and rollback instructions before changing
anything.
2. Round-trip parser gate
The adapter must decode target payload, encode it back without semantic
changes, and prove byte/structural equivalence where applicable.
3. Designer validation gate
Any metadata write must be validated by 1C tools, not only by SQL shape.
4. Saved-state gate
The adapter must distinguish active configuration from saved but not applied
state, including `ConfigSave` and extension saved-state equivalents.
5. Extension packaging gate
Extension changes must preserve manifest/CAS relationships and validate
extension load/update behavior.
6. Diff gate
Every proposed write must produce a human-readable diff at metadata level and
physical storage level.
7. Minimal write scope gate
Writes must target the smallest possible object or payload. Bulk Config
rewrites are forbidden unless no narrower route exists.
8. Recovery test gate
Restore/rollback must be tested on a disposable database before production
write support is enabled.
9. Full-path target gate
Every write intent must resolve to a full 1C path or a concrete saved-state
reference before planning. Short names and local BSL symbols are acceptable
only after they are bound to an object, form, module, routine, or query
context.
10. Layer provenance gate
Every write plan must identify whether the effective element comes from the
base configuration, a selected extension, generated extension source, or a
saved-state working copy. Effective text alone is not a writable target.
11. Extension control gate
Extension code operations must classify insert-before, insert-after,
replace, and replace-with-control separately. Replace-with-control writes
must verify the controlled fragment against the current source before apply.
## Allowed Now
- read metadata from SQL/XML/CAS;
- build object read views;
- resolve references;
- inspect forms/modules/templates;
- generate proposed code or metadata patches as files;
- produce diffs and implementation plans;
- plan saved-state form edits with `metadata.write` or
`metadata.form.element.write`;
- apply saved-state form proposals only when all are true:
`allow_sql_saved_state_apply=true`, the proposal contains
`validation.mode=path_preserve_format`, current sha1 matches, and backup
evidence is written before update;
- rollback saved-state form writes through `storage.saved_state.rollback`.
- build read-only write plans that resolve full 1C paths, provenance, and
required guards before a future apply route is selected.
- use `metadata.write` as the normal agent-facing BSL write facade. It accepts
canonical 1C paths, module/routine names, and code text, defaults to saving
into `ConfigSave`/`ConfigCASSave`, and hides physical storage details unless
`include_storage=true`. `code.write` remains a compatibility shortcut for
simple module edits.
## Forbidden Now
- direct updates to `_Reference*`, `_Document*`, register tables, `Config`,
`ConfigCAS`, or active extension CAS tables;
- saved-state writes that rewrite the whole serialized form payload instead of
patching exact scalar token spans;
- agent-facing code writes that require callers to know SQL tables, file names,
stream indexes, or brace paths;
- automatic Designer update/apply;
- extension rebuild/writeback;
- direct writes to effective views without layer provenance and write-plan
evidence;
- treating a local BSL expression such as `<Переменная>.<Поле>` as a metadata
path without code-symbol resolution in the current context;
- any write action that does not have rollback evidence.
## Saved-State Form Write Contract
Saved-state form writes must:
- resolve the display source before editing. If an element caption is empty and
inherited from a linked command, write the command caption, not the empty
element caption;
- use the form property registry for aliases and verification behavior instead
of ad-hoc property matching;
- return registry metadata for decoded parameter properties, including stable
canonical names such as `visible`, `enabled`, and `command_bar_location`;
- avoid silently writing a local element title when the displayed caption is
derived from `ПутьКДанным`; local title override must be explicit with
`source=local_override`;
- when an empty element title is derived through `ПутьКДанным`, route by source:
form attribute -> write the form attribute title; tabular form attribute field
such as `ТЗ.К1` -> write the field title; object/configuration attribute
(`Объект.<Реквизит>`) -> write the local form element title;
- patch a single scalar brace token in the original decoded text and preserve
surrounding formatting/compression envelope;
- return a semantic diff and physical sha1/byte metadata before apply;
- create rollback evidence under `ONEC_ADAPTER_BACKUP_DIR` or
`/data/adapter-apply-backups`;
- verify readback sha1 and re-decode the form for semantic verification.
## Saved-State BSL Write Contract
Agents edit modules as text, at the same level as a human programmer:
- read current code from the working/save state;
- replace the whole module with `module_text`, `full_text`, or `code`;
- replace one procedure/function with `routine_name` and `routine_text`;
- replace a smaller fragment with `old` and `new`. If `routine_name` or a
routine-level `canonical_path` is provided, uniqueness is checked inside that
procedure/function; otherwise `old` must occur exactly once in the current
saved module text.
The adapter maps that request to the physical saved-state route. For ordinary
module streams it uses the saved-state module writer. For embedded form modules
it patches only the module scalar token in the form payload and verifies that
the encoded payload changed no more than that token. If a fragment repeats in
the selected scope, the adapter must return `ambiguous_fragment` with `scope`
and `counts.occurrences` instead of choosing an occurrence.
## Saved-State Preparation Gate
An empty `ConfigSave` or `ConfigCASSave` means there are no unactivated
Configurator changes in that save layer. To test or apply saved-state writes
for an object, first prepare a working copy from the active storage family:
`Config -> ConfigSave` for base configuration objects, or
`ConfigCAS -> ConfigCASSave` for extension/CAS objects.
The preparation flow is intentionally split into reviewable stages:
```text
python scripts/plan_1c_saved_state_copy.py --base-id upo_test --target-table ConfigSave --report reports/1c-sql/upo_test/saved-state-copy-plan.json --json
python scripts/prepare_1c_saved_state_copy_sql.py --plan reports/1c-sql/upo_test/saved-state-copy-plan.json --expected-base-id upo_test --expected-target-table ConfigSave --sql-out reports/1c-sql/upo_test/prepare-saved-state-copy.sql --report reports/1c-sql/upo_test/prepare-saved-state-copy-sql.json --json
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\execute_1c_saved_state_copy_sql.ps1 -ExpectedBaseId upo_test -ExpectedTargetTable ConfigSave -IUnderstandThisWritesToSql
python scripts/verify_1c_saved_state_copy.py --plan reports/1c-sql/upo_test/saved-state-copy-plan.json --expected-base-id upo_test --expected-target-table ConfigSave --require-ready --json
```
Only `execute_1c_saved_state_copy_sql.ps1` performs SQL writes, and it requires
the explicit `-IUnderstandThisWritesToSql` gate. Normal adapter verification
generates the plan and reviewed SQL artifacts but does not execute them.
## Saved-State Write Route Smoke
After changing saved-state form write routing or redeploying the REST adapter,
run:
```text
python scripts/smoke_1c_saved_state_write_routes.py \
--report reports/1c-sql/upo_test/saved-state-write-routes-smoke.json
```
The smoke test uses only the REST adapter URL and performs `apply_and_rollback`
against `upo_test`. It must verify these route families:
- form element with `ПутьКДанным = А` writes the form attribute title;
- form element with `ПутьКДанным = ТЗ.К1` writes the tabular form attribute
field title;
- `command=КомандаПример1` writes the command title directly.
## Saved-State Write Matrix
To automatically enumerate decoded scalar values and learn which ones can be
changed safely, use the write matrix methods:
```text
python scripts/smoke_1c_write_matrix.py \
--max-candidates 50 \
--report reports/1c-sql/upo_test/write-matrix-smoke-50.json
```
For a full verified registry on the current `upo_test` learning form:
```text
python scripts/smoke_1c_write_matrix.py \
--max-candidates 1000 \
--report reports/1c-sql/upo_test/write-matrix-smoke-full.json
python scripts/build_1c_write_matrix_verified_registry.py \
--smoke-report reports/1c-sql/upo_test/write-matrix-smoke-full.json \
--output reports/1c-sql/upo_test/write-matrix-verified-registry.json \
--include-route-evidence
python scripts/build_1c_write_matrix_enum_registry.py \
--matrix-report reports/1c-sql/upo_test/write-matrix-build-v4.json \
--output reports/1c-sql/upo_test/write-matrix-enum-registry.json
python scripts/build_1c_write_matrix_scalar_registry.py \
--matrix-report reports/1c-sql/upo_test/write-matrix-build-v4.json \
--output reports/1c-sql/upo_test/write-matrix-scalar-registry.json
python scripts/build_1c_write_learning_plan.py \
--registry reports/1c-sql/upo_test/write-matrix-scalar-registry.json \
--output reports/1c-sql/upo_test/write-learning-plan-scalar.json
python scripts/analyze_1c_write_matrix_structural_diff.py \
--before reports/1c-sql/upo_test/write-matrix-build-v4.json \
--after reports/1c-sql/upo_test/write-matrix-build-after-move-a.json \
--output reports/1c-sql/upo_test/write-matrix-structural-diff-move-a-after-b.json
```
```json
{"method":"metadata.form.write_matrix.build","payload":{"base_id":"upo_test","table":"ConfigCASSave","file_name":"<form-file-name>"}}
```
```json
{"method":"metadata.form.write_matrix.smoke","payload":{"base_id":"upo_test","table":"ConfigCASSave","file_name":"<form-file-name>","allow_sql_saved_state_apply":true,"allow_sql_saved_state_rollback":true,"max_candidates":10,"learning_id":"upo-test-write-matrix"}}
```
`build` returns every decoded writable scalar candidate with `can_smoke` and a
reason when it is not safe for generic smoke. `smoke` runs only `can_smoke`
entries through `apply_and_rollback`, records verification status, and stores a
report under `ONEC_ADAPTER_WRITE_LEARNING_DIR` when `learning_id` is passed.
Matrix entries include `semantic_name`, `semantic_group`, and `semantic_source`
when the decoded form semantic map knows the parameter, so gap analysis can
group unresolved properties by configurator-facing meaning instead of only by
physical `Параметр N`.
For empty local strings, `build` also adds `codec_probe` diagnostics. Direct
empty `string` nodes are smoke-safe; composite empty string/list nodes remain
classified as `composite_node_requires_semantic_rule` after probing because
they usually represent typed structures such as layout, decoration, references,
or containers rather than plain strings.
Current generic smoke intentionally skips identity/binding values (`id`, `name`,
`ПутьКДанным`), enum/color values without known allowed sets, and composite
nodes that need a source-specific semantic write rule.
The scalar registry separates these skipped scalar values into learning queues:
`run_before_after_learning_for_parameter` is safe to learn with manual
Configurator before/after captures, `learn_reference_write_rule` needs a
source-specific reference/container rule, and `do_not_generic_write` remains
manual-only because it covers identity or structural values.
The learning plan turns the scalar or enum registry into an ordered queue of
manual learning cases. Each case contains a selector, current value, physical
write path, and the workflow:
`capture_before -> manual_configurator_change -> capture_after -> diff ->
infer_rule -> smoke_rule`.
Moving form elements is a structural operation, not a scalar property write.
`metadata.write_learning.diff` reports these as `target_moves` by stable target
identity (`section`, `name`, `id`) when a target path changes. Property
inference then returns `structural_move_rule_required` until a dedicated
saved-state move/reorder writer is implemented and smoke-tested.
The first structural writer is `metadata.form.target.move`. It currently
supports the learned safe primitive `swap_sibling_slots`: two form item nodes in
the same parent container are swapped by byte-preserving brace-text ranges
(`swap_paths`) and then passed through the same saved-state proposal,
backup/apply, and rollback gates as scalar writes. Example:
```json
{"method":"metadata.form.target.move","payload":{"base_id":"upo_test","table":"ConfigCASSave","file_name":"<form-file-name>","from_element":"А","to_element":"Б","allow_saved_state_write":true,"mode":"apply_and_rollback","allow_sql_saved_state_apply":true,"allow_sql_saved_state_rollback":true}}
```
The second structural primitive is `append_child`, used by
`metadata.form.command_button.write` to add a form command and a visible command
bar button. The method clones existing command/button nodes from the same form,
replaces name/title/action/GUID fields, appends the new nodes with
byte-preserving brace-text insertion, updates declared section counts such as
`{marker,count,record...}` when the append target uses them, then uses the same
proposal, backup, apply, verify, and rollback gates. If the save layer is empty, `plan` exposes
`metadata.saved_state.prepare`; apply modes with `allow_sql_saved_state_apply`
prepare saved state internally before retrying.
After `apply`/`apply_and_verify`, the method re-decodes the saved-state form
and returns `semantic_verify`: command present, button present, handler routine
present, command->handler link, button->command link, and the decoded
command/button paths. Repeated `upsert` calls are idempotent: if command and
button already exist, the method reports `idempotency.status=already_exists`,
updates/verifies the handler, and does not append duplicate structural nodes.
Successful saved-state module changes refresh `metadata_code_index_cache` and
vector chunks for the embedded form module, while SQL remains the freshness
source of truth.
Use `metadata.form.command_button.verify` when no write is needed. It accepts
the same public selectors (`extension`, `form`, `command_name`, `button_name`,
`handler_name`) and returns the same command/button/handler/link checks from
current SQL saved-state. Public command/button write and verify responses keep
saved-state search compact: `counts` plus `selected_form`, not the full list of
similar form candidates. Public form node addresses are exposed as `form_path`;
physical SQL `file_name` remains opt-in through `include_storage=true`.
By default the same method also upserts the form-module handler routine named by
`command_action`. For SQL form payloads where the module is embedded as a scalar
BSL string, the writer edits module path `2` with `replace_routine_text` and a
byte-preserving scalar proposal. Pass `include_handler=false` only for
diagnostics that intentionally validate the structural form append alone.
```json
{"method":"metadata.form.command_button.write","payload":{"base_id":"upo_test","extension":"test2","object_type":"CommonForm","object_name":"t_Форма","command_name":"РасчетС","command_title":"РасчетС","command_action":"РасчетС","allow_saved_state_write":true,"mode":"plan"}}
```
For ordinary agent work, prefer the high-level `metadata.write` entrypoint when
only the form module routine must be changed. It resolves
`ОбщаяФорма.<Form>.<Routine>` to the saved-state form payload, treats missing
routine names in `upsert` mode as an owner-only module search, edits embedded
module path `2`, and keeps the save-first defaults on apply modes:
```json
{"method":"metadata.write","payload":{"base_id":"upo_test","target":{"canonical_path":"ОбщаяФорма.t_Форма.РасчетС"},"mode":"apply_and_rollback","routine_operation":"upsert","routine_text":"&НаКлиенте\nПроцедура РасчетС(Команда)\n\t// ...\nКонецПроцедуры\n","allow_sql_saved_state_rollback":true}}
```
For form commands/buttons, `metadata.write` can route explicit paths such as
`ОбщаяФорма.<Form>.Команда.<Command>` or
`ОбщаяФорма.<Form>.Кнопка.<Button>` to
`metadata.form.command_button.write`. For existing form properties, use
`metadata.form.write_target.verify` first; it reports whether the saved-state
target is writable now or whether the adapter would need to prepare Save.
Every write call made through the public adapter entrypoint, including
`code.write`, receives an `operation_id`. Use `metadata.write.history` to
retrieve recent operations, backup ids, routed method, and verification result.
History accepts `operation_method`, `status`, `routed_method`, and `backup_id`
filters for quick audit lookup. Pass `include_summary=true` to get aggregate
counts by method, status, routed method, and operations with backups.
Use `metadata.write.rollback` to rollback by `operation_id` or `backup_id`;
it still requires the explicit `allow_sql_saved_state_rollback=true` gate.
Deployment verification runs `scripts/smoke_1c_write_rollback_safety.py` for
REST and MCP. The smoke does not apply rollback; it checks that
`metadata.write.rollback` is exposed, `metadata.write.history` is readable, and
rollback without the explicit gate returns `invalid_argument` for
`allow_sql_saved_state_rollback`.
To inspect what is currently pending in Save, use `metadata.saved_state.diff`.
It is read-only and compares the saved payload with its active SQL source:
```json
{"method":"metadata.saved_state.diff","payload":{"base_id":"upo_test","table":"ConfigCASSave","file_name":"<saved-state-file-name>","max_text_diff_lines":80}}
```
If the saved payload is missing, the result is `status=not_found` with
`needs_prepare=true` and a `metadata.saved_state.prepare` plan payload. If both
layers exist, the result is `changed` or `unchanged` with live SQL hashes.
Deployment verification runs `scripts/smoke_1c_saved_state_diff.py` for REST
and MCP and persists `saved-state-diff-smoke.json` plus
`saved-state-diff-mcp-smoke.json`.
For a save-layer overview, use `metadata.saved_state.status` before drilling
into a single file:
```json
{"method":"metadata.saved_state.status","payload":{"base_id":"upo_test","table":"ConfigCASSave","limit":500}}
```
It returns row/file counts and classifies files as `changed`, `unchanged`, or
`saved_only`. Each file includes a `diff_selector` for `metadata.saved_state.diff`.
For readback by public name, `metadata.form.decode` honors
`source_state=working`/`state=save`: it searches `ConfigCASSave`/`ConfigSave`
first and uses `extension` to narrow extension forms before falling back to
active metadata routes.
## Saved-State Copy Learning
An empty `ConfigSave`/`ConfigCASSave` means there is no pending Configurator
working copy. It does not mean the object is missing; the active object remains
in `Config`/`ConfigCAS`. To change an object through the saved-state write
path, first copy that same target object from the main configuration storage
into the save layer (`ConfigSave`/`ConfigCASSave`), then apply changes to the
save object and keep rollback evidence. To learn or verify the copy shape
safely, take a save-layer snapshot, create one minimal pending Configurator
change without applying it, then take another snapshot and compare the created
rows.
Before preparing a strict write smoke, generate a read-only copy plan:
```text
python scripts/plan_1c_saved_state_copy.py \
--base-id upo_test \
--target-table ConfigSave \
--report reports/1c-sql/upo_test/saved-state-copy-plan.json
```
The plan must be `plan_ready`, list active source rows from the matching storage
family, and report `target_collisions.status=clear`: `Config -> ConfigSave` for
base objects, `ConfigCAS -> ConfigCASSave` for extension/CAS objects. The
persisted report validator checks this plan by default, so a stale, colliding,
or family-mismatched save-layer target fails before any strict write smoke is
enabled.
After reviewing the copy plan, generate a guarded SQL preparation script without
executing it:
```text
python scripts/prepare_1c_saved_state_copy_sql.py \
--plan reports/1c-sql/upo_test/saved-state-copy-plan.json \
--expected-base-id upo_test \
--expected-target-table ConfigSave \
--sql-out reports/1c-sql/upo_test/prepare-saved-state-copy.sql \
--report reports/1c-sql/upo_test/prepare-saved-state-copy-sql.json
```
The generated script repeats the collision check inside a transaction, copies
only the reviewed `FileName` values, verifies the inserted row count, and
commits only if those guards pass.
After the SQL preparation is executed, verify that the save layer contains the
reviewed bytes:
```text
python scripts/verify_1c_saved_state_copy.py \
--plan reports/1c-sql/upo_test/saved-state-copy-plan.json \
--expected-base-id upo_test \
--expected-target-table ConfigSave \
--report reports/1c-sql/upo_test/saved-state-copy-verify.json \
--require-ready
```
Before executing the SQL preparation this should fail with
`blocked_missing_target_rows`; after preparation it must pass before strict
readiness or write-and-rollback smoke is enabled.
Generate the guarded cleanup script before executing the preparation SQL:
```text
python scripts/prepare_1c_saved_state_cleanup_sql.py \
--plan reports/1c-sql/upo_test/saved-state-copy-plan.json \
--expected-base-id upo_test \
--expected-target-table ConfigSave \
--sql-out reports/1c-sql/upo_test/cleanup-saved-state-copy.sql \
--report reports/1c-sql/upo_test/cleanup-saved-state-copy-sql.json
```
The cleanup script deletes only the reviewed `FileName`/`PartNo` rows from
`ConfigSave`, checks their `BinarySHA1` first, and rolls back if the save-layer
rows no longer match the plan.
Check strict readiness against the same target table:
```text
python scripts/check_1c_saved_state_strict_readiness.py \
--base-id upo_test \
--saved-state-table ConfigSave \
--report reports/1c-sql/upo_test/saved-state-strict-readiness.json
```
Observed on `upo_test`:
- a saved form edit created three `ConfigCASSave` rows:
`<object_guid>__<form_guid>`, `<object_guid>__<form_guid>.0`, and
`<object_guid>__configinfo`;
- a saved object-module edit added two module rows:
`<object_guid>__<module_guid>` and `<object_guid>__<module_guid>.0`, and
updated `<object_guid>__configinfo`;
- `ConfigSave` remained empty for these cases.
Direct saved-state module reads should use a concrete stream when known:
```json
{"method":"modules.read","payload":{"base_id":"upo_test","module_ref":"ConfigCASSave:<object_guid>__<module_guid>.0#stream:4","preview":true}}
```
For direct `ConfigSave`/`ConfigCASSave` module refs the adapter uses a synthetic
saved-state owner context and skips extension owner scans.
Do not pass a saved-state form payload container such as
`ConfigCASSave:<extension_guid>__<form_guid>.0` to
`metadata.module.write_apply` as if it were a concrete BSL stream. That ref is
readable for embedded form module analysis, but stream writes require
`#stream:<index>` or an explicit `stream_index`. If the ref has no stream index,
`metadata.write.plan` must return `ready_for_apply_method=false` for the stream
writer and include `next_resolution`.
The high-level `metadata.write` entrypoint may still accept such a saved-state
form payload container when the request identifies one routine with
`routine_name` and `routine_text`. In that case it must route to the embedded
form payload writer (`form_embedded_module_handler_write_apply`), edit the BSL
text at the decoded form module path, and preserve the payload format instead
of rewriting the serialized form payload as plain module text. For routine
replacement in form modules, existing leading BSL directives such as
`&НаКлиенте` are preserved when the replacement routine text omits a directive.
When the concrete module stream is not known, search saved-state modules first:
```json
{"method":"metadata.saved_state.modules.search","payload":{"base_id":"upo_test","tables":["ConfigCASSave"],"owner_guid":"<object_guid>","query":"<text-fragment>","limit":10}}
```
The search also accepts public object selectors. When
`object_type`/`object_name` resolve to one object, the adapter narrows the
saved-state scan by that owner GUID:
```json
{"method":"metadata.saved_state.modules.search","payload":{"base_id":"upo_test","tables":["ConfigCASSave"],"object_type":"Catalog","object_name":"Номенклатура","query":"ПередЗаписью","limit":10}}
```
The search result returns `streams[].module_ref`, `streams[].write_plan_target`,
payload sha1, stream encoding, and a short preview. Use `write_plan_target` as
the concrete module target for `metadata.write.plan`, then use the accepted plan
with `metadata.module.write_apply` or the generic `metadata.write` module
target.
For saved-state form payloads, the search also detects the embedded form module
stored inside the decoded form payload. These rows return
`payload.role=form_embedded_module_payload`, `streams[].module_path`, and a
`streams[].write_plan_target.module_ref` without `#stream`. That target is for
the generic `metadata.write` embedded-form route, not for
`metadata.module.write_apply`. When `object_type=CommonForm` and
`object_name` are supplied, module search first resolves the exact saved-state
form file through `metadata.saved_state.forms.search`, so similarly named forms
such as `t_Форма` and `tt_Форма3` do not broaden the module scan.
The generic `metadata.write` module target can use that selector-only path
directly. A form routine replacement may pass `target.kind=module`,
`target.object_type=CommonForm`, `target.object_name`, `routine_name`, and
`routine_text` without a `module_ref`; the adapter resolves the saved form
file, detects `form_embedded_module_payload`, and routes to the embedded-form
writer.
For common forms, the same path is available from a canonical path such as
`ОбщаяФорма.t_Форма.ЗаменаДомена`. The planner classifies it as a module
routine on a common form (`path_kind=module_routine`,
`section=form_module`), and `metadata.write` may resolve it through
`metadata.saved_state.modules.search` before routing to the embedded-form
writer.
Form module payload writes must never canonicalize the whole form payload. A
prior unsafe write showed that replacing path `2` with a normal structural
`path` edit can make Designer reject the saved form even when the adapter can
decode it. Embedded form module apply is therefore guarded in two places:
`form_embedded_module_handler_write_apply` verifies that the encoded payload is
exactly one byte-preserving scalar replacement, and
`storage.saved_state.apply_proposal` blocks form module path `2` edits unless
their edit mode is `path_preserve_format`.
Some saved form module scalars contain a trailing container marker such as
`///----`. This is not agent-facing BSL text. Public `code.read` and
`code.search` hide the trailing marker, while the embedded form module writer
preserves it automatically when an agent replaces the full module text.
Saved-state module writes use stream edits, not brace-path edits. Smoke the
route with apply-and-rollback:
```text
python scripts/smoke_1c_saved_state_module_write.py \
--file-name <object_guid>__<module_guid>.0 \
--stream-index 4 \
--expected-sha1 <current-payload-sha1> \
--report reports/1c-sql/upo_test/module-stream-write-smoke-script.json
```
The proposal must validate the BSL stream, update the stream header, apply with
a saved-state backup, verify readback sha1, and rollback to the original sha1.
If the disposable base currently has no pending Configurator saved-state rows,
pass `--allow-empty-saved-state` so the smoke records
`skipped_no_saved_state` instead of reporting a false route failure. For a real
write test, prepare the save layer by copying the module owner object from
`Config`/`ConfigCAS` into `ConfigSave`/`ConfigCASSave` first.
Agents should use the high-level method once the module saved-state row already
exists:
```json
{"method":"metadata.module.write_apply","payload":{"base_id":"upo_test","module_ref":"ConfigCASSave:<object_guid>__<module_guid>.0#stream:4","allow_saved_state_write":true,"mode":"apply_and_rollback","allow_sql_saved_state_apply":true,"allow_sql_saved_state_rollback":true,"expected_sha1":"<current-payload-sha1>","old":"<old-fragment>","new":"<new-fragment>","expected_contains":"<guard-fragment>"}}
```
Module text can be changed at three levels:
- full stream replacement: pass `text`;
- fragment replacement: pass `old` and `new`;
- routine replacement/append/upsert: pass `routine_name`, `routine_operation`,
and `routine_text`.
For full stream replacement, pass `expected_text_sha1` from the current module
text in addition to `expected_sha1` for the SQL payload. `expected_text_sha1`
uses normalized line endings, matching the code index freshness hash. This lets
agents verify that the BSL text they read is still the text being patched even
when SQL payload metadata or compression details differ.
This is a thin orchestrator over the proven stream writer: it normalizes
`module_ref`, builds `changes.propose`, applies through
`storage.saved_state.apply_proposal`, and can immediately rollback for smoke.
Before SQL apply it also runs `metadata.write.plan`; blocked plans, such as
`replace_with_control` without a control fragment, stop before the storage
apply call. If current-source evidence such as `current_text` or `source_text`
is supplied, `replace_with_control` also blocks with `control_fragment_drift`
when the controlled fragment no longer appears in that source.
The generic write entrypoint routes module targets to the same method:
```json
{"method":"metadata.write","payload":{"base_id":"upo_test","target":{"kind":"module","module_ref":"ConfigCASSave:<object_guid>__<module_guid>.0#stream:4"},"mode":"apply_and_rollback","allow_sql_saved_state_apply":true,"allow_sql_saved_state_rollback":true,"expected_sha1":"<current-payload-sha1>","old":"<old-fragment>","new":"<new-fragment>","expected_contains":"<guard-fragment>"}}
```
If `module_ref` is omitted, `metadata.write` can resolve exactly one saved-state
module stream with `metadata.saved_state.modules.search`:
```json
{"method":"metadata.write","payload":{"base_id":"upo_test","target":{"kind":"module","owner_guid":"<object_guid>"},"mode":"apply_and_rollback","allow_sql_saved_state_apply":true,"allow_sql_saved_state_rollback":true,"old":"<old-fragment>","new":"<new-fragment>","expected_contains":"<guard-fragment>"}}
```
Ambiguous or missing searches are reported as `module_target_not_resolved`; pass
`module_ref`, `file_name`, `stream_index`, or a narrower query.
For agents, saved-state preparation is an adapter concern, not a planning
choice. The agent asks to change the current module/form. The adapter reads the
active source from `Config`/`ConfigCAS`, creates or reuses the corresponding
working copy in `ConfigSave`/`ConfigCASSave`, writes only that saved-state copy,
and leaves activation to a human in 1C.
For the high-level `metadata.write` entrypoint, apply modes are save-first by
default. Agents should not ask users for SQL permission flags when the user asks
to save/edit metadata: call `metadata.write` with `mode=apply` and the adapter
sets `allow_sql_saved_state_apply=true`, `allow_sql_saved_state_prepare=true`,
and `auto_prepare_saved_state=true` internally. Low-level storage/apply tools
remain explicit, but the agent-facing writer has exactly one write destination:
`ConfigSave`/`ConfigCASSave`.
When `metadata.write` runs in `plan` mode and the save layer is empty, it returns
a `metadata.saved_state.prepare` plan. When `metadata.write` runs in an apply
mode, saved-state preparation is automatic by default. If a caller passes an active module ref such as
`ConfigCAS:<file>#stream:<n>`, the adapter prepares the matching saved-state ref
such as `ConfigCASSave:<file>#stream:<n>` and continues there. Direct writes to
`Config`/`ConfigCAS` remain blocked. Pass `auto_prepare_saved_state=false` only
for diagnostic tests that must prove the save layer is already present.
The same rule applies to form writes. If `metadata.write` receives a form target
in `Config`/`ConfigCAS`, it prepares the matching `ConfigSave`/`ConfigCASSave`
row and then routes the scalar form edit to
`metadata.form.element.write_apply`. If the saved-state form row is missing, a
`plan` response exposes `metadata.saved_state.prepare`; an apply mode with
`metadata.write` performs that preparation internally and retries against the
save table.
Routine-level module edits can be passed without manually building the low-level
`routine` object:
```json
{"method":"metadata.write","payload":{"base_id":"upo_test","target":{"kind":"module","owner_guid":"<object_guid>"},"mode":"plan","routine_name":"<procedure-or-function>","routine_operation":"replace","routine_text":"<full procedure/function text>","expected_old_contains":"<guard-fragment>"}}
```
Supported `routine_operation` values are `replace`, `append`, and `upsert`.
The routine writer preserves the surrounding module bytes: it keeps the module
EOL style and replaces only the exact routine span, not the separators before or
after it. Routine `expected_old_sha1` is calculated over that exact routine span.
## Preferred First Write Path
The first production-safe write path should be extension-first:
```text
generate extension source
-> validate in 1C tooling
-> package extension
-> load into disposable base
-> run smoke tests
-> produce human approval diff
```
Direct SQL metadata writes remain a research path until all gates above are
green.
+67
View File
@@ -0,0 +1,67 @@
# Architecture
## Decision
Используем архитектуру `core + plugins`.
Это промежуточный вариант между монолитом и микросервисами: быстро стартуем в одном репозитории, но держим границы так, чтобы любой плагин можно было вынести в отдельный сервис.
## Core
`core` содержит общие возможности платформы:
- `registry`: учет моделей, адаптеров, версий, лицензий и требований к ресурсам.
- `inference`: единый слой запуска моделей и совместимый API.
- `training`: общие пайплайны подготовки данных и дообучения.
- `evals`: общие правила оценки качества моделей.
- `deploy`: развертывание, включая GPU Docker host.
- `storage`: правила локального хранения моделей и датасетов.
- `monitoring`: метрики GPU, latency, ошибок и использования моделей.
`core` не должен содержать бизнес-логику конкретной задачи.
## Plugins
Плагины содержат прикладные направления:
- `text`: работа с текстом, суммаризация, анализ, генерация.
- `translation`: перевод.
- `audio`: speech-to-text, diarization, text-to-speech.
- `video`: извлечение кадров, аудио, анализ сцен и суммаризация.
- `image`: генерация изображений, inpainting и редактирование по маске.
- `1c`: помощник по 1С, BSL, запросы, метаданные, RAG и дообучение.
Каждый плагин должен иметь собственные:
- описание назначения;
- список моделей;
- пайплайны;
- датасеты;
- eval-тесты;
- настройки инференса;
- план будущего API.
## Plugin Boundary
Плагин может использовать `core`, но `core` не должен зависеть от конкретного плагина.
Правильное направление зависимости:
```text
plugins/* -> core/*
```
Неправильное направление:
```text
core/* -> plugins/*
```
## Growth Path
```text
Stage 1: one repository, core + plugins
Stage 2: heavy plugins run as separate containers
Stage 3: 1c/audio/video/image become standalone services
Stage 4: full service architecture if production load requires it
```
+26
View File
@@ -0,0 +1,26 @@
# Model Cards
Каждая модель, адаптер или embedding-модель должна иметь model card.
Model card отвечает на вопросы:
- что это за модель;
- для каких задач она используется;
- где лежат файлы модели;
- сколько VRAM нужно;
- какая лицензия;
- какие датасеты и eval-тесты использовались;
- можно ли использовать модель в production.
Шаблон: `registry/templates/model-card.yaml`.
Статусы:
- `draft`: модель описана, но не проверена.
- `candidate`: модель выбрана для экспериментов, но локальные файлы могут быть еще не загружены полностью.
- `staging`: модель проходит тесты.
- `production`: модель разрешена для рабочих сценариев.
- `archived`: модель не используется, но описание сохранено.
Проверка `scripts/check_model_storage.py` считает обязательными для полного локального хранения только `staging` и `production`.
Для полной инвентаризации всех карточек используйте `--strict`.
+27
View File
@@ -0,0 +1,27 @@
# First Model Selection
## Selected Model
`Qwen/Qwen3-4B-Instruct-2507`
Local served name: `qwen3-4b-instruct`.
## Why This Model
- Apache-2.0 license.
- 4B parameters, so it is a practical first GPU deployment target.
- Supports text generation, chat, coding, multilingual tasks and tool-use scenarios.
- Official model page includes vLLM usage.
- Native context length is 262,144 tokens, but the first deployment uses 32,768 tokens to reduce OOM risk until GPU VRAM is confirmed.
## Alternatives
- `Qwen/Qwen3-8B`: stronger base candidate, but requires more VRAM.
- `Qwen/Qwen3.5-4B`: newer and long-context oriented, but its model card currently recommends newer/mainline serving frameworks, so it is better as a second experiment after the first stable deployment.
- Mistral/DeepSeek coder models: useful future candidates for coding-specific comparison, but not the first default for Russian + 1C + general assistant coverage.
## Sources
- https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507
- https://huggingface.co/Qwen/Qwen3-8B
- https://huggingface.co/Qwen/Qwen3.5-4B
+455
View File
@@ -0,0 +1,455 @@
PROJECT CONTEXT FOR AI
Project name:
Local LLM Platform
Purpose:
We are building a local AI platform for running and operating LLM-based services on our own infrastructure, with a strong focus on 1C support. The platform is not just a chat wrapper around models. It is intended to become a reusable engineering base for:
- model registry and model selection;
- local inference services;
- GPU deployment and service switching;
- plugin-specific task pipelines;
- evaluation and smoke testing;
- safe 1C analysis, RAG, and eventually controlled code/change assistance.
Main idea:
The repository follows a "core + plugins" architecture.
- core = shared platform capabilities;
- plugins = task-specific domains that can later become standalone services.
This is intentionally a middle ground between a monolith and microservices:
- today we move faster in one repository;
- tomorrow heavy or mature domains can be extracted into separate services.
==================================================
1. WHAT WE ARE BUILDING
==================================================
We are building a local multi-plugin LLM platform for these domains:
- text;
- translation;
- audio;
- video;
- image;
- 1C.
The long-term goal is:
- one shared platform for model lifecycle and deployment;
- multiple domain plugins with their own prompts, datasets, evals, adapters, and APIs;
- safe operational workflows around real business systems, especially 1C.
The repository is not centered on cloud APIs. It is centered on self-hosted/local models and reproducible GPU deployment.
Primary GPU deployment target:
- docker-gpu.cin.su
Shared test Docker host:
- docker-test.cin.su
Important operational assumption:
- heavy model services are not expected to run all at once;
- the current GPU workflow is closer to "single heavy active model/service profile" than to "everything always on".
==================================================
2. ARCHITECTURE
==================================================
Top-level structure:
- core/
- plugins/
- registry/
- scripts/
- docs/
- tests/
- config/ and configs/
- reports/
Meaning of the main parts:
core/
- shared, reusable platform logic;
- should not depend on plugin-specific business logic.
Expected responsibilities in core:
- registry;
- inference;
- training;
- evals;
- deployment;
- storage;
- monitoring.
plugins/
- domain-specific logic;
- each plugin is designed as a future standalone service boundary.
registry/
- model cards and templates;
- stores metadata about models, adapters, versions, storage paths, resource requirements, and statuses;
- does not store model binaries in git.
scripts/
- the operational center of the repo;
- contains validation, smoke tests, deployment scripts, reporting, indexing, model download helpers, 1C tooling, and service control utilities.
docs/
- runbooks, architecture notes, roadmap, API contracts, and research notes.
Current architectural principle:
- plugins may depend on core;
- core must not depend on plugins.
==================================================
3. PLUGINS OVERVIEW
==================================================
Text plugin:
- general text/chat/code-style use cases;
- currently supported by local model registry and vLLM deployment patterns.
Translation plugin:
- local translation service route exists;
- transformers-based service deployment is prepared.
Audio plugin:
- speech-related plugin;
- service route and deployment assets exist;
- smoke and readiness tooling exist.
Video plugin:
- vision/video analysis direction;
- service route and deployment assets exist;
- intended for frame/video understanding tasks.
Image plugin:
- generation and editing;
- SDXL is the practical current image route;
- Qwen image edit experimentation exists, but is much heavier/slower in practice.
1C plugin:
- the most mature and strategically important plugin in the repository;
- includes RAG, metadata parsing, BSL/module/form analysis, adapter contracts, connector policies, an agent service, training artifacts, evals, and many safety checks.
==================================================
4. THE 1C DIRECTION: WHY IT MATTERS
==================================================
The 1C plugin is the deepest part of the project. It is not a simple prompt layer. It is evolving into a safe assistant stack for 1C development and analysis.
What the 1C plugin is meant to do:
- answer 1C and BSL questions;
- help analyze metadata and object structure;
- help with read-only 1C queries;
- support RAG over 1C documentation and internal knowledge;
- inspect forms, modules, templates, and related artifacts;
- support controlled change planning;
- eventually support fine-tuned adapters/LoRA after enough high-quality examples are collected.
Important strategic rule:
- first RAG and tooling;
- fine-tuning later.
This is a deliberate choice. The project is trying to avoid premature fine-tuning before having enough verified, safe, domain-correct examples.
==================================================
5. WHAT HAS ALREADY BEEN IMPLEMENTED
==================================================
This repository is already beyond the "empty skeleton" stage. It has real operational substance.
Implemented foundation:
- architecture and repository layout;
- model registry with model cards;
- plugin structure for all target domains;
- deployment assets for GPU-hosted inference services;
- local model chat UI and service control patterns;
- large collection of validation and smoke scripts;
- runbooks for deployment and operation.
Implemented model/platform side:
- model registry in registry/model-cards;
- plugin model bundle in plugins/model-bundle.yaml;
- vLLM deployment assets;
- llama.cpp deployment assets;
- transformers-based deployment assets for translation/audio/video/image;
- runtime profile and GPU profile configuration;
- scripts for model download, validation, indexing, status collection, and service management.
Implemented UX/operations side:
- model chat server and UI for manual model checks;
- management console web assets;
- platform status collection/reporting;
- image generation/edit job flows and report artifacts;
- service control endpoints for starting/stopping model-serving stacks.
Implemented 1C side:
- 1C metadata parsing modules;
- XML/form/payload/dbnames/config-related parsing utilities;
- 1C RAG corpus preparation and lexical/vector index scripts;
- 1C prompts and RAG profile routing;
- 1C connector and adapter service boundaries;
- 1C MCP adapter code;
- 1C agent server with its own web UI;
- policy files for read-only queries and change workflow;
- schema files for metadata snapshots, BSL snapshots, and moxel registry;
- training example scaffolding and LoRA config scaffolding;
- many focused checks, smoke tests, analysis scripts, and verification scripts.
Evidence of maturity:
- about 267 files in scripts/;
- about 20 test files under tests/1c alone;
- extensive runbook documentation;
- many contract-first docs for 1C behavior and write safety.
==================================================
6. CURRENT STATUS AS OF THE DOCUMENTED ROADMAP
==================================================
The roadmap file contains a concrete status snapshot dated 2026-06-20. Treat this as the latest documented status inside the repo unless newer evidence is added elsewhere.
Documented operating status at that point:
- main UI available on LAN;
- current GPU profile was "image";
- model chat UI container was running;
- image and translation services were online;
- text, audio, video, and some 1C-heavy routes were prepared but not always running at the same time;
- the platform intentionally used a single-heavy-model approach on RTX 4090 to avoid VRAM conflicts.
Documented plugin state:
- Text: ready to start via vLLM route.
- 1C: ready to start, with a strong GGUF route preferred for manual checks.
- Translation: online.
- Audio: ready to start.
- Video: ready to start.
- Image: online, SDXL verified.
Documented completed priorities in roadmap:
- first-class launcher for Qwen3-Coder Q6 on GPU;
- 1C route preference for the strongest practical GGUF model for manual checks;
- smoke test additions for more profiles;
- image model mode switching;
- status report generation from health endpoints.
Practical interpretation:
- the platform already has operational deployment logic;
- however, not every route is meant to be hot simultaneously;
- the operator workflow and profile switching are part of the design, not a temporary bug.
==================================================
7. MODEL STRATEGY
==================================================
The repository uses a curated "one practical model per plugin" strategy rather than trying to host every possible large model.
Examples from the bundle:
- text: Qwen3-4B-Instruct;
- translation: LMT-60-4B;
- audio: Whisper Large V3 Turbo;
- video: Qwen2.5-VL 7B;
- image: SDXL Base plus SDXL Inpainting;
- 1C: Qwen3-Coder GGUF variant as a practical code/1C route.
Why this matters:
- the project is optimizing for practical local deployment;
- VRAM and runtime footprint are first-class constraints;
- model selection is tied to plugin purpose, deployment realism, and operator workflow.
==================================================
8. 1C SAFETY MODEL
==================================================
This is one of the most important parts for any AI reading this project.
The 1C direction is safety-first.
The model may:
- inspect metadata;
- search/read modules;
- validate read-only queries;
- propose changes;
- prepare plans and evidence;
- help build reviewable artifacts.
The model must not:
- directly change a live 1C database;
- run destructive queries;
- invent metadata when the adapter/connector does not confirm it;
- treat effective runtime view as a directly writable surface.
Core safety principles already documented in the repo:
- read-only first;
- full semantic 1C paths are preferred over storage internals;
- effective read view and editor provenance must stay separate;
- every write must begin with a plan;
- direct active configuration writes are forbidden;
- saved-state and extension-aware workflows are preferred;
- concrete write targets, guards, and validation are mandatory.
This means the 1C system is being designed more like a controlled engineering assistant than a free-form coding bot.
==================================================
9. WHAT IS IMPORTANT ABOUT THE 1C ADAPTER/CONNECTOR DESIGN
==================================================
The repo already contains a detailed contract for how 1C interaction should work.
High-level design ideas:
- the adapter works through SQL storage and controlled analysis layers;
- XML exports and Form.xml are useful for analysis and learning, but are not the live write transport;
- agent-facing APIs should speak in full 1C semantic paths, not raw SQL/CAS/internal offsets;
- metadata path resolution and BSL symbol resolution are separate problems;
- the adapter must preserve origin/provenance information so we always know whether something belongs to base config, extension, or saved-state.
Planned/implemented adapter-side capabilities include:
- object resolution;
- fact resolution;
- BSL symbol resolution;
- object context retrieval;
- modules/form/template inspection;
- write planning;
- saved-state targeting and verification;
- evidence-preserving decode behavior;
- smoke checks and contract checks.
This is already a serious design effort, not just a TODO note.
==================================================
10. WHAT REMAINS INCOMPLETE OR STILL EVOLVING
==================================================
Even though a lot exists, the project is still in an active build-out phase.
Areas that are clearly still evolving:
- final extraction of heavy plugins into standalone services;
- broader production-grade monitoring and lifecycle management;
- full maturity of video/audio/image routes under routine operation;
- richer model cabinet / ingest / verification workflows;
- more complete 1C live integration loops;
- broader eval coverage across plugins;
- final training/fine-tuning loop for 1C after verified datasets mature;
- more end-to-end workflows that turn read/analysis into safely validated change pipelines.
The 1C write path especially is intentionally conservative:
- planning and verification are much further along than unrestricted applying;
- the project prefers blocked/refused actions over unsafe hidden behavior;
- saved-state and extension workflows are being formalized step by step.
==================================================
11. WHAT WE HAVE DONE VS WHAT WE STILL NEED TO DO
==================================================
Already done:
- defined the platform direction and architecture;
- created the plugin structure;
- built the model registry concept and model-card inventory;
- prepared deployment stacks for local/GPU model serving;
- built a manual model-chat/testing UI;
- created many operational scripts and checks;
- established the 1C plugin as the main advanced domain;
- implemented large portions of 1C parsing, RAG, adapter contracts, and safety policies;
- documented many workflows and contracts in runbooks and design docs.
Still need to do:
- continue hardening the platform from "engineering bench" into "repeatable service platform";
- improve standalone service boundaries for heavy plugins;
- keep refining GPU profile switching and operational ergonomics;
- expand eval coverage and regression checks for all plugins;
- continue building the 1C live integration loop with strong provenance and write safety;
- improve extension-aware and saved-state-aware write planning and validation;
- grow verified 1C datasets;
- train and compare 1C-specific LoRA/adapters only after enough trustworthy examples exist;
- keep all dangerous operations gated by explicit planning, validation, and rollback-capable flows.
==================================================
12. RECOMMENDED MENTAL MODEL FOR ANOTHER AI
==================================================
If you are another AI reading this file, understand the project like this:
This repository is a local AI platform with a strong operations and tooling focus.
It is not only about prompting models. It is about:
- selecting the right local models;
- deploying them on constrained GPU hardware;
- routing use cases through plugin-specific services;
- verifying quality with smoke/eval checks;
- building safe interfaces to real-world systems.
The 1C area is the flagship domain.
Its current priority is safe read/analysis/RAG/tooling with controlled change planning.
Unsafe convenience is not acceptable there.
You should assume:
- reproducibility matters;
- deployment realism matters;
- model size/VRAM tradeoffs matter;
- contracts and runbooks matter;
- safety gates matter more than agent autonomy in 1C live workflows.
==================================================
13. HOW TO CONTRIBUTE CORRECTLY
==================================================
When continuing work in this repo, prefer these behaviors:
- preserve the core + plugins boundary;
- keep model binaries and large datasets out of git;
- add or update model cards instead of hardcoding assumptions;
- prefer operational scripts and reproducible runbooks over one-off manual steps;
- for 1C, prefer read-only analysis and explicit planning before any write path work;
- preserve provenance/origin information;
- use full semantic 1C paths where possible;
- do not collapse safe abstractions into raw storage details in agent-facing flows;
- add checks/tests/docs together with new behavior;
- treat the scripts/ folder as part of the product, not as disposable glue.
==================================================
14. MOST IMPORTANT NEXT STEPS
==================================================
The most sensible next steps, based on the repository state, are:
1. Continue strengthening the 1C safe operational loop.
- Better write planning.
- Better target resolution.
- Better saved-state verification.
- Better extension provenance and conflict detection.
2. Improve end-to-end readiness of non-1C plugins.
- Make text/audio/video/image routes easier to operate repeatedly.
- Expand smoke tests and health reporting.
- Reduce ambiguity in runtime profiles and active service state.
3. Keep the model registry and deployment inventory clean.
- Ensure model cards, bundles, and runtime profiles stay aligned.
- Keep practical model choices explicit.
4. Build confidence through validation.
- Prefer contract checks, smoke checks, and generated reports.
- Keep adding regression coverage where workflows are safety-sensitive.
5. Delay aggressive fine-tuning until the data is worthy.
- RAG and tooling first.
- Verified examples next.
- LoRA/adapters after that.
==================================================
15. SHORT VERSION
==================================================
We are building a self-hosted local LLM platform with a plugin architecture.
The platform supports text, translation, audio, video, image, and especially 1C.
What is already done:
- architecture;
- model registry;
- deployment stacks;
- chat/testing UI;
- large operations script layer;
- strong 1C parsing/RAG/adapter/safety foundation.
What is most important now:
- mature the platform operationally;
- keep non-1C plugins practical to run;
- continue deepening the 1C assistant with strict safety, provenance, planning, and validation;
- only move to 1C fine-tuning after enough verified domain data exists.
End of file.
+420
View File
@@ -0,0 +1,420 @@
# Quality And Audit Logging Plan
## Goal
Build a minimal but complete logging and telemetry contour that lets us:
- reconstruct any user turn end to end;
- compare model, prompt, RAG, and tool variants;
- detect regressions and failure clusters;
- export clean datasets for evals and later fine-tuning;
- do this without leaking secrets into long-term storage.
This plan treats logging as a cross-cutting platform capability under `core`,
while each plugin keeps its domain-specific fields and labels.
## Current State
What already exists:
- `plugins/1c/agent` stores chat messages and payload JSON in SQLite.
- `1c-agent` persists outbound model context, inbound provider response,
`latency_ms`, tool results, and RAG context.
- `trace_id` exists at the API level for the agent.
What is missing:
- no centralized event schema across plugins and services;
- no guaranteed trace propagation across agent -> adapter/MCP -> model backend;
- no normalized turn-level analytics tables;
- no token/cost accounting;
- no quality labels or review workflow;
- no retention/sanitization/export policy for long-term analysis.
## Target Outcome
For every important request we should be able to answer:
1. What the user asked.
2. Which prompt, model, route, tools, and knowledge sources were used.
3. What the system returned.
4. Whether the result was successful, partial, wrong, unsafe, or abandoned.
5. Which change caused quality to improve or regress.
## Logging Layers
### 1. Access Log
Capture every inbound HTTP request and outbound response for platform services.
Required fields:
- `timestamp`
- `service`
- `instance`
- `environment`
- `request_id`
- `trace_id`
- `method`
- `path_template`
- `status_code`
- `duration_ms`
- `request_size_bytes`
- `response_size_bytes`
- `client_type`
- `error_code`
Notes:
- Do not store raw request/response bodies here by default.
- This is for traffic, latency, and error-rate analysis.
### 2. Turn Audit Log
Capture one normalized record per user-visible turn.
Required fields:
- `turn_id`
- `timestamp`
- `project_id`
- `chat_id`
- `plugin`
- `service`
- `trace_id`
- `request_id`
- `user_message_id`
- `assistant_message_id`
- `user_text`
- `assistant_text`
- `outcome`
- `failure_type`
- `duration_ms`
- `human_review_status`
Recommended enums:
- `outcome`: `success`, `partial`, `failure`, `refused`, `abandoned`
- `failure_type`: `none`, `routing`, `tool_misuse`, `hallucination`,
`format_error`, `timeout`, `provider_error`, `adapter_error`,
`rag_miss`, `policy_error`, `unknown`
### 3. Model Call Log
Capture each actual provider/model request.
Required fields:
- `model_call_id`
- `turn_id`
- `trace_id`
- `provider_id`
- `provider_type`
- `base_url`
- `model_registry_id`
- `served_model_name`
- `route_name`
- `temperature`
- `max_tokens`
- `prompt_messages_json`
- `response_json`
- `prompt_tokens`
- `completion_tokens`
- `total_tokens`
- `cost_estimate`
- `latency_ms`
- `finish_reason`
- `cache_hit`
Notes:
- `prompt_messages_json` is the exact payload sent to the model after prompt
assembly and tool/RAG injection.
- `response_json` is the raw provider response after secret stripping.
### 4. Tool Call Log
Capture every tool or adapter call.
Required fields:
- `tool_call_id`
- `turn_id`
- `trace_id`
- `tool_family`
- `tool_name`
- `target_service`
- `request_json`
- `response_json`
- `status`
- `duration_ms`
- `retry_count`
Recommended tool families:
- `adapter`
- `mcp`
- `rag`
- `internal`
- `external-http`
### 5. Retrieval Log
Capture what RAG actually did.
Required fields:
- `retrieval_id`
- `turn_id`
- `trace_id`
- `plugin`
- `profile`
- `index_version`
- `query_text`
- `top_k`
- `sources_json`
- `context_chars`
- `retrieval_latency_ms`
### 6. Review Log
Capture human or automated quality judgments.
Required fields:
- `review_id`
- `turn_id`
- `reviewer`
- `review_source`
- `score_helpfulness`
- `score_correctness`
- `score_tool_use`
- `score_safety`
- `label_primary_issue`
- `notes`
- `created_at`
Recommended `review_source` values:
- `human`
- `eval`
- `heuristic`
- `llm-judge`
## Trace Propagation Rules
Every inbound request creates or adopts:
- `request_id`
- `trace_id`
Propagation rules:
- agent must pass `trace_id` and `request_id` to adapter/MCP/model wrappers;
- adapter and MCP must echo them in responses and logs;
- background jobs must generate child spans but preserve the parent `trace_id`;
- exported reports must include source `trace_id` where possible.
If a downstream protocol cannot carry headers directly, include these IDs in the
JSON payload envelope.
## Secret And Privacy Rules
Never persist raw secrets in long-term logs.
Must redact or hash before storage:
- API keys
- bearer tokens
- cookies
- passwords
- connection strings with credentials
- user-uploaded files containing secrets or personal data
Recommended approach:
- keep raw payload only in short-lived memory for request execution;
- write sanitized JSON to persistent audit storage;
- store a `redaction_applied=true` flag and optional `redaction_rules_version`.
## Retention Policy
Use three storage horizons.
### Hot
- Purpose: active debugging and operator support.
- Storage: local SQLite or service-local JSONL.
- Retention: `7-14` days.
### Warm
- Purpose: quality analytics and regression analysis.
- Storage: central SQL tables or partitioned JSONL/Parquet under `reports/`.
- Retention: `30-90` days.
### Cold
- Purpose: curated datasets and incident forensics.
- Storage: exported reviewed samples only.
- Retention: explicit/manual.
## Minimal Schema Proposal
Add a new cross-cutting audit storage layer under `core`.
Suggested logical entities:
- `access_events`
- `turn_audit`
- `model_calls`
- `tool_calls`
- `retrieval_events`
- `quality_reviews`
Suggested implementation path:
- phase 1: SQLite in each service plus nightly export to JSONL
- phase 2: normalized central SQLite/Postgres
- phase 3: dashboards and automated quality reports
## Implementation Order
### Phase 1. Fast Wins
Goal: get useful forensic visibility with minimal code churn.
Tasks:
1. Add access logging middleware/pattern to `1c-agent` and other HTTP services.
2. Split current message journal into explicit `turn_audit` and `model_call`
records in addition to existing message history.
3. Add stable `turn_id` and `model_call_id`.
4. Start persisting token usage if the provider returns it.
5. Add redaction helper for secrets before writing payload JSON.
6. Export daily JSONL snapshots into `reports/observability/`.
Definition of done:
- any turn can be reconstructed from one `turn_id`;
- any provider failure can be grouped by model, route, and error code;
- secrets are not written as plain text.
### Phase 2. Cross-Service Correlation
Goal: see one trace across agent, adapter, MCP, and model route.
Tasks:
1. Propagate `trace_id` and `request_id` through agent -> adapter/MCP.
2. Add the same IDs to adapter logs and tool results.
3. Normalize tool-call audit records.
4. Record RAG index version and exact retrieved sources.
5. Add per-turn `outcome` and `failure_type`.
Definition of done:
- a bad answer can be traced to routing, retrieval, tool use, or model output;
- one trace can be joined across at least agent and adapter/MCP logs.
### Phase 3. Quality Loop
Goal: turn logs into a real quality improvement loop.
Tasks:
1. Add review/export scripts for bad turns and representative samples.
2. Add quality labels and scoring workflow.
3. Produce weekly aggregate reports:
- error rate by plugin
- failure types by route/model
- tool success rate
- latency percentiles
- token usage and cost
4. Feed reviewed samples into eval datasets and training candidates.
Definition of done:
- we can say which model/prompt/route is better using saved evidence;
- we can build eval slices from production-like failures.
## Metrics To Track
Minimum operational metrics:
- request count
- non-2xx rate
- timeout rate
- p50/p95/p99 latency
- tool-call success rate
- empty-response rate
- average prompt/completion tokens
Minimum quality metrics:
- turn success rate
- partial-answer rate
- hallucination rate
- tool-misuse rate
- retrieval-miss rate
- human review score averages
- regression rate after route/prompt/model changes
## Suggested Repo Additions
Recommended folders:
- `core/observability/`
- `docs/runbooks/observability.md`
- `scripts/export_audit_logs.py`
- `scripts/report_quality_metrics.py`
Recommended first shared modules:
- `core/observability/schema.py`
- `core/observability/redaction.py`
- `core/observability/trace.py`
- `core/observability/store.py`
## 1C-Specific Additions
For the `1c` plugin, also record:
- `base_id`
- adapter method list
- saved-state vs active/effective mode
- resolved object/module selectors
- truncation/partial flags from adapter
- extension/base layer hints
This is important because many 1C failures are not generic model failures. They
come from incomplete evidence, wrong search strategy, or extension-layer blind
spots.
## Immediate Next Step
The best next implementation step is:
1. introduce `core/observability` with redaction, trace helpers, and JSONL
writers;
2. wire `1c-agent` to emit `access_events`, `turn_audit`, and `model_calls`;
3. add one export script and one weekly report script;
4. then extend the same contract to other plugins.
This gives us a usable quality loop without waiting for a full observability
platform rollout.
## Current Implemented Foundation
Already present in the repo:
- `core/observability/trace.py`
- `core/observability/redaction.py`
- `core/observability/store.py`
- `plugins/1c/agent/agent_server.py` emits `access_events`, `turn_audit`,
`model_calls`, `tool_calls`, and `retrieval_events`
- `scripts/summarize_observability_reports.py`
- `scripts/report_failed_turns.py`
- `scripts/report_quality_metrics.py`
- `scripts/report_repeated_failures.py`
- `scripts/export_quality_snapshot.py`
+50
View File
@@ -0,0 +1,50 @@
# 1C Model Candidates
Дата анализа: 2026-06-19.
## Вывод
Публичной модели, явно обученной именно на 1С/BSL и при этом выглядящей сильнее современных coder-моделей, я не нашел.
Лучший кандидат для проверки под 1С сейчас: `Qwen/Qwen3-Coder-30B-A3B-Instruct`, в локальном GGUF-варианте `lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-GGUF`.
## Почему Qwen3-Coder 30B A3B
- Модель ориентирована на coding/agentic coding.
- Есть длинный контекст: 262,144 токена нативно, что важно для анализа модулей, метаданных и фрагментов конфигурации.
- 30.5B total / 3.3B active MoE: потенциально сильнее маленьких 4B/7B моделей, но легче активной части.
- Есть GGUF quant для llama.cpp/Ollama/LM Studio.
- Apache-2.0 у оригинальной модели.
Практичные GGUF-файлы:
- `Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf` - 18,632,186,176 bytes.
- `Qwen3-Coder-30B-A3B-Instruct-Q3_K_L.gguf` - 14,583,005,504 bytes.
## Сравнение
| Модель | Плюсы для 1С | Минусы |
| --- | --- | --- |
| `Qwen3-Coder-30B-A3B-Instruct` | coding, agentic workflows, long context, tool-use, repo-scale задачи | не обучена специально на 1С; Q4 крупнее Devstral Q4 |
| `Devstral-Small-2-24B-Instruct-2512` | agentic coding, Mistral family, Q4_K_M рекомендован в карточке quant | карточка quant предупреждает про ограничения tool calling в llama.cpp/mistral-vibe |
| `DeepSeek-Coder-V2-Lite-Instruct` | легче, coder-модель, GGUF Q4 около 10.36 GB | старее; вероятно слабее для длинного agentic/RAG сценария |
| `GLM-4.5-Air` | сильная agent/reasoning/coding модель | MoE 106B total / 12B active, тяжелее для локального контура; надо отдельно проверять runtime |
## Рекомендация
1. Не искать “магическую 1С-модель” как основу.
2. Взять `Qwen3-Coder-30B-A3B-Instruct Q4_K_M` как лучший следующий кандидат.
3. Сравнить с текущим `Devstral Q4_K_M` на `plugins/1c/evals/smoke.yaml`.
4. Для качества по 1С делать не ставку на память модели, а связку:
- RAG по документации и metadata snapshots;
- инструменты 1С;
- curated examples;
- LoRA/adapters после накопления датасета.
## Sources
- https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct
- https://huggingface.co/lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-GGUF
- https://huggingface.co/bartowski/mistralai_Devstral-Small-2-24B-Instruct-2512-GGUF
- https://huggingface.co/bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF
- https://huggingface.co/unsloth/GLM-4.5-Air-GGUF
+46
View File
@@ -0,0 +1,46 @@
# Plugin Model Selection
Дата: 2026-06-19.
Так как VRAM `docker-gpu.cin.su` пока не подтверждена по SSH, выбран не абсолютный максимум, а практичный набор сильных моделей, которые разумно пробовать на локальном GPU-хосте.
| Plugin | Model | Why |
| --- | --- | --- |
| `text` | `Qwen/Qwen3-4B-Instruct-2507` | 4B, Apache-2.0, длинный контекст, хороший общий assistant baseline. |
| `translation` | `NiuTrans/LMT-60-4B` | Apache-2.0, специализированная multilingual translation модель, легче 8B-варианта. |
| `audio` | `openai/whisper-large-v3-turbo` | MIT, сильный ASR/speech translation baseline, небольшой размер относительно LLM. |
| `video` | `Qwen/Qwen2.5-VL-7B-Instruct` | Apache-2.0, image/video/document understanding, long-video claims in model card. |
| `image` | `stabilityai/stable-diffusion-xl-base-1.0` + `diffusers/stable-diffusion-xl-1.0-inpainting-0.1` | Практичный SDXL baseline для генерации и masked editing на RTX 4090; качаем fp16 diffusers-вариант. |
| `1c` | `lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-GGUF` | лучший найденный кандидат для code/agentic/repository/tool-use; 1С-качество добираем RAG/tools/LoRA. |
## Bundle
Manifest: `plugins/model-bundle.yaml`.
Download:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/download_plugin_model_bundle.ps1
```
If the large 1C GGUF should be skipped:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/download_plugin_model_bundle.ps1 -SkipLarge1C
```
If image models should be skipped:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/download_plugin_model_bundle.ps1 -SkipImage
```
## Sources
- https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507
- https://huggingface.co/NiuTrans/LMT-60-4B
- https://huggingface.co/openai/whisper-large-v3-turbo
- https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct
- https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0
- https://huggingface.co/diffusers/stable-diffusion-xl-1.0-inpainting-0.1
- https://huggingface.co/lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-GGUF
+201
View File
@@ -0,0 +1,201 @@
# Roadmap
## Current Stand Status - 2026-06-20
Main UI:
- LAN URL: `http://192.168.220.91:8765/tools/model-chat/`
- Current GPU profile: `image`
- Running GPU containers: `llm-model-chat-ui`, `llm-transformers-image`, `llm-transformers-translation`
- Stopped heavy GPU containers: `llm-vllm-text`, `llm-transformers-audio`, `llm-transformers-video`, `llm-llama-qwen3-coder-q6-test`
- GPU mode is intentionally single-heavy-model: keep only the active task model loaded on RTX 4090.
Plugin status:
| Plugin | Current route | Status | Notes |
| --- | --- | --- | --- |
| Text | `qwen3-4b-instruct` on `vllm-text` | ready to start | Stopped in image profile to free VRAM. |
| 1C | `qwen3-coder-1c-q6` on `llama.cpp` port `8081` | ready to start | Default manual-check route now prefers Qwen3-Coder Q6; LoRA/RAG remain separate candidates. |
| Translation | `lmt-60-4b` on `translation-api` | online | API responds on `http://docker-gpu.cin.su:8010`. |
| Audio | `whisper-large-v3-turbo` | ready to start | Model files are present; service is stopped in image profile. |
| Video | `qwen2_5-vl-7b-instruct` | ready to start | Model files are present; service is stopped in image profile. |
| Image | `sdxl-image` | online | SDXL generation and inpainting are verified through UI proxy. |
Recent verified checks:
- Browser UI opens and selects `Stable Diffusion XL Base 1.0 [staging]` for plugin `Фото`.
- Image API route is `http://docker-gpu.cin.su:8040`, served model `sdxl-image`.
- SDXL generate smoke test completed at 512x512, 4 steps, after warmup in `2194 ms`.
- SDXL inpaint smoke test completed at 512x512, 4 steps, including first inpaint model load in `180649 ms`.
- SDXL generate smoke test completed after image service restart at 512x512, 1 step, in `86866 ms`.
- Qwen Image Edit files are present and `QwenImageEditPipeline` loads in about `31 s`, but a 512x512
1-step edit did not finish within `1800 s` on RTX 4090 with CPU offload; SDXL remains the default
practical image service.
- Qwen3-Coder Q6 GPU/CPU comparison exists: GPU `57.72 tok/s`, CPU `2.66 tok/s`, speedup about `x21.7`.
- `1C` route check returns `qwen3-coder-30b-a3b-instruct-q6_k` through `http://docker-gpu.cin.su:8081`.
Next technical priorities:
1. Done: add a first-class launcher for Qwen3-Coder Q6 on GPU instead of keeping it as an ad hoc test container.
2. Done: make the `1C` route prefer the strongest available GGUF model for manual checks, while keeping LoRA/RAG as separate modes.
3. Done: add smoke tests for `audio` and `video` profiles similar to the completed image smoke tests; live runs wait for profile switches.
4. Done: add an image model mode switch for SDXL vs Qwen Image Edit before trying Qwen Image Edit on RTX 4090.
5. Done: add a status report generator so this section can be regenerated from `/api/health`.
## Stage 1: Platform Skeleton
- Зафиксировать архитектуру `core + plugins`.
- Описать правила model registry.
- Создать шаблон model card.
- Подготовить структуру плагинов.
## Stage 2: First Inference
- Выбрать первую текстовую модель.
- Описать ее model card.
- Поднять inference на `docker-gpu.cin.su`.
- Проверить API и базовые eval-тесты.
Текущая основа:
- `core/deploy/docker-gpu/vllm/compose.yaml`
- `core/deploy/docker-gpu/vllm/.env.example`
- `docs/runbooks/first-vllm-inference.md`
- `registry/model-cards/qwen3-4b-instruct-2507.yaml`
- `scripts/download_hf_model.py`
- `scripts/list_model_cards.py`
- `scripts/build_model_index.py`
## Stage 3: 1C RAG
- Собрать документы и правила по 1С.
- Подготовить индекс для поиска.
- Добавить инструменты работы с метаданными 1С.
- Сделать eval-набор задач по BSL и запросам 1С.
Текущая основа:
- `plugins/1c/rag/manifests/corpus.yaml`
- `scripts/prepare_1c_rag_corpus.py`
- `plugins/1c/tools/tool-contract.yaml`
- `plugins/1c/prompts/system.md`
- `plugins/1c/evals/smoke.yaml`
- `scripts/build_1c_rag_index.py`
- `scripts/search_1c_rag.py`
- `scripts/ask_1c_rag.py`
- `scripts/check_1c_rag_prompt.py`
- `plugins/1c/metadata/schema.json`
- `scripts/validate_1c_metadata_snapshot.py`
- `scripts/convert_1c_metadata_to_rag.py`
## Stage 4: 1C Fine-Tuning
- Накопить проверенные пары вопрос/ответ.
- Очистить данные от секретов и персональных данных.
- Обучить LoRA/adapter.
- Сравнить base model, RAG и adapter.
Текущая основа:
- `plugins/1c/training/examples/instruction.examples.jsonl`
- `plugins/1c/training/manifests/dataset.yaml`
- `scripts/validate_1c_training_data.py`
- `scripts/prepare_1c_training_data.py`
- `plugins/1c/training/configs/qwen3-coder-30b-a3b-lora.yaml`
- `scripts/preflight_1c_training.py`
- `scripts/train_1c_lora.py`
- `core/deploy/docker-gpu/training/1c-lora.compose.yaml`
- `plugins/1c/adapters/qwen3-coder-30b-a3b-1c-lora-v1.yaml`
- `registry/model-cards/qwen3-coder-30b-a3b-1c-lora-v1.yaml`
## Stage 5: Service Split
- Вынести тяжелые плагины в отдельные контейнеры.
- Описать API-контракты.
- Добавить мониторинг и контроль версий моделей.
Текущая основа:
- `core/deploy/docker-gpu/transformers/translation.compose.yaml`
- `core/deploy/docker-gpu/transformers/audio.compose.yaml`
- `core/deploy/docker-gpu/transformers/video.compose.yaml`
- `scripts/transformers_plugin_server.py`
- `scripts/deploy_transformers_service.ps1`
## Stage 6: GPU Service Manager
- Управлять запуском/остановкой тяжелых сервисов моделей.
- Показывать VRAM и endpoint выбранной модели в UI.
- Исключить конфликт нескольких compose-проектов.
- Оставить ручной операторский путь через PowerShell.
Текущая основа:
- `scripts/manage_model_service.ps1`
- `POST /api/service-control` in `scripts/model_chat_server.py`
- service panel in `tools/model-chat/index.html`
## Stage 7: Model Cabinet
- Загружать модель файлом из личного кабинета.
- Импортировать модель по URL или серверному пути.
- Проверять наличие файлов, размер, checksum и карточку модели.
- Переводить проверенную модель из incoming в рабочий registry.
Текущая основа:
- `/api/model-ingest/upload`
- `/api/model-ingest/source`
- `/api/model-ingest/verify`
- `reports/model-ingest/jobs.jsonl`
## Stage 8: Video And Vision
- Реализовать image endpoint для Qwen2.5-VL.
- Добавить загрузку изображения в UI.
- Для видео сделать извлечение кадров и VLM-анализ.
- Добавить eval-набор для экранов, документов и видео-сцен.
## Stage 9: 1C Live Integration
- Подключить read-only коннектор к реальной базе 1С.
- Добавить быстрый operational loop без обязательной полной XML/EDT-синхронизации на каждый шаг.
- Индексировать реальные метаданные и BSL-модули через легкий 1C agent или JSON snapshot.
- Подключить intake задач из текста, Excel-файлов и скриншотов/макетов интерфейса.
- Ввести полные 1C-пути как основной язык агента:
`Справочник.Контрагенты.Наименование`, а не голое `Наименование`.
- Разделить effective-read и editor-provenance: агент читает итоговую картину
1С, но пишет только через origin/layer-aware write plan.
- Добавить отдельное разрешение BSL-символов внутри модуля/формы, чтобы
переменная `Номенклатура` не смешивалась с объектом
`Справочник.Номенклатура`.
- Расширить поддержку расширений: порядок применения, добавленные объекты,
adopted base objects, insert-before/after, replace, replace-with-control.
- Добавить quality gates перед любыми изменениями.
- Обучать LoRA только после RAG baseline и проверенного датасета.
Текущая основа:
- `docs/runbooks/1c-operational-coding.md`
- `docs/1c-extension-layer-plan.md`
- `docs/1c-adapter-api-contract.md`
- `plugins/1c/connector/contracts/openapi.yaml`
- `plugins/1c/connector/policies/read-only-query.yaml`
- `plugins/1c/connector/policies/change-workflow.yaml`
## Cross-Cutting: Evals
- `scripts/validate_evals.py`
- `scripts/run_1c_smoke_eval.py`
- `docs/runbooks/evals.md`
## Cross-Cutting: Observability And Quality Logging
- Add centralized access, turn, model-call, tool-call, and retrieval logging.
- Propagate `trace_id` and `request_id` across agent, adapter, MCP, and model routes.
- Add redaction, retention, and export rules for quality analysis.
- Add reviewed quality labels and aggregate reports for regressions and route comparisons.
Current plan:
- `docs/quality-audit-logging-plan.md`
+91
View File
@@ -0,0 +1,91 @@
# 1C Agent Coding Contract
This contract is the default rule set for coding agents that work through the
1C adapter.
## Default View
- Read current code and metadata from the working saved-state layer by default.
- For REST calls, use `state=working`.
- For MCP calls, use `source_state=working`.
- Treat saved-state objects as current programming state even when they are not
activated yet.
- Objects can exist only in saved-state and can later be activated or canceled.
Do not hide them from the agent view.
## Compare Views
- Use `state=both` or `source_state=all` only when the task needs a comparison
with activated runtime state.
- Show the effective working text first.
- Mark comparison details explicitly:
- `saved_state`: saved and not activated;
- `active`: activated runtime state;
- `text_source`: which layer produced the returned text;
- `comparison.differs`: whether both layers exist and differ.
## Read Workflow
Use public names and selectors:
1. `extension.objects.find` with `state=working` to find extension objects.
2. `code.search` with `state=working` to find routines or fragments.
3. `code.read` with `state=working` to read the module or routine.
4. `code.read` with `state=both` only for an explicit saved-vs-active check.
Agents should ask for and report object names, routine names, and code text.
They should not ask users for SQL tables, storage file names, stream indexes, or
saved-state write flags during normal coding work.
## Write Workflow
All normal BSL writes go through `code.write`.
Supported public edit shapes:
- replace a whole module with `module_text`, `full_text`, or `code`;
- replace one procedure or function with `routine_name` and `routine_text`;
- replace one unique fragment with `old` and `new`; when `routine_name` is
supplied, the adapter scopes the replacement to that routine.
`code.write` saves into saved-state automatically. A coding agent should simply
say "save this code" and send the desired code text. It must not ask whether SQL
saved-state apply flags are allowed.
Every successful `code.write` response must show:
- `write_mode.target=saved_state`;
- `write_mode.activation_state=not_activated`;
- `write_mode.production_apply=false`.
## Hidden Storage Details
The form module container marker `///----` is adapter-owned storage syntax.
Public `code.read` and `code.search` responses must not expose it as BSL.
Full-module writes must preserve the marker internally when the saved form
payload requires it.
Low-level methods such as `metadata.module.write_apply`, `metadata.write`, SQL
tables, stream refs, and saved-state apply flags are diagnostic tools. They are
not the default programming interface for agents.
## Agent Response Shape
When reporting a working saved-state result to a user, prefer concise wording:
```text
В working/save вижу формы:
t_Форма
tt_Форма3
ФормаЭлемента
Код читается из saved_state, еще не активирован.
```
If the user asks to compare active and saved state:
```text
Working/save: найдено, источник saved_state, не активировано.
Active: не найдено.
Эффективный код для программирования сейчас берется из saved_state.
```
+88
View File
@@ -0,0 +1,88 @@
# 1C Agent (отдельный подпроект)
## Быстрый запуск на test-docker
```powershell
# 1) Подготовить переменные (без секретов)
cd Z:\codex\LLM
Copy-Item core\deploy\docker\1c-agent\1c-agent.env.example .\core\deploy\docker\1c-agent\\.env
```
Редактируйте `.env`:
- `ONEC_ADAPTER_URL` → URL REST-адаптера;
- `ONEC_ADAPTER_TOKEN` → токен (если настроен).
- при необходимости `ONEC_AGENT_PROVIDERS` (JSON).
```powershell
$env:DOCKER_HOST = "ssh://test-docker"
docker compose -f core\deploy\docker\1c-agent\compose.yaml --env-file core\deploy\docker\1c-agent\.env up -d --build
```
Проверка:
```powershell
docker ps --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
curl http://docker-test.cin.su:8090/v1/health
```
Если внешний хост не отвечает из вашей сети, проверьте локально в контейнере:
```powershell
$env:DOCKER_HOST = "ssh://test-docker"
docker exec onec-agent python -c "import urllib.request, json; print(json.loads(urllib.request.urlopen('http://127.0.0.1:8090/v1/health', timeout=5).read().decode()))"
```
## Основные потоки
- `project` — единица бизнеса/сценария: имя, описание, политики, базовая настройка.
- `chat` — конкретная сессия: выбранная модель, провайдер, параметры генерации, локальные параметры RAG.
- `message` — отдельные сообщения внутри чата, используемые для контекста.
## Что лучше держать где
- Project-level:
- название/описание;
- долгоживущие политики доступа/ограничения;
- общие настройки безопасности и базовые параметры.
- Chat-level:
- выбранный провайдер/модель;
- `rag_profile`, `rag_limit`;
- `temperature`, `max_tokens`;
- системный prompt конкретной сессии.
## Взаимодействие с разными ИИ-провайдерами
Сервис уже умеет работать через `ONEC_AGENT_PROVIDERS` как через карту:
```json
{
"default": {
"type": "openai-compatible",
"base_url": "http://docker-gpu.cin.su:8000",
"model": "qwen3-4b-instruct-2507"
}
}
```
Если нужно подключить другой ИИ протокол, добавляем новый `type` в `plugins/1c/agent/agent_server.py` в диспетчер `call_model(...)`.
## Проверка turn
```powershell
curl -X POST "http://docker-test.cin.su:8090/v1/projects" `
-H "Content-Type: application/json" `
-d '{ "name":"demo-1c", "description":"Проверка пайплайна" }'
curl -X POST "http://docker-test.cin.su:8090/v1/projects/<project_id>/chats" `
-H "Content-Type: application/json" `
-d '{ "title":"Проверка" }'
curl -X POST "http://docker-test.cin.su:8090/v1/projects/<project_id>/chats/<chat_id>/turn" `
-H "Content-Type: application/json" `
-d '{ "message":"Какие реквизиты есть у справочника Номенклатура?", "use_rag": true }'
```
## Статус/поддержка
- Проверить, что сервис запущен: `GET /v1/health`.
- Проверить модели/провайдеры: `GET /v1/models`, `GET /v1/providers`.
+316
View File
@@ -0,0 +1,316 @@
# Controlled Designer-to-SQL Decoding
Use the disposable `upo_test` base to learn SQL encodings that cannot be
established from static samples. The mutation is performed by 1C Designer or a
1C Enterprise client. The adapter remains a read-only SQL observer. The active
rule is `plugins/1c/connector/policies/designer-sql-decoding-policy.yaml`.
## Credential rule
Keep the 1C user password outside the repository. Supply it for one process
through an environment variable or an operating-system credential store. Do
not add it to `.env`, YAML, JSON, test fixtures, reports, or command examples.
## Metadata experiment
1. Select an object by a public 1C name.
2. Confirm there are no unrelated pending Designer changes.
3. Capture the target from live SQL, including active and saved-state origin.
4. In Designer change exactly one property and save it without applying the
configuration when saved-state evidence is sufficient.
5. Capture `ConfigSave` or `ConfigCASSave` again and compute the structural and
byte-level diff.
6. Repeat with a second value; one pair is only a hypothesis.
7. Promote a decoder only after the name, type, path, and ownership are stable.
8. Revert through Designer and verify rollback in SQL.
Applying the configuration is a separate explicit phase because it can change
`Config`, `ConfigCAS`, and the physical application-data schema.
## Application-data experiment
1. Resolve the object through `data.schema` using its public name.
2. Capture `data.count` and a narrowly filtered `data.list`.
3. Create or edit one test record through 1C Enterprise, never through SQL.
4. Capture the same logical filter after the 1C transaction commits.
5. Correlate logical values with SQL columns, including composite branches.
6. Revert or delete through 1C Enterprise and verify rollback read-only.
## Noise controls
- Record configuration-check errors that existed before the experiment.
- Do not run two experiments against the same object concurrently.
- Separate base configuration and extension ownership.
- Treat timestamps, version bytes, caches, and background service data as
volatile unless explicitly targeted.
- Discard a run when more than one semantic property changed.
Each accepted experiment produces a manifest with the public selector,
intended change, before/after SQL hashes, changed paths or columns, semantic
rule, second verification case, rollback evidence, and regression tests. XML
may be attached as offline naming evidence but is not read by the adapter.
## Confirmed baseline
The first live `upo_test` CAS comparison is recorded in
`reports/1c-sql/upo_test/designer-sql-baseline-20260714.json`. It proves a BSL
module-text change at tree path `$.2`. The accompanying `pos` and `end` changes
are stream-directory offsets recalculated from the text length; they are not
independent metadata properties and must be filtered as derived evidence.
## Saved extension metadata descriptors
The `test2` experiment on 2026-07-15 added a minimal calculation register and
its required chart of calculation types through Designer, then saved the
extension without applying it. The objects remain intentionally `saved_only`
for adapter regression checks.
Observed read-only SQL signatures in `ConfigCASSave`:
- `CalculationRegister`: brace root marker `1`, root length `10`, metadata
block marker `21`;
- `ChartOfCalculationTypes`: brace root marker `1`, root length `8`, metadata
block marker `35`;
- saved extension descriptor names use
`<extension-guid>__<object-guid>`; child/module parts add a numeric suffix.
The runtime adapter derives these signatures only from SQL payloads. The XML
export is offline evidence used to confirm the public object kind, name, GUID,
and the register-to-chart relationship; it is not a runtime data source.
When `state=working`, `extension.objects.find` must overlay these descriptors
from `ConfigCASSave` and report `saved_only` or `saved_override`. With
`state=active`, the same unapplied objects must not be returned. Route-cache
rebuilds must reclassify active descriptors so an earlier guessed kind cannot
survive as a false match.
Public saved-state card selectors expose `extension_guid`, object `guid`, and
`table=ConfigCASSave`; they do not expose the physical descriptor name. The
adapter reconstructs `<extension-guid>__<object-guid>` internally before the
read-only SQL lookup.
For `ChartOfCalculationTypes`, the verified kind-specific map currently covers
17 properties: the five scalar code/name settings at paths `1.24``1.30`, six
default/auxiliary form references, five localized presentations, and
`ActionPeriodUse` at `1.57`. The paths were cross-checked on the live
`Начисления` and `Удержания` descriptors against their offline XML exports.
For the saved-only `CalculationRegister`, the verified header map covers 13
properties: periodicity, action/base-period flags, list-form references, chart
reference, standard-command/help flags, lock and full-text modes, and three
localized list presentations. `metadata.object.properties` accepts the public
`extension_guid` + object `guid` selector and reconstructs the `ConfigCASSave`
descriptor name internally. Runtime property responses remain SQL-only.
The saved register descriptor also confirms all seven variable child-part
collections: `Attribute` at root path `3`, `Recalculation` at `4`, `Template`
at `5`, `Resource` at `6`, `Form` at `7`, `Command` at `8`, and `Dimension` at
`9`. The three field roles were distinguished by a controlled Designer sample
containing one `Реквизит1`, `Ресурс1`, and `Измерение1`. Designer saved the
extension without applying it; the adapter identified the records read-only
from the resulting live `ConfigCASSave` payload.
Saved-only register fields are available through `metadata.object.attributes`
with `table=ConfigCASSave`, `extension_guid`, and the public object `guid`.
The response returns names and decoded types while reconstructing the physical
saved descriptor key internally. `metadata.object.related` accepts the same
selector for recalculations, templates, forms, and commands; saved child keys
are also prefixed internally and remain hidden unless storage diagnostics are
explicitly requested.
A second controlled Designer sample added one item to every related collection:
`Перерасчет1`, `Макет`, `ФормаСписка`, and `Команда1`. The root descriptor then
reported a declared count of one at paths `4`, `5`, `7`, and `8`. Recalculation,
template, and form descriptors use the internal
`<extension-guid>__<child-guid>` key. A saved command exposes its BSL payload as
`<extension-guid>__<command-guid>.2`; the additional command-class GUID inside
the owner record is type evidence, not a second related command. Public related
results therefore keep only the record identity GUID and probe the `.2` module
route internally when the direct saved command descriptor is absent.
The specialized `metadata.object.forms`, `metadata.object.form.details`,
`metadata.object.commands`, and `metadata.object.modules` methods accept the
same saved-state selector. Form enumeration and detail decoding use the
internally reconstructed `<extension-guid>__<form-guid>.0` payload. Object
commands are returned from the owner descriptor and their saved `.2` payload
is verified without exposing the physical key in normal responses. Each saved
object command also returns a ready public `modules.read` selector, so callers
select the command by name and never need to calculate its GUID or SQL route.
The `.2` payload is a raw-deflate multi-stream container; `modules.read`
automatically selects its single BSL-marked stream when whole-payload text
decoding is not applicable.
Object-scoped `modules.search` and its `code.search` wrapper include these
saved command modules alongside the owner's regular modules. A caller can
therefore search by the public register selector plus BSL text and receive a
ready `modules.read`/`code.read` selector for the matching command module.
`metadata.definition.find` with `areas=["modules"]` follows the same saved
selector, enumerates command-module routines, and returns the exact procedure
or function definition with a routine-scoped read selector.
`metadata.object.parts` and `metadata.object.decode` also reconstruct the
saved descriptor prefix before reading. The former enumerates the root and
suffix payloads under `<extension-guid>__<object-guid>`; the latter decodes the
root descriptor from that key while keeping physical storage coordinates
hidden by default.
The combined `metadata.object.full` profile promotes saved object-command
selectors into its `modules` collection as `command_module` handles. This
keeps the profile lightweight (no BSL text is loaded there) while ensuring the
reported module count includes code carriers owned by commands.
Targeted `metadata.code_index.build` runs on saved command `.2` containers now
retain only streams that are positively identified as BSL. Command owner
metadata cached from `metadata.object.commands` is inherited by the concrete
`#stream:N` index row, and obsolete non-BSL rows plus their vector chunks are
pruned when the source file is rebuilt. The extension GUID is recovered from
the saved module route and retained in the indexed owner metadata.
For saved extensions, `metadata.object.get` resolves an object name through
the saved extension manifest/state route before reading its descriptor.
The public `/rpc` dispatcher preserves this name selector when
`extension_guid` and `table=ConfigCASSave` are supplied; callers may use the
object name directly and do not have to resolve its GUID first. An explicit
`guid` still takes precedence when both selectors are present.
`metadata.code_index.build` can therefore be scoped by `kind` plus `name` and
`extension_guid`; it discovers owner and command module files internally and
does not require callers to pass a physical prefix or command GUID.
The same object-scoped build enumerates saved forms, decodes their embedded
modules separately from container streams, and indexes a form only when a
valid non-empty BSL module is present. Empty generated forms are not emitted
as code carriers. Build counts distinguish `forms_scanned`,
`form_modules_discovered`, `empty_form_modules`, and `form_module_errors`, so
coverage and decoding failures are observable separately.
`CalculationRegister` is included in the public register code-carrier matrix.
Its record-set/register/command module handles use the same name-first SQL-only
read and saved-state write contract as the other register kinds.
Offline `Form.xml` analysis uses the same public vocabulary as the SQL form
decoder for form commands, events, attributes and value-table columns, check
box fields, and search/view-status/search-control additions. The XML profile
also exposes semantic properties of the root `Form` node, including command
bar location and visibility. XML remains comparison evidence only and is never
consulted by runtime adapter calls.
The SQL/XML comparison report records property-route evidence as
`XML kind/property -> SQL marker/parameter/source`. Across the 11 controlled
test-extension forms, 764 elements and 3,361 properties compare without
mismatches or XML-only properties; 23 routes have at least two matching
examples and a single SQL route. `UsualGroup.Visible` remains intentionally
variant-aware because marker `22` uses parameter `26` or `28` in two observed
structures. A controlled Designer probe on
`t_FORM_ContainerTableBehaviorVariants` changed only
`ГруппаФорма.Visible=false -> true`: parameter `26` changed `0 -> 1`, while
parameter `10` and nested `Группа1` remained unchanged. In the nested shape,
parameter `26` is a GUID and parameter `28` is the boolean visibility slot.
Saved-state writes therefore select parameter `26` only when it is boolean;
otherwise they select boolean parameter `28`. The extension was restored with
`/RollbackCfg -Extension test`, and its `ConfigCASSave` prefix was verified
empty after the probe.
Root form properties are compared separately from element properties. A
controlled `ShowCommandBar=false -> true` Designer probe changed exactly two
form atoms: parameter `17` from `0` to `2` and parameter `56` from `0` to `1`.
The SQL profile exposes this as `ОтображатьКоманднуюПанель` with
`write_shape=paired_scalar`; it is readable but must not be routed through a
single-scalar writer. `АвтоКоманднаяПанель` is resolved from the decoded form
item with `id=-1`. Before the next controlled probe, 33 root properties
matched and 44 remained.
A controlled `WindowOpeningMode=DontBlock -> LockOwner` Designer probe changed
exactly form parameter `2` from `0` to `1` and companion parameter `54` from
`0` to `1`. The SQL profile exposes the confirmed values as
`РежимОткрытияОкна=DontBlock|LockOwner` with `write_shape=paired_scalar`.
Other platform enum values remain undecoded until separately observed. The
probe was loaded only into the saved extension configuration, then rolled back;
the `test` extension prefix in `ConfigCASSave` was verified empty afterwards.
Across the 11 fixtures, 44 root properties now match; the remaining 33 are
three scalar properties repeated on each form: `AutoSaveDataInSettings`, root
`Group`, and `CommandBarLocation`.
A controlled `AutoSaveDataInSettings=Use -> DontUse` Designer probe changed
only form parameter `7` from `1` to `0`. The SQL profile exposes this as
`АвтоСохранениеДанныхВНастройках=Use|DontUse` with
`write_shape=scalar_enum`. The extension was rolled back and the `test`
prefix in `ConfigCASSave` was verified empty. Across the 11 fixtures, 55 root
properties now match; the remaining 22 are root `Group` and
`CommandBarLocation`, each repeated on all forms.
A controlled root `Group=Vertical -> Horizontal` Designer probe changed four
form parameters together: `11`, `40`, `47`, and `57`, all from `0` to `1`.
The SQL profile exposes `Группировка=Vertical|Horizontal` with
`write_shape=composite_scalar`; partial single-atom writes are not safe. The
extension was rolled back and the `test` prefix was verified empty. Across the
11 fixtures, 66 root properties now match; only `CommandBarLocation` remains.
`CommandBarLocation` shares the same parameter pair as `ShowCommandBar`. With
the panel enabled, `Top` produced `[17,56]=[2,1]` and `Bottom` produced
`[3,1]`; the hidden state is `[0,0]` and is exposed as `None`. XML `None` with
`ShowCommandBar=true` normalizes to the same SQL state as `Top`, so SQL exposes
the effective position. The profile returns
`ПоложениеКоманднойПанели=None|Top|Bottom` with
`write_shape=paired_scalar_shared`; both properties must be encoded together.
After each probe the extension was rolled back and its saved prefix was empty.
All 77 root properties across the 11 controlled forms now match SQL to XML.
The platform's third window-opening mode was verified separately because it is
not present in the UPO XML inventory. The accepted XML literal is
`LockWholeInterface`; a controlled Designer probe changed form parameters
`[2,54]` from `[0,0]` to `[2,2]`. `РежимОткрытияОкна` now decodes the complete
confirmed enum: `DontBlock=[0,0]`, `LockOwner=[1,1]`, and
`LockWholeInterface=[2,2]`. The failed tentative `LockUI` literal was rejected
by XDTO before any SQL saved state was created. The successful probe was rolled
back and the extension saved prefix was verified empty.
The complete root `Group` enum observed in UPO was verified with two additional
Designer probes. `AlwaysHorizontal` maps to
`[11,40,47,57]=[1,1,3,3]`, while `HorizontalIfPossible` maps to
`[1,2,2,2]`. Together with `Vertical=[0,0,0,0]` and
`Horizontal=[1,1,1,1]`, the decoder now covers every root form grouping value
present in the XML inventory. Each probe was rolled back and the saved prefix
was verified empty.
Active base-configuration forms also use compact root layouts. Across multiple
SQL-only samples, form payload versions 49 and 50 store
`WindowOpeningMode` directly in parameter `2`; values `0` and `1` were
confirmed by `DontBlock` list forms and `LockOwner` item forms. In these
layouts, parameter `11=0` consistently identifies root `Group=Vertical` even
when the newer companion positions are absent. The decoder applies these
fallbacks only to versions 49/50 and observed values; runtime remains SQL-only.
The active base catalog form `ЗадачиАссистентаУправления.ФормаСписка`
confirmed the compact marker-55 dynamic-list layout. Parameter `54` is the
property-bag entry count; its typed key/value pairs decode keys `5`, `6`, `8`,
`9`, `11`, `12`, `14`, and `16` as `AutoRefresh`, `AutoRefreshPeriod`,
`ChoiceFoldersAndItems`, `RestoreCurrentRow`, `ShowRoot`, `AllowRootChoice`,
`UpdateOnDataChange`, and `AllowGettingCurrentRowURL`. The following tail
records contain the user-settings-group item id, `InitialTreeView`, and the
standard `DefaultPicture` field reference. The same live payload confirms
`CommandBarLocation=None` at parameter `6` and `DefaultItem=true` at parameter
`16`. Compact marker-22 command bars expose `Autofill` at parameter `28`.
Marker-35 label fields with nested subtype marker `11` store
`AutoMaxWidth/MaxWidth` in nested positions `15/16` of parameter `39`.
The SQL/XML comparison normalizes these public XML names to the decoder's
Russian semantic vocabulary and treats marker-55 `Динамический список` as the
SQL implementation of XML `Table`. XML is used only to validate the learned
routes; runtime decoding reads the SQL payload alone.
The base item form of the same catalog confirmed more compact-layout routes.
Root parameter `20` is a typed enum with type GUID
`59ef2b80-c86b-11d5-a3c1-0050bae0a776`; value `0` decodes
`UseForFoldersAndItems=Items`. Root event bindings may reside in a direct form
block such as `1.23`; they are discovered by the GUID/handler pair shape rather
than a fixed position. GUID `bf0ac0e1-bcbb-4dfe-8fc4-0b1923b461a6` identifies
`BeforeWriteAtServer`. Compact pages options `{3,1,...}` decode
`PagesRepresentation=TabsOnTop`.
Object-backed data paths resolve public standard fields `-2/-3/-4/-5` as
`Code/Description/Parent/Ref`; custom object and tabular-section fields are
resolved from the element name and owning table. XML `AdditionalColumns`
definitions are correlated with the physical SQL form element by their full
data path. Across both active forms of
`Catalog.ЗадачиАссистентаУправления`, the comparison now covers 144 XML
elements, including the logical additional column, with no missing elements,
property mismatches, or XML-only properties.
@@ -0,0 +1,176 @@
# 1C Form Command Binding Learning
This runbook tracks the two remaining `ТестНастройки` SQL/XML gaps after the
form decoder reached zero missing items and zero mismatches.
## Target
- Base: `upo_test`
- Adapter: `http://docker-gpu.cin.su:8011`
- Saved-state table: `ConfigCASSave`
- Form payload file:
`f96a0c45-3eff-11f1-8287-005056b0d483__25c39fbf-35a4-4b43-8e3f-cd1f91082c88.0`
- Form: `ТестНастройки`
- Current baseline payload SHA1:
`0ca61aa4ed041fc4219cada0126d5f144e59d30d`
## Before Captures
The baseline captures were created on 2026-07-02.
| Learning ID | Element | Expected XML binding | Snapshot |
| --- | --- | --- | --- |
| `form-command-binding-standard-customize-form` | `ТЗИзменитьФорму` | `Form.StandardCommand.CustomizeForm` | `38cab2dc00c9465e9dc42a17548d3170` |
| `form-command-binding-local-apply-command` | `ФормаКомандаОбновить` | `Form.Command.КомандаПрименить` | `e312ccc2e6ed45fbb6ebac187d30ba7a` |
Adapter-side capture paths:
- `/data/adapter-write-learning/form-command-binding-standard-customize-form/before-38cab2dc00c9465e9dc42a17548d3170.json`
- `/data/adapter-write-learning/form-command-binding-local-apply-command/before-e312ccc2e6ed45fbb6ebac187d30ba7a.json`
## XML Fixture Workflow
Use the XML workflow first, matching the moxel discovery flow: edit/generate XML,
load it into the test extension, then inspect the changed SQL payload.
Generate fixtures:
```powershell
python scripts/create_1c_form_command_binding_xml_fixtures.py
```
Generated files:
| Fixture | Purpose |
| --- | --- |
| `reports/1c-sql/upo_test/xml-command-binding-fixtures/00-original/Form.xml` | baseline copy |
| `reports/1c-sql/upo_test/xml-command-binding-fixtures/01-local-button-command-example1/Form.xml` | changes `ФормаКомандаОбновить` to `Form.Command.КомандаПример1` |
| `reports/1c-sql/upo_test/xml-command-binding-fixtures/02-standard-button-to-local-command/Form.xml` | changes `ТЗИзменитьФорму` to `Form.Command.КомандаПример1` |
| `reports/1c-sql/upo_test/xml-command-binding-fixtures/03-combined-command-binding-switches/Form.xml` | applies both existing-button changes |
| `reports/1c-sql/upo_test/xml-command-binding-fixtures/04-add-two-learning-buttons/Form.xml` | adds two new learning buttons under `Группа2` |
Preferred first load:
1. Load `01-local-button-command-example1/Form.xml` into
`фс_ДоработкиОбщее.DataProcessor.фс_НастройкаУсловногоОформления.Forms.ТестНастройки`.
2. Update/save the extension so `ConfigCASSave` receives a new form payload.
3. Run:
```powershell
python scripts/run_1c_form_command_binding_learning.py `
--wait-for-sha-change 0ca61aa4ed041fc4219cada0126d5f144e59d30d `
--max-wait-seconds 300 `
--report reports/1c-sql/upo_test/form-command-binding-learning-run.json
```
Then use the adapter diff/inference output to promote a decoder/write rule.
If the first load is clean, repeat with `02-standard-button-to-local-command`
or `03-combined-command-binding-switches`.
## Experiment A: Standard Command Binding
Goal: learn where marker `34` stores a standard command binding.
After any manual save, the preferred one-command runner is:
```powershell
python scripts/run_1c_form_command_binding_learning.py `
--wait-for-sha-change 0ca61aa4ed041fc4219cada0126d5f144e59d30d `
--max-wait-seconds 300 `
--report reports/1c-sql/upo_test/form-command-binding-learning-run.json
```
Manual edit in Designer:
1. Open form `ТестНастройки`.
2. Select button `ТЗИзменитьФорму`.
3. Change only `CommandName` from `Form.StandardCommand.CustomizeForm` to a
different standard command if Designer allows it.
4. Save the form once.
After save, capture:
```powershell
@'
import json, urllib.request
payload = {
"base_id": "upo_test",
"learning_id": "form-command-binding-standard-customize-form",
"table": "ConfigCASSave",
"file_name": "f96a0c45-3eff-11f1-8287-005056b0d483__25c39fbf-35a4-4b43-8e3f-cd1f91082c88.0",
"form": "ТестНастройки",
"element": "ТЗИзменитьФорму",
"property": "ИмяКоманды",
"timeout_seconds": 60,
"max_items": 5000
}
req = urllib.request.Request("http://docker-gpu.cin.su:8011/rpc", data=json.dumps({"method":"metadata.write_learning.capture_after","payload":payload}, ensure_ascii=False).encode("utf-8"), headers={"Content-Type":"application/json"})
print(urllib.request.urlopen(req, timeout=90).read().decode("utf-8"))
'@ | python -
```
Then run:
```powershell
@'
import json, urllib.request
for method in ("metadata.write_learning.diff", "metadata.write_learning.infer_rule"):
payload = {"learning_id": "form-command-binding-standard-customize-form"}
req = urllib.request.Request("http://docker-gpu.cin.su:8011/rpc", data=json.dumps({"method": method, "payload": payload}, ensure_ascii=False).encode("utf-8"), headers={"Content-Type":"application/json"})
print(urllib.request.urlopen(req, timeout=90).read().decode("utf-8"))
'@ | python -
```
## Experiment B: Local Form Command Binding
Goal: learn where marker `34` stores a local form command binding.
Use the same one-command runner above after the manual save.
Manual edit in Designer:
1. Open form `ТестНастройки`.
2. Select button `ФормаКомандаОбновить`.
3. Change only `CommandName` from `Form.Command.КомандаПрименить` to
`Form.Command.КомандаПример1` or `Form.Command.КомандаПример2`.
4. Save the form once.
After save, capture:
```powershell
@'
import json, urllib.request
payload = {
"base_id": "upo_test",
"learning_id": "form-command-binding-local-apply-command",
"table": "ConfigCASSave",
"file_name": "f96a0c45-3eff-11f1-8287-005056b0d483__25c39fbf-35a4-4b43-8e3f-cd1f91082c88.0",
"form": "ТестНастройки",
"element": "ФормаКомандаОбновить",
"property": "ИмяКоманды",
"timeout_seconds": 60,
"max_items": 5000
}
req = urllib.request.Request("http://docker-gpu.cin.su:8011/rpc", data=json.dumps({"method":"metadata.write_learning.capture_after","payload":payload}, ensure_ascii=False).encode("utf-8"), headers={"Content-Type":"application/json"})
print(urllib.request.urlopen(req, timeout=90).read().decode("utf-8"))
'@ | python -
```
Then run:
```powershell
@'
import json, urllib.request
for method in ("metadata.write_learning.diff", "metadata.write_learning.infer_rule"):
payload = {"learning_id": "form-command-binding-local-apply-command"}
req = urllib.request.Request("http://docker-gpu.cin.su:8011/rpc", data=json.dumps({"method": method, "payload": payload}, ensure_ascii=False).encode("utf-8"), headers={"Content-Type":"application/json"})
print(urllib.request.urlopen(req, timeout=90).read().decode("utf-8"))
'@ | python -
```
## Promotion Gate
Promote a decoder/write rule only when the after capture changes exactly one
intended command binding. If `metadata.write_learning.diff` reports
`no_changes`, use low-level `payload.diff` against the baseline payload SHA1
and implement a dedicated command-binding decoder from the scalar tree diff.
+291
View File
@@ -0,0 +1,291 @@
# 1C Form Discovery And Editing
This runbook adapts the MOXCEL discovery loop to managed forms. The goal is a
full SQL-side form decoder and safe saved-state editing through the test
extension, with XML exports used only as evidence fixtures.
## Current Baseline
- Default base: `upo_test`.
- Default adapter endpoint: `http://docker-gpu.cin.su:8011`.
- Primary test extension/object fixture:
`фс_ДоработкиОбщее` /
`DataProcessor.фс_НастройкаУсловногоОформления`.
- Primary form fixture: `ТестНастройки`.
- Existing form context evidence:
`reports/1c-sql/upo/form-context-test-nastroiki-title-resolution.json`.
- Existing saved-state smoke selectors:
`А`, `ТЗК1`, and `КомандаПример1`.
If `ConfigSave` or `ConfigCASSave` is empty, prepare the working saved-state
row through the reviewed saved-state copy flow in
`docs/1c-write-path-safety.md` before running write smokes.
## Current UPO Test Status
- Active form discovery object:
`Catalog.ЗадачиАссистентаУправления`.
- Working saved-state table: `ConfigSave`.
- Working form payloads:
`fa447250-c2a0-439d-8ba7-422923f57200.0` (`ФормаЭлемента`) and
`91ce61c5-6f4b-484a-9021-59f18be88550.0` (`ФормаСписка`).
- The saved-state copy planner now includes form payload rows (`*.0`) with
`role=form_payload`; descriptor-only copies are not enough for
`metadata.form.decode`.
- Latest decoder profile after container/dynamic-list baseline mapping:
`reports/1c-sql/upo_test/form-profile-zadachi-assistenta-configsave-after-map-live.md`.
Coverage is `484/2083` mapped, up from `252/2083`.
- Latest SQL/XML oracle comparison for `ФормаСписка`:
`reports/1c-sql/upo_test/form-sql-xml-compare-zadachi-assistenta-list-xmlmap.md`.
It currently shows `17` matched items, `44` matched properties, `41`
XML-only properties, and `12` mismatches after mapping SQL type code `8`
to `Контекстное меню`.
- `ТестНастройки` XML context is available at
`reports/1c-sql/upo_test/form-context-test-nastroiki-effective-refresh.json`,
and the live SQL saved-state rows are prepared in `ConfigCASSave` from local
CAS blobs:
`f96a0c45-3eff-11f1-8287-005056b0d483__25c39fbf-35a4-4b43-8e3f-cd1f91082c88`
plus `.0`.
- Latest `ТестНастройки` direct SQL decoder profile:
`reports/1c-sql/upo_test/form-profile-test-nastroiki-direct.md`.
Coverage is `552/2891` mapped after decoding `ExtendedTooltip` form
items from marker `12`, table additions from marker `6`, buttons from
marker `34`, form/element events, field layout details, group/table layout
properties, derived child item references, and section-level command/action
semantics.
- Latest `ТестНастройки` SQL/XML oracle comparison:
`reports/1c-sql/upo_test/form-sql-xml-compare-test-nastroiki-direct.md`.
It currently shows `68` matched items, `292` matched properties, `2`
XML-only properties, no missing SQL items, and no remaining value
mismatches after `ExtendedTooltip` decoding, form event decoding from
section `1.19`, marker `6`/`34` decoding, element event matching, derived
reference semantics, field/group/table layout semantics, plus section-aware
XML matching for dynamic-list columns and command-backed buttons. The only
remaining XML-only properties are command binding cases:
`ТЗИзменитьФорму -> Form.StandardCommand.CustomizeForm` and
`ФормаКомандаОбновить -> Form.Command.КомандаПрименить`.
- Latest `ТестНастройки` write matrix smoke:
`reports/1c-sql/upo_test/form-write-matrix-smoke-test-nastroiki-layout-table-100.json`.
It verified 100 candidates with zero failures. The current matrix has
`2912` entries, `1339` safe smoke candidates, and no not-writable entries.
A direct `metadata.write` smoke for `А.Заголовок` also completed as
`verified_and_rolled_back`.
- Latest write matrix smokes:
`reports/1c-sql/upo_test/form-write-matrix-smoke-element-50.json` and
`reports/1c-sql/upo_test/form-write-matrix-smoke-list-50.json`.
Both verified 50 candidates with zero failures.
- Latest direct `metadata.write` smoke changed `Список.Заголовок` through
`apply_and_rollback`; semantic readback verified the change and rollback
restored the original SHA1.
## Discovery Loop
Use the same shape as the MOXCEL work:
1. Create or update one controlled form fixture in the test extension.
2. Change exactly one visible form property in Designer.
3. Capture the SQL saved-state form payload before and after.
4. Compare the decoded SQL payload with exported `Ext/Form.xml` as an oracle.
5. Promote read rules only when SQL bytes reproduce XML-visible facts.
6. Promote write rules only after `apply_and_rollback` proves semantic readback
and rollback.
The adapter runtime remains SQL-only. Exported form XML is a labeling and
verification fixture, not a runtime input for adapter answers.
## Decoder Scope
The full decoder should expose these public form sections:
- form common properties and events;
- form items with stable `id`, `name`, parent/group, type, title, data path,
visibility, enabled/read-only flags, layout properties, and color/font
properties when decoded;
- form attributes, including value-table fields and dynamic-list fields;
- form commands, command bars, command-button bindings, and command handlers;
- table columns, pages, groups, decorations, input fields, labels, buttons,
extended tooltips, context menus, and dynamic lists;
- form module summary and routine/event/command link validation;
- source-aware display resolution for inherited captions:
command title, form attribute title, value-table field title, and local
form item override.
Every decoded scalar must carry enough evidence for future writes:
section, element identity, physical payload path, semantic group/name, current
value, value type, source, and verification rule.
## Test Extension Fixture Plan
Keep fixtures small and intentionally boring. Add form elements in
`ТестНастройки` or a sibling test form so each save isolates one concept:
- command button bound to `КомандаПример1`;
- local-only button title;
- element title inherited from a form command;
- element title inherited from a form attribute `А`;
- element title inherited from a value-table field `ТЗ.К1`;
- input field with visibility, availability, read-only, title location, choice
buttons, quick choice, and text editing flags;
- group/page/table layout properties: parent group, order, stretch, width,
height, command-bar location, and default item;
- color/font properties for label/button/input field;
- dynamic list with main table, custom query flag, query text, and columns;
- form events and element events with matching and missing BSL handlers;
- structural move/reorder cases inside one parent container.
Prefer one-property saves. Do not combine property, handler, and structural
changes in the same learning capture.
## Read-Side Commands
Decode a concrete saved-state form payload:
```powershell
python scripts/smoke_1c_write_matrix.py `
--base-url http://docker-gpu.cin.su:8011 `
--base-id upo_test `
--table ConfigCASSave `
--file-name <form-file-name> `
--build-only `
--report reports/1c-sql/upo_test/form-write-matrix-build.json
```
Read XML-backed form context for the test fixture:
```powershell
python scripts/get_1c_form_context.py `
--index reports/1c-sql/upo/unified-object-route-index.json `
--kind DataProcessor `
--name фс_НастройкаУсловногоОформления `
--form ТестНастройки `
--view effective `
--max-items 500 `
--output reports/1c-sql/upo_test/form-context-test-nastroiki-effective.json
```
Adapter RPC equivalents:
```json
{"method":"metadata.form.decode","payload":{"base_id":"upo_test","table":"ConfigCASSave","file_name":"<form-file-name>","include_parameters":true,"max_items":5000,"max_parameters":500}}
```
```json
{"method":"metadata.form.write_matrix.build","payload":{"base_id":"upo_test","table":"ConfigCASSave","file_name":"<form-file-name>"}}
```
Build a decoder coverage and gap profile for an object form:
```powershell
python scripts/profile_1c_forms.py `
--adapter-url http://docker-gpu.cin.su:8011 `
--base-id upo_test `
--kind Catalog `
--name ЗадачиАссистентаУправления `
--table Config `
--raw-output-json reports/1c-sql/upo_test/form-profile-zadachi-assistenta-details.json `
--output-json reports/1c-sql/upo_test/form-profile-zadachi-assistenta.json `
--output-markdown reports/1c-sql/upo_test/form-profile-zadachi-assistenta.md
```
Compare decoded SQL form semantics with exported `Form.xml` semantics:
```powershell
python scripts/compare_1c_form_sql_xml.py `
--sql-details reports/1c-sql/upo_test/form-profile-zadachi-assistenta-configsave-xmlmap-details.json `
--xml-context reports/1c-sql/upo_test/form-context-zadachi-assistenta-list-effective.json `
--output-json reports/1c-sql/upo_test/form-sql-xml-compare-zadachi-assistenta-list-xmlmap.json `
--output-markdown reports/1c-sql/upo_test/form-sql-xml-compare-zadachi-assistenta-list-xmlmap.md
```
## Write Learning
For a manual one-property Designer change:
```json
{"method":"metadata.write_learning.capture_before","payload":{"base_id":"upo_test","learning_id":"form-visible-case","table":"ConfigCASSave","form":"ТестНастройки","element":"<element-name>"}}
```
After the Designer save:
```json
{"method":"metadata.write_learning.capture_after","payload":{"base_id":"upo_test","learning_id":"form-visible-case","table":"ConfigCASSave","form":"ТестНастройки","element":"<element-name>"}}
{"method":"metadata.write_learning.diff","payload":{"learning_id":"form-visible-case"}}
{"method":"metadata.write_learning.infer_rule","payload":{"learning_id":"form-visible-case"}}
```
Promote a rule only when the diff changes exactly one intended semantic value
or one intended structural relation. Composite/list rewrites need a dedicated
source-specific rule, not a generic scalar writer.
## Write Verification
Run the existing source-aware route smoke:
```powershell
python scripts/smoke_1c_saved_state_write_routes.py `
--base-url http://docker-gpu.cin.su:8011 `
--base-id upo_test `
--table ConfigCASSave `
--file-name <form-file-name> `
--report reports/1c-sql/upo_test/saved-state-write-routes-smoke.json
```
Then run the matrix smoke:
```powershell
python scripts/smoke_1c_write_matrix.py `
--base-url http://docker-gpu.cin.su:8011 `
--base-id upo_test `
--table ConfigCASSave `
--file-name <form-file-name> `
--max-candidates 50 `
--learning-id upo-test-form-write-matrix `
--report reports/1c-sql/upo_test/form-write-matrix-smoke-50.json
```
Successful writes must use `apply_and_rollback`, explicit SQL apply and
rollback gates, sha1 preconditions, backup evidence, semantic readback through
`metadata.form.decode`, and rollback verification.
## Promotion Gates
Read rule promotion requires:
- SQL-only decoder output with stable semantic name and value type;
- XML fixture agreement for the same form element/property;
- no dependency on display strings when a stable id/path exists;
- regression coverage on the test form and at least one real extension form.
Write rule promotion requires:
- exact physical payload path or structural span evidence;
- source-aware routing for inherited display values;
- `metadata.form.write_target.resolve` success with a deterministic target;
- `metadata.form.element.write_apply` or `metadata.write` success in
`apply_and_rollback`;
- semantic verification and rollback readback success;
- registration in the scalar/enum/verified write matrix reports.
## Immediate Work Queue
1. Refresh the test form saved-state row for `upo_test` if
`ConfigCASSave`/`ConfigSave` is empty.
2. Capture a fresh `metadata.form.decode` baseline for `ТестНастройки`.
3. Build a form property gap report: decoded SQL semantics versus `Ext/Form.xml`
semantics from the test extension.
4. Learn the two remaining command binding cases with a one-property
before/after capture: standard command button binding and local button to
form command binding. Do not hard-code these from display names. Current
captures and exact after-capture commands are in
`docs/runbooks/1c-form-command-binding-learning.md`.
5. Expand `parser/form_payload.py` for the next write-relevant properties:
availability, read-only, title location, command-bar location, colors, font,
and dynamic-list query settings.
5. Rebuild the write matrix and split entries into verified scalar, enum,
composite-needs-rule, identity/binding, and structural queues.
6. Learn one property at a time through
`metadata.write_learning.capture_before/capture_after/diff/infer_rule`.
7. Promote safe scalar/enum routes into smoke coverage.
8. Add a structural movement scorecard for sibling reorder and parent/group
movement, then extend `metadata.form.target.move` beyond sibling swaps only
after controlled round-trip proof.
+55
View File
@@ -0,0 +1,55 @@
# 1C Live Interaction
Цель: безопасно связать модель с живыми базами 1С.
## Layers
1. Connector API: `plugins/1c/connector/contracts/openapi.yaml`
2. Metadata snapshots: `plugins/1c/schemas/metadata-snapshot-v2.schema.json`
3. BSL module snapshots: `plugins/1c/schemas/bsl-module-snapshot.schema.json`
4. Read-only query policy: `plugins/1c/connector/policies/read-only-query.yaml`
5. Change workflow policy: `plugins/1c/connector/policies/change-workflow.yaml`
For day-to-day development, use the faster operational loop instead of full XML/EDT sync on every task:
- run read-only SQL for diagnostics and data samples;
- get metadata and BSL through a lightweight 1C agent or exported JSON snapshot;
- refresh cached snapshots by configuration version/checksum;
- generate external reports, data processors, extensions, or reviewable patches.
Details: `docs/runbooks/1c-operational-coding.md`.
## Query Validation
```powershell
python scripts/validate_1c_readonly_query.py --query "ВЫБРАТЬ Первые 10 Ссылка ИЗ Справочник.Номенклатура"
```
Denied example:
```powershell
python scripts/validate_1c_readonly_query.py --query "УДАЛИТЬ ИЗ Справочник.Номенклатура"
```
## BSL Module Snapshot
```powershell
python scripts/validate_1c_bsl_modules.py plugins/1c/metadata/examples/bsl-modules.example.json
python scripts/convert_1c_bsl_modules_to_rag.py --input plugins/1c/metadata/examples/bsl-modules.example.json --output plugins/1c/rag/sources/bsl-modules.generated.md
```
## Safety Rule
The model may:
- inspect metadata;
- search/read modules;
- validate read-only queries;
- propose changes.
The model must not:
- directly change a live database;
- run destructive queries;
- reveal secrets or personal data;
- invent metadata when connector data is missing.
+120
View File
@@ -0,0 +1,120 @@
# 1C LoRA Training
Цель: обучить draft LoRA adapter `qwen3-coder-30b-a3b-1c-lora-v1` поверх `qwen3-coder-30b-a3b-instruct`.
## Preconditions
- Полностью скачана базовая модель: `/models/base/qwen3-coder-30b-a3b-instruct`.
- Подготовлен датасет: `plugins/1c/training/prepared/train.chat.jsonl`.
- Есть GPU/CUDA на `docker-gpu.cin.su`.
- В датасете достаточно проверенных примеров. Синтетические 2 записи подходят только для smoke-run, не для полезного качества.
## Current Preflight Status
На 2026-07-04 локальный preflight для нового Qwen3-Coder training contour не стартует, потому что:
- локально нет CUDA/GPU;
- training-зависимости не установлены в локальный Python;
- HF-база `qwen3-coder-30b-a3b-instruct` еще не лежит в `/models/base/qwen3-coder-30b-a3b-instruct` на текущем workspace path;
- запускать обучение нужно на `docker-gpu.cin.su`, потому что именно там есть GPU-контур для этой модели.
Если файл будет удален или поврежден, восстановить/докачать базовую модель можно так:
```powershell
python scripts/download_hf_range.py qwen3-coder-30b-a3b-instruct `
--local-dir models/base/qwen3-coder-30b-a3b-instruct `
--chunk-size 16mb `
--retries 20
```
## Prepare Dataset
```powershell
python scripts/validate_1c_training_data.py plugins/1c/training/examples/instruction.examples.jsonl
python scripts/prepare_1c_training_data.py --input plugins/1c/training/examples/instruction.examples.jsonl --output plugins/1c/training/prepared/train.chat.jsonl
```
## Local Preflight
```powershell
python scripts/preflight_1c_training.py
```
## Dry Run
```powershell
python scripts/train_1c_lora.py --dry-run
```
## GPU Docker Run
```powershell
docker --host ssh://docker-gpu.cin.su compose --env-file core/deploy/docker-gpu/training/1c-lora.env.example -f core/deploy/docker-gpu/training/1c-lora.compose.yaml up --abort-on-container-exit
```
Или через готовый wrapper:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/run_1c_lora_training_gpu.ps1
```
Full end-to-end orchestration for the current `Q6` route:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/train_and_publish_q6_lora.ps1
```
Preview the whole flow without executing:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/train_and_publish_q6_lora.ps1 -PlanOnly
```
Troubleshooting:
```text
docs/runbooks/q6-lora-troubleshooting.md
```
## Output
Adapter path:
```text
/models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1
```
After a successful training run:
1. Run `plugins/1c/evals/smoke.yaml`.
2. Convert the adapter for `llama.cpp` GGUF format or merge it before rebuilding the Q6 GGUF deployment artifact.
GGUF adapter export on `docker-gpu`:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/convert_1c_lora_to_gguf_gpu.ps1
```
The default output path is:
```text
/models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1.gguf
```
If the export container should reuse an existing `llama.cpp` checkout on the host without pulling updates:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/convert_1c_lora_to_gguf_gpu.ps1 -SkipClone
```
3. To launch the current GPU Q6 route with a converted adapter, use:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/manage_gpu_q6_service.ps1 `
-Action start `
-LoraPath /models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1.gguf
```
If you need a custom adapter scale, pass `-LoraScale 0.5` or another value.
4. Compare base, RAG, and adapter outputs.
5. Promote model card from `draft` only after expert review.
+50
View File
@@ -0,0 +1,50 @@
# 1C Metadata Snapshot
Цель: сохранять структуру 1С как локальный snapshot и использовать ее в RAG без выдумывания объектов конфигурации.
## Files
- Schema: `plugins/1c/metadata/schema.json`
- Example: `plugins/1c/metadata/examples/metadata.example.json`
- Local snapshots: `plugins/1c/metadata/snapshots`
- Converter: `scripts/convert_1c_metadata_to_rag.py`
## Snapshot Rule
Рабочие snapshot-файлы считаются локальными артефактами. Они не должны содержать:
- пароли;
- токены;
- строки подключения;
- персональные данные;
- клиентские секреты;
- выгрузки данных.
Snapshot описывает только структуру метаданных.
## Convert Example To RAG Source
```powershell
python scripts/validate_1c_metadata_snapshot.py plugins/1c/metadata/examples/metadata.example.json
```
```powershell
python scripts/convert_1c_metadata_to_rag.py --input plugins/1c/metadata/examples/metadata.example.json --output plugins/1c/rag/sources/metadata.example.generated.md
```
После этого можно перестроить корпус и индекс:
```powershell
python scripts/prepare_1c_rag_corpus.py
python scripts/build_1c_rag_index.py
```
Команды нужно выполнять последовательно: сначала подготовить JSONL-корпус, затем строить индекс.
## Ask
```powershell
python scripts/ask_1c_rag.py "Какие реквизиты есть у справочника Номенклатура?" --print-prompt
```
В prompt должны попасть реквизиты из snapshot и ссылка на source `metadata.example.generated.md`.
+337
View File
@@ -0,0 +1,337 @@
# 1C MOXCEL Discovery
This runbook describes the read-only discovery loop for tabular document
MOXCEL payloads.
## Current Artifacts
- `reports/1c-template-baselines/Primer3_moxel_schema_discovery.json`
contains inferred decoder rules from marker matrices and history diffs.
- `reports/1c-template-baselines/Primer3_moxel_property_experiments.json`
contains one-property experiment analysis and the next probe plan.
- `reports/1c-template-baselines/Primer3_moxel_property_experiments.manifest.json`
maps existing before/after summaries to property experiment labels.
## Schema Discovery
Run marker/history discovery:
```powershell
python scripts/discover_1c_moxel_schema.py `
--marker-matrix reports/1c-template-baselines/Primer3_marker_matrix_2026-06-27_latest.json `
--history-matrix reports/1c-template-baselines/Primer3_history_matrix.json `
--property-candidates reports/1c-template-baselines/Primer3_property_candidates.json `
--output-json reports/1c-template-baselines/Primer3_moxel_schema_discovery.json `
--output-markdown reports/1c-template-baselines/Primer3_moxel_schema_discovery.md
```
The current strongest rule is:
```text
moxel.inline_text_cell.column:
one_based_column = int(last_numeric(preceding_scalars)) + 1
```
Named range coordinates are partially proven from controlled moves:
```text
left/right: raw scalar indexes 2 and 4
top/bottom: raw scalar indexes 3 and 5
one_based = raw + 1
```
Use multi-cell and rectangular named ranges to split `left` from `right` and
`top` from `bottom`.
Build the stable registry after discovery:
```powershell
python scripts/build_1c_moxel_schema_registry.py `
--discovery reports/1c-template-probes/upo_test_auto_moxel_schema_discovery.json `
--discovery reports/1c-template-baselines/Primer3_moxel_schema_discovery.json `
--output-json plugins/1c/metadata/moxel-schema-registry.json `
--output-markdown reports/1c-template-baselines/moxel-schema-registry.md
```
Validate the registry safety contract:
```powershell
python scripts/check_1c_moxel_schema_registry.py `
--registry plugins/1c/metadata/moxel-schema-registry.json `
--output reports/1c-template-baselines/moxel-schema-registry-check.json
```
The registry allows read-side decoder rules only. Write-side MOXCEL mutation is
kept blocked until a disposable-base round-trip proves the exact scalar path.
Verify registry rules against concrete probe snapshots:
```powershell
python scripts/verify_1c_moxel_schema_registry.py `
--registry plugins/1c/metadata/moxel-schema-registry.json `
--probe reports/1c-template-probes/upo_test_auto_20260627T142419Z_670780b4.json `
--output reports/1c-template-baselines/moxel-schema-registry-verification.json
```
This check is data-backed: `verified_read` rules must pass on the supplied
probe snapshots, while `candidate_read` rules are reported as diagnostic
evidence and are not promoted automatically.
## Full Pipeline
After a new probe or one-property experiment has been captured, refresh all
MOXCEL discovery artifacts with one command:
```powershell
python scripts/run_1c_moxel_discovery_pipeline.py
```
To capture the latest live MOXCEL probe first and then refresh all artifacts:
```powershell
python scripts/run_1c_moxel_discovery_pipeline.py --capture-live --label pipeline-live
```
The pipeline runs schema discovery, named-range rule analysis, property
experiment analysis, registry build, registry safety check, registry
verification, and the next experiment plan. It writes:
- `reports/1c-template-baselines/moxel-discovery-pipeline.json`
- `reports/1c-template-baselines/moxel-discovery-pipeline.md`
- `reports/1c-template-probes/latest-live-probe.json` when `--capture-live` is used
- `reports/1c-template-baselines/moxel-named-range-rules.md`
- `reports/1c-template-baselines/moxel-next-experiments.md`
- `reports/1c-template-baselines/moxel-next-action.json`
- `reports/1c-template-baselines/moxel-next-action.md`
- `reports/1c-template-baselines/moxel-next-action-check.json`
- `reports/1c-template-baselines/moxel-status.md`
When `moxel-named-range-rules.json` contains high-confidence rectangular range
evidence, the registry build step automatically promotes the corresponding
named-range read rules from `candidate_read` to `verified_read`. Write status
still remains blocked until a separate round-trip proof exists.
## Property Experiments
Generate or refresh the property probe plan:
```powershell
python scripts/analyze_1c_moxel_property_experiments.py `
--emit-default-plan `
--output-json reports/1c-template-baselines/Primer3_moxel_property_experiments.json `
--output-markdown reports/1c-template-baselines/Primer3_moxel_property_experiments.md
```
Analyze existing before/after experiments:
```powershell
python scripts/analyze_1c_moxel_property_experiments.py `
--manifest reports/1c-template-baselines/Primer3_moxel_property_experiments.manifest.json `
--emit-default-plan `
--output-json reports/1c-template-baselines/Primer3_moxel_property_experiments.json `
--output-markdown reports/1c-template-baselines/Primer3_moxel_property_experiments.md
```
Each new experiment should change exactly one property on the same tracked cell
or range, then capture a fresh `templates.map`/summary snapshot.
To capture the next manual one-property save automatically, start the watcher
before changing and saving the template in 1C:
```powershell
python scripts/watch_1c_moxel_property_experiment.py `
--property ВертикальноеПоложение `
--target-text "Ячейка 7 - 2" `
--target-name R7C2_TEST `
--timeout-seconds 600 `
--run-pipeline-after
```
The watcher captures a `before` snapshot, waits for a new latest MOXCEL payload,
captures `after`, writes Markdown/JSON snapshots to
`reports/1c-template-probes`, and appends the experiment to
`reports/1c-template-baselines/Primer3_moxel_property_experiments.manifest.json`.
With `--run-pipeline-after`, it also refreshes schema discovery, property
analysis, registry build/check/verify, and the next experiment plan.
Prioritize:
- `ГоризонтальноеПоложение`
- `ВертикальноеПоложение`
- `ЦветТекста`
- `ЦветФона`
- `Шрифт.Имя`
- `Шрифт.Размер`
- borders
- `Защита`
- `Гиперссылка`
- wrapping
- column width
- row height
- merge ranges
Only promote a property path into write support after a disposable-base
round-trip proves that changing that scalar affects only the intended property.
## XML-Assisted Fixture Plan
Use XML exports only as analysis fixtures. The adapter runtime stays SQL-only:
it reads `Config`/`ConfigCAS` payloads, while exported `Ext/Template.xml`
files are used to label and verify decoder hypotheses.
Create a small extension with controlled templates and export it to XML after
each controlled save:
- one tabular document with sparse text cells, direct parameters, placeholders,
and empty formatted cells;
- one tabular document with horizontal, vertical, and rectangular merged cells;
- one tabular document with named areas and named ranges, including duplicate
names in different positions;
- one tabular document dedicated to format changes: column width, row height,
horizontal/vertical alignment, border, font, text color, background color,
protection, wrapping;
- one fixture per non-tabular template type where possible: text document,
binary data, HTML document, graphical/geographical schema, data composition
schema, data composition appearance template, external component.
The concrete merge fixture checklist is stored in:
- `reports/1c-template-baselines/moxel-controlled-merge-fixtures.json`
- `reports/1c-template-baselines/moxel-controlled-merge-fixtures.md`
Start with the merge fixtures `MOXEL_Merge_None_Grid`,
`MOXEL_Merge_H_R5C18_W3`, `MOXEL_Merge_H_R6C2_W15`,
`MOXEL_Merge_V_R5C2_H3`, `MOXEL_Merge_Rect_R5C2_R7C4`, and
`MOXEL_Merge_Mixed_4Ranges`. They are designed to split column edges, row
edges, width/height, and merge-record ordering without relying on the large
production print forms.
For MOXCEL discovery, change exactly one property per save, capture the SQL
payload, then compare it with the exported XML shape. Promote read rules only
when the SQL decoder can reproduce XML-visible facts from SQL bytes alone.
Use `scripts/analyze_1c_template_xml_profiles.py` to build XML fixture
profiles without feeding XML into the adapter runtime. Example:
```powershell
python scripts\analyze_1c_template_xml_profiles.py `
--root "Z:\codex\1C\XML\UPO\Структура базы 1с\Конфигурация\Documents\АвансовыйОтчет" `
--output-json reports\1c-template-baselines\xml-template-profiles-avansovy-otchet.json `
--output-markdown reports\1c-template-baselines\xml-template-profiles-avansovy-otchet.md
```
Current `АвансовыйОтчет` XML fixture facts:
- `ПФ_MXL_АвансовыйОтчет`: capacity/used `75x26`, `961` cells,
`196` text values, `77` parameters, `108` merges, `233` distinct format
indexes.
- `ПФ_MXL_АвансовыйОтчетВВалюте`: capacity/used `72x26`, `938` cells,
`188` text values, `75` parameters, `101` merges, `233` distinct format
indexes.
The SQL decoder should eventually reproduce these facts from SQL payloads:
`capacity_dimensions`/`used_dimensions`, coordinate-bound cell text and
parameters, authoritative `merged_ranges`, and format indexes/styles. XML
profiles are evidence for decoder hypotheses, not an input source for adapter
answers.
Compare the SQL-decoded baseline with the XML fixture profile after decoder
changes:
```powershell
python scripts\profile_1c_tabular_templates.py `
--adapter-url http://docker-gpu.cin.su:8011 `
--base-id upo_test `
--inventory-json reports\1c-template-baselines\upo_test_configuration_tabular_templates.json `
--output-json reports\1c-template-baselines\upo_test_tabular_template_profiles.json `
--output-markdown reports\1c-template-baselines\upo_test_tabular_template_profiles.md
```
```powershell
python scripts\compare_1c_template_sql_xml_profiles.py `
--sql-profile reports\1c-template-baselines\upo_test_tabular_template_profiles.json `
--xml-profile reports\1c-template-baselines\xml-template-profiles-avansovy-otchet.json `
--output-json reports\1c-template-baselines\sql-xml-template-profile-compare-avansovy-otchet.json `
--output-markdown reports\1c-template-baselines\sql-xml-template-profile-compare-avansovy-otchet.md
```
The current comparison intentionally reports gaps for both `АвансовыйОтчет`
templates. SQL capacity still reports the MOXCEL allocation `128x72`, while XML
spreadsheet dimensions are `75x26`/`72x26`. With `template_part_moxel_v8`,
hint-aware SQL `used_dimensions` improved to `74x25` and `71x25`; the remaining
edge likely depends on merge/format records. Limited cell/parameter counts,
missing authoritative `merged_ranges`, and missing format index coverage remain
open. Use these gaps as the next decoder scorecard; a gap should only disappear
when SQL bytes alone reproduce the XML-visible fact.
The compare report also shows `Progress signals`. A non-zero
`cell_coordinate_hints_available` signal means the SQL decoder recovered
coordinate evidence from hints, but the `cells_missing_or_limited` gap stays
open until authoritative cell rows/columns are decoded.
For merge-block row/size reverse engineering, regenerate the row-band report:
```powershell
python scripts\analyze_1c_moxel_merge_row_bands.py `
--template ПФ_MXL_АвансовыйОтчет `
--template ПФ_MXL_АвансовыйОтчетВВалюте `
--output-json reports\1c-template-baselines\moxel-merge-row-band-analysis-avansovy-otchet.json `
--output-markdown reports\1c-template-baselines\moxel-merge-row-band-analysis-avansovy-otchet.md
```
This report compares XML merge rows with SQL MOXCEL small-scalar bands and
packed `scalar/32` column-edge evidence. Treat `row_or_size_hints` as low
confidence until a controlled merge fixture proves which scalar positions are
row indexes versus widths, heights, or flags.
`merge_record_block_candidates[].evidence.record_analysis` exposes a normalized
SQL-only view of the candidate block: per-record shape/head, packed
`scalar/32` values, small scalars, scalar slot summaries, value-to-record runs,
and sample records. Use it for slot-formula discovery; it is diagnostic
evidence and does not make `merged_ranges` authoritative by itself.
To score candidate numeric slots against XML-visible merge fields, run:
```powershell
python scripts\analyze_1c_moxel_merge_slot_candidates.py `
--template ПФ_MXL_АвансовыйОтчет `
--template ПФ_MXL_АвансовыйОтчетВВалюте `
--output-json reports\1c-template-baselines\moxel-merge-slot-candidates-avansovy-otchet.json `
--output-markdown reports\1c-template-baselines\moxel-merge-slot-candidates-avansovy-otchet.md
```
The current large-form slot report is intentionally hypothesis-only. It shows
that simple value-set and ordered-offset matching are dominated by low-entropy
height/flag values and do not prove a top/left/bottom/right formula. Use the
controlled merge fixtures before promoting any SQL rule into authoritative
`merged_ranges`.
Inline text style candidates expose `coordinate_hints` for the column using the
discovered rule
`inline_text_column_from_last_preceding_scalar_plus_one`. Treat this as a
high-confidence hint for analysis and controlled experiments, not as an
authoritative cell coordinate until row and merge rules are proven by SQL/XML
round trips.
The adapter and SQL profile reports include `cell_style_coordinate_hints` in
counts. Track this count separately from `cells`: it measures how much
coordinate evidence was recovered from inline style records, while `cells`
remains reserved for decoded row/cell runs with authoritative coordinates.
`cell_coordinate_hints` combines the style-derived column hint with a matched
decoded cell row when the text/cell id can be linked. Use it to inspect
text-to-coordinate evidence during decoder discovery. Do not count it as
authoritative `cells` coverage until the row rule and merge interactions are
validated against SQL/XML fixtures.
For compact inspection, request only coordinate evidence from `templates.map`
or `templates.analyze` with `sections=coordinate_hints`. This returns
`cell_coordinate_hints` and `cell_style_coordinate_hints` from both the
structure and analysis layers without dumping all cells/styles.
Coordinate hints, hint-aware `used_dimensions`, merge-record block candidates,
SQL-only merge-block column-edge hints, low-confidence row/size scalar hints,
and merge-block `record_analysis` changed the decoded artifact schema, so
MOXCEL template cache uses `template_part_moxel_v9`. Refresh SQL baselines after
deploying the new adapter; old `v2`/`v3`/`v4` decoded artifacts will not contain
the current `cell_coordinate_hints`, merge-block evidence, and used-dimension
semantics.
+219
View File
@@ -0,0 +1,219 @@
# 1C Operational Coding Loop
Цель: сделать помощника по 1С, который работает в темпе реальной разработки, без постоянной полной выгрузки конфигурации в XML и без обязательного 1C:EDT на первом этапе.
## Problem
Полная выгрузка конфигурации в XML медленная. 1C:EDT требует отдельной установки, настройки проекта и дисциплины синхронизации. Для оперативной разработки помощнику нужны текущие данные почти сразу:
- структура базы и конфигурации;
- доступные объекты, реквизиты, табличные части, формы и команды;
- актуальные модули BSL;
- примеры реальных данных для read-only анализа;
- постановки задач из текста, Excel-файлов и скриншотов интерфейса.
## Decision
Используем двухконтурную схему.
Быстрый контур:
- read-only SQL для диагностики, выборок и проверки данных;
- легкий 1C agent внутри базы или рядом с ней для метаданных, модулей и управляемых операций;
- локальный кеш/snapshot с коротким TTL;
- RAG поверх актуального кеша;
- генерация патчей, внешних отчетов, обработок и расширений как артефактов.
Тяжелый контур:
- XML/EDT/хранилище конфигурации для периодической полной синхронизации;
- сборка, ревью, массовый рефакторинг и долгоживущие изменения;
- финальная проверка перед переносом в production.
## Live Sources
### SQL Read-Only
SQL удобен как быстрый источник данных, но не является главным источником метаданных 1С.
Разрешено:
- read-only запросы;
- выборки для отчетов и сверок;
- оценка объемов данных;
- поиск аномалий;
- проверка результата после изменения в тестовой базе.
Запрещено:
- DML/DDL;
- изменение таблиц платформы напрямую;
- запись в production;
- хранение строк подключения и паролей в репозитории.
## 1C Storage Layers (write/read boundary)
Принцип работы со слоями конфигурации:
- `Config` и `ConfigCAS`**active** (уже применённое в системе состояние). Для них разрешены только read-операции.
- `ConfigSave` и `ConfigCASSave`**saved, not yet applied** (сохранённое в конфигураторе состояние). Это целевые слои для формирования изменений через адаптер.
- Base-изменения пишутся в `ConfigSave`.
- Extension-изменения пишутся в `ConfigCASSave`.
Жёсткое правило:
- Коннектор/агент не пишет в `Config`/`ConfigCAS`.
- Изменения должны идти через saved-слои и проходить сравнение `ConfigSave↔Config`, `ConfigCASSave↔ConfigCAS` до ручного/внешнего apply в production.
- Если требуется production apply, это отдельный человеческий процесс контроля и проверки.
Мини-чеклист перед передачей на manual-apply:
1. Есть актуальный compare saved-vs-active.
2. Есть report с изменениями по объектам.
3. Есть отметка «не применено» в `ConfigSave`/`ConfigCASSave`.
4. Есть rollback-план и явное human approval.
### Working-State Read Policy
Для программирования и оперативного анализа помощник должен читать последнее
сохранённое состояние конфигуратора, а не только применённую конфигурацию.
По умолчанию:
- MCP-запросы к `extension.objects.find`, `modules.search`, `code.search` и
`metadata.resolve_overrides` используют `source_state=working`;
- REST-запросы к тем же методам используют `state=working`;
- `working` означает: сначала `ConfigSave`/`ConfigCASSave`, затем active-слой
как fallback;
- результаты помечаются `activation_state`: `saved_only`, `saved_override` или
`active`.
Когда нужно сравнение:
- `source_state=applied` / `state=active` — показать только применённое;
- `source_state=all` / `state=both` — показать оба слоя и различия;
- `full_scan=true` включается только осознанно для глубокого поиска по active
`ConfigCAS`, потому что такой поиск медленнее.
Если пользователь спрашивает естественным языком вроде «выдай список всех форм
в save только имена», агент должен трактовать это как working/save-first
срез, вернуть имена saved-форм и не отбрасывать объекты `saved_only`: они могут
быть ещё не активированы и всё равно являются текущим состоянием разработки.
### Code Write Policy
Для оперативного программирования агент не должен знать SQL-нюансы: таблицы,
имена файлов, stream indexes и brace paths являются внутренней реализацией
адаптера.
Стандартный write API для агента:
- `code.write` с `module_text`/`full_text`/`code` заменяет весь модуль;
- `code.write` с `routine_name` и `routine_text` заменяет только указанную
процедуру или функцию;
- `code.write` с `old` и `new` заменяет фрагмент только если `old` найден
ровно один раз в выбранной области. Если передан `routine_name` или
canonical path до процедуры, областью является эта процедура/функция; иначе
весь текущий saved-модуль.
По умолчанию `code.write` делает `mode=apply`, но это apply в saved-state
слой (`ConfigSave`/`ConfigCASSave`), а не применение конфигурации в runtime.
Адаптер сам выставляет save-first gates и сам выбирает физический маршрут.
Физические детали возвращаются только при `include_storage=true` для
диагностики. Ответ `code.write` всегда содержит `write_mode`: target
`saved_state`, activation_state `not_activated`, production_apply `false`.
Ответы `code.read` и `code.search` для saved-кода содержат `current_state`:
source `saved_state`, activation_state `not_activated`.
Для `code.read`: `state=working` означает save-first с fallback в active,
`state=save` читает только saved layer, `state=active` пропускает saved layer.
`state=both` читает saved и active отдельно, возвращает `layers` для обоих
слоев и `comparison.both_present` / `comparison.differs`. При `include_text=true`
верхнеуровневый `text` берется из saved-state, если он есть, иначе из active;
`text_source` показывает выбранный слой.
Для `code.search state=both` saved-совпадения идут первыми, active-слой
добирается отдельным проходом, а `counts.saved_matches` и
`counts.active_matches` показывают покрытие по слоям.
Если фрагмент повторяется, агент должен передать более узкий контекст
(`routine_name`) или заменить процедуру целиком. Адаптер в такой ситуации
возвращает `ambiguous_fragment`, `scope` и `counts.occurrences`, и не
записывает.
### 1C Agent
Для актуальной структуры и кода нужен небольшой агент на стороне 1С. Он может быть реализован как внешняя обработка, расширение или опубликованный HTTP-сервис.
Минимальные функции:
- вернуть список объектов метаданных;
- вернуть описание объекта: реквизиты, табличные части, формы, команды, модули;
- искать по BSL-модулям;
- читать текст выбранного модуля;
- выполнять read-only запрос языка запросов 1С с лимитами;
- отдавать версию/дату изменения конфигурации для инвалидации кеша;
- принимать change proposal, но не применять его автоматически в production.
Если публикация HTTP-сервиса невозможна, первым вариантом может быть ручной запуск внешней обработки, которая выгружает JSON snapshot в общую папку.
## Freshness Model
Модель не должна считать кеш вечным.
- SQL read-only данные читаются по запросу.
- Metadata snapshot имеет TTL и номер версии конфигурации.
- BSL-модули кешируются с checksum.
- Перед генерацией кода под конкретный объект помощник проверяет свежесть metadata snapshot.
- Если snapshot устарел или отсутствует, помощник запрашивает обновление через agent.
## Task Intake
Задачи приходят разными форматами:
- обычный текст;
- Excel-файл с требованиями, примером отчета или справочником полей;
- скриншот формы, нарисованный макет интерфейса или ошибка;
- фрагмент BSL;
- SQL/запрос 1С;
- описание бизнес-процесса.
Обработка:
- текст идет напрямую в модель;
- Excel парсится структурно: листы, заголовки, таблицы, примечания;
- скриншоты идут через vision/OCR и превращаются в описание интерфейса, полей, команд и ошибок;
- все извлеченные требования связываются с metadata snapshot и BSL search.
## Coding Outputs
Помощник должен уметь выдавать:
- BSL-фрагмент;
- полный модуль формы/объекта/общего модуля;
- текст запроса 1С;
- схему внешнего отчета или обработки;
- proposal для расширения;
- список изменений по формам и командам;
- чеклист проверки;
- тестовые сценарии;
- rollback plan.
Для production изменения не применяются напрямую. Нормальный поток:
1. Получить актуальные метаданные и модули.
2. Сформировать change proposal.
3. Проверить синтаксис и зависимости в тестовой базе.
4. Сформировать артефакт: внешняя обработка, отчет, расширение или патч.
5. Провести ревью.
6. Перенести через согласованный 1С-процесс.
## Practical Priority
Первый рабочий MVP:
1. SQL read-only connector с валидацией политики.
2. 1C metadata snapshot через внешнюю обработку в JSON.
3. BSL module snapshot и поиск по модулям.
4. Загрузка постановки из Excel.
5. Загрузка скриншота формы/ошибки в vision-модель.
6. Генерация внешнего отчета/обработки по актуальному snapshot.
7. Smoke eval: модель не выдумывает реквизиты и просит обновить snapshot, если он устарел.
+36
View File
@@ -0,0 +1,36 @@
# 1C Plugin Health
Цель: одной командой проверить, что 1С-плагин собран консистентно.
## Run
```powershell
python scripts/check_1c_plugin.py --print
```
Короткий статус:
```powershell
python scripts/status_1c_plugin.py
```
Отчет пишется в:
```text
reports/1c-plugin-health.json
```
`reports/` игнорируется git.
## Checks
Скрипт проверяет:
- наличие обязательных файлов плагина;
- metadata snapshot example;
- training examples и secret-scan;
- eval YAML;
- RAG prompt guardrails;
- training preflight.
`training_preflight` может быть `blocked`, если нет GPU, полной базовой модели или зависимостей. Это не ломает healthcheck как структурную проверку плагина.
+286
View File
@@ -0,0 +1,286 @@
# 1C RAG
Цель: подготовить локальный корпус знаний по 1С для поиска и ответов без дообучения модели.
## Principles
- Сначала RAG и инструменты, потом fine-tuning.
- Не загружать в репозиторий базы, выгрузки, приватные документы, персональные данные и секреты.
- Не выдумывать метаданные конкретной базы 1С. Если нужны реквизиты или объекты, использовать tool boundary.
## Add Sources
Кладем материалы в:
```text
plugins/1c/rag/sources
```
Поддерживаемые форматы:
- `.md`
- `.txt`
- `.bsl`
- `.os`
## Prepare Corpus
```powershell
python scripts/validate_1c_rag_sources.py --print
python scripts/prepare_1c_rag_corpus.py
```
Результат:
```text
plugins/1c/datasets/prepared/rag_corpus.jsonl
plugins/1c/datasets/prepared/rag_manifest.json
```
Этот файл не коммитится.
## Test With Example
```powershell
python scripts/prepare_1c_rag_corpus.py --source-dir plugins/1c/rag/examples --output plugins/1c/datasets/prepared/rag_corpus.example.jsonl
```
## Build Search Index
```powershell
python scripts/build_1c_rag_index.py
```
Для синтетического примера:
```powershell
python scripts/build_1c_rag_index.py --corpus plugins/1c/datasets/prepared/rag_corpus.example.jsonl --output plugins/1c/datasets/prepared/rag_index.example.json
```
## Build Vector Index
```powershell
python scripts/build_1c_rag_vector_index.py
```
Результат:
```text
plugins/1c/datasets/prepared/rag_vector_index.sqlite
```
Индекс хранит `corpus_hash`, `embedding_model`, размерность, дату сборки и метаданные чанков.
Текущий provider `local-hashing-v1` - это локальный deterministic vector baseline, а не нейросемантическая embedding-модель. Он нужен, чтобы отладить формат, freshness и hybrid retrieval без скачивания модели. Нейросемантический provider подключается следующим слоем без смены SQLite-схемы.
OpenAI-compatible embedding endpoint:
```powershell
$env:EMBEDDING_API_KEY = "<token-if-needed>"
python scripts/build_1c_rag_vector_index.py `
--embedding-provider openai-compatible `
--embedding-model "<embedding-model-name>" `
--embedding-base-url "http://docker-gpu.cin.su:8000" `
--embedding-api-key-env EMBEDDING_API_KEY
```
Для поиска по такому индексу query embedding должен считаться тем же provider/model:
```powershell
python scripts/search_1c_rag_hybrid.py "реквизиты справочника номенклатура" `
--profile metadata `
--embedding-base-url "http://docker-gpu.cin.su:8000" `
--embedding-api-key-env EMBEDDING_API_KEY
```
Ключ не пишется в индекс и читается только из переменной окружения.
Полная сборка knowledge base теперь строит corpus, lexical index и vector index:
```powershell
python scripts/build_1c_knowledge_base.py --include-example-bsl
```
Если vector index временно не нужен:
```powershell
python scripts/build_1c_knowledge_base.py --skip-vector-index
```
## Search
```powershell
python scripts/search_1c_rag.py "метаданные справочника" --index plugins/1c/datasets/prepared/rag_index.example.json
```
Поиск использует локальный lexical BM25-подобный индекс с нормализацией 1С-терминов и алиасами вроде `1с/bsl`, `справочник/catalog`, `метаданные/metadata`.
Профили RAG лежат в:
```text
plugins/1c/rag/profiles.yaml
```
Основные профили:
- `auto`
- `general`
- `metadata`
- `bsl`
- `query`
- `safe-change`
`auto` выбирает профиль по тексту вопроса. Например, вопросы про реквизиты уходят в `metadata`, вопросы про процедуры и ошибки BSL - в `bsl`, вопросы с `ВЫБРАТЬ` - в `query`.
При подготовке корпуса источники получают смысловой `source_type`:
- `metadata`
- `bsl`
- `query`
- `safety`
- `docs`
Полезные параметры:
```powershell
python scripts/search_1c_rag.py "реквизиты справочника номенклатура" `
--index plugins/1c/datasets/prepared/rag_index.example.json `
--profile metadata `
--limit 5 `
--candidate-limit 30 `
--dedupe-by-document
```
Vector search:
```powershell
python scripts/search_1c_rag_vector.py "реквизиты справочника номенклатура" --profile metadata --limit 5
```
Hybrid search объединяет lexical и vector результаты через reciprocal-rank fusion:
```powershell
python scripts/search_1c_rag_hybrid.py "реквизиты справочника номенклатура" --profile metadata --limit 5
```
Если `vector_freshness.status = stale`, нужно пересобрать vector index после обновления corpus.
## Ask With Context
Без вызова модели, только сборка prompt:
```powershell
python scripts/ask_1c_rag.py "Какие реквизиты есть у справочника Номенклатура?" --index plugins/1c/datasets/prepared/rag_index.example.json --print-prompt
```
С явным профилем:
```powershell
python scripts/ask_1c_rag.py "Проверь BSL-код процедуры ПередЗаписью" --profile bsl --print-prompt
```
Проверка обязательных правил в prompt:
```powershell
python scripts/check_1c_rag_prompt.py
```
Проверка качества поиска по smoke-набору:
```powershell
python scripts/check_1c_rag_quality.py --print
```
Проверка auto-routing профилей:
```powershell
python scripts/check_1c_rag_profiles.py --print
```
С запущенной моделью:
```powershell
python scripts/ask_1c_rag.py "Какие реквизиты есть у справочника Номенклатура?" --index plugins/1c/datasets/prepared/rag_index.example.json --base-url http://docker-gpu.cin.su:8000 --model qwen3-4b-instruct
```
## Tool Contract
Контракт инструментов:
```text
plugins/1c/tools/tool-contract.yaml
```
Главное правило: модель не должна придумывать структуру базы 1С. Если ответ зависит от метаданных, сначала нужен вызов инструмента.
## Metadata Snapshot Flow
Для проверки RAG на структуре 1С можно сгенерировать source из metadata snapshot:
```powershell
python scripts/validate_1c_metadata_snapshot.py plugins/1c/metadata/examples/metadata.example.json
python scripts/convert_1c_metadata_to_rag.py --input plugins/1c/metadata/examples/metadata.example.json --output plugins/1c/rag/sources/metadata.example.generated.md
python scripts/build_1c_knowledge_base.py --include-example-bsl
python scripts/ask_1c_rag.py "Какие реквизиты есть у справочника Номенклатура?" --print-prompt
```
`build_1c_knowledge_base.py` выполняет валидацию snapshot, конвертацию источников, сборку corpus и индекса за один проход.
Перед сборкой corpus он также запускает `validate_1c_rag_sources.py`, чтобы заблокировать неподдерживаемые расширения, пустые файлы и вероятные секреты.
Manifest содержит source path, source type, file type, content hash и число чанков. Его удобно использовать для аудита и будущего инкрементального обновления индекса.
Проверить, не устарел ли manifest относительно `plugins/1c/rag/sources`:
```powershell
python scripts/check_1c_rag_freshness.py --print
```
Статус `stale` означает, что появились новые файлы, изменился hash, был удален источник или изменился `source_type`. После этого нужно пересобрать knowledge base.
Проверить, не устарел ли vector index относительно corpus:
```powershell
python scripts/check_1c_rag_vector_freshness.py --print
```
## Semantic Cache Embedding Worker
Для live 1C-адаптера semantic cache заполняется отдельно от RAG corpus. Адаптер отдает pending документы с `document_id` и `content_sha1`; worker считает embedding и вызывает `semantic.cache.embedding.upsert`. Если документ изменился, адаптер отвергнет embedding по `content_sha1`.
Dry-run:
```powershell
python scripts/embed_1c_semantic_cache.py --base-id upo_test --kind Template --limit 20 --dry-run --json
```
Запись baseline-векторов:
```powershell
python scripts/embed_1c_semantic_cache.py --base-id upo_test --kind Template --limit 20
```
Поиск по semantic cache с автоматически рассчитанным query embedding:
```powershell
python scripts/search_1c_semantic_cache.py "ОбластьШапка" --base-id upo_test --kind Template --limit 5 --validate-candidates
```
Если нужно перед поиском автоматически дозаполнить pending embeddings:
```powershell
python scripts/search_1c_semantic_cache.py "ОбластьШапка" --base-id upo_test --kind Template --embed-pending --validate-candidates
```
С OpenAI-compatible embedding endpoint:
```powershell
$env:EMBEDDING_API_KEY = "<token-if-needed>"
python scripts/embed_1c_semantic_cache.py `
--base-id upo_test `
--kind Template `
--embedding-provider openai-compatible `
--embedding-model "<embedding-model-name>" `
--embedding-base-url "http://docker-gpu.cin.su:8000" `
--embedding-api-key-env EMBEDDING_API_KEY
```
Semantic cache остается candidate-only: перед программными изменениями использовать `validate_candidates=true` или live read-selector из результата.
+47
View File
@@ -0,0 +1,47 @@
# 1C Training Data
Цель: подготовить безопасный датасет для будущего LoRA/adapter fine-tuning по 1С.
## Principles
- Сначала RAG и инструменты, потом дообучение.
- Дообучение выполняется только на проверенных примерах.
- Запрещено добавлять пароли, токены, строки подключения, персональные данные и клиентские секреты.
- Нельзя обучать модель на сырых выгрузках баз 1С.
- Каждый пример должен пройти экспертное ревью.
## Files
- Example dataset: `plugins/1c/training/examples/instruction.examples.jsonl`
- Dataset manifest: `plugins/1c/training/manifests/dataset.yaml`
- Adapter manifest: `plugins/1c/adapters/qwen3-coder-30b-a3b-1c-lora-v1.yaml`
- Registry card: `registry/model-cards/qwen3-coder-30b-a3b-1c-lora-v1.yaml`
## Validate Example Data
```powershell
python scripts/validate_1c_training_data.py plugins/1c/training/examples/instruction.examples.jsonl
```
## Prepare Chat JSONL
```powershell
python scripts/prepare_1c_training_data.py --input plugins/1c/training/examples/instruction.examples.jsonl --output plugins/1c/training/prepared/train.chat.jsonl
```
## Update Registry
```powershell
python scripts/validate_model_cards.py
python scripts/build_model_index.py
```
## Promotion Rule
Adapter status stays `draft` until:
- enough examples are collected;
- secret scan passes;
- expert review is complete;
- `plugins/1c/evals/smoke.yaml` passes on base, RAG, and adapter;
- rollback path is documented.
+781
View File
@@ -0,0 +1,781 @@
# adapter-1c-mcp
Thin MCP proxy for the 1C REST adapter.
Target host:
```text
docker.cin.su
```
Runtime URL:
```text
http://docker.cin.su:8021
```
MCP endpoint for Codex:
```text
http://docker.cin.su:8021/mcp
```
The server responds with MCP protocol `2025-06-18` and returns
`Mcp-Session-Id` during `initialize`.
Legacy SSE endpoint:
```text
http://docker.cin.su:8021/sse
```
Direct JSON-RPC endpoint for smoke tests:
```text
http://docker.cin.su:8021/mcp
```
Codex config:
```toml
[mcp_servers.adapter-1c-mcp]
enabled = true
url = "http://docker.cin.su:8021/mcp"
```
Already running Codex Desktop windows can keep the MCP tool list from their
startup time. Restart Codex Desktop, or start a fresh session, after changing
MCP server configuration.
## How The Adapter URL Is Passed
The MCP proxy does not hard-code the 1C adapter address. Pass it with:
```text
ONEC_ADAPTER_URL=http://docker-gpu.cin.su:8011
```
Optional adapter bearer token:
```text
ONEC_ADAPTER_TOKEN=
```
Timeout for adapter HTTP calls:
```text
ONEC_ADAPTER_TIMEOUT_SECONDS=4
```
These variables are configured in:
```text
core/deploy/docker/adapter-1c-mcp/.env.example
```
For real deployment, create a non-committed `.env` next to the compose file and set the actual adapter URL/token there.
## Deploy
Deploy both REST adapter and MCP proxy, then run live verification when a test
base is available:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass `
-File scripts\deploy_1c_adapter_stack.ps1 `
-BaseId <base-id-from-project-context>
```
Multiple test bases can be verified in one deploy:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass `
-File scripts\deploy_1c_adapter_stack.ps1 `
-BaseId <base-id-1>,<base-id-2>
```
Duplicate base ids are rejected before verification starts, so persisted
reports are not overwritten by an accidental repeated base value.
To make post-deploy checks deterministic, pass an explicit metadata object
selector. This is optional; without it the smoke discovers a module-capable
object automatically.
```powershell
powershell -NoProfile -ExecutionPolicy Bypass `
-File scripts\deploy_1c_adapter_stack.ps1 `
-BaseId <base-id-from-project-context> `
-ObjectGuid <metadata-object-guid> `
-ObjectKind <metadata-kind>
```
Saved-state preparation defaults to `ConfigSave` for the base configuration
save layer. Use `-SavedStateTable ConfigCASSave` when the strict smoke should
target an extension/CAS save layer instead:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass `
-File scripts\deploy_1c_adapter_stack.ps1 `
-BaseId <base-id-from-project-context> `
-SavedStateTable ConfigCASSave
```
The deploy script prints Docker container summaries for both services, then the
verify script prints `/health` status and `contract_version` before running the
REST and MCP selector-chain smoke tests. The selector-chain smoke also checks,
when a routine name is available from module metadata, that
`metadata.resolve_overrides` returns `write_plan_evidence.next_resolution` for
`metadata.saved_state.modules.search` with the same `base_id` and routine query,
then follows that resolver. If a saved-state stream exposes `write_plan_target`,
the smoke composes it with `write_plan_evidence.target` and checks the resulting
read-only `metadata.write.plan`; when the test base has no saved-state stream,
that last composition step records `skipped_no_saved_state_target`.
These selector-chain reports are written under
`reports/1c-sql/<base-id>/selector-chain-rest-smoke.json` and
`reports/1c-sql/<base-id>/selector-chain-mcp-smoke.json`. Each report includes
a top-level `coverage` block summarizing whether override evidence was found,
whether the saved-state resolver ran, whether a `write_plan_target` was
available, and which steps were skipped. The verify script validates that both
selector-chain report files are written, parse as JSON, pass, and include the
expected `coverage` sections.
Use `-RequireSelectorChainWritePlanComposition` when the test base is expected
to contain a matching saved-state stream and the selector-chain smoke must fail
instead of accepting `skipped_no_saved_state_target`.
Verification also runs the read-only write-plan safety smoke through both REST
and MCP `onec_request`; the MCP smoke also checks that live methods without
`payload.base_id` are stopped by `adapter_1c_mcp_policy.v1` with
`base_id_required`, and that low-level `storage.*`/`query.*` fallback calls are
blocked as `diagnostic_method` unless diagnostics are explicitly requested. The
verify script validates the written write-plan safety reports after each smoke.
Then verification runs REST saved-state form/module write-loop smoke tests
through `metadata.write`; if the test base has no pending
`ConfigSave`/`ConfigCASSave` rows, those saved-state checks record
`skipped_no_saved_state` and pass by default. The saved-state report files are
also parsed and checked after the smoke commands. Before those smoke tests,
verification generates read-only `saved-state-strict-readiness.json` and
`saved-state-copy-plan.json` reports for the selected `-SavedStateTable`. The
copy plan is generated through `scripts/plan_1c_saved_state_copy.py` using the
same object selector arguments, so the later persisted-report check has fresh
copy-preparation evidence. At the end, verification runs
`scripts/check_1c_verify_reports.py` against the persisted report set, so the
same report validation can be repeated offline without calling the adapter.
The offline validator checks that selector-chain coverage includes
`metadata.resolve_overrides` evidence, the expected
`metadata.saved_state.modules.search` next step, saved-state resolution, and
attempted `metadata.write.plan` composition. It also verifies write-plan safety
outcomes: effective form writes must route to `metadata.write.plan`,
uncontrolled replacement must be blocked, controlled replacement must be
planned through `metadata.module.write_apply`, drift must be blocked, and MCP
policy checks must block missing `base_id` and diagnostic fallback.
Saved-state write smoke reports are checked in both modes: empty-state skips
must include successful preflight counts and `skipped=true`; real
write-and-rollback runs must include allowed write plans, expected apply
methods, and rollback evidence. The same persisted report validator also checks
`saved-state-strict-readiness.json` and `saved-state-copy-plan.json` when
saved-state write smoke reports are enabled. Readiness must match the requested
`base_id` and selected save-layer table; in strict mode it must be `ready=true`.
The copy plan must match the requested `base_id`, resolve a concrete object,
list active source rows from the matching storage family, be `plan_ready`, and
show clear target collision status for `ConfigSave`/`ConfigCASSave`. The family
must match the target save layer: `Config -> ConfigSave` for base changes and
`ConfigCAS -> ConfigCASSave` for extension/CAS changes. The validator also
checks that readiness, copy-plan target table, saved-state form smoke table, and
saved-state module smoke `module_ref` table are the same.
`scripts/check_powershell_scripts.py` also guards the verify/deploy wiring for
these strict flags and asserts that `check_1c_verify_reports.py --self-test`
covers selector-chain, safety, saved-state, and copy-plan failure classes.
Persisted report validation also pins the expected JSON `schema` value for each
report type, so stale or unrelated report files fail before their contents are
trusted.
The same validator checks report identity: each persisted JSON must match the
requested `base_id`, and REST/MCP reports must match their expected
`transport`. When verification provides expected endpoints, REST/MCP reports
must also match the configured `endpoint_url`.
To require an actual saved-state write-and-rollback instead of allowing the
empty-state skip:
```powershell
python scripts\check_1c_saved_state_strict_readiness.py `
--base-id <base-id-from-project-context> `
--saved-state-table ConfigSave `
--report reports\1c-sql\<base-id>\saved-state-strict-readiness.json `
--json
```
This readiness check is read-only. It checks REST adapter health, row counts in
the selected save-layer table, and saved-state form/module discovery. Use
`--saved-state-table ConfigSave` for the base save layer or
`--saved-state-table ConfigCASSave` for the CAS/extension save layer. If it
reports `blocked_no_saved_state_rows`, there are no unactivated Configurator
changes in the selected save layer. To prepare a strict write test, copy the
target object from the main configuration storage (`Config`/`ConfigCAS`) into
the selected save layer through the approved saved-state workflow, then rerun
readiness before enabling strict mode.
To produce a read-only copy preparation plan for a concrete object:
```powershell
python scripts\plan_1c_saved_state_copy.py `
--base-id <base-id-from-project-context> `
--kind <metadata-kind> `
--name <metadata-object-name> `
--target-table ConfigSave `
--report reports\1c-sql\<base-id>\saved-state-copy-plan.json `
--json
```
The copy plan does not write SQL. It lists active storage rows for the selected
object only from the matching source family, checks whether the intended
`ConfigSave`/`ConfigCASSave` target already contains the same `FileName` values,
and records the intended save-layer target. `--target-table ConfigSave` plans
from `Config`; `--target-table ConfigCASSave` plans from `ConfigCAS`. Normal
verification regenerates this file automatically before saved-state smoke tests
and uses `-SavedStateTable` to pick the same target table for the copy plan,
form smoke, and module smoke. Run the command manually when choosing a target
object for strict readiness or when preparing evidence without the full
deploy/verify flow. The offline persisted-report validator checks this file by
default, so regenerate the plan whenever the chosen object, target table, or
base changes.
To generate the reviewed SQL preparation script from that plan without
executing SQL:
```powershell
python scripts\prepare_1c_saved_state_copy_sql.py `
--plan reports\1c-sql\<base-id>\saved-state-copy-plan.json `
--expected-base-id <base-id-from-project-context> `
--expected-target-table ConfigSave `
--sql-out reports\1c-sql\<base-id>\prepare-saved-state-copy.sql `
--report reports\1c-sql\<base-id>\prepare-saved-state-copy-sql.json `
--json
```
This generator does not call the adapter and does not write SQL. It refuses
plans that are not `plan_ready`, have target collisions, or mix storage
families. The generated SQL is a manual review artifact: it starts a
transaction, rechecks that the save-layer target has no planned `FileName`
values, copies only the planned rows with `INSERT ... SELECT`, verifies the
copied row count, and commits only if those guards pass.
To execute the reviewed preparation SQL, use the explicit SQL-write gate. This
is not run by deploy/verify:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass `
-File scripts\execute_1c_saved_state_copy_sql.ps1 `
-Server <sql-server> `
-Database <sql-database> `
-User <sql-user> `
-Password <sql-password> `
-ExpectedBaseId <base-id-from-project-context> `
-ExpectedTargetTable ConfigSave `
-IUnderstandThisWritesToSql
```
The executor validates the generated SQL report, checks that the SQL file still
looks like the reviewed saved-state copy artifact, records the SQL SHA1, runs
the SQL through `System.Data.SqlClient`, and immediately calls the read-only
post-copy verifier. Credentials can also be supplied with `ONEC_SQL_SERVER`,
`ONEC_SQL_DATABASE`, `ONEC_SQL_USER`, and `ONEC_SQL_PASSWORD`. Do not commit
credentials or execution reports containing environment-specific connection
details.
After the SQL preparation has been executed, verify the saved-state copy
read-only before enabling strict write smoke:
```powershell
python scripts\verify_1c_saved_state_copy.py `
--plan reports\1c-sql\<base-id>\saved-state-copy-plan.json `
--expected-base-id <base-id-from-project-context> `
--expected-target-table ConfigSave `
--report reports\1c-sql\<base-id>\saved-state-copy-verify.json `
--require-ready `
--json
```
Before the SQL preparation is executed, this verifier is expected to report
`blocked_missing_target_rows`. After preparation it must report `ready=true` by
comparing the save-layer `FileName`, `PartNo`, byte sizes, and `BinarySHA1`
values against the reviewed active-source rows from the copy plan.
Also generate the guarded cleanup script before executing the preparation SQL,
so there is a reviewed way to remove the prepared working-copy rows later:
```powershell
python scripts\prepare_1c_saved_state_cleanup_sql.py `
--plan reports\1c-sql\<base-id>\saved-state-copy-plan.json `
--expected-base-id <base-id-from-project-context> `
--expected-target-table ConfigSave `
--sql-out reports\1c-sql\<base-id>\cleanup-saved-state-copy.sql `
--report reports\1c-sql\<base-id>\cleanup-saved-state-copy-sql.json `
--json
```
The cleanup SQL deletes only the planned `FileName`/`PartNo` rows from the
selected save layer, and only when their current `BinarySHA1` still matches the
reviewed copy plan. It rolls back if the rows are missing, extra/mismatched, or
if the deleted row count differs from the plan.
```powershell
powershell -NoProfile -ExecutionPolicy Bypass `
-File scripts\deploy_1c_adapter_stack.ps1 `
-BaseId <base-id-from-project-context> `
-RequireSavedStateWriteSmoke
```
To require selector-chain evidence to resolve all the way to a concrete
read-only `metadata.write.plan` composition:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass `
-File scripts\deploy_1c_adapter_stack.ps1 `
-BaseId <base-id-from-project-context> `
-RequireSelectorChainWritePlanComposition
```
To skip saved-state write-loop checks entirely:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass `
-File scripts\deploy_1c_adapter_stack.ps1 `
-BaseId <base-id-from-project-context> `
-SkipSavedStateWriteSmoke
```
To skip only the read-only write-plan safety smoke:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass `
-File scripts\deploy_1c_adapter_stack.ps1 `
-BaseId <base-id-from-project-context> `
-SkipWritePlanSafetySmoke
```
To validate the latest persisted reports without contacting REST or MCP:
```powershell
python scripts\check_1c_verify_reports.py --base-id <base-id-from-project-context>
```
Add `--rest-adapter-url <url> --mcp-url <url>` when the persisted REST/MCP
reports should be pinned to specific endpoints.
Add `--saved-state-table ConfigSave` or `--saved-state-table ConfigCASSave`
when the persisted saved-state reports must be pinned to a specific save-layer
table.
Add `--max-report-age-seconds <seconds>` when the persisted reports must also
be fresh enough for a deployment gate.
Use `--skip-saved-state-copy-plan` only when intentionally validating an older
report bundle that does not include `saved-state-copy-plan.json`.
To run the offline validator's synthetic soft/strict self-test:
```powershell
python scripts\check_1c_verify_reports.py --self-test
```
To run the whole offline/static verification-stack preflight:
```powershell
python scripts\check_1c_adapter_verification_stack.py --base-id <base-id-from-project-context>
```
The verification-stack preflight passes the default REST/MCP endpoint URLs to
the persisted report validator. Override them with `--rest-adapter-url` and
`--mcp-url` for non-default deployments.
It also passes `--saved-state-table`, defaulting to `ConfigSave`, so
persisted copy-plan/form/module reports must match the intended save-layer
table.
Pass multiple base ids after `--base-id` to validate several persisted report
directories in one run.
Use `--max-report-age-seconds` to reject stale report files during the
verification-stack preflight.
To save a machine-readable verification-stack summary:
```powershell
python scripts\check_1c_adapter_verification_stack.py `
--base-id <base-id-from-project-context> `
--json `
--report reports\1c-sql\<base-id>\adapter-verification-stack-check.json
```
The saved stack report keeps per-command return codes, compact parsed summaries
for JSON-producing checks, and truncated stdout/stderr tails for diagnostics.
For persisted verify-report checks, the parsed summary also includes per-base
REST/MCP selector-chain composition status, write-plan safety check counts, and
saved-state write smoke status, including the saved-state copy-plan target,
source-row count, and target collision status.
To deploy without live verification:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass `
-File scripts\deploy_1c_adapter_stack.ps1 `
-SkipVerify
```
Low-level MCP-only deploy:
```powershell
docker --host ssh://docker.cin.su compose `
--env-file core\deploy\docker\adapter-1c-mcp\.env.example `
-f core\deploy\docker\adapter-1c-mcp\compose.yaml `
up -d --build
```
Container name:
```text
adapter-1c-mcp
```
## Health
```powershell
Invoke-RestMethod http://docker.cin.su:8021/health
docker --host ssh://docker.cin.su ps --filter name=adapter-1c-mcp
docker --host ssh://docker.cin.su logs --tail 80 adapter-1c-mcp
```
Smoke test MCP initialize:
```powershell
$body = @{
jsonrpc = "2.0"
id = 1
method = "initialize"
params = @{
protocolVersion = "2025-06-18"
capabilities = @{}
clientInfo = @{ name = "smoke"; version = "1" }
}
} | ConvertTo-Json -Depth 10
Invoke-WebRequest `
-Uri "http://docker.cin.su:8021/mcp" `
-Method Post `
-ContentType "application/json" `
-Headers @{ Accept = "application/json, text/event-stream" } `
-Body $body
```
## Tools
The MCP proxy exposes a small stable tool surface:
```text
onec_health
onec_help
onec_request
onec_job_get
onec_job_cancel
access_role_users
access_role_profiles
access_role_audit_export
access_role_audit_analyze
access_user_explain
access_users_search
access_object_explain
access_keys_query
access_object_keys_resolve
access_object_roles
access_object_subjects
access_rls_discover
```
Use `onec_request(method, payload)` for uncommon or newly added adapter methods
until a dedicated alias is useful enough to keep stable.
The `access_*` tools are lightweight aliases for common access-audit workflows;
they only forward to REST adapter methods such as `access.role.users` and keep
all BSP/access-rights logic in the REST adapter.
For object access checks, pass object names/public refs first; the REST adapter
resolves GUIDs, SQL numbers, BSP identifiers, synonyms, and readable role-rights
names internally. Example:
```powershell
python scripts\smoke_1c_access_object.py `
--base-id upo_test `
--ref "РегистрСведений.УОП_АктуальныеСпецификации" `
--action write
```
Use `access_object_roles` or `access_object_subjects` for the question "which
roles/users can read or write this metadata object". Do not depend on
`metadata.objects.list kind=Role`: some live bases do not expose roles as regular
metadata objects through that route.
For BSP role-right checks, `action=write` means add or modify rights. Read-only
roles must not be returned for `write`; query `action=read` separately when you
need visibility roles. Returned permissions keep `source_fields` from
`ПраваРолей` so suspicious mappings can be verified against the live base.
`access.snapshot.extract` also supports this access graph and defaults to the
BSP extractor preset when `queries` are not supplied. Passing `preset=bsp`
explicitly is still fine, but the old "No extractor queries were provided"
discovery response should not appear for a normal BSP base.
When `access_object_subjects` is called with `include_access_key_scope=true`,
the regular `limit` only trims returned roles/profiles/groups/users. Scope
diagnostics use separate controls: `access_key_scope_subject_limit` limits how
many matched groups/users are checked, and `access_key_scope_limit` limits rows
read from each BSP access-key extractor. Check `access_key_scope.coverage` and
`access_key_scope.truncated` before treating the scope sample as complete.
For local audit artifacts, use the workspace script:
```powershell
python scripts\export_1c_access_role_audit.py `
--base-id upo_test `
--role "запись изменение номенклатура поставщиков" `
--user-threshold 30
```
It writes CSV, JSON export, analysis JSON, and a summary file under
`reports/1c-access/<base_id>/`. By default it also writes a self-contained
HTML report with findings, counters, artifact links, and a filterable user table;
pass `--no-html` to skip it.
Architecture contract:
- The REST adapter owns 1C behavior, payload validation, decoding, owner
resolution, RAG/tool-facing data shape, and method documentation through
`help.methods`.
- MCP is a transport proxy: JSON-RPC/MCP framing, adapter URL/token handling,
generic `onec_request`, job polling/cancellation helpers, and lightweight
agent-safety checks such as requiring `base_id` for live database methods.
- Do not add method-specific 1C business logic to MCP. Put it in the adapter,
expose it through `/rpc`, and make it discoverable via `help.methods`.
- Do not register an MCP unified/composite handler with the same name as an
adapter method; `onec_request(method=...)` must call the adapter method, not
shadow it inside MCP.
- When adding a new adapter method, verify it through MCP with
`onec_request({"method": "...", "payload": {...}})`. MCP should not require a
schema edit for ordinary adapter growth.
- Run `python scripts/check_1c_mcp_adapter_contract.py` before release; it
verifies that `onec_request` stays generic and that every method listed by the
REST adapter is forwarded through `/rpc`.
Example:
```json
{
"method": "metadata.object.get",
"payload": {
"base_id": "<base_id-from-project-context>",
"kind": "document",
"name": "ПриходнаяНакладная",
"view": "merged"
}
}
```
Do not copy placeholder or sample `base_id` values from documentation into real
requests. For live database methods, get `base_id` from the current project,
user request, environment, or another authoritative context.
## Agent Request Policy
The MCP proxy blocks live database methods without `payload.base_id` before
calling the REST adapter. This is intentional: agents must not probe metadata,
modules, extensions, queries, storage, or code without a concrete target base.
Affected method families include:
```text
metadata.*
modules.*
code.*
templates.*
diagnostics.*
extensions.*
query.*
storage.*
schema.*
codec.*
```
Allowed without `base_id`:
```text
onec_help / help.methods
onec_health without a base for generic service health
adapter.job.get / adapter.job.cancel
```
Agent rules:
- If `base_id` is unknown, ask for it or obtain it from project context.
- For object-scoped calls, use one of the public selector shapes supported by
the adapter: `ref`, `kind` + `name`, `guid`, or MCP-friendly
`object_type`/`object_name`/`object_guid`. Do not add MCP-side conditions for
concrete object names.
- If a prior result contains `module_ref`, `module_id`, GUID, or read selector,
use direct read methods before global search.
- If a search result contains `read_selector.method`, call that method with the
selector payload. `code.search` selectors point to `code.read`; `modules.search`
selectors point to `modules.read`.
- If a result contains `related_selectors`, prefer those payloads for the next
object-scoped call. They preserve `kind`/`name`/`guid` and include `ref` when
the adapter knows the canonical object kind and name.
- When `modules.search` or `code.search` resolves a module owner, keep both the
public owner selector (`ref`, `kind`, `name`, `guid`) and the opaque
`module_ref` in the next read payload. `module_ref` is the direct read handle;
`ref` makes the owner clear to the agent and later calls.
- When narrowing `code.search` to one module, pass `module_ordinal` together
with the object selector. The adapter response should remain a public
`onec_code_search.v1` object with `item.read_selector`, not a low-level tuple
or storage result.
- For BSL edits, prefer high-level `metadata.write` with a 1C canonical path,
`routine_text`, and `routine_operation`. Do not ask the user for SQL/save
gates: the adapter prepares saved-state when needed, writes only to the
saved-state working layer, and does not activate changes. Use
`metadata.form.command_button.write` for the complete form command + visible
button + handler workflow. `code.write` remains a compatibility shortcut for
simple module edits.
- For unresolved module owners, inspect `diagnostics.owner_resolution` and
`counts.owner_scan_limit_hit`; narrow by `kind`/`name`/`guid` or increase
`owner_scan_limit` before falling back to broader searches.
- Do not treat `not_found` from `metadata.definition.find` or scoped search as
proof that code is absent in extensions or ConfigCAS.
- Treat `partial`, `truncated=true`, scan limits, and timeouts as incomplete
evidence.
Offline selector-chain smoke:
```powershell
python scripts\smoke_1c_mcp_selector_chain.py --json --no-report
```
Saved-state BSL write smoke:
```powershell
python scripts\smoke_1c_code_write_saved_state.py `
--adapter-url http://docker-gpu.cin.su:8011 `
--base-id <base-id-from-project-context> `
--extension <extension-name> `
--object-type CommonForm `
--object-name <form-name> `
--routine-name <routine-name> `
--json
```
This smoke performs idempotent `code.write mode=apply` checks with the current
routine text, a unique `old`/`new` fragment, and the current full
`module_text`. It then verifies `code.read state=working`,
`code.read state=both`, marker-hiding for saved form modules, and saved-state
form indexing. Every write step must report `write_mode.target=saved_state`
and `activation_state=not_activated`.
Agent working-view report:
```powershell
python scripts\report_1c_agent_working_view.py `
--adapter-url http://docker-gpu.cin.su:8011 `
--base-id <base-id-from-project-context> `
--extension <extension-name> `
--object-type CommonForm `
--object-name <form-name> `
--routine-name <routine-name> `
--json
```
This read-only report shows the form names that an agent sees in the
working/save layer and verifies that `code.read state=working` and
`state=both` prefer `saved_state` when saved code exists.
The concise agent-facing coding rules are kept in
`docs/runbooks/1c-agent-coding-contract.md`.
Run this with the contract checks before release. It validates generic agent
routes through `onec_request` and rejects concrete object names in selector
examples.
Optional live selector-chain smoke against a real adapter/base:
```powershell
python scripts\smoke_1c_mcp_selector_chain.py `
--live `
--transport rest `
--adapter-url http://docker-gpu.cin.su:8011 `
--base-id <base-id-from-project-context> `
--json `
--no-report
```
To verify both deployed layers with one command:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass `
-File scripts\verify_1c_adapter_deployment.ps1 `
-BaseId <base-id-from-project-context>
```
By default this also writes
`reports/1c-sql/<base>/code-write-saved-state-rest-smoke.json` and
`reports/1c-sql/<base>/code-write-saved-state-mcp-smoke.json` when the default
saved-state code-write target exists. The REST verification also writes
`reports/1c-sql/<base>/agent-working-view.json` to pin the save-first working
view that agents should use for coding. Use
`-RequireCodeWriteSavedStateSmoke` to make that smoke mandatory, or
`-SkipCodeWriteSavedStateSmoke` to skip it for bases where the scenario is not
available.
Pass multiple base ids to run the same REST and MCP checks against each base:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass `
-File scripts\verify_1c_adapter_deployment.ps1 `
-BaseId <base-id-1>,<base-id-2>
```
Use `-ObjectRef`, or `-ObjectKind` with `-ObjectName`/`-ObjectGuid`, to verify a
specific module-capable metadata object instead of auto-discovery.
Use `-SavedStateTable ConfigSave` or `-SavedStateTable ConfigCASSave` to choose
which save-layer table is used by the copy plan and saved-state write smokes.
Add `-RequireSelectorChainWritePlanComposition` when the selected base/object
must have a saved-state stream that lets the selector-chain smoke compose a
concrete read-only `metadata.write.plan`.
To exercise the MCP proxy itself, switch transport and URL:
```powershell
python scripts\smoke_1c_mcp_selector_chain.py `
--live `
--transport mcp `
--mcp-url http://docker.cin.su:8021 `
--base-id <base-id-from-project-context> `
--json `
--no-report
```
Live mode discovers a module-capable object from metadata lists and then checks
`adapter.help.methods -> metadata.definition.find -> related_selectors.modules
-> metadata.object.modules -> modules.read -> code.search ->
item.read_selector -> code.read` using the returned selector and a token derived
from module text or routine metadata. The `adapter.help.methods` step verifies
live REST `contract_version` and `selector_capabilities`, so stale REST adapter
images fail before the longer chain. MCP transport also performs `initialize`,
checks `tools/list` for the same `contract_version` and the generic
`onec_request` selector schema, keeps `Mcp-Session-Id`, and sends adapter calls
through `tools/call` + `onec_request`. This catches stale MCP proxy images where
REST works but agents still see old tool descriptions. Do not commit real base
ids or captured reports from live runs.
- Avoid increasing global search limits when a direct selector is already
available.
- Do not request `include_storage=true` merely to read a module found by search;
use the public `read_selector` first.
The request is proxied to the REST adapter through `POST /rpc`:
```json
{
"method": "metadata.object.get",
"payload": {}
}
```
The MCP proxy intentionally does not know adapter-specific 1C methods. Add new
methods in the REST adapter and expose them through `help.methods`; MCP stays
unchanged unless the MCP transport itself changes.
+61
View File
@@ -0,0 +1,61 @@
# Artifact Portability
Модели, RAG-корпусы, выгрузки ИТС, снапшоты метаданных и тренировочные наборы
не хранятся в git. При переносе проекта или запуске в Docker их нужно переносить
и монтировать отдельно.
## Build Manifest
```powershell
python scripts/build_llm_artifact_manifest.py --output reports/llm-artifact-manifest.json
```
По умолчанию manifest хранит размеры, количество файлов, расширения и sample
файлов. Хеширование больших моделей выключено, чтобы команда не была медленной.
Для более строгой проверки:
```powershell
python scripts/build_llm_artifact_manifest.py --hash-files --max-files 2000
```
## Check Manifest
На той же машине:
```powershell
python scripts/check_llm_artifact_manifest.py `
--manifest reports/llm-artifact-manifest.json `
--output reports/llm-artifact-check.json
```
После переноса в другой workspace:
```powershell
python scripts/check_llm_artifact_manifest.py `
--manifest reports/llm-artifact-manifest.json `
--target-root Z:/codex/LLM `
--output reports/llm-artifact-check.json
```
## Docker Volumes
Контейнеры не должны хранить эти данные во внутреннем слое. Монтируем внешние
папки:
```yaml
volumes:
- Z:/codex/LLM/models:/app/models
- Z:/codex/LLM/plugins/1c/datasets:/app/plugins/1c/datasets
- Z:/codex/LLM/plugins/1c/rag/sources:/app/plugins/1c/rag/sources
- Z:/codex/LLM/plugins/1c/rag/official-docs:/app/plugins/1c/rag/official-docs
- Z:/codex/LLM/plugins/1c/metadata/snapshots:/app/plugins/1c/metadata/snapshots
```
## Rule
Перед удалением контейнеров, переносом на другой host или пересборкой volumes:
1. Собрать manifest.
2. Скопировать artifact directories.
3. Проверить manifest на целевом пути.
4. Только потом запускать Docker services.
+43
View File
@@ -0,0 +1,43 @@
# Ask 1C RAG
Цель: проверить полный локальный сценарий 1С RAG:
```text
question -> local search -> context prompt -> optional vLLM answer
```
## Prepare Example Corpus
```powershell
python scripts/prepare_1c_rag_corpus.py --source-dir plugins/1c/rag/examples --output plugins/1c/datasets/prepared/rag_corpus.example.jsonl
```
## Build Example Index
```powershell
python scripts/build_1c_rag_index.py --corpus plugins/1c/datasets/prepared/rag_corpus.example.jsonl --output plugins/1c/datasets/prepared/rag_index.example.json
```
## Render Prompt Only
```powershell
python scripts/ask_1c_rag.py "Какие реквизиты есть у справочника Номенклатура?" --index plugins/1c/datasets/prepared/rag_index.example.json --print-prompt
```
Этот режим не требует запущенной модели. Он нужен для проверки, какой контекст будет передан LLM.
## Check Prompt Guardrails
```powershell
python scripts/check_1c_rag_prompt.py
```
## Ask Running Model
После запуска vLLM:
```powershell
python scripts/ask_1c_rag.py "Какие реквизиты есть у справочника Номенклатура?" --index plugins/1c/datasets/prepared/rag_index.example.json --base-url http://docker-gpu.cin.su:8000 --model qwen3-4b-instruct
```
Ожидаемое поведение: модель не должна выдумывать реквизиты. Она должна сказать, что для ответа нужны метаданные 1С.
+55
View File
@@ -0,0 +1,55 @@
# Deploy llama.cpp GGUF
Цель: поднять GGUF-модель `Devstral Small 2 24B Instruct 2512 Q4_K_M` через `llama-server` на `docker-gpu.cin.su`.
## Model
- Repo: `bartowski/mistralai_Devstral-Small-2-24B-Instruct-2512-GGUF`
- File: `mistralai_Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf`
- Size: `14334438272` bytes
- Registry card: `registry/model-cards/devstral-small-2-24b-instruct-2512-q4_k_m.yaml`
## Download
Resume-safe local download:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/download_devstral_gguf.ps1
```
The script can be re-run after network failures.
## Compose
```text
core/deploy/docker-gpu/llama-cpp/compose.yaml
core/deploy/docker-gpu/llama-cpp/.env.example
```
## Run
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_llama_cpp.ps1 -Pull
```
Проверить compose без запуска:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_llama_cpp.ps1 -ConfigOnly
```
Остановить сервис:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_llama_cpp.ps1 -Down
```
## Check
```powershell
python scripts/check_inference_endpoint.py --base-url http://docker-gpu.cin.su:8080 --expected-model devstral-1c-q4 --print
```
```powershell
python scripts/smoke_chat.py --base-url http://docker-gpu.cin.su:8080 --model devstral-1c-q4
```
+82
View File
@@ -0,0 +1,82 @@
# Deploy vLLM
Цель: поднять первый OpenAI-compatible inference API на `docker-gpu.cin.su`.
## 1. Check GPU Host
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/check_gpu_host.ps1
```
Если SSH ругается на host key, см. `docs/runbooks/gpu-host-preflight.md`.
## 2. Prepare Env
Для реального запуска лучше создать локальный `.env` рядом с compose-файлом:
```powershell
Copy-Item core/deploy/docker-gpu/vllm/.env.example core/deploy/docker-gpu/vllm/.env
```
Отредактировать:
- `VLLM_MODEL_ID`
- `VLLM_SERVED_MODEL_NAME`
- `HOST_MODELS_DIR`
- `HOST_HF_CACHE_DIR`
- `HF_TOKEN`, если модель закрытая
На Windows GPU-хосте используйте проверенный образ
`VLLM_IMAGE=vllm/vllm-openai:v0.10.2`. С драйвером CUDA 12.8 новые образы могут
не стартовать из-за требования CUDA 13. После обновления драйвера до CUDA 13.x
образ `latest` проходит CUDA-проверку, но vLLM 0.23.0 на Docker Desktop/WSL
падает при старте движка с `UVA is not available`.
Файл `.env` не коммитится.
## 3. Validate Compose
Без запуска:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_vllm.ps1 -ConfigOnly
```
## 4. Deploy
С `.env.example`:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_vllm.ps1 -Pull
```
С реальным `.env`:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_vllm.ps1 -EnvFile core/deploy/docker-gpu/vllm/.env -Pull
```
## 5. Check Endpoint
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/check_vllm_endpoint.ps1
```
Или напрямую:
```powershell
python scripts/check_inference_endpoint.py --base-url http://docker-gpu.cin.su:8000 --expected-model qwen3-4b-instruct --print
python scripts/smoke_chat.py --base-url http://docker-gpu.cin.su:8000 --model qwen3-4b-instruct
```
## 6. Stop
```powershell
docker --host ssh://test@docker-gpu.cin.su compose --env-file core/deploy/docker-gpu/vllm/.env -f core/deploy/docker-gpu/vllm/compose.yaml down
```
Если для GPU-хоста нужен явный пользователь, передайте его в параметре:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_vllm.ps1 -DockerHost ssh://USER@docker-gpu.cin.su -ConfigOnly
```
+81
View File
@@ -0,0 +1,81 @@
# Download Model
Цель: загрузить модель из Hugging Face в локальное хранилище, не помещая веса модели в git.
## Install Dependencies
```powershell
pip install -r requirements.txt
```
## Dry Run
```powershell
python scripts/download_hf_model.py qwen3-4b-instruct-2507 --dry-run
```
## Download
На хосте, где доступен путь `/models`:
```powershell
python scripts/download_hf_model.py qwen3-4b-instruct-2507
```
Если команда запускается на Windows, а в model card указан Linux-путь `/models/...`, реальная загрузка без `--local-dir` будет остановлена. Это защита от случайной загрузки в неправильный локальный каталог.
Для закрытой модели токен передается через окружение:
```powershell
$env:HF_TOKEN="..."
python scripts/download_hf_model.py qwen3-4b-instruct-2507
```
Не сохраняйте токен в репозитории, `.env` или model card.
## Resume Large Files
Если сеть рвет большие файлы, используйте range-загрузчик. Он докачивает файл с текущего размера и проверяет итоговый размер по Hugging Face metadata.
```powershell
python scripts/download_hf_range.py qwen3-4b-instruct-2507 `
--local-dir models/base/qwen3-4b-instruct-2507 `
--allow-file model-00001-of-00003.safetensors `
--allow-file model-00002-of-00003.safetensors `
--chunk-size 16mb `
--retries 20
```
Эту команду можно запускать повторно до полного завершения.
## Override Path
Если нужно скачать в другой каталог:
```powershell
python scripts/download_hf_model.py qwen3-4b-instruct-2507 --local-dir D:\models\base\qwen3-4b-instruct-2507
```
## Registry Index
После добавления или изменения карточек моделей:
```powershell
python scripts/build_model_index.py
```
## Check Local Storage
Проверить, какие модели полностью скачаны, частично скачаны или отсутствуют:
```powershell
python scripts/check_model_storage.py --print
```
В обычном `check_all` эта проверка работает как warning, потому что часть моделей и адаптеров может быть запланирована, но еще не загружена.
Для текущего состояния всей платформы:
```powershell
python scripts/collect_platform_status.py --print
```
+89
View File
@@ -0,0 +1,89 @@
# Evals
Цель: проверять качество моделей, RAG и будущих адаптеров воспроизводимым способом.
## Validate Eval Files
```powershell
python scripts/validate_evals.py
```
## Run 1C Smoke Eval Without Model
Prompt-only режим проверяет, что eval-набор читается и превращается в отчет.
```powershell
python scripts/run_1c_smoke_eval.py --print
```
Отчет пишется в:
```text
reports/evals/1c-smoke.report.json
```
Папка `reports` не коммитится.
## Run 1C Smoke Eval Against vLLM
После запуска inference:
```powershell
python scripts/run_1c_smoke_eval.py --base-url http://docker-gpu.cin.su:8000 --model qwen3-4b-instruct
```
Runner сохраняет ответы и считает автоматические проверки, если критерий задан структурно.
Строковые критерии остаются ручными.
Поддерживаемые автоматические типы:
- `contains`
- `not_contains`
- `regex`
- `max_sentences`
- `refuses`
- `requires_metadata`
Для GGUF/llama.cpp:
```powershell
python scripts/run_1c_smoke_eval.py --base-url http://docker-gpu.cin.su:8080 --model devstral-1c-q4
```
Перед evals удобно проверить endpoint:
```powershell
python scripts/check_inference_endpoint.py --base-url http://docker-gpu.cin.su:8000 --expected-model qwen3-4b-instruct --print
python scripts/check_inference_endpoint.py --base-url http://docker-gpu.cin.su:8080 --expected-model devstral-1c-q4 --print
```
## Run Live Evals For Available Models
Единая команда проверяет endpoint-ы активного GPU-профиля и запускает smoke eval только для моделей этого профиля:
```powershell
python scripts/run_live_model_evals.py --profile text --print
```
Отчет пишется в:
```text
reports/evals/live-model-evals.json
```
По умолчанию live eval использует продуктовый 1С system prompt:
```text
plugins/1c/prompts/system.md
```
Если endpoint недоступен, соответствующая модель получает статус `blocked`.
Для полной ручной проверки взаимоисключающих профилей используйте:
```powershell
python scripts/run_live_model_evals.py --all-targets --print
```
## Promotion Rule
Модель или адаптер нельзя переводить из `draft/staging` в `production`, пока smoke-evals не пройдены и отчет не просмотрен.
+72
View File
@@ -0,0 +1,72 @@
# First vLLM Inference
Цель: поднять первый OpenAI-compatible inference API на `docker-gpu.cin.su`.
Основной runbook запуска: `docs/runbooks/deploy-vllm.md`.
Preflight GPU-хоста: `docs/runbooks/gpu-host-preflight.md`.
## Files
- `core/deploy/docker-gpu/vllm/compose.yaml`
- `core/deploy/docker-gpu/vllm/.env.example`
- `registry/model-cards/qwen3-4b-instruct-2507.yaml`
## Prepare
На GPU-хосте:
```powershell
docker --context default version
```
С локальной машины, если настроен Docker context:
```powershell
docker --host ssh://test@docker-gpu.cin.su info
```
## Configure
Скопировать `.env.example` в `.env` на стороне deployment-каталога и указать:
- `VLLM_MODEL_ID`
- `VLLM_SERVED_MODEL_NAME`
- `HOST_MODELS_DIR`
- `HOST_HF_CACHE_DIR`
- `HF_TOKEN`, только если нужен доступ к закрытой модели
Не коммитить `.env`.
## Run
```powershell
docker compose --env-file .env -f core/deploy/docker-gpu/vllm/compose.yaml up -d
```
## Check
```powershell
curl http://docker-gpu.cin.su:8000/v1/models
```
Пример запроса:
```powershell
curl http://docker-gpu.cin.su:8000/v1/chat/completions `
-H "Content-Type: application/json" `
-d '{"model":"qwen3-4b-instruct","messages":[{"role":"user","content":"Привет. Ответь коротко."}]}'
```
Или smoke-скриптом из корня проекта:
```powershell
python scripts/smoke_chat.py --base-url http://docker-gpu.cin.su:8000 --model qwen3-4b-instruct
```
## Notes
- Сначала используем одну текстовую модель.
- Первая модель: `Qwen/Qwen3-4B-Instruct-2507`.
- Первый запуск ограничен `VLLM_MAX_MODEL_LEN=32768`, чтобы снизить риск OOM.
- Для 1С позже подключим RAG и LoRA/adapters отдельно.
- Большие модели и cache лежат вне git.
+118
View File
@@ -0,0 +1,118 @@
# GPU Host Preflight
Цель: проверить, что `docker-gpu.cin.su` готов к запуску GPU-контейнеров.
Если `docker-gpu.cin.su` указывает на Windows-хост, используйте также отдельный runbook:
```text
docs/runbooks/windows-docker-gpu-host.md
```
## SSH Host Key
Если SSH сообщает `Host key verification failed`, нужно вручную проверить fingerprint хоста и добавить ключ в `known_hosts`.
Команда для просмотра ключа:
```powershell
ssh-keyscan docker-gpu.cin.su
```
После проверки fingerprint можно добавить ключ:
```powershell
ssh-keyscan docker-gpu.cin.su >> $env:USERPROFILE\.ssh\known_hosts
```
Не добавляйте ключ вслепую, если есть риск подмены DNS или хоста.
## Preflight
Из корня проекта:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/check_gpu_host.ps1
```
Если нужен явный пользователь SSH:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/check_gpu_host.ps1 -SshTarget USER@docker-gpu.cin.su
```
Скрипт проверяет:
- SSH-доступ;
- наличие GPU через `nvidia-smi`;
- Docker Engine;
- Docker Compose;
- запуск тестового CUDA-контейнера с `--gpus all`.
## Readiness Report
Чтобы одной командой проверить SSH preflight и endpoints активного GPU-профиля:
```powershell
python scripts/check_gpu_readiness.py --profile text --print
```
Отчет пишется в:
```text
reports/gpu-readiness.json
```
По умолчанию проверяется профиль `text` из `config/gpu_profiles.json`:
- SSH/GPU/Docker preflight через `scripts/check_gpu_host.ps1`.
- health/model endpoints из поля `wait` выбранного профиля.
Полная проверка взаимоисключающих OpenAI-compatible endpoints доступна отдельно:
```powershell
python scripts/check_gpu_readiness.py --all-endpoints --print
```
Если SSH возвращает `Permission denied`, нужно настроить SSH-ключ или явно указать пользователя:
```powershell
python scripts/check_gpu_readiness.py --ssh-target USER@docker-gpu.cin.su --print
```
## Full GPU Stack Flow
После настройки SSH-доступа можно запустить весь контур:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_gpu_stack.ps1 -Pull
```
Сценарий выполняет:
- GPU host preflight;
- deploy vLLM;
- deploy llama.cpp GGUF;
- ожидание `/v1/models` для обоих endpoint-ов;
- запуск live evals.
Посмотреть план без запуска:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_gpu_stack.ps1 -PlanOnly -Pull
```
Проверить compose-файлы без SSH preflight:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_gpu_stack.ps1 -ConfigOnly -SkipPreflight
```
## Expected Result
В выводе должны быть:
- имя хоста;
- модель GPU и объем VRAM;
- версия Docker;
- версия Docker Compose;
- результат `nvidia-smi` внутри контейнера.
+50
View File
@@ -0,0 +1,50 @@
# Management Console
Локальная web-консоль для управления рабочим контуром LLM/1C.
## Start
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/run_management_console.ps1
```
Default URL:
```text
http://127.0.0.1:8770/
```
## Current Capabilities
- overview of local model/RAG artifacts;
- model storage report view;
- 1C RAG and official 1C:ITS artifact status;
- 1C plugin health summary;
- whitelisted checks and maintenance actions;
- 1C:ITS cookie dialog launch;
- 1C:ITS fetch progress from `raw/progress.json`;
- job log with stdout/stderr tail.
The console intentionally does not expose arbitrary shell execution. Every
action must be present in the server whitelist in
`scripts/management_console_server.py`.
## Current Commands
- build artifact manifest;
- check artifact manifest;
- check model storage;
- check 1C plugin;
- check 1C RAG freshness;
- check official docs private artifacts;
- validate PowerShell scripts;
- open 1C:ITS cookie dialog.
## Next Expansion
- Docker service status and start/stop/restart for approved services;
- SQL connector status for known 1C bases;
- RAG rebuild with selected source groups;
- model download/ingest queue;
- vector index status when hybrid search is added;
- role-based action policy before any write-capable 1C operation.
+71
View File
@@ -0,0 +1,71 @@
# Model Chat Testbench
Цель: вручную проверять модели из `registry/index.json` через чат с выбором плагина и модели.
## Web UI
```powershell
python scripts/model_chat_server.py
```
Открыть:
```text
http://127.0.0.1:8765
```
Интерфейс читает модели из `registry/index.json`, группирует их по плагинам и отправляет запросы в OpenAI-compatible endpoint через локальный proxy.
Возможности:
- выбор endpoint preset: `vLLM text`, `llama.cpp GGUF`, `local`;
- проверка `/v1/models` и наличия выбранного `served_model_name`;
- выбор плагина и модели из registry;
- готовые тестовые prompt-пакеты по каждому плагину;
- сравнение нескольких моделей на одном prompt;
- сборка 1C RAG prompt с найденными источниками;
- сохранение каждого ответа и ошибки в JSONL;
- ручная оценка ответа: `ok`, `needs_review`, `bad`.
Endpoint по умолчанию:
```text
http://docker-gpu.cin.su:8000
```
## CLI Smoke Chat
```powershell
python scripts/model_chat_cli.py --plugin text
python scripts/model_chat_cli.py --plugin 1c --model-id qwen3-4b-instruct-2507
python scripts/model_chat_cli.py --plugin translation --prompt "Переведи на английский: Проверяем модель."
```
## Reports
Результаты сохраняются в:
```text
reports/model-chat/YYYYMMDD.jsonl
```
Записи `type=chat` содержат prompt, ответ, модель, endpoint, latency и статус. Записи `type=feedback` содержат ручную оценку и ссылку на `parent_id`.
Записи `type=compare` содержат общий prompt и ответы нескольких моделей. Записи `type=rag_prompt` содержат вопрос, количество найденных источников и список source chunks.
## 1C RAG Prompt
Для кнопки `1C RAG` нужен подготовленный индекс:
```powershell
python scripts/prepare_1c_rag_corpus.py
python scripts/build_1c_rag_index.py
```
Если индекс отсутствует, UI покажет команду подготовки.
## Notes
- Для vLLM используется `served_model_name` из model card.
- Для GGUF/llama.cpp укажите endpoint соответствующего сервиса, например `--base-url http://docker-gpu.cin.su:8080`.
- Если GPU endpoint недоступен, UI покажет ошибку запроса, но каталог моделей все равно загрузится.
+388
View File
@@ -0,0 +1,388 @@
# Model Chat UI Service
Цель: поднять личный кабинет проверки моделей в локальной сети.
LAN URL:
```text
http://192.168.220.91:8765/tools/model-chat/
```
## Deploy
Preflight before deployment:
```powershell
powershell -ExecutionPolicy Bypass -File scripts\preflight_model_chat_ui.ps1 `
-DockerHost ssh://docker-gpu `
-SshTarget docker-gpu `
-CheckLive
```
Fast local-only preflight:
```powershell
powershell -ExecutionPolicy Bypass -File scripts\preflight_model_chat_ui.ps1 -SkipDockerConfig
```
```powershell
powershell -ExecutionPolicy Bypass -File scripts/deploy_model_chat_ui.ps1 `
-DockerHost ssh://docker-gpu `
-SshTarget docker-gpu
```
Скрипт:
- собирает небольшой архив приложения без тяжелых `models/`;
- копирует его на Windows GPU-хост;
- распаковывает в `Z:\LLM\model-chat-app`;
- запускает compose-сервис `llm-model-chat-ui`;
- открывает Windows Firewall для TCP `8765`.
## Health
```powershell
Invoke-RestMethod http://192.168.220.91:8765/api/health
Invoke-RestMethod http://192.168.220.91:8765/api/catalog
Invoke-RestMethod http://192.168.220.91:8765/api/model-services
python scripts\plan_model_services.py
python scripts\generate_model_chat_status.py
```
The status generator writes a compact Markdown snapshot to:
```text
reports/model-chat/status.md
```
`/api/health` includes:
- endpoint status for vLLM, llama.cpp, translation, audio, and video service ports;
- selected route for each plugin;
- GPU profile readiness with missing services to start and conflicting services to stop;
- latest Model Chat UI preflight status from `reports/model-chat/preflight.json`;
- local storage status for every registered model;
- service plan with compose/deploy hints.
Service states:
- `online`: endpoint is running and reports the expected served model name;
- `ready_to_start`: model files are present, but the service is not online;
- `blocked`: local model files are missing or incomplete.
## Service Control
The UI has a `service` panel for the selected model. It calls `POST /api/service-control`:
```json
{
"model_id": "whisper-large-v3-turbo",
"action": "status"
}
```
Allowed actions are `start`, `stop`, `restart`, and `status`.
If the UI container has no Docker CLI or SSH key, use the operator script from the project folder:
```powershell
powershell -ExecutionPolicy Bypass -File scripts\manage_model_service.ps1 -Service audio-api -Action status
powershell -ExecutionPolicy Bypass -File scripts\manage_model_service.ps1 -Service audio-api -Action stop
powershell -ExecutionPolicy Bypass -File scripts\manage_model_service.ps1 -Service translation-api -Action start
```
Use `stop` on heavy services before starting another large model when VRAM is low.
## GPU Profiles
Use GPU profiles instead of manual container juggling when switching between heavy models:
```powershell
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile default
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile text
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile audio
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile video
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile image
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile gguf-1c
```
Profile definitions live in:
```text
config/gpu_profiles.json
```
The UI catalog, `/api/health` profile readiness, and `scripts\switch_gpu_profile.ps1` read the same file.
Validate it before deploy:
```powershell
python scripts\validate_gpu_profiles.py --print
```
Validate the UI deployment archive without uploading it:
```powershell
powershell -ExecutionPolicy Bypass -File scripts\deploy_model_chat_ui.ps1 -ArchiveOnly
```
Dry-run the plan without touching containers:
```powershell
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile video -PlanOnly
```
Profiles:
- `default` / `text`: vLLM text, translation, audio, and UI; video, image, and llama.cpp stopped.
- `audio`: audio, translation, vLLM text, and UI; video, image, and llama.cpp stopped.
- `video`: translation, video, and UI; vLLM text, audio, image, and llama.cpp stopped to free VRAM.
- `image`: translation, image, and UI; vLLM text, audio, video, and llama.cpp stopped to free VRAM.
- `gguf-1c`: llama.cpp, translation, and UI; vLLM text, audio, video, and image stopped.
The Model Chat UI also shows the recommended profile command for the selected plugin.
The profile hint shows `status: ready` when the current containers already match the profile.
Otherwise it lists services that should be started and services that should be stopped.
For `video` and `image`, the service starts quickly but the first request can still spend several minutes loading
the model into GPU memory.
## Runtime Profiles
Runtime profiles choose the execution host and endpoint used by the chat UI:
```text
config/runtime_profiles.json
```
Profiles:
- `gpu-fast`: default interactive profile on `docker-gpu.cin.su`; uses RTX 4090 endpoints.
- `cpu-test`: benchmark/fallback profile on `docker-test.cin.su`; currently maps Qwen3-Coder Q6 to `http://docker-test.cin.su:18086` with served model `qwen3-coder-1c-q6-cpu`.
- `background`: batch profile for downloads, RAG indexing and conversions; not intended for direct chat.
The UI applies `model_overrides` from the selected runtime profile. This allows the same registry model
to use a different endpoint or served model name on another host.
The service panel has `Benchmark GPU / CPU`. It calls `POST /api/benchmark/runtime`, runs
`scripts/benchmark_runtime_profiles.py`, and stores the raw report in:
```text
/reports/benchmarks/runtime-profiles-<model-id>-<timestamp>.json
```
The UI loads recent benchmark history for the currently selected model/plugin through
`GET /api/benchmark/history?model_id=<model-id>&plugin=<plugin-id>`.
When both `gpu-fast` and `cpu-test` succeed, the report includes `speedup.gpu_vs_cpu_ratio`.
Use larger generation limits, for example 192-384 tokens, for representative GPU/CPU ratios;
very short runs include more startup and request overhead.
The UI has a separate `Benchmark tokens` field, default `384`, so chat generation limits do not
accidentally make CPU comparison runs too long.
Before starting the long benchmark request, the UI performs a quick preflight for both `gpu-fast`
and `cpu-test`; if either served model is missing, the benchmark is not started.
```text
POST /api/benchmark/preflight
```
CLI check:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\check_runtime_preflight.ps1
```
Each history row links to the raw JSON report through:
```text
/api/benchmark/report?name=<runtime-profiles-report.json>
```
It also links to a generated Markdown summary:
```text
/api/benchmark/report.md?name=<runtime-profiles-report.json>
```
For `cpu-test`, start the heavyweight llama.cpp CPU server before sending chat requests:
```powershell
powershell -ExecutionPolicy Bypass -File scripts\manage_cpu_q6_service.ps1 -Action start
powershell -ExecutionPolicy Bypass -File scripts\manage_cpu_q6_service.ps1 -Action status
powershell -ExecutionPolicy Bypass -File scripts\manage_cpu_q6_service.ps1 -Action logs
```
For `gpu-fast` Qwen3-Coder Q6 checks on port `8081`, use the GPU launcher. Stop image/video/vLLM
first if VRAM is tight:
```powershell
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile gguf-1c
powershell -ExecutionPolicy Bypass -File scripts\manage_gpu_q6_service.ps1 -Action start
powershell -ExecutionPolicy Bypass -File scripts\manage_gpu_q6_service.ps1 -Action status
powershell -ExecutionPolicy Bypass -File scripts\manage_gpu_q6_service.ps1 -Action logs
```
After you convert a trained PEFT LoRA adapter to GGUF for `llama.cpp`, you can start the same
service with the adapter applied:
```powershell
powershell -ExecutionPolicy Bypass -File scripts\convert_1c_lora_to_gguf_gpu.ps1
powershell -ExecutionPolicy Bypass -File scripts\manage_gpu_q6_service.ps1 `
-Action start `
-LoraPath /models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1.gguf
```
Stop it after benchmarks on the shared host if it is no longer needed:
```powershell
powershell -ExecutionPolicy Bypass -File scripts\manage_cpu_q6_service.ps1 -Action stop
powershell -ExecutionPolicy Bypass -File scripts\manage_gpu_q6_service.ps1 -Action stop
```
## Containers
```powershell
docker -H ssh://docker-gpu ps
docker -H ssh://docker-gpu logs --tail 100 llm-model-chat-ui
docker -H ssh://docker-gpu logs --tail 100 llm-vllm-text
docker -H ssh://docker-gpu logs --tail 100 llm-transformers-translation
```
## Transformers Plugin Services
Translation, audio, video, and image services use the small OpenAI-like server in
`scripts/transformers_plugin_server.py`.
```powershell
powershell -ExecutionPolicy Bypass -File scripts\deploy_transformers_service.ps1 -Plugin translation
powershell -ExecutionPolicy Bypass -File scripts\deploy_transformers_service.ps1 -Plugin audio
powershell -ExecutionPolicy Bypass -File scripts\deploy_transformers_service.ps1 -Plugin video
powershell -ExecutionPolicy Bypass -File scripts\deploy_transformers_service.ps1 -Plugin image
```
By default the deploy script also refreshes `Z:\LLM\model-chat-app` before restarting the selected
service, so changes in `scripts/transformers_plugin_server.py` are applied to the mounted `/app`
directory. Use `-NoSyncApp` only for a fast container restart when the app files are already current.
Ports:
- translation: `8010`
- audio: `8020`
- video: `8030`
- image: `8040`
The translation service exposes `/health`, `/v1/models`, and `/v1/chat/completions`.
The audio service exposes `/health`, `/v1/models`, and `/v1/audio/transcriptions`; in the UI,
select plugin `Звук`, choose an audio file, then press `Отправить`.
The video service exposes `/health`, `/v1/models`, and `/v1/vision/analyze`; in the UI,
select plugin `Видео`, choose an image, enter the question in the prompt box, then press `Отправить`.
The image service exposes `/health`, `/v1/models`, `/v1/images/generations`, `/v1/images/edits`,
and async job endpoints under `/v1/images/jobs`. In the UI, select plugin `Фото`, choose
an image model, choose `generate` or `edit / inpaint`, enter the prompt, then press `Отправить`.
By default `LOAD_ON_START=1` and `BACKGROUND_LOAD_ON_START=1` for image generation, so the service
opens HTTP quickly and loads the SDXL base model in the background. The first request can show
`loading_model` for several minutes. Switching between SDXL base and inpaint may unload the other
pipeline to keep VRAM available.
Stop the large text vLLM container before loading another large model if VRAM is tight.
The UI can select `Qwen Image Edit`, but the image endpoint must actually serve
`qwen-image-edit`; otherwise `/api/image/submit` rejects the job instead of silently using SDXL.
To test Qwen Image Edit as a single-heavy-model experiment, restart the image service with the Qwen
env file:
```powershell
docker -H ssh://docker-gpu compose `
--env-file core/deploy/docker-gpu/transformers/image.qwen-edit.env.example `
-f core/deploy/docker-gpu/transformers/image.compose.yaml up -d
```
Restore the verified SDXL service with:
```powershell
powershell -ExecutionPolicy Bypass -File scripts\deploy_transformers_service.ps1 -Plugin image
```
Verified Qwen Image Edit notes from `2026-06-20`:
- model files are present under `Z:\LLM\models\image\qwen-image-edit`;
- `diffusers` detects `QwenImageEditPipeline` and the service can expose `/v1/models` as
`qwen-image-edit`;
- pipeline load completed in about `31 s` with CPU offload on RTX 4090;
- a 512x512 edit job with `1` inference step did not finish within `1800 s`, so Qwen Image Edit is
not practical on the current 24 GB GPU profile without a quantized/optimized runtime or a larger
GPU;
- keep SDXL as the default verified image service for now.
Smoke test image jobs through the Model Chat proxy:
```powershell
python scripts\smoke_image_jobs.py --operation generate --steps 4
python scripts\smoke_image_jobs.py --operation edit --steps 4
python scripts\smoke_image_jobs.py --operation edit --steps 40 --cancel --cancel-after 1
python scripts\smoke_image_jobs.py --model-id qwen-image-edit --model-mode qwen-image-edit --model qwen-image-edit --operation edit --steps 4
```
Audio and video smoke checks generate small synthetic inputs locally and send them through the
Model Chat UI proxy:
```powershell
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile audio
python scripts\smoke_audio_transcription.py
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile video
python scripts\smoke_video_analysis.py
```
When the matching service is intentionally stopped, use `--allow-unavailable` for a code-path check
that does not fail the local validation run.
Verified on `2026-06-20` with RTX 4090:
- generate, SDXL base, 512x512, 4 steps: completed in `2194 ms` after warmup;
- edit / inpaint, SDXL inpaint, 512x512, 4 steps: completed in `180649 ms` including first inpaint model load;
- generate, SDXL base, 512x512, 1 step: completed in `86866 ms` after image service restart and warmup;
- generated artifacts are served through `/generated-images/<date>/<file>.png`.
Direct artifact check uses `GET`; this minimal HTTP server does not implement `HEAD` for generated files.
## Model Downloads On GPU Host
Use this when model files should be written directly to `Z:\LLM\models` on the GPU host:
For large Hugging Face shard files, prefer the explicit range downloader. It writes final files directly
to `/models/...`, resumes by local file size, and avoids stale `.cache/huggingface/download/*.incomplete`
files left by interrupted Xet/snapshot downloads.
```powershell
powershell -ExecutionPolicy Bypass -File scripts\download_hf_range_gpu.ps1 `
-CardId qwen2_5-vl-7b-instruct `
-LocalDir /models/video/qwen2.5-vl-7b-instruct `
-AllowFile "model-00001-of-00005.safetensors,model-00002-of-00005.safetensors,model-00003-of-00005.safetensors,model-00004-of-00005.safetensors,model-00005-of-00005.safetensors" `
-Detached `
-ContainerName llm-hf-range-qwen-vl `
-ChunkSize 256mb
```
The registry/snapshot downloader is still useful for dry-run planning and small metadata files:
```powershell
powershell -ExecutionPolicy Bypass -File scripts\download_missing_hf_models_gpu.ps1 -DryRun
docker -H ssh://docker-gpu rm -f llm-model-download
powershell -ExecutionPolicy Bypass -File scripts\download_missing_hf_models_gpu.ps1 `
-Detached `
-ContainerName llm-model-download
```
Monitor:
```powershell
docker -H ssh://docker-gpu ps
docker -H ssh://docker-gpu logs --tail 100 llm-model-download
Invoke-RestMethod http://192.168.220.91:8765/api/model-services
```
## Notes
- UI is served by `scripts/model_chat_server.py`.
- Inference endpoint defaults to `http://docker-gpu.cin.su:8000`.
- Use `vllm/vllm-openai:v0.10.2` for the UI image because it already contains Python and is verified on this host.
- `vllm/vllm-openai:latest` passes CUDA after driver `595.97`, but vLLM `0.23.0` currently fails on Docker Desktop/WSL with `UVA is not available`.
+77
View File
@@ -0,0 +1,77 @@
# Model Ingest
Цель: принять новую модель от пользователя, агента или внешнего источника без ручного копирования в реестр.
## UI
Локальный кабинет моделей доступен в Model Chat Testbench:
```text
http://127.0.0.1:8765
```
Блок `Модели` поддерживает:
- загрузку файла из браузера;
- заявку на импорт из URL;
- заявку на импорт из пути, доступного серверу;
- просмотр последних ingest jobs.
## API
Последние заявки:
```powershell
curl http://127.0.0.1:8765/api/model-ingest/jobs
```
Загрузить файл raw stream:
```powershell
curl -X POST "http://127.0.0.1:8765/api/model-ingest/upload?model_id=my-model&plugin=1c&format=gguf&filename=model.gguf" `
--data-binary "@model.gguf"
```
Попросить сервер скачать из URL или импортировать путь:
```powershell
curl -X POST http://127.0.0.1:8765/api/model-ingest/source `
-H "Content-Type: application/json" `
-d "{\"model_id\":\"my-model\",\"plugin\":\"1c\",\"format\":\"gguf\",\"source\":\"https://example/model.gguf\"}"
```
## Storage
Файлы складываются в:
```text
models/incoming/<job_id>/
```
Метаданные задания:
```text
models/incoming/<job_id>/metadata.json
```
Журнал:
```text
reports/model-ingest/jobs.jsonl
```
## Promotion
После проверки размера, checksum, лицензии и runtime модель нужно вручную промоутить:
1. Перенести файл из `models/incoming/<job_id>/` в целевую папку `models/...`.
2. Создать или обновить `registry/model-cards/<model_id>.yaml`.
3. Запустить:
```powershell
python scripts/validate_model_cards.py
python scripts/build_model_index.py
python scripts/check_model_storage.py --warn-only
```
Пароли к NAS, Hugging Face токены и другие секреты не записываются в ingest job и не должны попадать в model card.
+151
View File
@@ -0,0 +1,151 @@
# Q6 LoRA Troubleshooting
Цель: короткая памятка по типовым проблемам при обучении, конвертации и публикации `Qwen3-Coder Q6` с LoRA.
## Main Rule
На `docker-gpu` training/download контейнеры должны видеть код из `Z:\LLM\model-chat-app`, а не напрямую из `Z:\codex\LLM`.
Актуальный поток такой:
1. локальный репозиторий архивируется;
2. архив синхронизируется в `Z:\LLM\model-chat-app`;
3. training/download контейнеры используют этот каталог как `/workspace` или `/app`.
Если новые `scripts`, `plugins` или `registry/model-cards` не попали в `model-chat-app`, контейнеры будут работать на старом коде.
## Quick Checks
Проверить GPU host:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/check_gpu_host.ps1 -SshTarget docker-gpu
```
Показать план полного пайплайна:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/train_and_publish_q6_lora.ps1 -PlanOnly
```
Прогнать только безопасный preflight:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/train_and_publish_q6_lora.ps1 -SkipDownload -SkipTrain -SkipConvert -SkipRestart -SkipSmoke
```
## Common Failures
### `Model card not found for id qwen3-coder-30b-a3b-instruct`
Причина:
- контейнер загрузки стартовал раньше, чем новый `model card` попал в `Z:\LLM\model-chat-app`.
Что делать:
```powershell
. .\scripts\app_archive.ps1
$archive = New-AppArchive
Test-AppArchive -Archive $archive
Sync-AppDirectory -SshTarget docker-gpu -RemoteArchive 'C:\ProgramData\LLM\model-chat-app.zip' -RemoteAppDir 'Z:\LLM\model-chat-app' -Archive $archive
docker -H ssh://docker-gpu rm -f llm-hf-range-qwen3-coder-base
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\download_hf_range_gpu.ps1 -CardId qwen3-coder-30b-a3b-instruct -LocalDir /models/base/qwen3-coder-30b-a3b-instruct -Detached -ContainerName llm-hf-range-qwen3-coder-base
```
### `requirements-training.txt` not found
Причина:
- training container смонтировал неправильный workspace path.
Правильное ожидание:
- `HOST_WORKSPACE_DIR=Z:/LLM/model-chat-app`
- wrapper `scripts/run_1c_lora_training_gpu.ps1` сам синхронизирует текущий код в `model-chat-app` перед `docker compose up`.
### `base model is incomplete at /models/base/qwen3-coder-30b-a3b-instruct`
Причина:
- HF-база еще не скачана полностью.
Что смотреть:
```powershell
docker -H ssh://docker-gpu logs -f llm-hf-range-qwen3-coder-base
docker -H ssh://docker-gpu run --rm -v Z:/LLM/models:/models alpine sh -lc "ls -lah /models/base/qwen3-coder-30b-a3b-instruct"
```
Хороший признак:
- в логе идут строки `OK <filename>`
- появились все `model-00001-of-00016.safetensors` ... `model-00016-of-00016.safetensors`
### `CUDA out of memory`
Причина:
- слишком тяжелая конфигурация для текущего режима RTX 4090.
Что делать:
- остановить лишние тяжелые сервисы;
- убедиться, что активен только нужный training/runtime контур;
- при необходимости уменьшить training нагрузку в `plugins/1c/training/configs/qwen3-coder-30b-a3b-lora.yaml`.
### `llama.cpp` стартовал без адаптера
Причина:
- не передан `-LoraPath`
- `.gguf` адаптер не был создан
- опубликован не тот путь
Проверка:
```powershell
docker -H ssh://docker-gpu run --rm -v Z:/LLM/models:/models alpine sh -lc "ls -lah /models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1.gguf"
powershell -ExecutionPolicy Bypass -File scripts\manage_gpu_q6_service.ps1 -Action status -LoraPath /models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1.gguf
```
## Monitoring Commands
Скачивание базы:
```powershell
docker -H ssh://docker-gpu logs -f llm-hf-range-qwen3-coder-base
```
Обучение:
```powershell
docker -H ssh://docker-gpu logs -f llm-train-1c-lora
```
Q6 runtime:
```powershell
docker -H ssh://docker-gpu logs -f llm-llama-qwen3-coder-q6-test
```
Endpoint check:
```powershell
python scripts\check_inference_endpoint.py --base-url http://docker-gpu.cin.su:8081 --expected-model qwen3-coder-1c-q6 --print
```
Smoke eval:
```powershell
python scripts\run_1c_smoke_eval.py --base-url http://docker-gpu.cin.su:8081 --model qwen3-coder-1c-q6
```
## Recommended Order
1. Дождаться полной загрузки HF-базы.
2. Прогнать `PreflightOnly`.
3. Запустить обучение.
4. Конвертировать LoRA в GGUF.
5. Перезапустить `Q6` с `-LoraPath`.
6. Прогнать endpoint check и smoke eval.

Some files were not shown because too many files have changed in this diff Show More