Initial SQL-only 1C adapter baseline
This commit is contained in:
@@ -0,0 +1,91 @@
|
||||
# 1C Agent Coding Contract
|
||||
|
||||
This contract is the default rule set for coding agents that work through the
|
||||
1C adapter.
|
||||
|
||||
## Default View
|
||||
|
||||
- Read current code and metadata from the working saved-state layer by default.
|
||||
- For REST calls, use `state=working`.
|
||||
- For MCP calls, use `source_state=working`.
|
||||
- Treat saved-state objects as current programming state even when they are not
|
||||
activated yet.
|
||||
- Objects can exist only in saved-state and can later be activated or canceled.
|
||||
Do not hide them from the agent view.
|
||||
|
||||
## Compare Views
|
||||
|
||||
- Use `state=both` or `source_state=all` only when the task needs a comparison
|
||||
with activated runtime state.
|
||||
- Show the effective working text first.
|
||||
- Mark comparison details explicitly:
|
||||
- `saved_state`: saved and not activated;
|
||||
- `active`: activated runtime state;
|
||||
- `text_source`: which layer produced the returned text;
|
||||
- `comparison.differs`: whether both layers exist and differ.
|
||||
|
||||
## Read Workflow
|
||||
|
||||
Use public names and selectors:
|
||||
|
||||
1. `extension.objects.find` with `state=working` to find extension objects.
|
||||
2. `code.search` with `state=working` to find routines or fragments.
|
||||
3. `code.read` with `state=working` to read the module or routine.
|
||||
4. `code.read` with `state=both` only for an explicit saved-vs-active check.
|
||||
|
||||
Agents should ask for and report object names, routine names, and code text.
|
||||
They should not ask users for SQL tables, storage file names, stream indexes, or
|
||||
saved-state write flags during normal coding work.
|
||||
|
||||
## Write Workflow
|
||||
|
||||
All normal BSL writes go through `code.write`.
|
||||
|
||||
Supported public edit shapes:
|
||||
|
||||
- replace a whole module with `module_text`, `full_text`, or `code`;
|
||||
- replace one procedure or function with `routine_name` and `routine_text`;
|
||||
- replace one unique fragment with `old` and `new`; when `routine_name` is
|
||||
supplied, the adapter scopes the replacement to that routine.
|
||||
|
||||
`code.write` saves into saved-state automatically. A coding agent should simply
|
||||
say "save this code" and send the desired code text. It must not ask whether SQL
|
||||
saved-state apply flags are allowed.
|
||||
|
||||
Every successful `code.write` response must show:
|
||||
|
||||
- `write_mode.target=saved_state`;
|
||||
- `write_mode.activation_state=not_activated`;
|
||||
- `write_mode.production_apply=false`.
|
||||
|
||||
## Hidden Storage Details
|
||||
|
||||
The form module container marker `///----` is adapter-owned storage syntax.
|
||||
Public `code.read` and `code.search` responses must not expose it as BSL.
|
||||
Full-module writes must preserve the marker internally when the saved form
|
||||
payload requires it.
|
||||
|
||||
Low-level methods such as `metadata.module.write_apply`, `metadata.write`, SQL
|
||||
tables, stream refs, and saved-state apply flags are diagnostic tools. They are
|
||||
not the default programming interface for agents.
|
||||
|
||||
## Agent Response Shape
|
||||
|
||||
When reporting a working saved-state result to a user, prefer concise wording:
|
||||
|
||||
```text
|
||||
В working/save вижу формы:
|
||||
t_Форма
|
||||
tt_Форма3
|
||||
ФормаЭлемента
|
||||
|
||||
Код читается из saved_state, еще не активирован.
|
||||
```
|
||||
|
||||
If the user asks to compare active and saved state:
|
||||
|
||||
```text
|
||||
Working/save: найдено, источник saved_state, не активировано.
|
||||
Active: не найдено.
|
||||
Эффективный код для программирования сейчас берется из saved_state.
|
||||
```
|
||||
@@ -0,0 +1,88 @@
|
||||
# 1C Agent (отдельный подпроект)
|
||||
|
||||
## Быстрый запуск на test-docker
|
||||
|
||||
```powershell
|
||||
# 1) Подготовить переменные (без секретов)
|
||||
cd Z:\codex\LLM
|
||||
Copy-Item core\deploy\docker\1c-agent\1c-agent.env.example .\core\deploy\docker\1c-agent\\.env
|
||||
```
|
||||
|
||||
Редактируйте `.env`:
|
||||
- `ONEC_ADAPTER_URL` → URL REST-адаптера;
|
||||
- `ONEC_ADAPTER_TOKEN` → токен (если настроен).
|
||||
- при необходимости `ONEC_AGENT_PROVIDERS` (JSON).
|
||||
|
||||
```powershell
|
||||
$env:DOCKER_HOST = "ssh://test-docker"
|
||||
docker compose -f core\deploy\docker\1c-agent\compose.yaml --env-file core\deploy\docker\1c-agent\.env up -d --build
|
||||
```
|
||||
|
||||
Проверка:
|
||||
|
||||
```powershell
|
||||
docker ps --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
|
||||
curl http://docker-test.cin.su:8090/v1/health
|
||||
```
|
||||
|
||||
Если внешний хост не отвечает из вашей сети, проверьте локально в контейнере:
|
||||
|
||||
```powershell
|
||||
$env:DOCKER_HOST = "ssh://test-docker"
|
||||
docker exec onec-agent python -c "import urllib.request, json; print(json.loads(urllib.request.urlopen('http://127.0.0.1:8090/v1/health', timeout=5).read().decode()))"
|
||||
```
|
||||
|
||||
## Основные потоки
|
||||
|
||||
- `project` — единица бизнеса/сценария: имя, описание, политики, базовая настройка.
|
||||
- `chat` — конкретная сессия: выбранная модель, провайдер, параметры генерации, локальные параметры RAG.
|
||||
- `message` — отдельные сообщения внутри чата, используемые для контекста.
|
||||
|
||||
## Что лучше держать где
|
||||
|
||||
- Project-level:
|
||||
- название/описание;
|
||||
- долгоживущие политики доступа/ограничения;
|
||||
- общие настройки безопасности и базовые параметры.
|
||||
- Chat-level:
|
||||
- выбранный провайдер/модель;
|
||||
- `rag_profile`, `rag_limit`;
|
||||
- `temperature`, `max_tokens`;
|
||||
- системный prompt конкретной сессии.
|
||||
|
||||
## Взаимодействие с разными ИИ-провайдерами
|
||||
|
||||
Сервис уже умеет работать через `ONEC_AGENT_PROVIDERS` как через карту:
|
||||
|
||||
```json
|
||||
{
|
||||
"default": {
|
||||
"type": "openai-compatible",
|
||||
"base_url": "http://docker-gpu.cin.su:8000",
|
||||
"model": "qwen3-4b-instruct-2507"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Если нужно подключить другой ИИ протокол, добавляем новый `type` в `plugins/1c/agent/agent_server.py` в диспетчер `call_model(...)`.
|
||||
|
||||
## Проверка turn
|
||||
|
||||
```powershell
|
||||
curl -X POST "http://docker-test.cin.su:8090/v1/projects" `
|
||||
-H "Content-Type: application/json" `
|
||||
-d '{ "name":"demo-1c", "description":"Проверка пайплайна" }'
|
||||
|
||||
curl -X POST "http://docker-test.cin.su:8090/v1/projects/<project_id>/chats" `
|
||||
-H "Content-Type: application/json" `
|
||||
-d '{ "title":"Проверка" }'
|
||||
|
||||
curl -X POST "http://docker-test.cin.su:8090/v1/projects/<project_id>/chats/<chat_id>/turn" `
|
||||
-H "Content-Type: application/json" `
|
||||
-d '{ "message":"Какие реквизиты есть у справочника Номенклатура?", "use_rag": true }'
|
||||
```
|
||||
|
||||
## Статус/поддержка
|
||||
|
||||
- Проверить, что сервис запущен: `GET /v1/health`.
|
||||
- Проверить модели/провайдеры: `GET /v1/models`, `GET /v1/providers`.
|
||||
@@ -0,0 +1,316 @@
|
||||
# Controlled Designer-to-SQL Decoding
|
||||
|
||||
Use the disposable `upo_test` base to learn SQL encodings that cannot be
|
||||
established from static samples. The mutation is performed by 1C Designer or a
|
||||
1C Enterprise client. The adapter remains a read-only SQL observer. The active
|
||||
rule is `plugins/1c/connector/policies/designer-sql-decoding-policy.yaml`.
|
||||
|
||||
## Credential rule
|
||||
|
||||
Keep the 1C user password outside the repository. Supply it for one process
|
||||
through an environment variable or an operating-system credential store. Do
|
||||
not add it to `.env`, YAML, JSON, test fixtures, reports, or command examples.
|
||||
|
||||
## Metadata experiment
|
||||
|
||||
1. Select an object by a public 1C name.
|
||||
2. Confirm there are no unrelated pending Designer changes.
|
||||
3. Capture the target from live SQL, including active and saved-state origin.
|
||||
4. In Designer change exactly one property and save it without applying the
|
||||
configuration when saved-state evidence is sufficient.
|
||||
5. Capture `ConfigSave` or `ConfigCASSave` again and compute the structural and
|
||||
byte-level diff.
|
||||
6. Repeat with a second value; one pair is only a hypothesis.
|
||||
7. Promote a decoder only after the name, type, path, and ownership are stable.
|
||||
8. Revert through Designer and verify rollback in SQL.
|
||||
|
||||
Applying the configuration is a separate explicit phase because it can change
|
||||
`Config`, `ConfigCAS`, and the physical application-data schema.
|
||||
|
||||
## Application-data experiment
|
||||
|
||||
1. Resolve the object through `data.schema` using its public name.
|
||||
2. Capture `data.count` and a narrowly filtered `data.list`.
|
||||
3. Create or edit one test record through 1C Enterprise, never through SQL.
|
||||
4. Capture the same logical filter after the 1C transaction commits.
|
||||
5. Correlate logical values with SQL columns, including composite branches.
|
||||
6. Revert or delete through 1C Enterprise and verify rollback read-only.
|
||||
|
||||
## Noise controls
|
||||
|
||||
- Record configuration-check errors that existed before the experiment.
|
||||
- Do not run two experiments against the same object concurrently.
|
||||
- Separate base configuration and extension ownership.
|
||||
- Treat timestamps, version bytes, caches, and background service data as
|
||||
volatile unless explicitly targeted.
|
||||
- Discard a run when more than one semantic property changed.
|
||||
|
||||
Each accepted experiment produces a manifest with the public selector,
|
||||
intended change, before/after SQL hashes, changed paths or columns, semantic
|
||||
rule, second verification case, rollback evidence, and regression tests. XML
|
||||
may be attached as offline naming evidence but is not read by the adapter.
|
||||
|
||||
## Confirmed baseline
|
||||
|
||||
The first live `upo_test` CAS comparison is recorded in
|
||||
`reports/1c-sql/upo_test/designer-sql-baseline-20260714.json`. It proves a BSL
|
||||
module-text change at tree path `$.2`. The accompanying `pos` and `end` changes
|
||||
are stream-directory offsets recalculated from the text length; they are not
|
||||
independent metadata properties and must be filtered as derived evidence.
|
||||
|
||||
## Saved extension metadata descriptors
|
||||
|
||||
The `test2` experiment on 2026-07-15 added a minimal calculation register and
|
||||
its required chart of calculation types through Designer, then saved the
|
||||
extension without applying it. The objects remain intentionally `saved_only`
|
||||
for adapter regression checks.
|
||||
|
||||
Observed read-only SQL signatures in `ConfigCASSave`:
|
||||
|
||||
- `CalculationRegister`: brace root marker `1`, root length `10`, metadata
|
||||
block marker `21`;
|
||||
- `ChartOfCalculationTypes`: brace root marker `1`, root length `8`, metadata
|
||||
block marker `35`;
|
||||
- saved extension descriptor names use
|
||||
`<extension-guid>__<object-guid>`; child/module parts add a numeric suffix.
|
||||
|
||||
The runtime adapter derives these signatures only from SQL payloads. The XML
|
||||
export is offline evidence used to confirm the public object kind, name, GUID,
|
||||
and the register-to-chart relationship; it is not a runtime data source.
|
||||
|
||||
When `state=working`, `extension.objects.find` must overlay these descriptors
|
||||
from `ConfigCASSave` and report `saved_only` or `saved_override`. With
|
||||
`state=active`, the same unapplied objects must not be returned. Route-cache
|
||||
rebuilds must reclassify active descriptors so an earlier guessed kind cannot
|
||||
survive as a false match.
|
||||
|
||||
Public saved-state card selectors expose `extension_guid`, object `guid`, and
|
||||
`table=ConfigCASSave`; they do not expose the physical descriptor name. The
|
||||
adapter reconstructs `<extension-guid>__<object-guid>` internally before the
|
||||
read-only SQL lookup.
|
||||
|
||||
For `ChartOfCalculationTypes`, the verified kind-specific map currently covers
|
||||
17 properties: the five scalar code/name settings at paths `1.24`–`1.30`, six
|
||||
default/auxiliary form references, five localized presentations, and
|
||||
`ActionPeriodUse` at `1.57`. The paths were cross-checked on the live
|
||||
`Начисления` and `Удержания` descriptors against their offline XML exports.
|
||||
|
||||
For the saved-only `CalculationRegister`, the verified header map covers 13
|
||||
properties: periodicity, action/base-period flags, list-form references, chart
|
||||
reference, standard-command/help flags, lock and full-text modes, and three
|
||||
localized list presentations. `metadata.object.properties` accepts the public
|
||||
`extension_guid` + object `guid` selector and reconstructs the `ConfigCASSave`
|
||||
descriptor name internally. Runtime property responses remain SQL-only.
|
||||
|
||||
The saved register descriptor also confirms all seven variable child-part
|
||||
collections: `Attribute` at root path `3`, `Recalculation` at `4`, `Template`
|
||||
at `5`, `Resource` at `6`, `Form` at `7`, `Command` at `8`, and `Dimension` at
|
||||
`9`. The three field roles were distinguished by a controlled Designer sample
|
||||
containing one `Реквизит1`, `Ресурс1`, and `Измерение1`. Designer saved the
|
||||
extension without applying it; the adapter identified the records read-only
|
||||
from the resulting live `ConfigCASSave` payload.
|
||||
|
||||
Saved-only register fields are available through `metadata.object.attributes`
|
||||
with `table=ConfigCASSave`, `extension_guid`, and the public object `guid`.
|
||||
The response returns names and decoded types while reconstructing the physical
|
||||
saved descriptor key internally. `metadata.object.related` accepts the same
|
||||
selector for recalculations, templates, forms, and commands; saved child keys
|
||||
are also prefixed internally and remain hidden unless storage diagnostics are
|
||||
explicitly requested.
|
||||
|
||||
A second controlled Designer sample added one item to every related collection:
|
||||
`Перерасчет1`, `Макет`, `ФормаСписка`, and `Команда1`. The root descriptor then
|
||||
reported a declared count of one at paths `4`, `5`, `7`, and `8`. Recalculation,
|
||||
template, and form descriptors use the internal
|
||||
`<extension-guid>__<child-guid>` key. A saved command exposes its BSL payload as
|
||||
`<extension-guid>__<command-guid>.2`; the additional command-class GUID inside
|
||||
the owner record is type evidence, not a second related command. Public related
|
||||
results therefore keep only the record identity GUID and probe the `.2` module
|
||||
route internally when the direct saved command descriptor is absent.
|
||||
|
||||
The specialized `metadata.object.forms`, `metadata.object.form.details`,
|
||||
`metadata.object.commands`, and `metadata.object.modules` methods accept the
|
||||
same saved-state selector. Form enumeration and detail decoding use the
|
||||
internally reconstructed `<extension-guid>__<form-guid>.0` payload. Object
|
||||
commands are returned from the owner descriptor and their saved `.2` payload
|
||||
is verified without exposing the physical key in normal responses. Each saved
|
||||
object command also returns a ready public `modules.read` selector, so callers
|
||||
select the command by name and never need to calculate its GUID or SQL route.
|
||||
The `.2` payload is a raw-deflate multi-stream container; `modules.read`
|
||||
automatically selects its single BSL-marked stream when whole-payload text
|
||||
decoding is not applicable.
|
||||
|
||||
Object-scoped `modules.search` and its `code.search` wrapper include these
|
||||
saved command modules alongside the owner's regular modules. A caller can
|
||||
therefore search by the public register selector plus BSL text and receive a
|
||||
ready `modules.read`/`code.read` selector for the matching command module.
|
||||
`metadata.definition.find` with `areas=["modules"]` follows the same saved
|
||||
selector, enumerates command-module routines, and returns the exact procedure
|
||||
or function definition with a routine-scoped read selector.
|
||||
|
||||
`metadata.object.parts` and `metadata.object.decode` also reconstruct the
|
||||
saved descriptor prefix before reading. The former enumerates the root and
|
||||
suffix payloads under `<extension-guid>__<object-guid>`; the latter decodes the
|
||||
root descriptor from that key while keeping physical storage coordinates
|
||||
hidden by default.
|
||||
|
||||
The combined `metadata.object.full` profile promotes saved object-command
|
||||
selectors into its `modules` collection as `command_module` handles. This
|
||||
keeps the profile lightweight (no BSL text is loaded there) while ensuring the
|
||||
reported module count includes code carriers owned by commands.
|
||||
|
||||
Targeted `metadata.code_index.build` runs on saved command `.2` containers now
|
||||
retain only streams that are positively identified as BSL. Command owner
|
||||
metadata cached from `metadata.object.commands` is inherited by the concrete
|
||||
`#stream:N` index row, and obsolete non-BSL rows plus their vector chunks are
|
||||
pruned when the source file is rebuilt. The extension GUID is recovered from
|
||||
the saved module route and retained in the indexed owner metadata.
|
||||
|
||||
For saved extensions, `metadata.object.get` resolves an object name through
|
||||
the saved extension manifest/state route before reading its descriptor.
|
||||
The public `/rpc` dispatcher preserves this name selector when
|
||||
`extension_guid` and `table=ConfigCASSave` are supplied; callers may use the
|
||||
object name directly and do not have to resolve its GUID first. An explicit
|
||||
`guid` still takes precedence when both selectors are present.
|
||||
`metadata.code_index.build` can therefore be scoped by `kind` plus `name` and
|
||||
`extension_guid`; it discovers owner and command module files internally and
|
||||
does not require callers to pass a physical prefix or command GUID.
|
||||
The same object-scoped build enumerates saved forms, decodes their embedded
|
||||
modules separately from container streams, and indexes a form only when a
|
||||
valid non-empty BSL module is present. Empty generated forms are not emitted
|
||||
as code carriers. Build counts distinguish `forms_scanned`,
|
||||
`form_modules_discovered`, `empty_form_modules`, and `form_module_errors`, so
|
||||
coverage and decoding failures are observable separately.
|
||||
|
||||
`CalculationRegister` is included in the public register code-carrier matrix.
|
||||
Its record-set/register/command module handles use the same name-first SQL-only
|
||||
read and saved-state write contract as the other register kinds.
|
||||
|
||||
Offline `Form.xml` analysis uses the same public vocabulary as the SQL form
|
||||
decoder for form commands, events, attributes and value-table columns, check
|
||||
box fields, and search/view-status/search-control additions. The XML profile
|
||||
also exposes semantic properties of the root `Form` node, including command
|
||||
bar location and visibility. XML remains comparison evidence only and is never
|
||||
consulted by runtime adapter calls.
|
||||
|
||||
The SQL/XML comparison report records property-route evidence as
|
||||
`XML kind/property -> SQL marker/parameter/source`. Across the 11 controlled
|
||||
test-extension forms, 764 elements and 3,361 properties compare without
|
||||
mismatches or XML-only properties; 23 routes have at least two matching
|
||||
examples and a single SQL route. `UsualGroup.Visible` remains intentionally
|
||||
variant-aware because marker `22` uses parameter `26` or `28` in two observed
|
||||
structures. A controlled Designer probe on
|
||||
`t_FORM_ContainerTableBehaviorVariants` changed only
|
||||
`ГруппаФорма.Visible=false -> true`: parameter `26` changed `0 -> 1`, while
|
||||
parameter `10` and nested `Группа1` remained unchanged. In the nested shape,
|
||||
parameter `26` is a GUID and parameter `28` is the boolean visibility slot.
|
||||
Saved-state writes therefore select parameter `26` only when it is boolean;
|
||||
otherwise they select boolean parameter `28`. The extension was restored with
|
||||
`/RollbackCfg -Extension test`, and its `ConfigCASSave` prefix was verified
|
||||
empty after the probe.
|
||||
|
||||
Root form properties are compared separately from element properties. A
|
||||
controlled `ShowCommandBar=false -> true` Designer probe changed exactly two
|
||||
form atoms: parameter `17` from `0` to `2` and parameter `56` from `0` to `1`.
|
||||
The SQL profile exposes this as `ОтображатьКоманднуюПанель` with
|
||||
`write_shape=paired_scalar`; it is readable but must not be routed through a
|
||||
single-scalar writer. `АвтоКоманднаяПанель` is resolved from the decoded form
|
||||
item with `id=-1`. Before the next controlled probe, 33 root properties
|
||||
matched and 44 remained.
|
||||
|
||||
A controlled `WindowOpeningMode=DontBlock -> LockOwner` Designer probe changed
|
||||
exactly form parameter `2` from `0` to `1` and companion parameter `54` from
|
||||
`0` to `1`. The SQL profile exposes the confirmed values as
|
||||
`РежимОткрытияОкна=DontBlock|LockOwner` with `write_shape=paired_scalar`.
|
||||
Other platform enum values remain undecoded until separately observed. The
|
||||
probe was loaded only into the saved extension configuration, then rolled back;
|
||||
the `test` extension prefix in `ConfigCASSave` was verified empty afterwards.
|
||||
Across the 11 fixtures, 44 root properties now match; the remaining 33 are
|
||||
three scalar properties repeated on each form: `AutoSaveDataInSettings`, root
|
||||
`Group`, and `CommandBarLocation`.
|
||||
|
||||
A controlled `AutoSaveDataInSettings=Use -> DontUse` Designer probe changed
|
||||
only form parameter `7` from `1` to `0`. The SQL profile exposes this as
|
||||
`АвтоСохранениеДанныхВНастройках=Use|DontUse` with
|
||||
`write_shape=scalar_enum`. The extension was rolled back and the `test`
|
||||
prefix in `ConfigCASSave` was verified empty. Across the 11 fixtures, 55 root
|
||||
properties now match; the remaining 22 are root `Group` and
|
||||
`CommandBarLocation`, each repeated on all forms.
|
||||
|
||||
A controlled root `Group=Vertical -> Horizontal` Designer probe changed four
|
||||
form parameters together: `11`, `40`, `47`, and `57`, all from `0` to `1`.
|
||||
The SQL profile exposes `Группировка=Vertical|Horizontal` with
|
||||
`write_shape=composite_scalar`; partial single-atom writes are not safe. The
|
||||
extension was rolled back and the `test` prefix was verified empty. Across the
|
||||
11 fixtures, 66 root properties now match; only `CommandBarLocation` remains.
|
||||
|
||||
`CommandBarLocation` shares the same parameter pair as `ShowCommandBar`. With
|
||||
the panel enabled, `Top` produced `[17,56]=[2,1]` and `Bottom` produced
|
||||
`[3,1]`; the hidden state is `[0,0]` and is exposed as `None`. XML `None` with
|
||||
`ShowCommandBar=true` normalizes to the same SQL state as `Top`, so SQL exposes
|
||||
the effective position. The profile returns
|
||||
`ПоложениеКоманднойПанели=None|Top|Bottom` with
|
||||
`write_shape=paired_scalar_shared`; both properties must be encoded together.
|
||||
After each probe the extension was rolled back and its saved prefix was empty.
|
||||
All 77 root properties across the 11 controlled forms now match SQL to XML.
|
||||
|
||||
The platform's third window-opening mode was verified separately because it is
|
||||
not present in the UPO XML inventory. The accepted XML literal is
|
||||
`LockWholeInterface`; a controlled Designer probe changed form parameters
|
||||
`[2,54]` from `[0,0]` to `[2,2]`. `РежимОткрытияОкна` now decodes the complete
|
||||
confirmed enum: `DontBlock=[0,0]`, `LockOwner=[1,1]`, and
|
||||
`LockWholeInterface=[2,2]`. The failed tentative `LockUI` literal was rejected
|
||||
by XDTO before any SQL saved state was created. The successful probe was rolled
|
||||
back and the extension saved prefix was verified empty.
|
||||
|
||||
The complete root `Group` enum observed in UPO was verified with two additional
|
||||
Designer probes. `AlwaysHorizontal` maps to
|
||||
`[11,40,47,57]=[1,1,3,3]`, while `HorizontalIfPossible` maps to
|
||||
`[1,2,2,2]`. Together with `Vertical=[0,0,0,0]` and
|
||||
`Horizontal=[1,1,1,1]`, the decoder now covers every root form grouping value
|
||||
present in the XML inventory. Each probe was rolled back and the saved prefix
|
||||
was verified empty.
|
||||
|
||||
Active base-configuration forms also use compact root layouts. Across multiple
|
||||
SQL-only samples, form payload versions 49 and 50 store
|
||||
`WindowOpeningMode` directly in parameter `2`; values `0` and `1` were
|
||||
confirmed by `DontBlock` list forms and `LockOwner` item forms. In these
|
||||
layouts, parameter `11=0` consistently identifies root `Group=Vertical` even
|
||||
when the newer companion positions are absent. The decoder applies these
|
||||
fallbacks only to versions 49/50 and observed values; runtime remains SQL-only.
|
||||
|
||||
The active base catalog form `ЗадачиАссистентаУправления.ФормаСписка`
|
||||
confirmed the compact marker-55 dynamic-list layout. Parameter `54` is the
|
||||
property-bag entry count; its typed key/value pairs decode keys `5`, `6`, `8`,
|
||||
`9`, `11`, `12`, `14`, and `16` as `AutoRefresh`, `AutoRefreshPeriod`,
|
||||
`ChoiceFoldersAndItems`, `RestoreCurrentRow`, `ShowRoot`, `AllowRootChoice`,
|
||||
`UpdateOnDataChange`, and `AllowGettingCurrentRowURL`. The following tail
|
||||
records contain the user-settings-group item id, `InitialTreeView`, and the
|
||||
standard `DefaultPicture` field reference. The same live payload confirms
|
||||
`CommandBarLocation=None` at parameter `6` and `DefaultItem=true` at parameter
|
||||
`16`. Compact marker-22 command bars expose `Autofill` at parameter `28`.
|
||||
Marker-35 label fields with nested subtype marker `11` store
|
||||
`AutoMaxWidth/MaxWidth` in nested positions `15/16` of parameter `39`.
|
||||
|
||||
The SQL/XML comparison normalizes these public XML names to the decoder's
|
||||
Russian semantic vocabulary and treats marker-55 `Динамический список` as the
|
||||
SQL implementation of XML `Table`. XML is used only to validate the learned
|
||||
routes; runtime decoding reads the SQL payload alone.
|
||||
|
||||
The base item form of the same catalog confirmed more compact-layout routes.
|
||||
Root parameter `20` is a typed enum with type GUID
|
||||
`59ef2b80-c86b-11d5-a3c1-0050bae0a776`; value `0` decodes
|
||||
`UseForFoldersAndItems=Items`. Root event bindings may reside in a direct form
|
||||
block such as `1.23`; they are discovered by the GUID/handler pair shape rather
|
||||
than a fixed position. GUID `bf0ac0e1-bcbb-4dfe-8fc4-0b1923b461a6` identifies
|
||||
`BeforeWriteAtServer`. Compact pages options `{3,1,...}` decode
|
||||
`PagesRepresentation=TabsOnTop`.
|
||||
|
||||
Object-backed data paths resolve public standard fields `-2/-3/-4/-5` as
|
||||
`Code/Description/Parent/Ref`; custom object and tabular-section fields are
|
||||
resolved from the element name and owning table. XML `AdditionalColumns`
|
||||
definitions are correlated with the physical SQL form element by their full
|
||||
data path. Across both active forms of
|
||||
`Catalog.ЗадачиАссистентаУправления`, the comparison now covers 144 XML
|
||||
elements, including the logical additional column, with no missing elements,
|
||||
property mismatches, or XML-only properties.
|
||||
@@ -0,0 +1,176 @@
|
||||
# 1C Form Command Binding Learning
|
||||
|
||||
This runbook tracks the two remaining `ТестНастройки` SQL/XML gaps after the
|
||||
form decoder reached zero missing items and zero mismatches.
|
||||
|
||||
## Target
|
||||
|
||||
- Base: `upo_test`
|
||||
- Adapter: `http://docker-gpu.cin.su:8011`
|
||||
- Saved-state table: `ConfigCASSave`
|
||||
- Form payload file:
|
||||
`f96a0c45-3eff-11f1-8287-005056b0d483__25c39fbf-35a4-4b43-8e3f-cd1f91082c88.0`
|
||||
- Form: `ТестНастройки`
|
||||
- Current baseline payload SHA1:
|
||||
`0ca61aa4ed041fc4219cada0126d5f144e59d30d`
|
||||
|
||||
## Before Captures
|
||||
|
||||
The baseline captures were created on 2026-07-02.
|
||||
|
||||
| Learning ID | Element | Expected XML binding | Snapshot |
|
||||
| --- | --- | --- | --- |
|
||||
| `form-command-binding-standard-customize-form` | `ТЗИзменитьФорму` | `Form.StandardCommand.CustomizeForm` | `38cab2dc00c9465e9dc42a17548d3170` |
|
||||
| `form-command-binding-local-apply-command` | `ФормаКомандаОбновить` | `Form.Command.КомандаПрименить` | `e312ccc2e6ed45fbb6ebac187d30ba7a` |
|
||||
|
||||
Adapter-side capture paths:
|
||||
|
||||
- `/data/adapter-write-learning/form-command-binding-standard-customize-form/before-38cab2dc00c9465e9dc42a17548d3170.json`
|
||||
- `/data/adapter-write-learning/form-command-binding-local-apply-command/before-e312ccc2e6ed45fbb6ebac187d30ba7a.json`
|
||||
|
||||
## XML Fixture Workflow
|
||||
|
||||
Use the XML workflow first, matching the moxel discovery flow: edit/generate XML,
|
||||
load it into the test extension, then inspect the changed SQL payload.
|
||||
|
||||
Generate fixtures:
|
||||
|
||||
```powershell
|
||||
python scripts/create_1c_form_command_binding_xml_fixtures.py
|
||||
```
|
||||
|
||||
Generated files:
|
||||
|
||||
| Fixture | Purpose |
|
||||
| --- | --- |
|
||||
| `reports/1c-sql/upo_test/xml-command-binding-fixtures/00-original/Form.xml` | baseline copy |
|
||||
| `reports/1c-sql/upo_test/xml-command-binding-fixtures/01-local-button-command-example1/Form.xml` | changes `ФормаКомандаОбновить` to `Form.Command.КомандаПример1` |
|
||||
| `reports/1c-sql/upo_test/xml-command-binding-fixtures/02-standard-button-to-local-command/Form.xml` | changes `ТЗИзменитьФорму` to `Form.Command.КомандаПример1` |
|
||||
| `reports/1c-sql/upo_test/xml-command-binding-fixtures/03-combined-command-binding-switches/Form.xml` | applies both existing-button changes |
|
||||
| `reports/1c-sql/upo_test/xml-command-binding-fixtures/04-add-two-learning-buttons/Form.xml` | adds two new learning buttons under `Группа2` |
|
||||
|
||||
Preferred first load:
|
||||
|
||||
1. Load `01-local-button-command-example1/Form.xml` into
|
||||
`фс_ДоработкиОбщее.DataProcessor.фс_НастройкаУсловногоОформления.Forms.ТестНастройки`.
|
||||
2. Update/save the extension so `ConfigCASSave` receives a new form payload.
|
||||
3. Run:
|
||||
|
||||
```powershell
|
||||
python scripts/run_1c_form_command_binding_learning.py `
|
||||
--wait-for-sha-change 0ca61aa4ed041fc4219cada0126d5f144e59d30d `
|
||||
--max-wait-seconds 300 `
|
||||
--report reports/1c-sql/upo_test/form-command-binding-learning-run.json
|
||||
```
|
||||
|
||||
Then use the adapter diff/inference output to promote a decoder/write rule.
|
||||
If the first load is clean, repeat with `02-standard-button-to-local-command`
|
||||
or `03-combined-command-binding-switches`.
|
||||
|
||||
## Experiment A: Standard Command Binding
|
||||
|
||||
Goal: learn where marker `34` stores a standard command binding.
|
||||
|
||||
After any manual save, the preferred one-command runner is:
|
||||
|
||||
```powershell
|
||||
python scripts/run_1c_form_command_binding_learning.py `
|
||||
--wait-for-sha-change 0ca61aa4ed041fc4219cada0126d5f144e59d30d `
|
||||
--max-wait-seconds 300 `
|
||||
--report reports/1c-sql/upo_test/form-command-binding-learning-run.json
|
||||
```
|
||||
|
||||
Manual edit in Designer:
|
||||
|
||||
1. Open form `ТестНастройки`.
|
||||
2. Select button `ТЗИзменитьФорму`.
|
||||
3. Change only `CommandName` from `Form.StandardCommand.CustomizeForm` to a
|
||||
different standard command if Designer allows it.
|
||||
4. Save the form once.
|
||||
|
||||
After save, capture:
|
||||
|
||||
```powershell
|
||||
@'
|
||||
import json, urllib.request
|
||||
payload = {
|
||||
"base_id": "upo_test",
|
||||
"learning_id": "form-command-binding-standard-customize-form",
|
||||
"table": "ConfigCASSave",
|
||||
"file_name": "f96a0c45-3eff-11f1-8287-005056b0d483__25c39fbf-35a4-4b43-8e3f-cd1f91082c88.0",
|
||||
"form": "ТестНастройки",
|
||||
"element": "ТЗИзменитьФорму",
|
||||
"property": "ИмяКоманды",
|
||||
"timeout_seconds": 60,
|
||||
"max_items": 5000
|
||||
}
|
||||
req = urllib.request.Request("http://docker-gpu.cin.su:8011/rpc", data=json.dumps({"method":"metadata.write_learning.capture_after","payload":payload}, ensure_ascii=False).encode("utf-8"), headers={"Content-Type":"application/json"})
|
||||
print(urllib.request.urlopen(req, timeout=90).read().decode("utf-8"))
|
||||
'@ | python -
|
||||
```
|
||||
|
||||
Then run:
|
||||
|
||||
```powershell
|
||||
@'
|
||||
import json, urllib.request
|
||||
for method in ("metadata.write_learning.diff", "metadata.write_learning.infer_rule"):
|
||||
payload = {"learning_id": "form-command-binding-standard-customize-form"}
|
||||
req = urllib.request.Request("http://docker-gpu.cin.su:8011/rpc", data=json.dumps({"method": method, "payload": payload}, ensure_ascii=False).encode("utf-8"), headers={"Content-Type":"application/json"})
|
||||
print(urllib.request.urlopen(req, timeout=90).read().decode("utf-8"))
|
||||
'@ | python -
|
||||
```
|
||||
|
||||
## Experiment B: Local Form Command Binding
|
||||
|
||||
Goal: learn where marker `34` stores a local form command binding.
|
||||
|
||||
Use the same one-command runner above after the manual save.
|
||||
|
||||
Manual edit in Designer:
|
||||
|
||||
1. Open form `ТестНастройки`.
|
||||
2. Select button `ФормаКомандаОбновить`.
|
||||
3. Change only `CommandName` from `Form.Command.КомандаПрименить` to
|
||||
`Form.Command.КомандаПример1` or `Form.Command.КомандаПример2`.
|
||||
4. Save the form once.
|
||||
|
||||
After save, capture:
|
||||
|
||||
```powershell
|
||||
@'
|
||||
import json, urllib.request
|
||||
payload = {
|
||||
"base_id": "upo_test",
|
||||
"learning_id": "form-command-binding-local-apply-command",
|
||||
"table": "ConfigCASSave",
|
||||
"file_name": "f96a0c45-3eff-11f1-8287-005056b0d483__25c39fbf-35a4-4b43-8e3f-cd1f91082c88.0",
|
||||
"form": "ТестНастройки",
|
||||
"element": "ФормаКомандаОбновить",
|
||||
"property": "ИмяКоманды",
|
||||
"timeout_seconds": 60,
|
||||
"max_items": 5000
|
||||
}
|
||||
req = urllib.request.Request("http://docker-gpu.cin.su:8011/rpc", data=json.dumps({"method":"metadata.write_learning.capture_after","payload":payload}, ensure_ascii=False).encode("utf-8"), headers={"Content-Type":"application/json"})
|
||||
print(urllib.request.urlopen(req, timeout=90).read().decode("utf-8"))
|
||||
'@ | python -
|
||||
```
|
||||
|
||||
Then run:
|
||||
|
||||
```powershell
|
||||
@'
|
||||
import json, urllib.request
|
||||
for method in ("metadata.write_learning.diff", "metadata.write_learning.infer_rule"):
|
||||
payload = {"learning_id": "form-command-binding-local-apply-command"}
|
||||
req = urllib.request.Request("http://docker-gpu.cin.su:8011/rpc", data=json.dumps({"method": method, "payload": payload}, ensure_ascii=False).encode("utf-8"), headers={"Content-Type":"application/json"})
|
||||
print(urllib.request.urlopen(req, timeout=90).read().decode("utf-8"))
|
||||
'@ | python -
|
||||
```
|
||||
|
||||
## Promotion Gate
|
||||
|
||||
Promote a decoder/write rule only when the after capture changes exactly one
|
||||
intended command binding. If `metadata.write_learning.diff` reports
|
||||
`no_changes`, use low-level `payload.diff` against the baseline payload SHA1
|
||||
and implement a dedicated command-binding decoder from the scalar tree diff.
|
||||
@@ -0,0 +1,291 @@
|
||||
# 1C Form Discovery And Editing
|
||||
|
||||
This runbook adapts the MOXCEL discovery loop to managed forms. The goal is a
|
||||
full SQL-side form decoder and safe saved-state editing through the test
|
||||
extension, with XML exports used only as evidence fixtures.
|
||||
|
||||
## Current Baseline
|
||||
|
||||
- Default base: `upo_test`.
|
||||
- Default adapter endpoint: `http://docker-gpu.cin.su:8011`.
|
||||
- Primary test extension/object fixture:
|
||||
`фс_ДоработкиОбщее` /
|
||||
`DataProcessor.фс_НастройкаУсловногоОформления`.
|
||||
- Primary form fixture: `ТестНастройки`.
|
||||
- Existing form context evidence:
|
||||
`reports/1c-sql/upo/form-context-test-nastroiki-title-resolution.json`.
|
||||
- Existing saved-state smoke selectors:
|
||||
`А`, `ТЗК1`, and `КомандаПример1`.
|
||||
|
||||
If `ConfigSave` or `ConfigCASSave` is empty, prepare the working saved-state
|
||||
row through the reviewed saved-state copy flow in
|
||||
`docs/1c-write-path-safety.md` before running write smokes.
|
||||
|
||||
## Current UPO Test Status
|
||||
|
||||
- Active form discovery object:
|
||||
`Catalog.ЗадачиАссистентаУправления`.
|
||||
- Working saved-state table: `ConfigSave`.
|
||||
- Working form payloads:
|
||||
`fa447250-c2a0-439d-8ba7-422923f57200.0` (`ФормаЭлемента`) and
|
||||
`91ce61c5-6f4b-484a-9021-59f18be88550.0` (`ФормаСписка`).
|
||||
- The saved-state copy planner now includes form payload rows (`*.0`) with
|
||||
`role=form_payload`; descriptor-only copies are not enough for
|
||||
`metadata.form.decode`.
|
||||
- Latest decoder profile after container/dynamic-list baseline mapping:
|
||||
`reports/1c-sql/upo_test/form-profile-zadachi-assistenta-configsave-after-map-live.md`.
|
||||
Coverage is `484/2083` mapped, up from `252/2083`.
|
||||
- Latest SQL/XML oracle comparison for `ФормаСписка`:
|
||||
`reports/1c-sql/upo_test/form-sql-xml-compare-zadachi-assistenta-list-xmlmap.md`.
|
||||
It currently shows `17` matched items, `44` matched properties, `41`
|
||||
XML-only properties, and `12` mismatches after mapping SQL type code `8`
|
||||
to `Контекстное меню`.
|
||||
- `ТестНастройки` XML context is available at
|
||||
`reports/1c-sql/upo_test/form-context-test-nastroiki-effective-refresh.json`,
|
||||
and the live SQL saved-state rows are prepared in `ConfigCASSave` from local
|
||||
CAS blobs:
|
||||
`f96a0c45-3eff-11f1-8287-005056b0d483__25c39fbf-35a4-4b43-8e3f-cd1f91082c88`
|
||||
plus `.0`.
|
||||
- Latest `ТестНастройки` direct SQL decoder profile:
|
||||
`reports/1c-sql/upo_test/form-profile-test-nastroiki-direct.md`.
|
||||
Coverage is `552/2891` mapped after decoding `ExtendedTooltip` form
|
||||
items from marker `12`, table additions from marker `6`, buttons from
|
||||
marker `34`, form/element events, field layout details, group/table layout
|
||||
properties, derived child item references, and section-level command/action
|
||||
semantics.
|
||||
- Latest `ТестНастройки` SQL/XML oracle comparison:
|
||||
`reports/1c-sql/upo_test/form-sql-xml-compare-test-nastroiki-direct.md`.
|
||||
It currently shows `68` matched items, `292` matched properties, `2`
|
||||
XML-only properties, no missing SQL items, and no remaining value
|
||||
mismatches after `ExtendedTooltip` decoding, form event decoding from
|
||||
section `1.19`, marker `6`/`34` decoding, element event matching, derived
|
||||
reference semantics, field/group/table layout semantics, plus section-aware
|
||||
XML matching for dynamic-list columns and command-backed buttons. The only
|
||||
remaining XML-only properties are command binding cases:
|
||||
`ТЗИзменитьФорму -> Form.StandardCommand.CustomizeForm` and
|
||||
`ФормаКомандаОбновить -> Form.Command.КомандаПрименить`.
|
||||
- Latest `ТестНастройки` write matrix smoke:
|
||||
`reports/1c-sql/upo_test/form-write-matrix-smoke-test-nastroiki-layout-table-100.json`.
|
||||
It verified 100 candidates with zero failures. The current matrix has
|
||||
`2912` entries, `1339` safe smoke candidates, and no not-writable entries.
|
||||
A direct `metadata.write` smoke for `А.Заголовок` also completed as
|
||||
`verified_and_rolled_back`.
|
||||
- Latest write matrix smokes:
|
||||
`reports/1c-sql/upo_test/form-write-matrix-smoke-element-50.json` and
|
||||
`reports/1c-sql/upo_test/form-write-matrix-smoke-list-50.json`.
|
||||
Both verified 50 candidates with zero failures.
|
||||
- Latest direct `metadata.write` smoke changed `Список.Заголовок` through
|
||||
`apply_and_rollback`; semantic readback verified the change and rollback
|
||||
restored the original SHA1.
|
||||
|
||||
## Discovery Loop
|
||||
|
||||
Use the same shape as the MOXCEL work:
|
||||
|
||||
1. Create or update one controlled form fixture in the test extension.
|
||||
2. Change exactly one visible form property in Designer.
|
||||
3. Capture the SQL saved-state form payload before and after.
|
||||
4. Compare the decoded SQL payload with exported `Ext/Form.xml` as an oracle.
|
||||
5. Promote read rules only when SQL bytes reproduce XML-visible facts.
|
||||
6. Promote write rules only after `apply_and_rollback` proves semantic readback
|
||||
and rollback.
|
||||
|
||||
The adapter runtime remains SQL-only. Exported form XML is a labeling and
|
||||
verification fixture, not a runtime input for adapter answers.
|
||||
|
||||
## Decoder Scope
|
||||
|
||||
The full decoder should expose these public form sections:
|
||||
|
||||
- form common properties and events;
|
||||
- form items with stable `id`, `name`, parent/group, type, title, data path,
|
||||
visibility, enabled/read-only flags, layout properties, and color/font
|
||||
properties when decoded;
|
||||
- form attributes, including value-table fields and dynamic-list fields;
|
||||
- form commands, command bars, command-button bindings, and command handlers;
|
||||
- table columns, pages, groups, decorations, input fields, labels, buttons,
|
||||
extended tooltips, context menus, and dynamic lists;
|
||||
- form module summary and routine/event/command link validation;
|
||||
- source-aware display resolution for inherited captions:
|
||||
command title, form attribute title, value-table field title, and local
|
||||
form item override.
|
||||
|
||||
Every decoded scalar must carry enough evidence for future writes:
|
||||
section, element identity, physical payload path, semantic group/name, current
|
||||
value, value type, source, and verification rule.
|
||||
|
||||
## Test Extension Fixture Plan
|
||||
|
||||
Keep fixtures small and intentionally boring. Add form elements in
|
||||
`ТестНастройки` or a sibling test form so each save isolates one concept:
|
||||
|
||||
- command button bound to `КомандаПример1`;
|
||||
- local-only button title;
|
||||
- element title inherited from a form command;
|
||||
- element title inherited from a form attribute `А`;
|
||||
- element title inherited from a value-table field `ТЗ.К1`;
|
||||
- input field with visibility, availability, read-only, title location, choice
|
||||
buttons, quick choice, and text editing flags;
|
||||
- group/page/table layout properties: parent group, order, stretch, width,
|
||||
height, command-bar location, and default item;
|
||||
- color/font properties for label/button/input field;
|
||||
- dynamic list with main table, custom query flag, query text, and columns;
|
||||
- form events and element events with matching and missing BSL handlers;
|
||||
- structural move/reorder cases inside one parent container.
|
||||
|
||||
Prefer one-property saves. Do not combine property, handler, and structural
|
||||
changes in the same learning capture.
|
||||
|
||||
## Read-Side Commands
|
||||
|
||||
Decode a concrete saved-state form payload:
|
||||
|
||||
```powershell
|
||||
python scripts/smoke_1c_write_matrix.py `
|
||||
--base-url http://docker-gpu.cin.su:8011 `
|
||||
--base-id upo_test `
|
||||
--table ConfigCASSave `
|
||||
--file-name <form-file-name> `
|
||||
--build-only `
|
||||
--report reports/1c-sql/upo_test/form-write-matrix-build.json
|
||||
```
|
||||
|
||||
Read XML-backed form context for the test fixture:
|
||||
|
||||
```powershell
|
||||
python scripts/get_1c_form_context.py `
|
||||
--index reports/1c-sql/upo/unified-object-route-index.json `
|
||||
--kind DataProcessor `
|
||||
--name фс_НастройкаУсловногоОформления `
|
||||
--form ТестНастройки `
|
||||
--view effective `
|
||||
--max-items 500 `
|
||||
--output reports/1c-sql/upo_test/form-context-test-nastroiki-effective.json
|
||||
```
|
||||
|
||||
Adapter RPC equivalents:
|
||||
|
||||
```json
|
||||
{"method":"metadata.form.decode","payload":{"base_id":"upo_test","table":"ConfigCASSave","file_name":"<form-file-name>","include_parameters":true,"max_items":5000,"max_parameters":500}}
|
||||
```
|
||||
|
||||
```json
|
||||
{"method":"metadata.form.write_matrix.build","payload":{"base_id":"upo_test","table":"ConfigCASSave","file_name":"<form-file-name>"}}
|
||||
```
|
||||
|
||||
Build a decoder coverage and gap profile for an object form:
|
||||
|
||||
```powershell
|
||||
python scripts/profile_1c_forms.py `
|
||||
--adapter-url http://docker-gpu.cin.su:8011 `
|
||||
--base-id upo_test `
|
||||
--kind Catalog `
|
||||
--name ЗадачиАссистентаУправления `
|
||||
--table Config `
|
||||
--raw-output-json reports/1c-sql/upo_test/form-profile-zadachi-assistenta-details.json `
|
||||
--output-json reports/1c-sql/upo_test/form-profile-zadachi-assistenta.json `
|
||||
--output-markdown reports/1c-sql/upo_test/form-profile-zadachi-assistenta.md
|
||||
```
|
||||
|
||||
Compare decoded SQL form semantics with exported `Form.xml` semantics:
|
||||
|
||||
```powershell
|
||||
python scripts/compare_1c_form_sql_xml.py `
|
||||
--sql-details reports/1c-sql/upo_test/form-profile-zadachi-assistenta-configsave-xmlmap-details.json `
|
||||
--xml-context reports/1c-sql/upo_test/form-context-zadachi-assistenta-list-effective.json `
|
||||
--output-json reports/1c-sql/upo_test/form-sql-xml-compare-zadachi-assistenta-list-xmlmap.json `
|
||||
--output-markdown reports/1c-sql/upo_test/form-sql-xml-compare-zadachi-assistenta-list-xmlmap.md
|
||||
```
|
||||
|
||||
## Write Learning
|
||||
|
||||
For a manual one-property Designer change:
|
||||
|
||||
```json
|
||||
{"method":"metadata.write_learning.capture_before","payload":{"base_id":"upo_test","learning_id":"form-visible-case","table":"ConfigCASSave","form":"ТестНастройки","element":"<element-name>"}}
|
||||
```
|
||||
|
||||
After the Designer save:
|
||||
|
||||
```json
|
||||
{"method":"metadata.write_learning.capture_after","payload":{"base_id":"upo_test","learning_id":"form-visible-case","table":"ConfigCASSave","form":"ТестНастройки","element":"<element-name>"}}
|
||||
{"method":"metadata.write_learning.diff","payload":{"learning_id":"form-visible-case"}}
|
||||
{"method":"metadata.write_learning.infer_rule","payload":{"learning_id":"form-visible-case"}}
|
||||
```
|
||||
|
||||
Promote a rule only when the diff changes exactly one intended semantic value
|
||||
or one intended structural relation. Composite/list rewrites need a dedicated
|
||||
source-specific rule, not a generic scalar writer.
|
||||
|
||||
## Write Verification
|
||||
|
||||
Run the existing source-aware route smoke:
|
||||
|
||||
```powershell
|
||||
python scripts/smoke_1c_saved_state_write_routes.py `
|
||||
--base-url http://docker-gpu.cin.su:8011 `
|
||||
--base-id upo_test `
|
||||
--table ConfigCASSave `
|
||||
--file-name <form-file-name> `
|
||||
--report reports/1c-sql/upo_test/saved-state-write-routes-smoke.json
|
||||
```
|
||||
|
||||
Then run the matrix smoke:
|
||||
|
||||
```powershell
|
||||
python scripts/smoke_1c_write_matrix.py `
|
||||
--base-url http://docker-gpu.cin.su:8011 `
|
||||
--base-id upo_test `
|
||||
--table ConfigCASSave `
|
||||
--file-name <form-file-name> `
|
||||
--max-candidates 50 `
|
||||
--learning-id upo-test-form-write-matrix `
|
||||
--report reports/1c-sql/upo_test/form-write-matrix-smoke-50.json
|
||||
```
|
||||
|
||||
Successful writes must use `apply_and_rollback`, explicit SQL apply and
|
||||
rollback gates, sha1 preconditions, backup evidence, semantic readback through
|
||||
`metadata.form.decode`, and rollback verification.
|
||||
|
||||
## Promotion Gates
|
||||
|
||||
Read rule promotion requires:
|
||||
|
||||
- SQL-only decoder output with stable semantic name and value type;
|
||||
- XML fixture agreement for the same form element/property;
|
||||
- no dependency on display strings when a stable id/path exists;
|
||||
- regression coverage on the test form and at least one real extension form.
|
||||
|
||||
Write rule promotion requires:
|
||||
|
||||
- exact physical payload path or structural span evidence;
|
||||
- source-aware routing for inherited display values;
|
||||
- `metadata.form.write_target.resolve` success with a deterministic target;
|
||||
- `metadata.form.element.write_apply` or `metadata.write` success in
|
||||
`apply_and_rollback`;
|
||||
- semantic verification and rollback readback success;
|
||||
- registration in the scalar/enum/verified write matrix reports.
|
||||
|
||||
## Immediate Work Queue
|
||||
|
||||
1. Refresh the test form saved-state row for `upo_test` if
|
||||
`ConfigCASSave`/`ConfigSave` is empty.
|
||||
2. Capture a fresh `metadata.form.decode` baseline for `ТестНастройки`.
|
||||
3. Build a form property gap report: decoded SQL semantics versus `Ext/Form.xml`
|
||||
semantics from the test extension.
|
||||
4. Learn the two remaining command binding cases with a one-property
|
||||
before/after capture: standard command button binding and local button to
|
||||
form command binding. Do not hard-code these from display names. Current
|
||||
captures and exact after-capture commands are in
|
||||
`docs/runbooks/1c-form-command-binding-learning.md`.
|
||||
5. Expand `parser/form_payload.py` for the next write-relevant properties:
|
||||
availability, read-only, title location, command-bar location, colors, font,
|
||||
and dynamic-list query settings.
|
||||
5. Rebuild the write matrix and split entries into verified scalar, enum,
|
||||
composite-needs-rule, identity/binding, and structural queues.
|
||||
6. Learn one property at a time through
|
||||
`metadata.write_learning.capture_before/capture_after/diff/infer_rule`.
|
||||
7. Promote safe scalar/enum routes into smoke coverage.
|
||||
8. Add a structural movement scorecard for sibling reorder and parent/group
|
||||
movement, then extend `metadata.form.target.move` beyond sibling swaps only
|
||||
after controlled round-trip proof.
|
||||
@@ -0,0 +1,55 @@
|
||||
# 1C Live Interaction
|
||||
|
||||
Цель: безопасно связать модель с живыми базами 1С.
|
||||
|
||||
## Layers
|
||||
|
||||
1. Connector API: `plugins/1c/connector/contracts/openapi.yaml`
|
||||
2. Metadata snapshots: `plugins/1c/schemas/metadata-snapshot-v2.schema.json`
|
||||
3. BSL module snapshots: `plugins/1c/schemas/bsl-module-snapshot.schema.json`
|
||||
4. Read-only query policy: `plugins/1c/connector/policies/read-only-query.yaml`
|
||||
5. Change workflow policy: `plugins/1c/connector/policies/change-workflow.yaml`
|
||||
|
||||
For day-to-day development, use the faster operational loop instead of full XML/EDT sync on every task:
|
||||
|
||||
- run read-only SQL for diagnostics and data samples;
|
||||
- get metadata and BSL through a lightweight 1C agent or exported JSON snapshot;
|
||||
- refresh cached snapshots by configuration version/checksum;
|
||||
- generate external reports, data processors, extensions, or reviewable patches.
|
||||
|
||||
Details: `docs/runbooks/1c-operational-coding.md`.
|
||||
|
||||
## Query Validation
|
||||
|
||||
```powershell
|
||||
python scripts/validate_1c_readonly_query.py --query "ВЫБРАТЬ Первые 10 Ссылка ИЗ Справочник.Номенклатура"
|
||||
```
|
||||
|
||||
Denied example:
|
||||
|
||||
```powershell
|
||||
python scripts/validate_1c_readonly_query.py --query "УДАЛИТЬ ИЗ Справочник.Номенклатура"
|
||||
```
|
||||
|
||||
## BSL Module Snapshot
|
||||
|
||||
```powershell
|
||||
python scripts/validate_1c_bsl_modules.py plugins/1c/metadata/examples/bsl-modules.example.json
|
||||
python scripts/convert_1c_bsl_modules_to_rag.py --input plugins/1c/metadata/examples/bsl-modules.example.json --output plugins/1c/rag/sources/bsl-modules.generated.md
|
||||
```
|
||||
|
||||
## Safety Rule
|
||||
|
||||
The model may:
|
||||
|
||||
- inspect metadata;
|
||||
- search/read modules;
|
||||
- validate read-only queries;
|
||||
- propose changes.
|
||||
|
||||
The model must not:
|
||||
|
||||
- directly change a live database;
|
||||
- run destructive queries;
|
||||
- reveal secrets or personal data;
|
||||
- invent metadata when connector data is missing.
|
||||
@@ -0,0 +1,120 @@
|
||||
# 1C LoRA Training
|
||||
|
||||
Цель: обучить draft LoRA adapter `qwen3-coder-30b-a3b-1c-lora-v1` поверх `qwen3-coder-30b-a3b-instruct`.
|
||||
|
||||
## Preconditions
|
||||
|
||||
- Полностью скачана базовая модель: `/models/base/qwen3-coder-30b-a3b-instruct`.
|
||||
- Подготовлен датасет: `plugins/1c/training/prepared/train.chat.jsonl`.
|
||||
- Есть GPU/CUDA на `docker-gpu.cin.su`.
|
||||
- В датасете достаточно проверенных примеров. Синтетические 2 записи подходят только для smoke-run, не для полезного качества.
|
||||
|
||||
## Current Preflight Status
|
||||
|
||||
На 2026-07-04 локальный preflight для нового Qwen3-Coder training contour не стартует, потому что:
|
||||
|
||||
- локально нет CUDA/GPU;
|
||||
- training-зависимости не установлены в локальный Python;
|
||||
- HF-база `qwen3-coder-30b-a3b-instruct` еще не лежит в `/models/base/qwen3-coder-30b-a3b-instruct` на текущем workspace path;
|
||||
- запускать обучение нужно на `docker-gpu.cin.su`, потому что именно там есть GPU-контур для этой модели.
|
||||
|
||||
Если файл будет удален или поврежден, восстановить/докачать базовую модель можно так:
|
||||
|
||||
```powershell
|
||||
python scripts/download_hf_range.py qwen3-coder-30b-a3b-instruct `
|
||||
--local-dir models/base/qwen3-coder-30b-a3b-instruct `
|
||||
--chunk-size 16mb `
|
||||
--retries 20
|
||||
```
|
||||
|
||||
## Prepare Dataset
|
||||
|
||||
```powershell
|
||||
python scripts/validate_1c_training_data.py plugins/1c/training/examples/instruction.examples.jsonl
|
||||
python scripts/prepare_1c_training_data.py --input plugins/1c/training/examples/instruction.examples.jsonl --output plugins/1c/training/prepared/train.chat.jsonl
|
||||
```
|
||||
|
||||
## Local Preflight
|
||||
|
||||
```powershell
|
||||
python scripts/preflight_1c_training.py
|
||||
```
|
||||
|
||||
## Dry Run
|
||||
|
||||
```powershell
|
||||
python scripts/train_1c_lora.py --dry-run
|
||||
```
|
||||
|
||||
## GPU Docker Run
|
||||
|
||||
```powershell
|
||||
docker --host ssh://docker-gpu.cin.su compose --env-file core/deploy/docker-gpu/training/1c-lora.env.example -f core/deploy/docker-gpu/training/1c-lora.compose.yaml up --abort-on-container-exit
|
||||
```
|
||||
|
||||
Или через готовый wrapper:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/run_1c_lora_training_gpu.ps1
|
||||
```
|
||||
|
||||
Full end-to-end orchestration for the current `Q6` route:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/train_and_publish_q6_lora.ps1
|
||||
```
|
||||
|
||||
Preview the whole flow without executing:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/train_and_publish_q6_lora.ps1 -PlanOnly
|
||||
```
|
||||
|
||||
Troubleshooting:
|
||||
|
||||
```text
|
||||
docs/runbooks/q6-lora-troubleshooting.md
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
Adapter path:
|
||||
|
||||
```text
|
||||
/models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1
|
||||
```
|
||||
|
||||
After a successful training run:
|
||||
|
||||
1. Run `plugins/1c/evals/smoke.yaml`.
|
||||
2. Convert the adapter for `llama.cpp` GGUF format or merge it before rebuilding the Q6 GGUF deployment artifact.
|
||||
|
||||
GGUF adapter export on `docker-gpu`:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/convert_1c_lora_to_gguf_gpu.ps1
|
||||
```
|
||||
|
||||
The default output path is:
|
||||
|
||||
```text
|
||||
/models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1.gguf
|
||||
```
|
||||
|
||||
If the export container should reuse an existing `llama.cpp` checkout on the host without pulling updates:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/convert_1c_lora_to_gguf_gpu.ps1 -SkipClone
|
||||
```
|
||||
|
||||
3. To launch the current GPU Q6 route with a converted adapter, use:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/manage_gpu_q6_service.ps1 `
|
||||
-Action start `
|
||||
-LoraPath /models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1.gguf
|
||||
```
|
||||
|
||||
If you need a custom adapter scale, pass `-LoraScale 0.5` or another value.
|
||||
4. Compare base, RAG, and adapter outputs.
|
||||
5. Promote model card from `draft` only after expert review.
|
||||
@@ -0,0 +1,50 @@
|
||||
# 1C Metadata Snapshot
|
||||
|
||||
Цель: сохранять структуру 1С как локальный snapshot и использовать ее в RAG без выдумывания объектов конфигурации.
|
||||
|
||||
## Files
|
||||
|
||||
- Schema: `plugins/1c/metadata/schema.json`
|
||||
- Example: `plugins/1c/metadata/examples/metadata.example.json`
|
||||
- Local snapshots: `plugins/1c/metadata/snapshots`
|
||||
- Converter: `scripts/convert_1c_metadata_to_rag.py`
|
||||
|
||||
## Snapshot Rule
|
||||
|
||||
Рабочие snapshot-файлы считаются локальными артефактами. Они не должны содержать:
|
||||
|
||||
- пароли;
|
||||
- токены;
|
||||
- строки подключения;
|
||||
- персональные данные;
|
||||
- клиентские секреты;
|
||||
- выгрузки данных.
|
||||
|
||||
Snapshot описывает только структуру метаданных.
|
||||
|
||||
## Convert Example To RAG Source
|
||||
|
||||
```powershell
|
||||
python scripts/validate_1c_metadata_snapshot.py plugins/1c/metadata/examples/metadata.example.json
|
||||
```
|
||||
|
||||
```powershell
|
||||
python scripts/convert_1c_metadata_to_rag.py --input plugins/1c/metadata/examples/metadata.example.json --output plugins/1c/rag/sources/metadata.example.generated.md
|
||||
```
|
||||
|
||||
После этого можно перестроить корпус и индекс:
|
||||
|
||||
```powershell
|
||||
python scripts/prepare_1c_rag_corpus.py
|
||||
python scripts/build_1c_rag_index.py
|
||||
```
|
||||
|
||||
Команды нужно выполнять последовательно: сначала подготовить JSONL-корпус, затем строить индекс.
|
||||
|
||||
## Ask
|
||||
|
||||
```powershell
|
||||
python scripts/ask_1c_rag.py "Какие реквизиты есть у справочника Номенклатура?" --print-prompt
|
||||
```
|
||||
|
||||
В prompt должны попасть реквизиты из snapshot и ссылка на source `metadata.example.generated.md`.
|
||||
@@ -0,0 +1,337 @@
|
||||
# 1C MOXCEL Discovery
|
||||
|
||||
This runbook describes the read-only discovery loop for tabular document
|
||||
MOXCEL payloads.
|
||||
|
||||
## Current Artifacts
|
||||
|
||||
- `reports/1c-template-baselines/Primer3_moxel_schema_discovery.json`
|
||||
contains inferred decoder rules from marker matrices and history diffs.
|
||||
- `reports/1c-template-baselines/Primer3_moxel_property_experiments.json`
|
||||
contains one-property experiment analysis and the next probe plan.
|
||||
- `reports/1c-template-baselines/Primer3_moxel_property_experiments.manifest.json`
|
||||
maps existing before/after summaries to property experiment labels.
|
||||
|
||||
## Schema Discovery
|
||||
|
||||
Run marker/history discovery:
|
||||
|
||||
```powershell
|
||||
python scripts/discover_1c_moxel_schema.py `
|
||||
--marker-matrix reports/1c-template-baselines/Primer3_marker_matrix_2026-06-27_latest.json `
|
||||
--history-matrix reports/1c-template-baselines/Primer3_history_matrix.json `
|
||||
--property-candidates reports/1c-template-baselines/Primer3_property_candidates.json `
|
||||
--output-json reports/1c-template-baselines/Primer3_moxel_schema_discovery.json `
|
||||
--output-markdown reports/1c-template-baselines/Primer3_moxel_schema_discovery.md
|
||||
```
|
||||
|
||||
The current strongest rule is:
|
||||
|
||||
```text
|
||||
moxel.inline_text_cell.column:
|
||||
one_based_column = int(last_numeric(preceding_scalars)) + 1
|
||||
```
|
||||
|
||||
Named range coordinates are partially proven from controlled moves:
|
||||
|
||||
```text
|
||||
left/right: raw scalar indexes 2 and 4
|
||||
top/bottom: raw scalar indexes 3 and 5
|
||||
one_based = raw + 1
|
||||
```
|
||||
|
||||
Use multi-cell and rectangular named ranges to split `left` from `right` and
|
||||
`top` from `bottom`.
|
||||
|
||||
Build the stable registry after discovery:
|
||||
|
||||
```powershell
|
||||
python scripts/build_1c_moxel_schema_registry.py `
|
||||
--discovery reports/1c-template-probes/upo_test_auto_moxel_schema_discovery.json `
|
||||
--discovery reports/1c-template-baselines/Primer3_moxel_schema_discovery.json `
|
||||
--output-json plugins/1c/metadata/moxel-schema-registry.json `
|
||||
--output-markdown reports/1c-template-baselines/moxel-schema-registry.md
|
||||
```
|
||||
|
||||
Validate the registry safety contract:
|
||||
|
||||
```powershell
|
||||
python scripts/check_1c_moxel_schema_registry.py `
|
||||
--registry plugins/1c/metadata/moxel-schema-registry.json `
|
||||
--output reports/1c-template-baselines/moxel-schema-registry-check.json
|
||||
```
|
||||
|
||||
The registry allows read-side decoder rules only. Write-side MOXCEL mutation is
|
||||
kept blocked until a disposable-base round-trip proves the exact scalar path.
|
||||
|
||||
Verify registry rules against concrete probe snapshots:
|
||||
|
||||
```powershell
|
||||
python scripts/verify_1c_moxel_schema_registry.py `
|
||||
--registry plugins/1c/metadata/moxel-schema-registry.json `
|
||||
--probe reports/1c-template-probes/upo_test_auto_20260627T142419Z_670780b4.json `
|
||||
--output reports/1c-template-baselines/moxel-schema-registry-verification.json
|
||||
```
|
||||
|
||||
This check is data-backed: `verified_read` rules must pass on the supplied
|
||||
probe snapshots, while `candidate_read` rules are reported as diagnostic
|
||||
evidence and are not promoted automatically.
|
||||
|
||||
## Full Pipeline
|
||||
|
||||
After a new probe or one-property experiment has been captured, refresh all
|
||||
MOXCEL discovery artifacts with one command:
|
||||
|
||||
```powershell
|
||||
python scripts/run_1c_moxel_discovery_pipeline.py
|
||||
```
|
||||
|
||||
To capture the latest live MOXCEL probe first and then refresh all artifacts:
|
||||
|
||||
```powershell
|
||||
python scripts/run_1c_moxel_discovery_pipeline.py --capture-live --label pipeline-live
|
||||
```
|
||||
|
||||
The pipeline runs schema discovery, named-range rule analysis, property
|
||||
experiment analysis, registry build, registry safety check, registry
|
||||
verification, and the next experiment plan. It writes:
|
||||
|
||||
- `reports/1c-template-baselines/moxel-discovery-pipeline.json`
|
||||
- `reports/1c-template-baselines/moxel-discovery-pipeline.md`
|
||||
- `reports/1c-template-probes/latest-live-probe.json` when `--capture-live` is used
|
||||
- `reports/1c-template-baselines/moxel-named-range-rules.md`
|
||||
- `reports/1c-template-baselines/moxel-next-experiments.md`
|
||||
- `reports/1c-template-baselines/moxel-next-action.json`
|
||||
- `reports/1c-template-baselines/moxel-next-action.md`
|
||||
- `reports/1c-template-baselines/moxel-next-action-check.json`
|
||||
- `reports/1c-template-baselines/moxel-status.md`
|
||||
|
||||
When `moxel-named-range-rules.json` contains high-confidence rectangular range
|
||||
evidence, the registry build step automatically promotes the corresponding
|
||||
named-range read rules from `candidate_read` to `verified_read`. Write status
|
||||
still remains blocked until a separate round-trip proof exists.
|
||||
|
||||
## Property Experiments
|
||||
|
||||
Generate or refresh the property probe plan:
|
||||
|
||||
```powershell
|
||||
python scripts/analyze_1c_moxel_property_experiments.py `
|
||||
--emit-default-plan `
|
||||
--output-json reports/1c-template-baselines/Primer3_moxel_property_experiments.json `
|
||||
--output-markdown reports/1c-template-baselines/Primer3_moxel_property_experiments.md
|
||||
```
|
||||
|
||||
Analyze existing before/after experiments:
|
||||
|
||||
```powershell
|
||||
python scripts/analyze_1c_moxel_property_experiments.py `
|
||||
--manifest reports/1c-template-baselines/Primer3_moxel_property_experiments.manifest.json `
|
||||
--emit-default-plan `
|
||||
--output-json reports/1c-template-baselines/Primer3_moxel_property_experiments.json `
|
||||
--output-markdown reports/1c-template-baselines/Primer3_moxel_property_experiments.md
|
||||
```
|
||||
|
||||
Each new experiment should change exactly one property on the same tracked cell
|
||||
or range, then capture a fresh `templates.map`/summary snapshot.
|
||||
|
||||
To capture the next manual one-property save automatically, start the watcher
|
||||
before changing and saving the template in 1C:
|
||||
|
||||
```powershell
|
||||
python scripts/watch_1c_moxel_property_experiment.py `
|
||||
--property ВертикальноеПоложение `
|
||||
--target-text "Ячейка 7 - 2" `
|
||||
--target-name R7C2_TEST `
|
||||
--timeout-seconds 600 `
|
||||
--run-pipeline-after
|
||||
```
|
||||
|
||||
The watcher captures a `before` snapshot, waits for a new latest MOXCEL payload,
|
||||
captures `after`, writes Markdown/JSON snapshots to
|
||||
`reports/1c-template-probes`, and appends the experiment to
|
||||
`reports/1c-template-baselines/Primer3_moxel_property_experiments.manifest.json`.
|
||||
With `--run-pipeline-after`, it also refreshes schema discovery, property
|
||||
analysis, registry build/check/verify, and the next experiment plan.
|
||||
|
||||
Prioritize:
|
||||
|
||||
- `ГоризонтальноеПоложение`
|
||||
- `ВертикальноеПоложение`
|
||||
- `ЦветТекста`
|
||||
- `ЦветФона`
|
||||
- `Шрифт.Имя`
|
||||
- `Шрифт.Размер`
|
||||
- borders
|
||||
- `Защита`
|
||||
- `Гиперссылка`
|
||||
- wrapping
|
||||
- column width
|
||||
- row height
|
||||
- merge ranges
|
||||
|
||||
Only promote a property path into write support after a disposable-base
|
||||
round-trip proves that changing that scalar affects only the intended property.
|
||||
|
||||
## XML-Assisted Fixture Plan
|
||||
|
||||
Use XML exports only as analysis fixtures. The adapter runtime stays SQL-only:
|
||||
it reads `Config`/`ConfigCAS` payloads, while exported `Ext/Template.xml`
|
||||
files are used to label and verify decoder hypotheses.
|
||||
|
||||
Create a small extension with controlled templates and export it to XML after
|
||||
each controlled save:
|
||||
|
||||
- one tabular document with sparse text cells, direct parameters, placeholders,
|
||||
and empty formatted cells;
|
||||
- one tabular document with horizontal, vertical, and rectangular merged cells;
|
||||
- one tabular document with named areas and named ranges, including duplicate
|
||||
names in different positions;
|
||||
- one tabular document dedicated to format changes: column width, row height,
|
||||
horizontal/vertical alignment, border, font, text color, background color,
|
||||
protection, wrapping;
|
||||
- one fixture per non-tabular template type where possible: text document,
|
||||
binary data, HTML document, graphical/geographical schema, data composition
|
||||
schema, data composition appearance template, external component.
|
||||
|
||||
The concrete merge fixture checklist is stored in:
|
||||
|
||||
- `reports/1c-template-baselines/moxel-controlled-merge-fixtures.json`
|
||||
- `reports/1c-template-baselines/moxel-controlled-merge-fixtures.md`
|
||||
|
||||
Start with the merge fixtures `MOXEL_Merge_None_Grid`,
|
||||
`MOXEL_Merge_H_R5C18_W3`, `MOXEL_Merge_H_R6C2_W15`,
|
||||
`MOXEL_Merge_V_R5C2_H3`, `MOXEL_Merge_Rect_R5C2_R7C4`, and
|
||||
`MOXEL_Merge_Mixed_4Ranges`. They are designed to split column edges, row
|
||||
edges, width/height, and merge-record ordering without relying on the large
|
||||
production print forms.
|
||||
|
||||
For MOXCEL discovery, change exactly one property per save, capture the SQL
|
||||
payload, then compare it with the exported XML shape. Promote read rules only
|
||||
when the SQL decoder can reproduce XML-visible facts from SQL bytes alone.
|
||||
|
||||
Use `scripts/analyze_1c_template_xml_profiles.py` to build XML fixture
|
||||
profiles without feeding XML into the adapter runtime. Example:
|
||||
|
||||
```powershell
|
||||
python scripts\analyze_1c_template_xml_profiles.py `
|
||||
--root "Z:\codex\1C\XML\UPO\Структура базы 1с\Конфигурация\Documents\АвансовыйОтчет" `
|
||||
--output-json reports\1c-template-baselines\xml-template-profiles-avansovy-otchet.json `
|
||||
--output-markdown reports\1c-template-baselines\xml-template-profiles-avansovy-otchet.md
|
||||
```
|
||||
|
||||
Current `АвансовыйОтчет` XML fixture facts:
|
||||
|
||||
- `ПФ_MXL_АвансовыйОтчет`: capacity/used `75x26`, `961` cells,
|
||||
`196` text values, `77` parameters, `108` merges, `233` distinct format
|
||||
indexes.
|
||||
- `ПФ_MXL_АвансовыйОтчетВВалюте`: capacity/used `72x26`, `938` cells,
|
||||
`188` text values, `75` parameters, `101` merges, `233` distinct format
|
||||
indexes.
|
||||
|
||||
The SQL decoder should eventually reproduce these facts from SQL payloads:
|
||||
`capacity_dimensions`/`used_dimensions`, coordinate-bound cell text and
|
||||
parameters, authoritative `merged_ranges`, and format indexes/styles. XML
|
||||
profiles are evidence for decoder hypotheses, not an input source for adapter
|
||||
answers.
|
||||
|
||||
Compare the SQL-decoded baseline with the XML fixture profile after decoder
|
||||
changes:
|
||||
|
||||
```powershell
|
||||
python scripts\profile_1c_tabular_templates.py `
|
||||
--adapter-url http://docker-gpu.cin.su:8011 `
|
||||
--base-id upo_test `
|
||||
--inventory-json reports\1c-template-baselines\upo_test_configuration_tabular_templates.json `
|
||||
--output-json reports\1c-template-baselines\upo_test_tabular_template_profiles.json `
|
||||
--output-markdown reports\1c-template-baselines\upo_test_tabular_template_profiles.md
|
||||
```
|
||||
|
||||
```powershell
|
||||
python scripts\compare_1c_template_sql_xml_profiles.py `
|
||||
--sql-profile reports\1c-template-baselines\upo_test_tabular_template_profiles.json `
|
||||
--xml-profile reports\1c-template-baselines\xml-template-profiles-avansovy-otchet.json `
|
||||
--output-json reports\1c-template-baselines\sql-xml-template-profile-compare-avansovy-otchet.json `
|
||||
--output-markdown reports\1c-template-baselines\sql-xml-template-profile-compare-avansovy-otchet.md
|
||||
```
|
||||
|
||||
The current comparison intentionally reports gaps for both `АвансовыйОтчет`
|
||||
templates. SQL capacity still reports the MOXCEL allocation `128x72`, while XML
|
||||
spreadsheet dimensions are `75x26`/`72x26`. With `template_part_moxel_v8`,
|
||||
hint-aware SQL `used_dimensions` improved to `74x25` and `71x25`; the remaining
|
||||
edge likely depends on merge/format records. Limited cell/parameter counts,
|
||||
missing authoritative `merged_ranges`, and missing format index coverage remain
|
||||
open. Use these gaps as the next decoder scorecard; a gap should only disappear
|
||||
when SQL bytes alone reproduce the XML-visible fact.
|
||||
The compare report also shows `Progress signals`. A non-zero
|
||||
`cell_coordinate_hints_available` signal means the SQL decoder recovered
|
||||
coordinate evidence from hints, but the `cells_missing_or_limited` gap stays
|
||||
open until authoritative cell rows/columns are decoded.
|
||||
|
||||
For merge-block row/size reverse engineering, regenerate the row-band report:
|
||||
|
||||
```powershell
|
||||
python scripts\analyze_1c_moxel_merge_row_bands.py `
|
||||
--template ПФ_MXL_АвансовыйОтчет `
|
||||
--template ПФ_MXL_АвансовыйОтчетВВалюте `
|
||||
--output-json reports\1c-template-baselines\moxel-merge-row-band-analysis-avansovy-otchet.json `
|
||||
--output-markdown reports\1c-template-baselines\moxel-merge-row-band-analysis-avansovy-otchet.md
|
||||
```
|
||||
|
||||
This report compares XML merge rows with SQL MOXCEL small-scalar bands and
|
||||
packed `scalar/32` column-edge evidence. Treat `row_or_size_hints` as low
|
||||
confidence until a controlled merge fixture proves which scalar positions are
|
||||
row indexes versus widths, heights, or flags.
|
||||
|
||||
`merge_record_block_candidates[].evidence.record_analysis` exposes a normalized
|
||||
SQL-only view of the candidate block: per-record shape/head, packed
|
||||
`scalar/32` values, small scalars, scalar slot summaries, value-to-record runs,
|
||||
and sample records. Use it for slot-formula discovery; it is diagnostic
|
||||
evidence and does not make `merged_ranges` authoritative by itself.
|
||||
|
||||
To score candidate numeric slots against XML-visible merge fields, run:
|
||||
|
||||
```powershell
|
||||
python scripts\analyze_1c_moxel_merge_slot_candidates.py `
|
||||
--template ПФ_MXL_АвансовыйОтчет `
|
||||
--template ПФ_MXL_АвансовыйОтчетВВалюте `
|
||||
--output-json reports\1c-template-baselines\moxel-merge-slot-candidates-avansovy-otchet.json `
|
||||
--output-markdown reports\1c-template-baselines\moxel-merge-slot-candidates-avansovy-otchet.md
|
||||
```
|
||||
|
||||
The current large-form slot report is intentionally hypothesis-only. It shows
|
||||
that simple value-set and ordered-offset matching are dominated by low-entropy
|
||||
height/flag values and do not prove a top/left/bottom/right formula. Use the
|
||||
controlled merge fixtures before promoting any SQL rule into authoritative
|
||||
`merged_ranges`.
|
||||
|
||||
Inline text style candidates expose `coordinate_hints` for the column using the
|
||||
discovered rule
|
||||
`inline_text_column_from_last_preceding_scalar_plus_one`. Treat this as a
|
||||
high-confidence hint for analysis and controlled experiments, not as an
|
||||
authoritative cell coordinate until row and merge rules are proven by SQL/XML
|
||||
round trips.
|
||||
|
||||
The adapter and SQL profile reports include `cell_style_coordinate_hints` in
|
||||
counts. Track this count separately from `cells`: it measures how much
|
||||
coordinate evidence was recovered from inline style records, while `cells`
|
||||
remains reserved for decoded row/cell runs with authoritative coordinates.
|
||||
|
||||
`cell_coordinate_hints` combines the style-derived column hint with a matched
|
||||
decoded cell row when the text/cell id can be linked. Use it to inspect
|
||||
text-to-coordinate evidence during decoder discovery. Do not count it as
|
||||
authoritative `cells` coverage until the row rule and merge interactions are
|
||||
validated against SQL/XML fixtures.
|
||||
|
||||
For compact inspection, request only coordinate evidence from `templates.map`
|
||||
or `templates.analyze` with `sections=coordinate_hints`. This returns
|
||||
`cell_coordinate_hints` and `cell_style_coordinate_hints` from both the
|
||||
structure and analysis layers without dumping all cells/styles.
|
||||
|
||||
Coordinate hints, hint-aware `used_dimensions`, merge-record block candidates,
|
||||
SQL-only merge-block column-edge hints, low-confidence row/size scalar hints,
|
||||
and merge-block `record_analysis` changed the decoded artifact schema, so
|
||||
MOXCEL template cache uses `template_part_moxel_v9`. Refresh SQL baselines after
|
||||
deploying the new adapter; old `v2`/`v3`/`v4` decoded artifacts will not contain
|
||||
the current `cell_coordinate_hints`, merge-block evidence, and used-dimension
|
||||
semantics.
|
||||
@@ -0,0 +1,219 @@
|
||||
# 1C Operational Coding Loop
|
||||
|
||||
Цель: сделать помощника по 1С, который работает в темпе реальной разработки, без постоянной полной выгрузки конфигурации в XML и без обязательного 1C:EDT на первом этапе.
|
||||
|
||||
## Problem
|
||||
|
||||
Полная выгрузка конфигурации в XML медленная. 1C:EDT требует отдельной установки, настройки проекта и дисциплины синхронизации. Для оперативной разработки помощнику нужны текущие данные почти сразу:
|
||||
|
||||
- структура базы и конфигурации;
|
||||
- доступные объекты, реквизиты, табличные части, формы и команды;
|
||||
- актуальные модули BSL;
|
||||
- примеры реальных данных для read-only анализа;
|
||||
- постановки задач из текста, Excel-файлов и скриншотов интерфейса.
|
||||
|
||||
## Decision
|
||||
|
||||
Используем двухконтурную схему.
|
||||
|
||||
Быстрый контур:
|
||||
|
||||
- read-only SQL для диагностики, выборок и проверки данных;
|
||||
- легкий 1C agent внутри базы или рядом с ней для метаданных, модулей и управляемых операций;
|
||||
- локальный кеш/snapshot с коротким TTL;
|
||||
- RAG поверх актуального кеша;
|
||||
- генерация патчей, внешних отчетов, обработок и расширений как артефактов.
|
||||
|
||||
Тяжелый контур:
|
||||
|
||||
- XML/EDT/хранилище конфигурации для периодической полной синхронизации;
|
||||
- сборка, ревью, массовый рефакторинг и долгоживущие изменения;
|
||||
- финальная проверка перед переносом в production.
|
||||
|
||||
## Live Sources
|
||||
|
||||
### SQL Read-Only
|
||||
|
||||
SQL удобен как быстрый источник данных, но не является главным источником метаданных 1С.
|
||||
|
||||
Разрешено:
|
||||
|
||||
- read-only запросы;
|
||||
- выборки для отчетов и сверок;
|
||||
- оценка объемов данных;
|
||||
- поиск аномалий;
|
||||
- проверка результата после изменения в тестовой базе.
|
||||
|
||||
Запрещено:
|
||||
|
||||
- DML/DDL;
|
||||
- изменение таблиц платформы напрямую;
|
||||
- запись в production;
|
||||
- хранение строк подключения и паролей в репозитории.
|
||||
|
||||
## 1C Storage Layers (write/read boundary)
|
||||
|
||||
Принцип работы со слоями конфигурации:
|
||||
|
||||
- `Config` и `ConfigCAS` — **active** (уже применённое в системе состояние). Для них разрешены только read-операции.
|
||||
- `ConfigSave` и `ConfigCASSave` — **saved, not yet applied** (сохранённое в конфигураторе состояние). Это целевые слои для формирования изменений через адаптер.
|
||||
- Base-изменения пишутся в `ConfigSave`.
|
||||
- Extension-изменения пишутся в `ConfigCASSave`.
|
||||
|
||||
Жёсткое правило:
|
||||
|
||||
- Коннектор/агент не пишет в `Config`/`ConfigCAS`.
|
||||
- Изменения должны идти через saved-слои и проходить сравнение `ConfigSave↔Config`, `ConfigCASSave↔ConfigCAS` до ручного/внешнего apply в production.
|
||||
- Если требуется production apply, это отдельный человеческий процесс контроля и проверки.
|
||||
|
||||
Мини-чеклист перед передачей на manual-apply:
|
||||
|
||||
1. Есть актуальный compare saved-vs-active.
|
||||
2. Есть report с изменениями по объектам.
|
||||
3. Есть отметка «не применено» в `ConfigSave`/`ConfigCASSave`.
|
||||
4. Есть rollback-план и явное human approval.
|
||||
|
||||
### Working-State Read Policy
|
||||
|
||||
Для программирования и оперативного анализа помощник должен читать последнее
|
||||
сохранённое состояние конфигуратора, а не только применённую конфигурацию.
|
||||
|
||||
По умолчанию:
|
||||
|
||||
- MCP-запросы к `extension.objects.find`, `modules.search`, `code.search` и
|
||||
`metadata.resolve_overrides` используют `source_state=working`;
|
||||
- REST-запросы к тем же методам используют `state=working`;
|
||||
- `working` означает: сначала `ConfigSave`/`ConfigCASSave`, затем active-слой
|
||||
как fallback;
|
||||
- результаты помечаются `activation_state`: `saved_only`, `saved_override` или
|
||||
`active`.
|
||||
|
||||
Когда нужно сравнение:
|
||||
|
||||
- `source_state=applied` / `state=active` — показать только применённое;
|
||||
- `source_state=all` / `state=both` — показать оба слоя и различия;
|
||||
- `full_scan=true` включается только осознанно для глубокого поиска по active
|
||||
`ConfigCAS`, потому что такой поиск медленнее.
|
||||
|
||||
Если пользователь спрашивает естественным языком вроде «выдай список всех форм
|
||||
в save только имена», агент должен трактовать это как working/save-first
|
||||
срез, вернуть имена saved-форм и не отбрасывать объекты `saved_only`: они могут
|
||||
быть ещё не активированы и всё равно являются текущим состоянием разработки.
|
||||
|
||||
### Code Write Policy
|
||||
|
||||
Для оперативного программирования агент не должен знать SQL-нюансы: таблицы,
|
||||
имена файлов, stream indexes и brace paths являются внутренней реализацией
|
||||
адаптера.
|
||||
|
||||
Стандартный write API для агента:
|
||||
|
||||
- `code.write` с `module_text`/`full_text`/`code` заменяет весь модуль;
|
||||
- `code.write` с `routine_name` и `routine_text` заменяет только указанную
|
||||
процедуру или функцию;
|
||||
- `code.write` с `old` и `new` заменяет фрагмент только если `old` найден
|
||||
ровно один раз в выбранной области. Если передан `routine_name` или
|
||||
canonical path до процедуры, областью является эта процедура/функция; иначе
|
||||
весь текущий saved-модуль.
|
||||
|
||||
По умолчанию `code.write` делает `mode=apply`, но это apply в saved-state
|
||||
слой (`ConfigSave`/`ConfigCASSave`), а не применение конфигурации в runtime.
|
||||
Адаптер сам выставляет save-first gates и сам выбирает физический маршрут.
|
||||
Физические детали возвращаются только при `include_storage=true` для
|
||||
диагностики. Ответ `code.write` всегда содержит `write_mode`: target
|
||||
`saved_state`, activation_state `not_activated`, production_apply `false`.
|
||||
Ответы `code.read` и `code.search` для saved-кода содержат `current_state`:
|
||||
source `saved_state`, activation_state `not_activated`.
|
||||
Для `code.read`: `state=working` означает save-first с fallback в active,
|
||||
`state=save` читает только saved layer, `state=active` пропускает saved layer.
|
||||
`state=both` читает saved и active отдельно, возвращает `layers` для обоих
|
||||
слоев и `comparison.both_present` / `comparison.differs`. При `include_text=true`
|
||||
верхнеуровневый `text` берется из saved-state, если он есть, иначе из active;
|
||||
`text_source` показывает выбранный слой.
|
||||
Для `code.search state=both` saved-совпадения идут первыми, active-слой
|
||||
добирается отдельным проходом, а `counts.saved_matches` и
|
||||
`counts.active_matches` показывают покрытие по слоям.
|
||||
|
||||
Если фрагмент повторяется, агент должен передать более узкий контекст
|
||||
(`routine_name`) или заменить процедуру целиком. Адаптер в такой ситуации
|
||||
возвращает `ambiguous_fragment`, `scope` и `counts.occurrences`, и не
|
||||
записывает.
|
||||
|
||||
### 1C Agent
|
||||
|
||||
Для актуальной структуры и кода нужен небольшой агент на стороне 1С. Он может быть реализован как внешняя обработка, расширение или опубликованный HTTP-сервис.
|
||||
|
||||
Минимальные функции:
|
||||
|
||||
- вернуть список объектов метаданных;
|
||||
- вернуть описание объекта: реквизиты, табличные части, формы, команды, модули;
|
||||
- искать по BSL-модулям;
|
||||
- читать текст выбранного модуля;
|
||||
- выполнять read-only запрос языка запросов 1С с лимитами;
|
||||
- отдавать версию/дату изменения конфигурации для инвалидации кеша;
|
||||
- принимать change proposal, но не применять его автоматически в production.
|
||||
|
||||
Если публикация HTTP-сервиса невозможна, первым вариантом может быть ручной запуск внешней обработки, которая выгружает JSON snapshot в общую папку.
|
||||
|
||||
## Freshness Model
|
||||
|
||||
Модель не должна считать кеш вечным.
|
||||
|
||||
- SQL read-only данные читаются по запросу.
|
||||
- Metadata snapshot имеет TTL и номер версии конфигурации.
|
||||
- BSL-модули кешируются с checksum.
|
||||
- Перед генерацией кода под конкретный объект помощник проверяет свежесть metadata snapshot.
|
||||
- Если snapshot устарел или отсутствует, помощник запрашивает обновление через agent.
|
||||
|
||||
## Task Intake
|
||||
|
||||
Задачи приходят разными форматами:
|
||||
|
||||
- обычный текст;
|
||||
- Excel-файл с требованиями, примером отчета или справочником полей;
|
||||
- скриншот формы, нарисованный макет интерфейса или ошибка;
|
||||
- фрагмент BSL;
|
||||
- SQL/запрос 1С;
|
||||
- описание бизнес-процесса.
|
||||
|
||||
Обработка:
|
||||
|
||||
- текст идет напрямую в модель;
|
||||
- Excel парсится структурно: листы, заголовки, таблицы, примечания;
|
||||
- скриншоты идут через vision/OCR и превращаются в описание интерфейса, полей, команд и ошибок;
|
||||
- все извлеченные требования связываются с metadata snapshot и BSL search.
|
||||
|
||||
## Coding Outputs
|
||||
|
||||
Помощник должен уметь выдавать:
|
||||
|
||||
- BSL-фрагмент;
|
||||
- полный модуль формы/объекта/общего модуля;
|
||||
- текст запроса 1С;
|
||||
- схему внешнего отчета или обработки;
|
||||
- proposal для расширения;
|
||||
- список изменений по формам и командам;
|
||||
- чеклист проверки;
|
||||
- тестовые сценарии;
|
||||
- rollback plan.
|
||||
|
||||
Для production изменения не применяются напрямую. Нормальный поток:
|
||||
|
||||
1. Получить актуальные метаданные и модули.
|
||||
2. Сформировать change proposal.
|
||||
3. Проверить синтаксис и зависимости в тестовой базе.
|
||||
4. Сформировать артефакт: внешняя обработка, отчет, расширение или патч.
|
||||
5. Провести ревью.
|
||||
6. Перенести через согласованный 1С-процесс.
|
||||
|
||||
## Practical Priority
|
||||
|
||||
Первый рабочий MVP:
|
||||
|
||||
1. SQL read-only connector с валидацией политики.
|
||||
2. 1C metadata snapshot через внешнюю обработку в JSON.
|
||||
3. BSL module snapshot и поиск по модулям.
|
||||
4. Загрузка постановки из Excel.
|
||||
5. Загрузка скриншота формы/ошибки в vision-модель.
|
||||
6. Генерация внешнего отчета/обработки по актуальному snapshot.
|
||||
7. Smoke eval: модель не выдумывает реквизиты и просит обновить snapshot, если он устарел.
|
||||
@@ -0,0 +1,36 @@
|
||||
# 1C Plugin Health
|
||||
|
||||
Цель: одной командой проверить, что 1С-плагин собран консистентно.
|
||||
|
||||
## Run
|
||||
|
||||
```powershell
|
||||
python scripts/check_1c_plugin.py --print
|
||||
```
|
||||
|
||||
Короткий статус:
|
||||
|
||||
```powershell
|
||||
python scripts/status_1c_plugin.py
|
||||
```
|
||||
|
||||
Отчет пишется в:
|
||||
|
||||
```text
|
||||
reports/1c-plugin-health.json
|
||||
```
|
||||
|
||||
`reports/` игнорируется git.
|
||||
|
||||
## Checks
|
||||
|
||||
Скрипт проверяет:
|
||||
|
||||
- наличие обязательных файлов плагина;
|
||||
- metadata snapshot example;
|
||||
- training examples и secret-scan;
|
||||
- eval YAML;
|
||||
- RAG prompt guardrails;
|
||||
- training preflight.
|
||||
|
||||
`training_preflight` может быть `blocked`, если нет GPU, полной базовой модели или зависимостей. Это не ломает healthcheck как структурную проверку плагина.
|
||||
@@ -0,0 +1,286 @@
|
||||
# 1C RAG
|
||||
|
||||
Цель: подготовить локальный корпус знаний по 1С для поиска и ответов без дообучения модели.
|
||||
|
||||
## Principles
|
||||
|
||||
- Сначала RAG и инструменты, потом fine-tuning.
|
||||
- Не загружать в репозиторий базы, выгрузки, приватные документы, персональные данные и секреты.
|
||||
- Не выдумывать метаданные конкретной базы 1С. Если нужны реквизиты или объекты, использовать tool boundary.
|
||||
|
||||
## Add Sources
|
||||
|
||||
Кладем материалы в:
|
||||
|
||||
```text
|
||||
plugins/1c/rag/sources
|
||||
```
|
||||
|
||||
Поддерживаемые форматы:
|
||||
|
||||
- `.md`
|
||||
- `.txt`
|
||||
- `.bsl`
|
||||
- `.os`
|
||||
|
||||
## Prepare Corpus
|
||||
|
||||
```powershell
|
||||
python scripts/validate_1c_rag_sources.py --print
|
||||
python scripts/prepare_1c_rag_corpus.py
|
||||
```
|
||||
|
||||
Результат:
|
||||
|
||||
```text
|
||||
plugins/1c/datasets/prepared/rag_corpus.jsonl
|
||||
plugins/1c/datasets/prepared/rag_manifest.json
|
||||
```
|
||||
|
||||
Этот файл не коммитится.
|
||||
|
||||
## Test With Example
|
||||
|
||||
```powershell
|
||||
python scripts/prepare_1c_rag_corpus.py --source-dir plugins/1c/rag/examples --output plugins/1c/datasets/prepared/rag_corpus.example.jsonl
|
||||
```
|
||||
|
||||
## Build Search Index
|
||||
|
||||
```powershell
|
||||
python scripts/build_1c_rag_index.py
|
||||
```
|
||||
|
||||
Для синтетического примера:
|
||||
|
||||
```powershell
|
||||
python scripts/build_1c_rag_index.py --corpus plugins/1c/datasets/prepared/rag_corpus.example.jsonl --output plugins/1c/datasets/prepared/rag_index.example.json
|
||||
```
|
||||
|
||||
## Build Vector Index
|
||||
|
||||
```powershell
|
||||
python scripts/build_1c_rag_vector_index.py
|
||||
```
|
||||
|
||||
Результат:
|
||||
|
||||
```text
|
||||
plugins/1c/datasets/prepared/rag_vector_index.sqlite
|
||||
```
|
||||
|
||||
Индекс хранит `corpus_hash`, `embedding_model`, размерность, дату сборки и метаданные чанков.
|
||||
Текущий provider `local-hashing-v1` - это локальный deterministic vector baseline, а не нейросемантическая embedding-модель. Он нужен, чтобы отладить формат, freshness и hybrid retrieval без скачивания модели. Нейросемантический provider подключается следующим слоем без смены SQLite-схемы.
|
||||
|
||||
OpenAI-compatible embedding endpoint:
|
||||
|
||||
```powershell
|
||||
$env:EMBEDDING_API_KEY = "<token-if-needed>"
|
||||
python scripts/build_1c_rag_vector_index.py `
|
||||
--embedding-provider openai-compatible `
|
||||
--embedding-model "<embedding-model-name>" `
|
||||
--embedding-base-url "http://docker-gpu.cin.su:8000" `
|
||||
--embedding-api-key-env EMBEDDING_API_KEY
|
||||
```
|
||||
|
||||
Для поиска по такому индексу query embedding должен считаться тем же provider/model:
|
||||
|
||||
```powershell
|
||||
python scripts/search_1c_rag_hybrid.py "реквизиты справочника номенклатура" `
|
||||
--profile metadata `
|
||||
--embedding-base-url "http://docker-gpu.cin.su:8000" `
|
||||
--embedding-api-key-env EMBEDDING_API_KEY
|
||||
```
|
||||
|
||||
Ключ не пишется в индекс и читается только из переменной окружения.
|
||||
|
||||
Полная сборка knowledge base теперь строит corpus, lexical index и vector index:
|
||||
|
||||
```powershell
|
||||
python scripts/build_1c_knowledge_base.py --include-example-bsl
|
||||
```
|
||||
|
||||
Если vector index временно не нужен:
|
||||
|
||||
```powershell
|
||||
python scripts/build_1c_knowledge_base.py --skip-vector-index
|
||||
```
|
||||
|
||||
## Search
|
||||
|
||||
```powershell
|
||||
python scripts/search_1c_rag.py "метаданные справочника" --index plugins/1c/datasets/prepared/rag_index.example.json
|
||||
```
|
||||
|
||||
Поиск использует локальный lexical BM25-подобный индекс с нормализацией 1С-терминов и алиасами вроде `1с/bsl`, `справочник/catalog`, `метаданные/metadata`.
|
||||
|
||||
Профили RAG лежат в:
|
||||
|
||||
```text
|
||||
plugins/1c/rag/profiles.yaml
|
||||
```
|
||||
|
||||
Основные профили:
|
||||
|
||||
- `auto`
|
||||
- `general`
|
||||
- `metadata`
|
||||
- `bsl`
|
||||
- `query`
|
||||
- `safe-change`
|
||||
|
||||
`auto` выбирает профиль по тексту вопроса. Например, вопросы про реквизиты уходят в `metadata`, вопросы про процедуры и ошибки BSL - в `bsl`, вопросы с `ВЫБРАТЬ` - в `query`.
|
||||
|
||||
При подготовке корпуса источники получают смысловой `source_type`:
|
||||
|
||||
- `metadata`
|
||||
- `bsl`
|
||||
- `query`
|
||||
- `safety`
|
||||
- `docs`
|
||||
|
||||
Полезные параметры:
|
||||
|
||||
```powershell
|
||||
python scripts/search_1c_rag.py "реквизиты справочника номенклатура" `
|
||||
--index plugins/1c/datasets/prepared/rag_index.example.json `
|
||||
--profile metadata `
|
||||
--limit 5 `
|
||||
--candidate-limit 30 `
|
||||
--dedupe-by-document
|
||||
```
|
||||
|
||||
Vector search:
|
||||
|
||||
```powershell
|
||||
python scripts/search_1c_rag_vector.py "реквизиты справочника номенклатура" --profile metadata --limit 5
|
||||
```
|
||||
|
||||
Hybrid search объединяет lexical и vector результаты через reciprocal-rank fusion:
|
||||
|
||||
```powershell
|
||||
python scripts/search_1c_rag_hybrid.py "реквизиты справочника номенклатура" --profile metadata --limit 5
|
||||
```
|
||||
|
||||
Если `vector_freshness.status = stale`, нужно пересобрать vector index после обновления corpus.
|
||||
|
||||
## Ask With Context
|
||||
|
||||
Без вызова модели, только сборка prompt:
|
||||
|
||||
```powershell
|
||||
python scripts/ask_1c_rag.py "Какие реквизиты есть у справочника Номенклатура?" --index plugins/1c/datasets/prepared/rag_index.example.json --print-prompt
|
||||
```
|
||||
|
||||
С явным профилем:
|
||||
|
||||
```powershell
|
||||
python scripts/ask_1c_rag.py "Проверь BSL-код процедуры ПередЗаписью" --profile bsl --print-prompt
|
||||
```
|
||||
|
||||
Проверка обязательных правил в prompt:
|
||||
|
||||
```powershell
|
||||
python scripts/check_1c_rag_prompt.py
|
||||
```
|
||||
|
||||
Проверка качества поиска по smoke-набору:
|
||||
|
||||
```powershell
|
||||
python scripts/check_1c_rag_quality.py --print
|
||||
```
|
||||
|
||||
Проверка auto-routing профилей:
|
||||
|
||||
```powershell
|
||||
python scripts/check_1c_rag_profiles.py --print
|
||||
```
|
||||
|
||||
С запущенной моделью:
|
||||
|
||||
```powershell
|
||||
python scripts/ask_1c_rag.py "Какие реквизиты есть у справочника Номенклатура?" --index plugins/1c/datasets/prepared/rag_index.example.json --base-url http://docker-gpu.cin.su:8000 --model qwen3-4b-instruct
|
||||
```
|
||||
|
||||
## Tool Contract
|
||||
|
||||
Контракт инструментов:
|
||||
|
||||
```text
|
||||
plugins/1c/tools/tool-contract.yaml
|
||||
```
|
||||
|
||||
Главное правило: модель не должна придумывать структуру базы 1С. Если ответ зависит от метаданных, сначала нужен вызов инструмента.
|
||||
|
||||
## Metadata Snapshot Flow
|
||||
|
||||
Для проверки RAG на структуре 1С можно сгенерировать source из metadata snapshot:
|
||||
|
||||
```powershell
|
||||
python scripts/validate_1c_metadata_snapshot.py plugins/1c/metadata/examples/metadata.example.json
|
||||
python scripts/convert_1c_metadata_to_rag.py --input plugins/1c/metadata/examples/metadata.example.json --output plugins/1c/rag/sources/metadata.example.generated.md
|
||||
python scripts/build_1c_knowledge_base.py --include-example-bsl
|
||||
python scripts/ask_1c_rag.py "Какие реквизиты есть у справочника Номенклатура?" --print-prompt
|
||||
```
|
||||
|
||||
`build_1c_knowledge_base.py` выполняет валидацию snapshot, конвертацию источников, сборку corpus и индекса за один проход.
|
||||
Перед сборкой corpus он также запускает `validate_1c_rag_sources.py`, чтобы заблокировать неподдерживаемые расширения, пустые файлы и вероятные секреты.
|
||||
|
||||
Manifest содержит source path, source type, file type, content hash и число чанков. Его удобно использовать для аудита и будущего инкрементального обновления индекса.
|
||||
|
||||
Проверить, не устарел ли manifest относительно `plugins/1c/rag/sources`:
|
||||
|
||||
```powershell
|
||||
python scripts/check_1c_rag_freshness.py --print
|
||||
```
|
||||
|
||||
Статус `stale` означает, что появились новые файлы, изменился hash, был удален источник или изменился `source_type`. После этого нужно пересобрать knowledge base.
|
||||
|
||||
Проверить, не устарел ли vector index относительно corpus:
|
||||
|
||||
```powershell
|
||||
python scripts/check_1c_rag_vector_freshness.py --print
|
||||
```
|
||||
|
||||
## Semantic Cache Embedding Worker
|
||||
|
||||
Для live 1C-адаптера semantic cache заполняется отдельно от RAG corpus. Адаптер отдает pending документы с `document_id` и `content_sha1`; worker считает embedding и вызывает `semantic.cache.embedding.upsert`. Если документ изменился, адаптер отвергнет embedding по `content_sha1`.
|
||||
|
||||
Dry-run:
|
||||
|
||||
```powershell
|
||||
python scripts/embed_1c_semantic_cache.py --base-id upo_test --kind Template --limit 20 --dry-run --json
|
||||
```
|
||||
|
||||
Запись baseline-векторов:
|
||||
|
||||
```powershell
|
||||
python scripts/embed_1c_semantic_cache.py --base-id upo_test --kind Template --limit 20
|
||||
```
|
||||
|
||||
Поиск по semantic cache с автоматически рассчитанным query embedding:
|
||||
|
||||
```powershell
|
||||
python scripts/search_1c_semantic_cache.py "ОбластьШапка" --base-id upo_test --kind Template --limit 5 --validate-candidates
|
||||
```
|
||||
|
||||
Если нужно перед поиском автоматически дозаполнить pending embeddings:
|
||||
|
||||
```powershell
|
||||
python scripts/search_1c_semantic_cache.py "ОбластьШапка" --base-id upo_test --kind Template --embed-pending --validate-candidates
|
||||
```
|
||||
|
||||
С OpenAI-compatible embedding endpoint:
|
||||
|
||||
```powershell
|
||||
$env:EMBEDDING_API_KEY = "<token-if-needed>"
|
||||
python scripts/embed_1c_semantic_cache.py `
|
||||
--base-id upo_test `
|
||||
--kind Template `
|
||||
--embedding-provider openai-compatible `
|
||||
--embedding-model "<embedding-model-name>" `
|
||||
--embedding-base-url "http://docker-gpu.cin.su:8000" `
|
||||
--embedding-api-key-env EMBEDDING_API_KEY
|
||||
```
|
||||
|
||||
Semantic cache остается candidate-only: перед программными изменениями использовать `validate_candidates=true` или live read-selector из результата.
|
||||
@@ -0,0 +1,47 @@
|
||||
# 1C Training Data
|
||||
|
||||
Цель: подготовить безопасный датасет для будущего LoRA/adapter fine-tuning по 1С.
|
||||
|
||||
## Principles
|
||||
|
||||
- Сначала RAG и инструменты, потом дообучение.
|
||||
- Дообучение выполняется только на проверенных примерах.
|
||||
- Запрещено добавлять пароли, токены, строки подключения, персональные данные и клиентские секреты.
|
||||
- Нельзя обучать модель на сырых выгрузках баз 1С.
|
||||
- Каждый пример должен пройти экспертное ревью.
|
||||
|
||||
## Files
|
||||
|
||||
- Example dataset: `plugins/1c/training/examples/instruction.examples.jsonl`
|
||||
- Dataset manifest: `plugins/1c/training/manifests/dataset.yaml`
|
||||
- Adapter manifest: `plugins/1c/adapters/qwen3-coder-30b-a3b-1c-lora-v1.yaml`
|
||||
- Registry card: `registry/model-cards/qwen3-coder-30b-a3b-1c-lora-v1.yaml`
|
||||
|
||||
## Validate Example Data
|
||||
|
||||
```powershell
|
||||
python scripts/validate_1c_training_data.py plugins/1c/training/examples/instruction.examples.jsonl
|
||||
```
|
||||
|
||||
## Prepare Chat JSONL
|
||||
|
||||
```powershell
|
||||
python scripts/prepare_1c_training_data.py --input plugins/1c/training/examples/instruction.examples.jsonl --output plugins/1c/training/prepared/train.chat.jsonl
|
||||
```
|
||||
|
||||
## Update Registry
|
||||
|
||||
```powershell
|
||||
python scripts/validate_model_cards.py
|
||||
python scripts/build_model_index.py
|
||||
```
|
||||
|
||||
## Promotion Rule
|
||||
|
||||
Adapter status stays `draft` until:
|
||||
|
||||
- enough examples are collected;
|
||||
- secret scan passes;
|
||||
- expert review is complete;
|
||||
- `plugins/1c/evals/smoke.yaml` passes on base, RAG, and adapter;
|
||||
- rollback path is documented.
|
||||
@@ -0,0 +1,781 @@
|
||||
# adapter-1c-mcp
|
||||
|
||||
Thin MCP proxy for the 1C REST adapter.
|
||||
|
||||
Target host:
|
||||
|
||||
```text
|
||||
docker.cin.su
|
||||
```
|
||||
|
||||
Runtime URL:
|
||||
|
||||
```text
|
||||
http://docker.cin.su:8021
|
||||
```
|
||||
|
||||
MCP endpoint for Codex:
|
||||
|
||||
```text
|
||||
http://docker.cin.su:8021/mcp
|
||||
```
|
||||
|
||||
The server responds with MCP protocol `2025-06-18` and returns
|
||||
`Mcp-Session-Id` during `initialize`.
|
||||
|
||||
Legacy SSE endpoint:
|
||||
|
||||
```text
|
||||
http://docker.cin.su:8021/sse
|
||||
```
|
||||
|
||||
Direct JSON-RPC endpoint for smoke tests:
|
||||
|
||||
```text
|
||||
http://docker.cin.su:8021/mcp
|
||||
```
|
||||
|
||||
Codex config:
|
||||
|
||||
```toml
|
||||
[mcp_servers.adapter-1c-mcp]
|
||||
enabled = true
|
||||
url = "http://docker.cin.su:8021/mcp"
|
||||
```
|
||||
|
||||
Already running Codex Desktop windows can keep the MCP tool list from their
|
||||
startup time. Restart Codex Desktop, or start a fresh session, after changing
|
||||
MCP server configuration.
|
||||
|
||||
## How The Adapter URL Is Passed
|
||||
|
||||
The MCP proxy does not hard-code the 1C adapter address. Pass it with:
|
||||
|
||||
```text
|
||||
ONEC_ADAPTER_URL=http://docker-gpu.cin.su:8011
|
||||
```
|
||||
|
||||
Optional adapter bearer token:
|
||||
|
||||
```text
|
||||
ONEC_ADAPTER_TOKEN=
|
||||
```
|
||||
|
||||
Timeout for adapter HTTP calls:
|
||||
|
||||
```text
|
||||
ONEC_ADAPTER_TIMEOUT_SECONDS=4
|
||||
```
|
||||
|
||||
These variables are configured in:
|
||||
|
||||
```text
|
||||
core/deploy/docker/adapter-1c-mcp/.env.example
|
||||
```
|
||||
|
||||
For real deployment, create a non-committed `.env` next to the compose file and set the actual adapter URL/token there.
|
||||
|
||||
## Deploy
|
||||
|
||||
Deploy both REST adapter and MCP proxy, then run live verification when a test
|
||||
base is available:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass `
|
||||
-File scripts\deploy_1c_adapter_stack.ps1 `
|
||||
-BaseId <base-id-from-project-context>
|
||||
```
|
||||
|
||||
Multiple test bases can be verified in one deploy:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass `
|
||||
-File scripts\deploy_1c_adapter_stack.ps1 `
|
||||
-BaseId <base-id-1>,<base-id-2>
|
||||
```
|
||||
|
||||
Duplicate base ids are rejected before verification starts, so persisted
|
||||
reports are not overwritten by an accidental repeated base value.
|
||||
|
||||
To make post-deploy checks deterministic, pass an explicit metadata object
|
||||
selector. This is optional; without it the smoke discovers a module-capable
|
||||
object automatically.
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass `
|
||||
-File scripts\deploy_1c_adapter_stack.ps1 `
|
||||
-BaseId <base-id-from-project-context> `
|
||||
-ObjectGuid <metadata-object-guid> `
|
||||
-ObjectKind <metadata-kind>
|
||||
```
|
||||
|
||||
Saved-state preparation defaults to `ConfigSave` for the base configuration
|
||||
save layer. Use `-SavedStateTable ConfigCASSave` when the strict smoke should
|
||||
target an extension/CAS save layer instead:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass `
|
||||
-File scripts\deploy_1c_adapter_stack.ps1 `
|
||||
-BaseId <base-id-from-project-context> `
|
||||
-SavedStateTable ConfigCASSave
|
||||
```
|
||||
|
||||
The deploy script prints Docker container summaries for both services, then the
|
||||
verify script prints `/health` status and `contract_version` before running the
|
||||
REST and MCP selector-chain smoke tests. The selector-chain smoke also checks,
|
||||
when a routine name is available from module metadata, that
|
||||
`metadata.resolve_overrides` returns `write_plan_evidence.next_resolution` for
|
||||
`metadata.saved_state.modules.search` with the same `base_id` and routine query,
|
||||
then follows that resolver. If a saved-state stream exposes `write_plan_target`,
|
||||
the smoke composes it with `write_plan_evidence.target` and checks the resulting
|
||||
read-only `metadata.write.plan`; when the test base has no saved-state stream,
|
||||
that last composition step records `skipped_no_saved_state_target`.
|
||||
These selector-chain reports are written under
|
||||
`reports/1c-sql/<base-id>/selector-chain-rest-smoke.json` and
|
||||
`reports/1c-sql/<base-id>/selector-chain-mcp-smoke.json`. Each report includes
|
||||
a top-level `coverage` block summarizing whether override evidence was found,
|
||||
whether the saved-state resolver ran, whether a `write_plan_target` was
|
||||
available, and which steps were skipped. The verify script validates that both
|
||||
selector-chain report files are written, parse as JSON, pass, and include the
|
||||
expected `coverage` sections.
|
||||
Use `-RequireSelectorChainWritePlanComposition` when the test base is expected
|
||||
to contain a matching saved-state stream and the selector-chain smoke must fail
|
||||
instead of accepting `skipped_no_saved_state_target`.
|
||||
Verification also runs the read-only write-plan safety smoke through both REST
|
||||
and MCP `onec_request`; the MCP smoke also checks that live methods without
|
||||
`payload.base_id` are stopped by `adapter_1c_mcp_policy.v1` with
|
||||
`base_id_required`, and that low-level `storage.*`/`query.*` fallback calls are
|
||||
blocked as `diagnostic_method` unless diagnostics are explicitly requested. The
|
||||
verify script validates the written write-plan safety reports after each smoke.
|
||||
Then verification runs REST saved-state form/module write-loop smoke tests
|
||||
through `metadata.write`; if the test base has no pending
|
||||
`ConfigSave`/`ConfigCASSave` rows, those saved-state checks record
|
||||
`skipped_no_saved_state` and pass by default. The saved-state report files are
|
||||
also parsed and checked after the smoke commands. Before those smoke tests,
|
||||
verification generates read-only `saved-state-strict-readiness.json` and
|
||||
`saved-state-copy-plan.json` reports for the selected `-SavedStateTable`. The
|
||||
copy plan is generated through `scripts/plan_1c_saved_state_copy.py` using the
|
||||
same object selector arguments, so the later persisted-report check has fresh
|
||||
copy-preparation evidence. At the end, verification runs
|
||||
`scripts/check_1c_verify_reports.py` against the persisted report set, so the
|
||||
same report validation can be repeated offline without calling the adapter.
|
||||
The offline validator checks that selector-chain coverage includes
|
||||
`metadata.resolve_overrides` evidence, the expected
|
||||
`metadata.saved_state.modules.search` next step, saved-state resolution, and
|
||||
attempted `metadata.write.plan` composition. It also verifies write-plan safety
|
||||
outcomes: effective form writes must route to `metadata.write.plan`,
|
||||
uncontrolled replacement must be blocked, controlled replacement must be
|
||||
planned through `metadata.module.write_apply`, drift must be blocked, and MCP
|
||||
policy checks must block missing `base_id` and diagnostic fallback.
|
||||
Saved-state write smoke reports are checked in both modes: empty-state skips
|
||||
must include successful preflight counts and `skipped=true`; real
|
||||
write-and-rollback runs must include allowed write plans, expected apply
|
||||
methods, and rollback evidence. The same persisted report validator also checks
|
||||
`saved-state-strict-readiness.json` and `saved-state-copy-plan.json` when
|
||||
saved-state write smoke reports are enabled. Readiness must match the requested
|
||||
`base_id` and selected save-layer table; in strict mode it must be `ready=true`.
|
||||
The copy plan must match the requested `base_id`, resolve a concrete object,
|
||||
list active source rows from the matching storage family, be `plan_ready`, and
|
||||
show clear target collision status for `ConfigSave`/`ConfigCASSave`. The family
|
||||
must match the target save layer: `Config -> ConfigSave` for base changes and
|
||||
`ConfigCAS -> ConfigCASSave` for extension/CAS changes. The validator also
|
||||
checks that readiness, copy-plan target table, saved-state form smoke table, and
|
||||
saved-state module smoke `module_ref` table are the same.
|
||||
`scripts/check_powershell_scripts.py` also guards the verify/deploy wiring for
|
||||
these strict flags and asserts that `check_1c_verify_reports.py --self-test`
|
||||
covers selector-chain, safety, saved-state, and copy-plan failure classes.
|
||||
Persisted report validation also pins the expected JSON `schema` value for each
|
||||
report type, so stale or unrelated report files fail before their contents are
|
||||
trusted.
|
||||
The same validator checks report identity: each persisted JSON must match the
|
||||
requested `base_id`, and REST/MCP reports must match their expected
|
||||
`transport`. When verification provides expected endpoints, REST/MCP reports
|
||||
must also match the configured `endpoint_url`.
|
||||
|
||||
To require an actual saved-state write-and-rollback instead of allowing the
|
||||
empty-state skip:
|
||||
|
||||
```powershell
|
||||
python scripts\check_1c_saved_state_strict_readiness.py `
|
||||
--base-id <base-id-from-project-context> `
|
||||
--saved-state-table ConfigSave `
|
||||
--report reports\1c-sql\<base-id>\saved-state-strict-readiness.json `
|
||||
--json
|
||||
```
|
||||
|
||||
This readiness check is read-only. It checks REST adapter health, row counts in
|
||||
the selected save-layer table, and saved-state form/module discovery. Use
|
||||
`--saved-state-table ConfigSave` for the base save layer or
|
||||
`--saved-state-table ConfigCASSave` for the CAS/extension save layer. If it
|
||||
reports `blocked_no_saved_state_rows`, there are no unactivated Configurator
|
||||
changes in the selected save layer. To prepare a strict write test, copy the
|
||||
target object from the main configuration storage (`Config`/`ConfigCAS`) into
|
||||
the selected save layer through the approved saved-state workflow, then rerun
|
||||
readiness before enabling strict mode.
|
||||
|
||||
To produce a read-only copy preparation plan for a concrete object:
|
||||
|
||||
```powershell
|
||||
python scripts\plan_1c_saved_state_copy.py `
|
||||
--base-id <base-id-from-project-context> `
|
||||
--kind <metadata-kind> `
|
||||
--name <metadata-object-name> `
|
||||
--target-table ConfigSave `
|
||||
--report reports\1c-sql\<base-id>\saved-state-copy-plan.json `
|
||||
--json
|
||||
```
|
||||
|
||||
The copy plan does not write SQL. It lists active storage rows for the selected
|
||||
object only from the matching source family, checks whether the intended
|
||||
`ConfigSave`/`ConfigCASSave` target already contains the same `FileName` values,
|
||||
and records the intended save-layer target. `--target-table ConfigSave` plans
|
||||
from `Config`; `--target-table ConfigCASSave` plans from `ConfigCAS`. Normal
|
||||
verification regenerates this file automatically before saved-state smoke tests
|
||||
and uses `-SavedStateTable` to pick the same target table for the copy plan,
|
||||
form smoke, and module smoke. Run the command manually when choosing a target
|
||||
object for strict readiness or when preparing evidence without the full
|
||||
deploy/verify flow. The offline persisted-report validator checks this file by
|
||||
default, so regenerate the plan whenever the chosen object, target table, or
|
||||
base changes.
|
||||
|
||||
To generate the reviewed SQL preparation script from that plan without
|
||||
executing SQL:
|
||||
|
||||
```powershell
|
||||
python scripts\prepare_1c_saved_state_copy_sql.py `
|
||||
--plan reports\1c-sql\<base-id>\saved-state-copy-plan.json `
|
||||
--expected-base-id <base-id-from-project-context> `
|
||||
--expected-target-table ConfigSave `
|
||||
--sql-out reports\1c-sql\<base-id>\prepare-saved-state-copy.sql `
|
||||
--report reports\1c-sql\<base-id>\prepare-saved-state-copy-sql.json `
|
||||
--json
|
||||
```
|
||||
|
||||
This generator does not call the adapter and does not write SQL. It refuses
|
||||
plans that are not `plan_ready`, have target collisions, or mix storage
|
||||
families. The generated SQL is a manual review artifact: it starts a
|
||||
transaction, rechecks that the save-layer target has no planned `FileName`
|
||||
values, copies only the planned rows with `INSERT ... SELECT`, verifies the
|
||||
copied row count, and commits only if those guards pass.
|
||||
|
||||
To execute the reviewed preparation SQL, use the explicit SQL-write gate. This
|
||||
is not run by deploy/verify:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass `
|
||||
-File scripts\execute_1c_saved_state_copy_sql.ps1 `
|
||||
-Server <sql-server> `
|
||||
-Database <sql-database> `
|
||||
-User <sql-user> `
|
||||
-Password <sql-password> `
|
||||
-ExpectedBaseId <base-id-from-project-context> `
|
||||
-ExpectedTargetTable ConfigSave `
|
||||
-IUnderstandThisWritesToSql
|
||||
```
|
||||
|
||||
The executor validates the generated SQL report, checks that the SQL file still
|
||||
looks like the reviewed saved-state copy artifact, records the SQL SHA1, runs
|
||||
the SQL through `System.Data.SqlClient`, and immediately calls the read-only
|
||||
post-copy verifier. Credentials can also be supplied with `ONEC_SQL_SERVER`,
|
||||
`ONEC_SQL_DATABASE`, `ONEC_SQL_USER`, and `ONEC_SQL_PASSWORD`. Do not commit
|
||||
credentials or execution reports containing environment-specific connection
|
||||
details.
|
||||
|
||||
After the SQL preparation has been executed, verify the saved-state copy
|
||||
read-only before enabling strict write smoke:
|
||||
|
||||
```powershell
|
||||
python scripts\verify_1c_saved_state_copy.py `
|
||||
--plan reports\1c-sql\<base-id>\saved-state-copy-plan.json `
|
||||
--expected-base-id <base-id-from-project-context> `
|
||||
--expected-target-table ConfigSave `
|
||||
--report reports\1c-sql\<base-id>\saved-state-copy-verify.json `
|
||||
--require-ready `
|
||||
--json
|
||||
```
|
||||
|
||||
Before the SQL preparation is executed, this verifier is expected to report
|
||||
`blocked_missing_target_rows`. After preparation it must report `ready=true` by
|
||||
comparing the save-layer `FileName`, `PartNo`, byte sizes, and `BinarySHA1`
|
||||
values against the reviewed active-source rows from the copy plan.
|
||||
|
||||
Also generate the guarded cleanup script before executing the preparation SQL,
|
||||
so there is a reviewed way to remove the prepared working-copy rows later:
|
||||
|
||||
```powershell
|
||||
python scripts\prepare_1c_saved_state_cleanup_sql.py `
|
||||
--plan reports\1c-sql\<base-id>\saved-state-copy-plan.json `
|
||||
--expected-base-id <base-id-from-project-context> `
|
||||
--expected-target-table ConfigSave `
|
||||
--sql-out reports\1c-sql\<base-id>\cleanup-saved-state-copy.sql `
|
||||
--report reports\1c-sql\<base-id>\cleanup-saved-state-copy-sql.json `
|
||||
--json
|
||||
```
|
||||
|
||||
The cleanup SQL deletes only the planned `FileName`/`PartNo` rows from the
|
||||
selected save layer, and only when their current `BinarySHA1` still matches the
|
||||
reviewed copy plan. It rolls back if the rows are missing, extra/mismatched, or
|
||||
if the deleted row count differs from the plan.
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass `
|
||||
-File scripts\deploy_1c_adapter_stack.ps1 `
|
||||
-BaseId <base-id-from-project-context> `
|
||||
-RequireSavedStateWriteSmoke
|
||||
```
|
||||
|
||||
To require selector-chain evidence to resolve all the way to a concrete
|
||||
read-only `metadata.write.plan` composition:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass `
|
||||
-File scripts\deploy_1c_adapter_stack.ps1 `
|
||||
-BaseId <base-id-from-project-context> `
|
||||
-RequireSelectorChainWritePlanComposition
|
||||
```
|
||||
|
||||
To skip saved-state write-loop checks entirely:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass `
|
||||
-File scripts\deploy_1c_adapter_stack.ps1 `
|
||||
-BaseId <base-id-from-project-context> `
|
||||
-SkipSavedStateWriteSmoke
|
||||
```
|
||||
|
||||
To skip only the read-only write-plan safety smoke:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass `
|
||||
-File scripts\deploy_1c_adapter_stack.ps1 `
|
||||
-BaseId <base-id-from-project-context> `
|
||||
-SkipWritePlanSafetySmoke
|
||||
```
|
||||
|
||||
To validate the latest persisted reports without contacting REST or MCP:
|
||||
|
||||
```powershell
|
||||
python scripts\check_1c_verify_reports.py --base-id <base-id-from-project-context>
|
||||
```
|
||||
|
||||
Add `--rest-adapter-url <url> --mcp-url <url>` when the persisted REST/MCP
|
||||
reports should be pinned to specific endpoints.
|
||||
Add `--saved-state-table ConfigSave` or `--saved-state-table ConfigCASSave`
|
||||
when the persisted saved-state reports must be pinned to a specific save-layer
|
||||
table.
|
||||
Add `--max-report-age-seconds <seconds>` when the persisted reports must also
|
||||
be fresh enough for a deployment gate.
|
||||
Use `--skip-saved-state-copy-plan` only when intentionally validating an older
|
||||
report bundle that does not include `saved-state-copy-plan.json`.
|
||||
|
||||
To run the offline validator's synthetic soft/strict self-test:
|
||||
|
||||
```powershell
|
||||
python scripts\check_1c_verify_reports.py --self-test
|
||||
```
|
||||
|
||||
To run the whole offline/static verification-stack preflight:
|
||||
|
||||
```powershell
|
||||
python scripts\check_1c_adapter_verification_stack.py --base-id <base-id-from-project-context>
|
||||
```
|
||||
|
||||
The verification-stack preflight passes the default REST/MCP endpoint URLs to
|
||||
the persisted report validator. Override them with `--rest-adapter-url` and
|
||||
`--mcp-url` for non-default deployments.
|
||||
It also passes `--saved-state-table`, defaulting to `ConfigSave`, so
|
||||
persisted copy-plan/form/module reports must match the intended save-layer
|
||||
table.
|
||||
Pass multiple base ids after `--base-id` to validate several persisted report
|
||||
directories in one run.
|
||||
Use `--max-report-age-seconds` to reject stale report files during the
|
||||
verification-stack preflight.
|
||||
|
||||
To save a machine-readable verification-stack summary:
|
||||
|
||||
```powershell
|
||||
python scripts\check_1c_adapter_verification_stack.py `
|
||||
--base-id <base-id-from-project-context> `
|
||||
--json `
|
||||
--report reports\1c-sql\<base-id>\adapter-verification-stack-check.json
|
||||
```
|
||||
|
||||
The saved stack report keeps per-command return codes, compact parsed summaries
|
||||
for JSON-producing checks, and truncated stdout/stderr tails for diagnostics.
|
||||
For persisted verify-report checks, the parsed summary also includes per-base
|
||||
REST/MCP selector-chain composition status, write-plan safety check counts, and
|
||||
saved-state write smoke status, including the saved-state copy-plan target,
|
||||
source-row count, and target collision status.
|
||||
|
||||
To deploy without live verification:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass `
|
||||
-File scripts\deploy_1c_adapter_stack.ps1 `
|
||||
-SkipVerify
|
||||
```
|
||||
|
||||
Low-level MCP-only deploy:
|
||||
|
||||
```powershell
|
||||
docker --host ssh://docker.cin.su compose `
|
||||
--env-file core\deploy\docker\adapter-1c-mcp\.env.example `
|
||||
-f core\deploy\docker\adapter-1c-mcp\compose.yaml `
|
||||
up -d --build
|
||||
```
|
||||
|
||||
Container name:
|
||||
|
||||
```text
|
||||
adapter-1c-mcp
|
||||
```
|
||||
|
||||
## Health
|
||||
|
||||
```powershell
|
||||
Invoke-RestMethod http://docker.cin.su:8021/health
|
||||
docker --host ssh://docker.cin.su ps --filter name=adapter-1c-mcp
|
||||
docker --host ssh://docker.cin.su logs --tail 80 adapter-1c-mcp
|
||||
```
|
||||
|
||||
Smoke test MCP initialize:
|
||||
|
||||
```powershell
|
||||
$body = @{
|
||||
jsonrpc = "2.0"
|
||||
id = 1
|
||||
method = "initialize"
|
||||
params = @{
|
||||
protocolVersion = "2025-06-18"
|
||||
capabilities = @{}
|
||||
clientInfo = @{ name = "smoke"; version = "1" }
|
||||
}
|
||||
} | ConvertTo-Json -Depth 10
|
||||
|
||||
Invoke-WebRequest `
|
||||
-Uri "http://docker.cin.su:8021/mcp" `
|
||||
-Method Post `
|
||||
-ContentType "application/json" `
|
||||
-Headers @{ Accept = "application/json, text/event-stream" } `
|
||||
-Body $body
|
||||
```
|
||||
|
||||
## Tools
|
||||
|
||||
The MCP proxy exposes a small stable tool surface:
|
||||
|
||||
```text
|
||||
onec_health
|
||||
onec_help
|
||||
onec_request
|
||||
onec_job_get
|
||||
onec_job_cancel
|
||||
access_role_users
|
||||
access_role_profiles
|
||||
access_role_audit_export
|
||||
access_role_audit_analyze
|
||||
access_user_explain
|
||||
access_users_search
|
||||
access_object_explain
|
||||
access_keys_query
|
||||
access_object_keys_resolve
|
||||
access_object_roles
|
||||
access_object_subjects
|
||||
access_rls_discover
|
||||
```
|
||||
|
||||
Use `onec_request(method, payload)` for uncommon or newly added adapter methods
|
||||
until a dedicated alias is useful enough to keep stable.
|
||||
The `access_*` tools are lightweight aliases for common access-audit workflows;
|
||||
they only forward to REST adapter methods such as `access.role.users` and keep
|
||||
all BSP/access-rights logic in the REST adapter.
|
||||
|
||||
For object access checks, pass object names/public refs first; the REST adapter
|
||||
resolves GUIDs, SQL numbers, BSP identifiers, synonyms, and readable role-rights
|
||||
names internally. Example:
|
||||
|
||||
```powershell
|
||||
python scripts\smoke_1c_access_object.py `
|
||||
--base-id upo_test `
|
||||
--ref "РегистрСведений.УОП_АктуальныеСпецификации" `
|
||||
--action write
|
||||
```
|
||||
|
||||
Use `access_object_roles` or `access_object_subjects` for the question "which
|
||||
roles/users can read or write this metadata object". Do not depend on
|
||||
`metadata.objects.list kind=Role`: some live bases do not expose roles as regular
|
||||
metadata objects through that route.
|
||||
|
||||
For BSP role-right checks, `action=write` means add or modify rights. Read-only
|
||||
roles must not be returned for `write`; query `action=read` separately when you
|
||||
need visibility roles. Returned permissions keep `source_fields` from
|
||||
`ПраваРолей` so suspicious mappings can be verified against the live base.
|
||||
|
||||
`access.snapshot.extract` also supports this access graph and defaults to the
|
||||
BSP extractor preset when `queries` are not supplied. Passing `preset=bsp`
|
||||
explicitly is still fine, but the old "No extractor queries were provided"
|
||||
discovery response should not appear for a normal BSP base.
|
||||
|
||||
When `access_object_subjects` is called with `include_access_key_scope=true`,
|
||||
the regular `limit` only trims returned roles/profiles/groups/users. Scope
|
||||
diagnostics use separate controls: `access_key_scope_subject_limit` limits how
|
||||
many matched groups/users are checked, and `access_key_scope_limit` limits rows
|
||||
read from each BSP access-key extractor. Check `access_key_scope.coverage` and
|
||||
`access_key_scope.truncated` before treating the scope sample as complete.
|
||||
|
||||
For local audit artifacts, use the workspace script:
|
||||
|
||||
```powershell
|
||||
python scripts\export_1c_access_role_audit.py `
|
||||
--base-id upo_test `
|
||||
--role "запись изменение номенклатура поставщиков" `
|
||||
--user-threshold 30
|
||||
```
|
||||
|
||||
It writes CSV, JSON export, analysis JSON, and a summary file under
|
||||
`reports/1c-access/<base_id>/`. By default it also writes a self-contained
|
||||
HTML report with findings, counters, artifact links, and a filterable user table;
|
||||
pass `--no-html` to skip it.
|
||||
|
||||
Architecture contract:
|
||||
|
||||
- The REST adapter owns 1C behavior, payload validation, decoding, owner
|
||||
resolution, RAG/tool-facing data shape, and method documentation through
|
||||
`help.methods`.
|
||||
- MCP is a transport proxy: JSON-RPC/MCP framing, adapter URL/token handling,
|
||||
generic `onec_request`, job polling/cancellation helpers, and lightweight
|
||||
agent-safety checks such as requiring `base_id` for live database methods.
|
||||
- Do not add method-specific 1C business logic to MCP. Put it in the adapter,
|
||||
expose it through `/rpc`, and make it discoverable via `help.methods`.
|
||||
- Do not register an MCP unified/composite handler with the same name as an
|
||||
adapter method; `onec_request(method=...)` must call the adapter method, not
|
||||
shadow it inside MCP.
|
||||
- When adding a new adapter method, verify it through MCP with
|
||||
`onec_request({"method": "...", "payload": {...}})`. MCP should not require a
|
||||
schema edit for ordinary adapter growth.
|
||||
- Run `python scripts/check_1c_mcp_adapter_contract.py` before release; it
|
||||
verifies that `onec_request` stays generic and that every method listed by the
|
||||
REST adapter is forwarded through `/rpc`.
|
||||
|
||||
Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"method": "metadata.object.get",
|
||||
"payload": {
|
||||
"base_id": "<base_id-from-project-context>",
|
||||
"kind": "document",
|
||||
"name": "ПриходнаяНакладная",
|
||||
"view": "merged"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Do not copy placeholder or sample `base_id` values from documentation into real
|
||||
requests. For live database methods, get `base_id` from the current project,
|
||||
user request, environment, or another authoritative context.
|
||||
|
||||
## Agent Request Policy
|
||||
|
||||
The MCP proxy blocks live database methods without `payload.base_id` before
|
||||
calling the REST adapter. This is intentional: agents must not probe metadata,
|
||||
modules, extensions, queries, storage, or code without a concrete target base.
|
||||
|
||||
Affected method families include:
|
||||
|
||||
```text
|
||||
metadata.*
|
||||
modules.*
|
||||
code.*
|
||||
templates.*
|
||||
diagnostics.*
|
||||
extensions.*
|
||||
query.*
|
||||
storage.*
|
||||
schema.*
|
||||
codec.*
|
||||
```
|
||||
|
||||
Allowed without `base_id`:
|
||||
|
||||
```text
|
||||
onec_help / help.methods
|
||||
onec_health without a base for generic service health
|
||||
adapter.job.get / adapter.job.cancel
|
||||
```
|
||||
|
||||
Agent rules:
|
||||
|
||||
- If `base_id` is unknown, ask for it or obtain it from project context.
|
||||
- For object-scoped calls, use one of the public selector shapes supported by
|
||||
the adapter: `ref`, `kind` + `name`, `guid`, or MCP-friendly
|
||||
`object_type`/`object_name`/`object_guid`. Do not add MCP-side conditions for
|
||||
concrete object names.
|
||||
- If a prior result contains `module_ref`, `module_id`, GUID, or read selector,
|
||||
use direct read methods before global search.
|
||||
- If a search result contains `read_selector.method`, call that method with the
|
||||
selector payload. `code.search` selectors point to `code.read`; `modules.search`
|
||||
selectors point to `modules.read`.
|
||||
- If a result contains `related_selectors`, prefer those payloads for the next
|
||||
object-scoped call. They preserve `kind`/`name`/`guid` and include `ref` when
|
||||
the adapter knows the canonical object kind and name.
|
||||
- When `modules.search` or `code.search` resolves a module owner, keep both the
|
||||
public owner selector (`ref`, `kind`, `name`, `guid`) and the opaque
|
||||
`module_ref` in the next read payload. `module_ref` is the direct read handle;
|
||||
`ref` makes the owner clear to the agent and later calls.
|
||||
- When narrowing `code.search` to one module, pass `module_ordinal` together
|
||||
with the object selector. The adapter response should remain a public
|
||||
`onec_code_search.v1` object with `item.read_selector`, not a low-level tuple
|
||||
or storage result.
|
||||
- For BSL edits, prefer high-level `metadata.write` with a 1C canonical path,
|
||||
`routine_text`, and `routine_operation`. Do not ask the user for SQL/save
|
||||
gates: the adapter prepares saved-state when needed, writes only to the
|
||||
saved-state working layer, and does not activate changes. Use
|
||||
`metadata.form.command_button.write` for the complete form command + visible
|
||||
button + handler workflow. `code.write` remains a compatibility shortcut for
|
||||
simple module edits.
|
||||
- For unresolved module owners, inspect `diagnostics.owner_resolution` and
|
||||
`counts.owner_scan_limit_hit`; narrow by `kind`/`name`/`guid` or increase
|
||||
`owner_scan_limit` before falling back to broader searches.
|
||||
- Do not treat `not_found` from `metadata.definition.find` or scoped search as
|
||||
proof that code is absent in extensions or ConfigCAS.
|
||||
- Treat `partial`, `truncated=true`, scan limits, and timeouts as incomplete
|
||||
evidence.
|
||||
|
||||
Offline selector-chain smoke:
|
||||
|
||||
```powershell
|
||||
python scripts\smoke_1c_mcp_selector_chain.py --json --no-report
|
||||
```
|
||||
|
||||
Saved-state BSL write smoke:
|
||||
|
||||
```powershell
|
||||
python scripts\smoke_1c_code_write_saved_state.py `
|
||||
--adapter-url http://docker-gpu.cin.su:8011 `
|
||||
--base-id <base-id-from-project-context> `
|
||||
--extension <extension-name> `
|
||||
--object-type CommonForm `
|
||||
--object-name <form-name> `
|
||||
--routine-name <routine-name> `
|
||||
--json
|
||||
```
|
||||
|
||||
This smoke performs idempotent `code.write mode=apply` checks with the current
|
||||
routine text, a unique `old`/`new` fragment, and the current full
|
||||
`module_text`. It then verifies `code.read state=working`,
|
||||
`code.read state=both`, marker-hiding for saved form modules, and saved-state
|
||||
form indexing. Every write step must report `write_mode.target=saved_state`
|
||||
and `activation_state=not_activated`.
|
||||
|
||||
Agent working-view report:
|
||||
|
||||
```powershell
|
||||
python scripts\report_1c_agent_working_view.py `
|
||||
--adapter-url http://docker-gpu.cin.su:8011 `
|
||||
--base-id <base-id-from-project-context> `
|
||||
--extension <extension-name> `
|
||||
--object-type CommonForm `
|
||||
--object-name <form-name> `
|
||||
--routine-name <routine-name> `
|
||||
--json
|
||||
```
|
||||
|
||||
This read-only report shows the form names that an agent sees in the
|
||||
working/save layer and verifies that `code.read state=working` and
|
||||
`state=both` prefer `saved_state` when saved code exists.
|
||||
|
||||
The concise agent-facing coding rules are kept in
|
||||
`docs/runbooks/1c-agent-coding-contract.md`.
|
||||
|
||||
Run this with the contract checks before release. It validates generic agent
|
||||
routes through `onec_request` and rejects concrete object names in selector
|
||||
examples.
|
||||
|
||||
Optional live selector-chain smoke against a real adapter/base:
|
||||
|
||||
```powershell
|
||||
python scripts\smoke_1c_mcp_selector_chain.py `
|
||||
--live `
|
||||
--transport rest `
|
||||
--adapter-url http://docker-gpu.cin.su:8011 `
|
||||
--base-id <base-id-from-project-context> `
|
||||
--json `
|
||||
--no-report
|
||||
```
|
||||
|
||||
To verify both deployed layers with one command:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass `
|
||||
-File scripts\verify_1c_adapter_deployment.ps1 `
|
||||
-BaseId <base-id-from-project-context>
|
||||
```
|
||||
|
||||
By default this also writes
|
||||
`reports/1c-sql/<base>/code-write-saved-state-rest-smoke.json` and
|
||||
`reports/1c-sql/<base>/code-write-saved-state-mcp-smoke.json` when the default
|
||||
saved-state code-write target exists. The REST verification also writes
|
||||
`reports/1c-sql/<base>/agent-working-view.json` to pin the save-first working
|
||||
view that agents should use for coding. Use
|
||||
`-RequireCodeWriteSavedStateSmoke` to make that smoke mandatory, or
|
||||
`-SkipCodeWriteSavedStateSmoke` to skip it for bases where the scenario is not
|
||||
available.
|
||||
|
||||
Pass multiple base ids to run the same REST and MCP checks against each base:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass `
|
||||
-File scripts\verify_1c_adapter_deployment.ps1 `
|
||||
-BaseId <base-id-1>,<base-id-2>
|
||||
```
|
||||
|
||||
Use `-ObjectRef`, or `-ObjectKind` with `-ObjectName`/`-ObjectGuid`, to verify a
|
||||
specific module-capable metadata object instead of auto-discovery.
|
||||
Use `-SavedStateTable ConfigSave` or `-SavedStateTable ConfigCASSave` to choose
|
||||
which save-layer table is used by the copy plan and saved-state write smokes.
|
||||
Add `-RequireSelectorChainWritePlanComposition` when the selected base/object
|
||||
must have a saved-state stream that lets the selector-chain smoke compose a
|
||||
concrete read-only `metadata.write.plan`.
|
||||
|
||||
To exercise the MCP proxy itself, switch transport and URL:
|
||||
|
||||
```powershell
|
||||
python scripts\smoke_1c_mcp_selector_chain.py `
|
||||
--live `
|
||||
--transport mcp `
|
||||
--mcp-url http://docker.cin.su:8021 `
|
||||
--base-id <base-id-from-project-context> `
|
||||
--json `
|
||||
--no-report
|
||||
```
|
||||
|
||||
Live mode discovers a module-capable object from metadata lists and then checks
|
||||
`adapter.help.methods -> metadata.definition.find -> related_selectors.modules
|
||||
-> metadata.object.modules -> modules.read -> code.search ->
|
||||
item.read_selector -> code.read` using the returned selector and a token derived
|
||||
from module text or routine metadata. The `adapter.help.methods` step verifies
|
||||
live REST `contract_version` and `selector_capabilities`, so stale REST adapter
|
||||
images fail before the longer chain. MCP transport also performs `initialize`,
|
||||
checks `tools/list` for the same `contract_version` and the generic
|
||||
`onec_request` selector schema, keeps `Mcp-Session-Id`, and sends adapter calls
|
||||
through `tools/call` + `onec_request`. This catches stale MCP proxy images where
|
||||
REST works but agents still see old tool descriptions. Do not commit real base
|
||||
ids or captured reports from live runs.
|
||||
- Avoid increasing global search limits when a direct selector is already
|
||||
available.
|
||||
- Do not request `include_storage=true` merely to read a module found by search;
|
||||
use the public `read_selector` first.
|
||||
|
||||
The request is proxied to the REST adapter through `POST /rpc`:
|
||||
|
||||
```json
|
||||
{
|
||||
"method": "metadata.object.get",
|
||||
"payload": {}
|
||||
}
|
||||
```
|
||||
|
||||
The MCP proxy intentionally does not know adapter-specific 1C methods. Add new
|
||||
methods in the REST adapter and expose them through `help.methods`; MCP stays
|
||||
unchanged unless the MCP transport itself changes.
|
||||
@@ -0,0 +1,61 @@
|
||||
# Artifact Portability
|
||||
|
||||
Модели, RAG-корпусы, выгрузки ИТС, снапшоты метаданных и тренировочные наборы
|
||||
не хранятся в git. При переносе проекта или запуске в Docker их нужно переносить
|
||||
и монтировать отдельно.
|
||||
|
||||
## Build Manifest
|
||||
|
||||
```powershell
|
||||
python scripts/build_llm_artifact_manifest.py --output reports/llm-artifact-manifest.json
|
||||
```
|
||||
|
||||
По умолчанию manifest хранит размеры, количество файлов, расширения и sample
|
||||
файлов. Хеширование больших моделей выключено, чтобы команда не была медленной.
|
||||
Для более строгой проверки:
|
||||
|
||||
```powershell
|
||||
python scripts/build_llm_artifact_manifest.py --hash-files --max-files 2000
|
||||
```
|
||||
|
||||
## Check Manifest
|
||||
|
||||
На той же машине:
|
||||
|
||||
```powershell
|
||||
python scripts/check_llm_artifact_manifest.py `
|
||||
--manifest reports/llm-artifact-manifest.json `
|
||||
--output reports/llm-artifact-check.json
|
||||
```
|
||||
|
||||
После переноса в другой workspace:
|
||||
|
||||
```powershell
|
||||
python scripts/check_llm_artifact_manifest.py `
|
||||
--manifest reports/llm-artifact-manifest.json `
|
||||
--target-root Z:/codex/LLM `
|
||||
--output reports/llm-artifact-check.json
|
||||
```
|
||||
|
||||
## Docker Volumes
|
||||
|
||||
Контейнеры не должны хранить эти данные во внутреннем слое. Монтируем внешние
|
||||
папки:
|
||||
|
||||
```yaml
|
||||
volumes:
|
||||
- Z:/codex/LLM/models:/app/models
|
||||
- Z:/codex/LLM/plugins/1c/datasets:/app/plugins/1c/datasets
|
||||
- Z:/codex/LLM/plugins/1c/rag/sources:/app/plugins/1c/rag/sources
|
||||
- Z:/codex/LLM/plugins/1c/rag/official-docs:/app/plugins/1c/rag/official-docs
|
||||
- Z:/codex/LLM/plugins/1c/metadata/snapshots:/app/plugins/1c/metadata/snapshots
|
||||
```
|
||||
|
||||
## Rule
|
||||
|
||||
Перед удалением контейнеров, переносом на другой host или пересборкой volumes:
|
||||
|
||||
1. Собрать manifest.
|
||||
2. Скопировать artifact directories.
|
||||
3. Проверить manifest на целевом пути.
|
||||
4. Только потом запускать Docker services.
|
||||
@@ -0,0 +1,43 @@
|
||||
# Ask 1C RAG
|
||||
|
||||
Цель: проверить полный локальный сценарий 1С RAG:
|
||||
|
||||
```text
|
||||
question -> local search -> context prompt -> optional vLLM answer
|
||||
```
|
||||
|
||||
## Prepare Example Corpus
|
||||
|
||||
```powershell
|
||||
python scripts/prepare_1c_rag_corpus.py --source-dir plugins/1c/rag/examples --output plugins/1c/datasets/prepared/rag_corpus.example.jsonl
|
||||
```
|
||||
|
||||
## Build Example Index
|
||||
|
||||
```powershell
|
||||
python scripts/build_1c_rag_index.py --corpus plugins/1c/datasets/prepared/rag_corpus.example.jsonl --output plugins/1c/datasets/prepared/rag_index.example.json
|
||||
```
|
||||
|
||||
## Render Prompt Only
|
||||
|
||||
```powershell
|
||||
python scripts/ask_1c_rag.py "Какие реквизиты есть у справочника Номенклатура?" --index plugins/1c/datasets/prepared/rag_index.example.json --print-prompt
|
||||
```
|
||||
|
||||
Этот режим не требует запущенной модели. Он нужен для проверки, какой контекст будет передан LLM.
|
||||
|
||||
## Check Prompt Guardrails
|
||||
|
||||
```powershell
|
||||
python scripts/check_1c_rag_prompt.py
|
||||
```
|
||||
|
||||
## Ask Running Model
|
||||
|
||||
После запуска vLLM:
|
||||
|
||||
```powershell
|
||||
python scripts/ask_1c_rag.py "Какие реквизиты есть у справочника Номенклатура?" --index plugins/1c/datasets/prepared/rag_index.example.json --base-url http://docker-gpu.cin.su:8000 --model qwen3-4b-instruct
|
||||
```
|
||||
|
||||
Ожидаемое поведение: модель не должна выдумывать реквизиты. Она должна сказать, что для ответа нужны метаданные 1С.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Deploy llama.cpp GGUF
|
||||
|
||||
Цель: поднять GGUF-модель `Devstral Small 2 24B Instruct 2512 Q4_K_M` через `llama-server` на `docker-gpu.cin.su`.
|
||||
|
||||
## Model
|
||||
|
||||
- Repo: `bartowski/mistralai_Devstral-Small-2-24B-Instruct-2512-GGUF`
|
||||
- File: `mistralai_Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf`
|
||||
- Size: `14334438272` bytes
|
||||
- Registry card: `registry/model-cards/devstral-small-2-24b-instruct-2512-q4_k_m.yaml`
|
||||
|
||||
## Download
|
||||
|
||||
Resume-safe local download:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/download_devstral_gguf.ps1
|
||||
```
|
||||
|
||||
The script can be re-run after network failures.
|
||||
|
||||
## Compose
|
||||
|
||||
```text
|
||||
core/deploy/docker-gpu/llama-cpp/compose.yaml
|
||||
core/deploy/docker-gpu/llama-cpp/.env.example
|
||||
```
|
||||
|
||||
## Run
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_llama_cpp.ps1 -Pull
|
||||
```
|
||||
|
||||
Проверить compose без запуска:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_llama_cpp.ps1 -ConfigOnly
|
||||
```
|
||||
|
||||
Остановить сервис:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_llama_cpp.ps1 -Down
|
||||
```
|
||||
|
||||
## Check
|
||||
|
||||
```powershell
|
||||
python scripts/check_inference_endpoint.py --base-url http://docker-gpu.cin.su:8080 --expected-model devstral-1c-q4 --print
|
||||
```
|
||||
|
||||
```powershell
|
||||
python scripts/smoke_chat.py --base-url http://docker-gpu.cin.su:8080 --model devstral-1c-q4
|
||||
```
|
||||
@@ -0,0 +1,82 @@
|
||||
# Deploy vLLM
|
||||
|
||||
Цель: поднять первый OpenAI-compatible inference API на `docker-gpu.cin.su`.
|
||||
|
||||
## 1. Check GPU Host
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/check_gpu_host.ps1
|
||||
```
|
||||
|
||||
Если SSH ругается на host key, см. `docs/runbooks/gpu-host-preflight.md`.
|
||||
|
||||
## 2. Prepare Env
|
||||
|
||||
Для реального запуска лучше создать локальный `.env` рядом с compose-файлом:
|
||||
|
||||
```powershell
|
||||
Copy-Item core/deploy/docker-gpu/vllm/.env.example core/deploy/docker-gpu/vllm/.env
|
||||
```
|
||||
|
||||
Отредактировать:
|
||||
|
||||
- `VLLM_MODEL_ID`
|
||||
- `VLLM_SERVED_MODEL_NAME`
|
||||
- `HOST_MODELS_DIR`
|
||||
- `HOST_HF_CACHE_DIR`
|
||||
- `HF_TOKEN`, если модель закрытая
|
||||
|
||||
На Windows GPU-хосте используйте проверенный образ
|
||||
`VLLM_IMAGE=vllm/vllm-openai:v0.10.2`. С драйвером CUDA 12.8 новые образы могут
|
||||
не стартовать из-за требования CUDA 13. После обновления драйвера до CUDA 13.x
|
||||
образ `latest` проходит CUDA-проверку, но vLLM 0.23.0 на Docker Desktop/WSL
|
||||
падает при старте движка с `UVA is not available`.
|
||||
|
||||
Файл `.env` не коммитится.
|
||||
|
||||
## 3. Validate Compose
|
||||
|
||||
Без запуска:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_vllm.ps1 -ConfigOnly
|
||||
```
|
||||
|
||||
## 4. Deploy
|
||||
|
||||
С `.env.example`:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_vllm.ps1 -Pull
|
||||
```
|
||||
|
||||
С реальным `.env`:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_vllm.ps1 -EnvFile core/deploy/docker-gpu/vllm/.env -Pull
|
||||
```
|
||||
|
||||
## 5. Check Endpoint
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/check_vllm_endpoint.ps1
|
||||
```
|
||||
|
||||
Или напрямую:
|
||||
|
||||
```powershell
|
||||
python scripts/check_inference_endpoint.py --base-url http://docker-gpu.cin.su:8000 --expected-model qwen3-4b-instruct --print
|
||||
python scripts/smoke_chat.py --base-url http://docker-gpu.cin.su:8000 --model qwen3-4b-instruct
|
||||
```
|
||||
|
||||
## 6. Stop
|
||||
|
||||
```powershell
|
||||
docker --host ssh://test@docker-gpu.cin.su compose --env-file core/deploy/docker-gpu/vllm/.env -f core/deploy/docker-gpu/vllm/compose.yaml down
|
||||
```
|
||||
|
||||
Если для GPU-хоста нужен явный пользователь, передайте его в параметре:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_vllm.ps1 -DockerHost ssh://USER@docker-gpu.cin.su -ConfigOnly
|
||||
```
|
||||
@@ -0,0 +1,81 @@
|
||||
# Download Model
|
||||
|
||||
Цель: загрузить модель из Hugging Face в локальное хранилище, не помещая веса модели в git.
|
||||
|
||||
## Install Dependencies
|
||||
|
||||
```powershell
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## Dry Run
|
||||
|
||||
```powershell
|
||||
python scripts/download_hf_model.py qwen3-4b-instruct-2507 --dry-run
|
||||
```
|
||||
|
||||
## Download
|
||||
|
||||
На хосте, где доступен путь `/models`:
|
||||
|
||||
```powershell
|
||||
python scripts/download_hf_model.py qwen3-4b-instruct-2507
|
||||
```
|
||||
|
||||
Если команда запускается на Windows, а в model card указан Linux-путь `/models/...`, реальная загрузка без `--local-dir` будет остановлена. Это защита от случайной загрузки в неправильный локальный каталог.
|
||||
|
||||
Для закрытой модели токен передается через окружение:
|
||||
|
||||
```powershell
|
||||
$env:HF_TOKEN="..."
|
||||
python scripts/download_hf_model.py qwen3-4b-instruct-2507
|
||||
```
|
||||
|
||||
Не сохраняйте токен в репозитории, `.env` или model card.
|
||||
|
||||
## Resume Large Files
|
||||
|
||||
Если сеть рвет большие файлы, используйте range-загрузчик. Он докачивает файл с текущего размера и проверяет итоговый размер по Hugging Face metadata.
|
||||
|
||||
```powershell
|
||||
python scripts/download_hf_range.py qwen3-4b-instruct-2507 `
|
||||
--local-dir models/base/qwen3-4b-instruct-2507 `
|
||||
--allow-file model-00001-of-00003.safetensors `
|
||||
--allow-file model-00002-of-00003.safetensors `
|
||||
--chunk-size 16mb `
|
||||
--retries 20
|
||||
```
|
||||
|
||||
Эту команду можно запускать повторно до полного завершения.
|
||||
|
||||
## Override Path
|
||||
|
||||
Если нужно скачать в другой каталог:
|
||||
|
||||
```powershell
|
||||
python scripts/download_hf_model.py qwen3-4b-instruct-2507 --local-dir D:\models\base\qwen3-4b-instruct-2507
|
||||
```
|
||||
|
||||
## Registry Index
|
||||
|
||||
После добавления или изменения карточек моделей:
|
||||
|
||||
```powershell
|
||||
python scripts/build_model_index.py
|
||||
```
|
||||
|
||||
## Check Local Storage
|
||||
|
||||
Проверить, какие модели полностью скачаны, частично скачаны или отсутствуют:
|
||||
|
||||
```powershell
|
||||
python scripts/check_model_storage.py --print
|
||||
```
|
||||
|
||||
В обычном `check_all` эта проверка работает как warning, потому что часть моделей и адаптеров может быть запланирована, но еще не загружена.
|
||||
|
||||
Для текущего состояния всей платформы:
|
||||
|
||||
```powershell
|
||||
python scripts/collect_platform_status.py --print
|
||||
```
|
||||
@@ -0,0 +1,89 @@
|
||||
# Evals
|
||||
|
||||
Цель: проверять качество моделей, RAG и будущих адаптеров воспроизводимым способом.
|
||||
|
||||
## Validate Eval Files
|
||||
|
||||
```powershell
|
||||
python scripts/validate_evals.py
|
||||
```
|
||||
|
||||
## Run 1C Smoke Eval Without Model
|
||||
|
||||
Prompt-only режим проверяет, что eval-набор читается и превращается в отчет.
|
||||
|
||||
```powershell
|
||||
python scripts/run_1c_smoke_eval.py --print
|
||||
```
|
||||
|
||||
Отчет пишется в:
|
||||
|
||||
```text
|
||||
reports/evals/1c-smoke.report.json
|
||||
```
|
||||
|
||||
Папка `reports` не коммитится.
|
||||
|
||||
## Run 1C Smoke Eval Against vLLM
|
||||
|
||||
После запуска inference:
|
||||
|
||||
```powershell
|
||||
python scripts/run_1c_smoke_eval.py --base-url http://docker-gpu.cin.su:8000 --model qwen3-4b-instruct
|
||||
```
|
||||
|
||||
Runner сохраняет ответы и считает автоматические проверки, если критерий задан структурно.
|
||||
Строковые критерии остаются ручными.
|
||||
|
||||
Поддерживаемые автоматические типы:
|
||||
|
||||
- `contains`
|
||||
- `not_contains`
|
||||
- `regex`
|
||||
- `max_sentences`
|
||||
- `refuses`
|
||||
- `requires_metadata`
|
||||
|
||||
Для GGUF/llama.cpp:
|
||||
|
||||
```powershell
|
||||
python scripts/run_1c_smoke_eval.py --base-url http://docker-gpu.cin.su:8080 --model devstral-1c-q4
|
||||
```
|
||||
|
||||
Перед evals удобно проверить endpoint:
|
||||
|
||||
```powershell
|
||||
python scripts/check_inference_endpoint.py --base-url http://docker-gpu.cin.su:8000 --expected-model qwen3-4b-instruct --print
|
||||
python scripts/check_inference_endpoint.py --base-url http://docker-gpu.cin.su:8080 --expected-model devstral-1c-q4 --print
|
||||
```
|
||||
|
||||
## Run Live Evals For Available Models
|
||||
|
||||
Единая команда проверяет endpoint-ы активного GPU-профиля и запускает smoke eval только для моделей этого профиля:
|
||||
|
||||
```powershell
|
||||
python scripts/run_live_model_evals.py --profile text --print
|
||||
```
|
||||
|
||||
Отчет пишется в:
|
||||
|
||||
```text
|
||||
reports/evals/live-model-evals.json
|
||||
```
|
||||
|
||||
По умолчанию live eval использует продуктовый 1С system prompt:
|
||||
|
||||
```text
|
||||
plugins/1c/prompts/system.md
|
||||
```
|
||||
|
||||
Если endpoint недоступен, соответствующая модель получает статус `blocked`.
|
||||
Для полной ручной проверки взаимоисключающих профилей используйте:
|
||||
|
||||
```powershell
|
||||
python scripts/run_live_model_evals.py --all-targets --print
|
||||
```
|
||||
|
||||
## Promotion Rule
|
||||
|
||||
Модель или адаптер нельзя переводить из `draft/staging` в `production`, пока smoke-evals не пройдены и отчет не просмотрен.
|
||||
@@ -0,0 +1,72 @@
|
||||
# First vLLM Inference
|
||||
|
||||
Цель: поднять первый OpenAI-compatible inference API на `docker-gpu.cin.su`.
|
||||
|
||||
Основной runbook запуска: `docs/runbooks/deploy-vllm.md`.
|
||||
Preflight GPU-хоста: `docs/runbooks/gpu-host-preflight.md`.
|
||||
|
||||
## Files
|
||||
|
||||
- `core/deploy/docker-gpu/vllm/compose.yaml`
|
||||
- `core/deploy/docker-gpu/vllm/.env.example`
|
||||
- `registry/model-cards/qwen3-4b-instruct-2507.yaml`
|
||||
|
||||
## Prepare
|
||||
|
||||
На GPU-хосте:
|
||||
|
||||
```powershell
|
||||
docker --context default version
|
||||
```
|
||||
|
||||
С локальной машины, если настроен Docker context:
|
||||
|
||||
```powershell
|
||||
docker --host ssh://test@docker-gpu.cin.su info
|
||||
```
|
||||
|
||||
## Configure
|
||||
|
||||
Скопировать `.env.example` в `.env` на стороне deployment-каталога и указать:
|
||||
|
||||
- `VLLM_MODEL_ID`
|
||||
- `VLLM_SERVED_MODEL_NAME`
|
||||
- `HOST_MODELS_DIR`
|
||||
- `HOST_HF_CACHE_DIR`
|
||||
- `HF_TOKEN`, только если нужен доступ к закрытой модели
|
||||
|
||||
Не коммитить `.env`.
|
||||
|
||||
## Run
|
||||
|
||||
```powershell
|
||||
docker compose --env-file .env -f core/deploy/docker-gpu/vllm/compose.yaml up -d
|
||||
```
|
||||
|
||||
## Check
|
||||
|
||||
```powershell
|
||||
curl http://docker-gpu.cin.su:8000/v1/models
|
||||
```
|
||||
|
||||
Пример запроса:
|
||||
|
||||
```powershell
|
||||
curl http://docker-gpu.cin.su:8000/v1/chat/completions `
|
||||
-H "Content-Type: application/json" `
|
||||
-d '{"model":"qwen3-4b-instruct","messages":[{"role":"user","content":"Привет. Ответь коротко."}]}'
|
||||
```
|
||||
|
||||
Или smoke-скриптом из корня проекта:
|
||||
|
||||
```powershell
|
||||
python scripts/smoke_chat.py --base-url http://docker-gpu.cin.su:8000 --model qwen3-4b-instruct
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- Сначала используем одну текстовую модель.
|
||||
- Первая модель: `Qwen/Qwen3-4B-Instruct-2507`.
|
||||
- Первый запуск ограничен `VLLM_MAX_MODEL_LEN=32768`, чтобы снизить риск OOM.
|
||||
- Для 1С позже подключим RAG и LoRA/adapters отдельно.
|
||||
- Большие модели и cache лежат вне git.
|
||||
@@ -0,0 +1,118 @@
|
||||
# GPU Host Preflight
|
||||
|
||||
Цель: проверить, что `docker-gpu.cin.su` готов к запуску GPU-контейнеров.
|
||||
|
||||
Если `docker-gpu.cin.su` указывает на Windows-хост, используйте также отдельный runbook:
|
||||
|
||||
```text
|
||||
docs/runbooks/windows-docker-gpu-host.md
|
||||
```
|
||||
|
||||
## SSH Host Key
|
||||
|
||||
Если SSH сообщает `Host key verification failed`, нужно вручную проверить fingerprint хоста и добавить ключ в `known_hosts`.
|
||||
|
||||
Команда для просмотра ключа:
|
||||
|
||||
```powershell
|
||||
ssh-keyscan docker-gpu.cin.su
|
||||
```
|
||||
|
||||
После проверки fingerprint можно добавить ключ:
|
||||
|
||||
```powershell
|
||||
ssh-keyscan docker-gpu.cin.su >> $env:USERPROFILE\.ssh\known_hosts
|
||||
```
|
||||
|
||||
Не добавляйте ключ вслепую, если есть риск подмены DNS или хоста.
|
||||
|
||||
## Preflight
|
||||
|
||||
Из корня проекта:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/check_gpu_host.ps1
|
||||
```
|
||||
|
||||
Если нужен явный пользователь SSH:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/check_gpu_host.ps1 -SshTarget USER@docker-gpu.cin.su
|
||||
```
|
||||
|
||||
Скрипт проверяет:
|
||||
|
||||
- SSH-доступ;
|
||||
- наличие GPU через `nvidia-smi`;
|
||||
- Docker Engine;
|
||||
- Docker Compose;
|
||||
- запуск тестового CUDA-контейнера с `--gpus all`.
|
||||
|
||||
## Readiness Report
|
||||
|
||||
Чтобы одной командой проверить SSH preflight и endpoints активного GPU-профиля:
|
||||
|
||||
```powershell
|
||||
python scripts/check_gpu_readiness.py --profile text --print
|
||||
```
|
||||
|
||||
Отчет пишется в:
|
||||
|
||||
```text
|
||||
reports/gpu-readiness.json
|
||||
```
|
||||
|
||||
По умолчанию проверяется профиль `text` из `config/gpu_profiles.json`:
|
||||
|
||||
- SSH/GPU/Docker preflight через `scripts/check_gpu_host.ps1`.
|
||||
- health/model endpoints из поля `wait` выбранного профиля.
|
||||
|
||||
Полная проверка взаимоисключающих OpenAI-compatible endpoints доступна отдельно:
|
||||
|
||||
```powershell
|
||||
python scripts/check_gpu_readiness.py --all-endpoints --print
|
||||
```
|
||||
|
||||
Если SSH возвращает `Permission denied`, нужно настроить SSH-ключ или явно указать пользователя:
|
||||
|
||||
```powershell
|
||||
python scripts/check_gpu_readiness.py --ssh-target USER@docker-gpu.cin.su --print
|
||||
```
|
||||
|
||||
## Full GPU Stack Flow
|
||||
|
||||
После настройки SSH-доступа можно запустить весь контур:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_gpu_stack.ps1 -Pull
|
||||
```
|
||||
|
||||
Сценарий выполняет:
|
||||
|
||||
- GPU host preflight;
|
||||
- deploy vLLM;
|
||||
- deploy llama.cpp GGUF;
|
||||
- ожидание `/v1/models` для обоих endpoint-ов;
|
||||
- запуск live evals.
|
||||
|
||||
Посмотреть план без запуска:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_gpu_stack.ps1 -PlanOnly -Pull
|
||||
```
|
||||
|
||||
Проверить compose-файлы без SSH preflight:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_gpu_stack.ps1 -ConfigOnly -SkipPreflight
|
||||
```
|
||||
|
||||
## Expected Result
|
||||
|
||||
В выводе должны быть:
|
||||
|
||||
- имя хоста;
|
||||
- модель GPU и объем VRAM;
|
||||
- версия Docker;
|
||||
- версия Docker Compose;
|
||||
- результат `nvidia-smi` внутри контейнера.
|
||||
@@ -0,0 +1,50 @@
|
||||
# Management Console
|
||||
|
||||
Локальная web-консоль для управления рабочим контуром LLM/1C.
|
||||
|
||||
## Start
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/run_management_console.ps1
|
||||
```
|
||||
|
||||
Default URL:
|
||||
|
||||
```text
|
||||
http://127.0.0.1:8770/
|
||||
```
|
||||
|
||||
## Current Capabilities
|
||||
|
||||
- overview of local model/RAG artifacts;
|
||||
- model storage report view;
|
||||
- 1C RAG and official 1C:ITS artifact status;
|
||||
- 1C plugin health summary;
|
||||
- whitelisted checks and maintenance actions;
|
||||
- 1C:ITS cookie dialog launch;
|
||||
- 1C:ITS fetch progress from `raw/progress.json`;
|
||||
- job log with stdout/stderr tail.
|
||||
|
||||
The console intentionally does not expose arbitrary shell execution. Every
|
||||
action must be present in the server whitelist in
|
||||
`scripts/management_console_server.py`.
|
||||
|
||||
## Current Commands
|
||||
|
||||
- build artifact manifest;
|
||||
- check artifact manifest;
|
||||
- check model storage;
|
||||
- check 1C plugin;
|
||||
- check 1C RAG freshness;
|
||||
- check official docs private artifacts;
|
||||
- validate PowerShell scripts;
|
||||
- open 1C:ITS cookie dialog.
|
||||
|
||||
## Next Expansion
|
||||
|
||||
- Docker service status and start/stop/restart for approved services;
|
||||
- SQL connector status for known 1C bases;
|
||||
- RAG rebuild with selected source groups;
|
||||
- model download/ingest queue;
|
||||
- vector index status when hybrid search is added;
|
||||
- role-based action policy before any write-capable 1C operation.
|
||||
@@ -0,0 +1,71 @@
|
||||
# Model Chat Testbench
|
||||
|
||||
Цель: вручную проверять модели из `registry/index.json` через чат с выбором плагина и модели.
|
||||
|
||||
## Web UI
|
||||
|
||||
```powershell
|
||||
python scripts/model_chat_server.py
|
||||
```
|
||||
|
||||
Открыть:
|
||||
|
||||
```text
|
||||
http://127.0.0.1:8765
|
||||
```
|
||||
|
||||
Интерфейс читает модели из `registry/index.json`, группирует их по плагинам и отправляет запросы в OpenAI-compatible endpoint через локальный proxy.
|
||||
|
||||
Возможности:
|
||||
|
||||
- выбор endpoint preset: `vLLM text`, `llama.cpp GGUF`, `local`;
|
||||
- проверка `/v1/models` и наличия выбранного `served_model_name`;
|
||||
- выбор плагина и модели из registry;
|
||||
- готовые тестовые prompt-пакеты по каждому плагину;
|
||||
- сравнение нескольких моделей на одном prompt;
|
||||
- сборка 1C RAG prompt с найденными источниками;
|
||||
- сохранение каждого ответа и ошибки в JSONL;
|
||||
- ручная оценка ответа: `ok`, `needs_review`, `bad`.
|
||||
|
||||
Endpoint по умолчанию:
|
||||
|
||||
```text
|
||||
http://docker-gpu.cin.su:8000
|
||||
```
|
||||
|
||||
## CLI Smoke Chat
|
||||
|
||||
```powershell
|
||||
python scripts/model_chat_cli.py --plugin text
|
||||
python scripts/model_chat_cli.py --plugin 1c --model-id qwen3-4b-instruct-2507
|
||||
python scripts/model_chat_cli.py --plugin translation --prompt "Переведи на английский: Проверяем модель."
|
||||
```
|
||||
|
||||
## Reports
|
||||
|
||||
Результаты сохраняются в:
|
||||
|
||||
```text
|
||||
reports/model-chat/YYYYMMDD.jsonl
|
||||
```
|
||||
|
||||
Записи `type=chat` содержат prompt, ответ, модель, endpoint, latency и статус. Записи `type=feedback` содержат ручную оценку и ссылку на `parent_id`.
|
||||
|
||||
Записи `type=compare` содержат общий prompt и ответы нескольких моделей. Записи `type=rag_prompt` содержат вопрос, количество найденных источников и список source chunks.
|
||||
|
||||
## 1C RAG Prompt
|
||||
|
||||
Для кнопки `1C RAG` нужен подготовленный индекс:
|
||||
|
||||
```powershell
|
||||
python scripts/prepare_1c_rag_corpus.py
|
||||
python scripts/build_1c_rag_index.py
|
||||
```
|
||||
|
||||
Если индекс отсутствует, UI покажет команду подготовки.
|
||||
|
||||
## Notes
|
||||
|
||||
- Для vLLM используется `served_model_name` из model card.
|
||||
- Для GGUF/llama.cpp укажите endpoint соответствующего сервиса, например `--base-url http://docker-gpu.cin.su:8080`.
|
||||
- Если GPU endpoint недоступен, UI покажет ошибку запроса, но каталог моделей все равно загрузится.
|
||||
@@ -0,0 +1,388 @@
|
||||
# Model Chat UI Service
|
||||
|
||||
Цель: поднять личный кабинет проверки моделей в локальной сети.
|
||||
|
||||
LAN URL:
|
||||
|
||||
```text
|
||||
http://192.168.220.91:8765/tools/model-chat/
|
||||
```
|
||||
|
||||
## Deploy
|
||||
|
||||
Preflight before deployment:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\preflight_model_chat_ui.ps1 `
|
||||
-DockerHost ssh://docker-gpu `
|
||||
-SshTarget docker-gpu `
|
||||
-CheckLive
|
||||
```
|
||||
|
||||
Fast local-only preflight:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\preflight_model_chat_ui.ps1 -SkipDockerConfig
|
||||
```
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts/deploy_model_chat_ui.ps1 `
|
||||
-DockerHost ssh://docker-gpu `
|
||||
-SshTarget docker-gpu
|
||||
```
|
||||
|
||||
Скрипт:
|
||||
|
||||
- собирает небольшой архив приложения без тяжелых `models/`;
|
||||
- копирует его на Windows GPU-хост;
|
||||
- распаковывает в `Z:\LLM\model-chat-app`;
|
||||
- запускает compose-сервис `llm-model-chat-ui`;
|
||||
- открывает Windows Firewall для TCP `8765`.
|
||||
|
||||
## Health
|
||||
|
||||
```powershell
|
||||
Invoke-RestMethod http://192.168.220.91:8765/api/health
|
||||
Invoke-RestMethod http://192.168.220.91:8765/api/catalog
|
||||
Invoke-RestMethod http://192.168.220.91:8765/api/model-services
|
||||
python scripts\plan_model_services.py
|
||||
python scripts\generate_model_chat_status.py
|
||||
```
|
||||
|
||||
The status generator writes a compact Markdown snapshot to:
|
||||
|
||||
```text
|
||||
reports/model-chat/status.md
|
||||
```
|
||||
|
||||
`/api/health` includes:
|
||||
|
||||
- endpoint status for vLLM, llama.cpp, translation, audio, and video service ports;
|
||||
- selected route for each plugin;
|
||||
- GPU profile readiness with missing services to start and conflicting services to stop;
|
||||
- latest Model Chat UI preflight status from `reports/model-chat/preflight.json`;
|
||||
- local storage status for every registered model;
|
||||
- service plan with compose/deploy hints.
|
||||
|
||||
Service states:
|
||||
|
||||
- `online`: endpoint is running and reports the expected served model name;
|
||||
- `ready_to_start`: model files are present, but the service is not online;
|
||||
- `blocked`: local model files are missing or incomplete.
|
||||
|
||||
## Service Control
|
||||
|
||||
The UI has a `service` panel for the selected model. It calls `POST /api/service-control`:
|
||||
|
||||
```json
|
||||
{
|
||||
"model_id": "whisper-large-v3-turbo",
|
||||
"action": "status"
|
||||
}
|
||||
```
|
||||
|
||||
Allowed actions are `start`, `stop`, `restart`, and `status`.
|
||||
|
||||
If the UI container has no Docker CLI or SSH key, use the operator script from the project folder:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\manage_model_service.ps1 -Service audio-api -Action status
|
||||
powershell -ExecutionPolicy Bypass -File scripts\manage_model_service.ps1 -Service audio-api -Action stop
|
||||
powershell -ExecutionPolicy Bypass -File scripts\manage_model_service.ps1 -Service translation-api -Action start
|
||||
```
|
||||
|
||||
Use `stop` on heavy services before starting another large model when VRAM is low.
|
||||
|
||||
## GPU Profiles
|
||||
|
||||
Use GPU profiles instead of manual container juggling when switching between heavy models:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile default
|
||||
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile text
|
||||
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile audio
|
||||
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile video
|
||||
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile image
|
||||
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile gguf-1c
|
||||
```
|
||||
|
||||
Profile definitions live in:
|
||||
|
||||
```text
|
||||
config/gpu_profiles.json
|
||||
```
|
||||
|
||||
The UI catalog, `/api/health` profile readiness, and `scripts\switch_gpu_profile.ps1` read the same file.
|
||||
Validate it before deploy:
|
||||
|
||||
```powershell
|
||||
python scripts\validate_gpu_profiles.py --print
|
||||
```
|
||||
|
||||
Validate the UI deployment archive without uploading it:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\deploy_model_chat_ui.ps1 -ArchiveOnly
|
||||
```
|
||||
|
||||
Dry-run the plan without touching containers:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile video -PlanOnly
|
||||
```
|
||||
|
||||
Profiles:
|
||||
|
||||
- `default` / `text`: vLLM text, translation, audio, and UI; video, image, and llama.cpp stopped.
|
||||
- `audio`: audio, translation, vLLM text, and UI; video, image, and llama.cpp stopped.
|
||||
- `video`: translation, video, and UI; vLLM text, audio, image, and llama.cpp stopped to free VRAM.
|
||||
- `image`: translation, image, and UI; vLLM text, audio, video, and llama.cpp stopped to free VRAM.
|
||||
- `gguf-1c`: llama.cpp, translation, and UI; vLLM text, audio, video, and image stopped.
|
||||
|
||||
The Model Chat UI also shows the recommended profile command for the selected plugin.
|
||||
The profile hint shows `status: ready` when the current containers already match the profile.
|
||||
Otherwise it lists services that should be started and services that should be stopped.
|
||||
For `video` and `image`, the service starts quickly but the first request can still spend several minutes loading
|
||||
the model into GPU memory.
|
||||
|
||||
## Runtime Profiles
|
||||
|
||||
Runtime profiles choose the execution host and endpoint used by the chat UI:
|
||||
|
||||
```text
|
||||
config/runtime_profiles.json
|
||||
```
|
||||
|
||||
Profiles:
|
||||
|
||||
- `gpu-fast`: default interactive profile on `docker-gpu.cin.su`; uses RTX 4090 endpoints.
|
||||
- `cpu-test`: benchmark/fallback profile on `docker-test.cin.su`; currently maps Qwen3-Coder Q6 to `http://docker-test.cin.su:18086` with served model `qwen3-coder-1c-q6-cpu`.
|
||||
- `background`: batch profile for downloads, RAG indexing and conversions; not intended for direct chat.
|
||||
|
||||
The UI applies `model_overrides` from the selected runtime profile. This allows the same registry model
|
||||
to use a different endpoint or served model name on another host.
|
||||
|
||||
The service panel has `Benchmark GPU / CPU`. It calls `POST /api/benchmark/runtime`, runs
|
||||
`scripts/benchmark_runtime_profiles.py`, and stores the raw report in:
|
||||
|
||||
```text
|
||||
/reports/benchmarks/runtime-profiles-<model-id>-<timestamp>.json
|
||||
```
|
||||
|
||||
The UI loads recent benchmark history for the currently selected model/plugin through
|
||||
`GET /api/benchmark/history?model_id=<model-id>&plugin=<plugin-id>`.
|
||||
When both `gpu-fast` and `cpu-test` succeed, the report includes `speedup.gpu_vs_cpu_ratio`.
|
||||
Use larger generation limits, for example 192-384 tokens, for representative GPU/CPU ratios;
|
||||
very short runs include more startup and request overhead.
|
||||
The UI has a separate `Benchmark tokens` field, default `384`, so chat generation limits do not
|
||||
accidentally make CPU comparison runs too long.
|
||||
Before starting the long benchmark request, the UI performs a quick preflight for both `gpu-fast`
|
||||
and `cpu-test`; if either served model is missing, the benchmark is not started.
|
||||
|
||||
```text
|
||||
POST /api/benchmark/preflight
|
||||
```
|
||||
|
||||
CLI check:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\check_runtime_preflight.ps1
|
||||
```
|
||||
|
||||
Each history row links to the raw JSON report through:
|
||||
|
||||
```text
|
||||
/api/benchmark/report?name=<runtime-profiles-report.json>
|
||||
```
|
||||
|
||||
It also links to a generated Markdown summary:
|
||||
|
||||
```text
|
||||
/api/benchmark/report.md?name=<runtime-profiles-report.json>
|
||||
```
|
||||
|
||||
For `cpu-test`, start the heavyweight llama.cpp CPU server before sending chat requests:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\manage_cpu_q6_service.ps1 -Action start
|
||||
powershell -ExecutionPolicy Bypass -File scripts\manage_cpu_q6_service.ps1 -Action status
|
||||
powershell -ExecutionPolicy Bypass -File scripts\manage_cpu_q6_service.ps1 -Action logs
|
||||
```
|
||||
|
||||
For `gpu-fast` Qwen3-Coder Q6 checks on port `8081`, use the GPU launcher. Stop image/video/vLLM
|
||||
first if VRAM is tight:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile gguf-1c
|
||||
powershell -ExecutionPolicy Bypass -File scripts\manage_gpu_q6_service.ps1 -Action start
|
||||
powershell -ExecutionPolicy Bypass -File scripts\manage_gpu_q6_service.ps1 -Action status
|
||||
powershell -ExecutionPolicy Bypass -File scripts\manage_gpu_q6_service.ps1 -Action logs
|
||||
```
|
||||
|
||||
After you convert a trained PEFT LoRA adapter to GGUF for `llama.cpp`, you can start the same
|
||||
service with the adapter applied:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\convert_1c_lora_to_gguf_gpu.ps1
|
||||
|
||||
powershell -ExecutionPolicy Bypass -File scripts\manage_gpu_q6_service.ps1 `
|
||||
-Action start `
|
||||
-LoraPath /models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1.gguf
|
||||
```
|
||||
|
||||
Stop it after benchmarks on the shared host if it is no longer needed:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\manage_cpu_q6_service.ps1 -Action stop
|
||||
powershell -ExecutionPolicy Bypass -File scripts\manage_gpu_q6_service.ps1 -Action stop
|
||||
```
|
||||
|
||||
## Containers
|
||||
|
||||
```powershell
|
||||
docker -H ssh://docker-gpu ps
|
||||
docker -H ssh://docker-gpu logs --tail 100 llm-model-chat-ui
|
||||
docker -H ssh://docker-gpu logs --tail 100 llm-vllm-text
|
||||
docker -H ssh://docker-gpu logs --tail 100 llm-transformers-translation
|
||||
```
|
||||
|
||||
## Transformers Plugin Services
|
||||
|
||||
Translation, audio, video, and image services use the small OpenAI-like server in
|
||||
`scripts/transformers_plugin_server.py`.
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\deploy_transformers_service.ps1 -Plugin translation
|
||||
powershell -ExecutionPolicy Bypass -File scripts\deploy_transformers_service.ps1 -Plugin audio
|
||||
powershell -ExecutionPolicy Bypass -File scripts\deploy_transformers_service.ps1 -Plugin video
|
||||
powershell -ExecutionPolicy Bypass -File scripts\deploy_transformers_service.ps1 -Plugin image
|
||||
```
|
||||
|
||||
By default the deploy script also refreshes `Z:\LLM\model-chat-app` before restarting the selected
|
||||
service, so changes in `scripts/transformers_plugin_server.py` are applied to the mounted `/app`
|
||||
directory. Use `-NoSyncApp` only for a fast container restart when the app files are already current.
|
||||
|
||||
Ports:
|
||||
|
||||
- translation: `8010`
|
||||
- audio: `8020`
|
||||
- video: `8030`
|
||||
- image: `8040`
|
||||
|
||||
The translation service exposes `/health`, `/v1/models`, and `/v1/chat/completions`.
|
||||
The audio service exposes `/health`, `/v1/models`, and `/v1/audio/transcriptions`; in the UI,
|
||||
select plugin `Звук`, choose an audio file, then press `Отправить`.
|
||||
The video service exposes `/health`, `/v1/models`, and `/v1/vision/analyze`; in the UI,
|
||||
select plugin `Видео`, choose an image, enter the question in the prompt box, then press `Отправить`.
|
||||
The image service exposes `/health`, `/v1/models`, `/v1/images/generations`, `/v1/images/edits`,
|
||||
and async job endpoints under `/v1/images/jobs`. In the UI, select plugin `Фото`, choose
|
||||
an image model, choose `generate` or `edit / inpaint`, enter the prompt, then press `Отправить`.
|
||||
By default `LOAD_ON_START=1` and `BACKGROUND_LOAD_ON_START=1` for image generation, so the service
|
||||
opens HTTP quickly and loads the SDXL base model in the background. The first request can show
|
||||
`loading_model` for several minutes. Switching between SDXL base and inpaint may unload the other
|
||||
pipeline to keep VRAM available.
|
||||
Stop the large text vLLM container before loading another large model if VRAM is tight.
|
||||
|
||||
The UI can select `Qwen Image Edit`, but the image endpoint must actually serve
|
||||
`qwen-image-edit`; otherwise `/api/image/submit` rejects the job instead of silently using SDXL.
|
||||
To test Qwen Image Edit as a single-heavy-model experiment, restart the image service with the Qwen
|
||||
env file:
|
||||
|
||||
```powershell
|
||||
docker -H ssh://docker-gpu compose `
|
||||
--env-file core/deploy/docker-gpu/transformers/image.qwen-edit.env.example `
|
||||
-f core/deploy/docker-gpu/transformers/image.compose.yaml up -d
|
||||
```
|
||||
|
||||
Restore the verified SDXL service with:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\deploy_transformers_service.ps1 -Plugin image
|
||||
```
|
||||
|
||||
Verified Qwen Image Edit notes from `2026-06-20`:
|
||||
|
||||
- model files are present under `Z:\LLM\models\image\qwen-image-edit`;
|
||||
- `diffusers` detects `QwenImageEditPipeline` and the service can expose `/v1/models` as
|
||||
`qwen-image-edit`;
|
||||
- pipeline load completed in about `31 s` with CPU offload on RTX 4090;
|
||||
- a 512x512 edit job with `1` inference step did not finish within `1800 s`, so Qwen Image Edit is
|
||||
not practical on the current 24 GB GPU profile without a quantized/optimized runtime or a larger
|
||||
GPU;
|
||||
- keep SDXL as the default verified image service for now.
|
||||
|
||||
Smoke test image jobs through the Model Chat proxy:
|
||||
|
||||
```powershell
|
||||
python scripts\smoke_image_jobs.py --operation generate --steps 4
|
||||
python scripts\smoke_image_jobs.py --operation edit --steps 4
|
||||
python scripts\smoke_image_jobs.py --operation edit --steps 40 --cancel --cancel-after 1
|
||||
python scripts\smoke_image_jobs.py --model-id qwen-image-edit --model-mode qwen-image-edit --model qwen-image-edit --operation edit --steps 4
|
||||
```
|
||||
|
||||
Audio and video smoke checks generate small synthetic inputs locally and send them through the
|
||||
Model Chat UI proxy:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile audio
|
||||
python scripts\smoke_audio_transcription.py
|
||||
|
||||
powershell -ExecutionPolicy Bypass -File scripts\switch_gpu_profile.ps1 -Profile video
|
||||
python scripts\smoke_video_analysis.py
|
||||
```
|
||||
|
||||
When the matching service is intentionally stopped, use `--allow-unavailable` for a code-path check
|
||||
that does not fail the local validation run.
|
||||
|
||||
Verified on `2026-06-20` with RTX 4090:
|
||||
|
||||
- generate, SDXL base, 512x512, 4 steps: completed in `2194 ms` after warmup;
|
||||
- edit / inpaint, SDXL inpaint, 512x512, 4 steps: completed in `180649 ms` including first inpaint model load;
|
||||
- generate, SDXL base, 512x512, 1 step: completed in `86866 ms` after image service restart and warmup;
|
||||
- generated artifacts are served through `/generated-images/<date>/<file>.png`.
|
||||
|
||||
Direct artifact check uses `GET`; this minimal HTTP server does not implement `HEAD` for generated files.
|
||||
|
||||
## Model Downloads On GPU Host
|
||||
|
||||
Use this when model files should be written directly to `Z:\LLM\models` on the GPU host:
|
||||
|
||||
For large Hugging Face shard files, prefer the explicit range downloader. It writes final files directly
|
||||
to `/models/...`, resumes by local file size, and avoids stale `.cache/huggingface/download/*.incomplete`
|
||||
files left by interrupted Xet/snapshot downloads.
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\download_hf_range_gpu.ps1 `
|
||||
-CardId qwen2_5-vl-7b-instruct `
|
||||
-LocalDir /models/video/qwen2.5-vl-7b-instruct `
|
||||
-AllowFile "model-00001-of-00005.safetensors,model-00002-of-00005.safetensors,model-00003-of-00005.safetensors,model-00004-of-00005.safetensors,model-00005-of-00005.safetensors" `
|
||||
-Detached `
|
||||
-ContainerName llm-hf-range-qwen-vl `
|
||||
-ChunkSize 256mb
|
||||
```
|
||||
|
||||
The registry/snapshot downloader is still useful for dry-run planning and small metadata files:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File scripts\download_missing_hf_models_gpu.ps1 -DryRun
|
||||
|
||||
docker -H ssh://docker-gpu rm -f llm-model-download
|
||||
powershell -ExecutionPolicy Bypass -File scripts\download_missing_hf_models_gpu.ps1 `
|
||||
-Detached `
|
||||
-ContainerName llm-model-download
|
||||
```
|
||||
|
||||
Monitor:
|
||||
|
||||
```powershell
|
||||
docker -H ssh://docker-gpu ps
|
||||
docker -H ssh://docker-gpu logs --tail 100 llm-model-download
|
||||
Invoke-RestMethod http://192.168.220.91:8765/api/model-services
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- UI is served by `scripts/model_chat_server.py`.
|
||||
- Inference endpoint defaults to `http://docker-gpu.cin.su:8000`.
|
||||
- Use `vllm/vllm-openai:v0.10.2` for the UI image because it already contains Python and is verified on this host.
|
||||
- `vllm/vllm-openai:latest` passes CUDA after driver `595.97`, but vLLM `0.23.0` currently fails on Docker Desktop/WSL with `UVA is not available`.
|
||||
@@ -0,0 +1,77 @@
|
||||
# Model Ingest
|
||||
|
||||
Цель: принять новую модель от пользователя, агента или внешнего источника без ручного копирования в реестр.
|
||||
|
||||
## UI
|
||||
|
||||
Локальный кабинет моделей доступен в Model Chat Testbench:
|
||||
|
||||
```text
|
||||
http://127.0.0.1:8765
|
||||
```
|
||||
|
||||
Блок `Модели` поддерживает:
|
||||
|
||||
- загрузку файла из браузера;
|
||||
- заявку на импорт из URL;
|
||||
- заявку на импорт из пути, доступного серверу;
|
||||
- просмотр последних ingest jobs.
|
||||
|
||||
## API
|
||||
|
||||
Последние заявки:
|
||||
|
||||
```powershell
|
||||
curl http://127.0.0.1:8765/api/model-ingest/jobs
|
||||
```
|
||||
|
||||
Загрузить файл raw stream:
|
||||
|
||||
```powershell
|
||||
curl -X POST "http://127.0.0.1:8765/api/model-ingest/upload?model_id=my-model&plugin=1c&format=gguf&filename=model.gguf" `
|
||||
--data-binary "@model.gguf"
|
||||
```
|
||||
|
||||
Попросить сервер скачать из URL или импортировать путь:
|
||||
|
||||
```powershell
|
||||
curl -X POST http://127.0.0.1:8765/api/model-ingest/source `
|
||||
-H "Content-Type: application/json" `
|
||||
-d "{\"model_id\":\"my-model\",\"plugin\":\"1c\",\"format\":\"gguf\",\"source\":\"https://example/model.gguf\"}"
|
||||
```
|
||||
|
||||
## Storage
|
||||
|
||||
Файлы складываются в:
|
||||
|
||||
```text
|
||||
models/incoming/<job_id>/
|
||||
```
|
||||
|
||||
Метаданные задания:
|
||||
|
||||
```text
|
||||
models/incoming/<job_id>/metadata.json
|
||||
```
|
||||
|
||||
Журнал:
|
||||
|
||||
```text
|
||||
reports/model-ingest/jobs.jsonl
|
||||
```
|
||||
|
||||
## Promotion
|
||||
|
||||
После проверки размера, checksum, лицензии и runtime модель нужно вручную промоутить:
|
||||
|
||||
1. Перенести файл из `models/incoming/<job_id>/` в целевую папку `models/...`.
|
||||
2. Создать или обновить `registry/model-cards/<model_id>.yaml`.
|
||||
3. Запустить:
|
||||
|
||||
```powershell
|
||||
python scripts/validate_model_cards.py
|
||||
python scripts/build_model_index.py
|
||||
python scripts/check_model_storage.py --warn-only
|
||||
```
|
||||
|
||||
Пароли к NAS, Hugging Face токены и другие секреты не записываются в ingest job и не должны попадать в model card.
|
||||
@@ -0,0 +1,151 @@
|
||||
# Q6 LoRA Troubleshooting
|
||||
|
||||
Цель: короткая памятка по типовым проблемам при обучении, конвертации и публикации `Qwen3-Coder Q6` с LoRA.
|
||||
|
||||
## Main Rule
|
||||
|
||||
На `docker-gpu` training/download контейнеры должны видеть код из `Z:\LLM\model-chat-app`, а не напрямую из `Z:\codex\LLM`.
|
||||
|
||||
Актуальный поток такой:
|
||||
|
||||
1. локальный репозиторий архивируется;
|
||||
2. архив синхронизируется в `Z:\LLM\model-chat-app`;
|
||||
3. training/download контейнеры используют этот каталог как `/workspace` или `/app`.
|
||||
|
||||
Если новые `scripts`, `plugins` или `registry/model-cards` не попали в `model-chat-app`, контейнеры будут работать на старом коде.
|
||||
|
||||
## Quick Checks
|
||||
|
||||
Проверить GPU host:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/check_gpu_host.ps1 -SshTarget docker-gpu
|
||||
```
|
||||
|
||||
Показать план полного пайплайна:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/train_and_publish_q6_lora.ps1 -PlanOnly
|
||||
```
|
||||
|
||||
Прогнать только безопасный preflight:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/train_and_publish_q6_lora.ps1 -SkipDownload -SkipTrain -SkipConvert -SkipRestart -SkipSmoke
|
||||
```
|
||||
|
||||
## Common Failures
|
||||
|
||||
### `Model card not found for id qwen3-coder-30b-a3b-instruct`
|
||||
|
||||
Причина:
|
||||
|
||||
- контейнер загрузки стартовал раньше, чем новый `model card` попал в `Z:\LLM\model-chat-app`.
|
||||
|
||||
Что делать:
|
||||
|
||||
```powershell
|
||||
. .\scripts\app_archive.ps1
|
||||
$archive = New-AppArchive
|
||||
Test-AppArchive -Archive $archive
|
||||
Sync-AppDirectory -SshTarget docker-gpu -RemoteArchive 'C:\ProgramData\LLM\model-chat-app.zip' -RemoteAppDir 'Z:\LLM\model-chat-app' -Archive $archive
|
||||
docker -H ssh://docker-gpu rm -f llm-hf-range-qwen3-coder-base
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\download_hf_range_gpu.ps1 -CardId qwen3-coder-30b-a3b-instruct -LocalDir /models/base/qwen3-coder-30b-a3b-instruct -Detached -ContainerName llm-hf-range-qwen3-coder-base
|
||||
```
|
||||
|
||||
### `requirements-training.txt` not found
|
||||
|
||||
Причина:
|
||||
|
||||
- training container смонтировал неправильный workspace path.
|
||||
|
||||
Правильное ожидание:
|
||||
|
||||
- `HOST_WORKSPACE_DIR=Z:/LLM/model-chat-app`
|
||||
- wrapper `scripts/run_1c_lora_training_gpu.ps1` сам синхронизирует текущий код в `model-chat-app` перед `docker compose up`.
|
||||
|
||||
### `base model is incomplete at /models/base/qwen3-coder-30b-a3b-instruct`
|
||||
|
||||
Причина:
|
||||
|
||||
- HF-база еще не скачана полностью.
|
||||
|
||||
Что смотреть:
|
||||
|
||||
```powershell
|
||||
docker -H ssh://docker-gpu logs -f llm-hf-range-qwen3-coder-base
|
||||
docker -H ssh://docker-gpu run --rm -v Z:/LLM/models:/models alpine sh -lc "ls -lah /models/base/qwen3-coder-30b-a3b-instruct"
|
||||
```
|
||||
|
||||
Хороший признак:
|
||||
|
||||
- в логе идут строки `OK <filename>`
|
||||
- появились все `model-00001-of-00016.safetensors` ... `model-00016-of-00016.safetensors`
|
||||
|
||||
### `CUDA out of memory`
|
||||
|
||||
Причина:
|
||||
|
||||
- слишком тяжелая конфигурация для текущего режима RTX 4090.
|
||||
|
||||
Что делать:
|
||||
|
||||
- остановить лишние тяжелые сервисы;
|
||||
- убедиться, что активен только нужный training/runtime контур;
|
||||
- при необходимости уменьшить training нагрузку в `plugins/1c/training/configs/qwen3-coder-30b-a3b-lora.yaml`.
|
||||
|
||||
### `llama.cpp` стартовал без адаптера
|
||||
|
||||
Причина:
|
||||
|
||||
- не передан `-LoraPath`
|
||||
- `.gguf` адаптер не был создан
|
||||
- опубликован не тот путь
|
||||
|
||||
Проверка:
|
||||
|
||||
```powershell
|
||||
docker -H ssh://docker-gpu run --rm -v Z:/LLM/models:/models alpine sh -lc "ls -lah /models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1.gguf"
|
||||
powershell -ExecutionPolicy Bypass -File scripts\manage_gpu_q6_service.ps1 -Action status -LoraPath /models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1.gguf
|
||||
```
|
||||
|
||||
## Monitoring Commands
|
||||
|
||||
Скачивание базы:
|
||||
|
||||
```powershell
|
||||
docker -H ssh://docker-gpu logs -f llm-hf-range-qwen3-coder-base
|
||||
```
|
||||
|
||||
Обучение:
|
||||
|
||||
```powershell
|
||||
docker -H ssh://docker-gpu logs -f llm-train-1c-lora
|
||||
```
|
||||
|
||||
Q6 runtime:
|
||||
|
||||
```powershell
|
||||
docker -H ssh://docker-gpu logs -f llm-llama-qwen3-coder-q6-test
|
||||
```
|
||||
|
||||
Endpoint check:
|
||||
|
||||
```powershell
|
||||
python scripts\check_inference_endpoint.py --base-url http://docker-gpu.cin.su:8081 --expected-model qwen3-coder-1c-q6 --print
|
||||
```
|
||||
|
||||
Smoke eval:
|
||||
|
||||
```powershell
|
||||
python scripts\run_1c_smoke_eval.py --base-url http://docker-gpu.cin.su:8081 --model qwen3-coder-1c-q6
|
||||
```
|
||||
|
||||
## Recommended Order
|
||||
|
||||
1. Дождаться полной загрузки HF-базы.
|
||||
2. Прогнать `PreflightOnly`.
|
||||
3. Запустить обучение.
|
||||
4. Конвертировать LoRA в GGUF.
|
||||
5. Перезапустить `Q6` с `-LoraPath`.
|
||||
6. Прогнать endpoint check и smoke eval.
|
||||
@@ -0,0 +1,158 @@
|
||||
# Windows Docker GPU Host
|
||||
|
||||
Целевой хост: `docker-gpu.cin.su` / `192.168.220.91`.
|
||||
|
||||
Назначение: Windows-машина с NVIDIA GPU, Docker Desktop/Engine и контейнерами/моделями на диске `Z:`.
|
||||
|
||||
## Что сейчас видно с рабочей машины
|
||||
|
||||
- DNS `docker-gpu.cin.su` указывает на `192.168.220.91`.
|
||||
- Открыты SSH `22` и RDP `3389`.
|
||||
- Docker API `2375/2376`, WinRM `5985/5986`, Portainer `9000`, inference `8000/8080` снаружи не открыты.
|
||||
- SSH под `test@docker-gpu.cin.su` ключ не принимает: нужен правильный пользователь или добавление публичного ключа.
|
||||
|
||||
## Доступ
|
||||
|
||||
Предпочтительный вариант - SSH-ключ, без хранения паролей в репозитории.
|
||||
|
||||
Публичный ключ пользователя, который нужно добавить на Windows-хост:
|
||||
|
||||
```text
|
||||
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGS167Ors2oDzveI1iNlRiuGSqdR90x0O05JrcoQ4UBL llm-docker-gpu
|
||||
```
|
||||
|
||||
Для обычного Windows-пользователя ключ добавляется в:
|
||||
|
||||
```powershell
|
||||
C:\Users\<USER>\.ssh\authorized_keys
|
||||
```
|
||||
|
||||
Для пользователей из группы Administrators OpenSSH Server обычно читает:
|
||||
|
||||
```powershell
|
||||
C:\ProgramData\ssh\administrators_authorized_keys
|
||||
```
|
||||
|
||||
После добавления ключа права на файл администраторского ключа должны быть строгими:
|
||||
|
||||
```powershell
|
||||
icacls C:\ProgramData\ssh\administrators_authorized_keys /inheritance:r
|
||||
icacls C:\ProgramData\ssh\administrators_authorized_keys /grant "Administrators:F" /grant "SYSTEM:F"
|
||||
```
|
||||
|
||||
Проверка:
|
||||
|
||||
```powershell
|
||||
ssh <USER>@docker-gpu.cin.su "whoami; hostname"
|
||||
```
|
||||
|
||||
Для Codex настроен локальный SSH alias:
|
||||
|
||||
```powershell
|
||||
ssh docker-gpu "whoami & hostname"
|
||||
docker -H ssh://docker-gpu info
|
||||
```
|
||||
|
||||
## Проверка Windows GPU/Docker
|
||||
|
||||
На самом Windows-хосте:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/check_windows_gpu_host.ps1
|
||||
```
|
||||
|
||||
Через SSH, когда ключ уже добавлен:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/check_windows_gpu_host.ps1 -SshTarget <USER>@docker-gpu.cin.su
|
||||
```
|
||||
|
||||
Скрипт проверяет:
|
||||
|
||||
- `nvidia-smi`;
|
||||
- Docker Engine;
|
||||
- Docker Compose;
|
||||
- наличие/создание `Z:\LLM\models`;
|
||||
- запуск CUDA-контейнера с `--gpus all`.
|
||||
|
||||
## Docker Desktop в SSH-сессии
|
||||
|
||||
На Windows-хосте Docker Desktop должен быть запущен в интерактивной пользовательской сессии. Если `docker version` показывает клиент, но не может открыть pipe `dockerDesktopLinuxEngine`, запустите Docker Desktop в активной RDP/console-сессии пользователя.
|
||||
|
||||
Проверка через Docker-over-SSH:
|
||||
|
||||
```powershell
|
||||
docker -H ssh://docker-gpu version
|
||||
docker -H ssh://docker-gpu run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi
|
||||
```
|
||||
|
||||
Если `ssh docker-gpu "docker pull ..."` падает с ошибкой `A specified logon session does not exist`, причиной может быть `credsStore: desktop` в `%USERPROFILE%\.docker\config.json`. Для деплоя предпочтительно тянуть образы локальным Docker CLI через `docker -H ssh://docker-gpu ...`; такой режим не требует удаленного desktop credential helper.
|
||||
|
||||
## Автозапуск Docker Desktop
|
||||
|
||||
На `M7` настроена задача Windows Task Scheduler:
|
||||
|
||||
```text
|
||||
LLM-DockerDesktop-Autostart
|
||||
```
|
||||
|
||||
Задача запускает:
|
||||
|
||||
```text
|
||||
C:\ProgramData\LLM\Ensure-DockerDesktop.ps1
|
||||
```
|
||||
|
||||
Режим:
|
||||
|
||||
- при старте Windows с задержкой 2 минуты;
|
||||
- watchdog каждые 5 минут;
|
||||
- запуск от пользователя `m7\m` с повышенными правами;
|
||||
- пароль хранится в Windows Task Scheduler, не в репозитории и не в скрипте.
|
||||
|
||||
Лог:
|
||||
|
||||
```text
|
||||
C:\ProgramData\LLM\logs\docker-desktop-autostart.log
|
||||
```
|
||||
|
||||
Проверка задачи:
|
||||
|
||||
```powershell
|
||||
ssh docker-gpu "schtasks /Query /TN LLM-DockerDesktop-Autostart /V /FO LIST"
|
||||
ssh docker-gpu "cmd /c type C:\ProgramData\LLM\logs\docker-desktop-autostart.log"
|
||||
```
|
||||
|
||||
После проверочной перезагрузки `2026-06-19` задача подняла Docker Desktop автоматически, а GPU smoke test прошел через `docker -H ssh://docker-gpu`.
|
||||
|
||||
## Env-файлы под Z:
|
||||
|
||||
Для Windows-хоста подготовлены отдельные примеры:
|
||||
|
||||
```text
|
||||
core/deploy/docker-gpu/vllm/.env.windows-z.example
|
||||
core/deploy/docker-gpu/llama-cpp/.env.windows-z.example
|
||||
```
|
||||
|
||||
В них используются bind mount пути вида:
|
||||
|
||||
```text
|
||||
HOST_MODELS_DIR=Z:/LLM/models
|
||||
HOST_HF_CACHE_DIR=Z:/LLM/models/cache/huggingface
|
||||
```
|
||||
|
||||
Внутри контейнеров модели видны как `/models`.
|
||||
|
||||
## Следующий запуск
|
||||
|
||||
После появления SSH-доступа и подтверждения GPU:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_gpu_stack.ps1 `
|
||||
-SshTarget <USER>@docker-gpu.cin.su `
|
||||
-DockerHost ssh://<USER>@docker-gpu.cin.su `
|
||||
-VllmEnvFile core/deploy/docker-gpu/vllm/.env.windows-z.example `
|
||||
-LlamaEnvFile core/deploy/docker-gpu/llama-cpp/.env.windows-z.example `
|
||||
-Pull
|
||||
```
|
||||
|
||||
Если Docker Desktop на Windows не принимает remote Docker over SSH напрямую, запуск compose нужно выполнить интерактивно на самом хосте через RDP/SSH PowerShell, используя те же `.env.windows-z.example`.
|
||||
Reference in New Issue
Block a user