121 lines
3.9 KiB
Markdown
121 lines
3.9 KiB
Markdown
# 1C LoRA Training
|
|
|
|
Цель: обучить draft LoRA adapter `qwen3-coder-30b-a3b-1c-lora-v1` поверх `qwen3-coder-30b-a3b-instruct`.
|
|
|
|
## Preconditions
|
|
|
|
- Полностью скачана базовая модель: `/models/base/qwen3-coder-30b-a3b-instruct`.
|
|
- Подготовлен датасет: `plugins/1c/training/prepared/train.chat.jsonl`.
|
|
- Есть GPU/CUDA на `docker-gpu.cin.su`.
|
|
- В датасете достаточно проверенных примеров. Синтетические 2 записи подходят только для smoke-run, не для полезного качества.
|
|
|
|
## Current Preflight Status
|
|
|
|
На 2026-07-04 локальный preflight для нового Qwen3-Coder training contour не стартует, потому что:
|
|
|
|
- локально нет CUDA/GPU;
|
|
- training-зависимости не установлены в локальный Python;
|
|
- HF-база `qwen3-coder-30b-a3b-instruct` еще не лежит в `/models/base/qwen3-coder-30b-a3b-instruct` на текущем workspace path;
|
|
- запускать обучение нужно на `docker-gpu.cin.su`, потому что именно там есть GPU-контур для этой модели.
|
|
|
|
Если файл будет удален или поврежден, восстановить/докачать базовую модель можно так:
|
|
|
|
```powershell
|
|
python scripts/download_hf_range.py qwen3-coder-30b-a3b-instruct `
|
|
--local-dir models/base/qwen3-coder-30b-a3b-instruct `
|
|
--chunk-size 16mb `
|
|
--retries 20
|
|
```
|
|
|
|
## Prepare Dataset
|
|
|
|
```powershell
|
|
python scripts/validate_1c_training_data.py plugins/1c/training/examples/instruction.examples.jsonl
|
|
python scripts/prepare_1c_training_data.py --input plugins/1c/training/examples/instruction.examples.jsonl --output plugins/1c/training/prepared/train.chat.jsonl
|
|
```
|
|
|
|
## Local Preflight
|
|
|
|
```powershell
|
|
python scripts/preflight_1c_training.py
|
|
```
|
|
|
|
## Dry Run
|
|
|
|
```powershell
|
|
python scripts/train_1c_lora.py --dry-run
|
|
```
|
|
|
|
## GPU Docker Run
|
|
|
|
```powershell
|
|
docker --host ssh://docker-gpu.cin.su compose --env-file core/deploy/docker-gpu/training/1c-lora.env.example -f core/deploy/docker-gpu/training/1c-lora.compose.yaml up --abort-on-container-exit
|
|
```
|
|
|
|
Или через готовый wrapper:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/run_1c_lora_training_gpu.ps1
|
|
```
|
|
|
|
Full end-to-end orchestration for the current `Q6` route:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/train_and_publish_q6_lora.ps1
|
|
```
|
|
|
|
Preview the whole flow without executing:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/train_and_publish_q6_lora.ps1 -PlanOnly
|
|
```
|
|
|
|
Troubleshooting:
|
|
|
|
```text
|
|
docs/runbooks/q6-lora-troubleshooting.md
|
|
```
|
|
|
|
## Output
|
|
|
|
Adapter path:
|
|
|
|
```text
|
|
/models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1
|
|
```
|
|
|
|
After a successful training run:
|
|
|
|
1. Run `plugins/1c/evals/smoke.yaml`.
|
|
2. Convert the adapter for `llama.cpp` GGUF format or merge it before rebuilding the Q6 GGUF deployment artifact.
|
|
|
|
GGUF adapter export on `docker-gpu`:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/convert_1c_lora_to_gguf_gpu.ps1
|
|
```
|
|
|
|
The default output path is:
|
|
|
|
```text
|
|
/models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1.gguf
|
|
```
|
|
|
|
If the export container should reuse an existing `llama.cpp` checkout on the host without pulling updates:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/convert_1c_lora_to_gguf_gpu.ps1 -SkipClone
|
|
```
|
|
|
|
3. To launch the current GPU Q6 route with a converted adapter, use:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/manage_gpu_q6_service.ps1 `
|
|
-Action start `
|
|
-LoraPath /models/adapters/1c/qwen3-coder-30b-a3b-1c-lora-v1.gguf
|
|
```
|
|
|
|
If you need a custom adapter scale, pass `-LoraScale 0.5` or another value.
|
|
4. Compare base, RAG, and adapter outputs.
|
|
5. Promote model card from `draft` only after expert review.
|