Initial SQL-only 1C adapter baseline

This commit is contained in:
2026-07-22 03:03:47 +03:00
commit e2503b77e7
545 changed files with 184711 additions and 0 deletions
+55
View File
@@ -0,0 +1,55 @@
# Deploy llama.cpp GGUF
Цель: поднять GGUF-модель `Devstral Small 2 24B Instruct 2512 Q4_K_M` через `llama-server` на `docker-gpu.cin.su`.
## Model
- Repo: `bartowski/mistralai_Devstral-Small-2-24B-Instruct-2512-GGUF`
- File: `mistralai_Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf`
- Size: `14334438272` bytes
- Registry card: `registry/model-cards/devstral-small-2-24b-instruct-2512-q4_k_m.yaml`
## Download
Resume-safe local download:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/download_devstral_gguf.ps1
```
The script can be re-run after network failures.
## Compose
```text
core/deploy/docker-gpu/llama-cpp/compose.yaml
core/deploy/docker-gpu/llama-cpp/.env.example
```
## Run
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_llama_cpp.ps1 -Pull
```
Проверить compose без запуска:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_llama_cpp.ps1 -ConfigOnly
```
Остановить сервис:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_llama_cpp.ps1 -Down
```
## Check
```powershell
python scripts/check_inference_endpoint.py --base-url http://docker-gpu.cin.su:8080 --expected-model devstral-1c-q4 --print
```
```powershell
python scripts/smoke_chat.py --base-url http://docker-gpu.cin.su:8080 --model devstral-1c-q4
```