OpenAI-compatible AI gateway with Vue admin UI, multi-provider egress, ingress key governance, monitoring, and security controls. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
renjue
@@ -0,0 +1,34 @@
|
||||
# VCS / CI
|
||||
.git
|
||||
.gitignore
|
||||
.gitea
|
||||
|
||||
# Docs (not needed in image)
|
||||
*.md
|
||||
web/docs
|
||||
|
||||
# Local runtime / secrets
|
||||
data/
|
||||
.env
|
||||
.env.*
|
||||
|
||||
# Build artifacts
|
||||
luminary
|
||||
luminary-recover
|
||||
.docker-bin/
|
||||
*.db
|
||||
*.test
|
||||
coverage/
|
||||
|
||||
# Frontend (rebuilt in Dockerfile)
|
||||
web/node_modules
|
||||
web/dist
|
||||
|
||||
# IDE / editor
|
||||
.idea
|
||||
.vscode
|
||||
.cursor
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
@@ -0,0 +1,182 @@
|
||||
# Gitea Actions CI 模板（Go + Vue）
|
||||
|
||||
复制整个 `.gitea/` 目录到新仓库即可启用 CI：**可选 go test** + **Docker 构建/推送**（Go 编译与 Vue 构建在 Dockerfile 内完成）。
|
||||
|
||||
## 项目约定
|
||||
|
||||
| 路径 | 说明 |
|
||||
|------|------|
|
||||
| `go.mod` | 仓库根目录 |
|
||||
| `Dockerfile` | 仓库根目录，多阶段构建（含前端 `web/`） |
|
||||
| `web/` | 可选；由 Dockerfile 内 `npm run build` 处理，CI 不再单独构建 |
|
||||
|
||||
## Dockerfile 要求
|
||||
|
||||
工作流的 **Build image** 步骤在仓库根目录（或 Variable `DOCKERFILE` 指定路径）执行 `docker buildx build`，**不会**在 CI 里单独装 Node / 跑 `npm`。因此 Dockerfile 必须能独立完成构建与（运行时）启动。
|
||||
|
||||
### 基本要求
|
||||
|
||||
| 项 | 要求 |
|
||||
|----|------|
|
||||
| 位置 | 默认仓库根目录 `Dockerfile`；其他路径用 Variable `DOCKERFILE` |
|
||||
| 构建上下文 | 仓库根目录 `.`（整个仓库会被 `COPY` 进镜像） |
|
||||
| Go 版本 | `go.mod` 里 `go` 行与 `golang` 基础镜像大版本一致 |
|
||||
| 多架构 | `go build` 须使用 `GOARCH="$TARGETARCH"`（buildx 注入）；推荐 `CGO_ENABLED=0` |
|
||||
| Go 模块代理 | **必须**在镜像内显式设置 `GOPROXY` / `GOSUMDB`（见下）；容器内不会自动读取 `go.env` |
|
||||
| 基础镜像加速 | 支持构建参数 `IMAGE_PREFIX`（CI 默认 `docker.1panel.live/library/`） |
|
||||
| 前端（可选） | 存在 `web/` 时在 Dockerfile 内完成 `npm ci` + `npm run build` |
|
||||
|
||||
### CI 自动传入的构建参数
|
||||
|
||||
| 参数 | 默认 | 说明 |
|
||||
|------|------|------|
|
||||
| `IMAGE_PREFIX` | `docker.1panel.live/library/` | 基础镜像前缀；`hub` 表示 Docker Hub |
|
||||
| `GOPROXY` | `https://goproxy.cn,direct` | 与 workflow / Variable 一致 |
|
||||
| `GOSUMDB` | `sum.golang.google.cn` | checksum 数据库 |
|
||||
|
||||
buildx 还会注入 `TARGETARCH`、`BUILDPLATFORM` 等，无需在 workflow 里写。
|
||||
|
||||
### 推荐：Go + Vue 多阶段模板
|
||||
|
||||
```dockerfile
|
||||
# syntax=docker/dockerfile:1
|
||||
ARG IMAGE_PREFIX=
|
||||
|
||||
# 1) 前端（无 web/ 时可删整个 stage，并去掉 go-builder 里 COPY dist）
|
||||
FROM --platform=$BUILDPLATFORM ${IMAGE_PREFIX}node:20-alpine AS web-builder
|
||||
WORKDIR /src/web
|
||||
COPY web/package.json web/package-lock.json web/.npmrc ./
|
||||
RUN npm ci
|
||||
COPY web/ ./
|
||||
RUN npm run build
|
||||
|
||||
# 2) Go 编译
|
||||
FROM --platform=$BUILDPLATFORM ${IMAGE_PREFIX}golang:1.25-alpine AS go-builder
|
||||
ARG TARGETARCH
|
||||
ARG GOPROXY=https://goproxy.cn,direct
|
||||
ARG GOSUMDB=sum.golang.google.cn
|
||||
ENV GOPROXY=${GOPROXY} GOSUMDB=${GOSUMDB}
|
||||
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN --mount=type=cache,target=/go/pkg/mod go mod download
|
||||
|
||||
COPY cmd/ ./cmd/
|
||||
COPY internal/ ./internal/
|
||||
# 若静态资源嵌入 Go：COPY --from=web-builder /src/web/dist/ ./path/to/static/
|
||||
|
||||
RUN --mount=type=cache,target=/go/pkg/mod \
|
||||
--mount=type=cache,target=/root/.cache/go-build \
|
||||
CGO_ENABLED=0 GOOS=linux GOARCH="$TARGETARCH" \
|
||||
go build -ldflags="-w -s" -o /app ./cmd/yourapp
|
||||
|
||||
# 3) 运行镜像
|
||||
FROM ${IMAGE_PREFIX}alpine:3.20
|
||||
RUN apk add --no-cache ca-certificates tzdata
|
||||
COPY --from=go-builder /app /usr/local/bin/yourapp
|
||||
ENTRYPOINT ["/usr/local/bin/yourapp"]
|
||||
```
|
||||
|
||||
按项目调整：`./cmd/yourapp`、静态资源路径、运行用户、`EXPOSE` / `VOLUME` 等。
|
||||
|
||||
### 仅 Go（无前端）
|
||||
|
||||
删除 `web-builder` stage；`go-builder` 中不要 `COPY` 前端产物；其余 `GOPROXY` / `TARGETARCH` / `IMAGE_PREFIX` 要求相同。
|
||||
|
||||
### 前端 npm 源（国内）
|
||||
|
||||
在 `web/.npmrc` 配置 registry，并在 Dockerfile 里与 `package.json` 一并 `COPY`：
|
||||
|
||||
```ini
|
||||
registry=https://registry.npmmirror.com
|
||||
```
|
||||
|
||||
### 本地验证（与 CI 一致）
|
||||
|
||||
```bash
|
||||
docker buildx build --platform linux/amd64 \
|
||||
--build-arg IMAGE_PREFIX=docker.1panel.live/library/ \
|
||||
--build-arg GOPROXY=https://goproxy.cn,direct \
|
||||
-f Dockerfile -t myapp:test .
|
||||
```
|
||||
|
||||
### 常见 Dockerfile 构建错误
|
||||
|
||||
| 报错 | 原因 | 处理 |
|
||||
|------|------|------|
|
||||
| `proxy.golang.org` timeout | 镜像内未设 `ENV GOPROXY` | go-builder 阶段加 `ARG`/`ENV GOPROXY` |
|
||||
| `node` / Vite 版本不符 | 基础镜像 Node 过旧 | 使用 `node:20-alpine` 及以上 |
|
||||
| 某架构 build 失败 | 未使用 `TARGETARCH` | `GOARCH="$TARGETARCH"` |
|
||||
| 拉基础镜像 timeout | Hub 不可达 | `--build-arg IMAGE_PREFIX=docker.1panel.live/library/` 或 1Panel 配镜像加速 |
|
||||
|
||||
## 一次性配置
|
||||
|
||||
1. **Runner**：部署 act_runner，标签含 `ubuntu-latest`，并挂载 `docker.sock`（见 [act-runner/README.md](act-runner/README.md)）。
|
||||
2. **Secret**：仓库 Settings → Actions → Secrets，添加 `REGISTRY_TOKEN`（PAT，`write:package` 权限）。
|
||||
3. **Variable（推荐）**：若 runner 与 Gitea 同机、或 `gitea.server_url` 为内网地址，必须设置 `REGISTRY=你的公网域名`（如 `git.example.com`，**不要**填 `172.17.0.1:13827`）。
|
||||
4. **Gitea Registry**：服务端 `[packages] ENABLED = true`，`ROOT_URL` 正确；穿透场景建议 `PUBLIC_URL_DETECTION = never`（Gitea 1.26+）。
|
||||
|
||||
`REGISTRY` 未设置时，会从 `GITEA_ROOT_URL` 或 `gitea.server_url` 推断；均为内网地址时 workflow 会提前失败并提示。
|
||||
|
||||
## 可选 Variables
|
||||
|
||||
仓库 Settings → Actions → Variables（留空则用默认值）：
|
||||
|
||||
| 变量 | 默认 | 说明 |
|
||||
|------|------|------|
|
||||
| `REGISTRY` | 见下方推断顺序 | **公网** Registry 主机名，如 `git.example.com`（勿用内网 IP:13827） |
|
||||
| `GITEA_ROOT_URL` | （空） | 当 `gitea.server_url` 为内网时，可设 `https://git.example.com/` |
|
||||
| `IMAGE_NAME` | 仓库名小写 | 镜像名，非 owner/repo 全路径 |
|
||||
| `DOCKERFILE` | `Dockerfile` | Dockerfile 路径 |
|
||||
| `DOCKER_PLATFORMS` | `linux/amd64,linux/arm64` | push 时 buildx 平台 |
|
||||
| `GO_TEST_SCOPE` | `./...` | `go test` 包路径 |
|
||||
| `RUN_GO_TEST` | （空，即运行） | 设 `false` 跳过 go test，仅 Docker 构建 |
|
||||
| `DOCKER_IMAGE_PREFIX` | `1panel` | 基础镜像前缀；可选 `hub` / `daocloud` |
|
||||
| `GOPROXY` | `https://goproxy.cn,direct` | Go 模块代理 |
|
||||
| `GOSUMDB` | `sum.golang.google.cn` | Go checksum 数据库 |
|
||||
|
||||
## 触发与镜像 tag
|
||||
|
||||
- **pull_request**：go test（可关）+ 单架构 `docker build` 验证（不推送）
|
||||
- **push main/master**：go test + 多架构构建并推送 `:latest`、`:sha-xxxxxxx`
|
||||
- **push tag v\***：额外推送 `:v1.2.3` 等
|
||||
|
||||
示例（仓库 `rose_cat707/Prism`，Gitea 在 `git.example.com`）：
|
||||
|
||||
```text
|
||||
git.example.com/rose_cat707/prism:latest
|
||||
git.example.com/rose_cat707/prism:sha-35b3b48
|
||||
```
|
||||
|
||||
## 目录结构
|
||||
|
||||
```text
|
||||
.gitea/
|
||||
├── README.md # 本文件
|
||||
├── workflows/
|
||||
│ └── ci.yml # 主工作流
|
||||
└── act-runner/ # runner 部署参考（可选）
|
||||
├── README.md
|
||||
├── config.yaml
|
||||
├── docker-compose.yml
|
||||
└── .env.example
|
||||
```
|
||||
|
||||
## 本地验证 Registry
|
||||
|
||||
```bash
|
||||
host=$(echo "https://你的-gitea-地址/" | sed -e 's|^https://||' -e 's|/.*||')
|
||||
curl -s -D - "https://${host}/v2/" -o /dev/null | grep -i www-authenticate
|
||||
docker pull "${host}/owner/image:latest" # 公开 Registry 无需 login
|
||||
```
|
||||
|
||||
推送镜像（CI）仍需仓库 Secret `REGISTRY_TOKEN`（`write:package`）。仅拉取公开包不需要登录。
|
||||
|
||||
`realm` 应指向公网 Gitea 域名，而非 `127.0.0.1`。
|
||||
|
||||
## 常见 Registry 错误
|
||||
|
||||
| 报错 | 原因 | 处理 |
|
||||
|------|------|------|
|
||||
| `Get "https://172.17.0.1:13827/v2/"` HTTP/HTTPS | Variable `REGISTRY` 或 `gitea.server_url` 为内网地址 | 设 `REGISTRY=git.example.com` |
|
||||
| token 指向 `127.0.0.1` | Gitea `realm` 配置错误 | `PUBLIC_URL_DETECTION=never` + 正确 `ROOT_URL` |
|
||||
@@ -0,0 +1,3 @@
|
||||
GITEA_INSTANCE_URL=https://git.example.com
|
||||
GITEA_RUNNER_REGISTRATION_TOKEN=从仓库_Settings_Actions_Runners_复制
|
||||
GITEA_RUNNER_NAME=go-vue-ci-runner
|
||||
@@ -0,0 +1,177 @@
|
||||
# act_runner 配置参考
|
||||
|
||||
工作流 `runs-on: ubuntu-latest`；`docker` job 额外使用 `catthehacker/ubuntu:act-22.04` 容器（含 Docker CLI）。
|
||||
|
||||
## 构建镜像必须：挂载 Docker Socket
|
||||
|
||||
`docker` job 需要访问宿主机 Docker 引擎。在 **runner 所在机器** 的 `config.yaml` 中配置：
|
||||
|
||||
```yaml
|
||||
container:
|
||||
options: -v /var/run/docker.sock:/var/run/docker.sock
|
||||
valid_volumes:
|
||||
- /var/run/docker.sock
|
||||
```
|
||||
|
||||
推荐 job 镜像（含 Node + Docker CLI）：
|
||||
|
||||
```yaml
|
||||
runner:
|
||||
labels:
|
||||
- "ubuntu-latest:docker://catthehacker/ubuntu:act-22.04"
|
||||
- "ubuntu-22.04:docker://catthehacker/ubuntu:act-22.04"
|
||||
```
|
||||
|
||||
修改后 **重启 runner**（例如 `docker restart <runner容器>` 或重启 Gitea Runner 服务）。
|
||||
|
||||
## 1Panel 宿主机：配置 Docker 镜像加速（推荐）
|
||||
|
||||
Runner 通过 `docker.sock` 使用宿主机 Docker。在 **1Panel** 中配置加速器后，普通 `docker pull` 会走加速；**buildx 多架构构建**仍建议配合 workflow 内的 `IMAGE_PREFIX`（默认 `docker.1panel.live`）。
|
||||
|
||||
1. 登录 1Panel → **容器** → **配置**
|
||||
2. **镜像加速地址** 填入：
|
||||
```text
|
||||
https://docker.1panel.live
|
||||
```
|
||||
3. 保存并 **重启 Docker**
|
||||
|
||||
验证（在 runner 宿主机）：
|
||||
|
||||
```bash
|
||||
docker info | grep -A5 'Registry Mirrors'
|
||||
docker pull alpine:3.20
|
||||
```
|
||||
|
||||
等价 `daemon.json`：
|
||||
|
||||
```json
|
||||
{
|
||||
"registry-mirrors": ["https://docker.1panel.live"]
|
||||
}
|
||||
```
|
||||
|
||||
CI 工作流默认 `IMAGE_PREFIX=docker.1panel.live/library/`（buildkit 拉基础镜像）。若 1Panel 加速不可用，可在仓库 Variables 设 `DOCKER_IMAGE_PREFIX=daocloud` 或 `hub`。
|
||||
|
||||
参考：[1Panel 容器配置文档](https://1panel.cn/docs/user_manual/containers/setting)
|
||||
|
||||
## 验证
|
||||
|
||||
在 runner 宿主机执行：
|
||||
|
||||
```bash
|
||||
docker info
|
||||
ls -l /var/run/docker.sock
|
||||
```
|
||||
|
||||
## 从零部署 runner（可选）
|
||||
|
||||
```bash
|
||||
cd .gitea/act-runner
|
||||
cp .env.example .env # 填入 Registration Token
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
## GitHub Actions 镜像（替代 ghfast）
|
||||
|
||||
日志里出现 `git clone 'https://ghfast.top/https://github.com/actions/checkout'` 说明 **runner 宿主机** 的 `config.yaml` 配置了 `github_mirror`，与仓库 workflow 无关。
|
||||
|
||||
在 runner 的 `config.yaml` 中修改（修改后重启 runner）：
|
||||
|
||||
```yaml
|
||||
runner:
|
||||
github_mirror: 'https://gitea.com' # 推荐
|
||||
```
|
||||
|
||||
常用替代方案：
|
||||
|
||||
| `github_mirror` 值 | 说明 |
|
||||
|------------------|------|
|
||||
| `''`（留空） | 直连 `github.com`，网络可达时最简单 |
|
||||
| `https://gitea.com` | Gitea 官方 actions 镜像，国内较稳 |
|
||||
| `https://gitclone.com/github.com` | 第三方 GitHub 克隆镜像 |
|
||||
| `https://ghfast.top/https://github.com` | 部分环境需代理认证，易报 `Proxy Authentication Required` |
|
||||
|
||||
前提：Gitea `app.ini` 中 `[actions] DEFAULT_ACTIONS_URL = github`（默认）。
|
||||
|
||||
**不依赖 runner 镜像** 的写法（workflow 内写绝对 URL，仅改写 `github.com` 的请求）：
|
||||
|
||||
```yaml
|
||||
uses: https://gitea.com/actions/checkout@v4
|
||||
```
|
||||
|
||||
本模板 `ci.yml` 已采用此写法。若其他 workflow 仍写 `uses: actions/checkout@v4`，仍需配置 `github_mirror` 或改为绝对 URL。
|
||||
|
||||
修改 `github_mirror` 后建议清理 runner 缓存目录（如 `/root/.cache/act`），否则旧缓存可能仍指向 ghfast。
|
||||
|
||||
## 常见错误
|
||||
|
||||
| 报错 | 原因 | 处理 |
|
||||
|------|------|------|
|
||||
| `registry-1.docker.io` i/o timeout | Docker Hub 不可达 | 1Panel 配 `docker.1panel.live`；Variable `DOCKER_IMAGE_PREFIX=1panel` |
|
||||
| `ghfast.top` Proxy Authentication Required | runner `github_mirror` 指向 ghfast 且需代理认证 | 改 `github_mirror` 或删掉；见上文「GitHub Actions 镜像」 |
|
||||
| `docker: command not found` | job 容器无 Docker CLI | 工作流已指定 act 镜像；或 runner 改用 catthehacker/ubuntu |
|
||||
| `Cannot connect to Docker daemon` | 未挂载 docker.sock | 按上文修改 config.yaml 并重启 runner |
|
||||
| `node not in PATH` | job 镜像无 Node | 标签映射改用 catthehacker/ubuntu:act-22.04 |
|
||||
| `http: server gave HTTP response to HTTPS client` 且 token 指向 `127.0.0.1` | **Gitea Registry 配置/反代错误** | 见下文「Registry 登录失败」 |
|
||||
|
||||
## Registry 登录失败（127.0.0.1 / HTTP vs HTTPS）
|
||||
|
||||
若 `docker login git.rc707blog.top` 报错类似：
|
||||
|
||||
```text
|
||||
Get "https://127.0.0.1:xxxxx/v2/token?...": http: server gave HTTP response to HTTPS client
|
||||
```
|
||||
|
||||
说明 Gitea 把 **Docker 认证 token 地址** 配成了本机内网地址，CI runner 访问不到。需在 **Gitea 服务器** 修复，而非改 workflow。
|
||||
|
||||
### 1. 检查 `app.ini`
|
||||
|
||||
```ini
|
||||
[server]
|
||||
ROOT_URL = https://git.example.com/
|
||||
LOCAL_ROOT_URL = http://127.0.0.1:3000/
|
||||
; 内网穿透 / 错误 Host 时（Gitea 1.26+）：
|
||||
; PUBLIC_URL_DETECTION = never
|
||||
|
||||
[packages]
|
||||
ENABLED = true
|
||||
```
|
||||
|
||||
`ROOT_URL` 必须与浏览器访问 Gitea 的 **HTTPS 外网地址** 完全一致（含末尾 `/`）。
|
||||
|
||||
### 2. 反向代理必须转发 `/v2` 并带上头
|
||||
|
||||
Container Registry 固定使用根路径 `/v2`。Nginx 示例：
|
||||
|
||||
```nginx
|
||||
location / {
|
||||
client_max_body_size 0;
|
||||
proxy_pass http://127.0.0.1:3000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
```
|
||||
|
||||
关键：`X-Forwarded-Proto: https` 和正确的 `Host`（与 Gitea `ROOT_URL` 域名一致）。
|
||||
|
||||
### 3. 在 runner 宿主机验证（修复后）
|
||||
|
||||
```bash
|
||||
GITEA_HOST=git.example.com # 改成你的 Gitea 域名
|
||||
curl -s -D - "https://${GITEA_HOST}/v2/" -o /dev/null | grep -i www-authenticate
|
||||
echo "$REGISTRY_TOKEN" | docker login "${GITEA_HOST}" -u 你的用户名 --password-stdin
|
||||
```
|
||||
|
||||
应返回 `401 Unauthorized`（正常，表示 registry 可达）且 `docker login` 显示 **Login Succeeded**。
|
||||
|
||||
参考：[Gitea 反向代理文档](https://docs.gitea.com/administration/reverse-proxies)
|
||||
|
||||
## Gitea Runner v0.6.x（个人 runner）
|
||||
|
||||
1. 找到 runner 的配置文件或环境（安装目录 / docker compose）
|
||||
2. 确保 runner 进程能访问宿主机 `/var/run/docker.sock`
|
||||
3. Runners 页标签含 `ubuntu-latest` 且状态 **空闲/在线**
|
||||
|
||||
若使用 Gitea 网页注册的个人 runner（docker 模式），通常需在 runner 启动参数或 `config.yaml` 里加入 socket 挂载，具体路径取决于你的安装方式。
|
||||
@@ -0,0 +1,34 @@
|
||||
# act_runner 配置（可选参考部署）
|
||||
# 工作流 runs-on: ubuntu-latest
|
||||
|
||||
log:
|
||||
level: info
|
||||
|
||||
runner:
|
||||
file: .runner
|
||||
capacity: 2
|
||||
timeout: 3h
|
||||
insecure: false
|
||||
fetch_timeout: 5s
|
||||
fetch_interval: 2s
|
||||
# 拉取 uses: actions/checkout@v4 等 GitHub Action 时的镜像（替换 https://github.com）
|
||||
# 需 Gitea app.ini 中 [actions] DEFAULT_ACTIONS_URL = github
|
||||
# 留空则直连 github.com；第三方镜像不稳定时可改用 workflow 绝对 URL（见 .gitea/README.md）
|
||||
#
|
||||
# github_mirror: '' # 直连 GitHub
|
||||
# github_mirror: 'https://gitea.com' # 推荐：Gitea 官方 actions 镜像
|
||||
# github_mirror: 'https://gitclone.com/github.com' # 第三方 GitHub 克隆镜像
|
||||
# github_mirror: 'https://ghfast.top/https://github.com' # 需代理认证时易失败，不推荐
|
||||
labels:
|
||||
- "ubuntu-latest:docker://catthehacker/ubuntu:act-22.04"
|
||||
- "ubuntu-22.04:docker://catthehacker/ubuntu:act-22.04"
|
||||
|
||||
cache:
|
||||
enabled: false
|
||||
|
||||
container:
|
||||
network: bridge
|
||||
privileged: false
|
||||
options: -v /var/run/docker.sock:/var/run/docker.sock
|
||||
valid_volumes:
|
||||
- /var/run/docker.sock
|
||||
@@ -0,0 +1,22 @@
|
||||
# Gitea act_runner — 可选参考部署（标签 default，与工作流一致）
|
||||
#
|
||||
# 若已在 Gitea 注册个人/仓库 runner 且标签为 default，无需使用本目录。
|
||||
|
||||
services:
|
||||
act-runner:
|
||||
image: docker.io/gitea/act_runner:0.2.12
|
||||
container_name: prism-act-runner
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
CONFIG_FILE: /config.yaml
|
||||
GITEA_INSTANCE_URL: ${GITEA_INSTANCE_URL:-https://git.rc707blog.top}
|
||||
GITEA_RUNNER_REGISTRATION_TOKEN: ${GITEA_RUNNER_REGISTRATION_TOKEN}
|
||||
GITEA_RUNNER_NAME: ${GITEA_RUNNER_NAME:-prism-ci-runner}
|
||||
volumes:
|
||||
- ./config.yaml:/config.yaml:ro
|
||||
- act-runner-data:/data
|
||||
- /var/run/docker.sock:/var/run/docker.sock
|
||||
working_dir: /data
|
||||
|
||||
volumes:
|
||||
act-runner-data:
|
||||
@@ -0,0 +1,236 @@
|
||||
# 通用 Go + Vue 项目 CI（Gitea Actions）
|
||||
#
|
||||
# 单 job：可选 go test + Docker 多架构构建/推送
|
||||
# 约定：go.mod、Dockerfile（详见 .gitea/README.md「Dockerfile 要求」）；前端（可选）在 web/
|
||||
#
|
||||
# 前置：act_runner（ubuntu-latest + docker.sock）、Secret REGISTRY_TOKEN
|
||||
# Variables：REGISTRY、IMAGE_NAME、DOCKER_IMAGE_PREFIX、RUN_GO_TEST 等
|
||||
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main, master]
|
||||
tags: ['v*']
|
||||
pull_request:
|
||||
|
||||
env:
|
||||
REGISTRY: ${{ vars.REGISTRY }}
|
||||
GITEA_ROOT_URL: ${{ vars.GITEA_ROOT_URL }}
|
||||
IMAGE_NAME: ${{ vars.IMAGE_NAME }}
|
||||
DOCKERFILE: ${{ vars.DOCKERFILE }}
|
||||
DOCKER_PLATFORMS: ${{ vars.DOCKER_PLATFORMS }}
|
||||
GO_TEST_SCOPE: ${{ vars.GO_TEST_SCOPE }}
|
||||
RUN_GO_TEST: ${{ vars.RUN_GO_TEST }}
|
||||
DOCKER_IMAGE_PREFIX: ${{ vars.DOCKER_IMAGE_PREFIX }}
|
||||
GOPROXY: ${{ vars.GOPROXY }}
|
||||
GOSUMDB: ${{ vars.GOSUMDB }}
|
||||
|
||||
jobs:
|
||||
docker:
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: catthehacker/ubuntu:act-22.04
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: https://gitea.com/actions/checkout@v4
|
||||
|
||||
- name: Run Go tests
|
||||
if: vars.RUN_GO_TEST != 'false'
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
export GOPROXY="${GOPROXY:-https://goproxy.cn,direct}"
|
||||
export GOSUMDB="${GOSUMDB:-sum.golang.google.cn}"
|
||||
|
||||
GO_VERSION=$(grep '^go ' go.mod | awk '{print $2}')
|
||||
ARCH=$(uname -m | sed 's/x86_64/amd64/; s/aarch64/arm64/')
|
||||
TAR="go${GO_VERSION}.linux-${ARCH}.tar.gz"
|
||||
|
||||
if ! command -v go >/dev/null 2>&1 || ! go version | grep -q "go${GO_VERSION} "; then
|
||||
downloaded=0
|
||||
for url in \
|
||||
"https://mirrors.aliyun.com/golang/${TAR}" \
|
||||
"https://golang.google.cn/dl/${TAR}" \
|
||||
"https://go.dev/dl/${TAR}"; do
|
||||
echo "trying ${url}"
|
||||
if curl -fsSL --connect-timeout 20 --retry 3 --retry-delay 5 --max-time 600 \
|
||||
"$url" -o /tmp/go.tgz; then
|
||||
downloaded=1
|
||||
break
|
||||
fi
|
||||
done
|
||||
[ "$downloaded" -eq 1 ] || { echo "failed to download Go ${GO_VERSION}" >&2; exit 1; }
|
||||
rm -rf /usr/local/go
|
||||
tar -C /usr/local -xzf /tmp/go.tgz
|
||||
export PATH="/usr/local/go/bin:${PATH}"
|
||||
fi
|
||||
go version
|
||||
go test "${GO_TEST_SCOPE:-./...}"
|
||||
|
||||
- name: Setup Docker
|
||||
run: |
|
||||
set -eux
|
||||
if ! command -v docker >/dev/null 2>&1; then
|
||||
apt-get update
|
||||
DEBIAN_FRONTEND=noninteractive apt-get install -y docker.io
|
||||
fi
|
||||
if [ ! -S /var/run/docker.sock ]; then
|
||||
echo "ERROR: /var/run/docker.sock 未挂载到 job 容器。" >&2
|
||||
echo "请在 act_runner config.yaml 中配置：" >&2
|
||||
echo " container.options: -v /var/run/docker.sock:/var/run/docker.sock" >&2
|
||||
echo " container.valid_volumes: [/var/run/docker.sock]" >&2
|
||||
exit 1
|
||||
fi
|
||||
docker info
|
||||
|
||||
- name: Resolve registry
|
||||
id: reg
|
||||
if: gitea.event_name == 'push'
|
||||
shell: bash
|
||||
env:
|
||||
GITEA_SERVER_URL: ${{ gitea.server_url }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
url_to_host() {
|
||||
echo "$1" | sed -e 's|^https://||' -e 's|^http://||' -e 's|/.*||'
|
||||
}
|
||||
|
||||
is_internal_host() {
|
||||
local host="${1%%:*}"
|
||||
case "$host" in
|
||||
localhost|127.*|10.*|192.168.*) return 0 ;;
|
||||
172.*)
|
||||
local second
|
||||
second=$(echo "$host" | cut -d. -f2)
|
||||
[ "$second" -ge 16 ] && [ "$second" -le 31 ]
|
||||
return
|
||||
;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
registry="${REGISTRY:-}"
|
||||
if [ -z "$registry" ] && [ -n "${GITEA_ROOT_URL:-}" ]; then
|
||||
registry=$(url_to_host "${GITEA_ROOT_URL}")
|
||||
fi
|
||||
if [ -z "$registry" ]; then
|
||||
registry=$(url_to_host "${GITEA_SERVER_URL}")
|
||||
fi
|
||||
|
||||
if is_internal_host "$registry"; then
|
||||
echo "ERROR: Registry 主机 '${registry}' 是内网地址。" >&2
|
||||
echo "请设置 Variable REGISTRY=公网 Gitea 域名（如 git.example.com）。" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "registry host: ${registry}"
|
||||
|
||||
auth_header=""
|
||||
if auth_header=$(curl -fsS -D - "https://${registry}/v2/" -o /dev/null 2>&1 | grep -i '^www-authenticate:'); then
|
||||
if echo "$auth_header" | grep -qiE '127\.0\.0\.1|172\.(1[6-9]|2[0-9]|3[01])\.|localhost|:13827'; then
|
||||
echo "ERROR: Registry token realm 指向内网地址：${auth_header}" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "host=${registry}" >> "${GITHUB_OUTPUT}"
|
||||
|
||||
- name: Log in to Container Registry
|
||||
if: gitea.event_name == 'push'
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
echo "${{ secrets.REGISTRY_TOKEN }}" | \
|
||||
docker login "${{ steps.reg.outputs.host }}" -u "${{ gitea.actor }}" --password-stdin
|
||||
|
||||
- name: Build image
|
||||
shell: bash
|
||||
env:
|
||||
GITEA_SHA: ${{ gitea.sha }}
|
||||
GITEA_REF: ${{ gitea.ref }}
|
||||
GITEA_REPOSITORY: ${{ gitea.repository }}
|
||||
GITEA_REPOSITORY_OWNER: ${{ gitea.repository_owner }}
|
||||
GITEA_EVENT_NAME: ${{ gitea.event_name }}
|
||||
REGISTRY_HOST: ${{ steps.reg.outputs.host }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
dockerfile="${DOCKERFILE:-Dockerfile}"
|
||||
|
||||
if [ "${DOCKER_IMAGE_PREFIX:-}" = "hub" ] || [ "${DOCKER_IMAGE_PREFIX:-}" = "docker.io" ]; then
|
||||
image_prefix=""
|
||||
elif [ -z "${DOCKER_IMAGE_PREFIX:-}" ] || [ "${DOCKER_IMAGE_PREFIX}" = "1panel" ]; then
|
||||
image_prefix="docker.1panel.live/library/"
|
||||
elif [ "${DOCKER_IMAGE_PREFIX}" = "daocloud" ]; then
|
||||
image_prefix="docker.m.daocloud.io/library/"
|
||||
else
|
||||
image_prefix="${DOCKER_IMAGE_PREFIX}"
|
||||
[[ "${image_prefix}" == */ ]] || image_prefix="${image_prefix}/"
|
||||
fi
|
||||
echo "using IMAGE_PREFIX=${image_prefix:-<docker.io>}"
|
||||
build_args=(
|
||||
--build-arg "IMAGE_PREFIX=${image_prefix}"
|
||||
--build-arg "GOPROXY=${GOPROXY:-https://goproxy.cn,direct}"
|
||||
--build-arg "GOSUMDB=${GOSUMDB:-sum.golang.google.cn}"
|
||||
)
|
||||
|
||||
builder_id="gitea-buildx-${GITEA_REPOSITORY//\//-}"
|
||||
|
||||
if [ "${GITEA_EVENT_NAME}" = "push" ]; then
|
||||
install_binfmt() {
|
||||
for img in \
|
||||
"docker.1panel.live/tonistiigi/binfmt:latest" \
|
||||
"docker.m.daocloud.io/tonistiigi/binfmt:latest" \
|
||||
"tonistiigi/binfmt:latest"; do
|
||||
echo "trying binfmt image: ${img}"
|
||||
if docker run --privileged --rm "${img}" --install all; then
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
install_binfmt || true
|
||||
|
||||
registry="${REGISTRY_HOST}"
|
||||
image_name="${IMAGE_NAME:-$(echo "${GITEA_REPOSITORY##*/}" | tr '[:upper:]' '[:lower:]')}"
|
||||
platforms="${DOCKER_PLATFORMS:-linux/amd64,linux/arm64}"
|
||||
image="${registry}/${GITEA_REPOSITORY_OWNER}/${image_name}"
|
||||
tags="${image}:sha-${GITEA_SHA:0:7}"
|
||||
case "${GITEA_REF}" in
|
||||
refs/heads/main|refs/heads/master)
|
||||
tags="${tags},${image}:latest"
|
||||
;;
|
||||
refs/tags/v*)
|
||||
tags="${tags},${image}:${GITEA_REF#refs/tags/}"
|
||||
;;
|
||||
esac
|
||||
|
||||
docker buildx create --name "${builder_id}" --use 2>/dev/null || docker buildx use "${builder_id}"
|
||||
docker buildx inspect --bootstrap
|
||||
|
||||
tag_args=()
|
||||
IFS=',' read -r -a tag_list <<< "$tags"
|
||||
for t in "${tag_list[@]}"; do
|
||||
tag_args+=(-t "$t")
|
||||
done
|
||||
|
||||
docker buildx build \
|
||||
--platform "${platforms}" \
|
||||
-f "${dockerfile}" \
|
||||
"${build_args[@]}" \
|
||||
"${tag_args[@]}" \
|
||||
--push \
|
||||
.
|
||||
else
|
||||
# PR：仅单架构构建验证 Dockerfile，不推送
|
||||
docker buildx create --name "${builder_id}" --use 2>/dev/null || docker buildx use "${builder_id}"
|
||||
docker buildx inspect --bootstrap
|
||||
docker buildx build \
|
||||
--platform linux/amd64 \
|
||||
-f "${dockerfile}" \
|
||||
"${build_args[@]}" \
|
||||
--load \
|
||||
.
|
||||
fi
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
# Binaries
|
||||
/luminary
|
||||
/luminary-recover
|
||||
*.exe
|
||||
*.exe~
|
||||
*.dll
|
||||
*.so
|
||||
*.dylib
|
||||
*.test
|
||||
|
||||
# Go
|
||||
go.work
|
||||
go.work.sum
|
||||
*.out
|
||||
coverage/
|
||||
*.coverprofile
|
||||
|
||||
# Luminary runtime data
|
||||
data/
|
||||
/data/
|
||||
*.db
|
||||
|
||||
# Docker local prebuilt
|
||||
.docker-bin/
|
||||
|
||||
# Frontend build output (keep placeholder for go:embed in CI / go test)
|
||||
web/dist/*
|
||||
!web/dist/index.html
|
||||
web/node_modules/
|
||||
|
||||
# Environment
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
|
||||
# IDE / editor
|
||||
.idea/
|
||||
.vscode/
|
||||
.cursor/
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
# Luminary AI Gateway（Go + Vue 管理台）
|
||||
#
|
||||
# 构建:
|
||||
# docker build -t luminary:latest .
|
||||
#
|
||||
# 运行:
|
||||
# docker run -d --name luminary \
|
||||
# -p 8293:8293 \
|
||||
# -v luminary-data:/data \
|
||||
# luminary:latest
|
||||
#
|
||||
# 生产镜像由 Gitea Actions 构建，见 README 与 .gitea/README.md
|
||||
|
||||
# syntax=docker/dockerfile:1
|
||||
|
||||
# 基础镜像默认 Docker Hub；CI 默认 1Panel 加速（docker.1panel.live），本地可传：
|
||||
# docker build --build-arg IMAGE_PREFIX=docker.1panel.live/library/ -t luminary:latest .
|
||||
ARG IMAGE_PREFIX=
|
||||
|
||||
FROM --platform=$BUILDPLATFORM ${IMAGE_PREFIX}node:20-alpine AS web-builder
|
||||
|
||||
WORKDIR /src/web
|
||||
COPY web/package.json web/package-lock.json web/.npmrc ./
|
||||
RUN npm ci
|
||||
COPY web/src ./src
|
||||
COPY web/public ./public
|
||||
COPY web/index.html web/tsconfig.json web/vite.config.ts ./
|
||||
RUN npm run build
|
||||
|
||||
FROM --platform=$BUILDPLATFORM ${IMAGE_PREFIX}golang:1.25-alpine AS go-builder
|
||||
|
||||
ARG TARGETARCH=amd64
|
||||
ARG GOPROXY=https://goproxy.cn,direct
|
||||
ARG GOSUMDB=sum.golang.google.cn
|
||||
ENV CGO_ENABLED=0 GOPROXY=${GOPROXY} GOSUMDB=${GOSUMDB}
|
||||
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum ./
|
||||
RUN --mount=type=cache,target=/go/pkg/mod \
|
||||
go mod download
|
||||
|
||||
COPY cmd/ ./cmd/
|
||||
COPY internal/ ./internal/
|
||||
COPY web/static_embed.go ./web/
|
||||
COPY --from=web-builder /src/web/dist ./web/dist
|
||||
|
||||
RUN --mount=type=cache,target=/go/pkg/mod \
|
||||
--mount=type=cache,target=/root/.cache/go-build \
|
||||
GOOS=linux GOARCH="$TARGETARCH" \
|
||||
go build -trimpath -ldflags="-w -s" -o /luminary ./cmd/luminary \
|
||||
&& go build -trimpath -ldflags="-w -s" -o /luminary-recover ./cmd/luminary-recover
|
||||
|
||||
ARG IMAGE_PREFIX=
|
||||
FROM ${IMAGE_PREFIX}alpine:3.20
|
||||
|
||||
ARG APK_MIRROR=mirrors.aliyun.com
|
||||
RUN sed -i "s/dl-cdn.alpinelinux.org/${APK_MIRROR}/g" /etc/apk/repositories \
|
||||
&& apk add --no-cache ca-certificates tzdata tini
|
||||
|
||||
ENV LUMINARY_DATA=/data \
|
||||
LUMINARY_ADDR=:8293
|
||||
|
||||
COPY --from=go-builder /luminary /usr/local/bin/luminary
|
||||
COPY --from=go-builder /luminary-recover /usr/local/bin/luminary-recover
|
||||
COPY docker/entrypoint.sh /entrypoint.sh
|
||||
COPY scripts/luminary-recover.sh /usr/local/bin/luminary-recover.sh
|
||||
RUN chmod +x /entrypoint.sh /usr/local/bin/luminary-recover.sh
|
||||
|
||||
VOLUME ["/data"]
|
||||
EXPOSE 8293
|
||||
ENTRYPOINT ["/sbin/tini", "--", "/entrypoint.sh"]
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Luminary Contributors
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,63 @@
|
||||
.PHONY: dev build build-web run tidy test recover docker-build docker-push docker-push-prebuilt
|
||||
|
||||
export GOPROXY ?= https://goproxy.cn,direct
|
||||
export GOSUMDB ?= sum.golang.google.cn
|
||||
export GOPRIVATE ?= git.rc707blog.top
|
||||
export IMAGE_PREFIX ?= docker.1panel.live/library/
|
||||
export APK_MIRROR ?= mirrors.aliyun.com
|
||||
export DOCKER_PLATFORM ?= linux/amd64
|
||||
export LUMINARY_IMAGE ?= registry.rc707blog.top/rose_cat707/luminary:latest
|
||||
|
||||
dev:
|
||||
@echo "Start backend: go run ./cmd/luminary"
|
||||
@echo "Start frontend: cd web && npm run dev"
|
||||
|
||||
build-web:
|
||||
cd web && npm install --registry=https://registry.npmmirror.com && npm run build
|
||||
|
||||
build: build-web
|
||||
go build -o luminary ./cmd/luminary
|
||||
|
||||
recover:
|
||||
go build -o luminary-recover ./cmd/luminary-recover
|
||||
|
||||
run: build
|
||||
./luminary
|
||||
|
||||
tidy:
|
||||
go mod tidy
|
||||
|
||||
test:
|
||||
go test ./...
|
||||
|
||||
docker-build:
|
||||
docker buildx build --platform $(DOCKER_PLATFORM) \
|
||||
-t $(LUMINARY_IMAGE) \
|
||||
--build-arg IMAGE_PREFIX=$(IMAGE_PREFIX) \
|
||||
--build-arg APK_MIRROR=$(APK_MIRROR) \
|
||||
--build-arg GOPROXY=$(GOPROXY) \
|
||||
--build-arg GOSUMDB=$(GOSUMDB) \
|
||||
--load \
|
||||
.
|
||||
|
||||
docker-push:
|
||||
docker buildx build --platform $(DOCKER_PLATFORM) \
|
||||
-t $(LUMINARY_IMAGE) \
|
||||
--build-arg IMAGE_PREFIX=$(IMAGE_PREFIX) \
|
||||
--build-arg APK_MIRROR=$(APK_MIRROR) \
|
||||
--build-arg GOPROXY=$(GOPROXY) \
|
||||
--build-arg GOSUMDB=$(GOSUMDB) \
|
||||
--push \
|
||||
.
|
||||
|
||||
docker-push-prebuilt: build-web
|
||||
mkdir -p .docker-bin
|
||||
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o .docker-bin/luminary ./cmd/luminary
|
||||
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o .docker-bin/luminary-recover ./cmd/luminary-recover
|
||||
docker buildx build --platform $(DOCKER_PLATFORM) \
|
||||
-f docker/Dockerfile.prebuilt \
|
||||
-t $(LUMINARY_IMAGE) \
|
||||
--build-arg IMAGE_PREFIX=$(IMAGE_PREFIX) \
|
||||
--build-arg APK_MIRROR=$(APK_MIRROR) \
|
||||
--push \
|
||||
.
|
||||
@@ -0,0 +1,357 @@
|
||||
# Luminary
|
||||
|
||||
OpenAI 兼容 AI 网关，带 Web 管理台。支持多 Provider 出口、入口密钥治理、看板监控与安全管控。控制面自研（Go + SQLite），数据面提供 OpenAI 兼容 `/v1/*` 推理 API。
|
||||
|
||||
所有运行参数在 Web 管理台完成配置，**无需手写配置文件**。管理台支持**简体中文、英文**两种界面语言，可在顶栏随时切换。
|
||||
|
||||
## 界面预览
|
||||
|
||||
以下为 Web 管理台主要页面（点击可查看原图）。
|
||||
|
||||
| 登录 | 看板监控 |
|
||||
|:---:|:---:|
|
||||
|  |  |
|
||||
| 默认密码 `admin123`，首次登录后请修改 | Provider 健康、24h 请求/Token 与入口用量排行 |
|
||||
|
||||
| 出口管理 | 入口管理 |
|
||||
|:---:|:---:|
|
||||
|  |  |
|
||||
| 多 Provider、多 Key 负载均衡与故障转移 | 入口密钥（`sk-lum-*`）、路由规则与预算限流 |
|
||||
|
||||
| 请求日志 | IP 规则 |
|
||||
|:---:|:---:|
|
||||
|  |  |
|
||||
| 全量请求记录与成本估算 | 精确 / CIDR 白黑名单（admin / proxy 范围） |
|
||||
|
||||
| 系统设置 | |
|
||||
|:---:|:---:|
|
||||
|  | |
|
||||
| 加密密钥、会话、登录限流、健康检查与重试策略 | |
|
||||
|
||||
## 功能概览
|
||||
|
||||
| 模块 | 说明 |
|
||||
|------|------|
|
||||
| 出口管理 | OpenAI / Anthropic / Azure OpenAI / OpenRouter / Ollama / Custom，多 Key 加权负载均衡与故障转移 |
|
||||
| 入口管理 | OpenAI 兼容 API，入口密钥（`sk-lum-*`），Virtual Key 路由、预算与 RPM/TPM 限流 |
|
||||
| 看板监控 | Provider 健康检查、24h 请求/Token 统计、入口用量排行 |
|
||||
| 请求日志 | 全量请求记录、按模型定价的成本估算 |
|
||||
| IP 安全 | 管理员与推理入口分范围的白名单 / 黑名单（精确、CIDR） |
|
||||
| 系统设置 | 加密密钥、信任代理、会话、登录限流、用量刷盘、网关重试 |
|
||||
|
||||
## 架构
|
||||
|
||||
```
|
||||
客户端 SDK / curl
|
||||
→ /v1/*（入口密钥 sk-lum-*）
|
||||
→ Luminary Gateway（路由、限流、故障转移）
|
||||
→ Provider 出口（多 Key 加权负载均衡）
|
||||
↑
|
||||
管理台 REST API + Vue UI (:8293)
|
||||
↓
|
||||
SQLite + 内存缓存
|
||||
```
|
||||
|
||||
## 快速开始（开发）
|
||||
|
||||
### 依赖
|
||||
|
||||
- Go 1.25+（见 `go.mod`）
|
||||
- Node.js 20+（仅开发/构建前端）
|
||||
|
||||
依赖镜像（已写入 `Makefile`、`web/.npmrc`，国内网络友好）：
|
||||
|
||||
- Go：`GOPROXY=https://goproxy.cn,direct`
|
||||
- npm：`registry=https://registry.npmmirror.com`
|
||||
|
||||
海外用户可将 `GOPROXY` 改为 `https://proxy.golang.org,direct`，npm 使用默认 registry 即可。
|
||||
|
||||
### 启动
|
||||
|
||||
```bash
|
||||
cd web && npm install # 安装前端依赖
|
||||
|
||||
# 终端 1：后端
|
||||
go run ./cmd/luminary
|
||||
|
||||
# 终端 2：前端（Vite 代理 /api、/v1 → :8293）
|
||||
cd web && npm run dev
|
||||
```
|
||||
|
||||
访问 http://localhost:5173 ，默认账号 `admin` / `admin123`（**首次登录后请立即修改**）。
|
||||
|
||||
```bash
|
||||
go test ./... # 运行单元测试
|
||||
make build # 构建 luminary 二进制（含嵌入前端）
|
||||
```
|
||||
|
||||
## 部署
|
||||
|
||||
镜像由 Gitea Actions 自动构建并发布到 Container Registry，**无需自行编译**。直接拉取运行即可。
|
||||
|
||||
```bash
|
||||
docker pull git.rc707blog.top/rose_cat707/luminary:latest
|
||||
```
|
||||
|
||||
| Tag | 说明 |
|
||||
|-----|------|
|
||||
| `latest` | `main` / `master` 分支最新构建 |
|
||||
| `sha-xxxxxxx` | 对应 commit 短 SHA |
|
||||
| `v1.2.3` | 推送 Git tag `v1.2.3` 时发布 |
|
||||
|
||||
### docker run
|
||||
|
||||
```bash
|
||||
docker pull git.rc707blog.top/rose_cat707/luminary:latest
|
||||
|
||||
docker run -d --name luminary \
|
||||
--restart unless-stopped \
|
||||
-p 8293:8293 \
|
||||
-v luminary-data:/data \
|
||||
git.rc707blog.top/rose_cat707/luminary:latest
|
||||
```
|
||||
|
||||
或使用项目内 `docker compose up -d`（通过环境变量 `LUMINARY_IMAGE` 指定镜像）。
|
||||
|
||||
### 本地自行构建
|
||||
|
||||
```bash
|
||||
make docker-build
|
||||
# 或
|
||||
docker build \
|
||||
--build-arg IMAGE_PREFIX=docker.1panel.live/library/ \
|
||||
--build-arg GOPROXY=https://goproxy.cn,direct \
|
||||
-t luminary:local .
|
||||
```
|
||||
|
||||
**端口（默认）**
|
||||
|
||||
| 服务 | 端口 | 说明 |
|
||||
|------|------|------|
|
||||
| 管理 API / Web UI / 推理入口 | 8293 | 管理台与 `/v1/*` 共用同一监听地址 |
|
||||
|
||||
监听地址可通过启动参数 `-addr` 或环境变量 `LUMINARY_ADDR` 修改。
|
||||
|
||||
## 数据目录
|
||||
|
||||
数据默认存储在 `./data/luminary.db`（SQLite），Docker 部署时挂载为 `/data`：
|
||||
|
||||
```
|
||||
data/
|
||||
└── luminary.db # Provider、入口密钥、规则、日志、系统设置
|
||||
```
|
||||
|
||||
首次启动自动创建数据库与默认系统设置（含随机加密密钥）。默认管理账号 `admin` / `admin123`（仅当数据库中尚无管理员时创建）。
|
||||
|
||||
## 入口 API（推理）
|
||||
|
||||
客户端使用在管理台「入口管理」创建的**入口密钥**（`sk-lum-*`）调用，与 OpenAI SDK 兼容。
|
||||
|
||||
**鉴权**
|
||||
|
||||
```http
|
||||
Authorization: Bearer sk-lum-<your-key>
|
||||
```
|
||||
|
||||
**Base URL**：`http://<host>:8293/v1`
|
||||
|
||||
| 方法 | 路径 | 说明 | 状态 |
|
||||
|------|------|------|------|
|
||||
| `GET` | `/v1/models` | 列出当前入口密钥可用的模型 | 已支持 |
|
||||
| `POST` | `/v1/chat/completions` | 对话补全；`stream: true` 时返回 SSE 流式 | 已支持 |
|
||||
| `POST` | `/v1/responses` | OpenAI Responses API；`stream: true` 时返回 SSE 流式 | 已支持 |
|
||||
| `POST` | `/v1/completions` | 文本补全（Legacy） | 已支持 |
|
||||
| `POST` | `/v1/embeddings` | 向量嵌入 | 已支持 |
|
||||
| `POST` | `/v1/rerank` | Cohere 兼容重排序 | 已支持 |
|
||||
| `POST` | `/v1/audio/speech` | 语音合成 | 预留 |
|
||||
| `POST` | `/v1/audio/transcriptions` | 语音转写 | 预留 |
|
||||
| `POST` | `/v1/images/generations` | 图像生成 | 预留 |
|
||||
|
||||
> 实际可访问的接口取决于上游 Provider 在「出口管理」中启用的 endpoint；未启用的接口网关不会转发。
|
||||
>
|
||||
> **Responses 适配**：若出口仅启用 `chat_completion` 而未启用 `responses`，网关会将 `/v1/responses` 自动转换为 `/v1/chat/completions` 请求，并将响应转回 Responses API 格式（含流式）。
|
||||
|
||||
**示例：对话补全**
|
||||
|
||||
```bash
|
||||
curl http://localhost:8293/v1/chat/completions \
|
||||
-H "Authorization: Bearer sk-lum-..." \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Hello"}]}'
|
||||
```
|
||||
|
||||
**示例：流式对话**
|
||||
|
||||
```bash
|
||||
curl http://localhost:8293/v1/chat/completions \
|
||||
-H "Authorization: Bearer sk-lum-..." \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model":"gpt-4o-mini","stream":true,"messages":[{"role":"user","content":"Hello"}]}'
|
||||
```
|
||||
|
||||
**示例：Embeddings**
|
||||
|
||||
```bash
|
||||
curl http://localhost:8293/v1/embeddings \
|
||||
-H "Authorization: Bearer sk-lum-..." \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model":"text-embedding-3-small","input":"hello"}'
|
||||
```
|
||||
|
||||
**示例：Rerank（Cohere 兼容）**
|
||||
|
||||
```bash
|
||||
curl http://localhost:8293/v1/rerank \
|
||||
-H "Authorization: Bearer sk-lum-..." \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model":"bge-reranker-v2-m3","query":"What is the capital of France?","documents":["Paris is the capital of France.","Berlin is in Germany."],"top_n":2}'
|
||||
```
|
||||
|
||||
**示例：列出模型**
|
||||
|
||||
```bash
|
||||
curl http://localhost:8293/v1/models \
|
||||
-H "Authorization: Bearer sk-lum-..."
|
||||
```
|
||||
|
||||
入口密钥的 Provider / Model 白名单、预算、RPM/TPM 限流与路由规则均在管理台「入口管理」配置；白名单为空表示不限制。
|
||||
|
||||
## 使用攻略（简要）
|
||||
|
||||
### 1. 首次登录
|
||||
|
||||
1. 打开 `http://<主机>:8293`
|
||||
2. 默认账号 `admin` / `admin123`，登录后进入「系统设置」修改密码
|
||||
3. 顶栏切换界面语言（中文 / English）
|
||||
|
||||
### 2. 配置出口
|
||||
|
||||
1. 「出口管理」→ 新建 Provider，选择类型并填写 Base URL
|
||||
2. 添加 API Key，配置加权与故障转移
|
||||
3. 启用所需 endpoint 分类（语言模型 / 向量嵌入 / 重排序 / 图像等）
|
||||
|
||||
### 3. 配置入口
|
||||
|
||||
1. 「入口管理」→ 新建入口密钥（`sk-lum-*`）
|
||||
2. 配置 Provider / Model 白名单、预算与 RPM/TPM 限流
|
||||
3. 按需设置 Virtual Key 路由规则
|
||||
|
||||
### 4. IP 安全
|
||||
|
||||
1. 「IP 规则」添加白名单 / 黑名单（支持 CIDR）
|
||||
2. 规则可按 `admin`（管理台）或 `proxy`（推理入口）范围生效
|
||||
|
||||
### 5. 日常运维
|
||||
|
||||
| 操作 | 位置 |
|
||||
|------|------|
|
||||
| 查看流量与健康 | 看板监控 |
|
||||
| 管理 Provider 与 Key | 出口管理 |
|
||||
| 管理入口密钥 | 入口管理 |
|
||||
| 查看请求与成本 | 请求日志 |
|
||||
| 修改运行参数 | 系统设置 |
|
||||
|
||||
## 系统设置
|
||||
|
||||
所有运行参数在管理台 **系统设置** 页面维护，存入 SQLite。
|
||||
|
||||
| 设置项 | 说明 |
|
||||
|------|------|
|
||||
| 加密密钥 | Provider API Key 加密密钥（首次启动自动生成） |
|
||||
| 信任代理 | 信任的反向代理 IP/CIDR；仅此时才读取 `X-Forwarded-For` |
|
||||
| 会话 / 登录限制 | 管理员 Session 有效期、登录尝试次数与锁定时长 |
|
||||
| 用量刷盘 / 健康检查 | 后台任务间隔 |
|
||||
| 网关重试 | 上游故障时的重试次数与退避 |
|
||||
|
||||
启动参数（非阻塞，仅决定进程如何监听与数据存放位置）：
|
||||
|
||||
| 参数 / 环境变量 | 说明 | 默认 |
|
||||
|------|------|------|
|
||||
| `-data-dir` / `LUMINARY_DATA` | SQLite 数据目录 | `./data` |
|
||||
| `-addr` / `LUMINARY_ADDR` | HTTP 监听地址 | `:8293` |
|
||||
|
||||
可选环境变量 `LUMINARY_ENCRYPTION_KEY`：仅在首次初始化系统设置时写入加密密钥（一般无需设置，管理台可改）。
|
||||
|
||||
## 应急恢复（容器内）
|
||||
|
||||
当 IP 白名单或登录冷却导致无法进入管理台时，可在容器内执行恢复工具（已内置在镜像中）：
|
||||
|
||||
```bash
|
||||
# 一键：重置密码为 admin123、清空 IP 规则与会话，并重启进程（清除内存中的登录冷却）
|
||||
docker exec luminary luminary-recover.sh --all
|
||||
|
||||
# 指定新密码
|
||||
docker exec luminary luminary-recover.sh --all --password 'your-new-password'
|
||||
|
||||
# 分项执行（不重启）
|
||||
docker exec luminary luminary-recover.sh \
|
||||
--reset-password --clear-ip --clear-sessions --password 'your-new-password'
|
||||
|
||||
# 仅清除登录冷却（需重启进程；登录冷却存在内存中）
|
||||
docker exec luminary luminary-recover.sh --restart
|
||||
```
|
||||
|
||||
本地非容器环境：
|
||||
|
||||
```bash
|
||||
chmod +x scripts/luminary-recover.sh
|
||||
./scripts/luminary-recover.sh --all
|
||||
# 或
|
||||
go run ./cmd/luminary-recover --all
|
||||
```
|
||||
|
||||
| 参数 | 说明 |
|
||||
|------|------|
|
||||
| `--all` | 重置密码 + 清空 IP 规则 + 清空会话 + 重启进程 |
|
||||
| `--reset-password` | 重置管理员密码 |
|
||||
| `--clear-ip` | 删除全部 IP 规则 |
|
||||
| `--clear-sessions` | 删除全部管理员 Session |
|
||||
| `--restart` | 向 PID 1 发送 SIGTERM，由容器重启策略拉起新进程 |
|
||||
| `--password` | 新密码（默认 `admin123`） |
|
||||
| `--username` | 管理员用户名（默认 `admin`） |
|
||||
| `-data-dir` / `LUMINARY_DATA` | 数据目录（默认 `./data`，容器内为 `/data`） |
|
||||
|
||||
## 命令行
|
||||
|
||||
| 命令 | 说明 |
|
||||
|------|------|
|
||||
| `luminary` | 启动网关（默认 `-data-dir ./data`、`-addr :8293`） |
|
||||
| `luminary-recover` | 应急恢复 CLI（容器内 `luminary-recover.sh` 封装） |
|
||||
|
||||
## 项目结构
|
||||
|
||||
```
|
||||
cmd/luminary/ # 主服务入口
|
||||
cmd/luminary-recover/ # 应急恢复 CLI
|
||||
internal/
|
||||
gateway/ # 路由、故障转移、健康检查
|
||||
handler/ # 管理 API、推理代理、静态资源
|
||||
provider/ # 各 Provider 客户端与适配
|
||||
store/ # SQLite 与内存缓存
|
||||
auth/ # 密码、登录限流
|
||||
middleware/ # IP 过滤、客户端 IP
|
||||
monitor/ # 监控统计
|
||||
pricing/ # 成本估算
|
||||
web/ # Vue 3 管理台
|
||||
src/i18n/ # 多语言资源
|
||||
src/views/ # 页面
|
||||
docs/
|
||||
image/ # README 界面截图
|
||||
UI-DESIGN-SYSTEM.md
|
||||
.gitea/workflows/ # Gitea Actions CI
|
||||
```
|
||||
|
||||
## CI / 镜像发布
|
||||
|
||||
推送 `main` / `master` 或 tag `v*` 时，Gitea Actions 构建多架构镜像并推送到 Registry。工作流与 Dockerfile 约定见 [.gitea/README.md](.gitea/README.md)；act_runner 部署参考见 [.gitea/act-runner/](.gitea/act-runner/)。
|
||||
|
||||
**一次性配置**（仓库 Settings → Actions）：
|
||||
|
||||
| 类型 | 名称 | 说明 |
|
||||
|------|------|------|
|
||||
| Secret | `REGISTRY_TOKEN` | Gitea PAT，需 `write:package` 权限 |
|
||||
| Variable | `REGISTRY` | 公网 Gitea 域名，如 `git.rc707blog.top` |
|
||||
|
||||
## 许可
|
||||
|
||||
[Luminary 采用 MIT License](LICENSE).
|
||||
@@ -0,0 +1,156 @@
|
||||
// luminary-recover: emergency recovery CLI (reset password, clear IP rules, restart).
|
||||
// Intended for docker exec when admin login is locked out.
|
||||
package main
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/glebarez/sqlite"
|
||||
"github.com/rose_cat707/luminary/internal/auth"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/settings"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/logger"
|
||||
)
|
||||
|
||||
func main() {
|
||||
os.Exit(run())
|
||||
}
|
||||
|
||||
func run() int {
|
||||
dataDir := flag.String("data-dir", envOr("LUMINARY_DATA", "./data"), "directory containing luminary.db")
|
||||
deprecatedConfig := flag.String("config", "", "deprecated: ignored; use -data-dir / LUMINARY_DATA")
|
||||
username := flag.String("username", "", "admin username (default: admin)")
|
||||
password := flag.String("password", "", "new admin password (default: admin123)")
|
||||
doAll := flag.Bool("all", false, "reset password, clear IP rules and sessions, then restart")
|
||||
doPassword := flag.Bool("reset-password", false, "reset admin password")
|
||||
doIP := flag.Bool("clear-ip", false, "delete all IP rules")
|
||||
doSessions := flag.Bool("clear-sessions", false, "delete all admin sessions")
|
||||
doRestart := flag.Bool("restart", false, "send SIGTERM to PID 1 to restart server (clears login cooldown)")
|
||||
flag.Parse()
|
||||
if *deprecatedConfig != "" {
|
||||
fmt.Fprintln(os.Stderr, "warning: -config is deprecated and ignored; use -data-dir / LUMINARY_DATA")
|
||||
}
|
||||
|
||||
if !*doAll && !*doPassword && !*doIP && !*doSessions && !*doRestart {
|
||||
fmt.Fprintln(os.Stderr, "usage: luminary-recover --all [--password NEW] [--restart]")
|
||||
fmt.Fprintln(os.Stderr, " luminary-recover --reset-password --clear-ip --clear-sessions [--restart]")
|
||||
flag.PrintDefaults()
|
||||
return 2
|
||||
}
|
||||
|
||||
user := *username
|
||||
pass := *password
|
||||
if user == "" || pass == "" {
|
||||
legacyUser, legacyPass := settings.LegacyAdminCredentials()
|
||||
if user == "" {
|
||||
user = legacyUser
|
||||
}
|
||||
if pass == "" {
|
||||
pass = legacyPass
|
||||
}
|
||||
}
|
||||
|
||||
if *doAll {
|
||||
*doPassword = true
|
||||
*doIP = true
|
||||
*doSessions = true
|
||||
*doRestart = true
|
||||
}
|
||||
|
||||
dbPath := filepath.Join(*dataDir, "luminary.db")
|
||||
db, err := openDB(dbPath)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "open database: %v\n", err)
|
||||
return 1
|
||||
}
|
||||
|
||||
if *doPassword {
|
||||
if pass == "" {
|
||||
fmt.Fprintln(os.Stderr, "password is required (use --password)")
|
||||
return 1
|
||||
}
|
||||
if err := resetPassword(db, user, pass); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "reset password: %v\n", err)
|
||||
return 1
|
||||
}
|
||||
fmt.Printf("admin password reset for user %q\n", user)
|
||||
}
|
||||
|
||||
if *doIP {
|
||||
res := db.Exec("DELETE FROM ip_rules")
|
||||
if res.Error != nil {
|
||||
fmt.Fprintf(os.Stderr, "clear IP rules: %v\n", res.Error)
|
||||
return 1
|
||||
}
|
||||
fmt.Printf("cleared %d IP rule(s)\n", res.RowsAffected)
|
||||
}
|
||||
|
||||
if *doSessions {
|
||||
res := db.Exec("DELETE FROM admin_sessions")
|
||||
if res.Error != nil {
|
||||
fmt.Fprintf(os.Stderr, "clear sessions: %v\n", res.Error)
|
||||
return 1
|
||||
}
|
||||
fmt.Printf("cleared %d admin session(s)\n", res.RowsAffected)
|
||||
}
|
||||
|
||||
if *doRestart {
|
||||
fmt.Println("sending SIGTERM to PID 1 (restart server to clear in-memory login cooldown)...")
|
||||
if err := syscall.Kill(1, syscall.SIGTERM); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "restart: %v (you may need to restart the container manually)\n", err)
|
||||
return 1
|
||||
}
|
||||
}
|
||||
|
||||
fmt.Println("done")
|
||||
return 0
|
||||
}
|
||||
|
||||
func openDB(path string) (*gorm.DB, error) {
|
||||
dsn := path + "?_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)"
|
||||
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{
|
||||
Logger: logger.Default.LogMode(logger.Silent),
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sqlDB, err := db.DB()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sqlDB.SetMaxOpenConns(1)
|
||||
return db, nil
|
||||
}
|
||||
|
||||
func resetPassword(db *gorm.DB, username, password string) error {
|
||||
hash, err := auth.HashPassword(password)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var user model.AdminUser
|
||||
err = db.Where("username = ?", username).First(&user).Error
|
||||
if err == gorm.ErrRecordNotFound {
|
||||
return db.Create(&model.AdminUser{
|
||||
Username: username,
|
||||
PasswordHash: hash,
|
||||
CreatedAt: time.Now(),
|
||||
}).Error
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return db.Model(&user).Update("password_hash", hash).Error
|
||||
}
|
||||
|
||||
func envOr(key, fallback string) string {
|
||||
if v := os.Getenv(key); v != "" {
|
||||
return v
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
@@ -0,0 +1,129 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"syscall"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
webembed "github.com/rose_cat707/luminary/web"
|
||||
"github.com/rose_cat707/luminary/internal/gateway"
|
||||
"github.com/rose_cat707/luminary/internal/handler/admin"
|
||||
"github.com/rose_cat707/luminary/internal/handler/files"
|
||||
"github.com/rose_cat707/luminary/internal/handler/proxy"
|
||||
"github.com/rose_cat707/luminary/internal/handler/replicate"
|
||||
"github.com/rose_cat707/luminary/internal/handler/static"
|
||||
"github.com/rose_cat707/luminary/internal/middleware"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/monitor"
|
||||
"github.com/rose_cat707/luminary/internal/prediction"
|
||||
"github.com/rose_cat707/luminary/internal/settings"
|
||||
"github.com/rose_cat707/luminary/internal/storage/imagestore"
|
||||
"github.com/rose_cat707/luminary/internal/store/cache"
|
||||
sqlitestore "github.com/rose_cat707/luminary/internal/store/sqlite"
|
||||
)
|
||||
|
||||
func main() {
|
||||
dataDir := flag.String("data-dir", envOr("LUMINARY_DATA", "./data"), "directory for SQLite database")
|
||||
addr := flag.String("addr", envOr("LUMINARY_ADDR", ":8293"), "HTTP listen address")
|
||||
deprecatedConfig := flag.String("config", "", "deprecated: ignored; settings are stored in the database")
|
||||
flag.Parse()
|
||||
if *deprecatedConfig != "" {
|
||||
log.Printf("warning: -config is deprecated and ignored; use -data-dir / LUMINARY_DATA and admin UI settings")
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(*dataDir, 0o755); err != nil {
|
||||
log.Fatalf("data dir: %v", err)
|
||||
}
|
||||
dbPath := filepath.Join(*dataDir, "luminary.db")
|
||||
|
||||
db, err := sqlitestore.New(dbPath)
|
||||
if err != nil {
|
||||
log.Fatalf("database: %v", err)
|
||||
}
|
||||
|
||||
rt := settings.NewRuntime(db.DB())
|
||||
if err := rt.Load(); err != nil {
|
||||
log.Fatalf("settings: %v", err)
|
||||
}
|
||||
|
||||
memCache := cache.New()
|
||||
repo := sqlitestore.NewRepository(db, memCache)
|
||||
if err := repo.EnsureAdmin(); err != nil {
|
||||
log.Fatalf("admin bootstrap: %v", err)
|
||||
}
|
||||
if err := repo.BootstrapFromDB(); err != nil {
|
||||
log.Fatalf("cache bootstrap: %v", err)
|
||||
}
|
||||
rt.AfterLoad()
|
||||
|
||||
imageDir := filepath.Join(*dataDir, rt.ImageStoragePath())
|
||||
publicURL := func() string {
|
||||
if u := rt.PublicBaseURL(); u != "" {
|
||||
return u
|
||||
}
|
||||
a := *addr
|
||||
if strings.HasPrefix(a, ":") {
|
||||
a = "localhost" + a
|
||||
}
|
||||
return "http://" + a
|
||||
}
|
||||
imgStore := imagestore.New(db.DB(), imageDir, publicURL(), rt.EncryptionKey(), rt.SignedURLTTL())
|
||||
_ = imgStore.EnsureDir()
|
||||
|
||||
predSvc := prediction.NewService(db.DB(), memCache, imgStore, publicURL(), rt.PredictionWaitSeconds())
|
||||
|
||||
gw := gateway.New(memCache, rt)
|
||||
mon := monitor.New(gw, memCache, db, rt)
|
||||
mon.Start(repo.FlushUsage)
|
||||
|
||||
gin.SetMode(gin.ReleaseMode)
|
||||
r := gin.New()
|
||||
r.Use(middleware.Logger(), gin.Recovery())
|
||||
|
||||
r.GET("/health", func(c *gin.Context) {
|
||||
c.JSON(200, gin.H{"status": "ok"})
|
||||
})
|
||||
|
||||
adminHandler := admin.New(repo, db, memCache, gw, rt, mon, predSvc, imgStore, publicURL)
|
||||
adminGroup := r.Group("/api/admin", middleware.IPFilter(memCache, model.IPScopeAdmin))
|
||||
adminHandler.Register(adminGroup)
|
||||
|
||||
proxyHandler := proxy.New(gw)
|
||||
v1 := r.Group("/v1", middleware.IPFilter(memCache, model.IPScopeProxy), middleware.IngressAuth(gw))
|
||||
proxyHandler.Register(v1)
|
||||
|
||||
replicateHandler := replicate.New(predSvc, publicURL)
|
||||
replicateHandler.Register(v1)
|
||||
|
||||
filesHandler := files.New(imgStore)
|
||||
filesHandler.Register(r)
|
||||
|
||||
staticHandler := static.New(webembed.WebFS)
|
||||
staticHandler.Register(r)
|
||||
|
||||
go func() {
|
||||
log.Printf("Luminary listening on %s (data=%s)", *addr, *dataDir)
|
||||
if err := r.Run(*addr); err != nil {
|
||||
log.Fatalf("server: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
quit := make(chan os.Signal, 1)
|
||||
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
|
||||
<-quit
|
||||
log.Println("shutting down...")
|
||||
mon.Stop()
|
||||
repo.FlushUsage(memCache.DrainUsageDeltas())
|
||||
}
|
||||
|
||||
func envOr(key, fallback string) string {
|
||||
if v := os.Getenv(key); v != "" {
|
||||
return v
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
services:
|
||||
luminary:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
args:
|
||||
IMAGE_PREFIX: docker.1panel.live/library/
|
||||
APK_MIRROR: mirrors.aliyun.com
|
||||
GOPROXY: https://goproxy.cn,direct
|
||||
GOSUMDB: sum.golang.google.cn
|
||||
image: ${LUMINARY_IMAGE:-registry.rc707blog.top/rose_cat707/luminary:latest}
|
||||
container_name: luminary
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "8293:8293"
|
||||
environment:
|
||||
LUMINARY_DATA: /data
|
||||
LUMINARY_ADDR: ":8293"
|
||||
volumes:
|
||||
- luminary-data:/data
|
||||
|
||||
volumes:
|
||||
luminary-data:
|
||||
@@ -0,0 +1,20 @@
|
||||
# Runtime image using locally cross-compiled linux/amd64 binaries (avoids QEMU segfault in buildx).
|
||||
ARG IMAGE_PREFIX=docker.1panel.live/library/
|
||||
ARG APK_MIRROR=mirrors.aliyun.com
|
||||
FROM ${IMAGE_PREFIX}alpine:3.20 AS runtime
|
||||
ARG APK_MIRROR=mirrors.aliyun.com
|
||||
RUN sed -i "s/dl-cdn.alpinelinux.org/${APK_MIRROR}/g" /etc/apk/repositories \
|
||||
&& apk add --no-cache ca-certificates tini
|
||||
|
||||
ENV LUMINARY_DATA=/data \
|
||||
LUMINARY_ADDR=:8293
|
||||
|
||||
COPY .docker-bin/luminary /usr/local/bin/luminary
|
||||
COPY .docker-bin/luminary-recover /usr/local/bin/luminary-recover
|
||||
COPY docker/entrypoint.sh /entrypoint.sh
|
||||
COPY scripts/luminary-recover.sh /usr/local/bin/luminary-recover.sh
|
||||
RUN chmod +x /entrypoint.sh /usr/local/bin/luminary-recover.sh
|
||||
|
||||
VOLUME ["/data"]
|
||||
EXPOSE 8293
|
||||
ENTRYPOINT ["/sbin/tini", "--", "/entrypoint.sh"]
|
||||
@@ -0,0 +1,10 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
DATA_DIR="${LUMINARY_DATA:-/data}"
|
||||
ADDR="${LUMINARY_ADDR:-:8293}"
|
||||
|
||||
mkdir -p "$DATA_DIR"
|
||||
|
||||
echo "[luminary] $(date '+%Y-%m-%d %H:%M:%S') starting (addr=${ADDR}, data=${DATA_DIR})"
|
||||
exec /usr/local/bin/luminary -data-dir "$DATA_DIR" -addr "$ADDR"
|
||||
File diff suppressed because it is too large
Load Diff
Binary file not shown.
|
After Width: | Height: | Size: 104 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 202 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 192 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 74 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 239 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 209 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 328 KiB |
@@ -0,0 +1,52 @@
|
||||
module github.com/rose_cat707/luminary
|
||||
|
||||
go 1.25.0
|
||||
|
||||
require (
|
||||
github.com/gin-gonic/gin v1.12.0
|
||||
github.com/glebarez/sqlite v1.11.0
|
||||
github.com/google/uuid v1.3.0
|
||||
github.com/gorilla/websocket v1.5.3
|
||||
golang.org/x/crypto v0.53.0
|
||||
gorm.io/gorm v1.31.1
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/bytedance/gopkg v0.1.3 // indirect
|
||||
github.com/bytedance/sonic v1.15.0 // indirect
|
||||
github.com/bytedance/sonic/loader v0.5.0 // indirect
|
||||
github.com/cloudwego/base64x v0.1.6 // indirect
|
||||
github.com/dustin/go-humanize v1.0.1 // indirect
|
||||
github.com/gabriel-vasile/mimetype v1.4.12 // indirect
|
||||
github.com/gin-contrib/sse v1.1.0 // indirect
|
||||
github.com/glebarez/go-sqlite v1.21.2 // indirect
|
||||
github.com/go-playground/locales v0.14.1 // indirect
|
||||
github.com/go-playground/universal-translator v0.18.1 // indirect
|
||||
github.com/go-playground/validator/v10 v10.30.1 // indirect
|
||||
github.com/goccy/go-json v0.10.5 // indirect
|
||||
github.com/goccy/go-yaml v1.19.2 // indirect
|
||||
github.com/jinzhu/inflection v1.0.0 // indirect
|
||||
github.com/jinzhu/now v1.1.5 // indirect
|
||||
github.com/json-iterator/go v1.1.12 // indirect
|
||||
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
|
||||
github.com/leodido/go-urn v1.4.0 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
|
||||
github.com/modern-go/reflect2 v1.0.2 // indirect
|
||||
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
|
||||
github.com/quic-go/qpack v0.6.0 // indirect
|
||||
github.com/quic-go/quic-go v0.59.0 // indirect
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
|
||||
github.com/ugorji/go/codec v1.3.1 // indirect
|
||||
go.mongodb.org/mongo-driver/v2 v2.5.0 // indirect
|
||||
golang.org/x/arch v0.22.0 // indirect
|
||||
golang.org/x/net v0.55.0 // indirect
|
||||
golang.org/x/sys v0.46.0 // indirect
|
||||
golang.org/x/text v0.38.0 // indirect
|
||||
google.golang.org/protobuf v1.36.10 // indirect
|
||||
modernc.org/libc v1.22.5 // indirect
|
||||
modernc.org/mathutil v1.5.0 // indirect
|
||||
modernc.org/memory v1.5.0 // indirect
|
||||
modernc.org/sqlite v1.23.1 // indirect
|
||||
)
|
||||
@@ -0,0 +1,118 @@
|
||||
github.com/bytedance/gopkg v0.1.3 h1:TPBSwH8RsouGCBcMBktLt1AymVo2TVsBVCY4b6TnZ/M=
|
||||
github.com/bytedance/gopkg v0.1.3/go.mod h1:576VvJ+eJgyCzdjS+c4+77QF3p7ubbtiKARP3TxducM=
|
||||
github.com/bytedance/sonic v1.15.0 h1:/PXeWFaR5ElNcVE84U0dOHjiMHQOwNIx3K4ymzh/uSE=
|
||||
github.com/bytedance/sonic v1.15.0/go.mod h1:tFkWrPz0/CUCLEF4ri4UkHekCIcdnkqXw9VduqpJh0k=
|
||||
github.com/bytedance/sonic/loader v0.5.0 h1:gXH3KVnatgY7loH5/TkeVyXPfESoqSBSBEiDd5VjlgE=
|
||||
github.com/bytedance/sonic/loader v0.5.0/go.mod h1:AR4NYCk5DdzZizZ5djGqQ92eEhCCcdf5x77udYiSJRo=
|
||||
github.com/cloudwego/base64x v0.1.6 h1:t11wG9AECkCDk5fMSoxmufanudBtJ+/HemLstXDLI2M=
|
||||
github.com/cloudwego/base64x v0.1.6/go.mod h1:OFcloc187FXDaYHvrNIjxSe8ncn0OOM8gEHfghB2IPU=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
|
||||
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
|
||||
github.com/gabriel-vasile/mimetype v1.4.12 h1:e9hWvmLYvtp846tLHam2o++qitpguFiYCKbn0w9jyqw=
|
||||
github.com/gabriel-vasile/mimetype v1.4.12/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=
|
||||
github.com/gin-contrib/sse v1.1.0 h1:n0w2GMuUpWDVp7qSpvze6fAu9iRxJY4Hmj6AmBOU05w=
|
||||
github.com/gin-contrib/sse v1.1.0/go.mod h1:hxRZ5gVpWMT7Z0B0gSNYqqsSCNIJMjzvm6fqCz9vjwM=
|
||||
github.com/gin-gonic/gin v1.12.0 h1:b3YAbrZtnf8N//yjKeU2+MQsh2mY5htkZidOM7O0wG8=
|
||||
github.com/gin-gonic/gin v1.12.0/go.mod h1:VxccKfsSllpKshkBWgVgRniFFAzFb9csfngsqANjnLc=
|
||||
github.com/glebarez/go-sqlite v1.21.2 h1:3a6LFC4sKahUunAmynQKLZceZCOzUthkRkEAl9gAXWo=
|
||||
github.com/glebarez/go-sqlite v1.21.2/go.mod h1:sfxdZyhQjTM2Wry3gVYWaW072Ri1WMdWJi0k6+3382k=
|
||||
github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw=
|
||||
github.com/glebarez/sqlite v1.11.0/go.mod h1:h8/o8j5wiAsqSPoWELDUdJXhjAhsVliSn7bWZjOhrgQ=
|
||||
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
|
||||
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
|
||||
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
|
||||
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
|
||||
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
|
||||
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
|
||||
github.com/go-playground/validator/v10 v10.30.1 h1:f3zDSN/zOma+w6+1Wswgd9fLkdwy06ntQJp0BBvFG0w=
|
||||
github.com/go-playground/validator/v10 v10.30.1/go.mod h1:oSuBIQzuJxL//3MelwSLD5hc2Tu889bF0Idm9Dg26cM=
|
||||
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
|
||||
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
|
||||
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
|
||||
github.com/goccy/go-yaml v1.19.2/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
|
||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
||||
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
|
||||
github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26 h1:Xim43kblpZXfIBQsbuBVKCudVG457BR2GZFIz3uw3hQ=
|
||||
github.com/google/pprof v0.0.0-20221118152302-e6195bd50e26/go.mod h1:dDKJzRmX4S37WGHujM7tX//fmj1uioxKzKxz3lo4HJo=
|
||||
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
|
||||
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
|
||||
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
|
||||
github.com/jinzhu/inflection v1.0.0 h1:K317FqzuhWc8YvSVlFMCCUb36O/S9MCKRDI7QkRKD/E=
|
||||
github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc=
|
||||
github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
|
||||
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
|
||||
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
|
||||
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
|
||||
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
|
||||
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
|
||||
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
|
||||
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
|
||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
||||
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
|
||||
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
||||
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
|
||||
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
|
||||
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
|
||||
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
|
||||
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
|
||||
github.com/quic-go/quic-go v0.59.0 h1:OLJkp1Mlm/aS7dpKgTc6cnpynnD2Xg7C1pwL6vy/SAw=
|
||||
github.com/quic-go/quic-go v0.59.0/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
||||
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
||||
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
|
||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
|
||||
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
|
||||
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
|
||||
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
|
||||
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
|
||||
github.com/ugorji/go/codec v1.3.1 h1:waO7eEiFDwidsBN6agj1vJQ4AG7lh2yqXyOXqhgQuyY=
|
||||
github.com/ugorji/go/codec v1.3.1/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4=
|
||||
go.mongodb.org/mongo-driver/v2 v2.5.0 h1:yXUhImUjjAInNcpTcAlPHiT7bIXhshCTL3jVBkF3xaE=
|
||||
go.mongodb.org/mongo-driver/v2 v2.5.0/go.mod h1:yOI9kBsufol30iFsl1slpdq1I0eHPzybRWdyYUs8K/0=
|
||||
go.uber.org/mock v0.6.0 h1:hyF9dfmbgIX5EfOdasqLsWD6xqpNZlXblLB/Dbnwv3Y=
|
||||
go.uber.org/mock v0.6.0/go.mod h1:KiVJ4BqZJaMj4svdfmHM0AUx4NJYO8ZNpPnZn1Z+BBU=
|
||||
golang.org/x/arch v0.22.0 h1:c/Zle32i5ttqRXjdLyyHZESLD/bB90DCU1g9l/0YBDI=
|
||||
golang.org/x/arch v0.22.0/go.mod h1:dNHoOeKiyja7GTvF9NJS1l3Z2yntpQNzgrjh1cU103A=
|
||||
golang.org/x/crypto v0.53.0 h1:QZ4Muo8THX6CizN2vPPd5fBGHyogrdK9fG4wLPFUsto=
|
||||
golang.org/x/crypto v0.53.0/go.mod h1:DNLU434OwVakk9PzuwV8w62mAJpRJL3vsgcfp4Qnsio=
|
||||
golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8=
|
||||
golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
|
||||
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
|
||||
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
|
||||
google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE=
|
||||
google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gorm.io/gorm v1.31.1 h1:7CA8FTFz/gRfgqgpeKIBcervUn3xSyPUmr6B2WXJ7kg=
|
||||
gorm.io/gorm v1.31.1/go.mod h1:XyQVbO2k6YkOis7C2437jSit3SsDK72s7n7rsSHd+Gs=
|
||||
modernc.org/libc v1.22.5 h1:91BNch/e5B0uPbJFgqbxXuOnxBQjlS//icfQEGmvyjE=
|
||||
modernc.org/libc v1.22.5/go.mod h1:jj+Z7dTNX8fBScMVNRAYZ/jF91K8fdT2hYMThc3YjBY=
|
||||
modernc.org/mathutil v1.5.0 h1:rV0Ko/6SfM+8G+yKiyI830l3Wuz1zRutdslNoQ0kfiQ=
|
||||
modernc.org/mathutil v1.5.0/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E=
|
||||
modernc.org/memory v1.5.0 h1:N+/8c5rE6EqugZwHii4IFsaJ7MUhoWX07J5tC/iI5Ds=
|
||||
modernc.org/memory v1.5.0/go.mod h1:PkUhL0Mugw21sHPeskwZW4D6VscE/GQJOnIpCnW6pSU=
|
||||
modernc.org/sqlite v1.23.1 h1:nrSBg4aRQQwq59JpvGEQ15tNxoO5pX/kUjcRNwSAGQM=
|
||||
modernc.org/sqlite v1.23.1/go.mod h1:OrDj17Mggn6MhE+iPbBNf7RGKODDE9NFT0f3EwDzJqk=
|
||||
@@ -0,0 +1,74 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"crypto/aes"
|
||||
"crypto/cipher"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
)
|
||||
|
||||
func deriveKey(secret string) []byte {
|
||||
h := sha256.Sum256([]byte(secret))
|
||||
return h[:]
|
||||
}
|
||||
|
||||
func Encrypt(plaintext, secret string) (string, error) {
|
||||
key := deriveKey(secret)
|
||||
block, err := aes.NewCipher(key)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
gcm, err := cipher.NewGCM(block)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
nonce := make([]byte, gcm.NonceSize())
|
||||
if _, err := io.ReadFull(rand.Reader, nonce); err != nil {
|
||||
return "", err
|
||||
}
|
||||
ciphertext := gcm.Seal(nonce, nonce, []byte(plaintext), nil)
|
||||
return base64.StdEncoding.EncodeToString(ciphertext), nil
|
||||
}
|
||||
|
||||
func Decrypt(encoded, secret string) (string, error) {
|
||||
data, err := base64.StdEncoding.DecodeString(encoded)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
key := deriveKey(secret)
|
||||
block, err := aes.NewCipher(key)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
gcm, err := cipher.NewGCM(block)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
nonceSize := gcm.NonceSize()
|
||||
if len(data) < nonceSize {
|
||||
return "", errors.New("ciphertext too short")
|
||||
}
|
||||
nonce, ciphertext := data[:nonceSize], data[nonceSize:]
|
||||
plain, err := gcm.Open(nil, nonce, ciphertext, nil)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return string(plain), nil
|
||||
}
|
||||
|
||||
func HashToken(token string) string {
|
||||
h := sha256.Sum256([]byte(token))
|
||||
return fmt.Sprintf("%x", h)
|
||||
}
|
||||
|
||||
func GenerateToken(n int) (string, error) {
|
||||
b := make([]byte, n)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return base64.URLEncoding.EncodeToString(b)[:n], nil
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
package loginlimit
|
||||
|
||||
import (
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Limiter tracks failed admin login attempts per client IP.
|
||||
type Limiter struct {
|
||||
mu sync.Mutex
|
||||
attempts map[string]*record
|
||||
maxAttempts int
|
||||
lockout time.Duration
|
||||
}
|
||||
|
||||
type record struct {
|
||||
failures int
|
||||
lockedUntil time.Time
|
||||
}
|
||||
|
||||
func New(maxAttempts int, lockout time.Duration) *Limiter {
|
||||
if maxAttempts <= 0 {
|
||||
maxAttempts = 5
|
||||
}
|
||||
if lockout <= 0 {
|
||||
lockout = 15 * time.Minute
|
||||
}
|
||||
return &Limiter{
|
||||
attempts: make(map[string]*record),
|
||||
maxAttempts: maxAttempts,
|
||||
lockout: lockout,
|
||||
}
|
||||
}
|
||||
|
||||
// Check reports whether the IP is locked and how long to wait.
|
||||
func (l *Limiter) Check(ip string) (locked bool, retryAfter time.Duration) {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
r := l.attempts[ip]
|
||||
if r == nil {
|
||||
return false, 0
|
||||
}
|
||||
now := time.Now()
|
||||
if now.Before(r.lockedUntil) {
|
||||
return true, r.lockedUntil.Sub(now)
|
||||
}
|
||||
if r.failures >= l.maxAttempts {
|
||||
delete(l.attempts, ip)
|
||||
}
|
||||
return false, 0
|
||||
}
|
||||
|
||||
// RecordFailure increments the failure count and locks the IP when the limit is reached.
|
||||
func (l *Limiter) RecordFailure(ip string) {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
now := time.Now()
|
||||
r := l.attempts[ip]
|
||||
if r == nil {
|
||||
r = &record{}
|
||||
l.attempts[ip] = r
|
||||
}
|
||||
if now.Before(r.lockedUntil) {
|
||||
return
|
||||
}
|
||||
if r.failures >= l.maxAttempts && !r.lockedUntil.IsZero() && now.After(r.lockedUntil) {
|
||||
r.failures = 0
|
||||
r.lockedUntil = time.Time{}
|
||||
}
|
||||
r.failures++
|
||||
if r.failures >= l.maxAttempts {
|
||||
r.lockedUntil = now.Add(l.lockout)
|
||||
}
|
||||
}
|
||||
|
||||
// Configure updates rate limit parameters at runtime.
|
||||
func (l *Limiter) Configure(maxAttempts int, lockout time.Duration) {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
if maxAttempts > 0 {
|
||||
l.maxAttempts = maxAttempts
|
||||
}
|
||||
if lockout > 0 {
|
||||
l.lockout = lockout
|
||||
}
|
||||
}
|
||||
|
||||
// Reset clears failure state after a successful login.
|
||||
func (l *Limiter) Reset(ip string) {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
delete(l.attempts, ip)
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
package loginlimit
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestLimiterLockout(t *testing.T) {
|
||||
l := New(3, time.Minute)
|
||||
ip := "192.168.1.1"
|
||||
|
||||
for i := 0; i < 2; i++ {
|
||||
if locked, _ := l.Check(ip); locked {
|
||||
t.Fatalf("unexpected lock at attempt %d", i)
|
||||
}
|
||||
l.RecordFailure(ip)
|
||||
}
|
||||
|
||||
if locked, _ := l.Check(ip); locked {
|
||||
t.Fatal("should not lock before max attempts")
|
||||
}
|
||||
l.RecordFailure(ip)
|
||||
|
||||
locked, retry := l.Check(ip)
|
||||
if !locked {
|
||||
t.Fatal("expected lock after max failures")
|
||||
}
|
||||
if retry <= 0 {
|
||||
t.Fatal("expected positive retry duration")
|
||||
}
|
||||
|
||||
l.Reset(ip)
|
||||
if locked, _ := l.Check(ip); locked {
|
||||
t.Fatal("expected unlock after reset")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"golang.org/x/crypto/bcrypt"
|
||||
)
|
||||
|
||||
func HashPassword(password string) (string, error) {
|
||||
b, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
|
||||
return string(b), err
|
||||
}
|
||||
|
||||
func CheckPassword(hash, password string) bool {
|
||||
return bcrypt.CompareHashAndPassword([]byte(hash), []byte(password)) == nil
|
||||
}
|
||||
|
||||
// HashIngressKey stores a SHA-256 hex digest for O(1) cache lookup.
|
||||
func HashIngressKey(key string) string {
|
||||
return HashToken(key)
|
||||
}
|
||||
|
||||
func IsLegacyIngressHash(hash string) bool {
|
||||
return strings.HasPrefix(hash, "$2a$") || strings.HasPrefix(hash, "$2b$")
|
||||
}
|
||||
|
||||
// CheckIngressKey verifies legacy bcrypt-hashed ingress keys.
|
||||
func CheckIngressKey(hash, key string) bool {
|
||||
return bcrypt.CompareHashAndPassword([]byte(hash), []byte(key)) == nil
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
package balancer
|
||||
|
||||
import (
|
||||
"math/rand"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
)
|
||||
|
||||
type KeyCandidate struct {
|
||||
Key model.ProviderKey
|
||||
Weight float64
|
||||
}
|
||||
|
||||
func SelectWeighted(candidates []KeyCandidate) *model.ProviderKey {
|
||||
if len(candidates) == 0 {
|
||||
return nil
|
||||
}
|
||||
if len(candidates) == 1 {
|
||||
k := candidates[0].Key
|
||||
return &k
|
||||
}
|
||||
var total float64
|
||||
for _, c := range candidates {
|
||||
w := c.Weight
|
||||
if w <= 0 {
|
||||
w = 1
|
||||
}
|
||||
total += w
|
||||
}
|
||||
if total <= 0 {
|
||||
k := candidates[0].Key
|
||||
return &k
|
||||
}
|
||||
r := rand.Float64() * total
|
||||
var acc float64
|
||||
for _, c := range candidates {
|
||||
w := c.Weight
|
||||
if w <= 0 {
|
||||
w = 1
|
||||
}
|
||||
acc += w
|
||||
if r <= acc {
|
||||
k := c.Key
|
||||
return &k
|
||||
}
|
||||
}
|
||||
k := candidates[len(candidates)-1].Key
|
||||
return &k
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
package balancer_test
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/balancer"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
)
|
||||
|
||||
func TestSelectWeighted(t *testing.T) {
|
||||
candidates := []balancer.KeyCandidate{
|
||||
{Key: model.ProviderKey{ID: 1, Name: "a"}, Weight: 1},
|
||||
{Key: model.ProviderKey{ID: 2, Name: "b"}, Weight: 3},
|
||||
}
|
||||
counts := map[uint]int{}
|
||||
for i := 0; i < 1000; i++ {
|
||||
k := balancer.SelectWeighted(candidates)
|
||||
if k == nil {
|
||||
t.Fatal("nil key")
|
||||
}
|
||||
counts[k.ID]++
|
||||
}
|
||||
if counts[2] <= counts[1] {
|
||||
t.Fatalf("expected heavier weight key selected more often: %v", counts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSelectWeightedEmpty(t *testing.T) {
|
||||
if k := balancer.SelectWeighted(nil); k != nil {
|
||||
t.Fatal("expected nil")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
package gateway
|
||||
|
||||
import "errors"
|
||||
|
||||
var (
|
||||
ErrModelNotAllowed = errors.New("model not allowed")
|
||||
ErrBudgetExceeded = errors.New("budget exceeded")
|
||||
ErrRateLimitExceeded = errors.New("rate limit exceeded")
|
||||
ErrInvalidJSON = errors.New("invalid json")
|
||||
ErrModelRequired = errors.New("model is required")
|
||||
)
|
||||
@@ -0,0 +1,160 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"sort"
|
||||
"time"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/balancer"
|
||||
"github.com/rose_cat707/luminary/internal/limiter"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/provider"
|
||||
"github.com/rose_cat707/luminary/internal/store/cache"
|
||||
)
|
||||
|
||||
type attempt struct {
|
||||
provider *model.Provider
|
||||
key *model.ProviderKey
|
||||
apiKey string
|
||||
}
|
||||
|
||||
func (s *Service) buildAttempts(providers []model.Provider, modelName string, route *routeTarget) ([]attempt, error) {
|
||||
var attempts []attempt
|
||||
var decryptFailures int
|
||||
var eligibleKeys int
|
||||
for i := range providers {
|
||||
p := providers[i]
|
||||
keys := s.listEligibleKeys(p.ID, modelName, route)
|
||||
eligibleKeys += len(keys)
|
||||
for _, k := range keys {
|
||||
plain, err := s.DecryptAPIKey(k.APIKeyEnc)
|
||||
if err != nil {
|
||||
decryptFailures++
|
||||
continue
|
||||
}
|
||||
attempts = append(attempts, attempt{provider: &p, key: &k, apiKey: plain})
|
||||
}
|
||||
}
|
||||
if len(attempts) == 0 {
|
||||
if decryptFailures > 0 {
|
||||
return nil, fmt.Errorf("no available provider keys: %d key(s) failed to decrypt (check encryption_key in system settings)", decryptFailures)
|
||||
}
|
||||
if eligibleKeys == 0 {
|
||||
return nil, fmt.Errorf("no available provider keys: none match model %q or are disabled/rate-limited", modelName)
|
||||
}
|
||||
return nil, fmt.Errorf("no available provider keys")
|
||||
}
|
||||
sort.SliceStable(attempts, func(i, j int) bool {
|
||||
return attempts[i].key.Weight > attempts[j].key.Weight
|
||||
})
|
||||
return attempts, nil
|
||||
}
|
||||
|
||||
func (s *Service) listEligibleKeys(providerID uint, modelName string, route *routeTarget) []model.ProviderKey {
|
||||
if route != nil && route.ProviderKeyID > 0 && route.ProviderID == providerID {
|
||||
if k, ok := s.cache.GetProviderKey(route.ProviderKeyID); ok && k.Enabled && limiter.IsProviderKeyAvailable(k) {
|
||||
return []model.ProviderKey{k}
|
||||
}
|
||||
}
|
||||
var candidates []balancer.KeyCandidate
|
||||
for _, k := range s.cache.ListProviderKeys(providerID) {
|
||||
if !limiter.IsProviderKeyAvailable(k) {
|
||||
continue
|
||||
}
|
||||
allowed := cache.ParseJSONList(k.ModelsJSON)
|
||||
if modelName != "" && !cache.MatchesList(allowed, "*") {
|
||||
upstream := s.resolveUpstreamModelID(providerID, modelName)
|
||||
if !cache.MatchesList(allowed, modelName) && !cache.MatchesList(allowed, upstream) {
|
||||
continue
|
||||
}
|
||||
}
|
||||
candidates = append(candidates, balancer.KeyCandidate{Key: k, Weight: k.Weight})
|
||||
}
|
||||
// collect unique keys by weighted random sampling order
|
||||
seen := map[uint]bool{}
|
||||
var out []model.ProviderKey
|
||||
for len(out) < len(candidates) {
|
||||
selected := balancer.SelectWeighted(candidates)
|
||||
if selected == nil {
|
||||
break
|
||||
}
|
||||
if seen[selected.ID] {
|
||||
// remove from candidates
|
||||
filtered := candidates[:0]
|
||||
for _, c := range candidates {
|
||||
if c.Key.ID != selected.ID {
|
||||
filtered = append(filtered, c)
|
||||
}
|
||||
}
|
||||
candidates = filtered
|
||||
if len(candidates) == 0 {
|
||||
break
|
||||
}
|
||||
continue
|
||||
}
|
||||
seen[selected.ID] = true
|
||||
out = append(out, *selected)
|
||||
filtered := candidates[:0]
|
||||
for _, c := range candidates {
|
||||
if c.Key.ID != selected.ID {
|
||||
filtered = append(filtered, c)
|
||||
}
|
||||
}
|
||||
candidates = filtered
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (s *Service) forwardWithFailover(ctx context.Context, attempts []attempt, endpoint provider.EndpointType, body []byte) (*provider.ChatResponse, *attempt, error) {
|
||||
var lastErr error
|
||||
var lastAtt *attempt
|
||||
var lastResp *provider.ChatResponse
|
||||
maxRetries := s.settings.GatewayMaxRetries()
|
||||
if maxRetries < 1 {
|
||||
maxRetries = 1
|
||||
}
|
||||
backoffMs := s.settings.GatewayRetryBackoffMs()
|
||||
for _, att := range attempts {
|
||||
if !s.canServeEndpoint(att.provider, endpoint) {
|
||||
continue
|
||||
}
|
||||
for try := 0; try < maxRetries; try++ {
|
||||
if try > 0 {
|
||||
time.Sleep(time.Duration(backoffMs) * time.Millisecond)
|
||||
}
|
||||
resp, err := s.forward(ctx, att.provider, att.apiKey, endpoint, body)
|
||||
cur := att
|
||||
lastAtt = &cur
|
||||
if resp != nil {
|
||||
lastResp = resp
|
||||
}
|
||||
if err != nil {
|
||||
lastErr = err
|
||||
continue
|
||||
}
|
||||
if resp == nil {
|
||||
lastErr = fmt.Errorf("empty upstream response")
|
||||
continue
|
||||
}
|
||||
if resp.StatusCode < 400 {
|
||||
return resp, &att, nil
|
||||
}
|
||||
lastErr = fmt.Errorf("%s", describeForwardError(nil, resp))
|
||||
if passThroughUpstreamStatus(resp.StatusCode) {
|
||||
return resp, &att, nil
|
||||
}
|
||||
if tryNextKeyOnUpstreamStatus(resp.StatusCode) {
|
||||
break // try next provider key
|
||||
}
|
||||
if retryableUpstreamStatus(resp.StatusCode) {
|
||||
continue // retry same key
|
||||
}
|
||||
return resp, &att, nil
|
||||
}
|
||||
}
|
||||
if lastErr == nil {
|
||||
lastErr = fmt.Errorf("all providers failed")
|
||||
}
|
||||
return lastResp, lastAtt, lastErr
|
||||
}
|
||||
@@ -0,0 +1,190 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/provider"
|
||||
)
|
||||
|
||||
func (s *Service) healthCheckEndpoints(p *model.Provider) []provider.EndpointType {
|
||||
var out []provider.EndpointType
|
||||
if s.providerAllowsEndpoint(p, provider.EndpointListModels) {
|
||||
out = append(out, provider.EndpointListModels)
|
||||
}
|
||||
switch p.Category {
|
||||
case model.CategoryEmbedding:
|
||||
if s.providerAllowsEndpoint(p, provider.EndpointEmbeddings) {
|
||||
out = append(out, provider.EndpointEmbeddings)
|
||||
}
|
||||
case model.CategoryRerank:
|
||||
if s.providerAllowsEndpoint(p, provider.EndpointRerank) {
|
||||
out = append(out, provider.EndpointRerank)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (s *Service) runHealthChecks(
|
||||
ctx context.Context,
|
||||
p *model.Provider,
|
||||
keys []model.ProviderKey,
|
||||
cfg provider.ProviderConfig,
|
||||
) (model.ProviderStatus, int64, string) {
|
||||
endpoints := s.healthCheckEndpoints(p)
|
||||
if len(endpoints) == 0 {
|
||||
return model.ProviderUnhealthy, 0, "no health check endpoint enabled"
|
||||
}
|
||||
start := time.Now()
|
||||
var lastErr string
|
||||
for _, ep := range endpoints {
|
||||
status, _, msg := s.runHealthCheck(ctx, p, keys, cfg, ep)
|
||||
if status == model.ProviderHealthy {
|
||||
return model.ProviderHealthy, time.Since(start).Milliseconds(), ""
|
||||
}
|
||||
if msg != "" {
|
||||
lastErr = msg
|
||||
}
|
||||
}
|
||||
if lastErr == "" {
|
||||
lastErr = "health check failed"
|
||||
}
|
||||
return model.ProviderUnhealthy, time.Since(start).Milliseconds(), lastErr
|
||||
}
|
||||
|
||||
func (s *Service) runHealthCheck(
|
||||
ctx context.Context,
|
||||
p *model.Provider,
|
||||
keys []model.ProviderKey,
|
||||
cfg provider.ProviderConfig,
|
||||
endpoint provider.EndpointType,
|
||||
) (model.ProviderStatus, int64, string) {
|
||||
start := time.Now()
|
||||
latency := func() int64 { return time.Since(start).Milliseconds() }
|
||||
|
||||
switch endpoint {
|
||||
case provider.EndpointListModels:
|
||||
if len(keys) == 0 && p.Type == model.ProviderOllama {
|
||||
if err := s.openai.HealthCheckWithConfig(ctx, "", cfg); err == nil {
|
||||
return model.ProviderHealthy, latency(), ""
|
||||
}
|
||||
return model.ProviderUnhealthy, latency(), "health check failed"
|
||||
}
|
||||
for _, k := range keys {
|
||||
if !k.Enabled {
|
||||
continue
|
||||
}
|
||||
apiKey, err := s.DecryptAPIKey(k.APIKeyEnc)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
var checkErr error
|
||||
if p.Type == model.ProviderAnthropic {
|
||||
checkErr = s.anthropic.HealthCheckWithConfig(ctx, apiKey, cfg)
|
||||
} else {
|
||||
checkErr = s.openai.HealthCheckWithConfig(ctx, apiKey, cfg)
|
||||
}
|
||||
if checkErr == nil {
|
||||
return model.ProviderHealthy, latency(), ""
|
||||
}
|
||||
}
|
||||
return model.ProviderUnhealthy, latency(), "health check failed"
|
||||
|
||||
case provider.EndpointEmbeddings, provider.EndpointRerank:
|
||||
body, err := healthProbeBody(s, p, endpoint)
|
||||
if err != nil {
|
||||
return model.ProviderUnhealthy, latency(), err.Error()
|
||||
}
|
||||
if len(keys) == 0 && p.Type == model.ProviderOllama {
|
||||
ok, msg := s.probeEndpoint(ctx, "", cfg, endpoint, body)
|
||||
if ok {
|
||||
return model.ProviderHealthy, latency(), ""
|
||||
}
|
||||
return model.ProviderUnhealthy, latency(), msg
|
||||
}
|
||||
var lastErr string
|
||||
for _, k := range keys {
|
||||
if !k.Enabled {
|
||||
continue
|
||||
}
|
||||
apiKey, err := s.DecryptAPIKey(k.APIKeyEnc)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if ok, msg := s.probeEndpoint(ctx, apiKey, cfg, endpoint, body); ok {
|
||||
return model.ProviderHealthy, latency(), ""
|
||||
} else if msg != "" {
|
||||
lastErr = msg
|
||||
}
|
||||
}
|
||||
if lastErr == "" {
|
||||
lastErr = "health check failed"
|
||||
}
|
||||
return model.ProviderUnhealthy, latency(), lastErr
|
||||
default:
|
||||
return model.ProviderUnhealthy, latency(), "unsupported health check endpoint"
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) probeEndpoint(
|
||||
ctx context.Context,
|
||||
apiKey string,
|
||||
cfg provider.ProviderConfig,
|
||||
endpoint provider.EndpointType,
|
||||
body []byte,
|
||||
) (bool, string) {
|
||||
resp, err := s.openai.ForwardWithConfig(ctx, apiKey, cfg, endpoint, body)
|
||||
if err != nil {
|
||||
return false, err.Error()
|
||||
}
|
||||
if resp == nil {
|
||||
return false, "empty response"
|
||||
}
|
||||
if isHealthCheckHTTPStatus(resp.StatusCode) {
|
||||
return true, ""
|
||||
}
|
||||
return false, fmt.Sprintf("%s: HTTP %d", endpoint, resp.StatusCode)
|
||||
}
|
||||
|
||||
func healthProbeBody(s *Service, p *model.Provider, endpoint provider.EndpointType) ([]byte, error) {
|
||||
modelID := probeModelID(s, p.ID, endpoint)
|
||||
switch endpoint {
|
||||
case provider.EndpointEmbeddings:
|
||||
return json.Marshal(map[string]string{"model": modelID, "input": "."})
|
||||
case provider.EndpointRerank:
|
||||
return json.Marshal(map[string]any{
|
||||
"model": modelID,
|
||||
"query": "ping",
|
||||
"documents": []string{"ping"},
|
||||
})
|
||||
default:
|
||||
return nil, fmt.Errorf("unsupported probe endpoint %s", endpoint)
|
||||
}
|
||||
}
|
||||
|
||||
func probeModelID(s *Service, providerID uint, endpoint provider.EndpointType) string {
|
||||
for _, m := range s.cache.GetModels(providerID) {
|
||||
if m.Enabled && m.ModelID != "" {
|
||||
return m.ModelID
|
||||
}
|
||||
}
|
||||
switch endpoint {
|
||||
case provider.EndpointEmbeddings:
|
||||
return "text-embedding-3-small"
|
||||
case provider.EndpointRerank:
|
||||
return "rerank-multilingual-v3.0"
|
||||
default:
|
||||
return "health-check"
|
||||
}
|
||||
}
|
||||
|
||||
func isHealthCheckHTTPStatus(code int) bool {
|
||||
if code >= 200 && code < 300 {
|
||||
return true
|
||||
}
|
||||
// Upstream reachable but rejected payload/model — still counts as healthy.
|
||||
return code == 400 || code == 422
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/provider"
|
||||
)
|
||||
|
||||
func TestIsHealthCheckHTTPStatus(t *testing.T) {
|
||||
cases := map[int]bool{
|
||||
200: true,
|
||||
204: true,
|
||||
400: true,
|
||||
422: true,
|
||||
401: false,
|
||||
404: false,
|
||||
500: false,
|
||||
}
|
||||
for code, want := range cases {
|
||||
if got := isHealthCheckHTTPStatus(code); got != want {
|
||||
t.Fatalf("code %d: got %v want %v", code, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestHealthCheckEndpointsEmbedding(t *testing.T) {
|
||||
s := &Service{}
|
||||
p := &model.Provider{
|
||||
Category: model.CategoryEmbedding,
|
||||
Type: model.ProviderOpenAI,
|
||||
AllowedEndpointsJSON: `["embeddings"]`,
|
||||
}
|
||||
eps := s.healthCheckEndpoints(p)
|
||||
if len(eps) != 1 || eps[0] != provider.EndpointEmbeddings {
|
||||
t.Fatalf("got %v", eps)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHealthCheckEndpointsEmbeddingWithListModels(t *testing.T) {
|
||||
s := &Service{}
|
||||
p := &model.Provider{
|
||||
Category: model.CategoryEmbedding,
|
||||
Type: model.ProviderOpenAI,
|
||||
AllowedEndpointsJSON: `["list_models","embeddings"]`,
|
||||
}
|
||||
eps := s.healthCheckEndpoints(p)
|
||||
if len(eps) != 2 || eps[0] != provider.EndpointListModels || eps[1] != provider.EndpointEmbeddings {
|
||||
t.Fatalf("got %v", eps)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/provider"
|
||||
)
|
||||
|
||||
const maxLogBodyBytes = 16384
|
||||
|
||||
func truncateLogBody(b []byte) string {
|
||||
if len(b) == 0 {
|
||||
return ""
|
||||
}
|
||||
if len(b) <= maxLogBodyBytes {
|
||||
return string(b)
|
||||
}
|
||||
return string(b[:maxLogBodyBytes]) + "\n...(truncated)"
|
||||
}
|
||||
|
||||
func extractModelName(body []byte) string {
|
||||
if len(body) == 0 {
|
||||
return ""
|
||||
}
|
||||
var payload struct {
|
||||
Model string `json:"model"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &payload); err != nil {
|
||||
return ""
|
||||
}
|
||||
return payload.Model
|
||||
}
|
||||
|
||||
func describeForwardError(err error, resp *provider.ChatResponse) string {
|
||||
if err != nil {
|
||||
return err.Error()
|
||||
}
|
||||
if resp == nil {
|
||||
return "unknown error"
|
||||
}
|
||||
if resp.StatusCode < 400 {
|
||||
return ""
|
||||
}
|
||||
if msg := extractAPIErrorMessage(resp.Body); msg != "" {
|
||||
return fmt.Sprintf("status %d: %s", resp.StatusCode, msg)
|
||||
}
|
||||
if len(resp.Body) > 0 {
|
||||
return fmt.Sprintf("status %d: %s", resp.StatusCode, truncateLogBody(resp.Body))
|
||||
}
|
||||
return fmt.Sprintf("status %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
func extractAPIErrorMessage(body []byte) string {
|
||||
if len(body) == 0 {
|
||||
return ""
|
||||
}
|
||||
var openAI struct {
|
||||
Error struct {
|
||||
Message string `json:"message"`
|
||||
Type string `json:"type"`
|
||||
} `json:"error"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &openAI); err == nil {
|
||||
if openAI.Error.Message != "" {
|
||||
if openAI.Error.Type != "" {
|
||||
return openAI.Error.Type + ": " + openAI.Error.Message
|
||||
}
|
||||
return openAI.Error.Message
|
||||
}
|
||||
}
|
||||
var simple struct {
|
||||
Message string `json:"message"`
|
||||
Error string `json:"error"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &simple); err == nil {
|
||||
if simple.Message != "" {
|
||||
return simple.Message
|
||||
}
|
||||
if simple.Error != "" {
|
||||
return simple.Error
|
||||
}
|
||||
}
|
||||
s := strings.TrimSpace(string(body))
|
||||
if len(s) > 240 {
|
||||
return s[:240] + "..."
|
||||
}
|
||||
return s
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/provider"
|
||||
)
|
||||
|
||||
func TestExtractAPIErrorMessage_OpenAI(t *testing.T) {
|
||||
body := []byte(`{"error":{"message":"Invalid API key","type":"invalid_request_error"}}`)
|
||||
got := extractAPIErrorMessage(body)
|
||||
if got != "invalid_request_error: Invalid API key" {
|
||||
t.Fatalf("got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDescribeForwardError_WithBody(t *testing.T) {
|
||||
got := describeForwardError(nil, &provider.ChatResponse{
|
||||
StatusCode: 400,
|
||||
Body: []byte(`{"error":{"message":"model not found"}}`),
|
||||
})
|
||||
if got != "status 400: model not found" {
|
||||
t.Fatalf("got %q", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
)
|
||||
|
||||
// IngressModelID returns the model identifier exposed to ingress clients.
|
||||
func IngressModelID(m model.ModelEntry) string {
|
||||
return m.IngressID()
|
||||
}
|
||||
|
||||
func modelEntryMatches(m model.ModelEntry, name string) bool {
|
||||
return m.MatchesIngressName(name)
|
||||
}
|
||||
|
||||
func (s *Service) resolveUpstreamModelID(providerID uint, ingressModel string) string {
|
||||
if ingressModel == "" {
|
||||
return ""
|
||||
}
|
||||
for _, m := range s.cache.GetModels(providerID) {
|
||||
if modelEntryMatches(m, ingressModel) {
|
||||
return m.ModelID
|
||||
}
|
||||
}
|
||||
return ingressModel
|
||||
}
|
||||
|
||||
func rewriteModelInBody(body []byte, upstreamModel string) ([]byte, error) {
|
||||
if upstreamModel == "" || len(body) == 0 {
|
||||
return body, nil
|
||||
}
|
||||
var payload map[string]any
|
||||
if err := json.Unmarshal(body, &payload); err != nil {
|
||||
return body, nil
|
||||
}
|
||||
current, _ := payload["model"].(string)
|
||||
if current == "" || current == upstreamModel {
|
||||
return body, nil
|
||||
}
|
||||
payload["model"] = upstreamModel
|
||||
return json.Marshal(payload)
|
||||
}
|
||||
|
||||
func (s *Service) rewriteBodyForProvider(providerID uint, body []byte) ([]byte, error) {
|
||||
ingressModel := extractModelName(body)
|
||||
if ingressModel == "" {
|
||||
return body, nil
|
||||
}
|
||||
upstream := s.resolveUpstreamModelID(providerID, ingressModel)
|
||||
return rewriteModelInBody(body, upstream)
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
)
|
||||
|
||||
func TestIngressModelID(t *testing.T) {
|
||||
if got := IngressModelID(model.ModelEntry{ModelID: "text-embedding-3-small", Alias: "my-embed"}); got != "my-embed" {
|
||||
t.Fatalf("got %q", got)
|
||||
}
|
||||
if got := (model.ModelEntry{ModelID: "gpt-4o"}).IngressID(); got != "gpt-4o" {
|
||||
t.Fatalf("got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRewriteModelInBody(t *testing.T) {
|
||||
body := []byte(`{"model":"my-embed","input":"hi"}`)
|
||||
out, err := rewriteModelInBody(body, "text-embedding-3-small")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var payload map[string]string
|
||||
if err := json.Unmarshal(out, &payload); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if payload["model"] != "text-embedding-3-small" {
|
||||
t.Fatalf("got %q", payload["model"])
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,104 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/store/cache"
|
||||
)
|
||||
|
||||
type routeTarget struct {
|
||||
ProviderID uint
|
||||
ProviderKeyID uint // 0 = auto load balance
|
||||
}
|
||||
|
||||
func (s *Service) resolveRoute(ingressID uint, modelName string) *routeTarget {
|
||||
rules := s.cache.ListRoutingRules(ingressID)
|
||||
var best *model.RoutingRule
|
||||
for i := range rules {
|
||||
r := rules[i]
|
||||
if !r.Enabled {
|
||||
continue
|
||||
}
|
||||
if !matchPattern(r.ModelPattern, modelName) {
|
||||
continue
|
||||
}
|
||||
if best == nil || r.Priority > best.Priority {
|
||||
best = &r
|
||||
}
|
||||
}
|
||||
if best == nil {
|
||||
return nil
|
||||
}
|
||||
return &routeTarget{ProviderID: best.ProviderID, ProviderKeyID: best.ProviderKeyID}
|
||||
}
|
||||
|
||||
func matchPattern(pattern, model string) bool {
|
||||
if pattern == "" || pattern == "*" {
|
||||
return true
|
||||
}
|
||||
if strings.HasSuffix(pattern, "*") {
|
||||
return strings.HasPrefix(model, strings.TrimSuffix(pattern, "*"))
|
||||
}
|
||||
return pattern == model
|
||||
}
|
||||
|
||||
func (s *Service) findProvidersForModel(modelName string, allowed []string, category model.ProviderCategory, route *routeTarget) ([]model.Provider, error) {
|
||||
if route != nil {
|
||||
if p, ok := s.cache.GetProvider(route.ProviderID); ok && p.Category == category && p.Enabled {
|
||||
if cache.MatchesList(allowed, p.Name) || cache.MatchesList(allowed, string(p.Type)) || cache.MatchesList(allowed, "*") {
|
||||
return []model.Provider{p}, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
var matched []model.Provider
|
||||
for _, p := range s.cache.ListProviders() {
|
||||
if p.Category != category || !model.IsCategorySupported(p.Category) || !p.Enabled {
|
||||
continue
|
||||
}
|
||||
if !cache.MatchesList(allowed, p.Name) && !cache.MatchesList(allowed, string(p.Type)) {
|
||||
continue
|
||||
}
|
||||
if s.providerSupportsModel(&p, modelName) {
|
||||
matched = append(matched, p)
|
||||
}
|
||||
}
|
||||
if len(matched) == 0 {
|
||||
if parts := strings.SplitN(modelName, "/", 2); len(parts) == 2 {
|
||||
if p, ok := s.cache.GetProviderByName(parts[0]); ok && p.Category == category && p.Enabled {
|
||||
if s.providerSupportsModel(&p, parts[1]) {
|
||||
return []model.Provider{p}, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil, errNoProvider(modelName)
|
||||
}
|
||||
return matched, nil
|
||||
}
|
||||
|
||||
func (s *Service) providerSupportsModel(p *model.Provider, modelName string) bool {
|
||||
models := s.cache.GetModels(p.ID)
|
||||
if len(models) == 0 {
|
||||
for _, k := range s.cache.ListProviderKeys(p.ID) {
|
||||
allowed := cache.ParseJSONList(k.ModelsJSON)
|
||||
if cache.MatchesList(allowed, modelName) || cache.MatchesList(allowed, "*") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
for _, m := range models {
|
||||
if m.Enabled && modelEntryMatches(m, modelName) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func errNoProvider(model string) error {
|
||||
return &gatewayError{msg: "no provider found for model " + model}
|
||||
}
|
||||
|
||||
type gatewayError struct{ msg string }
|
||||
|
||||
func (e *gatewayError) Error() string { return e.msg }
|
||||
@@ -0,0 +1,475 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/auth"
|
||||
"github.com/rose_cat707/luminary/internal/limiter"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/pricing"
|
||||
"github.com/rose_cat707/luminary/internal/provider"
|
||||
"github.com/rose_cat707/luminary/internal/provider/anthropic"
|
||||
"github.com/rose_cat707/luminary/internal/provider/comfyui"
|
||||
"github.com/rose_cat707/luminary/internal/provider/openai"
|
||||
"github.com/rose_cat707/luminary/internal/settings"
|
||||
"github.com/rose_cat707/luminary/internal/store/cache"
|
||||
)
|
||||
|
||||
type Service struct {
|
||||
cache *cache.Store
|
||||
settings *settings.Runtime
|
||||
openai *openai.Client
|
||||
anthropic *anthropic.Client
|
||||
streamClient *provider.HTTPClientWrapper
|
||||
}
|
||||
|
||||
func New(c *cache.Store, rt *settings.Runtime) *Service {
|
||||
return &Service{
|
||||
cache: c,
|
||||
settings: rt,
|
||||
openai: openai.New(),
|
||||
anthropic: anthropic.New(),
|
||||
streamClient: &provider.HTTPClientWrapper{Client: provider.NewStreamHTTPClient()},
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) ResolveIngressKey(plainKey string) (*model.IngressKey, error) {
|
||||
k, ok := s.cache.LookupIngressKey(plainKey)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("invalid api key")
|
||||
}
|
||||
if k.ExpiresAt != nil && k.ExpiresAt.Before(time.Now()) {
|
||||
return nil, fmt.Errorf("api key expired")
|
||||
}
|
||||
return k, nil
|
||||
}
|
||||
|
||||
func (s *Service) providerConfig(p *model.Provider) provider.ProviderConfig {
|
||||
return provider.ProviderConfig{
|
||||
Type: p.Type,
|
||||
Category: p.Category,
|
||||
BaseURL: p.BaseURL,
|
||||
EndpointPaths: provider.ParseEndpointPaths(p.EndpointPathsJSON),
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) DecryptAPIKey(enc string) (string, error) {
|
||||
return auth.Decrypt(enc, s.settings.EncryptionKey())
|
||||
}
|
||||
|
||||
func (s *Service) providerAllowsEndpoint(p *model.Provider, endpoint provider.EndpointType) bool {
|
||||
allowed := provider.ParseAllowedEndpoints(p.AllowedEndpointsJSON, p.Category)
|
||||
return provider.IsEndpointAllowed(allowed, endpoint)
|
||||
}
|
||||
|
||||
func (s *Service) canServeEndpoint(p *model.Provider, endpoint provider.EndpointType) bool {
|
||||
if s.providerAllowsEndpoint(p, endpoint) {
|
||||
return true
|
||||
}
|
||||
return endpoint == provider.EndpointResponses && s.providerAllowsEndpoint(p, provider.EndpointChatCompletion)
|
||||
}
|
||||
|
||||
func (s *Service) shouldAdaptResponses(p *model.Provider) bool {
|
||||
return !s.providerAllowsEndpoint(p, provider.EndpointResponses) && s.providerAllowsEndpoint(p, provider.EndpointChatCompletion)
|
||||
}
|
||||
|
||||
func (s *Service) forward(ctx context.Context, p *model.Provider, apiKey string, endpoint provider.EndpointType, body []byte) (*provider.ChatResponse, error) {
|
||||
var err error
|
||||
body, err = s.rewriteBodyForProvider(p.ID, body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
actualEndpoint := endpoint
|
||||
actualBody := body
|
||||
adaptResponses := false
|
||||
|
||||
if endpoint == provider.EndpointResponses {
|
||||
if s.shouldAdaptResponses(p) {
|
||||
adapted, err := provider.ResponsesToChatCompletion(body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
actualBody = adapted
|
||||
actualEndpoint = provider.EndpointChatCompletion
|
||||
adaptResponses = true
|
||||
} else if !s.providerAllowsEndpoint(p, endpoint) {
|
||||
return nil, fmt.Errorf("endpoint %s not enabled on provider %s", endpoint, p.Name)
|
||||
}
|
||||
} else if !s.providerAllowsEndpoint(p, endpoint) {
|
||||
return nil, fmt.Errorf("endpoint %s not enabled on provider %s", endpoint, p.Name)
|
||||
}
|
||||
|
||||
resp, err := s.forwardRaw(ctx, p, apiKey, actualEndpoint, actualBody)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if endpoint == provider.EndpointResponses && !adaptResponses && resp != nil && resp.StatusCode == http.StatusNotFound && s.providerAllowsEndpoint(p, provider.EndpointChatCompletion) {
|
||||
adaptedBody, adaptErr := provider.ResponsesToChatCompletion(body)
|
||||
if adaptErr == nil {
|
||||
if resp2, err2 := s.forwardRaw(ctx, p, apiKey, provider.EndpointChatCompletion, adaptedBody); err2 == nil && resp2 != nil {
|
||||
resp = resp2
|
||||
adaptResponses = true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if adaptResponses && resp != nil && resp.StatusCode < 400 {
|
||||
converted, convErr := provider.ChatCompletionToResponses(resp.Body)
|
||||
if convErr == nil {
|
||||
resp.Body = converted
|
||||
}
|
||||
}
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
func (s *Service) forwardRaw(ctx context.Context, p *model.Provider, apiKey string, endpoint provider.EndpointType, body []byte) (*provider.ChatResponse, error) {
|
||||
cfg := s.providerConfig(p)
|
||||
if p.Type == model.ProviderAnthropic && endpoint == provider.EndpointChatCompletion {
|
||||
return s.anthropic.ForwardWithConfig(ctx, apiKey, cfg, endpoint, body)
|
||||
}
|
||||
return s.openai.ForwardWithConfig(ctx, apiKey, cfg, endpoint, body)
|
||||
}
|
||||
|
||||
func (s *Service) forwardStream(ctx context.Context, p *model.Provider, apiKey string, endpoint provider.EndpointType, body []byte, w io.Writer, flusher http.Flusher) (*provider.StreamResult, error) {
|
||||
var err error
|
||||
body, err = s.rewriteBodyForProvider(p.ID, body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
actualEndpoint := endpoint
|
||||
actualBody := body
|
||||
adaptResponses := false
|
||||
|
||||
if endpoint == provider.EndpointResponses {
|
||||
if s.shouldAdaptResponses(p) {
|
||||
adapted, err := provider.ResponsesToChatCompletion(body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
actualBody = adapted
|
||||
actualEndpoint = provider.EndpointChatCompletion
|
||||
adaptResponses = true
|
||||
} else if !s.providerAllowsEndpoint(p, endpoint) {
|
||||
return nil, fmt.Errorf("endpoint %s not enabled", endpoint)
|
||||
}
|
||||
} else if !s.providerAllowsEndpoint(p, endpoint) {
|
||||
return nil, fmt.Errorf("endpoint %s not enabled", endpoint)
|
||||
}
|
||||
|
||||
cfg := s.providerConfig(p)
|
||||
url := cfg.ResolveEndpointURL(actualEndpoint)
|
||||
|
||||
if adaptResponses {
|
||||
pr, pw := io.Pipe()
|
||||
errCh := make(chan error, 1)
|
||||
go func() {
|
||||
_, err := provider.ForwardStreamHTTP(ctx, s.streamClient.Client, "POST", url, apiKey, p.Type, actualBody, pw, nil)
|
||||
_ = pw.Close()
|
||||
errCh <- err
|
||||
}()
|
||||
result, err := provider.AdaptResponsesStream(body, pr, w, flusher)
|
||||
if streamErr := <-errCh; err == nil && streamErr != nil {
|
||||
err = streamErr
|
||||
}
|
||||
return result, err
|
||||
}
|
||||
|
||||
return provider.ForwardStreamHTTP(ctx, s.streamClient.Client, "POST", url, apiKey, p.Type, actualBody, w, flusher)
|
||||
}
|
||||
|
||||
type forwardParams struct {
|
||||
ingress *model.IngressKey
|
||||
endpoint provider.EndpointType
|
||||
category model.ProviderCategory
|
||||
body []byte
|
||||
stream bool
|
||||
}
|
||||
|
||||
func (s *Service) prepareForward(params forwardParams) (modelName string, providers []model.Provider, route *routeTarget, err error) {
|
||||
if len(params.body) > 0 {
|
||||
var payload map[string]any
|
||||
if err := json.Unmarshal(params.body, &payload); err != nil {
|
||||
return "", nil, nil, fmt.Errorf("%w: %v", ErrInvalidJSON, err)
|
||||
}
|
||||
modelName, _ = payload["model"].(string)
|
||||
}
|
||||
if params.endpoint != provider.EndpointListModels && modelName == "" {
|
||||
return "", nil, nil, ErrModelRequired
|
||||
}
|
||||
allowedProviders := cache.ParseJSONList(params.ingress.AllowedProvidersJSON)
|
||||
if modelName != "" {
|
||||
allowedModels := cache.ParseJSONList(params.ingress.AllowedModelsJSON)
|
||||
if !cache.MatchesList(allowedModels, modelName) && !cache.MatchesList(allowedModels, "*") {
|
||||
return "", nil, nil, ErrModelNotAllowed
|
||||
}
|
||||
}
|
||||
if !limiter.CheckIngressBudget(params.ingress, 0) {
|
||||
return "", nil, nil, ErrBudgetExceeded
|
||||
}
|
||||
if !limiter.CheckRateLimit(s.cache, params.ingress.ID, params.ingress.RateLimitRPM, params.ingress.RateLimitTPM, 1000) {
|
||||
return "", nil, nil, ErrRateLimitExceeded
|
||||
}
|
||||
route = s.resolveRoute(params.ingress.ID, modelName)
|
||||
providers, err = s.findProvidersForModel(modelName, allowedProviders, params.category, route)
|
||||
return modelName, providers, route, err
|
||||
}
|
||||
|
||||
func (s *Service) recordUsage(ingress *model.IngressKey, att *attempt, clientIP, modelName, endpoint string, stream bool, tokensIn, tokensOut int, latency int64, status, errMsg, requestBody, responseBody string) {
|
||||
var providerKeyID, providerID uint
|
||||
if att != nil {
|
||||
providerKeyID = att.key.ID
|
||||
providerID = att.provider.ID
|
||||
}
|
||||
cost := pricing.Estimate(modelName, tokensIn, tokensOut, endpoint)
|
||||
s.cache.RecordUsage(cache.UsageDelta{
|
||||
IngressKeyID: ingress.ID,
|
||||
ProviderKeyID: providerKeyID,
|
||||
ProviderID: providerID,
|
||||
ClientIP: clientIP,
|
||||
Endpoint: endpoint,
|
||||
Model: modelName,
|
||||
TokensIn: tokensIn,
|
||||
TokensOut: tokensOut,
|
||||
Cost: cost,
|
||||
LatencyMs: latency,
|
||||
Status: status,
|
||||
Stream: stream,
|
||||
ErrorMsg: errMsg,
|
||||
RequestBody: requestBody,
|
||||
ResponseBody: responseBody,
|
||||
BudgetUsed: cost,
|
||||
Requests: 1,
|
||||
Tokens: int64(tokensIn + tokensOut),
|
||||
})
|
||||
}
|
||||
|
||||
func (s *Service) ForwardEndpoint(ctx context.Context, ingress *model.IngressKey, endpoint provider.EndpointType, rawBody []byte, clientIP string) (*provider.ChatResponse, error) {
|
||||
category := categoryForEndpoint(endpoint)
|
||||
modelName, providers, route, err := s.prepareForward(forwardParams{
|
||||
ingress: ingress, endpoint: endpoint, category: category, body: rawBody,
|
||||
})
|
||||
if err != nil {
|
||||
s.recordUsage(ingress, nil, clientIP, modelNameOrBody(rawBody, modelName), string(endpoint), false, 0, 0, 0, "error", err.Error(), truncateLogBody(rawBody), "")
|
||||
return nil, err
|
||||
}
|
||||
attempts, err := s.buildAttempts(providers, modelName, route)
|
||||
if err != nil {
|
||||
s.recordUsage(ingress, nil, clientIP, modelName, string(endpoint), false, 0, 0, 0, "error", err.Error(), truncateLogBody(rawBody), "")
|
||||
return nil, err
|
||||
}
|
||||
start := time.Now()
|
||||
resp, att, err := s.forwardWithFailover(ctx, attempts, endpoint, rawBody)
|
||||
latency := time.Since(start).Milliseconds()
|
||||
status := "success"
|
||||
if err != nil || (resp != nil && resp.StatusCode >= 400) {
|
||||
status = "error"
|
||||
}
|
||||
errMsg := describeForwardError(err, resp)
|
||||
tokensIn, tokensOut := 0, 0
|
||||
var respBody []byte
|
||||
if resp != nil {
|
||||
tokensIn, tokensOut = resp.TokensIn, resp.TokensOut
|
||||
respBody = resp.Body
|
||||
}
|
||||
s.recordUsage(ingress, att, clientIP, modelName, string(endpoint), false, tokensIn, tokensOut, latency, status, errMsg, truncateLogBody(rawBody), truncateLogBody(respBody))
|
||||
if err != nil && resp != nil && resp.StatusCode >= 400 {
|
||||
return resp, nil
|
||||
}
|
||||
return resp, err
|
||||
}
|
||||
|
||||
func modelNameOrBody(body []byte, modelName string) string {
|
||||
if modelName != "" {
|
||||
return modelName
|
||||
}
|
||||
return extractModelName(body)
|
||||
}
|
||||
|
||||
func (s *Service) ForwardStream(ctx context.Context, ingress *model.IngressKey, endpoint provider.EndpointType, rawBody []byte, clientIP string, w io.Writer, flusher http.Flusher) error {
|
||||
category := categoryForEndpoint(endpoint)
|
||||
modelName, providers, route, err := s.prepareForward(forwardParams{
|
||||
ingress: ingress, endpoint: endpoint, category: category, body: rawBody, stream: true,
|
||||
})
|
||||
if err != nil {
|
||||
s.recordUsage(ingress, nil, clientIP, modelNameOrBody(rawBody, modelName), string(endpoint), true, 0, 0, 0, "error", err.Error(), truncateLogBody(rawBody), "")
|
||||
return err
|
||||
}
|
||||
attempts, err := s.buildAttempts(providers, modelName, route)
|
||||
if err != nil {
|
||||
s.recordUsage(ingress, nil, clientIP, modelName, string(endpoint), true, 0, 0, 0, "error", err.Error(), truncateLogBody(rawBody), "")
|
||||
return err
|
||||
}
|
||||
var lastErr error
|
||||
var lastAtt *attempt
|
||||
maxRetries := s.settings.GatewayMaxRetries()
|
||||
if maxRetries < 1 {
|
||||
maxRetries = 1
|
||||
}
|
||||
backoffMs := s.settings.GatewayRetryBackoffMs()
|
||||
for _, att := range attempts {
|
||||
if !s.canServeEndpoint(att.provider, endpoint) {
|
||||
continue
|
||||
}
|
||||
for try := 0; try < maxRetries; try++ {
|
||||
if try > 0 {
|
||||
time.Sleep(time.Duration(backoffMs) * time.Millisecond)
|
||||
}
|
||||
start := time.Now()
|
||||
result, err := s.forwardStream(ctx, att.provider, att.apiKey, endpoint, rawBody, w, flusher)
|
||||
latency := time.Since(start).Milliseconds()
|
||||
lastAtt = &att
|
||||
if err != nil {
|
||||
lastErr = err
|
||||
continue
|
||||
}
|
||||
if result == nil {
|
||||
lastErr = fmt.Errorf("empty upstream stream response")
|
||||
continue
|
||||
}
|
||||
if result.StatusCode < 400 {
|
||||
s.recordUsage(ingress, &att, clientIP, modelName, string(endpoint), true, result.TokensIn, result.TokensOut, latency, "success", "", truncateLogBody(rawBody), "[SSE stream]")
|
||||
return nil
|
||||
}
|
||||
errMsg := describeForwardError(nil, &provider.ChatResponse{StatusCode: result.StatusCode, Body: result.Body})
|
||||
lastErr = fmt.Errorf("%s", errMsg)
|
||||
respBody := "[SSE stream]"
|
||||
if len(result.Body) > 0 {
|
||||
respBody = truncateLogBody(result.Body)
|
||||
}
|
||||
if passThroughUpstreamStatus(result.StatusCode) {
|
||||
s.recordUsage(ingress, &att, clientIP, modelName, string(endpoint), true, result.TokensIn, result.TokensOut, latency, "error", errMsg, truncateLogBody(rawBody), respBody)
|
||||
return nil
|
||||
}
|
||||
if tryNextKeyOnUpstreamStatus(result.StatusCode) {
|
||||
break
|
||||
}
|
||||
if retryableUpstreamStatus(result.StatusCode) {
|
||||
continue
|
||||
}
|
||||
s.recordUsage(ingress, &att, clientIP, modelName, string(endpoint), true, result.TokensIn, result.TokensOut, latency, "error", errMsg, truncateLogBody(rawBody), respBody)
|
||||
return nil
|
||||
}
|
||||
}
|
||||
if lastErr == nil {
|
||||
lastErr = fmt.Errorf("all providers failed")
|
||||
}
|
||||
s.recordUsage(ingress, lastAtt, clientIP, modelName, string(endpoint), true, 0, 0, 0, "error", lastErr.Error(), truncateLogBody(rawBody), "[SSE stream]")
|
||||
return lastErr
|
||||
}
|
||||
|
||||
func categoryForEndpoint(ep provider.EndpointType) model.ProviderCategory {
|
||||
switch ep {
|
||||
case provider.EndpointEmbeddings:
|
||||
return model.CategoryEmbedding
|
||||
case provider.EndpointRerank:
|
||||
return model.CategoryRerank
|
||||
default:
|
||||
return model.CategoryLLM
|
||||
}
|
||||
}
|
||||
|
||||
func listModelsCategories() []model.ProviderCategory {
|
||||
return []model.ProviderCategory{
|
||||
model.CategoryLLM,
|
||||
model.CategoryEmbedding,
|
||||
model.CategoryRerank,
|
||||
}
|
||||
}
|
||||
|
||||
func providerInListModelsCategory(cat model.ProviderCategory) bool {
|
||||
for _, c := range listModelsCategories() {
|
||||
if c == cat {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (s *Service) ListModels(ctx context.Context, ingress *model.IngressKey) ([]map[string]any, error) {
|
||||
allowedProviders := cache.ParseJSONList(ingress.AllowedProvidersJSON)
|
||||
var result []map[string]any
|
||||
var hasLLM bool
|
||||
for _, p := range s.cache.ListProviders() {
|
||||
if !providerInListModelsCategory(p.Category) {
|
||||
continue
|
||||
}
|
||||
if !cache.MatchesList(allowedProviders, p.Name) && !cache.MatchesList(allowedProviders, string(p.Type)) {
|
||||
continue
|
||||
}
|
||||
if !s.providerAllowsEndpoint(&p, provider.EndpointListModels) {
|
||||
continue
|
||||
}
|
||||
if p.Category == model.CategoryLLM {
|
||||
hasLLM = true
|
||||
}
|
||||
for _, m := range s.cache.GetModels(p.ID) {
|
||||
if m.Enabled {
|
||||
result = append(result, map[string]any{
|
||||
"id": IngressModelID(m), "object": "model", "owned_by": p.Name,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(result) == 0 && hasLLM {
|
||||
result = append(result, map[string]any{"id": "gpt-4o-mini", "object": "model", "owned_by": "luminary"})
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Service) FetchModels(ctx context.Context, providerID uint) ([]provider.ModelInfo, error) {
|
||||
p, ok := s.cache.GetProvider(providerID)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("provider not found")
|
||||
}
|
||||
if !s.providerAllowsEndpoint(&p, provider.EndpointListModels) {
|
||||
return nil, fmt.Errorf("list_models endpoint not enabled")
|
||||
}
|
||||
keys := s.cache.ListProviderKeys(providerID)
|
||||
var firstKey string
|
||||
for _, k := range keys {
|
||||
if k.Enabled {
|
||||
plain, err := s.DecryptAPIKey(k.APIKeyEnc)
|
||||
if err == nil {
|
||||
firstKey = plain
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
if firstKey == "" && p.Type != model.ProviderOllama {
|
||||
return nil, fmt.Errorf("no enabled keys")
|
||||
}
|
||||
cfg := s.providerConfig(&p)
|
||||
if p.Type == model.ProviderAnthropic {
|
||||
return s.anthropic.ListModelsWithConfig(ctx, firstKey, cfg)
|
||||
}
|
||||
return s.openai.ListModelsWithConfig(ctx, firstKey, cfg)
|
||||
}
|
||||
|
||||
func (s *Service) HealthCheckProvider(ctx context.Context, providerID uint) (model.ProviderStatus, int64, string) {
|
||||
p, ok := s.cache.GetProvider(providerID)
|
||||
if !ok {
|
||||
return model.ProviderUnhealthy, 0, "not found"
|
||||
}
|
||||
if !model.IsCategorySupported(p.Category) {
|
||||
return model.ProviderUnhealthy, 0, "category not supported"
|
||||
}
|
||||
if p.Type == model.ProviderComfyUI {
|
||||
start := time.Now()
|
||||
if err := comfyui.New(p.BaseURL).HealthCheck(ctx); err != nil {
|
||||
return model.ProviderUnhealthy, time.Since(start).Milliseconds(), err.Error()
|
||||
}
|
||||
return model.ProviderHealthy, time.Since(start).Milliseconds(), ""
|
||||
}
|
||||
keys := s.cache.ListProviderKeys(providerID)
|
||||
if len(keys) == 0 && p.Type != model.ProviderOllama {
|
||||
return model.ProviderUnhealthy, 0, "no provider keys configured"
|
||||
}
|
||||
cfg := s.providerConfig(&p)
|
||||
return s.runHealthChecks(ctx, &p, keys, cfg)
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
package gateway
|
||||
|
||||
import "net/http"
|
||||
|
||||
// retryableUpstreamStatus reports HTTP codes that should trigger retry / failover.
|
||||
func retryableUpstreamStatus(code int) bool {
|
||||
switch code {
|
||||
case http.StatusTooManyRequests,
|
||||
http.StatusRequestTimeout,
|
||||
http.StatusBadGateway,
|
||||
http.StatusServiceUnavailable,
|
||||
http.StatusGatewayTimeout:
|
||||
return true
|
||||
default:
|
||||
return code >= 500
|
||||
}
|
||||
}
|
||||
|
||||
// tryNextKeyOnUpstreamStatus reports auth/rate errors where another provider key may work.
|
||||
func tryNextKeyOnUpstreamStatus(code int) bool {
|
||||
switch code {
|
||||
case http.StatusUnauthorized,
|
||||
http.StatusForbidden,
|
||||
http.StatusTooManyRequests,
|
||||
http.StatusPaymentRequired:
|
||||
return true
|
||||
default:
|
||||
return retryableUpstreamStatus(code)
|
||||
}
|
||||
}
|
||||
|
||||
// passThroughUpstreamStatus reports client errors that should be returned as-is.
|
||||
func passThroughUpstreamStatus(code int) bool {
|
||||
switch code {
|
||||
case http.StatusBadRequest,
|
||||
http.StatusNotFound,
|
||||
http.StatusUnprocessableEntity,
|
||||
http.StatusUnsupportedMediaType:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRetryableUpstreamStatus(t *testing.T) {
|
||||
if !retryableUpstreamStatus(http.StatusTooManyRequests) {
|
||||
t.Fatal("429 should be retryable")
|
||||
}
|
||||
if !retryableUpstreamStatus(http.StatusServiceUnavailable) {
|
||||
t.Fatal("503 should be retryable")
|
||||
}
|
||||
if retryableUpstreamStatus(http.StatusBadRequest) {
|
||||
t.Fatal("400 should not be retryable")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTryNextKeyOnUpstreamStatus(t *testing.T) {
|
||||
if !tryNextKeyOnUpstreamStatus(http.StatusTooManyRequests) {
|
||||
t.Fatal("429 should try next key")
|
||||
}
|
||||
if !tryNextKeyOnUpstreamStatus(http.StatusUnauthorized) {
|
||||
t.Fatal("401 should try next key")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,273 @@
|
||||
package admin
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/rose_cat707/luminary/internal/auth"
|
||||
"github.com/rose_cat707/luminary/internal/auth/loginlimit"
|
||||
"github.com/rose_cat707/luminary/internal/gateway"
|
||||
"github.com/rose_cat707/luminary/internal/middleware"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/monitor"
|
||||
"github.com/rose_cat707/luminary/internal/prediction"
|
||||
"github.com/rose_cat707/luminary/internal/settings"
|
||||
"github.com/rose_cat707/luminary/internal/storage/imagestore"
|
||||
"github.com/rose_cat707/luminary/internal/store/cache"
|
||||
sqlitestore "github.com/rose_cat707/luminary/internal/store/sqlite"
|
||||
)
|
||||
|
||||
type Handler struct {
|
||||
repo *sqlitestore.Repository
|
||||
db *sqlitestore.Store
|
||||
cache *cache.Store
|
||||
gateway *gateway.Service
|
||||
settings *settings.Runtime
|
||||
monitor *monitor.Service
|
||||
loginLimit *loginlimit.Limiter
|
||||
predictions *prediction.Service
|
||||
images *imagestore.Store
|
||||
publicURL func() string
|
||||
}
|
||||
|
||||
func New(repo *sqlitestore.Repository, db *sqlitestore.Store, c *cache.Store, gw *gateway.Service, rt *settings.Runtime, mon *monitor.Service, pred *prediction.Service, imgs *imagestore.Store, publicURL func() string) *Handler {
|
||||
return &Handler{
|
||||
repo: repo,
|
||||
db: db,
|
||||
cache: c,
|
||||
gateway: gw,
|
||||
settings: rt,
|
||||
monitor: mon,
|
||||
loginLimit: loginlimit.New(rt.LoginMaxAttempts(), rt.LoginLockout()),
|
||||
predictions: pred,
|
||||
images: imgs,
|
||||
publicURL: publicURL,
|
||||
}
|
||||
}
|
||||
|
||||
func (h *Handler) publicBaseURL() string {
|
||||
if h.publicURL != nil {
|
||||
return h.publicURL()
|
||||
}
|
||||
return "http://localhost:8293"
|
||||
}
|
||||
|
||||
func (h *Handler) Register(r *gin.RouterGroup) {
|
||||
r.POST("/login", h.Login)
|
||||
r.GET("/auth/status", h.AuthStatus)
|
||||
|
||||
authGroup := r.Group("", middleware.AdminAuth(h.validateSession))
|
||||
authGroup.POST("/logout", h.Logout)
|
||||
authGroup.PUT("/password", h.ChangePassword)
|
||||
|
||||
authGroup.GET("/endpoint-meta", h.ListEndpointMeta)
|
||||
authGroup.GET("/providers", h.ListProviders)
|
||||
authGroup.POST("/providers", h.CreateProvider)
|
||||
authGroup.GET("/providers/:id", h.GetProvider)
|
||||
authGroup.PUT("/providers/:id", h.UpdateProvider)
|
||||
authGroup.DELETE("/providers/:id", h.DeleteProvider)
|
||||
authGroup.POST("/providers/:id/sync-models", h.SyncModels)
|
||||
|
||||
authGroup.GET("/providers/:id/keys", h.ListProviderKeys)
|
||||
authGroup.POST("/providers/:id/keys", h.CreateProviderKey)
|
||||
authGroup.PUT("/provider-keys/:keyId", h.UpdateProviderKey)
|
||||
authGroup.DELETE("/provider-keys/:keyId", h.DeleteProviderKey)
|
||||
|
||||
authGroup.GET("/providers/:id/models", h.ListModels)
|
||||
authGroup.POST("/providers/:id/models", h.CreateModel)
|
||||
authGroup.PUT("/providers/:id/models/:modelId", h.UpdateModel)
|
||||
authGroup.DELETE("/providers/:id/models/:modelId", h.DeleteModel)
|
||||
authGroup.PUT("/models/:modelId", h.UpdateModel)
|
||||
authGroup.DELETE("/models/:modelId", h.DeleteModel)
|
||||
|
||||
authGroup.GET("/ingress-keys", h.ListIngressKeys)
|
||||
authGroup.POST("/ingress-keys", h.CreateIngressKey)
|
||||
authGroup.GET("/ingress-keys/:id", h.GetIngressKey)
|
||||
authGroup.PUT("/ingress-keys/:id", h.UpdateIngressKey)
|
||||
authGroup.DELETE("/ingress-keys/:id", h.DeleteIngressKey)
|
||||
authGroup.GET("/ingress-keys/:id/usage", h.IngressUsage)
|
||||
|
||||
authGroup.GET("/ingress-keys/:id/routing-rules", h.ListRoutingRules)
|
||||
authGroup.POST("/ingress-keys/:id/routing-rules", h.CreateRoutingRule)
|
||||
authGroup.PUT("/routing-rules/:ruleId", h.UpdateRoutingRule)
|
||||
authGroup.DELETE("/routing-rules/:ruleId", h.DeleteRoutingRule)
|
||||
|
||||
authGroup.GET("/workflows/param-meta", h.GetWorkflowParamMeta)
|
||||
authGroup.POST("/workflows/analyze", h.AnalyzeWorkflow)
|
||||
authGroup.GET("/predictions/:id", h.GetPrediction)
|
||||
authGroup.GET("/images/:id", h.ServeAdminImage)
|
||||
|
||||
authGroup.GET("/request-logs", h.ListRequestLogs)
|
||||
authGroup.GET("/request-logs/:id", h.GetRequestLog)
|
||||
authGroup.GET("/pricing/catalog", h.PricingCatalog)
|
||||
|
||||
authGroup.GET("/ip-rules", h.ListIPRules)
|
||||
authGroup.POST("/ip-rules", h.CreateIPRule)
|
||||
authGroup.PUT("/ip-rules/:id", h.UpdateIPRule)
|
||||
authGroup.DELETE("/ip-rules/:id", h.DeleteIPRule)
|
||||
|
||||
authGroup.GET("/settings", h.GetSettings)
|
||||
authGroup.PUT("/settings", h.UpdateSettings)
|
||||
|
||||
authGroup.GET("/dashboard/overview", h.DashboardOverview)
|
||||
authGroup.GET("/dashboard/providers", h.DashboardProviders)
|
||||
authGroup.GET("/dashboard/ingress-usage", h.DashboardIngressUsage)
|
||||
authGroup.GET("/dashboard/egress-usage", h.DashboardEgressUsage)
|
||||
authGroup.GET("/dashboard/provider-model-usage", h.DashboardProviderModelUsage)
|
||||
authGroup.GET("/dashboard/client-ip-usage", h.DashboardClientIPUsage)
|
||||
authGroup.GET("/dashboard/request-trend", h.DashboardRequestTrend)
|
||||
}
|
||||
|
||||
func (h *Handler) validateSession(token string) (bool, time.Time) {
|
||||
hash := auth.HashToken(token)
|
||||
var session model.AdminSession
|
||||
if err := h.db.DB().Where("token_hash = ?", hash).First(&session).Error; err != nil {
|
||||
return false, time.Time{}
|
||||
}
|
||||
return session.ExpiresAt.After(time.Now()), session.ExpiresAt
|
||||
}
|
||||
|
||||
func (h *Handler) Login(c *gin.Context) {
|
||||
ip := middleware.ClientIP(c)
|
||||
if locked, retryAfter := h.loginLimit.Check(ip); locked {
|
||||
sec := int(retryAfter.Seconds()) + 1
|
||||
c.Header("Retry-After", strconv.Itoa(sec))
|
||||
h.recordLoginLog(ip, "", "error", "登录尝试次数过多", "", fmt.Sprintf(`{"retry_after_sec":%d}`, sec))
|
||||
c.JSON(http.StatusTooManyRequests, gin.H{
|
||||
"error": "登录尝试次数过多，请稍后再试",
|
||||
"retry_after_sec": sec,
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
var req struct {
|
||||
Username string `json:"username"`
|
||||
Password string `json:"password"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
h.recordLoginLog(ip, "", "error", "invalid request", "", "")
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
reqBody := loginRequestBody(req.Username)
|
||||
|
||||
var user model.AdminUser
|
||||
if err := h.db.DB().Where("username = ?", req.Username).First(&user).Error; err != nil {
|
||||
h.loginLimit.RecordFailure(ip)
|
||||
h.recordLoginLog(ip, req.Username, "error", "invalid credentials", reqBody, "")
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"error": "invalid credentials"})
|
||||
return
|
||||
}
|
||||
if !auth.CheckPassword(user.PasswordHash, req.Password) {
|
||||
h.loginLimit.RecordFailure(ip)
|
||||
h.recordLoginLog(ip, req.Username, "error", "invalid credentials", reqBody, "")
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"error": "invalid credentials"})
|
||||
return
|
||||
}
|
||||
h.loginLimit.Reset(ip)
|
||||
token, err := generateSessionToken()
|
||||
if err != nil {
|
||||
h.recordLoginLog(ip, req.Username, "error", "session error", reqBody, "")
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "session error"})
|
||||
return
|
||||
}
|
||||
session := model.AdminSession{
|
||||
TokenHash: auth.HashToken(token),
|
||||
ExpiresAt: time.Now().Add(h.settings.SessionTTL()),
|
||||
}
|
||||
if err := h.db.DB().Create(&session).Error; err != nil {
|
||||
h.recordLoginLog(ip, req.Username, "error", "session error", reqBody, "")
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "session error"})
|
||||
return
|
||||
}
|
||||
respBody := fmt.Sprintf(`{"expires_at":%q}`, session.ExpiresAt.Format(time.RFC3339))
|
||||
h.recordLoginLog(ip, req.Username, "success", "", reqBody, respBody)
|
||||
c.SetSameSite(http.SameSiteStrictMode)
|
||||
c.SetCookie(middleware.SessionCookieName(), token, int(h.settings.SessionTTL().Seconds()), "/", "", false, true)
|
||||
c.JSON(http.StatusOK, gin.H{"token": token, "expires_at": session.ExpiresAt})
|
||||
}
|
||||
|
||||
func loginRequestBody(username string) string {
|
||||
b, _ := json.Marshal(map[string]string{"username": username})
|
||||
return string(b)
|
||||
}
|
||||
|
||||
func (h *Handler) recordLoginLog(ip, username, status, errMsg, requestBody, responseBody string) {
|
||||
h.repo.RecordRequestLog(model.UsageRecord{
|
||||
ClientIP: ip,
|
||||
Endpoint: model.EndpointAdminLogin,
|
||||
Model: username,
|
||||
Status: status,
|
||||
ErrorMsg: errMsg,
|
||||
RequestBody: requestBody,
|
||||
ResponseBody: responseBody,
|
||||
})
|
||||
}
|
||||
|
||||
func (h *Handler) Logout(c *gin.Context) {
|
||||
token, _ := c.Get("session_token")
|
||||
if t, ok := token.(string); ok {
|
||||
h.db.DB().Where("token_hash = ?", auth.HashToken(t)).Delete(&model.AdminSession{})
|
||||
}
|
||||
c.SetCookie(middleware.SessionCookieName(), "", -1, "/", "", false, true)
|
||||
c.JSON(http.StatusOK, gin.H{"ok": true})
|
||||
}
|
||||
|
||||
func (h *Handler) AuthStatus(c *gin.Context) {
|
||||
token := c.GetHeader("Authorization")
|
||||
if len(token) > 7 {
|
||||
token = token[7:]
|
||||
}
|
||||
if token == "" {
|
||||
if cookie, err := c.Cookie(middleware.SessionCookieName()); err == nil {
|
||||
token = cookie
|
||||
}
|
||||
}
|
||||
valid := false
|
||||
if token != "" {
|
||||
valid, _ = h.validateSession(token)
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{"authenticated": valid})
|
||||
}
|
||||
|
||||
func (h *Handler) ChangePassword(c *gin.Context) {
|
||||
var req struct {
|
||||
OldPassword string `json:"old_password"`
|
||||
NewPassword string `json:"new_password"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil || req.NewPassword == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
var user model.AdminUser
|
||||
if err := h.db.DB().First(&user).Error; err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "user not found"})
|
||||
return
|
||||
}
|
||||
if !auth.CheckPassword(user.PasswordHash, req.OldPassword) {
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"error": "invalid old password"})
|
||||
return
|
||||
}
|
||||
hash, err := auth.HashPassword(req.NewPassword)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "hash error"})
|
||||
return
|
||||
}
|
||||
user.PasswordHash = hash
|
||||
h.db.DB().Save(&user)
|
||||
c.JSON(http.StatusOK, gin.H{"ok": true})
|
||||
}
|
||||
|
||||
func generateSessionToken() (string, error) {
|
||||
b := make([]byte, 32)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return hex.EncodeToString(b), nil
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
package admin
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strconv"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/provider"
|
||||
"github.com/rose_cat707/luminary/internal/provider/anthropic"
|
||||
"github.com/rose_cat707/luminary/internal/provider/custom"
|
||||
"github.com/rose_cat707/luminary/internal/provider/openai"
|
||||
)
|
||||
|
||||
func parseUint(s string) uint {
|
||||
v, _ := strconv.ParseUint(s, 10, 64)
|
||||
return uint(v)
|
||||
}
|
||||
|
||||
func (h *Handler) gatewayClient(t model.ProviderType) (provider.Client, error) {
|
||||
switch t {
|
||||
case model.ProviderOpenAI, model.ProviderAzureOpenAI, model.ProviderOpenRouter, model.ProviderOllama:
|
||||
return openai.New(), nil
|
||||
case model.ProviderAnthropic:
|
||||
return anthropic.New(), nil
|
||||
case model.ProviderCustom:
|
||||
return custom.New(), nil
|
||||
default:
|
||||
return nil, fmt.Errorf("unsupported provider type: %s", t)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
package admin
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/rose_cat707/luminary/internal/auth"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/store/cache"
|
||||
)
|
||||
|
||||
func (h *Handler) ListIngressKeys(c *gin.Context) {
|
||||
keys := h.cache.ListIngressKeys()
|
||||
c.JSON(http.StatusOK, gin.H{"data": keys})
|
||||
}
|
||||
|
||||
func (h *Handler) CreateIngressKey(c *gin.Context) {
|
||||
var req struct {
|
||||
Name string `json:"name"`
|
||||
BudgetLimit float64 `json:"budget_limit"`
|
||||
RateLimitRPM int `json:"rate_limit_rpm"`
|
||||
RateLimitTPM int `json:"rate_limit_tpm"`
|
||||
AllowedProviders []string `json:"allowed_providers"`
|
||||
AllowedModels []string `json:"allowed_models"`
|
||||
ExpiresAt *time.Time `json:"expires_at"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil || req.Name == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
plainKey, err := generateIngressKey()
|
||||
if err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "key generation failed"})
|
||||
return
|
||||
}
|
||||
hash := auth.HashIngressKey(plainKey)
|
||||
providersJSON := `["*"]`
|
||||
modelsJSON := `["*"]`
|
||||
if req.AllowedProviders != nil {
|
||||
if len(req.AllowedProviders) == 0 {
|
||||
providersJSON = `["*"]`
|
||||
} else {
|
||||
providersJSON = cache.ModelsToJSON(req.AllowedProviders)
|
||||
}
|
||||
}
|
||||
if req.AllowedModels != nil {
|
||||
if len(req.AllowedModels) == 0 {
|
||||
modelsJSON = `["*"]`
|
||||
} else {
|
||||
modelsJSON = cache.ModelsToJSON(req.AllowedModels)
|
||||
}
|
||||
}
|
||||
k := model.IngressKey{
|
||||
Name: req.Name,
|
||||
KeyHash: hash,
|
||||
Prefix: plainKey[:12] + "...",
|
||||
Enabled: true,
|
||||
BudgetLimit: req.BudgetLimit,
|
||||
RateLimitRPM: req.RateLimitRPM,
|
||||
RateLimitTPM: req.RateLimitTPM,
|
||||
AllowedProvidersJSON: providersJSON,
|
||||
AllowedModelsJSON: modelsJSON,
|
||||
ExpiresAt: req.ExpiresAt,
|
||||
}
|
||||
if err := h.db.DB().Create(&k).Error; err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
h.cache.SetIngressKey(k)
|
||||
c.JSON(http.StatusCreated, gin.H{"key": k, "plain_key": plainKey})
|
||||
}
|
||||
|
||||
func (h *Handler) GetIngressKey(c *gin.Context) {
|
||||
id := parseUint(c.Param("id"))
|
||||
k, ok := h.cache.GetIngressKey(id)
|
||||
if !ok {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, k)
|
||||
}
|
||||
|
||||
func (h *Handler) UpdateIngressKey(c *gin.Context) {
|
||||
id := parseUint(c.Param("id"))
|
||||
k, ok := h.cache.GetIngressKey(id)
|
||||
if !ok {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Name string `json:"name"`
|
||||
Enabled *bool `json:"enabled"`
|
||||
BudgetLimit *float64 `json:"budget_limit"`
|
||||
RateLimitRPM *int `json:"rate_limit_rpm"`
|
||||
RateLimitTPM *int `json:"rate_limit_tpm"`
|
||||
AllowedProviders []string `json:"allowed_providers"`
|
||||
AllowedModels []string `json:"allowed_models"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
if req.Name != "" {
|
||||
k.Name = req.Name
|
||||
}
|
||||
if req.Enabled != nil {
|
||||
k.Enabled = *req.Enabled
|
||||
}
|
||||
if req.BudgetLimit != nil {
|
||||
k.BudgetLimit = *req.BudgetLimit
|
||||
}
|
||||
if req.RateLimitRPM != nil {
|
||||
k.RateLimitRPM = *req.RateLimitRPM
|
||||
}
|
||||
if req.RateLimitTPM != nil {
|
||||
k.RateLimitTPM = *req.RateLimitTPM
|
||||
}
|
||||
if req.AllowedProviders != nil {
|
||||
if len(req.AllowedProviders) == 0 {
|
||||
k.AllowedProvidersJSON = `["*"]`
|
||||
} else {
|
||||
k.AllowedProvidersJSON = cache.ModelsToJSON(req.AllowedProviders)
|
||||
}
|
||||
}
|
||||
if req.AllowedModels != nil {
|
||||
if len(req.AllowedModels) == 0 {
|
||||
k.AllowedModelsJSON = `["*"]`
|
||||
} else {
|
||||
k.AllowedModelsJSON = cache.ModelsToJSON(req.AllowedModels)
|
||||
}
|
||||
}
|
||||
h.db.DB().Save(&k)
|
||||
h.cache.SetIngressKey(k)
|
||||
c.JSON(http.StatusOK, k)
|
||||
}
|
||||
|
||||
func (h *Handler) DeleteIngressKey(c *gin.Context) {
|
||||
id := parseUint(c.Param("id"))
|
||||
h.db.DB().Where("ingress_key_id = ?", id).Delete(&model.RoutingRule{})
|
||||
h.db.DB().Delete(&model.IngressKey{}, id)
|
||||
h.cache.DeleteIngressKey(id)
|
||||
h.cache.DeleteRoutingRulesForIngress(id)
|
||||
c.JSON(http.StatusOK, gin.H{"ok": true})
|
||||
}
|
||||
|
||||
func (h *Handler) IngressUsage(c *gin.Context) {
|
||||
id := parseUint(c.Param("id"))
|
||||
var records []model.UsageRecord
|
||||
h.db.DB().Where("ingress_key_id = ?", id).Order("created_at desc").Limit(50).Find(&records)
|
||||
k, _ := h.cache.GetIngressKey(id)
|
||||
c.JSON(http.StatusOK, gin.H{"key": k, "recent": records})
|
||||
}
|
||||
|
||||
func generateIngressKey() (string, error) {
|
||||
b := make([]byte, 24)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return "sk-lum-" + hex.EncodeToString(b), nil
|
||||
}
|
||||
@@ -0,0 +1,243 @@
|
||||
package admin
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
)
|
||||
|
||||
func (h *Handler) ListIPRules(c *gin.Context) {
|
||||
c.JSON(http.StatusOK, gin.H{"data": h.cache.ListIPRules()})
|
||||
}
|
||||
|
||||
func (h *Handler) CreateIPRule(c *gin.Context) {
|
||||
var req struct {
|
||||
CIDR string `json:"cidr"`
|
||||
Type model.IPRuleType `json:"type"`
|
||||
Scope model.IPRuleScope `json:"scope"`
|
||||
Enabled *bool `json:"enabled"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil || req.CIDR == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
if req.Type == "" {
|
||||
req.Type = model.IPRuleAllow
|
||||
}
|
||||
if req.Scope == "" {
|
||||
req.Scope = model.IPScopeProxy
|
||||
}
|
||||
rule := model.IPRule{CIDR: req.CIDR, Type: req.Type, Scope: req.Scope, Enabled: true}
|
||||
if req.Enabled != nil {
|
||||
rule.Enabled = *req.Enabled
|
||||
}
|
||||
if err := h.db.DB().Create(&rule).Error; err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
rules := h.cache.ListIPRules()
|
||||
rules = append(rules, rule)
|
||||
h.cache.SetIPRules(rules)
|
||||
c.JSON(http.StatusCreated, rule)
|
||||
}
|
||||
|
||||
func (h *Handler) UpdateIPRule(c *gin.Context) {
|
||||
id := parseUint(c.Param("id"))
|
||||
var rule model.IPRule
|
||||
if err := h.db.DB().First(&rule, id).Error; err != nil {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
CIDR string `json:"cidr"`
|
||||
Type model.IPRuleType `json:"type"`
|
||||
Scope model.IPRuleScope `json:"scope"`
|
||||
Enabled *bool `json:"enabled"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
if req.CIDR != "" {
|
||||
rule.CIDR = req.CIDR
|
||||
}
|
||||
if req.Type != "" {
|
||||
rule.Type = req.Type
|
||||
}
|
||||
if req.Scope != "" {
|
||||
rule.Scope = req.Scope
|
||||
}
|
||||
if req.Enabled != nil {
|
||||
rule.Enabled = *req.Enabled
|
||||
}
|
||||
h.db.DB().Save(&rule)
|
||||
h.reloadIPRules()
|
||||
c.JSON(http.StatusOK, rule)
|
||||
}
|
||||
|
||||
func (h *Handler) DeleteIPRule(c *gin.Context) {
|
||||
id := parseUint(c.Param("id"))
|
||||
h.db.DB().Delete(&model.IPRule{}, id)
|
||||
h.reloadIPRules()
|
||||
c.JSON(http.StatusOK, gin.H{"ok": true})
|
||||
}
|
||||
|
||||
func (h *Handler) reloadIPRules() {
|
||||
var rules []model.IPRule
|
||||
h.db.DB().Find(&rules)
|
||||
h.cache.SetIPRules(rules)
|
||||
}
|
||||
|
||||
func (h *Handler) DashboardOverview(c *gin.Context) {
|
||||
since := time.Now().Add(-24 * time.Hour)
|
||||
var totalRequests int64
|
||||
var totalTokens int64
|
||||
var errorCount int64
|
||||
h.db.DB().Model(&model.UsageRecord{}).Where("created_at >= ?", since).Count(&totalRequests)
|
||||
h.db.DB().Model(&model.UsageRecord{}).Where("created_at >= ?", since).Select("COALESCE(SUM(tokens_in + tokens_out), 0)").Scan(&totalTokens)
|
||||
h.db.DB().Model(&model.UsageRecord{}).Where("created_at >= ? AND status = ?", since, "error").Count(&errorCount)
|
||||
activeKeys := len(h.cache.ListIngressKeys())
|
||||
errorRate := 0.0
|
||||
if totalRequests > 0 {
|
||||
errorRate = float64(errorCount) / float64(totalRequests)
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"total_requests_24h": totalRequests,
|
||||
"total_tokens_24h": totalTokens,
|
||||
"error_rate": errorRate,
|
||||
"active_ingress_keys": activeKeys,
|
||||
"providers": len(h.cache.ListProviders()),
|
||||
})
|
||||
}
|
||||
|
||||
func (h *Handler) DashboardProviders(c *gin.Context) {
|
||||
providers := h.cache.ListProviders()
|
||||
health := h.cache.GetHealthStatus()
|
||||
type item struct {
|
||||
model.Provider
|
||||
Health model.HealthCheckRecord `json:"health"`
|
||||
}
|
||||
var out []item
|
||||
for _, p := range providers {
|
||||
out = append(out, item{Provider: p, Health: health[p.ID]})
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{"data": out})
|
||||
}
|
||||
|
||||
func (h *Handler) DashboardIngressUsage(c *gin.Context) {
|
||||
keys := h.cache.ListIngressKeys()
|
||||
c.JSON(http.StatusOK, gin.H{"data": keys})
|
||||
}
|
||||
|
||||
func (h *Handler) DashboardEgressUsage(c *gin.Context) {
|
||||
keys := h.cache.ListAllProviderKeys()
|
||||
c.JSON(http.StatusOK, gin.H{"data": keys})
|
||||
}
|
||||
|
||||
func (h *Handler) DashboardProviderModelUsage(c *gin.Context) {
|
||||
since := time.Now().Add(-24 * time.Hour)
|
||||
type row struct {
|
||||
ProviderID uint `json:"provider_id"`
|
||||
Model string `json:"model"`
|
||||
RequestCount int64 `json:"request_count"`
|
||||
TokenCount int64 `json:"token_count"`
|
||||
Cost float64 `json:"cost"`
|
||||
}
|
||||
var rows []row
|
||||
h.db.DB().Model(&model.UsageRecord{}).
|
||||
Select("provider_id, model, COUNT(*) as request_count, COALESCE(SUM(tokens_in + tokens_out), 0) as token_count, COALESCE(SUM(cost), 0) as cost").
|
||||
Where("created_at >= ? AND provider_id > 0 AND model != ''", since).
|
||||
Group("provider_id, model").
|
||||
Order("request_count desc").
|
||||
Limit(50).
|
||||
Scan(&rows)
|
||||
|
||||
type item struct {
|
||||
ProviderID uint `json:"provider_id"`
|
||||
ProviderName string `json:"provider_name"`
|
||||
Model string `json:"model"`
|
||||
RequestCount int64 `json:"request_count"`
|
||||
TokenCount int64 `json:"token_count"`
|
||||
Cost float64 `json:"cost"`
|
||||
}
|
||||
out := make([]item, 0, len(rows))
|
||||
for _, r := range rows {
|
||||
name := ""
|
||||
if p, ok := h.cache.GetProvider(r.ProviderID); ok {
|
||||
name = p.Name
|
||||
}
|
||||
out = append(out, item{
|
||||
ProviderID: r.ProviderID,
|
||||
ProviderName: name,
|
||||
Model: r.Model,
|
||||
RequestCount: r.RequestCount,
|
||||
TokenCount: r.TokenCount,
|
||||
Cost: r.Cost,
|
||||
})
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{"data": out})
|
||||
}
|
||||
|
||||
func (h *Handler) DashboardClientIPUsage(c *gin.Context) {
|
||||
since := time.Now().Add(-24 * time.Hour)
|
||||
type item struct {
|
||||
ClientIP string `json:"client_ip"`
|
||||
RequestCount int64 `json:"request_count"`
|
||||
TokenCount int64 `json:"token_count"`
|
||||
Cost float64 `json:"cost"`
|
||||
}
|
||||
var out []item
|
||||
h.db.DB().Model(&model.UsageRecord{}).
|
||||
Select("client_ip, COUNT(*) as request_count, COALESCE(SUM(tokens_in + tokens_out), 0) as token_count, COALESCE(SUM(cost), 0) as cost").
|
||||
Where("created_at >= ? AND client_ip != ''", since).
|
||||
Group("client_ip").
|
||||
Order("request_count desc").
|
||||
Limit(50).
|
||||
Scan(&out)
|
||||
c.JSON(http.StatusOK, gin.H{"data": out})
|
||||
}
|
||||
|
||||
func (h *Handler) DashboardRequestTrend(c *gin.Context) {
|
||||
since := time.Now().Add(-23 * time.Hour).Truncate(time.Hour)
|
||||
type aggRow struct {
|
||||
Hour string `gorm:"column:hour"`
|
||||
Requests int64 `gorm:"column:requests"`
|
||||
Errors int64 `gorm:"column:errors"`
|
||||
Tokens int64 `gorm:"column:tokens"`
|
||||
}
|
||||
var rows []aggRow
|
||||
h.db.DB().Model(&model.UsageRecord{}).
|
||||
Select(`strftime('%Y-%m-%d %H:00', created_at) as hour,
|
||||
COUNT(*) as requests,
|
||||
SUM(CASE WHEN status = 'error' THEN 1 ELSE 0 END) as errors,
|
||||
COALESCE(SUM(tokens_in + tokens_out), 0) as tokens`).
|
||||
Where("created_at >= ?", since).
|
||||
Group("hour").
|
||||
Order("hour").
|
||||
Scan(&rows)
|
||||
|
||||
byHour := make(map[string]aggRow, len(rows))
|
||||
for _, r := range rows {
|
||||
byHour[r.Hour] = r
|
||||
}
|
||||
|
||||
type bucket struct {
|
||||
Hour string `json:"hour"`
|
||||
Requests int64 `json:"requests"`
|
||||
Errors int64 `json:"errors"`
|
||||
Tokens int64 `json:"tokens"`
|
||||
}
|
||||
out := make([]bucket, 24)
|
||||
for i := 0; i < 24; i++ {
|
||||
t := since.Add(time.Duration(i) * time.Hour)
|
||||
key := t.Format("2006-01-02 15:00")
|
||||
if r, ok := byHour[key]; ok {
|
||||
out[i] = bucket{Hour: key, Requests: r.Requests, Errors: r.Errors, Tokens: r.Tokens}
|
||||
} else {
|
||||
out[i] = bucket{Hour: key}
|
||||
}
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{"data": out})
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
package admin
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/provider"
|
||||
"github.com/rose_cat707/luminary/internal/store/cache"
|
||||
)
|
||||
|
||||
func (h *Handler) ListEndpointMeta(c *gin.Context) {
|
||||
category := model.ProviderCategory(c.Query("category"))
|
||||
if category == "" {
|
||||
category = model.CategoryLLM
|
||||
}
|
||||
providerType := model.ProviderType(c.Query("type"))
|
||||
if providerType == "" {
|
||||
providerType = provider.DefaultProviderType(category)
|
||||
}
|
||||
endpoints := provider.EndpointsForCategory(category)
|
||||
providerTypes := provider.TypesForCategory(category)
|
||||
type item struct {
|
||||
provider.EndpointMeta
|
||||
DefaultPath string `json:"default_path"`
|
||||
}
|
||||
out := make([]item, 0, len(endpoints))
|
||||
for _, e := range endpoints {
|
||||
out = append(out, item{
|
||||
EndpointMeta: e,
|
||||
DefaultPath: provider.DefaultPath(providerType, e.Key),
|
||||
})
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"categories": []gin.H{
|
||||
{"key": model.CategoryLLM, "label": provider.CategoryLabel(model.CategoryLLM), "supported": true},
|
||||
{"key": model.CategoryEmbedding, "label": provider.CategoryLabel(model.CategoryEmbedding), "supported": true},
|
||||
{"key": model.CategoryRerank, "label": provider.CategoryLabel(model.CategoryRerank), "supported": true},
|
||||
{"key": model.CategorySpeech, "label": provider.CategoryLabel(model.CategorySpeech), "supported": false},
|
||||
{"key": model.CategoryImage, "label": provider.CategoryLabel(model.CategoryImage), "supported": true},
|
||||
{"key": model.CategoryAudio, "label": provider.CategoryLabel(model.CategoryAudio), "supported": false},
|
||||
},
|
||||
"provider_types": providerTypes,
|
||||
"all_provider_types": provider.AllProviderTypes(),
|
||||
"endpoints": out,
|
||||
})
|
||||
}
|
||||
|
||||
func (h *Handler) ListProviders(c *gin.Context) {
|
||||
category := c.Query("category")
|
||||
providers := h.cache.ListProviders()
|
||||
if category != "" {
|
||||
filtered := providers[:0]
|
||||
for _, p := range providers {
|
||||
if string(p.Category) == category {
|
||||
filtered = append(filtered, p)
|
||||
}
|
||||
}
|
||||
providers = filtered
|
||||
}
|
||||
out := make([]model.Provider, 0, len(providers))
|
||||
for _, p := range providers {
|
||||
p.Models = h.cache.GetModels(p.ID)
|
||||
out = append(out, p)
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{"data": out})
|
||||
}
|
||||
|
||||
func (h *Handler) CreateProvider(c *gin.Context) {
|
||||
var req struct {
|
||||
Name string `json:"name"`
|
||||
Type model.ProviderType `json:"type"`
|
||||
Category model.ProviderCategory `json:"category"`
|
||||
BaseURL string `json:"base_url"`
|
||||
AllowedEndpoints []string `json:"allowed_endpoints"`
|
||||
EndpointPaths map[string]string `json:"endpoint_paths"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil || req.Name == "" || req.Type == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
if req.Category == "" {
|
||||
req.Category = model.CategoryLLM
|
||||
}
|
||||
if !model.IsCategorySupported(req.Category) {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "category not supported"})
|
||||
return
|
||||
}
|
||||
if !provider.IsTypeValidForCategory(req.Category, req.Type) {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "protocol type not supported for this category"})
|
||||
return
|
||||
}
|
||||
if h.providerNameTaken(req.Name, 0) {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "provider name already exists"})
|
||||
return
|
||||
}
|
||||
allowed := req.AllowedEndpoints
|
||||
if len(allowed) == 0 {
|
||||
allowed = provider.DefaultEndpointsForCategory(req.Category)
|
||||
}
|
||||
if err := provider.ValidateCategoryEndpoints(req.Category, allowed); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
pathsJSON, _ := json.Marshal(req.EndpointPaths)
|
||||
if req.EndpointPaths == nil {
|
||||
pathsJSON = []byte("{}")
|
||||
}
|
||||
p := model.Provider{
|
||||
Name: req.Name,
|
||||
Type: req.Type,
|
||||
Category: req.Category,
|
||||
BaseURL: req.BaseURL,
|
||||
AllowedEndpointsJSON: cache.ModelsToJSON(allowed),
|
||||
EndpointPathsJSON: string(pathsJSON),
|
||||
Status: model.ProviderUnknown,
|
||||
}
|
||||
if err := h.db.DB().Create(&p).Error; err != nil {
|
||||
if h.providerNameTaken(req.Name, 0) {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "provider name already exists"})
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
h.cache.SetProvider(p)
|
||||
c.JSON(http.StatusCreated, p)
|
||||
}
|
||||
|
||||
func (h *Handler) GetProvider(c *gin.Context) {
|
||||
id := parseUint(c.Param("id"))
|
||||
p, ok := h.cache.GetProvider(id)
|
||||
if !ok {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
p.Keys = h.cache.ListProviderKeys(id)
|
||||
p.Models = h.cache.GetModels(id)
|
||||
c.JSON(http.StatusOK, p)
|
||||
}
|
||||
|
||||
func (h *Handler) UpdateProvider(c *gin.Context) {
|
||||
id := parseUint(c.Param("id"))
|
||||
p, ok := h.cache.GetProvider(id)
|
||||
if !ok {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Name string `json:"name"`
|
||||
BaseURL string `json:"base_url"`
|
||||
Category model.ProviderCategory `json:"category"`
|
||||
AllowedEndpoints []string `json:"allowed_endpoints"`
|
||||
EndpointPaths map[string]string `json:"endpoint_paths"`
|
||||
Enabled *bool `json:"enabled"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
if req.Name != "" && req.Name != p.Name {
|
||||
if h.providerNameTaken(req.Name, p.ID) {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "provider name already exists"})
|
||||
return
|
||||
}
|
||||
p.Name = req.Name
|
||||
}
|
||||
if req.BaseURL != "" {
|
||||
p.BaseURL = req.BaseURL
|
||||
}
|
||||
if req.Category != "" {
|
||||
if !model.IsCategorySupported(req.Category) {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "category not supported yet"})
|
||||
return
|
||||
}
|
||||
p.Category = req.Category
|
||||
}
|
||||
if req.AllowedEndpoints != nil {
|
||||
if err := provider.ValidateCategoryEndpoints(p.Category, req.AllowedEndpoints); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
p.AllowedEndpointsJSON = cache.ModelsToJSON(req.AllowedEndpoints)
|
||||
}
|
||||
if req.EndpointPaths != nil {
|
||||
b, _ := json.Marshal(req.EndpointPaths)
|
||||
p.EndpointPathsJSON = string(b)
|
||||
}
|
||||
if req.Enabled != nil {
|
||||
p.Enabled = *req.Enabled
|
||||
}
|
||||
h.db.DB().Save(&p)
|
||||
h.cache.SetProvider(p)
|
||||
c.JSON(http.StatusOK, p)
|
||||
}
|
||||
@@ -0,0 +1,418 @@
|
||||
package admin
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"slices"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/rose_cat707/luminary/internal/auth"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/prediction"
|
||||
"github.com/rose_cat707/luminary/internal/provider"
|
||||
"github.com/rose_cat707/luminary/internal/store/cache"
|
||||
)
|
||||
|
||||
func (h *Handler) DeleteProvider(c *gin.Context) {
|
||||
id := parseUint(c.Param("id"))
|
||||
h.db.DB().Where("provider_id = ?", id).Delete(&model.ProviderKey{})
|
||||
h.db.DB().Where("provider_id = ?", id).Delete(&model.ModelEntry{})
|
||||
h.db.DB().Delete(&model.Provider{}, id)
|
||||
h.cache.DeleteProvider(id)
|
||||
c.JSON(http.StatusOK, gin.H{"ok": true})
|
||||
}
|
||||
|
||||
func (h *Handler) SyncModels(c *gin.Context) {
|
||||
id := parseUint(c.Param("id"))
|
||||
p, ok := h.cache.GetProvider(id)
|
||||
if !ok {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "provider not found"})
|
||||
return
|
||||
}
|
||||
allowed := provider.ParseAllowedEndpoints(p.AllowedEndpointsJSON, p.Category)
|
||||
if !slices.Contains(allowed, string(provider.EndpointListModels)) {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "list_models endpoint not enabled"})
|
||||
return
|
||||
}
|
||||
models, err := h.gateway.FetchModels(c.Request.Context(), id)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadGateway, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
existing := h.cache.GetModels(id)
|
||||
existingByModelID := make(map[string]model.ModelEntry, len(existing))
|
||||
for _, e := range existing {
|
||||
existingByModelID[e.ModelID] = e
|
||||
}
|
||||
upstreamIDs := make(map[string]bool, len(models))
|
||||
var entries []model.ModelEntry
|
||||
for _, m := range models {
|
||||
upstreamIDs[m.ID] = true
|
||||
if old, ok := existingByModelID[m.ID]; ok {
|
||||
old.DisplayName = m.DisplayName
|
||||
if old.DisplayName == "" {
|
||||
old.DisplayName = m.ID
|
||||
}
|
||||
if err := h.db.DB().Save(&old).Error; err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
entries = append(entries, old)
|
||||
continue
|
||||
}
|
||||
displayName := m.DisplayName
|
||||
if displayName == "" {
|
||||
displayName = m.ID
|
||||
}
|
||||
entry := model.ModelEntry{
|
||||
ProviderID: id,
|
||||
ModelID: m.ID,
|
||||
DisplayName: displayName,
|
||||
Enabled: true,
|
||||
}
|
||||
if err := h.db.DB().Create(&entry).Error; err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
entries = append(entries, entry)
|
||||
}
|
||||
for _, e := range existing {
|
||||
if !upstreamIDs[e.ModelID] {
|
||||
entries = append(entries, e)
|
||||
}
|
||||
}
|
||||
h.cache.SetModels(id, entries)
|
||||
c.JSON(http.StatusOK, gin.H{"synced": len(models), "models": entries})
|
||||
}
|
||||
|
||||
func (h *Handler) ListProviderKeys(c *gin.Context) {
|
||||
id := parseUint(c.Param("id"))
|
||||
c.JSON(http.StatusOK, gin.H{"data": h.cache.ListProviderKeys(id)})
|
||||
}
|
||||
|
||||
func (h *Handler) CreateProviderKey(c *gin.Context) {
|
||||
providerID := parseUint(c.Param("id"))
|
||||
var req struct {
|
||||
Name string `json:"name"`
|
||||
APIKey string `json:"api_key"`
|
||||
Weight float64 `json:"weight"`
|
||||
Models []string `json:"models"`
|
||||
Enabled *bool `json:"enabled"`
|
||||
MaxRequestsPerDay int `json:"max_requests_per_day"`
|
||||
MaxTokensPerDay int64 `json:"max_tokens_per_day"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil || req.Name == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
if req.APIKey == "" {
|
||||
var p model.Provider
|
||||
h.db.DB().First(&p, providerID)
|
||||
if p.Type != model.ProviderOllama {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "api_key required"})
|
||||
return
|
||||
}
|
||||
req.APIKey = "ollama"
|
||||
}
|
||||
enc, err := auth.Encrypt(req.APIKey, h.settings.EncryptionKey())
|
||||
if err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "encrypt failed"})
|
||||
return
|
||||
}
|
||||
modelsJSON := cache.ModelsToJSON(req.Models)
|
||||
if req.Models == nil {
|
||||
modelsJSON = `["*"]`
|
||||
}
|
||||
k := model.ProviderKey{
|
||||
ProviderID: providerID,
|
||||
Name: req.Name,
|
||||
APIKeyEnc: enc,
|
||||
Weight: req.Weight,
|
||||
ModelsJSON: modelsJSON,
|
||||
Enabled: true,
|
||||
MaxRequestsPerDay: req.MaxRequestsPerDay,
|
||||
MaxTokensPerDay: req.MaxTokensPerDay,
|
||||
}
|
||||
if req.Weight <= 0 {
|
||||
k.Weight = 1
|
||||
}
|
||||
if req.Enabled != nil {
|
||||
k.Enabled = *req.Enabled
|
||||
}
|
||||
if err := h.db.DB().Create(&k).Error; err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
h.cache.SetProviderKey(k)
|
||||
c.JSON(http.StatusCreated, k)
|
||||
}
|
||||
|
||||
func (h *Handler) UpdateProviderKey(c *gin.Context) {
|
||||
keyID := parseUint(c.Param("keyId"))
|
||||
k, ok := h.cache.GetProviderKey(keyID)
|
||||
if !ok {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Name string `json:"name"`
|
||||
APIKey string `json:"api_key"`
|
||||
Weight float64 `json:"weight"`
|
||||
Models []string `json:"models"`
|
||||
Enabled *bool `json:"enabled"`
|
||||
MaxRequestsPerDay int `json:"max_requests_per_day"`
|
||||
MaxTokensPerDay int64 `json:"max_tokens_per_day"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
if req.Name != "" {
|
||||
k.Name = req.Name
|
||||
}
|
||||
if req.APIKey != "" {
|
||||
enc, err := auth.Encrypt(req.APIKey, h.settings.EncryptionKey())
|
||||
if err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "encrypt failed"})
|
||||
return
|
||||
}
|
||||
k.APIKeyEnc = enc
|
||||
}
|
||||
if req.Weight > 0 {
|
||||
k.Weight = req.Weight
|
||||
}
|
||||
if req.Models != nil {
|
||||
k.ModelsJSON = cache.ModelsToJSON(req.Models)
|
||||
}
|
||||
if req.Enabled != nil {
|
||||
k.Enabled = *req.Enabled
|
||||
}
|
||||
if req.MaxRequestsPerDay >= 0 {
|
||||
k.MaxRequestsPerDay = req.MaxRequestsPerDay
|
||||
}
|
||||
if req.MaxTokensPerDay >= 0 {
|
||||
k.MaxTokensPerDay = req.MaxTokensPerDay
|
||||
}
|
||||
h.db.DB().Save(&k)
|
||||
h.cache.SetProviderKey(k)
|
||||
c.JSON(http.StatusOK, k)
|
||||
}
|
||||
|
||||
func (h *Handler) DeleteProviderKey(c *gin.Context) {
|
||||
keyID := parseUint(c.Param("keyId"))
|
||||
h.db.DB().Delete(&model.ProviderKey{}, keyID)
|
||||
h.cache.DeleteProviderKey(keyID)
|
||||
c.JSON(http.StatusOK, gin.H{"ok": true})
|
||||
}
|
||||
|
||||
func (h *Handler) ListModels(c *gin.Context) {
|
||||
id := parseUint(c.Param("id"))
|
||||
if _, ok := h.cache.GetProvider(id); !ok {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "provider not found"})
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{"data": h.cache.GetModels(id)})
|
||||
}
|
||||
|
||||
func (h *Handler) CreateModel(c *gin.Context) {
|
||||
providerID := parseUint(c.Param("id"))
|
||||
prov, ok := h.cache.GetProvider(providerID)
|
||||
if !ok {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "provider not found"})
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
ModelID string `json:"model_id"`
|
||||
Alias string `json:"alias"`
|
||||
DisplayName string `json:"display_name"`
|
||||
WorkflowType model.WorkflowType `json:"workflow_type"`
|
||||
WorkflowJSON string `json:"workflow_json"`
|
||||
InputBindingJSON string `json:"input_binding_json"`
|
||||
InputBinding map[string]*prediction.NodeField `json:"input_binding"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil || req.ModelID == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
for _, existing := range h.cache.GetModels(providerID) {
|
||||
if existing.ModelID == req.ModelID {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "model already exists"})
|
||||
return
|
||||
}
|
||||
}
|
||||
if msg := validateModelAlias(h, req.Alias, 0); msg != "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": msg})
|
||||
return
|
||||
}
|
||||
if prov.Category == model.CategoryImage {
|
||||
if err := validateImageModelCreate(struct {
|
||||
ModelID, DisplayName string
|
||||
WorkflowType model.WorkflowType
|
||||
WorkflowJSON, InputBindingJSON string
|
||||
InputBinding map[string]*prediction.NodeField
|
||||
}{req.ModelID, req.DisplayName, req.WorkflowType, req.WorkflowJSON, req.InputBindingJSON, req.InputBinding}); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
}
|
||||
bindingJSON := req.InputBindingJSON
|
||||
if bindingJSON == "" && req.InputBinding != nil {
|
||||
b, _ := json.Marshal(req.InputBinding)
|
||||
bindingJSON = string(b)
|
||||
}
|
||||
wt := req.WorkflowType
|
||||
if wt == "" {
|
||||
wt = model.WorkflowTxt2Img
|
||||
}
|
||||
m := model.ModelEntry{
|
||||
ProviderID: providerID,
|
||||
ModelID: req.ModelID,
|
||||
Alias: normalizeAlias(req.Alias),
|
||||
DisplayName: req.DisplayName,
|
||||
WorkflowType: wt,
|
||||
WorkflowJSON: req.WorkflowJSON,
|
||||
InputBindingJSON: bindingJSON,
|
||||
VersionHash: prediction.HashWorkflowJSON(req.WorkflowJSON),
|
||||
Enabled: true,
|
||||
}
|
||||
if m.DisplayName == "" {
|
||||
m.DisplayName = req.ModelID
|
||||
}
|
||||
if err := h.db.DB().Create(&m).Error; err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
models := h.cache.GetModels(providerID)
|
||||
models = append(models, m)
|
||||
h.cache.SetModels(providerID, models)
|
||||
c.JSON(http.StatusCreated, m)
|
||||
}
|
||||
|
||||
func (h *Handler) UpdateModel(c *gin.Context) {
|
||||
providerID := parseUint(c.Param("id"))
|
||||
modelID := parseUint(c.Param("modelId"))
|
||||
var req struct {
|
||||
Enabled *bool `json:"enabled"`
|
||||
DisplayName string `json:"display_name"`
|
||||
Alias *string `json:"alias"`
|
||||
ModelID string `json:"model_id"`
|
||||
WorkflowType model.WorkflowType `json:"workflow_type"`
|
||||
WorkflowJSON string `json:"workflow_json"`
|
||||
InputBindingJSON string `json:"input_binding_json"`
|
||||
InputBinding map[string]*prediction.NodeField `json:"input_binding"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
m, ok := h.findModelEntry(providerID, modelID, req.ModelID)
|
||||
if !ok {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "model not found"})
|
||||
return
|
||||
}
|
||||
if req.Enabled != nil {
|
||||
m.Enabled = *req.Enabled
|
||||
}
|
||||
if req.DisplayName != "" {
|
||||
m.DisplayName = req.DisplayName
|
||||
}
|
||||
if req.Alias != nil {
|
||||
normalized := normalizeAlias(*req.Alias)
|
||||
if msg := validateModelAlias(h, normalized, m.ID); msg != "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": msg})
|
||||
return
|
||||
}
|
||||
m.Alias = normalized
|
||||
}
|
||||
if req.ModelID != "" && req.ModelID != m.ModelID {
|
||||
for _, existing := range h.cache.GetModels(m.ProviderID) {
|
||||
if existing.ID != m.ID && existing.ModelID == req.ModelID {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "model already exists"})
|
||||
return
|
||||
}
|
||||
}
|
||||
m.ModelID = req.ModelID
|
||||
}
|
||||
if req.WorkflowType != "" {
|
||||
m.WorkflowType = req.WorkflowType
|
||||
}
|
||||
if req.WorkflowJSON != "" {
|
||||
m.WorkflowJSON = req.WorkflowJSON
|
||||
m.VersionHash = prediction.HashWorkflowJSON(req.WorkflowJSON)
|
||||
}
|
||||
if req.InputBindingJSON != "" {
|
||||
m.InputBindingJSON = req.InputBindingJSON
|
||||
} else if req.InputBinding != nil {
|
||||
b, _ := json.Marshal(req.InputBinding)
|
||||
m.InputBindingJSON = string(b)
|
||||
}
|
||||
prov, _ := h.cache.GetProvider(m.ProviderID)
|
||||
if prov.Category == model.CategoryImage && m.WorkflowJSON != "" {
|
||||
if err := validateImageModelCreate(struct {
|
||||
ModelID, DisplayName string
|
||||
WorkflowType model.WorkflowType
|
||||
WorkflowJSON, InputBindingJSON string
|
||||
InputBinding map[string]*prediction.NodeField
|
||||
}{m.ModelID, m.DisplayName, m.WorkflowType, m.WorkflowJSON, m.InputBindingJSON, nil}); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
}
|
||||
if err := h.db.DB().Save(&m).Error; err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
models := h.cache.GetModels(m.ProviderID)
|
||||
for i := range models {
|
||||
if models[i].ID == m.ID {
|
||||
models[i] = m
|
||||
break
|
||||
}
|
||||
}
|
||||
h.cache.SetModels(m.ProviderID, models)
|
||||
c.JSON(http.StatusOK, m)
|
||||
}
|
||||
|
||||
func (h *Handler) findModelEntry(providerID, entryID uint, modelID string) (model.ModelEntry, bool) {
|
||||
var m model.ModelEntry
|
||||
if entryID > 0 {
|
||||
if err := h.db.DB().First(&m, entryID).Error; err == nil {
|
||||
if providerID > 0 && m.ProviderID != providerID {
|
||||
return model.ModelEntry{}, false
|
||||
}
|
||||
return m, true
|
||||
}
|
||||
}
|
||||
if providerID > 0 && modelID != "" {
|
||||
if err := h.db.DB().Where("provider_id = ? AND model_id = ?", providerID, modelID).First(&m).Error; err == nil {
|
||||
return m, true
|
||||
}
|
||||
}
|
||||
return model.ModelEntry{}, false
|
||||
}
|
||||
|
||||
func (h *Handler) DeleteModel(c *gin.Context) {
|
||||
modelID := parseUint(c.Param("modelId"))
|
||||
var m model.ModelEntry
|
||||
if err := h.db.DB().First(&m, modelID).Error; err != nil {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
if providerID := parseUint(c.Param("id")); providerID > 0 && m.ProviderID != providerID {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
if err := h.db.DB().Delete(&m).Error; err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
models := h.cache.GetModels(m.ProviderID)
|
||||
filtered := models[:0]
|
||||
for _, item := range models {
|
||||
if item.ID != modelID {
|
||||
filtered = append(filtered, item)
|
||||
}
|
||||
}
|
||||
h.cache.SetModels(m.ProviderID, filtered)
|
||||
c.JSON(http.StatusOK, gin.H{"ok": true})
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
package admin
|
||||
|
||||
import (
|
||||
"strings"
|
||||
)
|
||||
|
||||
func (h *Handler) providerNameTaken(name string, excludeID uint) bool {
|
||||
name = strings.TrimSpace(name)
|
||||
if name == "" {
|
||||
return false
|
||||
}
|
||||
for _, p := range h.cache.ListProviders() {
|
||||
if p.ID != excludeID && p.Name == name {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (h *Handler) modelAliasTaken(alias string, excludeModelID uint) bool {
|
||||
alias = strings.TrimSpace(alias)
|
||||
if alias == "" {
|
||||
return false
|
||||
}
|
||||
for _, p := range h.cache.ListProviders() {
|
||||
for _, m := range h.cache.GetModels(p.ID) {
|
||||
if m.ID == excludeModelID {
|
||||
continue
|
||||
}
|
||||
if m.Alias == alias {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func normalizeAlias(alias string) string {
|
||||
return strings.TrimSpace(alias)
|
||||
}
|
||||
|
||||
func validateModelAlias(h *Handler, alias string, excludeModelID uint) string {
|
||||
alias = normalizeAlias(alias)
|
||||
if alias == "" {
|
||||
return ""
|
||||
}
|
||||
if h.modelAliasTaken(alias, excludeModelID) {
|
||||
return "model alias already exists"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -0,0 +1,220 @@
|
||||
package admin
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/pricing"
|
||||
)
|
||||
|
||||
func (h *Handler) ListRoutingRules(c *gin.Context) {
|
||||
ingressID := parseUint(c.Param("id"))
|
||||
c.JSON(http.StatusOK, gin.H{"data": h.cache.ListRoutingRules(ingressID)})
|
||||
}
|
||||
|
||||
func (h *Handler) CreateRoutingRule(c *gin.Context) {
|
||||
ingressID := parseUint(c.Param("id"))
|
||||
var req struct {
|
||||
ModelPattern string `json:"model_pattern"`
|
||||
ProviderID uint `json:"provider_id"`
|
||||
ProviderKeyID uint `json:"provider_key_id"`
|
||||
Priority int `json:"priority"`
|
||||
Enabled *bool `json:"enabled"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil || req.ModelPattern == "" || req.ProviderID == 0 {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
rule := model.RoutingRule{
|
||||
IngressKeyID: ingressID,
|
||||
ModelPattern: req.ModelPattern,
|
||||
ProviderID: req.ProviderID,
|
||||
ProviderKeyID: req.ProviderKeyID,
|
||||
Priority: req.Priority,
|
||||
Enabled: true,
|
||||
}
|
||||
if req.Enabled != nil {
|
||||
rule.Enabled = *req.Enabled
|
||||
}
|
||||
if err := h.db.DB().Create(&rule).Error; err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
rules := h.cache.ListRoutingRules(ingressID)
|
||||
rules = append(rules, rule)
|
||||
h.cache.SetRoutingRules(ingressID, rules)
|
||||
c.JSON(http.StatusCreated, rule)
|
||||
}
|
||||
|
||||
func (h *Handler) UpdateRoutingRule(c *gin.Context) {
|
||||
ruleID := parseUint(c.Param("ruleId"))
|
||||
var rule model.RoutingRule
|
||||
if err := h.db.DB().First(&rule, ruleID).Error; err != nil {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
ModelPattern string `json:"model_pattern"`
|
||||
ProviderID uint `json:"provider_id"`
|
||||
ProviderKeyID uint `json:"provider_key_id"`
|
||||
Priority int `json:"priority"`
|
||||
Enabled *bool `json:"enabled"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
if req.ModelPattern != "" {
|
||||
rule.ModelPattern = req.ModelPattern
|
||||
}
|
||||
if req.ProviderID > 0 {
|
||||
rule.ProviderID = req.ProviderID
|
||||
}
|
||||
if req.ProviderKeyID >= 0 {
|
||||
rule.ProviderKeyID = req.ProviderKeyID
|
||||
}
|
||||
rule.Priority = req.Priority
|
||||
if req.Enabled != nil {
|
||||
rule.Enabled = *req.Enabled
|
||||
}
|
||||
h.db.DB().Save(&rule)
|
||||
h.reloadRoutingRules(rule.IngressKeyID)
|
||||
c.JSON(http.StatusOK, rule)
|
||||
}
|
||||
|
||||
func (h *Handler) DeleteRoutingRule(c *gin.Context) {
|
||||
ruleID := parseUint(c.Param("ruleId"))
|
||||
var rule model.RoutingRule
|
||||
if err := h.db.DB().First(&rule, ruleID).Error; err != nil {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
h.db.DB().Delete(&rule)
|
||||
h.reloadRoutingRules(rule.IngressKeyID)
|
||||
c.JSON(http.StatusOK, gin.H{"ok": true})
|
||||
}
|
||||
|
||||
func (h *Handler) reloadRoutingRules(ingressID uint) {
|
||||
var rules []model.RoutingRule
|
||||
h.db.DB().Where("ingress_key_id = ?", ingressID).Find(&rules)
|
||||
h.cache.SetRoutingRules(ingressID, rules)
|
||||
}
|
||||
|
||||
func (h *Handler) ListRequestLogs(c *gin.Context) {
|
||||
page, _ := strconv.Atoi(c.DefaultQuery("page", "1"))
|
||||
pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "20"))
|
||||
if page < 1 {
|
||||
page = 1
|
||||
}
|
||||
if pageSize <= 0 || pageSize > 50 {
|
||||
pageSize = 20
|
||||
}
|
||||
|
||||
q := h.db.DB().Model(&model.UsageRecord{})
|
||||
if ingressID := c.Query("ingress_key_id"); ingressID != "" {
|
||||
q = q.Where("ingress_key_id = ?", ingressID)
|
||||
}
|
||||
if status := c.Query("status"); status != "" {
|
||||
q = q.Where("status = ?", status)
|
||||
}
|
||||
if endpoint := c.Query("endpoint"); endpoint != "" {
|
||||
q = q.Where("endpoint = ?", endpoint)
|
||||
}
|
||||
if clientIP := c.Query("client_ip"); clientIP != "" {
|
||||
q = q.Where("client_ip LIKE ?", "%"+clientIP+"%")
|
||||
}
|
||||
if modelName := c.Query("model"); modelName != "" {
|
||||
q = q.Where("model LIKE ?", "%"+modelName+"%")
|
||||
}
|
||||
|
||||
var total int64
|
||||
q.Count(&total)
|
||||
|
||||
var logs []model.UsageRecord
|
||||
q.Order("created_at desc").
|
||||
Offset((page - 1) * pageSize).
|
||||
Limit(pageSize).
|
||||
Find(&logs)
|
||||
|
||||
type item struct {
|
||||
ID uint `json:"id"`
|
||||
IngressKeyID uint `json:"ingress_key_id"`
|
||||
IngressName string `json:"ingress_name"`
|
||||
ClientIP string `json:"client_ip"`
|
||||
Endpoint string `json:"endpoint"`
|
||||
Model string `json:"model"`
|
||||
LatencyMs int64 `json:"latency_ms"`
|
||||
Status string `json:"status"`
|
||||
Stream bool `json:"stream"`
|
||||
ErrorMsg string `json:"error_msg"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
out := make([]item, 0, len(logs))
|
||||
for _, log := range logs {
|
||||
name := ""
|
||||
if log.Endpoint == model.EndpointAdminLogin {
|
||||
name = "管理台"
|
||||
} else if k, ok := h.cache.GetIngressKey(log.IngressKeyID); ok {
|
||||
name = k.Name
|
||||
}
|
||||
out = append(out, item{
|
||||
ID: log.ID,
|
||||
IngressKeyID: log.IngressKeyID,
|
||||
IngressName: name,
|
||||
ClientIP: log.ClientIP,
|
||||
Endpoint: log.Endpoint,
|
||||
Model: log.Model,
|
||||
LatencyMs: log.LatencyMs,
|
||||
Status: log.Status,
|
||||
Stream: log.Stream,
|
||||
ErrorMsg: log.ErrorMsg,
|
||||
CreatedAt: log.CreatedAt,
|
||||
})
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"data": out,
|
||||
"total": total,
|
||||
"page": page,
|
||||
"page_size": pageSize,
|
||||
"max_kept": model.MaxUsageLogRecords,
|
||||
})
|
||||
}
|
||||
|
||||
func (h *Handler) GetRequestLog(c *gin.Context) {
|
||||
id := parseUint(c.Param("id"))
|
||||
var log model.UsageRecord
|
||||
if err := h.db.DB().First(&log, id).Error; err != nil {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
name := ""
|
||||
if log.Endpoint == model.EndpointAdminLogin {
|
||||
name = "管理台"
|
||||
} else if k, ok := h.cache.GetIngressKey(log.IngressKeyID); ok {
|
||||
name = k.Name
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"data": gin.H{
|
||||
"id": log.ID,
|
||||
"ingress_key_id": log.IngressKeyID,
|
||||
"ingress_name": name,
|
||||
"client_ip": log.ClientIP,
|
||||
"endpoint": log.Endpoint,
|
||||
"model": log.Model,
|
||||
"latency_ms": log.LatencyMs,
|
||||
"status": log.Status,
|
||||
"stream": log.Stream,
|
||||
"error_msg": log.ErrorMsg,
|
||||
"request_body": log.RequestBody,
|
||||
"response_body": log.ResponseBody,
|
||||
"created_at": log.CreatedAt,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func (h *Handler) PricingCatalog(c *gin.Context) {
|
||||
c.JSON(http.StatusOK, gin.H{"data": pricing.ListCatalog()})
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
package admin
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/rose_cat707/luminary/internal/settings"
|
||||
)
|
||||
|
||||
func (h *Handler) GetSettings(c *gin.Context) {
|
||||
c.JSON(http.StatusOK, h.settings.PublicView())
|
||||
}
|
||||
|
||||
func (h *Handler) UpdateSettings(c *gin.Context) {
|
||||
var in settings.UpdateInput
|
||||
if err := c.ShouldBindJSON(&in); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid request"})
|
||||
return
|
||||
}
|
||||
if err := h.settings.Update(in); err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
h.loginLimit.Configure(h.settings.LoginMaxAttempts(), h.settings.LoginLockout())
|
||||
if h.monitor != nil {
|
||||
h.monitor.NotifySettingsChanged()
|
||||
}
|
||||
c.JSON(http.StatusOK, h.settings.PublicView())
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
package admin
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/prediction"
|
||||
)
|
||||
|
||||
func (h *Handler) GetWorkflowParamMeta(c *gin.Context) {
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"txt2img": prediction.Txt2ImgParams,
|
||||
"img2img": prediction.Img2ImgParams,
|
||||
})
|
||||
}
|
||||
|
||||
func (h *Handler) AnalyzeWorkflow(c *gin.Context) {
|
||||
var req struct {
|
||||
WorkflowJSON string `json:"workflow_json"`
|
||||
WorkflowType model.WorkflowType `json:"workflow_type"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil || req.WorkflowJSON == "" {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "workflow_json required"})
|
||||
return
|
||||
}
|
||||
if req.WorkflowType == "" {
|
||||
req.WorkflowType = model.WorkflowTxt2Img
|
||||
}
|
||||
result, err := prediction.AnalyzeWorkflow(req.WorkflowJSON, req.WorkflowType)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, result)
|
||||
}
|
||||
|
||||
func (h *Handler) GetPrediction(c *gin.Context) {
|
||||
if h.predictions == nil {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
p, err := h.predictions.GetDBPrediction(c.Param("id"))
|
||||
if err != nil {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
images, _ := h.images.ListByPrediction(p.ID)
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"prediction": h.predictions.APIView(p, h.publicBaseURL()),
|
||||
"images": images,
|
||||
})
|
||||
}
|
||||
|
||||
func (h *Handler) ServeAdminImage(c *gin.Context) {
|
||||
if h.images == nil {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
img, err := h.images.Get(c.Param("id"))
|
||||
if err != nil {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
c.Header("Content-Type", img.Mime)
|
||||
c.File(img.LocalPath)
|
||||
}
|
||||
|
||||
func bindingFromSuggestions(suggestions []prediction.BindingSuggestion, overrides map[string]*prediction.NodeField) prediction.InputBinding {
|
||||
binding := prediction.InputBinding{}
|
||||
for _, s := range suggestions {
|
||||
if o, ok := overrides[s.Param]; ok && o != nil {
|
||||
if o.Node != "" && o.Field != "" {
|
||||
binding[s.Param] = o
|
||||
}
|
||||
continue
|
||||
}
|
||||
if s.Node != "" && s.Field != "" {
|
||||
binding[s.Param] = &prediction.NodeField{Node: s.Node, Field: s.Field}
|
||||
}
|
||||
}
|
||||
return binding
|
||||
}
|
||||
|
||||
func validateImageModelCreate(req struct {
|
||||
ModelID string
|
||||
DisplayName string
|
||||
WorkflowType model.WorkflowType
|
||||
WorkflowJSON string
|
||||
InputBindingJSON string
|
||||
InputBinding map[string]*prediction.NodeField
|
||||
}) error {
|
||||
if req.WorkflowJSON == "" {
|
||||
return nil
|
||||
}
|
||||
var workflow map[string]any
|
||||
if err := json.Unmarshal([]byte(req.WorkflowJSON), &workflow); err != nil {
|
||||
return err
|
||||
}
|
||||
var binding prediction.InputBinding
|
||||
if req.InputBindingJSON != "" {
|
||||
b, err := prediction.ParseBinding(req.InputBindingJSON)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
binding = b
|
||||
} else if req.InputBinding != nil {
|
||||
binding = req.InputBinding
|
||||
}
|
||||
wt := req.WorkflowType
|
||||
if wt == "" {
|
||||
wt = model.WorkflowTxt2Img
|
||||
}
|
||||
return binding.Validate(workflow, prediction.RequiredBindings(wt))
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
package files
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strconv"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/rose_cat707/luminary/internal/storage/imagestore"
|
||||
)
|
||||
|
||||
type Handler struct {
|
||||
store *imagestore.Store
|
||||
}
|
||||
|
||||
func New(store *imagestore.Store) *Handler {
|
||||
return &Handler{store: store}
|
||||
}
|
||||
|
||||
func (h *Handler) Register(r *gin.Engine) {
|
||||
r.GET("/files/:id", h.Serve)
|
||||
}
|
||||
|
||||
func (h *Handler) Serve(c *gin.Context) {
|
||||
id := c.Param("id")
|
||||
expStr := c.Query("exp")
|
||||
sig := c.Query("sig")
|
||||
exp, err := strconv.ParseInt(expStr, 10, 64)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid exp"})
|
||||
return
|
||||
}
|
||||
if !h.store.Verify(id, exp, sig) {
|
||||
c.JSON(http.StatusForbidden, gin.H{"error": "invalid signature"})
|
||||
return
|
||||
}
|
||||
img, err := h.store.Get(id)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
c.Header("Content-Type", img.Mime)
|
||||
c.File(img.LocalPath)
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
package proxy
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/gateway"
|
||||
)
|
||||
|
||||
func gatewayHTTPStatus(err error) int {
|
||||
switch {
|
||||
case errors.Is(err, gateway.ErrRateLimitExceeded):
|
||||
return http.StatusTooManyRequests
|
||||
case errors.Is(err, gateway.ErrModelNotAllowed),
|
||||
errors.Is(err, gateway.ErrBudgetExceeded):
|
||||
return http.StatusForbidden
|
||||
case errors.Is(err, gateway.ErrInvalidJSON),
|
||||
errors.Is(err, gateway.ErrModelRequired):
|
||||
return http.StatusBadRequest
|
||||
}
|
||||
msg := err.Error()
|
||||
if strings.Contains(msg, "not enabled") || strings.Contains(msg, "not allowed") {
|
||||
return http.StatusForbidden
|
||||
}
|
||||
if strings.Contains(msg, "invalid api key") || strings.Contains(msg, "expired") {
|
||||
return http.StatusUnauthorized
|
||||
}
|
||||
return http.StatusBadGateway
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
package proxy
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"io"
|
||||
"net/http"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/rose_cat707/luminary/internal/gateway"
|
||||
"github.com/rose_cat707/luminary/internal/middleware"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/provider"
|
||||
)
|
||||
|
||||
type Handler struct {
|
||||
gateway *gateway.Service
|
||||
}
|
||||
|
||||
func New(gw *gateway.Service) *Handler {
|
||||
return &Handler{gateway: gw}
|
||||
}
|
||||
|
||||
func (h *Handler) Register(r *gin.RouterGroup) {
|
||||
r.GET("/models", h.ListModels)
|
||||
r.POST("/chat/completions", h.forward(provider.EndpointChatCompletion))
|
||||
r.POST("/completions", h.forward(provider.EndpointTextCompletion))
|
||||
r.POST("/responses", h.forward(provider.EndpointResponses))
|
||||
r.POST("/embeddings", h.forward(provider.EndpointEmbeddings))
|
||||
r.POST("/rerank", h.forward(provider.EndpointRerank))
|
||||
}
|
||||
|
||||
func (h *Handler) ingressKey(c *gin.Context) (*model.IngressKey, bool) {
|
||||
v, ok := c.Get("ingress_key")
|
||||
if !ok {
|
||||
return nil, false
|
||||
}
|
||||
k, ok := v.(*model.IngressKey)
|
||||
return k, ok
|
||||
}
|
||||
|
||||
func (h *Handler) ListModels(c *gin.Context) {
|
||||
ingress, ok := h.ingressKey(c)
|
||||
if !ok {
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"error": "unauthorized"})
|
||||
return
|
||||
}
|
||||
models, err := h.gateway.ListModels(c.Request.Context(), ingress)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{"object": "list", "data": models})
|
||||
}
|
||||
|
||||
func (h *Handler) forward(endpoint provider.EndpointType) gin.HandlerFunc {
|
||||
return func(c *gin.Context) {
|
||||
ingress, ok := h.ingressKey(c)
|
||||
if !ok {
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"error": "unauthorized"})
|
||||
return
|
||||
}
|
||||
body, err := io.ReadAll(c.Request.Body)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid body"})
|
||||
return
|
||||
}
|
||||
if provider.IsStreamRequest(body) && provider.SupportsStreaming(endpoint) {
|
||||
h.handleStream(c, ingress, endpoint, body)
|
||||
return
|
||||
}
|
||||
clientIP := middleware.ClientIP(c)
|
||||
resp, err := h.gateway.ForwardEndpoint(c.Request.Context(), ingress, endpoint, body, clientIP)
|
||||
if err != nil {
|
||||
c.JSON(gatewayHTTPStatus(err), gin.H{"error": gin.H{"message": err.Error(), "type": "gateway_error"}})
|
||||
return
|
||||
}
|
||||
c.Data(resp.StatusCode, "application/json", resp.Body)
|
||||
}
|
||||
}
|
||||
|
||||
func (h *Handler) handleStream(c *gin.Context, ingress *model.IngressKey, endpoint provider.EndpointType, body []byte) {
|
||||
c.Header("Content-Type", "text/event-stream")
|
||||
c.Header("Cache-Control", "no-cache")
|
||||
c.Header("Connection", "keep-alive")
|
||||
c.Status(http.StatusOK)
|
||||
|
||||
flusher, ok := c.Writer.(http.Flusher)
|
||||
if !ok {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"error": "streaming not supported"})
|
||||
return
|
||||
}
|
||||
w := bufio.NewWriter(c.Writer)
|
||||
clientIP := middleware.ClientIP(c)
|
||||
err := h.gateway.ForwardStream(c.Request.Context(), ingress, endpoint, body, clientIP, w, flusher)
|
||||
w.Flush()
|
||||
if err != nil {
|
||||
_, _ = c.Writer.Write([]byte("data: {\"error\":\"" + err.Error() + "\"}\n\n"))
|
||||
flusher.Flush()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,159 @@
|
||||
package replicate
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/prediction"
|
||||
)
|
||||
|
||||
type Handler struct {
|
||||
svc *prediction.Service
|
||||
baseURL func() string
|
||||
}
|
||||
|
||||
func New(svc *prediction.Service, baseURL func() string) *Handler {
|
||||
return &Handler{svc: svc, baseURL: baseURL}
|
||||
}
|
||||
|
||||
func (h *Handler) Register(r *gin.RouterGroup) {
|
||||
r.POST("/predictions", h.Create)
|
||||
r.GET("/predictions", h.List)
|
||||
r.GET("/predictions/:id", h.Get)
|
||||
r.POST("/predictions/:id/cancel", h.Cancel)
|
||||
r.GET("/predictions/:id/stream", h.Stream)
|
||||
}
|
||||
|
||||
func (h *Handler) ingress(c *gin.Context) (*model.IngressKey, bool) {
|
||||
v, ok := c.Get("ingress_key")
|
||||
if !ok {
|
||||
return nil, false
|
||||
}
|
||||
k, ok := v.(*model.IngressKey)
|
||||
return k, ok
|
||||
}
|
||||
|
||||
func (h *Handler) Create(c *gin.Context) {
|
||||
ingress, ok := h.ingress(c)
|
||||
if !ok {
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"detail": "unauthorized"})
|
||||
return
|
||||
}
|
||||
var req prediction.CreateRequest
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"detail": "invalid request body"})
|
||||
return
|
||||
}
|
||||
wait := strings.Contains(c.GetHeader("Prefer"), "wait")
|
||||
p, err := h.svc.Create(c.Request.Context(), ingress, req, c.ClientIP(), wait)
|
||||
if err != nil {
|
||||
code := http.StatusBadRequest
|
||||
if strings.Contains(err.Error(), "not found") {
|
||||
code = http.StatusNotFound
|
||||
}
|
||||
c.JSON(code, gin.H{"detail": err.Error()})
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusCreated, h.svc.APIView(p, h.baseURL()))
|
||||
}
|
||||
|
||||
func (h *Handler) Get(c *gin.Context) {
|
||||
ingress, ok := h.ingress(c)
|
||||
if !ok {
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"detail": "unauthorized"})
|
||||
return
|
||||
}
|
||||
p, err := h.svc.Get(c.Request.Context(), c.Param("id"), ingress)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusNotFound, gin.H{"detail": err.Error()})
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, h.svc.APIView(p, h.baseURL()))
|
||||
}
|
||||
|
||||
func (h *Handler) List(c *gin.Context) {
|
||||
ingress, ok := h.ingress(c)
|
||||
if !ok {
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"detail": "unauthorized"})
|
||||
return
|
||||
}
|
||||
items, err := h.svc.List(c.Request.Context(), ingress, c.Query("cursor"), 100)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusInternalServerError, gin.H{"detail": err.Error()})
|
||||
return
|
||||
}
|
||||
results := make([]map[string]any, 0, len(items))
|
||||
for _, p := range items {
|
||||
results = append(results, h.svc.APIView(p, h.baseURL()))
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{"results": results})
|
||||
}
|
||||
|
||||
func (h *Handler) Cancel(c *gin.Context) {
|
||||
ingress, ok := h.ingress(c)
|
||||
if !ok {
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"detail": "unauthorized"})
|
||||
return
|
||||
}
|
||||
p, err := h.svc.Cancel(c.Request.Context(), c.Param("id"), ingress)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusNotFound, gin.H{"detail": err.Error()})
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, h.svc.APIView(p, h.baseURL()))
|
||||
}
|
||||
|
||||
func (h *Handler) Stream(c *gin.Context) {
|
||||
ingress, ok := h.ingress(c)
|
||||
if !ok {
|
||||
c.JSON(http.StatusUnauthorized, gin.H{"detail": "unauthorized"})
|
||||
return
|
||||
}
|
||||
id := c.Param("id")
|
||||
if _, err := h.svc.Get(c.Request.Context(), id, ingress); err != nil {
|
||||
c.JSON(http.StatusNotFound, gin.H{"detail": err.Error()})
|
||||
return
|
||||
}
|
||||
|
||||
c.Header("Content-Type", "text/event-stream")
|
||||
c.Header("Cache-Control", "no-cache")
|
||||
c.Header("Connection", "keep-alive")
|
||||
c.Status(http.StatusOK)
|
||||
|
||||
flusher, ok := c.Writer.(http.Flusher)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
w := bufio.NewWriter(c.Writer)
|
||||
ch := h.svc.Hub().Subscribe(id)
|
||||
defer h.svc.Hub().Unsubscribe(id, ch)
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-c.Request.Context().Done():
|
||||
return
|
||||
case ev, open := <-ch:
|
||||
if !open {
|
||||
return
|
||||
}
|
||||
payload, _ := json.Marshal(map[string]any{
|
||||
"id": id, "status": ev.Status, "logs": ev.Logs,
|
||||
"metrics": ev.Metrics, "output": ev.Output, "error": ev.Error,
|
||||
})
|
||||
_, _ = fmt.Fprintf(w, "data: %s\n\n", payload)
|
||||
w.Flush()
|
||||
flusher.Flush()
|
||||
if ev.Done {
|
||||
_, _ = fmt.Fprintf(w, "event: done\ndata: {}\n\n")
|
||||
w.Flush()
|
||||
flusher.Flush()
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
package static
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"io/fs"
|
||||
"mime"
|
||||
"net/http"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
type Handler struct {
|
||||
content embed.FS
|
||||
}
|
||||
|
||||
func New(content embed.FS) *Handler {
|
||||
return &Handler{content: content}
|
||||
}
|
||||
|
||||
func (h *Handler) Register(r *gin.Engine) {
|
||||
sub, err := fs.Sub(h.content, "dist")
|
||||
if err != nil {
|
||||
sub = h.content
|
||||
}
|
||||
r.NoRoute(func(c *gin.Context) {
|
||||
if strings.HasPrefix(c.Request.URL.Path, "/api/") || strings.HasPrefix(c.Request.URL.Path, "/v1/") {
|
||||
c.JSON(http.StatusNotFound, gin.H{"error": "not found"})
|
||||
return
|
||||
}
|
||||
h.serve(c, sub)
|
||||
})
|
||||
r.GET("/", func(c *gin.Context) { h.serve(c, sub) })
|
||||
r.GET("/assets/*filepath", func(c *gin.Context) { h.serve(c, sub) })
|
||||
}
|
||||
|
||||
func (h *Handler) serve(c *gin.Context, sub fs.FS) {
|
||||
path := strings.TrimPrefix(c.Request.URL.Path, "/")
|
||||
if path == "" {
|
||||
path = "index.html"
|
||||
}
|
||||
data, err := fs.ReadFile(sub, path)
|
||||
if err != nil {
|
||||
data, err = fs.ReadFile(sub, "index.html")
|
||||
if err != nil {
|
||||
c.String(http.StatusNotFound, "UI not built. Run: make build-web")
|
||||
return
|
||||
}
|
||||
path = "index.html"
|
||||
}
|
||||
ext := filepath.Ext(path)
|
||||
ct := mime.TypeByExtension(ext)
|
||||
if ct == "" {
|
||||
ct = "text/html"
|
||||
}
|
||||
if strings.HasPrefix(path, "assets/") {
|
||||
c.Header("Cache-Control", "public, max-age=31536000, immutable")
|
||||
} else {
|
||||
c.Header("Cache-Control", "no-cache")
|
||||
}
|
||||
c.Data(http.StatusOK, ct, data)
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
package limiter
|
||||
|
||||
import (
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/store/cache"
|
||||
)
|
||||
|
||||
func IsProviderKeyAvailable(k model.ProviderKey) bool {
|
||||
if !k.Enabled {
|
||||
return false
|
||||
}
|
||||
if k.MaxRequestsPerDay > 0 && k.RequestsToday >= k.MaxRequestsPerDay {
|
||||
return false
|
||||
}
|
||||
if k.MaxTokensPerDay > 0 && k.TokensToday >= k.MaxTokensPerDay {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func CheckIngressBudget(k *model.IngressKey, estimatedCost float64) bool {
|
||||
if k.BudgetLimit <= 0 {
|
||||
return true
|
||||
}
|
||||
return k.BudgetUsed+estimatedCost <= k.BudgetLimit
|
||||
}
|
||||
|
||||
func CheckRateLimit(c *cache.Store, ingressKeyID uint, rpm, tpm, tokens int) bool {
|
||||
return c.CheckRateLimit(ingressKeyID, rpm, tpm, tokens)
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"net"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
var (
|
||||
trustedMu sync.RWMutex
|
||||
trustedProxies []*net.IPNet
|
||||
trustedIPs []net.IP
|
||||
)
|
||||
|
||||
// ConfigureTrustedProxies sets CIDRs/IPs allowed to set X-Forwarded-For / X-Real-IP.
|
||||
// When empty, only the direct remote address is used.
|
||||
func ConfigureTrustedProxies(cidrs []string) {
|
||||
trustedMu.Lock()
|
||||
defer trustedMu.Unlock()
|
||||
trustedProxies = nil
|
||||
trustedIPs = nil
|
||||
for _, cidr := range cidrs {
|
||||
cidr = strings.TrimSpace(cidr)
|
||||
if cidr == "" {
|
||||
continue
|
||||
}
|
||||
if strings.Contains(cidr, "/") {
|
||||
_, network, err := net.ParseCIDR(cidr)
|
||||
if err == nil {
|
||||
trustedProxies = append(trustedProxies, network)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if ip := net.ParseIP(cidr); ip != nil {
|
||||
trustedIPs = append(trustedIPs, ip)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func isTrustedProxy(ip net.IP) bool {
|
||||
trustedMu.RLock()
|
||||
defer trustedMu.RUnlock()
|
||||
if len(trustedProxies) == 0 && len(trustedIPs) == 0 {
|
||||
return false
|
||||
}
|
||||
for _, tip := range trustedIPs {
|
||||
if tip.Equal(ip) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
for _, network := range trustedProxies {
|
||||
if network.Contains(ip) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func directRemoteIP(ctx *gin.Context) string {
|
||||
host, _, err := net.SplitHostPort(ctx.Request.RemoteAddr)
|
||||
if err != nil {
|
||||
return strings.TrimSpace(ctx.Request.RemoteAddr)
|
||||
}
|
||||
return host
|
||||
}
|
||||
|
||||
// ClientIP returns the client IP. Forwarded headers are honored only when the
|
||||
// direct remote address is a configured trusted proxy.
|
||||
func ClientIP(ctx *gin.Context) string {
|
||||
remote := directRemoteIP(ctx)
|
||||
remoteIP := net.ParseIP(remote)
|
||||
if remoteIP == nil || !isTrustedProxy(remoteIP) {
|
||||
if remoteIP != nil {
|
||||
return remoteIP.String()
|
||||
}
|
||||
return ctx.ClientIP()
|
||||
}
|
||||
if xff := ctx.GetHeader("X-Forwarded-For"); xff != "" {
|
||||
parts := strings.Split(xff, ",")
|
||||
return strings.TrimSpace(parts[0])
|
||||
}
|
||||
if xri := ctx.GetHeader("X-Real-IP"); xri != "" {
|
||||
return strings.TrimSpace(xri)
|
||||
}
|
||||
return remoteIP.String()
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
func TestClientIP_IgnoresXFFWithoutTrustedProxy(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
ConfigureTrustedProxies(nil)
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
c, _ := gin.CreateTestContext(w)
|
||||
c.Request = httptest.NewRequest("GET", "/", nil)
|
||||
c.Request.RemoteAddr = "203.0.113.10:12345"
|
||||
c.Request.Header.Set("X-Forwarded-For", "1.2.3.4")
|
||||
|
||||
if got := ClientIP(c); got != "203.0.113.10" {
|
||||
t.Fatalf("got %s want direct remote", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClientIP_UsesXFFFromTrustedProxy(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
ConfigureTrustedProxies([]string{"127.0.0.1"})
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
c, _ := gin.CreateTestContext(w)
|
||||
c.Request = httptest.NewRequest("GET", "/", nil)
|
||||
c.Request.RemoteAddr = "127.0.0.1:12345"
|
||||
c.Request.Header.Set("X-Forwarded-For", "1.2.3.4")
|
||||
|
||||
if got := ClientIP(c); got != "1.2.3.4" {
|
||||
t.Fatalf("got %s want forwarded client", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"net"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestIPMatchesRule_LoopbackEquivalence(t *testing.T) {
|
||||
v4 := net.ParseIP("127.0.0.1")
|
||||
v6 := net.ParseIP("::1")
|
||||
|
||||
if !ipMatchesRule("127.0.0.1", v6) {
|
||||
t.Fatal("::1 should match 127.0.0.1 allow rule")
|
||||
}
|
||||
if !ipMatchesRule("::1", v4) {
|
||||
t.Fatal("127.0.0.1 should match ::1 allow rule")
|
||||
}
|
||||
if !ipMatchesRule("127.0.0.0/8", v6) {
|
||||
t.Fatal("::1 should match 127.0.0.0/8 via loopback equivalence")
|
||||
}
|
||||
}
|
||||
|
||||
func TestIPMatchesRule_CIDR(t *testing.T) {
|
||||
ip := net.ParseIP("10.0.0.5")
|
||||
if !ipMatchesRule("10.0.0.0/24", ip) {
|
||||
t.Fatal("expected CIDR match")
|
||||
}
|
||||
if ipMatchesRule("192.168.1.0/24", ip) {
|
||||
t.Fatal("expected no match")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,166 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/store/cache"
|
||||
)
|
||||
|
||||
func IPFilter(c *cache.Store, scope model.IPRuleScope) gin.HandlerFunc {
|
||||
return func(ctx *gin.Context) {
|
||||
rules := c.ListIPRules()
|
||||
if len(rules) == 0 {
|
||||
ctx.Next()
|
||||
return
|
||||
}
|
||||
clientIP := clientIP(ctx)
|
||||
ip := net.ParseIP(clientIP)
|
||||
if ip == nil {
|
||||
ctx.AbortWithStatusJSON(http.StatusForbidden, gin.H{"error": "invalid client ip"})
|
||||
return
|
||||
}
|
||||
var allowRules, denyRules []model.IPRule
|
||||
for _, r := range rules {
|
||||
if !r.Enabled {
|
||||
continue
|
||||
}
|
||||
if r.Scope != scope && r.Scope != model.IPScopeAll {
|
||||
continue
|
||||
}
|
||||
if r.Type == model.IPRuleAllow {
|
||||
allowRules = append(allowRules, r)
|
||||
} else {
|
||||
denyRules = append(denyRules, r)
|
||||
}
|
||||
}
|
||||
for _, r := range denyRules {
|
||||
if ipMatchesRule(r.CIDR, ip) {
|
||||
ctx.AbortWithStatusJSON(http.StatusForbidden, gin.H{"error": "ip denied"})
|
||||
return
|
||||
}
|
||||
}
|
||||
if len(allowRules) > 0 {
|
||||
allowed := false
|
||||
for _, r := range allowRules {
|
||||
if ipMatchesRule(r.CIDR, ip) {
|
||||
allowed = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !allowed {
|
||||
ctx.AbortWithStatusJSON(http.StatusForbidden, gin.H{"error": "ip not allowed"})
|
||||
return
|
||||
}
|
||||
}
|
||||
ctx.Next()
|
||||
}
|
||||
}
|
||||
|
||||
func cidrContains(cidr string, ip net.IP) bool {
|
||||
if !strings.Contains(cidr, "/") {
|
||||
return ip.String() == cidr
|
||||
}
|
||||
_, network, err := net.ParseCIDR(cidr)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
return network.Contains(ip)
|
||||
}
|
||||
|
||||
// ipMatchesRule checks CIDR/IP match, treating IPv4/IPv6 loopback as equivalent.
|
||||
func ipMatchesRule(cidr string, ip net.IP) bool {
|
||||
if cidrContains(cidr, ip) {
|
||||
return true
|
||||
}
|
||||
if !ip.IsLoopback() {
|
||||
return false
|
||||
}
|
||||
alt := loopbackAlt(ip)
|
||||
return alt != nil && cidrContains(cidr, alt)
|
||||
}
|
||||
|
||||
func loopbackAlt(ip net.IP) net.IP {
|
||||
if v4 := ip.To4(); v4 != nil && v4.IsLoopback() {
|
||||
return net.ParseIP("::1")
|
||||
}
|
||||
if ip.Equal(net.ParseIP("::1")) {
|
||||
return net.ParseIP("127.0.0.1")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func clientIP(ctx *gin.Context) string {
|
||||
return ClientIP(ctx)
|
||||
}
|
||||
|
||||
const sessionCookie = "luminary_session"
|
||||
|
||||
type SessionValidator func(token string) (bool, time.Time)
|
||||
|
||||
func AdminAuth(validate SessionValidator) gin.HandlerFunc {
|
||||
return func(ctx *gin.Context) {
|
||||
token := extractToken(ctx)
|
||||
if token == "" {
|
||||
ctx.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "unauthorized"})
|
||||
return
|
||||
}
|
||||
ok, expires := validate(token)
|
||||
if !ok || time.Now().After(expires) {
|
||||
ctx.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "session expired"})
|
||||
return
|
||||
}
|
||||
ctx.Set("session_token", token)
|
||||
ctx.Next()
|
||||
}
|
||||
}
|
||||
|
||||
func extractToken(ctx *gin.Context) string {
|
||||
if auth := ctx.GetHeader("Authorization"); strings.HasPrefix(auth, "Bearer ") {
|
||||
return strings.TrimPrefix(auth, "Bearer ")
|
||||
}
|
||||
if cookie, err := ctx.Cookie(sessionCookie); err == nil {
|
||||
return cookie
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func SessionCookieName() string { return sessionCookie }
|
||||
|
||||
func IngressAuth(gw IngressKeyResolver) gin.HandlerFunc {
|
||||
return func(ctx *gin.Context) {
|
||||
key := extractIngressKey(ctx)
|
||||
if key == "" {
|
||||
ctx.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "missing api key"})
|
||||
return
|
||||
}
|
||||
ingress, err := gw.ResolveIngressKey(key)
|
||||
if err != nil {
|
||||
ctx.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "invalid api key"})
|
||||
return
|
||||
}
|
||||
ctx.Set("ingress_key", ingress)
|
||||
ctx.Next()
|
||||
}
|
||||
}
|
||||
|
||||
func extractIngressKey(ctx *gin.Context) string {
|
||||
if auth := ctx.GetHeader("Authorization"); strings.HasPrefix(auth, "Bearer ") {
|
||||
return strings.TrimPrefix(auth, "Bearer ")
|
||||
}
|
||||
return ctx.GetHeader("x-api-key")
|
||||
}
|
||||
|
||||
type IngressKeyResolver interface {
|
||||
ResolveIngressKey(key string) (*model.IngressKey, error)
|
||||
}
|
||||
|
||||
func Logger() gin.HandlerFunc {
|
||||
return gin.LoggerWithFormatter(func(param gin.LogFormatterParams) string {
|
||||
return param.TimeStamp.Format(time.RFC3339) + " " + param.Method + " " + param.Path + " " + param.ClientIP + "\n"
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
package model
|
||||
|
||||
// ProviderCategory 出口服务分类
|
||||
type ProviderCategory string
|
||||
|
||||
const (
|
||||
CategoryLLM ProviderCategory = "llm" // 语言模型
|
||||
CategoryEmbedding ProviderCategory = "embedding" // 向量嵌入
|
||||
CategoryRerank ProviderCategory = "rerank" // 重排序
|
||||
CategorySpeech ProviderCategory = "speech" // 语音合成（预留）
|
||||
CategoryImage ProviderCategory = "image" // 图像生成
|
||||
CategoryAudio ProviderCategory = "audio" // 音频转写（预留）
|
||||
)
|
||||
|
||||
func SupportedCategories() []ProviderCategory {
|
||||
return []ProviderCategory{CategoryLLM, CategoryEmbedding, CategoryRerank, CategoryImage}
|
||||
}
|
||||
|
||||
func IsCategorySupported(c ProviderCategory) bool {
|
||||
for _, s := range SupportedCategories() {
|
||||
if s == c {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,243 @@
|
||||
package model
|
||||
|
||||
import (
|
||||
"time"
|
||||
)
|
||||
|
||||
type ProviderType string
|
||||
|
||||
const (
|
||||
ProviderOpenAI ProviderType = "openai"
|
||||
ProviderAnthropic ProviderType = "anthropic"
|
||||
ProviderCustom ProviderType = "custom"
|
||||
ProviderAzureOpenAI ProviderType = "azure_openai"
|
||||
ProviderOpenRouter ProviderType = "openrouter"
|
||||
ProviderOllama ProviderType = "ollama"
|
||||
ProviderComfyUI ProviderType = "comfyui"
|
||||
)
|
||||
|
||||
type WorkflowType string
|
||||
|
||||
const (
|
||||
WorkflowTxt2Img WorkflowType = "txt2img"
|
||||
WorkflowImg2Img WorkflowType = "img2img"
|
||||
)
|
||||
|
||||
type PredictionStatus string
|
||||
|
||||
const (
|
||||
PredictionStarting PredictionStatus = "starting"
|
||||
PredictionProcessing PredictionStatus = "processing"
|
||||
PredictionSucceeded PredictionStatus = "succeeded"
|
||||
PredictionFailed PredictionStatus = "failed"
|
||||
PredictionCanceled PredictionStatus = "canceled"
|
||||
)
|
||||
|
||||
type ProviderStatus string
|
||||
|
||||
const (
|
||||
ProviderHealthy ProviderStatus = "healthy"
|
||||
ProviderUnhealthy ProviderStatus = "unhealthy"
|
||||
ProviderUnknown ProviderStatus = "unknown"
|
||||
)
|
||||
|
||||
type Provider struct {
|
||||
ID uint `gorm:"primaryKey" json:"id"`
|
||||
Name string `gorm:"uniqueIndex;not null" json:"name"`
|
||||
Type ProviderType `gorm:"not null" json:"type"`
|
||||
Category ProviderCategory `gorm:"not null;default:llm" json:"category"`
|
||||
BaseURL string `json:"base_url"`
|
||||
AllowedEndpointsJSON string `gorm:"default:'[\"list_models\",\"chat_completion\"]'" json:"allowed_endpoints_json"`
|
||||
EndpointPathsJSON string `gorm:"default:'{}'" json:"endpoint_paths_json"`
|
||||
Status ProviderStatus `gorm:"default:unknown" json:"status"`
|
||||
Enabled bool `gorm:"default:true" json:"enabled"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
Keys []ProviderKey `gorm:"foreignKey:ProviderID" json:"keys,omitempty"`
|
||||
Models []ModelEntry `gorm:"foreignKey:ProviderID" json:"models,omitempty"`
|
||||
}
|
||||
|
||||
type ProviderKey struct {
|
||||
ID uint `gorm:"primaryKey" json:"id"`
|
||||
ProviderID uint `gorm:"index;not null" json:"provider_id"`
|
||||
Name string `gorm:"not null" json:"name"`
|
||||
APIKeyEnc string `gorm:"not null" json:"-"`
|
||||
Weight float64 `gorm:"default:1" json:"weight"`
|
||||
ModelsJSON string `gorm:"default:'[\"*\"]'" json:"models_json"`
|
||||
Enabled bool `gorm:"default:true" json:"enabled"`
|
||||
MaxRequestsPerDay int `gorm:"default:0" json:"max_requests_per_day"`
|
||||
MaxTokensPerDay int64 `gorm:"default:0" json:"max_tokens_per_day"`
|
||||
RequestsToday int `gorm:"default:0" json:"requests_today"`
|
||||
TokensToday int64 `gorm:"default:0" json:"tokens_today"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
}
|
||||
|
||||
type ModelEntry struct {
|
||||
ID uint `gorm:"primaryKey" json:"id"`
|
||||
ProviderID uint `gorm:"index;not null" json:"provider_id"`
|
||||
ModelID string `gorm:"not null" json:"model_id"`
|
||||
Alias string `gorm:"index" json:"alias"`
|
||||
DisplayName string `json:"display_name"`
|
||||
WorkflowType WorkflowType `gorm:"default:txt2img" json:"workflow_type"`
|
||||
WorkflowJSON string `gorm:"type:text" json:"workflow_json"`
|
||||
InputBindingJSON string `gorm:"type:text" json:"input_binding_json"`
|
||||
VersionHash string `gorm:"index" json:"version_hash"`
|
||||
Enabled bool `gorm:"default:true" json:"enabled"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
type IngressKey struct {
|
||||
ID uint `gorm:"primaryKey" json:"id"`
|
||||
Name string `gorm:"not null" json:"name"`
|
||||
KeyHash string `gorm:"uniqueIndex;not null" json:"-"`
|
||||
Prefix string `gorm:"not null" json:"prefix"`
|
||||
Enabled bool `gorm:"default:true" json:"enabled"`
|
||||
BudgetLimit float64 `gorm:"default:0" json:"budget_limit"`
|
||||
BudgetUsed float64 `gorm:"default:0" json:"budget_used"`
|
||||
RateLimitRPM int `gorm:"default:0" json:"rate_limit_rpm"`
|
||||
RateLimitTPM int `gorm:"default:0" json:"rate_limit_tpm"`
|
||||
AllowedProvidersJSON string `gorm:"default:'[\"*\"]'" json:"allowed_providers_json"`
|
||||
AllowedModelsJSON string `gorm:"default:'[\"*\"]'" json:"allowed_models_json"`
|
||||
RequestCount int64 `gorm:"default:0" json:"request_count"`
|
||||
TokenCount int64 `gorm:"default:0" json:"token_count"`
|
||||
ExpiresAt *time.Time `json:"expires_at"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
}
|
||||
|
||||
type AdminUser struct {
|
||||
ID uint `gorm:"primaryKey" json:"id"`
|
||||
Username string `gorm:"uniqueIndex;not null" json:"username"`
|
||||
PasswordHash string `gorm:"not null" json:"-"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
type AdminSession struct {
|
||||
ID uint `gorm:"primaryKey" json:"id"`
|
||||
TokenHash string `gorm:"uniqueIndex;not null" json:"-"`
|
||||
ExpiresAt time.Time `gorm:"index" json:"expires_at"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
type IPRuleType string
|
||||
|
||||
const (
|
||||
IPRuleAllow IPRuleType = "allow"
|
||||
IPRuleDeny IPRuleType = "deny"
|
||||
)
|
||||
|
||||
type IPRuleScope string
|
||||
|
||||
const (
|
||||
IPScopeAdmin IPRuleScope = "admin"
|
||||
IPScopeProxy IPRuleScope = "proxy"
|
||||
IPScopeAll IPRuleScope = "all"
|
||||
)
|
||||
|
||||
type IPRule struct {
|
||||
ID uint `gorm:"primaryKey" json:"id"`
|
||||
CIDR string `gorm:"column:cidr;not null" json:"cidr"`
|
||||
Type IPRuleType `gorm:"not null" json:"type"`
|
||||
Scope IPRuleScope `gorm:"not null;default:proxy" json:"scope"`
|
||||
Enabled bool `gorm:"default:true" json:"enabled"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
type UsageRecord struct {
|
||||
ID uint `gorm:"primaryKey" json:"id"`
|
||||
IngressKeyID uint `gorm:"index" json:"ingress_key_id"`
|
||||
ProviderID uint `gorm:"index" json:"provider_id"`
|
||||
ProviderKeyID uint `gorm:"index" json:"provider_key_id"`
|
||||
ClientIP string `gorm:"index" json:"client_ip"`
|
||||
Endpoint string `json:"endpoint"`
|
||||
Model string `json:"model"`
|
||||
TokensIn int `json:"tokens_in"`
|
||||
TokensOut int `json:"tokens_out"`
|
||||
Cost float64 `json:"cost"`
|
||||
LatencyMs int64 `json:"latency_ms"`
|
||||
Status string `json:"status"`
|
||||
Stream bool `json:"stream"`
|
||||
ErrorMsg string `json:"error_msg"`
|
||||
RequestBody string `gorm:"type:text" json:"request_body,omitempty"`
|
||||
ResponseBody string `gorm:"type:text" json:"response_body,omitempty"`
|
||||
CreatedAt time.Time `gorm:"index" json:"created_at"`
|
||||
}
|
||||
|
||||
const (
|
||||
MaxUsageLogRecords = 500
|
||||
EndpointAdminLogin = "admin_login"
|
||||
EndpointPrediction = "prediction"
|
||||
)
|
||||
|
||||
type Prediction struct {
|
||||
ID string `gorm:"primaryKey" json:"id"`
|
||||
IngressKeyID uint `gorm:"index;not null" json:"ingress_key_id"`
|
||||
ProviderID uint `gorm:"index;not null" json:"provider_id"`
|
||||
ProviderKeyID uint `gorm:"index" json:"provider_key_id"`
|
||||
ModelEntryID uint `gorm:"index" json:"model_entry_id"`
|
||||
Version string `json:"version"`
|
||||
Model string `json:"model"`
|
||||
InputJSON string `gorm:"type:text" json:"input_json"`
|
||||
Status PredictionStatus `gorm:"index;not null;default:starting" json:"status"`
|
||||
ComfyPromptID string `json:"comfy_prompt_id"`
|
||||
ClientID string `json:"client_id"`
|
||||
OutputJSON string `gorm:"type:text" json:"output_json"`
|
||||
Error string `gorm:"type:text" json:"error"`
|
||||
Logs string `gorm:"type:text" json:"logs"`
|
||||
MetricsJSON string `gorm:"type:text" json:"metrics_json"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
StartedAt *time.Time `json:"started_at"`
|
||||
CompletedAt *time.Time `json:"completed_at"`
|
||||
}
|
||||
|
||||
type StoredImage struct {
|
||||
ID string `gorm:"primaryKey" json:"id"`
|
||||
PredictionID string `gorm:"index;not null" json:"prediction_id"`
|
||||
Filename string `json:"filename"`
|
||||
LocalPath string `json:"-"`
|
||||
Mime string `json:"mime"`
|
||||
Size int64 `json:"size"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
// RoutingRule Virtual Key 路由：按模型匹配绑定 Provider / ProviderKey
|
||||
type RoutingRule struct {
|
||||
ID uint `gorm:"primaryKey" json:"id"`
|
||||
IngressKeyID uint `gorm:"index;not null" json:"ingress_key_id"`
|
||||
ModelPattern string `gorm:"not null" json:"model_pattern"`
|
||||
ProviderID uint `gorm:"not null" json:"provider_id"`
|
||||
ProviderKeyID uint `gorm:"default:0" json:"provider_key_id"`
|
||||
Priority int `gorm:"default:0" json:"priority"`
|
||||
Enabled bool `gorm:"default:true" json:"enabled"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
// SystemSettings singleton (id=1) for runtime configuration managed via admin UI.
|
||||
type SystemSettings struct {
|
||||
ID uint `gorm:"primaryKey" json:"id"`
|
||||
EncryptionKey string `gorm:"not null" json:"-"`
|
||||
SessionTTLSeconds int64 `gorm:"not null;default:86400" json:"session_ttl_seconds"`
|
||||
LoginMaxAttempts int `gorm:"not null;default:5" json:"login_max_attempts"`
|
||||
LoginLockoutSeconds int64 `gorm:"not null;default:900" json:"login_lockout_seconds"`
|
||||
TrustedProxiesJSON string `gorm:"default:'[]'" json:"trusted_proxies_json"`
|
||||
UsageFlushSeconds int64 `gorm:"not null;default:5" json:"usage_flush_seconds"`
|
||||
HealthCheckSeconds int64 `gorm:"not null;default:30" json:"health_check_seconds"`
|
||||
GatewayMaxRetries int `gorm:"not null;default:3" json:"gateway_max_retries"`
|
||||
GatewayRetryBackoffMs int `gorm:"not null;default:200" json:"gateway_retry_backoff_ms"`
|
||||
ImageStoragePath string `gorm:"default:'images'" json:"image_storage_path"`
|
||||
PublicBaseURL string `json:"public_base_url"`
|
||||
SignedURLTTLSeconds int64 `gorm:"not null;default:3600" json:"signed_url_ttl_seconds"`
|
||||
PredictionWaitSeconds int64 `gorm:"not null;default:60" json:"prediction_wait_seconds"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
}
|
||||
|
||||
type HealthCheckRecord struct {
|
||||
ID uint `gorm:"primaryKey" json:"id"`
|
||||
ProviderID uint `gorm:"index" json:"provider_id"`
|
||||
Status ProviderStatus `json:"status"`
|
||||
LatencyMs int64 `json:"latency_ms"`
|
||||
ErrorMsg string `json:"error_msg"`
|
||||
CheckedAt time.Time `gorm:"index" json:"checked_at"`
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
package model
|
||||
|
||||
// IngressID returns the model name exposed to ingress API clients.
|
||||
func (m ModelEntry) IngressID() string {
|
||||
if m.Alias != "" {
|
||||
return m.Alias
|
||||
}
|
||||
return m.ModelID
|
||||
}
|
||||
|
||||
// MatchesIngressName reports whether the entry matches an ingress model identifier.
|
||||
func (m ModelEntry) MatchesIngressName(name string) bool {
|
||||
if name == "" || !m.Enabled {
|
||||
return false
|
||||
}
|
||||
if m.ModelID == name || m.VersionHash == name {
|
||||
return true
|
||||
}
|
||||
return m.Alias != "" && m.Alias == name
|
||||
}
|
||||
@@ -0,0 +1,175 @@
|
||||
package monitor
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/gateway"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/settings"
|
||||
"github.com/rose_cat707/luminary/internal/store/cache"
|
||||
sqlitestore "github.com/rose_cat707/luminary/internal/store/sqlite"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
type Service struct {
|
||||
gateway *gateway.Service
|
||||
cache *cache.Store
|
||||
db *gorm.DB
|
||||
settings *settings.Runtime
|
||||
stop chan struct{}
|
||||
healthRestart chan struct{}
|
||||
flushRestart chan struct{}
|
||||
flushFn func([]cache.UsageDelta)
|
||||
}
|
||||
|
||||
func New(gw *gateway.Service, c *cache.Store, store *sqlitestore.Store, rt *settings.Runtime) *Service {
|
||||
return &Service{
|
||||
gateway: gw,
|
||||
cache: c,
|
||||
db: store.DB(),
|
||||
settings: rt,
|
||||
stop: make(chan struct{}),
|
||||
healthRestart: make(chan struct{}, 1),
|
||||
flushRestart: make(chan struct{}, 1),
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) Start(flushFn func([]cache.UsageDelta)) {
|
||||
s.flushFn = flushFn
|
||||
go s.healthLoop()
|
||||
go s.usageFlushLoop()
|
||||
go s.dailyResetLoop()
|
||||
}
|
||||
|
||||
func (s *Service) Stop() {
|
||||
close(s.stop)
|
||||
}
|
||||
|
||||
func (s *Service) NotifySettingsChanged() {
|
||||
select {
|
||||
case s.healthRestart <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
select {
|
||||
case s.flushRestart <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) healthInterval() time.Duration {
|
||||
interval := s.settings.HealthCheckInterval()
|
||||
if interval < time.Second {
|
||||
return 30 * time.Second
|
||||
}
|
||||
return interval
|
||||
}
|
||||
|
||||
func (s *Service) flushInterval() time.Duration {
|
||||
interval := s.settings.UsageFlushInterval()
|
||||
if interval < time.Second {
|
||||
return 5 * time.Second
|
||||
}
|
||||
return interval
|
||||
}
|
||||
|
||||
func (s *Service) healthLoop() {
|
||||
var ticker *time.Ticker
|
||||
resetTicker := func() {
|
||||
if ticker != nil {
|
||||
ticker.Stop()
|
||||
}
|
||||
ticker = time.NewTicker(s.healthInterval())
|
||||
}
|
||||
resetTicker()
|
||||
defer ticker.Stop()
|
||||
s.runChecks()
|
||||
for {
|
||||
select {
|
||||
case <-ticker.C:
|
||||
s.runChecks()
|
||||
case <-s.healthRestart:
|
||||
resetTicker()
|
||||
case <-s.stop:
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) usageFlushLoop() {
|
||||
var ticker *time.Ticker
|
||||
resetTicker := func() {
|
||||
if ticker != nil {
|
||||
ticker.Stop()
|
||||
}
|
||||
ticker = time.NewTicker(s.flushInterval())
|
||||
}
|
||||
resetTicker()
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ticker.C:
|
||||
s.flushUsage()
|
||||
case <-s.flushRestart:
|
||||
resetTicker()
|
||||
case <-s.stop:
|
||||
s.flushUsage()
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) flushUsage() {
|
||||
if s.flushFn == nil {
|
||||
return
|
||||
}
|
||||
deltas := s.cache.DrainUsageDeltas()
|
||||
if len(deltas) > 0 {
|
||||
s.flushFn(deltas)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) runChecks() {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
|
||||
defer cancel()
|
||||
for _, p := range s.cache.ListProviders() {
|
||||
status, latency, errMsg := s.gateway.HealthCheckProvider(ctx, p.ID)
|
||||
rec := model.HealthCheckRecord{
|
||||
ProviderID: p.ID,
|
||||
Status: status,
|
||||
LatencyMs: latency,
|
||||
ErrorMsg: errMsg,
|
||||
CheckedAt: time.Now(),
|
||||
}
|
||||
s.cache.SetHealthStatus(p.ID, rec)
|
||||
if err := s.db.Create(&rec).Error; err != nil {
|
||||
log.Printf("save health check: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) dailyResetLoop() {
|
||||
for {
|
||||
now := time.Now()
|
||||
next := time.Date(now.Year(), now.Month(), now.Day()+1, 0, 0, 0, 0, now.Location())
|
||||
timer := time.NewTimer(time.Until(next))
|
||||
select {
|
||||
case <-timer.C:
|
||||
s.resetDailyCounters()
|
||||
case <-s.stop:
|
||||
timer.Stop()
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) resetDailyCounters() {
|
||||
s.cache.ResetProviderKeyDailyCounts()
|
||||
if err := s.db.Model(&model.ProviderKey{}).Updates(map[string]any{
|
||||
"requests_today": 0,
|
||||
"tokens_today": 0,
|
||||
}).Error; err != nil {
|
||||
log.Printf("reset daily provider key counters: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
package prediction
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
)
|
||||
|
||||
type NodeField struct {
|
||||
Node string `json:"node"`
|
||||
Field string `json:"field"`
|
||||
}
|
||||
|
||||
type InputBinding map[string]*NodeField
|
||||
|
||||
func ParseBinding(raw string) (InputBinding, error) {
|
||||
if raw == "" {
|
||||
return InputBinding{}, nil
|
||||
}
|
||||
var b InputBinding
|
||||
if err := json.Unmarshal([]byte(raw), &b); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return b, nil
|
||||
}
|
||||
|
||||
func (b InputBinding) Validate(workflow map[string]any, required []string) error {
|
||||
for _, name := range required {
|
||||
f, ok := b[name]
|
||||
if !ok || f == nil || f.Node == "" || f.Field == "" {
|
||||
return fmt.Errorf("required binding missing: %s", name)
|
||||
}
|
||||
}
|
||||
for name, f := range b {
|
||||
if f == nil || f.Node == "" || f.Field == "" {
|
||||
continue
|
||||
}
|
||||
if err := validateNodeField(workflow, f.Node, f.Field); err != nil {
|
||||
return fmt.Errorf("binding %s: %w", name, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func validateNodeField(workflow map[string]any, nodeID, field string) error {
|
||||
node, ok := workflow[nodeID].(map[string]any)
|
||||
if !ok {
|
||||
return fmt.Errorf("node %s not found", nodeID)
|
||||
}
|
||||
inputs, ok := node["inputs"].(map[string]any)
|
||||
if !ok {
|
||||
return fmt.Errorf("node %s has no inputs", nodeID)
|
||||
}
|
||||
if _, ok := inputs[field]; !ok {
|
||||
return fmt.Errorf("field %s not found on node %s", field, nodeID)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
package prediction
|
||||
|
||||
import (
|
||||
"sync"
|
||||
)
|
||||
|
||||
type Event struct {
|
||||
Status string `json:"status"`
|
||||
Logs string `json:"logs,omitempty"`
|
||||
Metrics map[string]any `json:"metrics,omitempty"`
|
||||
Output any `json:"output,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
Done bool `json:"-"`
|
||||
}
|
||||
|
||||
type Hub struct {
|
||||
mu sync.RWMutex
|
||||
subs map[string]map[chan Event]struct{}
|
||||
}
|
||||
|
||||
func NewHub() *Hub {
|
||||
return &Hub{subs: make(map[string]map[chan Event]struct{})}
|
||||
}
|
||||
|
||||
func (h *Hub) Subscribe(predictionID string) chan Event {
|
||||
ch := make(chan Event, 16)
|
||||
h.mu.Lock()
|
||||
if h.subs[predictionID] == nil {
|
||||
h.subs[predictionID] = make(map[chan Event]struct{})
|
||||
}
|
||||
h.subs[predictionID][ch] = struct{}{}
|
||||
h.mu.Unlock()
|
||||
return ch
|
||||
}
|
||||
|
||||
func (h *Hub) Unsubscribe(predictionID string, ch chan Event) {
|
||||
h.mu.Lock()
|
||||
if m := h.subs[predictionID]; m != nil {
|
||||
delete(m, ch)
|
||||
if len(m) == 0 {
|
||||
delete(h.subs, predictionID)
|
||||
}
|
||||
}
|
||||
h.mu.Unlock()
|
||||
close(ch)
|
||||
}
|
||||
|
||||
func (h *Hub) Publish(predictionID string, ev Event) {
|
||||
h.mu.RLock()
|
||||
defer h.mu.RUnlock()
|
||||
for ch := range h.subs[predictionID] {
|
||||
select {
|
||||
case ch <- ev:
|
||||
default:
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
package prediction
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"math/big"
|
||||
)
|
||||
|
||||
func BuildPrompt(workflowJSON string, binding InputBinding, input map[string]any) (map[string]any, error) {
|
||||
var workflow map[string]any
|
||||
if err := json.Unmarshal([]byte(workflowJSON), &workflow); err != nil {
|
||||
return nil, fmt.Errorf("invalid workflow: %w", err)
|
||||
}
|
||||
prompt := deepCopyMap(workflow)
|
||||
for name, target := range binding {
|
||||
if target == nil || target.Node == "" || target.Field == "" {
|
||||
continue
|
||||
}
|
||||
val, ok := input[name]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if name == "seed" {
|
||||
if iv, ok := val.(int); ok && iv < 0 {
|
||||
n, _ := rand.Int(rand.Reader, big.NewInt(1<<31-1))
|
||||
val = int(n.Int64())
|
||||
}
|
||||
}
|
||||
node, ok := prompt[target.Node].(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("node %s not found", target.Node)
|
||||
}
|
||||
inputs, ok := node["inputs"].(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("node %s inputs missing", target.Node)
|
||||
}
|
||||
inputs[target.Field] = val
|
||||
}
|
||||
return prompt, nil
|
||||
}
|
||||
|
||||
func deepCopyMap(src map[string]any) map[string]any {
|
||||
b, _ := json.Marshal(src)
|
||||
var dst map[string]any
|
||||
_ = json.Unmarshal(b, &dst)
|
||||
return dst
|
||||
}
|
||||
@@ -0,0 +1,122 @@
|
||||
package prediction
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
)
|
||||
|
||||
type ParamType string
|
||||
|
||||
const (
|
||||
ParamString ParamType = "string"
|
||||
ParamInt ParamType = "int"
|
||||
ParamFloat ParamType = "float"
|
||||
)
|
||||
|
||||
type ParamDef struct {
|
||||
Name string `json:"name"`
|
||||
Label string `json:"label"`
|
||||
Type ParamType `json:"type"`
|
||||
Required bool `json:"required"`
|
||||
Default any `json:"default,omitempty"`
|
||||
}
|
||||
|
||||
var Txt2ImgParams = []ParamDef{
|
||||
{Name: "prompt", Label: "正向提示词", Type: ParamString, Required: true},
|
||||
{Name: "negative_prompt", Label: "负向提示词", Type: ParamString, Default: ""},
|
||||
{Name: "seed", Label: "随机种子", Type: ParamInt, Default: -1},
|
||||
{Name: "width", Label: "宽度", Type: ParamInt, Default: 1024},
|
||||
{Name: "height", Label: "高度", Type: ParamInt, Default: 1024},
|
||||
{Name: "steps", Label: "采样步数", Type: ParamInt, Default: 20},
|
||||
{Name: "cfg_scale", Label: "CFG", Type: ParamFloat, Default: 7.0},
|
||||
}
|
||||
|
||||
var Img2ImgParams = []ParamDef{
|
||||
{Name: "prompt", Label: "正向提示词", Type: ParamString, Required: true},
|
||||
{Name: "negative_prompt", Label: "负向提示词", Type: ParamString, Default: ""},
|
||||
{Name: "image", Label: "输入图 URL", Type: ParamString, Required: true},
|
||||
{Name: "seed", Label: "随机种子", Type: ParamInt, Default: -1},
|
||||
{Name: "steps", Label: "采样步数", Type: ParamInt, Default: 20},
|
||||
{Name: "cfg_scale", Label: "CFG", Type: ParamFloat, Default: 7.0},
|
||||
{Name: "denoise", Label: "重绘幅度", Type: ParamFloat, Default: 0.75},
|
||||
}
|
||||
|
||||
func ParamsForWorkflowType(wt model.WorkflowType) []ParamDef {
|
||||
switch wt {
|
||||
case model.WorkflowImg2Img:
|
||||
return Img2ImgParams
|
||||
default:
|
||||
return Txt2ImgParams
|
||||
}
|
||||
}
|
||||
|
||||
func RequiredBindings(wt model.WorkflowType) []string {
|
||||
switch wt {
|
||||
case model.WorkflowImg2Img:
|
||||
return []string{"prompt", "image"}
|
||||
default:
|
||||
return []string{"prompt"}
|
||||
}
|
||||
}
|
||||
|
||||
func ValidateInput(wt model.WorkflowType, input map[string]any) (map[string]any, error) {
|
||||
defs := ParamsForWorkflowType(wt)
|
||||
out := make(map[string]any, len(defs))
|
||||
for _, d := range defs {
|
||||
v, ok := input[d.Name]
|
||||
if !ok || v == nil {
|
||||
if d.Required {
|
||||
return nil, fmt.Errorf("missing required field: %s", d.Name)
|
||||
}
|
||||
if d.Default != nil {
|
||||
out[d.Name] = d.Default
|
||||
}
|
||||
continue
|
||||
}
|
||||
coerced, err := coerceValue(d, v)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", d.Name, err)
|
||||
}
|
||||
out[d.Name] = coerced
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func coerceValue(d ParamDef, v any) (any, error) {
|
||||
switch d.Type {
|
||||
case ParamString:
|
||||
s, ok := v.(string)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("expected string")
|
||||
}
|
||||
if d.Required && s == "" {
|
||||
return nil, fmt.Errorf("cannot be empty")
|
||||
}
|
||||
return s, nil
|
||||
case ParamInt:
|
||||
switch n := v.(type) {
|
||||
case float64:
|
||||
return int(n), nil
|
||||
case int:
|
||||
return n, nil
|
||||
case int64:
|
||||
return int(n), nil
|
||||
default:
|
||||
return nil, fmt.Errorf("expected integer")
|
||||
}
|
||||
case ParamFloat:
|
||||
switch n := v.(type) {
|
||||
case float64:
|
||||
return n, nil
|
||||
case int:
|
||||
return float64(n), nil
|
||||
case int64:
|
||||
return float64(n), nil
|
||||
default:
|
||||
return nil, fmt.Errorf("expected number")
|
||||
}
|
||||
default:
|
||||
return v, nil
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,475 @@
|
||||
package prediction
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"path"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/provider/comfyui"
|
||||
"github.com/rose_cat707/luminary/internal/storage/imagestore"
|
||||
"github.com/rose_cat707/luminary/internal/store/cache"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
type Service struct {
|
||||
db *gorm.DB
|
||||
cache *cache.Store
|
||||
images *imagestore.Store
|
||||
hub *Hub
|
||||
publicURL string
|
||||
waitSec int64
|
||||
mu sync.Mutex
|
||||
running map[string]context.CancelFunc
|
||||
}
|
||||
|
||||
func NewService(db *gorm.DB, c *cache.Store, images *imagestore.Store, publicURL string, waitSec int64) *Service {
|
||||
if waitSec <= 0 {
|
||||
waitSec = 60
|
||||
}
|
||||
return &Service{
|
||||
db: db, cache: c, images: images, hub: NewHub(),
|
||||
publicURL: strings.TrimRight(publicURL, "/"), waitSec: waitSec,
|
||||
running: make(map[string]context.CancelFunc),
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) Hub() *Hub { return s.hub }
|
||||
|
||||
func NewPredictionID() (string, error) {
|
||||
b := make([]byte, 12)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return hex.EncodeToString(b), nil
|
||||
}
|
||||
|
||||
type CreateRequest struct {
|
||||
Version string `json:"version"`
|
||||
Input map[string]any `json:"input"`
|
||||
}
|
||||
|
||||
func (s *Service) Create(ctx context.Context, ingress *model.IngressKey, req CreateRequest, clientIP string, wait bool) (*model.Prediction, error) {
|
||||
if req.Version == "" {
|
||||
return nil, fmt.Errorf("version is required")
|
||||
}
|
||||
entry, provider, err := s.resolveModel(req.Version, ingress)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !provider.Enabled {
|
||||
return nil, fmt.Errorf("provider disabled")
|
||||
}
|
||||
input, err := ValidateInput(entry.WorkflowType, req.Input)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
binding, err := ParseBinding(entry.InputBindingJSON)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var workflow map[string]any
|
||||
_ = json.Unmarshal([]byte(entry.WorkflowJSON), &workflow)
|
||||
if err := binding.Validate(workflow, RequiredBindings(entry.WorkflowType)); err != nil {
|
||||
return nil, fmt.Errorf("model binding invalid: %w", err)
|
||||
}
|
||||
|
||||
id, err := NewPredictionID()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
inputJSON, _ := json.Marshal(input)
|
||||
p := model.Prediction{
|
||||
ID: id,
|
||||
IngressKeyID: ingress.ID,
|
||||
ProviderID: provider.ID,
|
||||
ModelEntryID: entry.ID,
|
||||
Version: req.Version,
|
||||
Model: entry.ModelID,
|
||||
InputJSON: string(inputJSON),
|
||||
Status: model.PredictionStarting,
|
||||
ClientID: "luminary-" + id,
|
||||
CreatedAt: time.Now(),
|
||||
}
|
||||
if err := s.db.Create(&p).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
runCtx, cancel := context.WithCancel(context.Background())
|
||||
s.mu.Lock()
|
||||
s.running[id] = cancel
|
||||
s.mu.Unlock()
|
||||
|
||||
go s.execute(runCtx, &p, entry, &provider, binding, input, clientIP)
|
||||
|
||||
if wait {
|
||||
deadline := time.After(time.Duration(s.waitSec) * time.Second)
|
||||
for {
|
||||
select {
|
||||
case <-deadline:
|
||||
return s.Get(ctx, id, ingress)
|
||||
case <-time.After(200 * time.Millisecond):
|
||||
cur, err := s.Get(ctx, id, ingress)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if cur.Status == model.PredictionSucceeded || cur.Status == model.PredictionFailed || cur.Status == model.PredictionCanceled {
|
||||
return cur, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return s.toAPI(&p), nil
|
||||
}
|
||||
|
||||
func (s *Service) resolveModel(version string, ingress *model.IngressKey) (model.ModelEntry, model.Provider, error) {
|
||||
allowedProviders := cache.ParseJSONList(ingress.AllowedProvidersJSON)
|
||||
allowedModels := cache.ParseJSONList(ingress.AllowedModelsJSON)
|
||||
var candidates []model.ModelEntry
|
||||
for _, p := range s.cache.ListProviders() {
|
||||
if p.Category != model.CategoryImage || !p.Enabled {
|
||||
continue
|
||||
}
|
||||
if !cache.MatchesList(allowedProviders, p.Name) && !cache.MatchesList(allowedProviders, string(p.Type)) && !cache.MatchesList(allowedProviders, "*") {
|
||||
continue
|
||||
}
|
||||
for _, m := range s.cache.GetModels(p.ID) {
|
||||
if !m.Enabled {
|
||||
continue
|
||||
}
|
||||
ingressID := m.IngressID()
|
||||
if !m.MatchesIngressName(version) {
|
||||
continue
|
||||
}
|
||||
if !cache.MatchesList(allowedModels, ingressID) && !cache.MatchesList(allowedModels, m.ModelID) && !cache.MatchesList(allowedModels, "*") {
|
||||
continue
|
||||
}
|
||||
candidates = append(candidates, m)
|
||||
}
|
||||
}
|
||||
if len(candidates) == 0 {
|
||||
return model.ModelEntry{}, model.Provider{}, fmt.Errorf("no model found for version %s", version)
|
||||
}
|
||||
m := candidates[0]
|
||||
p, ok := s.cache.GetProvider(m.ProviderID)
|
||||
if !ok {
|
||||
return model.ModelEntry{}, model.Provider{}, fmt.Errorf("provider not found")
|
||||
}
|
||||
return m, p, nil
|
||||
}
|
||||
|
||||
func (s *Service) execute(ctx context.Context, p *model.Prediction, entry model.ModelEntry, provider *model.Provider, binding InputBinding, input map[string]any, clientIP string) {
|
||||
defer func() {
|
||||
s.mu.Lock()
|
||||
delete(s.running, p.ID)
|
||||
s.mu.Unlock()
|
||||
}()
|
||||
|
||||
start := time.Now()
|
||||
s.updateStatus(p, model.PredictionProcessing, "", nil)
|
||||
s.hub.Publish(p.ID, Event{Status: string(model.PredictionProcessing)})
|
||||
|
||||
client := comfyui.New(provider.BaseURL)
|
||||
runInput := cloneMap(input)
|
||||
if entry.WorkflowType == model.WorkflowImg2Img {
|
||||
imageURL, _ := runInput["image"].(string)
|
||||
data, err := downloadURL(ctx, imageURL)
|
||||
if err != nil {
|
||||
s.fail(p, clientIP, start, fmt.Errorf("download image: %w", err))
|
||||
return
|
||||
}
|
||||
name := path.Base(imageURL)
|
||||
if name == "" || name == "." || name == "/" {
|
||||
name = "input.png"
|
||||
}
|
||||
uploaded, err := client.UploadImage(ctx, data, name)
|
||||
if err != nil {
|
||||
s.fail(p, clientIP, start, fmt.Errorf("upload image: %w", err))
|
||||
return
|
||||
}
|
||||
runInput["image"] = uploaded
|
||||
}
|
||||
|
||||
prompt, err := BuildPrompt(entry.WorkflowJSON, binding, runInput)
|
||||
if err != nil {
|
||||
s.fail(p, clientIP, start, err)
|
||||
return
|
||||
}
|
||||
|
||||
wsCtx, wsCancel := context.WithCancel(ctx)
|
||||
defer wsCancel()
|
||||
go client.Watch(wsCtx, p.ClientID, func(ev comfyui.ProgressEvent) {
|
||||
switch ev.Type {
|
||||
case "progress":
|
||||
if ev.Max > 0 {
|
||||
logs := fmt.Sprintf("progress %.0f/%.0f", ev.Value, ev.Max)
|
||||
s.appendLogs(p, logs)
|
||||
s.hub.Publish(p.ID, Event{Status: string(model.PredictionProcessing), Logs: logs})
|
||||
}
|
||||
case "execution_error":
|
||||
s.appendLogs(p, ev.ErrorText)
|
||||
}
|
||||
})
|
||||
|
||||
submit, err := client.SubmitPrompt(ctx, prompt, p.ClientID)
|
||||
if err != nil {
|
||||
s.fail(p, clientIP, start, err)
|
||||
return
|
||||
}
|
||||
p.ComfyPromptID = submit.PromptID
|
||||
now := time.Now()
|
||||
p.StartedAt = &now
|
||||
s.db.Model(p).Updates(map[string]any{"comfy_prompt_id": p.ComfyPromptID, "started_at": p.StartedAt})
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
_ = client.Interrupt(context.Background())
|
||||
s.updateStatus(p, model.PredictionCanceled, "canceled", nil)
|
||||
s.hub.Publish(p.ID, Event{Status: string(model.PredictionCanceled), Done: true})
|
||||
s.recordUsage(ingressByID(s.cache, p.IngressKeyID), provider, clientIP, p, start, "canceled", "canceled")
|
||||
return
|
||||
case <-time.After(2 * time.Second):
|
||||
history, err := client.GetHistory(ctx, p.ComfyPromptID)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if _, ok := history[p.ComfyPromptID]; !ok {
|
||||
continue
|
||||
}
|
||||
images := comfyui.CollectOutputImages(history, p.ComfyPromptID)
|
||||
if len(images) == 0 {
|
||||
continue
|
||||
}
|
||||
var urls []string
|
||||
for _, img := range images {
|
||||
data, mime, err := client.DownloadView(ctx, img.Filename, img.Subfolder, img.Type)
|
||||
if err != nil {
|
||||
s.fail(p, clientIP, start, err)
|
||||
return
|
||||
}
|
||||
stored, err := s.images.Save(p.ID, img.Filename, data, mime)
|
||||
if err != nil {
|
||||
s.fail(p, clientIP, start, err)
|
||||
return
|
||||
}
|
||||
urls = append(urls, s.images.SignURL(stored.ID))
|
||||
}
|
||||
outJSON, _ := json.Marshal(urls)
|
||||
p.OutputJSON = string(outJSON)
|
||||
completed := time.Now()
|
||||
p.CompletedAt = &completed
|
||||
metrics, _ := json.Marshal(map[string]any{
|
||||
"predict_time": time.Since(start).Seconds(),
|
||||
"total_time": completed.Sub(p.CreatedAt).Seconds(),
|
||||
})
|
||||
p.MetricsJSON = string(metrics)
|
||||
s.updateStatus(p, model.PredictionSucceeded, "", &completed)
|
||||
s.hub.Publish(p.ID, Event{
|
||||
Status: string(model.PredictionSucceeded), Output: urls,
|
||||
Metrics: map[string]any{"predict_time": time.Since(start).Seconds()},
|
||||
Done: true,
|
||||
})
|
||||
s.recordUsage(ingressByID(s.cache, p.IngressKeyID), provider, clientIP, p, start, "success", "")
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Service) fail(p *model.Prediction, clientIP string, start time.Time, err error) {
|
||||
completed := time.Now()
|
||||
p.CompletedAt = &completed
|
||||
p.Error = err.Error()
|
||||
s.updateStatus(p, model.PredictionFailed, err.Error(), &completed)
|
||||
s.hub.Publish(p.ID, Event{Status: string(model.PredictionFailed), Error: err.Error(), Done: true})
|
||||
prov, _ := s.cache.GetProvider(p.ProviderID)
|
||||
s.recordUsage(ingressByID(s.cache, p.IngressKeyID), &prov, clientIP, p, start, "error", err.Error())
|
||||
}
|
||||
|
||||
func (s *Service) updateStatus(p *model.Prediction, status model.PredictionStatus, errMsg string, completed *time.Time) {
|
||||
p.Status = status
|
||||
if errMsg != "" {
|
||||
p.Error = errMsg
|
||||
}
|
||||
updates := map[string]any{"status": status, "error": p.Error, "logs": p.Logs, "output_json": p.OutputJSON, "metrics_json": p.MetricsJSON}
|
||||
if completed != nil {
|
||||
updates["completed_at"] = completed
|
||||
}
|
||||
s.db.Model(p).Where("id = ?", p.ID).Updates(updates)
|
||||
}
|
||||
|
||||
func (s *Service) appendLogs(p *model.Prediction, line string) {
|
||||
if p.Logs != "" {
|
||||
p.Logs += "\n"
|
||||
}
|
||||
p.Logs += line
|
||||
s.db.Model(p).Where("id = ?", p.ID).Update("logs", p.Logs)
|
||||
}
|
||||
|
||||
func (s *Service) Get(ctx context.Context, id string, ingress *model.IngressKey) (*model.Prediction, error) {
|
||||
var p model.Prediction
|
||||
if err := s.db.First(&p, "id = ?", id).Error; err != nil {
|
||||
return nil, fmt.Errorf("prediction not found")
|
||||
}
|
||||
if ingress != nil && p.IngressKeyID != ingress.ID {
|
||||
return nil, fmt.Errorf("prediction not found")
|
||||
}
|
||||
return s.toAPI(&p), nil
|
||||
}
|
||||
|
||||
func (s *Service) List(ctx context.Context, ingress *model.IngressKey, cursor string, limit int) ([]*model.Prediction, error) {
|
||||
if limit <= 0 || limit > 100 {
|
||||
limit = 100
|
||||
}
|
||||
q := s.db.Where("ingress_key_id = ?", ingress.ID).Order("created_at desc").Limit(limit)
|
||||
if cursor != "" {
|
||||
q = q.Where("id < ?", cursor)
|
||||
}
|
||||
var rows []model.Prediction
|
||||
if err := q.Find(&rows).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := make([]*model.Prediction, 0, len(rows))
|
||||
for i := range rows {
|
||||
out = append(out, s.toAPI(&rows[i]))
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (s *Service) Cancel(ctx context.Context, id string, ingress *model.IngressKey) (*model.Prediction, error) {
|
||||
p, err := s.Get(ctx, id, ingress)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if p.Status == model.PredictionSucceeded || p.Status == model.PredictionFailed || p.Status == model.PredictionCanceled {
|
||||
return p, nil
|
||||
}
|
||||
s.mu.Lock()
|
||||
if cancel, ok := s.running[id]; ok {
|
||||
cancel()
|
||||
}
|
||||
s.mu.Unlock()
|
||||
return s.Get(ctx, id, ingress)
|
||||
}
|
||||
|
||||
func (s *Service) toAPI(p *model.Prediction) *model.Prediction {
|
||||
return p
|
||||
}
|
||||
|
||||
func (s *Service) APIView(p *model.Prediction, baseURL string) map[string]any {
|
||||
var input any
|
||||
_ = json.Unmarshal([]byte(p.InputJSON), &input)
|
||||
var output any
|
||||
if p.OutputJSON != "" {
|
||||
_ = json.Unmarshal([]byte(p.OutputJSON), &output)
|
||||
}
|
||||
var metrics any
|
||||
if p.MetricsJSON != "" {
|
||||
_ = json.Unmarshal([]byte(p.MetricsJSON), &metrics)
|
||||
}
|
||||
base := strings.TrimRight(baseURL, "/")
|
||||
return map[string]any{
|
||||
"id": p.ID,
|
||||
"version": p.Version,
|
||||
"input": input,
|
||||
"output": output,
|
||||
"error": nilIfEmpty(p.Error),
|
||||
"status": p.Status,
|
||||
"created_at": p.CreatedAt.UTC().Format(time.RFC3339Nano),
|
||||
"started_at": formatTime(p.StartedAt),
|
||||
"completed_at": formatTime(p.CompletedAt),
|
||||
"logs": p.Logs,
|
||||
"metrics": metrics,
|
||||
"urls": map[string]string{
|
||||
"get": fmt.Sprintf("%s/v1/predictions/%s", base, p.ID),
|
||||
"cancel": fmt.Sprintf("%s/v1/predictions/%s/cancel", base, p.ID),
|
||||
"stream": fmt.Sprintf("%s/v1/predictions/%s/stream", base, p.ID),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func nilIfEmpty(s string) any {
|
||||
if s == "" {
|
||||
return nil
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func formatTime(t *time.Time) any {
|
||||
if t == nil {
|
||||
return nil
|
||||
}
|
||||
return t.UTC().Format(time.RFC3339Nano)
|
||||
}
|
||||
|
||||
func downloadURL(ctx context.Context, raw string) ([]byte, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, raw, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode >= 400 {
|
||||
return nil, fmt.Errorf("http %d", resp.StatusCode)
|
||||
}
|
||||
return io.ReadAll(resp.Body)
|
||||
}
|
||||
|
||||
func cloneMap(in map[string]any) map[string]any {
|
||||
b, _ := json.Marshal(in)
|
||||
var out map[string]any
|
||||
_ = json.Unmarshal(b, &out)
|
||||
return out
|
||||
}
|
||||
|
||||
func ingressByID(c *cache.Store, id uint) *model.IngressKey {
|
||||
for _, k := range c.ListIngressKeys() {
|
||||
if k.ID == id {
|
||||
kCopy := k
|
||||
return &kCopy
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Service) recordUsage(ingress *model.IngressKey, provider *model.Provider, clientIP string, p *model.Prediction, start time.Time, status, errMsg string) {
|
||||
if ingress == nil {
|
||||
return
|
||||
}
|
||||
var providerID uint
|
||||
if provider != nil {
|
||||
providerID = provider.ID
|
||||
}
|
||||
s.cache.RecordUsage(cache.UsageDelta{
|
||||
IngressKeyID: ingress.ID,
|
||||
ProviderID: providerID,
|
||||
ClientIP: clientIP,
|
||||
Endpoint: model.EndpointPrediction,
|
||||
Model: p.ID,
|
||||
LatencyMs: time.Since(start).Milliseconds(),
|
||||
Status: status,
|
||||
ErrorMsg: errMsg,
|
||||
RequestBody: p.InputJSON,
|
||||
ResponseBody: p.OutputJSON,
|
||||
Requests: 1,
|
||||
})
|
||||
}
|
||||
|
||||
func (s *Service) GetDBPrediction(id string) (*model.Prediction, error) {
|
||||
var p model.Prediction
|
||||
if err := s.db.First(&p, "id = ?", id).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &p, nil
|
||||
}
|
||||
@@ -0,0 +1,268 @@
|
||||
package prediction
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
)
|
||||
|
||||
type WorkflowNode struct {
|
||||
ID string `json:"id"`
|
||||
ClassType string `json:"class_type"`
|
||||
Title string `json:"title"`
|
||||
InjectableFields []string `json:"injectable_fields"`
|
||||
}
|
||||
|
||||
type BindingCandidate struct {
|
||||
Node string `json:"node"`
|
||||
Field string `json:"field"`
|
||||
}
|
||||
|
||||
type BindingSuggestion struct {
|
||||
Param string `json:"param"`
|
||||
Node string `json:"node,omitempty"`
|
||||
Field string `json:"field,omitempty"`
|
||||
Type ParamType `json:"type"`
|
||||
DefaultFromWorkflow any `json:"default_from_workflow,omitempty"`
|
||||
Confidence string `json:"confidence"`
|
||||
Candidates []BindingCandidate `json:"candidates"`
|
||||
}
|
||||
|
||||
type AnalyzeResult struct {
|
||||
VersionHashPreview string `json:"version_hash_preview"`
|
||||
Nodes []WorkflowNode `json:"nodes"`
|
||||
Suggestions []BindingSuggestion `json:"suggestions"`
|
||||
}
|
||||
|
||||
func HashWorkflowJSON(raw string) string {
|
||||
sum := sha256.Sum256([]byte(raw))
|
||||
return hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
func AnalyzeWorkflow(workflowJSON string, wt model.WorkflowType) (*AnalyzeResult, error) {
|
||||
var workflow map[string]any
|
||||
if err := json.Unmarshal([]byte(workflowJSON), &workflow); err != nil {
|
||||
return nil, fmt.Errorf("invalid workflow json: %w", err)
|
||||
}
|
||||
nodes := parseNodes(workflow)
|
||||
suggestions := suggestBindings(workflow, nodes, wt)
|
||||
return &AnalyzeResult{
|
||||
VersionHashPreview: HashWorkflowJSON(workflowJSON),
|
||||
Nodes: nodes,
|
||||
Suggestions: suggestions,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func parseNodes(workflow map[string]any) []WorkflowNode {
|
||||
var out []WorkflowNode
|
||||
for id, raw := range workflow {
|
||||
node, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
classType, _ := node["class_type"].(string)
|
||||
title := ""
|
||||
if meta, ok := node["_meta"].(map[string]any); ok {
|
||||
title, _ = meta["title"].(string)
|
||||
}
|
||||
var fields []string
|
||||
if inputs, ok := node["inputs"].(map[string]any); ok {
|
||||
for k, v := range inputs {
|
||||
if isInjectableValue(v) {
|
||||
fields = append(fields, k)
|
||||
}
|
||||
}
|
||||
}
|
||||
out = append(out, WorkflowNode{
|
||||
ID: id, ClassType: classType, Title: title, InjectableFields: fields,
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func isInjectableValue(v any) bool {
|
||||
switch v.(type) {
|
||||
case string, float64, bool, int, int64:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
func suggestBindings(workflow map[string]any, nodes []WorkflowNode, wt model.WorkflowType) []BindingSuggestion {
|
||||
defs := ParamsForWorkflowType(wt)
|
||||
var clipNodes []WorkflowNode
|
||||
var samplerNodes []WorkflowNode
|
||||
var latentNodes []WorkflowNode
|
||||
var loadImageNodes []WorkflowNode
|
||||
for _, n := range nodes {
|
||||
switch n.ClassType {
|
||||
case "CLIPTextEncode":
|
||||
clipNodes = append(clipNodes, n)
|
||||
case "KSampler", "KSamplerAdvanced", "RandomNoise":
|
||||
samplerNodes = append(samplerNodes, n)
|
||||
case "EmptyLatentImage":
|
||||
latentNodes = append(latentNodes, n)
|
||||
case "LoadImage":
|
||||
loadImageNodes = append(loadImageNodes, n)
|
||||
}
|
||||
}
|
||||
var out []BindingSuggestion
|
||||
for _, d := range defs {
|
||||
s := BindingSuggestion{Param: d.Name, Type: d.Type, Confidence: "low"}
|
||||
switch d.Name {
|
||||
case "prompt":
|
||||
s = suggestCLIP(clipNodes, workflow, false)
|
||||
case "negative_prompt":
|
||||
s = suggestCLIP(clipNodes, workflow, true)
|
||||
case "seed":
|
||||
s = suggestSamplerField(samplerNodes, workflow, "seed", d.Type)
|
||||
if s.Node == "" {
|
||||
s = suggestSamplerField(samplerNodes, workflow, "noise_seed", d.Type)
|
||||
}
|
||||
case "steps":
|
||||
s = suggestSamplerField(samplerNodes, workflow, "steps", d.Type)
|
||||
case "cfg_scale":
|
||||
s = suggestSamplerField(samplerNodes, workflow, "cfg", d.Type)
|
||||
case "denoise":
|
||||
s = suggestSamplerField(samplerNodes, workflow, "denoise", d.Type)
|
||||
case "width":
|
||||
s = suggestLatentField(latentNodes, workflow, "width", d.Type)
|
||||
case "height":
|
||||
s = suggestLatentField(latentNodes, workflow, "height", d.Type)
|
||||
case "image":
|
||||
s = suggestLoadImage(loadImageNodes, workflow, d.Type)
|
||||
default:
|
||||
s.Param = d.Name
|
||||
s.Type = d.Type
|
||||
}
|
||||
if s.Param == "" {
|
||||
s.Param = d.Name
|
||||
s.Type = d.Type
|
||||
}
|
||||
if s.Node != "" {
|
||||
s.Confidence = "high"
|
||||
}
|
||||
out = append(out, s)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func suggestCLIP(nodes []WorkflowNode, workflow map[string]any, negative bool) BindingSuggestion {
|
||||
s := BindingSuggestion{Param: "prompt", Type: ParamString, Confidence: "low"}
|
||||
if negative {
|
||||
s.Param = "negative_prompt"
|
||||
}
|
||||
var candidates []BindingCandidate
|
||||
var picked *WorkflowNode
|
||||
for i := range nodes {
|
||||
n := nodes[i]
|
||||
titleLower := strings.ToLower(n.Title)
|
||||
isNeg := strings.Contains(titleLower, "negative") || strings.Contains(titleLower, "neg")
|
||||
if !hasField(n, "text") {
|
||||
continue
|
||||
}
|
||||
candidates = append(candidates, BindingCandidate{Node: n.ID, Field: "text"})
|
||||
if negative && isNeg && picked == nil {
|
||||
picked = &nodes[i]
|
||||
}
|
||||
if !negative && !isNeg && picked == nil {
|
||||
picked = &nodes[i]
|
||||
}
|
||||
}
|
||||
if picked == nil && len(candidates) > 0 {
|
||||
idx := 0
|
||||
if negative && len(candidates) > 1 {
|
||||
idx = 1
|
||||
}
|
||||
for i := range nodes {
|
||||
if nodes[i].ID == candidates[idx].Node {
|
||||
picked = &nodes[i]
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
s.Candidates = candidates
|
||||
if picked != nil {
|
||||
s.Node = picked.ID
|
||||
s.Field = "text"
|
||||
s.DefaultFromWorkflow = nodeFieldValue(workflow, picked.ID, "text")
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func suggestSamplerField(nodes []WorkflowNode, workflow map[string]any, field string, pt ParamType) BindingSuggestion {
|
||||
paramName := field
|
||||
if field == "cfg" {
|
||||
paramName = "cfg_scale"
|
||||
}
|
||||
s := BindingSuggestion{Param: paramName, Type: pt, Confidence: "low"}
|
||||
var candidates []BindingCandidate
|
||||
for _, n := range nodes {
|
||||
if hasField(n, field) {
|
||||
candidates = append(candidates, BindingCandidate{Node: n.ID, Field: field})
|
||||
if s.Node == "" {
|
||||
s.Node = n.ID
|
||||
s.Field = field
|
||||
s.DefaultFromWorkflow = nodeFieldValue(workflow, n.ID, field)
|
||||
}
|
||||
}
|
||||
}
|
||||
s.Candidates = candidates
|
||||
return s
|
||||
}
|
||||
|
||||
func suggestLatentField(nodes []WorkflowNode, workflow map[string]any, field string, pt ParamType) BindingSuggestion {
|
||||
s := BindingSuggestion{Param: field, Type: pt, Confidence: "low"}
|
||||
for _, n := range nodes {
|
||||
if hasField(n, field) {
|
||||
s.Candidates = append(s.Candidates, BindingCandidate{Node: n.ID, Field: field})
|
||||
if s.Node == "" {
|
||||
s.Node = n.ID
|
||||
s.Field = field
|
||||
s.DefaultFromWorkflow = nodeFieldValue(workflow, n.ID, field)
|
||||
}
|
||||
}
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func suggestLoadImage(nodes []WorkflowNode, workflow map[string]any, pt ParamType) BindingSuggestion {
|
||||
s := BindingSuggestion{Param: "image", Type: pt, Confidence: "low"}
|
||||
for _, n := range nodes {
|
||||
if hasField(n, "image") {
|
||||
s.Candidates = append(s.Candidates, BindingCandidate{Node: n.ID, Field: "image"})
|
||||
if s.Node == "" {
|
||||
s.Node = n.ID
|
||||
s.Field = "image"
|
||||
s.DefaultFromWorkflow = nodeFieldValue(workflow, n.ID, "image")
|
||||
}
|
||||
}
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func hasField(n WorkflowNode, field string) bool {
|
||||
for _, f := range n.InjectableFields {
|
||||
if f == field {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func nodeFieldValue(workflow map[string]any, nodeID, field string) any {
|
||||
node, ok := workflow[nodeID].(map[string]any)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
inputs, ok := node["inputs"].(map[string]any)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
return inputs[field]
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
package pricing
|
||||
|
||||
import "strings"
|
||||
|
||||
// 价格单位：美元 / 1M tokens
|
||||
type ModelPrice struct {
|
||||
InputPer1M float64
|
||||
OutputPer1M float64
|
||||
}
|
||||
|
||||
var catalog = map[string]ModelPrice{
|
||||
"gpt-4o": {InputPer1M: 2.5, OutputPer1M: 10},
|
||||
"gpt-4o-mini": {InputPer1M: 0.15, OutputPer1M: 0.6},
|
||||
"gpt-4-turbo": {InputPer1M: 10, OutputPer1M: 30},
|
||||
"gpt-3.5-turbo": {InputPer1M: 0.5, OutputPer1M: 1.5},
|
||||
"claude-3-5-sonnet": {InputPer1M: 3, OutputPer1M: 15},
|
||||
"claude-3-5-haiku": {InputPer1M: 0.8, OutputPer1M: 4},
|
||||
"claude-3-opus": {InputPer1M: 15, OutputPer1M: 75},
|
||||
"text-embedding-3-small": {InputPer1M: 0.02, OutputPer1M: 0},
|
||||
"text-embedding-3-large": {InputPer1M: 0.13, OutputPer1M: 0},
|
||||
"text-embedding-ada-002": {InputPer1M: 0.1, OutputPer1M: 0},
|
||||
"bge-reranker-v2-m3": {InputPer1M: 0.1, OutputPer1M: 0},
|
||||
"rerank-english-v3.0": {InputPer1M: 2.0, OutputPer1M: 0},
|
||||
"rerank-multilingual-v3.0": {InputPer1M: 2.0, OutputPer1M: 0},
|
||||
}
|
||||
|
||||
func Estimate(model string, tokensIn, tokensOut int, endpoint string) float64 {
|
||||
if strings.Contains(endpoint, "embeddings") || strings.Contains(endpoint, "rerank") {
|
||||
tokensOut = 0
|
||||
}
|
||||
price, ok := lookup(model)
|
||||
if !ok {
|
||||
// fallback: average small model pricing
|
||||
price = ModelPrice{InputPer1M: 0.5, OutputPer1M: 1.5}
|
||||
}
|
||||
inCost := float64(tokensIn) / 1_000_000 * price.InputPer1M
|
||||
outCost := float64(tokensOut) / 1_000_000 * price.OutputPer1M
|
||||
return inCost + outCost
|
||||
}
|
||||
|
||||
func lookup(model string) (ModelPrice, bool) {
|
||||
if p, ok := catalog[model]; ok {
|
||||
return p, true
|
||||
}
|
||||
// strip provider prefix
|
||||
if i := strings.LastIndex(model, "/"); i >= 0 {
|
||||
if p, ok := catalog[model[i+1:]]; ok {
|
||||
return p, true
|
||||
}
|
||||
}
|
||||
// prefix match
|
||||
for k, v := range catalog {
|
||||
if strings.HasPrefix(model, k) {
|
||||
return v, true
|
||||
}
|
||||
}
|
||||
return ModelPrice{}, false
|
||||
}
|
||||
|
||||
func ListCatalog() map[string]ModelPrice {
|
||||
out := make(map[string]ModelPrice, len(catalog))
|
||||
for k, v := range catalog {
|
||||
out[k] = v
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,283 @@
|
||||
package anthropic
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/provider"
|
||||
)
|
||||
|
||||
type Client struct {
|
||||
http *provider.HTTPClientWrapper
|
||||
}
|
||||
|
||||
func New() *Client {
|
||||
return &Client{http: provider.NewHTTPClientWrapper()}
|
||||
}
|
||||
|
||||
func (c *Client) Type() string { return "anthropic" }
|
||||
|
||||
func (c *Client) ListModels(ctx context.Context, apiKey, baseURL string) ([]provider.ModelInfo, error) {
|
||||
return c.ListModelsWithConfig(ctx, apiKey, provider.ProviderConfig{
|
||||
Type: model.ProviderAnthropic,
|
||||
BaseURL: provider.NormalizeBaseURL(baseURL, model.ProviderAnthropic),
|
||||
})
|
||||
}
|
||||
|
||||
func (c *Client) ListModelsWithConfig(ctx context.Context, apiKey string, cfg provider.ProviderConfig) ([]provider.ModelInfo, error) {
|
||||
cfg.Type = model.ProviderAnthropic
|
||||
cfg.BaseURL = provider.NormalizeBaseURL(cfg.BaseURL, model.ProviderAnthropic)
|
||||
url := cfg.ResolveEndpointURL(provider.EndpointListModels)
|
||||
resp, err := provider.ForwardHTTP(ctx, c.http.Client, "GET", url, apiKey, model.ProviderAnthropic, nil, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.StatusCode >= 400 {
|
||||
return nil, fmt.Errorf("anthropic list models: status %d: %s", resp.StatusCode, string(resp.Body))
|
||||
}
|
||||
var out struct {
|
||||
Data []struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"display_name"`
|
||||
} `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Body, &out); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
models := make([]provider.ModelInfo, 0, len(out.Data))
|
||||
for _, m := range out.Data {
|
||||
name := m.Name
|
||||
if name == "" {
|
||||
name = m.ID
|
||||
}
|
||||
models = append(models, provider.ModelInfo{ID: m.ID, DisplayName: name})
|
||||
}
|
||||
return models, nil
|
||||
}
|
||||
|
||||
func (c *Client) ChatCompletion(ctx context.Context, apiKey, baseURL string, req *provider.ChatRequest) (*provider.ChatResponse, error) {
|
||||
cfg := provider.ProviderConfig{
|
||||
Type: model.ProviderAnthropic,
|
||||
BaseURL: provider.NormalizeBaseURL(baseURL, model.ProviderAnthropic),
|
||||
}
|
||||
anthropicReq, err := convertOpenAIToAnthropic(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
body, err := json.Marshal(anthropicReq)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
url := cfg.ResolveEndpointURL(provider.EndpointChatCompletion)
|
||||
resp, err := provider.ForwardHTTP(ctx, c.http.Client, "POST", url, apiKey, model.ProviderAnthropic, body, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.StatusCode >= 400 {
|
||||
return &provider.ChatResponse{StatusCode: resp.StatusCode, Body: resp.Body}, nil
|
||||
}
|
||||
openAIResp, tokensIn, tokensOut := convertAnthropicToOpenAI(resp.Body, req.Model)
|
||||
return &provider.ChatResponse{
|
||||
StatusCode: 200,
|
||||
Body: openAIResp,
|
||||
TokensIn: tokensIn,
|
||||
TokensOut: tokensOut,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (c *Client) ForwardWithConfig(ctx context.Context, apiKey string, cfg provider.ProviderConfig, endpoint provider.EndpointType, body []byte) (*provider.ChatResponse, error) {
|
||||
cfg.Type = model.ProviderAnthropic
|
||||
cfg.BaseURL = provider.NormalizeBaseURL(cfg.BaseURL, model.ProviderAnthropic)
|
||||
if endpoint == provider.EndpointChatCompletion {
|
||||
var payload map[string]any
|
||||
if err := json.Unmarshal(body, &payload); err == nil {
|
||||
modelName, _ := payload["model"].(string)
|
||||
anthropicReq, err := convertOpenAIToAnthropic(&provider.ChatRequest{Model: modelName, Raw: body})
|
||||
if err == nil {
|
||||
body, _ = json.Marshal(anthropicReq)
|
||||
}
|
||||
}
|
||||
}
|
||||
url := cfg.ResolveEndpointURL(endpoint)
|
||||
resp, err := provider.ForwardHTTP(ctx, c.http.Client, "POST", url, apiKey, model.ProviderAnthropic, body, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if endpoint == provider.EndpointChatCompletion && resp.StatusCode < 400 {
|
||||
var payload map[string]any
|
||||
modelName := ""
|
||||
_ = json.Unmarshal(body, &payload)
|
||||
if m, ok := payload["model"].(string); ok {
|
||||
modelName = m
|
||||
}
|
||||
openAIResp, tokensIn, tokensOut := convertAnthropicToOpenAI(resp.Body, modelName)
|
||||
return &provider.ChatResponse{StatusCode: 200, Body: openAIResp, TokensIn: tokensIn, TokensOut: tokensOut}, nil
|
||||
}
|
||||
return &provider.ChatResponse{StatusCode: resp.StatusCode, Body: resp.Body, TokensIn: resp.TokensIn, TokensOut: resp.TokensOut}, nil
|
||||
}
|
||||
|
||||
func (c *Client) HealthCheck(ctx context.Context, apiKey, baseURL string) error {
|
||||
_, err := c.ListModels(ctx, apiKey, baseURL)
|
||||
return err
|
||||
}
|
||||
|
||||
func (c *Client) HealthCheckWithConfig(ctx context.Context, apiKey string, cfg provider.ProviderConfig) error {
|
||||
_, err := c.ListModelsWithConfig(ctx, apiKey, cfg)
|
||||
return err
|
||||
}
|
||||
|
||||
// convertOpenAIToAnthropic and convertAnthropicToOpenAI unchanged below
|
||||
|
||||
func convertOpenAIToAnthropic(req *provider.ChatRequest) (map[string]any, error) {
|
||||
var payload map[string]any
|
||||
raw := req.Raw
|
||||
if len(raw) == 0 {
|
||||
b, _ := json.Marshal(req)
|
||||
raw = b
|
||||
}
|
||||
if err := json.Unmarshal(raw, &payload); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
model, _ := payload["model"].(string)
|
||||
if model == "" {
|
||||
model = req.Model
|
||||
}
|
||||
msgs, _ := payload["messages"].([]any)
|
||||
var system string
|
||||
var anthropicMsgs []map[string]any
|
||||
for _, m := range msgs {
|
||||
msg, _ := m.(map[string]any)
|
||||
role, _ := msg["role"].(string)
|
||||
if role == "system" {
|
||||
system = extractOpenAIContent(msg["content"])
|
||||
continue
|
||||
}
|
||||
if role != "assistant" {
|
||||
role = "user"
|
||||
}
|
||||
content := convertOpenAIContentBlocks(msg["content"])
|
||||
anthropicMsgs = append(anthropicMsgs, map[string]any{"role": role, "content": content})
|
||||
}
|
||||
out := map[string]any{
|
||||
"model": model,
|
||||
"max_tokens": 4096,
|
||||
"messages": anthropicMsgs,
|
||||
}
|
||||
if system != "" {
|
||||
out["system"] = system
|
||||
}
|
||||
if maxTok, ok := payload["max_tokens"].(float64); ok {
|
||||
out["max_tokens"] = int(maxTok)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func extractOpenAIContent(content any) string {
|
||||
switch v := content.(type) {
|
||||
case string:
|
||||
return v
|
||||
case []any:
|
||||
var parts []string
|
||||
for _, p := range v {
|
||||
part, ok := p.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if text, ok := part["text"].(string); ok {
|
||||
parts = append(parts, text)
|
||||
}
|
||||
}
|
||||
return strings.Join(parts, "\n")
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
func convertOpenAIContentBlocks(content any) any {
|
||||
switch v := content.(type) {
|
||||
case string:
|
||||
if v == "" {
|
||||
return ""
|
||||
}
|
||||
return v
|
||||
case []any:
|
||||
blocks := make([]map[string]any, 0, len(v))
|
||||
for _, p := range v {
|
||||
part, ok := p.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
partType, _ := part["type"].(string)
|
||||
switch partType {
|
||||
case "text":
|
||||
if text, ok := part["text"].(string); ok {
|
||||
blocks = append(blocks, map[string]any{"type": "text", "text": text})
|
||||
}
|
||||
case "image_url":
|
||||
if imageURL, ok := part["image_url"].(map[string]any); ok {
|
||||
if url, ok := imageURL["url"].(string); ok && url != "" {
|
||||
blocks = append(blocks, map[string]any{
|
||||
"type": "image",
|
||||
"source": map[string]any{
|
||||
"type": "url",
|
||||
"url": url,
|
||||
},
|
||||
})
|
||||
}
|
||||
}
|
||||
case "image":
|
||||
if source, ok := part["source"].(map[string]any); ok {
|
||||
blocks = append(blocks, map[string]any{"type": "image", "source": source})
|
||||
}
|
||||
default:
|
||||
if text, ok := part["text"].(string); ok {
|
||||
blocks = append(blocks, map[string]any{"type": "text", "text": text})
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(blocks) == 1 {
|
||||
if text, ok := blocks[0]["text"].(string); ok && blocks[0]["type"] == "text" {
|
||||
return text
|
||||
}
|
||||
}
|
||||
if len(blocks) > 0 {
|
||||
return blocks
|
||||
}
|
||||
return ""
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
func convertAnthropicToOpenAI(body []byte, model string) ([]byte, int, int) {
|
||||
var resp struct {
|
||||
Content []struct {
|
||||
Text string `json:"text"`
|
||||
} `json:"content"`
|
||||
Usage struct {
|
||||
InputTokens int `json:"input_tokens"`
|
||||
OutputTokens int `json:"output_tokens"`
|
||||
} `json:"usage"`
|
||||
}
|
||||
_ = json.Unmarshal(body, &resp)
|
||||
text := ""
|
||||
if len(resp.Content) > 0 {
|
||||
text = resp.Content[0].Text
|
||||
}
|
||||
openAI := map[string]any{
|
||||
"id": "chatcmpl-luminary",
|
||||
"object": "chat.completion",
|
||||
"model": model,
|
||||
"choices": []any{map[string]any{"index": 0, "message": map[string]string{"role": "assistant", "content": text}, "finish_reason": "stop"}},
|
||||
"usage": map[string]int{
|
||||
"prompt_tokens": resp.Usage.InputTokens,
|
||||
"completion_tokens": resp.Usage.OutputTokens,
|
||||
"total_tokens": resp.Usage.InputTokens + resp.Usage.OutputTokens,
|
||||
},
|
||||
}
|
||||
b, _ := json.Marshal(openAI)
|
||||
return b, resp.Usage.InputTokens, resp.Usage.OutputTokens
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
package anthropic
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestConvertOpenAIContentBlocks_TextAndImage(t *testing.T) {
|
||||
content := []any{
|
||||
map[string]any{"type": "text", "text": "describe this"},
|
||||
map[string]any{"type": "image_url", "image_url": map[string]any{"url": "https://example.com/a.png"}},
|
||||
}
|
||||
blocks, ok := convertOpenAIContentBlocks(content).([]map[string]any)
|
||||
if !ok || len(blocks) != 2 {
|
||||
t.Fatalf("expected 2 blocks, got %#v", convertOpenAIContentBlocks(content))
|
||||
}
|
||||
if blocks[0]["text"] != "describe this" {
|
||||
t.Fatalf("unexpected text block: %#v", blocks[0])
|
||||
}
|
||||
if blocks[1]["type"] != "image" {
|
||||
t.Fatalf("unexpected image block: %#v", blocks[1])
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractOpenAIContent_String(t *testing.T) {
|
||||
if got := extractOpenAIContent("hello"); got != "hello" {
|
||||
t.Fatalf("got %q", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,290 @@
|
||||
package comfyui
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/gorilla/websocket"
|
||||
)
|
||||
|
||||
type Client struct {
|
||||
BaseURL string
|
||||
HTTPClient *http.Client
|
||||
}
|
||||
|
||||
func New(baseURL string) *Client {
|
||||
return &Client{
|
||||
BaseURL: strings.TrimRight(baseURL, "/"),
|
||||
HTTPClient: &http.Client{Timeout: 120 * time.Second},
|
||||
}
|
||||
}
|
||||
|
||||
type PromptResponse struct {
|
||||
PromptID string `json:"prompt_id"`
|
||||
Number int `json:"number"`
|
||||
}
|
||||
|
||||
func (c *Client) SubmitPrompt(ctx context.Context, prompt map[string]any, clientID string) (*PromptResponse, error) {
|
||||
body, _ := json.Marshal(map[string]any{
|
||||
"prompt": prompt,
|
||||
"client_id": clientID,
|
||||
})
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.BaseURL+"/prompt", bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
resp, err := c.HTTPClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
data, _ := io.ReadAll(resp.Body)
|
||||
if resp.StatusCode >= 400 {
|
||||
return nil, fmt.Errorf("comfyui prompt %d: %s", resp.StatusCode, string(data))
|
||||
}
|
||||
var out PromptResponse
|
||||
if err := json.Unmarshal(data, &out); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &out, nil
|
||||
}
|
||||
|
||||
func (c *Client) Interrupt(ctx context.Context) error {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.BaseURL+"/interrupt", nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
resp, err := c.HTTPClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode >= 400 {
|
||||
data, _ := io.ReadAll(resp.Body)
|
||||
return fmt.Errorf("comfyui interrupt %d: %s", resp.StatusCode, string(data))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type HistoryEntry struct {
|
||||
Outputs map[string]HistoryNodeOutput `json:"outputs"`
|
||||
Status map[string]any `json:"status"`
|
||||
}
|
||||
|
||||
type HistoryNodeOutput struct {
|
||||
Images []OutputImage `json:"images"`
|
||||
}
|
||||
|
||||
type OutputImage struct {
|
||||
Filename string `json:"filename"`
|
||||
Subfolder string `json:"subfolder"`
|
||||
Type string `json:"type"`
|
||||
}
|
||||
|
||||
func (c *Client) GetHistory(ctx context.Context, promptID string) (map[string]HistoryEntry, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, c.BaseURL+"/history/"+promptID, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resp, err := c.HTTPClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
data, _ := io.ReadAll(resp.Body)
|
||||
if resp.StatusCode >= 400 {
|
||||
return nil, fmt.Errorf("comfyui history %d: %s", resp.StatusCode, string(data))
|
||||
}
|
||||
var out map[string]HistoryEntry
|
||||
if err := json.Unmarshal(data, &out); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *Client) DownloadView(ctx context.Context, filename, subfolder, imageType string) ([]byte, string, error) {
|
||||
q := url.Values{}
|
||||
q.Set("filename", filename)
|
||||
if subfolder != "" {
|
||||
q.Set("subfolder", subfolder)
|
||||
}
|
||||
if imageType == "" {
|
||||
imageType = "output"
|
||||
}
|
||||
q.Set("type", imageType)
|
||||
u := c.BaseURL + "/view?" + q.Encode()
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
resp, err := c.HTTPClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
data, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
if resp.StatusCode >= 400 {
|
||||
return nil, "", fmt.Errorf("comfyui view %d", resp.StatusCode)
|
||||
}
|
||||
mime := resp.Header.Get("Content-Type")
|
||||
if mime == "" {
|
||||
mime = "image/png"
|
||||
}
|
||||
return data, mime, nil
|
||||
}
|
||||
|
||||
func (c *Client) UploadImage(ctx context.Context, data []byte, filename string) (string, error) {
|
||||
var buf bytes.Buffer
|
||||
w := multipart.NewWriter(&buf)
|
||||
part, err := w.CreateFormFile("image", filename)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if _, err := part.Write(data); err != nil {
|
||||
return "", err
|
||||
}
|
||||
_ = w.WriteField("overwrite", "true")
|
||||
_ = w.Close()
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.BaseURL+"/upload/image", &buf)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
req.Header.Set("Content-Type", w.FormDataContentType())
|
||||
resp, err := c.HTTPClient.Do(req)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
if resp.StatusCode >= 400 {
|
||||
return "", fmt.Errorf("comfyui upload %d: %s", resp.StatusCode, string(body))
|
||||
}
|
||||
var out struct {
|
||||
Name string `json:"name"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &out); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return out.Name, nil
|
||||
}
|
||||
|
||||
func (c *Client) HealthCheck(ctx context.Context) error {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, c.BaseURL+"/system_stats", nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
resp, err := c.HTTPClient.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode >= 400 {
|
||||
return fmt.Errorf("status %d", resp.StatusCode)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type ProgressEvent struct {
|
||||
Type string
|
||||
Data map[string]any
|
||||
Node string
|
||||
Value float64
|
||||
Max float64
|
||||
PromptID string
|
||||
ErrorText string
|
||||
}
|
||||
|
||||
func (c *Client) wsURL(clientID string) string {
|
||||
u, _ := url.Parse(c.BaseURL)
|
||||
scheme := "ws"
|
||||
if u.Scheme == "https" {
|
||||
scheme = "wss"
|
||||
}
|
||||
return fmt.Sprintf("%s://%s/ws?clientId=%s", scheme, u.Host, url.QueryEscape(clientID))
|
||||
}
|
||||
|
||||
func (c *Client) Watch(ctx context.Context, clientID string, onEvent func(ProgressEvent)) error {
|
||||
conn, _, err := websocket.DefaultDialer.DialContext(ctx, c.wsURL(clientID), nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer conn.Close()
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
defer close(done)
|
||||
for {
|
||||
_, msg, err := conn.ReadMessage()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
var envelope struct {
|
||||
Type string `json:"type"`
|
||||
Data json.RawMessage `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(msg, &envelope); err != nil {
|
||||
continue
|
||||
}
|
||||
ev := ProgressEvent{Type: envelope.Type}
|
||||
var data map[string]any
|
||||
_ = json.Unmarshal(envelope.Data, &data)
|
||||
ev.Data = data
|
||||
if envelope.Type == "progress" {
|
||||
if n, ok := data["node"].(string); ok {
|
||||
ev.Node = n
|
||||
}
|
||||
if v, ok := data["value"].(float64); ok {
|
||||
ev.Value = v
|
||||
}
|
||||
if m, ok := data["max"].(float64); ok {
|
||||
ev.Max = m
|
||||
}
|
||||
}
|
||||
if envelope.Type == "executing" {
|
||||
if n, ok := data["node"].(string); ok {
|
||||
ev.Node = n
|
||||
}
|
||||
if pid, ok := data["prompt_id"].(string); ok {
|
||||
ev.PromptID = pid
|
||||
}
|
||||
}
|
||||
if envelope.Type == "execution_error" {
|
||||
if msgs, ok := data["exception_message"].(string); ok {
|
||||
ev.ErrorText = msgs
|
||||
}
|
||||
}
|
||||
onEvent(ev)
|
||||
}
|
||||
}()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
_ = conn.WriteMessage(websocket.CloseMessage, websocket.FormatCloseMessage(websocket.CloseNormalClosure, ""))
|
||||
<-done
|
||||
return ctx.Err()
|
||||
case <-done:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func CollectOutputImages(history map[string]HistoryEntry, promptID string) []OutputImage {
|
||||
entry, ok := history[promptID]
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
var images []OutputImage
|
||||
for _, out := range entry.Outputs {
|
||||
images = append(images, out.Images...)
|
||||
}
|
||||
return images
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
package custom
|
||||
|
||||
import (
|
||||
"github.com/rose_cat707/luminary/internal/provider/openai"
|
||||
)
|
||||
|
||||
// Custom providers use OpenAI-compatible API at a custom base URL.
|
||||
type Client struct {
|
||||
*openai.Client
|
||||
}
|
||||
|
||||
func New() *Client {
|
||||
return &Client{Client: openai.New()}
|
||||
}
|
||||
|
||||
func (c *Client) Type() string { return "custom" }
|
||||
@@ -0,0 +1,176 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
)
|
||||
|
||||
// EndpointType 接口类型，对应入口 /v1/* 与上游 path 映射
|
||||
type EndpointType string
|
||||
|
||||
const (
|
||||
EndpointListModels EndpointType = "list_models"
|
||||
EndpointTextCompletion EndpointType = "text_completion"
|
||||
EndpointChatCompletion EndpointType = "chat_completion"
|
||||
EndpointResponses EndpointType = "responses"
|
||||
EndpointEmbeddings EndpointType = "embeddings"
|
||||
EndpointRerank EndpointType = "rerank"
|
||||
EndpointSpeech EndpointType = "speech"
|
||||
EndpointTranscription EndpointType = "transcription"
|
||||
EndpointImageGeneration EndpointType = "image_generation"
|
||||
)
|
||||
|
||||
type EndpointMeta struct {
|
||||
Key EndpointType `json:"key"`
|
||||
Label string `json:"label"`
|
||||
Method string `json:"method"`
|
||||
IngressPath string `json:"ingress_path"`
|
||||
Categories []model.ProviderCategory `json:"categories"`
|
||||
}
|
||||
|
||||
var AllEndpoints = []EndpointMeta{
|
||||
{Key: EndpointListModels, Label: "List Models", Method: "GET", IngressPath: "/v1/models", Categories: []model.ProviderCategory{model.CategoryLLM, model.CategoryEmbedding, model.CategoryRerank}},
|
||||
{Key: EndpointTextCompletion, Label: "Text Completion", Method: "POST", IngressPath: "/v1/completions", Categories: []model.ProviderCategory{model.CategoryLLM}},
|
||||
{Key: EndpointChatCompletion, Label: "Chat Completion", Method: "POST", IngressPath: "/v1/chat/completions", Categories: []model.ProviderCategory{model.CategoryLLM}},
|
||||
{Key: EndpointResponses, Label: "Responses", Method: "POST", IngressPath: "/v1/responses", Categories: []model.ProviderCategory{model.CategoryLLM}},
|
||||
{Key: EndpointEmbeddings, Label: "Embeddings", Method: "POST", IngressPath: "/v1/embeddings", Categories: []model.ProviderCategory{model.CategoryEmbedding}},
|
||||
{Key: EndpointRerank, Label: "Rerank", Method: "POST", IngressPath: "/v1/rerank", Categories: []model.ProviderCategory{model.CategoryRerank}},
|
||||
{Key: EndpointSpeech, Label: "Speech", Method: "POST", IngressPath: "/v1/audio/speech", Categories: []model.ProviderCategory{model.CategorySpeech}},
|
||||
{Key: EndpointTranscription, Label: "Transcription", Method: "POST", IngressPath: "/v1/audio/transcriptions", Categories: []model.ProviderCategory{model.CategoryAudio}},
|
||||
{Key: EndpointImageGeneration, Label: "Image Generation", Method: "POST", IngressPath: "/v1/images/generations", Categories: []model.ProviderCategory{model.CategoryImage}},
|
||||
}
|
||||
|
||||
// DefaultPaths 各 Provider 类型的默认上游 path（相对 base_url）
|
||||
var DefaultPaths = map[model.ProviderType]map[EndpointType]string{
|
||||
model.ProviderOpenAI: {
|
||||
EndpointListModels: "/models",
|
||||
EndpointTextCompletion: "/completions",
|
||||
EndpointChatCompletion: "/chat/completions",
|
||||
EndpointResponses: "/responses",
|
||||
EndpointEmbeddings: "/embeddings",
|
||||
EndpointRerank: "/rerank",
|
||||
EndpointSpeech: "/audio/speech",
|
||||
EndpointTranscription: "/audio/transcriptions",
|
||||
EndpointImageGeneration: "/images/generations",
|
||||
},
|
||||
model.ProviderAnthropic: {
|
||||
EndpointListModels: "/models",
|
||||
EndpointChatCompletion: "/messages",
|
||||
},
|
||||
model.ProviderCustom: {
|
||||
EndpointListModels: "/models",
|
||||
EndpointTextCompletion: "/completions",
|
||||
EndpointChatCompletion: "/chat/completions",
|
||||
EndpointResponses: "/responses",
|
||||
EndpointEmbeddings: "/embeddings",
|
||||
EndpointRerank: "/rerank",
|
||||
EndpointSpeech: "/audio/speech",
|
||||
EndpointTranscription: "/audio/transcriptions",
|
||||
EndpointImageGeneration: "/images/generations",
|
||||
},
|
||||
model.ProviderAzureOpenAI: {
|
||||
EndpointListModels: "/models",
|
||||
EndpointChatCompletion: "/chat/completions",
|
||||
EndpointResponses: "/responses",
|
||||
EndpointEmbeddings: "/embeddings",
|
||||
EndpointRerank: "/rerank",
|
||||
},
|
||||
model.ProviderOpenRouter: {
|
||||
EndpointListModels: "/models",
|
||||
EndpointChatCompletion: "/chat/completions",
|
||||
EndpointResponses: "/responses",
|
||||
EndpointEmbeddings: "/embeddings",
|
||||
EndpointRerank: "/rerank",
|
||||
},
|
||||
model.ProviderOllama: {
|
||||
EndpointListModels: "/models",
|
||||
EndpointChatCompletion: "/chat/completions",
|
||||
EndpointEmbeddings: "/embeddings",
|
||||
EndpointRerank: "/rerank",
|
||||
},
|
||||
model.ProviderComfyUI: {},
|
||||
}
|
||||
|
||||
func DefaultPath(providerType model.ProviderType, endpoint EndpointType) string {
|
||||
if m, ok := DefaultPaths[providerType]; ok {
|
||||
if p, ok := m[endpoint]; ok {
|
||||
return p
|
||||
}
|
||||
}
|
||||
if p, ok := DefaultPaths[model.ProviderOpenAI][endpoint]; ok {
|
||||
return p
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// DefaultEndpointsForCategory 创建出口时默认启用的 endpoint keys
|
||||
func DefaultEndpointsForCategory(cat model.ProviderCategory) []string {
|
||||
switch cat {
|
||||
case model.CategoryEmbedding:
|
||||
return []string{string(EndpointListModels), string(EndpointEmbeddings)}
|
||||
case model.CategoryRerank:
|
||||
return []string{string(EndpointListModels), string(EndpointRerank)}
|
||||
case model.CategoryImage:
|
||||
return []string{}
|
||||
default:
|
||||
return []string{string(EndpointListModels), string(EndpointChatCompletion)}
|
||||
}
|
||||
}
|
||||
|
||||
func EndpointsForCategory(cat model.ProviderCategory) []EndpointMeta {
|
||||
var out []EndpointMeta
|
||||
for _, e := range AllEndpoints {
|
||||
for _, c := range e.Categories {
|
||||
if c == cat {
|
||||
out = append(out, e)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// ResolveURL 解析最终请求 URL：override 可为相对 path 或完整 URL
|
||||
func ResolveURL(baseURL, override, defaultPath string) string {
|
||||
if override != "" {
|
||||
if strings.HasPrefix(override, "http://") || strings.HasPrefix(override, "https://") {
|
||||
return override
|
||||
}
|
||||
return joinURL(baseURL, override)
|
||||
}
|
||||
return joinURL(baseURL, defaultPath)
|
||||
}
|
||||
|
||||
func joinURL(base, path string) string {
|
||||
base = strings.TrimRight(base, "/")
|
||||
if path == "" {
|
||||
return base
|
||||
}
|
||||
if !strings.HasPrefix(path, "/") {
|
||||
path = "/" + path
|
||||
}
|
||||
if base == "" {
|
||||
return path
|
||||
}
|
||||
return base + path
|
||||
}
|
||||
|
||||
func CategoryLabel(cat model.ProviderCategory) string {
|
||||
switch cat {
|
||||
case model.CategoryLLM:
|
||||
return "语言模型"
|
||||
case model.CategoryEmbedding:
|
||||
return "向量嵌入"
|
||||
case model.CategoryRerank:
|
||||
return "重排序"
|
||||
case model.CategorySpeech:
|
||||
return "语音合成"
|
||||
case model.CategoryImage:
|
||||
return "图像生成"
|
||||
case model.CategoryAudio:
|
||||
return "音频转写"
|
||||
default:
|
||||
return string(cat)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,198 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
)
|
||||
|
||||
type ForwardRequest struct {
|
||||
Endpoint EndpointType
|
||||
Method string
|
||||
Body []byte
|
||||
Headers map[string]string
|
||||
}
|
||||
|
||||
type ForwardResponse struct {
|
||||
StatusCode int
|
||||
Body []byte
|
||||
TokensIn int
|
||||
TokensOut int
|
||||
}
|
||||
|
||||
type ProviderConfig struct {
|
||||
Type model.ProviderType
|
||||
Category model.ProviderCategory
|
||||
BaseURL string
|
||||
EndpointPaths map[string]string // endpoint key -> path override
|
||||
}
|
||||
|
||||
func ParseEndpointPaths(jsonStr string) map[string]string {
|
||||
out := map[string]string{}
|
||||
if jsonStr == "" || jsonStr == "{}" {
|
||||
return out
|
||||
}
|
||||
_ = json.Unmarshal([]byte(jsonStr), &out)
|
||||
return out
|
||||
}
|
||||
|
||||
func ParseAllowedEndpoints(jsonStr string, category model.ProviderCategory) []string {
|
||||
var out []string
|
||||
if jsonStr == "" {
|
||||
return DefaultEndpointsForCategory(category)
|
||||
}
|
||||
_ = json.Unmarshal([]byte(jsonStr), &out)
|
||||
if len(out) == 0 {
|
||||
return DefaultEndpointsForCategory(category)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (cfg *ProviderConfig) ResolveEndpointURL(endpoint EndpointType) string {
|
||||
override := cfg.EndpointPaths[string(endpoint)]
|
||||
defaultPath := DefaultPath(cfg.Type, endpoint)
|
||||
return ResolveURL(cfg.BaseURL, override, defaultPath)
|
||||
}
|
||||
|
||||
func ForwardHTTP(ctx context.Context, client *http.Client, method, url string, apiKey string, providerType model.ProviderType, body []byte, extraHeaders map[string]string) (*ForwardResponse, error) {
|
||||
var reader io.Reader
|
||||
if len(body) > 0 {
|
||||
reader = bytes.NewReader(body)
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, method, url, reader)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
setAuthHeaders(req, apiKey, providerType)
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
for k, v := range extraHeaders {
|
||||
req.Header.Set(k, v)
|
||||
}
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
respBody, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tokensIn, tokensOut := parseUsageJSON(respBody)
|
||||
return &ForwardResponse{
|
||||
StatusCode: resp.StatusCode,
|
||||
Body: respBody,
|
||||
TokensIn: tokensIn,
|
||||
TokensOut: tokensOut,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func setAuthHeaders(req *http.Request, apiKey string, providerType model.ProviderType) {
|
||||
switch providerType {
|
||||
case model.ProviderAnthropic:
|
||||
req.Header.Set("x-api-key", apiKey)
|
||||
req.Header.Set("anthropic-version", "2023-06-01")
|
||||
case model.ProviderAzureOpenAI:
|
||||
if apiKey != "" {
|
||||
req.Header.Set("api-key", apiKey)
|
||||
}
|
||||
case model.ProviderOllama:
|
||||
if apiKey != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+apiKey)
|
||||
}
|
||||
default:
|
||||
if apiKey != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+apiKey)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func parseUsageJSON(body []byte) (int, int) {
|
||||
var u struct {
|
||||
Usage struct {
|
||||
PromptTokens int `json:"prompt_tokens"`
|
||||
CompletionTokens int `json:"completion_tokens"`
|
||||
InputTokens int `json:"input_tokens"`
|
||||
OutputTokens int `json:"output_tokens"`
|
||||
} `json:"usage"`
|
||||
}
|
||||
_ = json.Unmarshal(body, &u)
|
||||
in := u.Usage.PromptTokens
|
||||
out := u.Usage.CompletionTokens
|
||||
if in == 0 {
|
||||
in = u.Usage.InputTokens
|
||||
}
|
||||
if out == 0 {
|
||||
out = u.Usage.OutputTokens
|
||||
}
|
||||
return in, out
|
||||
}
|
||||
|
||||
func DefaultBaseURL(providerType model.ProviderType) string {
|
||||
switch providerType {
|
||||
case model.ProviderAnthropic:
|
||||
return "https://api.anthropic.com/v1"
|
||||
case model.ProviderAzureOpenAI:
|
||||
return "" // must be configured per deployment
|
||||
case model.ProviderOpenRouter:
|
||||
return "https://openrouter.ai/api/v1"
|
||||
case model.ProviderOllama:
|
||||
return "http://127.0.0.1:11434/v1"
|
||||
default:
|
||||
return "https://api.openai.com/v1"
|
||||
}
|
||||
}
|
||||
|
||||
func IsOpenAICompatible(t model.ProviderType) bool {
|
||||
switch t {
|
||||
case model.ProviderOpenAI, model.ProviderCustom, model.ProviderAzureOpenAI, model.ProviderOpenRouter, model.ProviderOllama:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
func NormalizeBaseURL(baseURL string, providerType model.ProviderType) string {
|
||||
if baseURL == "" {
|
||||
return DefaultBaseURL(providerType)
|
||||
}
|
||||
return baseURL
|
||||
}
|
||||
|
||||
func IsEndpointAllowed(allowed []string, endpoint EndpointType) bool {
|
||||
for _, a := range allowed {
|
||||
if a == string(endpoint) || a == "*" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func NewHTTPClient() *http.Client {
|
||||
return &http.Client{Timeout: 120 * time.Second}
|
||||
}
|
||||
|
||||
func ValidateCategoryEndpoints(category model.ProviderCategory, allowed []string) error {
|
||||
if !model.IsCategorySupported(category) {
|
||||
return fmt.Errorf("category %s is not supported yet", category)
|
||||
}
|
||||
valid := EndpointsForCategory(category)
|
||||
validSet := map[string]bool{}
|
||||
for _, e := range valid {
|
||||
validSet[string(e.Key)] = true
|
||||
}
|
||||
for _, a := range allowed {
|
||||
if a == "*" {
|
||||
continue
|
||||
}
|
||||
if !validSet[a] {
|
||||
return fmt.Errorf("endpoint %s is not available for category %s", a, category)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
package provider
|
||||
|
||||
import "net/http"
|
||||
|
||||
type HTTPClientWrapper struct {
|
||||
Client *http.Client
|
||||
}
|
||||
|
||||
func NewHTTPClientWrapper() *HTTPClientWrapper {
|
||||
return &HTTPClientWrapper{Client: NewHTTPClient()}
|
||||
}
|
||||
@@ -0,0 +1,94 @@
|
||||
package openai
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/provider"
|
||||
)
|
||||
|
||||
type Client struct {
|
||||
http *provider.HTTPClientWrapper
|
||||
}
|
||||
|
||||
func New() *Client {
|
||||
return &Client{http: provider.NewHTTPClientWrapper()}
|
||||
}
|
||||
|
||||
func (c *Client) Type() string { return "openai" }
|
||||
|
||||
func (c *Client) cfg(baseURL string) provider.ProviderConfig {
|
||||
return provider.ProviderConfig{
|
||||
Type: model.ProviderOpenAI,
|
||||
Category: model.CategoryLLM,
|
||||
BaseURL: provider.NormalizeBaseURL(baseURL, model.ProviderOpenAI),
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Client) ListModels(ctx context.Context, apiKey, baseURL string) ([]provider.ModelInfo, error) {
|
||||
return c.ListModelsWithConfig(ctx, apiKey, provider.ProviderConfig{
|
||||
Type: model.ProviderOpenAI,
|
||||
BaseURL: provider.NormalizeBaseURL(baseURL, model.ProviderOpenAI),
|
||||
})
|
||||
}
|
||||
|
||||
func (c *Client) ListModelsWithConfig(ctx context.Context, apiKey string, cfg provider.ProviderConfig) ([]provider.ModelInfo, error) {
|
||||
cfg.Type = model.ProviderOpenAI
|
||||
cfg.BaseURL = provider.NormalizeBaseURL(cfg.BaseURL, model.ProviderOpenAI)
|
||||
url := cfg.ResolveEndpointURL(provider.EndpointListModels)
|
||||
resp, err := provider.ForwardHTTP(ctx, c.http.Client, "GET", url, apiKey, cfg.Type, nil, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.StatusCode >= 400 {
|
||||
return nil, fmt.Errorf("list models: %s", string(resp.Body))
|
||||
}
|
||||
var out struct {
|
||||
Data []struct {
|
||||
ID string `json:"id"`
|
||||
} `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(resp.Body, &out); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
models := make([]provider.ModelInfo, 0, len(out.Data))
|
||||
for _, m := range out.Data {
|
||||
models = append(models, provider.ModelInfo{ID: m.ID, DisplayName: m.ID})
|
||||
}
|
||||
return models, nil
|
||||
}
|
||||
|
||||
func (c *Client) ChatCompletion(ctx context.Context, apiKey, baseURL string, req *provider.ChatRequest) (*provider.ChatResponse, error) {
|
||||
return c.ForwardWithConfig(ctx, apiKey, provider.ProviderConfig{
|
||||
Type: model.ProviderOpenAI,
|
||||
BaseURL: provider.NormalizeBaseURL(baseURL, model.ProviderOpenAI),
|
||||
}, provider.EndpointChatCompletion, req.Raw)
|
||||
}
|
||||
|
||||
func (c *Client) ForwardWithConfig(ctx context.Context, apiKey string, cfg provider.ProviderConfig, endpoint provider.EndpointType, body []byte) (*provider.ChatResponse, error) {
|
||||
cfg.Type = model.ProviderOpenAI
|
||||
cfg.BaseURL = provider.NormalizeBaseURL(cfg.BaseURL, model.ProviderOpenAI)
|
||||
url := cfg.ResolveEndpointURL(endpoint)
|
||||
resp, err := provider.ForwardHTTP(ctx, c.http.Client, "POST", url, apiKey, cfg.Type, body, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &provider.ChatResponse{
|
||||
StatusCode: resp.StatusCode,
|
||||
Body: resp.Body,
|
||||
TokensIn: resp.TokensIn,
|
||||
TokensOut: resp.TokensOut,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (c *Client) HealthCheck(ctx context.Context, apiKey, baseURL string) error {
|
||||
_, err := c.ListModels(ctx, apiKey, baseURL)
|
||||
return err
|
||||
}
|
||||
|
||||
func (c *Client) HealthCheckWithConfig(ctx context.Context, apiKey string, cfg provider.ProviderConfig) error {
|
||||
_, err := c.ListModelsWithConfig(ctx, apiKey, cfg)
|
||||
return err
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
)
|
||||
|
||||
type ModelInfo struct {
|
||||
ID string `json:"id"`
|
||||
DisplayName string `json:"display_name,omitempty"`
|
||||
}
|
||||
|
||||
type ChatRequest struct {
|
||||
Model string `json:"model"`
|
||||
Messages json.RawMessage `json:"messages"`
|
||||
Stream bool `json:"stream"`
|
||||
Raw json.RawMessage `json:"-"`
|
||||
}
|
||||
|
||||
type ChatResponse struct {
|
||||
StatusCode int
|
||||
Body []byte
|
||||
Headers map[string]string
|
||||
TokensIn int
|
||||
TokensOut int
|
||||
}
|
||||
|
||||
type Client interface {
|
||||
Type() string
|
||||
ListModels(ctx context.Context, apiKey, baseURL string) ([]ModelInfo, error)
|
||||
ChatCompletion(ctx context.Context, apiKey, baseURL string, req *ChatRequest) (*ChatResponse, error)
|
||||
HealthCheck(ctx context.Context, apiKey, baseURL string) error
|
||||
}
|
||||
@@ -0,0 +1,640 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// CanAdaptResponses reports whether responses ingress can be served via chat completions.
|
||||
func CanAdaptResponses(allowsResponses, allowsChat bool) bool {
|
||||
return allowsChat && !allowsResponses
|
||||
}
|
||||
|
||||
// ResponsesToChatCompletion converts an OpenAI Responses API request to chat completions format.
|
||||
func ResponsesToChatCompletion(body []byte) ([]byte, error) {
|
||||
var req map[string]json.RawMessage
|
||||
if err := json.Unmarshal(body, &req); err != nil {
|
||||
return nil, fmt.Errorf("invalid json: %w", err)
|
||||
}
|
||||
|
||||
out := make(map[string]any)
|
||||
if raw, ok := req["model"]; ok {
|
||||
var model string
|
||||
if err := json.Unmarshal(raw, &model); err != nil || model == "" {
|
||||
return nil, fmt.Errorf("model is required")
|
||||
}
|
||||
out["model"] = model
|
||||
}
|
||||
|
||||
messages := make([]map[string]any, 0, 4)
|
||||
if raw, ok := req["instructions"]; ok {
|
||||
var instructions string
|
||||
if err := json.Unmarshal(raw, &instructions); err == nil && instructions != "" {
|
||||
messages = append(messages, map[string]any{"role": "system", "content": instructions})
|
||||
} else {
|
||||
var instructionParts []json.RawMessage
|
||||
if err := json.Unmarshal(raw, &instructionParts); err == nil {
|
||||
if text := joinContentPartTexts(instructionParts); text != "" {
|
||||
messages = append(messages, map[string]any{"role": "system", "content": text})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if raw, ok := req["input"]; ok {
|
||||
inputMsgs, err := parseResponsesInput(raw)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
messages = append(messages, inputMsgs...)
|
||||
}
|
||||
if len(messages) == 0 {
|
||||
return nil, fmt.Errorf("input is required")
|
||||
}
|
||||
out["messages"] = messages
|
||||
|
||||
copyField(req, out, "stream")
|
||||
copyField(req, out, "temperature")
|
||||
copyField(req, out, "top_p")
|
||||
copyField(req, out, "user")
|
||||
copyField(req, out, "stop")
|
||||
copyField(req, out, "tools")
|
||||
copyField(req, out, "tool_choice")
|
||||
copyField(req, out, "parallel_tool_calls")
|
||||
copyField(req, out, "seed")
|
||||
copyField(req, out, "n")
|
||||
copyField(req, out, "presence_penalty")
|
||||
copyField(req, out, "frequency_penalty")
|
||||
copyField(req, out, "logit_bias")
|
||||
copyField(req, out, "response_format")
|
||||
|
||||
if raw, ok := req["max_output_tokens"]; ok {
|
||||
var v int
|
||||
if err := json.Unmarshal(raw, &v); err == nil && v > 0 {
|
||||
out["max_tokens"] = v
|
||||
}
|
||||
} else {
|
||||
copyField(req, out, "max_tokens")
|
||||
}
|
||||
|
||||
if raw, ok := req["text"]; ok {
|
||||
if rf := responsesTextToResponseFormat(raw); rf != nil {
|
||||
out["response_format"] = rf
|
||||
}
|
||||
}
|
||||
|
||||
return json.Marshal(out)
|
||||
}
|
||||
|
||||
func copyField(src map[string]json.RawMessage, dst map[string]any, key string) {
|
||||
if raw, ok := src[key]; ok && len(raw) > 0 && string(raw) != "null" {
|
||||
var v any
|
||||
if json.Unmarshal(raw, &v) == nil {
|
||||
dst[key] = v
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func responsesTextToResponseFormat(raw json.RawMessage) any {
|
||||
var text struct {
|
||||
Format struct {
|
||||
Type string `json:"type"`
|
||||
} `json:"format"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &text); err != nil || text.Format.Type == "" {
|
||||
return nil
|
||||
}
|
||||
switch text.Format.Type {
|
||||
case "json_object":
|
||||
return map[string]any{"type": "json_object"}
|
||||
case "text":
|
||||
return map[string]any{"type": "text"}
|
||||
default:
|
||||
return map[string]any{"type": text.Format.Type}
|
||||
}
|
||||
}
|
||||
|
||||
func parseResponsesInput(raw json.RawMessage) ([]map[string]any, error) {
|
||||
var asString string
|
||||
if err := json.Unmarshal(raw, &asString); err == nil {
|
||||
if asString == "" {
|
||||
return nil, fmt.Errorf("input is required")
|
||||
}
|
||||
return []map[string]any{{"role": "user", "content": asString}}, nil
|
||||
}
|
||||
|
||||
var asArray []json.RawMessage
|
||||
if err := json.Unmarshal(raw, &asArray); err != nil {
|
||||
return nil, fmt.Errorf("invalid input")
|
||||
}
|
||||
out := make([]map[string]any, 0, len(asArray))
|
||||
for _, item := range asArray {
|
||||
msg, err := parseResponsesInputItem(item)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, msg)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func parseResponsesInputItem(raw json.RawMessage) (map[string]any, error) {
|
||||
var asString string
|
||||
if err := json.Unmarshal(raw, &asString); err == nil {
|
||||
return map[string]any{"role": "user", "content": asString}, nil
|
||||
}
|
||||
|
||||
var item map[string]json.RawMessage
|
||||
if err := json.Unmarshal(raw, &item); err != nil {
|
||||
return nil, fmt.Errorf("invalid input item")
|
||||
}
|
||||
|
||||
itemType := ""
|
||||
if rawType, ok := item["type"]; ok {
|
||||
_ = json.Unmarshal(rawType, &itemType)
|
||||
}
|
||||
|
||||
role := "user"
|
||||
if rawRole, ok := item["role"]; ok {
|
||||
_ = json.Unmarshal(rawRole, &role)
|
||||
}
|
||||
if role == "" {
|
||||
role = "user"
|
||||
}
|
||||
role = normalizeResponsesRole(role)
|
||||
|
||||
switch itemType {
|
||||
case "message":
|
||||
if rawContent, ok := item["content"]; ok {
|
||||
content, err := parseResponsesContent(rawContent)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return map[string]any{"role": role, "content": content}, nil
|
||||
}
|
||||
case "input_text":
|
||||
var text string
|
||||
if rawText, ok := item["text"]; ok {
|
||||
_ = json.Unmarshal(rawText, &text)
|
||||
}
|
||||
if text != "" {
|
||||
return map[string]any{"role": role, "content": text}, nil
|
||||
}
|
||||
case "function_call_output":
|
||||
return parseFunctionCallOutputItem(item, role)
|
||||
}
|
||||
|
||||
if rawContent, ok := item["content"]; ok {
|
||||
content, err := parseResponsesContent(rawContent)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return map[string]any{"role": role, "content": content}, nil
|
||||
}
|
||||
if rawText, ok := item["text"]; ok {
|
||||
var text string
|
||||
if err := json.Unmarshal(rawText, &text); err == nil {
|
||||
return map[string]any{"role": role, "content": text}, nil
|
||||
}
|
||||
}
|
||||
if rawInput, ok := item["input"]; ok {
|
||||
var text string
|
||||
if err := json.Unmarshal(rawInput, &text); err == nil {
|
||||
return map[string]any{"role": role, "content": text}, nil
|
||||
}
|
||||
}
|
||||
return nil, fmt.Errorf("unsupported input item")
|
||||
}
|
||||
|
||||
func normalizeResponsesRole(role string) string {
|
||||
switch role {
|
||||
case "developer":
|
||||
return "system"
|
||||
default:
|
||||
return role
|
||||
}
|
||||
}
|
||||
|
||||
func parseResponsesContent(raw json.RawMessage) (any, error) {
|
||||
var asString string
|
||||
if err := json.Unmarshal(raw, &asString); err == nil {
|
||||
return asString, nil
|
||||
}
|
||||
var parts []json.RawMessage
|
||||
if err := json.Unmarshal(raw, &parts); err != nil {
|
||||
return nil, fmt.Errorf("invalid content")
|
||||
}
|
||||
converted := make([]any, 0, len(parts))
|
||||
for _, part := range parts {
|
||||
if item, ok := convertResponsesContentPart(part); ok {
|
||||
converted = append(converted, item)
|
||||
}
|
||||
}
|
||||
if len(converted) == 0 {
|
||||
return "", nil
|
||||
}
|
||||
if len(converted) == 1 {
|
||||
if text, ok := converted[0].(string); ok {
|
||||
return text, nil
|
||||
}
|
||||
if m, ok := converted[0].(map[string]any); ok {
|
||||
if m["type"] == "text" {
|
||||
if text, ok := m["text"].(string); ok {
|
||||
return text, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return converted, nil
|
||||
}
|
||||
|
||||
func convertResponsesContentPart(raw json.RawMessage) (any, bool) {
|
||||
var asString string
|
||||
if err := json.Unmarshal(raw, &asString); err == nil {
|
||||
return asString, true
|
||||
}
|
||||
var part map[string]json.RawMessage
|
||||
if err := json.Unmarshal(raw, &part); err != nil {
|
||||
return nil, false
|
||||
}
|
||||
typ := ""
|
||||
if rawType, ok := part["type"]; ok {
|
||||
_ = json.Unmarshal(rawType, &typ)
|
||||
}
|
||||
switch typ {
|
||||
case "input_text", "text", "output_text":
|
||||
var text string
|
||||
if rawText, ok := part["text"]; ok {
|
||||
_ = json.Unmarshal(rawText, &text)
|
||||
}
|
||||
if text == "" {
|
||||
return nil, false
|
||||
}
|
||||
return map[string]any{"type": "text", "text": text}, true
|
||||
case "input_image":
|
||||
out := map[string]any{"type": "image_url", "image_url": map[string]any{}}
|
||||
img := out["image_url"].(map[string]any)
|
||||
if rawURL, ok := part["image_url"]; ok {
|
||||
var url string
|
||||
if err := json.Unmarshal(rawURL, &url); err == nil {
|
||||
img["url"] = url
|
||||
} else {
|
||||
var nested map[string]any
|
||||
if err := json.Unmarshal(rawURL, &nested); err == nil {
|
||||
for k, v := range nested {
|
||||
img[k] = v
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if rawDetail, ok := part["detail"]; ok {
|
||||
var detail string
|
||||
if json.Unmarshal(rawDetail, &detail) == nil {
|
||||
img["detail"] = detail
|
||||
}
|
||||
}
|
||||
if img["url"] == nil {
|
||||
return nil, false
|
||||
}
|
||||
return out, true
|
||||
case "image_url":
|
||||
var v any
|
||||
_ = json.Unmarshal(raw, &v)
|
||||
return v, true
|
||||
case "input_file":
|
||||
var fileID, filename string
|
||||
if rawID, ok := part["file_id"]; ok {
|
||||
_ = json.Unmarshal(rawID, &fileID)
|
||||
}
|
||||
if rawName, ok := part["filename"]; ok {
|
||||
_ = json.Unmarshal(rawName, &filename)
|
||||
}
|
||||
label := filename
|
||||
if label == "" {
|
||||
label = fileID
|
||||
}
|
||||
if label == "" {
|
||||
return nil, false
|
||||
}
|
||||
return map[string]any{"type": "text", "text": "[file:" + label + "]"}, true
|
||||
default:
|
||||
return nil, false
|
||||
}
|
||||
}
|
||||
|
||||
func joinContentPartTexts(parts []json.RawMessage) string {
|
||||
var texts []string
|
||||
for _, part := range parts {
|
||||
if item, ok := convertResponsesContentPart(part); ok {
|
||||
if text, ok := item.(string); ok {
|
||||
texts = append(texts, text)
|
||||
} else if m, ok := item.(map[string]any); ok && m["type"] == "text" {
|
||||
if text, ok := m["text"].(string); ok {
|
||||
texts = append(texts, text)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return strings.Join(texts, "\n")
|
||||
}
|
||||
|
||||
func parseFunctionCallOutputItem(item map[string]json.RawMessage, role string) (map[string]any, error) {
|
||||
var output, callID string
|
||||
if rawOutput, ok := item["output"]; ok {
|
||||
_ = json.Unmarshal(rawOutput, &output)
|
||||
}
|
||||
if rawCallID, ok := item["call_id"]; ok {
|
||||
_ = json.Unmarshal(rawCallID, &callID)
|
||||
}
|
||||
if output == "" {
|
||||
return nil, fmt.Errorf("unsupported input item")
|
||||
}
|
||||
msg := map[string]any{"role": "tool", "content": output}
|
||||
if callID != "" {
|
||||
msg["tool_call_id"] = callID
|
||||
}
|
||||
if role != "" && role != "user" {
|
||||
msg["role"] = role
|
||||
}
|
||||
return msg, nil
|
||||
}
|
||||
|
||||
// ChatCompletionToResponses converts a chat completions response to Responses API format.
|
||||
func ChatCompletionToResponses(body []byte) ([]byte, error) {
|
||||
var chat map[string]any
|
||||
if err := json.Unmarshal(body, &chat); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, ok := chat["error"]; ok {
|
||||
return body, nil
|
||||
}
|
||||
|
||||
content := extractChatContent(chat)
|
||||
chatID, _ := chat["id"].(string)
|
||||
model, _ := chat["model"].(string)
|
||||
created := extractCreatedAt(chat)
|
||||
respID := responsesIDFromChatID(chatID)
|
||||
msgID := messageIDFromResponseID(respID)
|
||||
usage := chatUsageToResponses(chat["usage"])
|
||||
|
||||
resp := buildResponseObject(respID, msgID, model, created, content, usage, "completed")
|
||||
return json.Marshal(resp)
|
||||
}
|
||||
|
||||
func extractCreatedAt(chat map[string]any) int64 {
|
||||
switch v := chat["created"].(type) {
|
||||
case float64:
|
||||
return int64(v)
|
||||
case int64:
|
||||
return v
|
||||
case int:
|
||||
return int64(v)
|
||||
default:
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
func responsesIDFromChatID(chatID string) string {
|
||||
if chatID == "" {
|
||||
return "resp_adapted"
|
||||
}
|
||||
if strings.HasPrefix(chatID, "resp_") {
|
||||
return chatID
|
||||
}
|
||||
if strings.HasPrefix(chatID, "chatcmpl-") {
|
||||
return "resp_" + strings.TrimPrefix(chatID, "chatcmpl-")
|
||||
}
|
||||
return "resp_" + chatID
|
||||
}
|
||||
|
||||
func messageIDFromResponseID(respID string) string {
|
||||
if strings.HasPrefix(respID, "resp_") {
|
||||
return "msg_" + strings.TrimPrefix(respID, "resp_")
|
||||
}
|
||||
return "msg_" + respID
|
||||
}
|
||||
|
||||
func buildOutputTextContent(text string) map[string]any {
|
||||
return map[string]any{
|
||||
"type": "output_text",
|
||||
"text": text,
|
||||
"annotations": []any{},
|
||||
}
|
||||
}
|
||||
|
||||
func buildOutputMessage(msgID, content, status string) map[string]any {
|
||||
return map[string]any{
|
||||
"type": "message",
|
||||
"id": msgID,
|
||||
"role": "assistant",
|
||||
"status": status,
|
||||
"content": []map[string]any{buildOutputTextContent(content)},
|
||||
}
|
||||
}
|
||||
|
||||
func buildResponseObject(respID, msgID, model string, created int64, content string, usage map[string]any, status string) map[string]any {
|
||||
msgStatus := status
|
||||
if status == "in_progress" || status == "queued" {
|
||||
msgStatus = "in_progress"
|
||||
}
|
||||
resp := map[string]any{
|
||||
"id": respID,
|
||||
"object": "response",
|
||||
"created_at": created,
|
||||
"status": status,
|
||||
"model": model,
|
||||
"output_text": content,
|
||||
"error": nil,
|
||||
"incomplete_details": nil,
|
||||
"parallel_tool_calls": true,
|
||||
"output": []map[string]any{buildOutputMessage(msgID, content, msgStatus)},
|
||||
}
|
||||
if usage != nil {
|
||||
resp["usage"] = usage
|
||||
}
|
||||
return resp
|
||||
}
|
||||
|
||||
func extractChatContent(chat map[string]any) string {
|
||||
choices, ok := chat["choices"].([]any)
|
||||
if !ok || len(choices) == 0 {
|
||||
return ""
|
||||
}
|
||||
choice, ok := choices[0].(map[string]any)
|
||||
if !ok {
|
||||
return ""
|
||||
}
|
||||
if msg, ok := choice["message"].(map[string]any); ok {
|
||||
if content, ok := msg["content"].(string); ok {
|
||||
return content
|
||||
}
|
||||
}
|
||||
if delta, ok := choice["delta"].(map[string]any); ok {
|
||||
if content, ok := delta["content"].(string); ok {
|
||||
return content
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func chatUsageToResponses(usage any) map[string]any {
|
||||
u, ok := usage.(map[string]any)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
in, hasIn := numberAsInt(u["prompt_tokens"])
|
||||
out, hasOut := numberAsInt(u["completion_tokens"])
|
||||
total, hasTotal := numberAsInt(u["total_tokens"])
|
||||
if !hasIn && !hasOut && !hasTotal {
|
||||
return nil
|
||||
}
|
||||
if !hasTotal && hasIn && hasOut {
|
||||
total = in + out
|
||||
}
|
||||
return map[string]any{
|
||||
"input_tokens": in,
|
||||
"output_tokens": out,
|
||||
"total_tokens": total,
|
||||
"input_tokens_details": map[string]any{
|
||||
"cached_tokens": 0,
|
||||
},
|
||||
"output_tokens_details": map[string]any{
|
||||
"reasoning_tokens": 0,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func numberAsInt(v any) (int, bool) {
|
||||
switch n := v.(type) {
|
||||
case float64:
|
||||
return int(n), true
|
||||
case int:
|
||||
return n, true
|
||||
case int64:
|
||||
return int(n), true
|
||||
default:
|
||||
return 0, false
|
||||
}
|
||||
}
|
||||
|
||||
// AdaptResponsesStream wraps a chat-completions SSE stream as Responses API SSE events.
|
||||
func AdaptResponsesStream(chatBody []byte, upstream io.Reader, w io.Writer, flusher http.Flusher) (*StreamResult, error) {
|
||||
var req map[string]any
|
||||
_ = json.Unmarshal(chatBody, &req)
|
||||
model, _ := req["model"].(string)
|
||||
|
||||
result := &StreamResult{StatusCode: http.StatusOK}
|
||||
respID := "resp_adapted"
|
||||
msgID := "msg_adapted"
|
||||
seq := 0
|
||||
var fullText string
|
||||
var usage map[string]any
|
||||
started := false
|
||||
|
||||
writeEvent := func(eventType string, payload map[string]any) error {
|
||||
payload["type"] = eventType
|
||||
payload["sequence_number"] = seq
|
||||
seq++
|
||||
b, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := fmt.Fprintf(w, "event: %s\ndata: %s\n\n", eventType, b); err != nil {
|
||||
return err
|
||||
}
|
||||
if flusher != nil {
|
||||
flusher.Flush()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
emitCreated := func(status string) error {
|
||||
response := buildResponseObject(respID, msgID, model, 0, "", nil, status)
|
||||
response["output"] = []any{}
|
||||
response["output_text"] = ""
|
||||
return writeEvent("response.created", map[string]any{"response": response})
|
||||
}
|
||||
|
||||
scanner := bufio.NewScanner(upstream)
|
||||
scanner.Buffer(make([]byte, 64*1024), 1024*1024)
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
if !strings.HasPrefix(line, "data: ") {
|
||||
continue
|
||||
}
|
||||
data := strings.TrimPrefix(line, "data: ")
|
||||
if data == "[DONE]" {
|
||||
break
|
||||
}
|
||||
tokensIn, tokensOut := parseStreamChunk([]byte(data))
|
||||
result.TokensIn += tokensIn
|
||||
result.TokensOut += tokensOut
|
||||
|
||||
var chunk map[string]any
|
||||
if err := json.Unmarshal([]byte(data), &chunk); err != nil {
|
||||
continue
|
||||
}
|
||||
if id, ok := chunk["id"].(string); ok && id != "" {
|
||||
respID = responsesIDFromChatID(id)
|
||||
msgID = messageIDFromResponseID(respID)
|
||||
}
|
||||
if m, ok := chunk["model"].(string); ok && m != "" {
|
||||
model = m
|
||||
}
|
||||
if u := chatUsageToResponses(chunk["usage"]); u != nil {
|
||||
usage = u
|
||||
}
|
||||
|
||||
delta := extractChatContent(chunk)
|
||||
if delta == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
if !started {
|
||||
started = true
|
||||
if err := emitCreated("in_progress"); err != nil {
|
||||
return result, err
|
||||
}
|
||||
if err := writeEvent("response.in_progress", map[string]any{
|
||||
"response": buildResponseObject(respID, msgID, model, 0, "", nil, "in_progress"),
|
||||
}); err != nil {
|
||||
return result, err
|
||||
}
|
||||
if err := writeEvent("response.output_item.added", map[string]any{
|
||||
"output_index": 0,
|
||||
"item": map[string]any{
|
||||
"type": "message", "id": msgID, "role": "assistant", "status": "in_progress", "content": []any{},
|
||||
},
|
||||
}); err != nil {
|
||||
return result, err
|
||||
}
|
||||
}
|
||||
|
||||
fullText += delta
|
||||
if err := writeEvent("response.output_text.delta", map[string]any{
|
||||
"item_id": msgID, "output_index": 0, "content_index": 0, "delta": delta, "logprobs": []any{},
|
||||
}); err != nil {
|
||||
return result, err
|
||||
}
|
||||
}
|
||||
if err := scanner.Err(); err != nil {
|
||||
return result, err
|
||||
}
|
||||
|
||||
if started {
|
||||
_ = writeEvent("response.output_text.done", map[string]any{
|
||||
"item_id": msgID, "output_index": 0, "content_index": 0, "text": fullText, "logprobs": []any{},
|
||||
})
|
||||
doneItem := buildOutputMessage(msgID, fullText, "completed")
|
||||
_ = writeEvent("response.output_item.done", map[string]any{
|
||||
"output_index": 0,
|
||||
"item": doneItem,
|
||||
})
|
||||
final := buildResponseObject(respID, msgID, model, 0, fullText, usage, "completed")
|
||||
_ = writeEvent("response.completed", map[string]any{"response": final})
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
@@ -0,0 +1,179 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestResponsesToChatCompletion_StringInput(t *testing.T) {
|
||||
body := []byte(`{"model":"gpt-4o-mini","instructions":"Be brief","input":"Hello"}`)
|
||||
out, err := ResponsesToChatCompletion(body)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var req map[string]any
|
||||
if err := json.Unmarshal(out, &req); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if req["model"] != "gpt-4o-mini" {
|
||||
t.Fatalf("model: %v", req["model"])
|
||||
}
|
||||
msgs := req["messages"].([]any)
|
||||
if len(msgs) != 2 {
|
||||
t.Fatalf("messages len: %d", len(msgs))
|
||||
}
|
||||
sys := msgs[0].(map[string]any)
|
||||
user := msgs[1].(map[string]any)
|
||||
if sys["role"] != "system" || sys["content"] != "Be brief" {
|
||||
t.Fatalf("system: %#v", sys)
|
||||
}
|
||||
if user["role"] != "user" || user["content"] != "Hello" {
|
||||
t.Fatalf("user: %#v", user)
|
||||
}
|
||||
}
|
||||
|
||||
func TestChatCompletionToResponses(t *testing.T) {
|
||||
body := []byte(`{"id":"chatcmpl-abc123","model":"gpt-4o-mini","created":123,"choices":[{"message":{"role":"assistant","content":"Hi"}}],"usage":{"prompt_tokens":3,"completion_tokens":1,"total_tokens":4}}`)
|
||||
out, err := ChatCompletionToResponses(body)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var resp map[string]any
|
||||
if err := json.Unmarshal(out, &resp); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if resp["object"] != "response" {
|
||||
t.Fatalf("object: %v", resp["object"])
|
||||
}
|
||||
if resp["id"] != "resp_abc123" {
|
||||
t.Fatalf("id: %v", resp["id"])
|
||||
}
|
||||
if resp["output_text"] != "Hi" {
|
||||
t.Fatalf("output_text: %v", resp["output_text"])
|
||||
}
|
||||
output := resp["output"].([]any)
|
||||
msg := output[0].(map[string]any)
|
||||
content := msg["content"].([]any)
|
||||
text := content[0].(map[string]any)
|
||||
if text["type"] != "output_text" || text["text"] != "Hi" {
|
||||
t.Fatalf("text: %#v", text)
|
||||
}
|
||||
if _, ok := text["annotations"]; !ok {
|
||||
t.Fatalf("missing annotations: %#v", text)
|
||||
}
|
||||
usage := resp["usage"].(map[string]any)
|
||||
if usage["input_tokens"].(float64) != 3 {
|
||||
t.Fatalf("usage: %#v", usage)
|
||||
}
|
||||
if _, ok := usage["input_tokens_details"]; !ok {
|
||||
t.Fatalf("missing input_tokens_details")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResponsesToChatCompletion_InputTextParts(t *testing.T) {
|
||||
body := []byte(`{
|
||||
"model":"gpt-4o-mini",
|
||||
"input":[{"role":"user","content":[{"type":"input_text","text":"Hello"}]}]
|
||||
}`)
|
||||
out, err := ResponsesToChatCompletion(body)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var req map[string]any
|
||||
if err := json.Unmarshal(out, &req); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
msgs := req["messages"].([]any)
|
||||
user := msgs[0].(map[string]any)
|
||||
if user["content"] != "Hello" {
|
||||
t.Fatalf("content: %#v", user["content"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestResponsesToChatCompletion_InputTextAndImage(t *testing.T) {
|
||||
body := []byte(`{
|
||||
"model":"gpt-4o-mini",
|
||||
"input":[{"role":"user","content":[
|
||||
{"type":"input_text","text":"describe"},
|
||||
{"type":"input_image","image_url":"https://example.com/a.png","detail":"auto"}
|
||||
]}]
|
||||
}`)
|
||||
out, err := ResponsesToChatCompletion(body)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var req map[string]any
|
||||
if err := json.Unmarshal(out, &req); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
msgs := req["messages"].([]any)
|
||||
user := msgs[0].(map[string]any)
|
||||
parts := user["content"].([]any)
|
||||
if len(parts) != 2 {
|
||||
t.Fatalf("parts: %#v", parts)
|
||||
}
|
||||
textPart := parts[0].(map[string]any)
|
||||
if textPart["type"] != "text" || textPart["text"] != "describe" {
|
||||
t.Fatalf("text part: %#v", textPart)
|
||||
}
|
||||
imgPart := parts[1].(map[string]any)
|
||||
if imgPart["type"] != "image_url" {
|
||||
t.Fatalf("image part: %#v", imgPart)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResponsesToChatCompletion_MessageItem(t *testing.T) {
|
||||
body := []byte(`{
|
||||
"model":"gpt-4o-mini",
|
||||
"input":[{"type":"message","role":"user","content":[{"type":"input_text","text":"Hi"}]}]
|
||||
}`)
|
||||
out, err := ResponsesToChatCompletion(body)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(string(out), "input_text") {
|
||||
t.Fatalf("should not contain input_text: %s", out)
|
||||
}
|
||||
var req map[string]any
|
||||
if err := json.Unmarshal(out, &req); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
msgs := req["messages"].([]any)
|
||||
user := msgs[0].(map[string]any)
|
||||
if user["content"] != "Hi" {
|
||||
t.Fatalf("content: %#v", user["content"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdaptResponsesStream(t *testing.T) {
|
||||
upstream := strings.NewReader("data: {\"id\":\"chatcmpl-1\",\"model\":\"gpt-4o-mini\",\"choices\":[{\"delta\":{\"content\":\"Hi\"}}]}\n\ndata: [DONE]\n\n")
|
||||
var out strings.Builder
|
||||
result, err := AdaptResponsesStream([]byte(`{"model":"gpt-4o-mini","stream":true}`), upstream, &out, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if result.StatusCode != 200 {
|
||||
t.Fatalf("status: %d", result.StatusCode)
|
||||
}
|
||||
s := out.String()
|
||||
for _, event := range []string{
|
||||
"response.created",
|
||||
"response.in_progress",
|
||||
"response.output_item.added",
|
||||
"response.output_text.delta",
|
||||
"response.output_text.done",
|
||||
"response.output_item.done",
|
||||
"response.completed",
|
||||
} {
|
||||
if !strings.Contains(s, event) {
|
||||
t.Fatalf("missing event %s: %s", event, s)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(s, `"delta":"Hi"`) {
|
||||
t.Fatalf("missing delta text: %s", s)
|
||||
}
|
||||
if !strings.Contains(s, `"output_text":"Hi"`) {
|
||||
t.Fatalf("missing output_text in completed: %s", s)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
)
|
||||
|
||||
type StreamResult struct {
|
||||
StatusCode int
|
||||
TokensIn int
|
||||
TokensOut int
|
||||
Body []byte // populated for non-2xx responses
|
||||
}
|
||||
|
||||
func ForwardStreamHTTP(ctx context.Context, client *http.Client, method, url string, apiKey string, providerType model.ProviderType, body []byte, w io.Writer, flusher http.Flusher) (*StreamResult, error) {
|
||||
var reader io.Reader
|
||||
if len(body) > 0 {
|
||||
reader = bytes.NewReader(body)
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, method, url, reader)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
setAuthHeaders(req, apiKey, providerType)
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Accept", "text/event-stream")
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode >= 400 {
|
||||
b, _ := io.ReadAll(resp.Body)
|
||||
if flusher != nil {
|
||||
w.Write(b)
|
||||
flusher.Flush()
|
||||
}
|
||||
return &StreamResult{StatusCode: resp.StatusCode, Body: b}, nil
|
||||
}
|
||||
|
||||
result := &StreamResult{StatusCode: resp.StatusCode}
|
||||
scanner := bufio.NewScanner(resp.Body)
|
||||
scanner.Buffer(make([]byte, 64*1024), 1024*1024)
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
if strings.HasPrefix(line, "data: ") {
|
||||
data := strings.TrimPrefix(line, "data: ")
|
||||
if data == "[DONE]" {
|
||||
continue
|
||||
}
|
||||
tokensIn, tokensOut := parseStreamChunk([]byte(data))
|
||||
result.TokensIn += tokensIn
|
||||
result.TokensOut += tokensOut
|
||||
}
|
||||
if _, err := w.Write([]byte(line + "\n")); err != nil {
|
||||
return result, err
|
||||
}
|
||||
if flusher != nil {
|
||||
flusher.Flush()
|
||||
}
|
||||
}
|
||||
return result, scanner.Err()
|
||||
}
|
||||
|
||||
func parseStreamChunk(data []byte) (int, int) {
|
||||
var chunk struct {
|
||||
Usage struct {
|
||||
PromptTokens int `json:"prompt_tokens"`
|
||||
CompletionTokens int `json:"completion_tokens"`
|
||||
} `json:"usage"`
|
||||
}
|
||||
if err := json.Unmarshal(data, &chunk); err == nil && chunk.Usage.PromptTokens+chunk.Usage.CompletionTokens > 0 {
|
||||
return chunk.Usage.PromptTokens, chunk.Usage.CompletionTokens
|
||||
}
|
||||
return 0, 0
|
||||
}
|
||||
|
||||
func NewStreamHTTPClient() *http.Client {
|
||||
return &http.Client{} // no timeout for streams
|
||||
}
|
||||
|
||||
func IsStreamRequest(body []byte) bool {
|
||||
var payload struct {
|
||||
Stream bool `json:"stream"`
|
||||
}
|
||||
_ = json.Unmarshal(body, &payload)
|
||||
return payload.Stream
|
||||
}
|
||||
|
||||
func SupportsStreaming(endpoint EndpointType) bool {
|
||||
return endpoint == EndpointChatCompletion || endpoint == EndpointResponses
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
package provider
|
||||
|
||||
import "github.com/rose_cat707/luminary/internal/model"
|
||||
|
||||
type ProviderTypeMeta struct {
|
||||
Key model.ProviderType `json:"key"`
|
||||
Label string `json:"label"`
|
||||
}
|
||||
|
||||
var allProviderTypes = []ProviderTypeMeta{
|
||||
{Key: model.ProviderOpenAI, Label: "OpenAI"},
|
||||
{Key: model.ProviderAnthropic, Label: "Anthropic"},
|
||||
{Key: model.ProviderAzureOpenAI, Label: "Azure OpenAI"},
|
||||
{Key: model.ProviderOpenRouter, Label: "OpenRouter"},
|
||||
{Key: model.ProviderOllama, Label: "Ollama"},
|
||||
{Key: model.ProviderCustom, Label: "Custom (OpenAI compatible)"},
|
||||
{Key: model.ProviderComfyUI, Label: "ComfyUI"},
|
||||
}
|
||||
|
||||
func ProviderTypeLabel(t model.ProviderType) string {
|
||||
for _, m := range allProviderTypes {
|
||||
if m.Key == t {
|
||||
return m.Label
|
||||
}
|
||||
}
|
||||
return string(t)
|
||||
}
|
||||
|
||||
// TypesForCategory 各出口分类支持的协议类型
|
||||
func TypesForCategory(cat model.ProviderCategory) []ProviderTypeMeta {
|
||||
switch cat {
|
||||
case model.CategoryImage:
|
||||
return []ProviderTypeMeta{{Key: model.ProviderComfyUI, Label: "ComfyUI"}}
|
||||
case model.CategoryEmbedding:
|
||||
return filterTypes(
|
||||
model.ProviderOpenAI,
|
||||
model.ProviderAzureOpenAI,
|
||||
model.ProviderOpenRouter,
|
||||
model.ProviderOllama,
|
||||
model.ProviderCustom,
|
||||
)
|
||||
case model.CategoryRerank:
|
||||
return filterTypes(
|
||||
model.ProviderOpenAI,
|
||||
model.ProviderAzureOpenAI,
|
||||
model.ProviderOpenRouter,
|
||||
model.ProviderOllama,
|
||||
model.ProviderCustom,
|
||||
)
|
||||
default:
|
||||
return filterTypes(
|
||||
model.ProviderOpenAI,
|
||||
model.ProviderAnthropic,
|
||||
model.ProviderAzureOpenAI,
|
||||
model.ProviderOpenRouter,
|
||||
model.ProviderOllama,
|
||||
model.ProviderCustom,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
func filterTypes(keys ...model.ProviderType) []ProviderTypeMeta {
|
||||
set := map[model.ProviderType]bool{}
|
||||
for _, k := range keys {
|
||||
set[k] = true
|
||||
}
|
||||
out := make([]ProviderTypeMeta, 0, len(keys))
|
||||
for _, m := range allProviderTypes {
|
||||
if set[m.Key] {
|
||||
out = append(out, m)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func DefaultProviderType(cat model.ProviderCategory) model.ProviderType {
|
||||
types := TypesForCategory(cat)
|
||||
if len(types) == 0 {
|
||||
return model.ProviderOpenAI
|
||||
}
|
||||
return types[0].Key
|
||||
}
|
||||
|
||||
func IsTypeValidForCategory(cat model.ProviderCategory, t model.ProviderType) bool {
|
||||
for _, m := range TypesForCategory(cat) {
|
||||
if m.Key == t {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// AllProviderTypes 返回全部协议类型（用于管理台展示标签）
|
||||
func AllProviderTypes() []ProviderTypeMeta {
|
||||
return append([]ProviderTypeMeta(nil), allProviderTypes...)
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
)
|
||||
|
||||
func TestTypesForCategory(t *testing.T) {
|
||||
llm := TypesForCategory(model.CategoryLLM)
|
||||
if len(llm) != 6 {
|
||||
t.Fatalf("llm types: got %d want 6", len(llm))
|
||||
}
|
||||
image := TypesForCategory(model.CategoryImage)
|
||||
if len(image) != 1 || image[0].Key != model.ProviderComfyUI {
|
||||
t.Fatalf("image types: %+v", image)
|
||||
}
|
||||
embedding := TypesForCategory(model.CategoryEmbedding)
|
||||
for _, m := range embedding {
|
||||
if m.Key == model.ProviderAnthropic {
|
||||
t.Fatal("anthropic should not support embedding category")
|
||||
}
|
||||
}
|
||||
if !IsTypeValidForCategory(model.CategoryRerank, model.ProviderCustom) {
|
||||
t.Fatal("custom should be valid for rerank")
|
||||
}
|
||||
if IsTypeValidForCategory(model.CategoryRerank, model.ProviderAnthropic) {
|
||||
t.Fatal("anthropic should not be valid for rerank")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
package settings
|
||||
|
||||
import (
|
||||
"log"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/auth"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// encryptionKeyCandidates returns possible keys ordered for repair attempts.
|
||||
func encryptionKeyCandidates() []string {
|
||||
seen := map[string]bool{}
|
||||
var out []string
|
||||
add := func(key string) {
|
||||
key = strings.TrimSpace(key)
|
||||
if key == "" || seen[key] {
|
||||
return
|
||||
}
|
||||
seen[key] = true
|
||||
out = append(out, key)
|
||||
}
|
||||
|
||||
add(os.Getenv("LUMINARY_ENCRYPTION_KEY"))
|
||||
add(legacyDefaultEncryptionKey)
|
||||
return out
|
||||
}
|
||||
|
||||
// EnsureEncryptionKeyWorks verifies provider keys can be decrypted and repairs the
|
||||
// stored encryption_key from legacy sources when needed.
|
||||
func (r *Runtime) EnsureEncryptionKeyWorks() {
|
||||
var sample model.ProviderKey
|
||||
err := r.db.Where("api_key_enc <> ''").First(&sample).Error
|
||||
if err == gorm.ErrRecordNotFound {
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
log.Printf("settings: check encryption key: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
current := r.EncryptionKey()
|
||||
if _, err := auth.Decrypt(sample.APIKeyEnc, current); err == nil {
|
||||
return
|
||||
}
|
||||
|
||||
for _, candidate := range encryptionKeyCandidates() {
|
||||
if candidate == current {
|
||||
continue
|
||||
}
|
||||
if _, err := auth.Decrypt(sample.APIKeyEnc, candidate); err != nil {
|
||||
continue
|
||||
}
|
||||
key := candidate
|
||||
if err := r.Update(UpdateInput{EncryptionKey: &key}); err != nil {
|
||||
log.Printf("settings: repair encryption_key: %v", err)
|
||||
return
|
||||
}
|
||||
log.Printf("settings: repaired encryption_key from legacy source")
|
||||
return
|
||||
}
|
||||
|
||||
log.Printf("settings: provider keys cannot be decrypted; set the correct encryption_key in admin system settings or re-save provider keys")
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
package settings
|
||||
|
||||
import (
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
)
|
||||
|
||||
const legacyDefaultEncryptionKey = "change-me-in-production-32bytes!!"
|
||||
|
||||
func mergeLegacySettings(s model.SystemSettings) model.SystemSettings {
|
||||
if key := strings.TrimSpace(os.Getenv("LUMINARY_ENCRYPTION_KEY")); key != "" {
|
||||
s.EncryptionKey = key
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// LegacyAdminCredentials returns default bootstrap credentials for first admin.
|
||||
func LegacyAdminCredentials() (username, password string) {
|
||||
return "admin", "admin123"
|
||||
}
|
||||
@@ -0,0 +1,262 @@
|
||||
package settings
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/middleware"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// Runtime holds mutable system settings loaded from the database.
|
||||
type Runtime struct {
|
||||
mu sync.RWMutex
|
||||
db *gorm.DB
|
||||
data model.SystemSettings
|
||||
}
|
||||
|
||||
func NewRuntime(db *gorm.DB) *Runtime {
|
||||
return &Runtime{db: db}
|
||||
}
|
||||
|
||||
func (r *Runtime) Load() error {
|
||||
var s model.SystemSettings
|
||||
err := r.db.First(&s, 1).Error
|
||||
if err == gorm.ErrRecordNotFound {
|
||||
s = defaultSettings()
|
||||
s = mergeLegacySettings(s)
|
||||
if err := r.db.Create(&s).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
} else if err != nil {
|
||||
return err
|
||||
}
|
||||
r.mu.Lock()
|
||||
r.data = s
|
||||
r.mu.Unlock()
|
||||
r.applySideEffects()
|
||||
return nil
|
||||
}
|
||||
|
||||
// AfterLoad runs post-bootstrap checks that depend on other tables.
|
||||
func (r *Runtime) AfterLoad() {
|
||||
r.EnsureEncryptionKeyWorks()
|
||||
}
|
||||
|
||||
func defaultSettings() model.SystemSettings {
|
||||
key := make([]byte, 32)
|
||||
_, _ = rand.Read(key)
|
||||
return model.SystemSettings{
|
||||
ID: 1,
|
||||
EncryptionKey: hex.EncodeToString(key),
|
||||
SessionTTLSeconds: int64((24 * time.Hour).Seconds()),
|
||||
LoginMaxAttempts: 5,
|
||||
LoginLockoutSeconds: int64((15 * time.Minute).Seconds()),
|
||||
TrustedProxiesJSON: "[]",
|
||||
UsageFlushSeconds: 5,
|
||||
HealthCheckSeconds: 30,
|
||||
GatewayMaxRetries: 3,
|
||||
GatewayRetryBackoffMs: 200,
|
||||
ImageStoragePath: "images",
|
||||
SignedURLTTLSeconds: 3600,
|
||||
PredictionWaitSeconds: 60,
|
||||
}
|
||||
}
|
||||
|
||||
func (r *Runtime) Snapshot() model.SystemSettings {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
return r.data
|
||||
}
|
||||
|
||||
func (r *Runtime) EncryptionKey() string {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
return r.data.EncryptionKey
|
||||
}
|
||||
|
||||
func (r *Runtime) SessionTTL() time.Duration {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
return time.Duration(r.data.SessionTTLSeconds) * time.Second
|
||||
}
|
||||
|
||||
func (r *Runtime) LoginMaxAttempts() int {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
return r.data.LoginMaxAttempts
|
||||
}
|
||||
|
||||
func (r *Runtime) LoginLockout() time.Duration {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
return time.Duration(r.data.LoginLockoutSeconds) * time.Second
|
||||
}
|
||||
|
||||
func (r *Runtime) TrustedProxies() []string {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
var out []string
|
||||
_ = json.Unmarshal([]byte(r.data.TrustedProxiesJSON), &out)
|
||||
return out
|
||||
}
|
||||
|
||||
func (r *Runtime) UsageFlushInterval() time.Duration {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
return time.Duration(r.data.UsageFlushSeconds) * time.Second
|
||||
}
|
||||
|
||||
func (r *Runtime) HealthCheckInterval() time.Duration {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
return time.Duration(r.data.HealthCheckSeconds) * time.Second
|
||||
}
|
||||
|
||||
func (r *Runtime) GatewayMaxRetries() int {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
return r.data.GatewayMaxRetries
|
||||
}
|
||||
|
||||
func (r *Runtime) GatewayRetryBackoffMs() int {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
return r.data.GatewayRetryBackoffMs
|
||||
}
|
||||
|
||||
func (r *Runtime) ImageStoragePath() string {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
if r.data.ImageStoragePath == "" {
|
||||
return "images"
|
||||
}
|
||||
return r.data.ImageStoragePath
|
||||
}
|
||||
|
||||
func (r *Runtime) PublicBaseURL() string {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
return r.data.PublicBaseURL
|
||||
}
|
||||
|
||||
func (r *Runtime) SignedURLTTL() time.Duration {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
sec := r.data.SignedURLTTLSeconds
|
||||
if sec <= 0 {
|
||||
sec = 3600
|
||||
}
|
||||
return time.Duration(sec) * time.Second
|
||||
}
|
||||
|
||||
func (r *Runtime) PredictionWaitSeconds() int64 {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
if r.data.PredictionWaitSeconds <= 0 {
|
||||
return 60
|
||||
}
|
||||
return r.data.PredictionWaitSeconds
|
||||
}
|
||||
|
||||
type UpdateInput struct {
|
||||
EncryptionKey *string `json:"encryption_key"`
|
||||
SessionTTLSeconds *int64 `json:"session_ttl_seconds"`
|
||||
LoginMaxAttempts *int `json:"login_max_attempts"`
|
||||
LoginLockoutSeconds *int64 `json:"login_lockout_seconds"`
|
||||
TrustedProxies []string `json:"trusted_proxies"`
|
||||
UsageFlushSeconds *int64 `json:"usage_flush_seconds"`
|
||||
HealthCheckSeconds *int64 `json:"health_check_seconds"`
|
||||
GatewayMaxRetries *int `json:"gateway_max_retries"`
|
||||
GatewayRetryBackoffMs *int `json:"gateway_retry_backoff_ms"`
|
||||
ImageStoragePath *string `json:"image_storage_path"`
|
||||
PublicBaseURL *string `json:"public_base_url"`
|
||||
SignedURLTTLSeconds *int64 `json:"signed_url_ttl_seconds"`
|
||||
PredictionWaitSeconds *int64 `json:"prediction_wait_seconds"`
|
||||
}
|
||||
|
||||
func (r *Runtime) Update(in UpdateInput) error {
|
||||
r.mu.Lock()
|
||||
s := r.data
|
||||
if in.EncryptionKey != nil && *in.EncryptionKey != "" {
|
||||
s.EncryptionKey = *in.EncryptionKey
|
||||
}
|
||||
if in.SessionTTLSeconds != nil && *in.SessionTTLSeconds > 0 {
|
||||
s.SessionTTLSeconds = *in.SessionTTLSeconds
|
||||
}
|
||||
if in.LoginMaxAttempts != nil && *in.LoginMaxAttempts > 0 {
|
||||
s.LoginMaxAttempts = *in.LoginMaxAttempts
|
||||
}
|
||||
if in.LoginLockoutSeconds != nil && *in.LoginLockoutSeconds > 0 {
|
||||
s.LoginLockoutSeconds = *in.LoginLockoutSeconds
|
||||
}
|
||||
if in.TrustedProxies != nil {
|
||||
b, _ := json.Marshal(in.TrustedProxies)
|
||||
s.TrustedProxiesJSON = string(b)
|
||||
}
|
||||
if in.UsageFlushSeconds != nil && *in.UsageFlushSeconds > 0 {
|
||||
s.UsageFlushSeconds = *in.UsageFlushSeconds
|
||||
}
|
||||
if in.HealthCheckSeconds != nil && *in.HealthCheckSeconds > 0 {
|
||||
s.HealthCheckSeconds = *in.HealthCheckSeconds
|
||||
}
|
||||
if in.GatewayMaxRetries != nil && *in.GatewayMaxRetries > 0 {
|
||||
s.GatewayMaxRetries = *in.GatewayMaxRetries
|
||||
}
|
||||
if in.GatewayRetryBackoffMs != nil && *in.GatewayRetryBackoffMs >= 0 {
|
||||
s.GatewayRetryBackoffMs = *in.GatewayRetryBackoffMs
|
||||
}
|
||||
if in.ImageStoragePath != nil {
|
||||
s.ImageStoragePath = *in.ImageStoragePath
|
||||
}
|
||||
if in.PublicBaseURL != nil {
|
||||
s.PublicBaseURL = *in.PublicBaseURL
|
||||
}
|
||||
if in.SignedURLTTLSeconds != nil && *in.SignedURLTTLSeconds > 0 {
|
||||
s.SignedURLTTLSeconds = *in.SignedURLTTLSeconds
|
||||
}
|
||||
if in.PredictionWaitSeconds != nil && *in.PredictionWaitSeconds > 0 {
|
||||
s.PredictionWaitSeconds = *in.PredictionWaitSeconds
|
||||
}
|
||||
if err := r.db.Save(&s).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
r.data = s
|
||||
r.mu.Unlock()
|
||||
r.applySideEffects()
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *Runtime) applySideEffects() {
|
||||
middleware.ConfigureTrustedProxies(r.TrustedProxies())
|
||||
}
|
||||
|
||||
func (r *Runtime) PublicView() map[string]any {
|
||||
s := r.Snapshot()
|
||||
masked := "(已设置)"
|
||||
if s.EncryptionKey == "" {
|
||||
masked = "(未设置)"
|
||||
}
|
||||
var proxies []string
|
||||
_ = json.Unmarshal([]byte(s.TrustedProxiesJSON), &proxies)
|
||||
return map[string]any{
|
||||
"encryption_key_set": s.EncryptionKey != "",
|
||||
"encryption_key_hint": masked,
|
||||
"session_ttl_seconds": s.SessionTTLSeconds,
|
||||
"login_max_attempts": s.LoginMaxAttempts,
|
||||
"login_lockout_seconds": s.LoginLockoutSeconds,
|
||||
"trusted_proxies": proxies,
|
||||
"usage_flush_seconds": s.UsageFlushSeconds,
|
||||
"health_check_seconds": s.HealthCheckSeconds,
|
||||
"gateway_max_retries": s.GatewayMaxRetries,
|
||||
"gateway_retry_backoff_ms": s.GatewayRetryBackoffMs,
|
||||
"image_storage_path": s.ImageStoragePath,
|
||||
"public_base_url": s.PublicBaseURL,
|
||||
"signed_url_ttl_seconds": s.SignedURLTTLSeconds,
|
||||
"prediction_wait_seconds": s.PredictionWaitSeconds,
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
package imagestore
|
||||
|
||||
import (
|
||||
"crypto/hmac"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
type Store struct {
|
||||
db *gorm.DB
|
||||
baseDir string
|
||||
signKey string
|
||||
publicURL string
|
||||
ttl time.Duration
|
||||
}
|
||||
|
||||
func New(db *gorm.DB, baseDir, publicURL, signKey string, ttl time.Duration) *Store {
|
||||
return &Store{db: db, baseDir: baseDir, publicURL: strings.TrimRight(publicURL, "/"), signKey: signKey, ttl: ttl}
|
||||
}
|
||||
|
||||
func (s *Store) EnsureDir() error {
|
||||
return os.MkdirAll(s.baseDir, 0o755)
|
||||
}
|
||||
|
||||
func (s *Store) Save(predictionID, filename string, data []byte, mime string) (*model.StoredImage, error) {
|
||||
if err := s.EnsureDir(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
dir := filepath.Join(s.baseDir, predictionID)
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
localPath := filepath.Join(dir, filename)
|
||||
if err := os.WriteFile(localPath, data, 0o644); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
img := model.StoredImage{
|
||||
ID: uuid.NewString(),
|
||||
PredictionID: predictionID,
|
||||
Filename: filename,
|
||||
LocalPath: localPath,
|
||||
Mime: mime,
|
||||
Size: int64(len(data)),
|
||||
CreatedAt: time.Now(),
|
||||
}
|
||||
if err := s.db.Create(&img).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &img, nil
|
||||
}
|
||||
|
||||
func (s *Store) Get(id string) (*model.StoredImage, error) {
|
||||
var img model.StoredImage
|
||||
if err := s.db.First(&img, "id = ?", id).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &img, nil
|
||||
}
|
||||
|
||||
func (s *Store) ListByPrediction(predictionID string) ([]model.StoredImage, error) {
|
||||
var imgs []model.StoredImage
|
||||
err := s.db.Where("prediction_id = ?", predictionID).Order("created_at asc").Find(&imgs).Error
|
||||
return imgs, err
|
||||
}
|
||||
|
||||
func (s *Store) SignURL(imageID string) string {
|
||||
exp := time.Now().Add(s.ttl).Unix()
|
||||
sig := s.sign(imageID, exp)
|
||||
return fmt.Sprintf("%s/files/%s?exp=%d&sig=%s", s.publicURL, imageID, exp, sig)
|
||||
}
|
||||
|
||||
func (s *Store) Verify(imageID string, exp int64, sig string) bool {
|
||||
if time.Now().Unix() > exp {
|
||||
return false
|
||||
}
|
||||
return hmac.Equal([]byte(s.sign(imageID, exp)), []byte(sig))
|
||||
}
|
||||
|
||||
func (s *Store) sign(imageID string, exp int64) string {
|
||||
mac := hmac.New(sha256.New, []byte(s.signKey))
|
||||
mac.Write([]byte(imageID + "|" + strconv.FormatInt(exp, 10)))
|
||||
return hex.EncodeToString(mac.Sum(nil))
|
||||
}
|
||||
Vendored
+455
@@ -0,0 +1,455 @@
|
||||
package cache
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/auth"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
)
|
||||
|
||||
type IngressKeySnapshot struct {
|
||||
Key model.IngressKey
|
||||
KeyPlain string // only set on create, not persisted in cache long-term
|
||||
}
|
||||
|
||||
type UsageDelta struct {
|
||||
IngressKeyID uint
|
||||
ProviderKeyID uint
|
||||
ProviderID uint
|
||||
ClientIP string
|
||||
Endpoint string
|
||||
Model string
|
||||
TokensIn int
|
||||
TokensOut int
|
||||
Cost float64
|
||||
LatencyMs int64
|
||||
Status string
|
||||
Stream bool
|
||||
ErrorMsg string
|
||||
RequestBody string
|
||||
ResponseBody string
|
||||
BudgetUsed float64
|
||||
Requests int64
|
||||
Tokens int64
|
||||
}
|
||||
|
||||
type Store struct {
|
||||
mu sync.RWMutex
|
||||
|
||||
providers map[uint]*model.Provider
|
||||
providerKeys map[uint]*model.ProviderKey
|
||||
providerByName map[string]*model.Provider
|
||||
models map[uint][]model.ModelEntry
|
||||
|
||||
ingressKeys map[uint]*model.IngressKey
|
||||
ingressByHash map[string]*model.IngressKey
|
||||
|
||||
ipRules []model.IPRule
|
||||
routingRules map[uint][]model.RoutingRule // ingressKeyID -> rules
|
||||
|
||||
healthStatus map[uint]model.HealthCheckRecord
|
||||
|
||||
usageDeltas []UsageDelta
|
||||
usageMu sync.Mutex
|
||||
|
||||
// rate limit counters: ingressKeyID -> window start + counts
|
||||
rateMu sync.Mutex
|
||||
rateWindows map[uint]*rateWindow
|
||||
}
|
||||
|
||||
type rateWindow struct {
|
||||
start time.Time
|
||||
requests int
|
||||
tokens int
|
||||
}
|
||||
|
||||
func New() *Store {
|
||||
return &Store{
|
||||
providers: make(map[uint]*model.Provider),
|
||||
providerKeys: make(map[uint]*model.ProviderKey),
|
||||
providerByName: make(map[string]*model.Provider),
|
||||
models: make(map[uint][]model.ModelEntry),
|
||||
ingressKeys: make(map[uint]*model.IngressKey),
|
||||
ingressByHash: make(map[string]*model.IngressKey),
|
||||
routingRules: make(map[uint][]model.RoutingRule),
|
||||
healthStatus: make(map[uint]model.HealthCheckRecord),
|
||||
rateWindows: make(map[uint]*rateWindow),
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Store) LoadProviders(providers []model.Provider) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.providers = make(map[uint]*model.Provider)
|
||||
c.providerByName = make(map[string]*model.Provider)
|
||||
for i := range providers {
|
||||
p := providers[i]
|
||||
c.providers[p.ID] = &p
|
||||
c.providerByName[p.Name] = &p
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Store) LoadProviderKeys(keys []model.ProviderKey) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.providerKeys = make(map[uint]*model.ProviderKey)
|
||||
for i := range keys {
|
||||
k := keys[i]
|
||||
c.providerKeys[k.ID] = &k
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Store) LoadModels(models []model.ModelEntry) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.models = make(map[uint][]model.ModelEntry)
|
||||
for _, m := range models {
|
||||
c.models[m.ProviderID] = append(c.models[m.ProviderID], m)
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Store) LoadIngressKeys(keys []model.IngressKey) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.ingressKeys = make(map[uint]*model.IngressKey)
|
||||
c.ingressByHash = make(map[string]*model.IngressKey)
|
||||
for i := range keys {
|
||||
k := keys[i]
|
||||
c.ingressKeys[k.ID] = &k
|
||||
c.ingressByHash[k.KeyHash] = &k
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Store) LoadIPRules(rules []model.IPRule) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.ipRules = rules
|
||||
}
|
||||
|
||||
func (c *Store) GetProvider(id uint) (model.Provider, bool) {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
p, ok := c.providers[id]
|
||||
if !ok {
|
||||
return model.Provider{}, false
|
||||
}
|
||||
return *p, true
|
||||
}
|
||||
|
||||
func (c *Store) GetProviderByName(name string) (model.Provider, bool) {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
p, ok := c.providerByName[name]
|
||||
if !ok {
|
||||
return model.Provider{}, false
|
||||
}
|
||||
return *p, true
|
||||
}
|
||||
|
||||
func (c *Store) ListProviders() []model.Provider {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
out := make([]model.Provider, 0, len(c.providers))
|
||||
for _, p := range c.providers {
|
||||
out = append(out, *p)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (c *Store) SetProvider(p model.Provider) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.providers[p.ID] = &p
|
||||
c.providerByName[p.Name] = &p
|
||||
}
|
||||
|
||||
func (c *Store) DeleteProvider(id uint) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
if p, ok := c.providers[id]; ok {
|
||||
delete(c.providerByName, p.Name)
|
||||
}
|
||||
delete(c.providers, id)
|
||||
delete(c.models, id)
|
||||
}
|
||||
|
||||
func (c *Store) GetProviderKey(id uint) (model.ProviderKey, bool) {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
k, ok := c.providerKeys[id]
|
||||
if !ok {
|
||||
return model.ProviderKey{}, false
|
||||
}
|
||||
return *k, true
|
||||
}
|
||||
|
||||
func (c *Store) ListProviderKeys(providerID uint) []model.ProviderKey {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
var out []model.ProviderKey
|
||||
for _, k := range c.providerKeys {
|
||||
if k.ProviderID == providerID {
|
||||
out = append(out, *k)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (c *Store) ListAllProviderKeys() []model.ProviderKey {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
out := make([]model.ProviderKey, 0, len(c.providerKeys))
|
||||
for _, k := range c.providerKeys {
|
||||
out = append(out, *k)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (c *Store) SetProviderKey(k model.ProviderKey) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.providerKeys[k.ID] = &k
|
||||
}
|
||||
|
||||
func (c *Store) DeleteProviderKey(id uint) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
delete(c.providerKeys, id)
|
||||
}
|
||||
|
||||
func (c *Store) GetModels(providerID uint) []model.ModelEntry {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
models := c.models[providerID]
|
||||
if models == nil {
|
||||
return []model.ModelEntry{}
|
||||
}
|
||||
return models
|
||||
}
|
||||
|
||||
func (c *Store) SetModels(providerID uint, models []model.ModelEntry) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.models[providerID] = models
|
||||
}
|
||||
|
||||
func (c *Store) ListIngressKeys() []model.IngressKey {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
out := make([]model.IngressKey, 0, len(c.ingressKeys))
|
||||
for _, k := range c.ingressKeys {
|
||||
out = append(out, *k)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (c *Store) GetIngressKey(id uint) (model.IngressKey, bool) {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
k, ok := c.ingressKeys[id]
|
||||
if !ok {
|
||||
return model.IngressKey{}, false
|
||||
}
|
||||
return *k, true
|
||||
}
|
||||
|
||||
func (c *Store) SetIngressKey(k model.IngressKey) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.ingressKeys[k.ID] = &k
|
||||
c.ingressByHash[k.KeyHash] = &k
|
||||
}
|
||||
|
||||
func (c *Store) DeleteIngressKey(id uint) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
if k, ok := c.ingressKeys[id]; ok {
|
||||
delete(c.ingressByHash, k.KeyHash)
|
||||
}
|
||||
delete(c.ingressKeys, id)
|
||||
}
|
||||
|
||||
func (c *Store) LookupIngressKey(plain string) (*model.IngressKey, bool) {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
hash := auth.HashIngressKey(plain)
|
||||
if k, ok := c.ingressByHash[hash]; ok && k.Enabled {
|
||||
cp := *k
|
||||
return &cp, true
|
||||
}
|
||||
for _, k := range c.ingressKeys {
|
||||
if !k.Enabled || !auth.IsLegacyIngressHash(k.KeyHash) {
|
||||
continue
|
||||
}
|
||||
if auth.CheckIngressKey(k.KeyHash, plain) {
|
||||
cp := *k
|
||||
return &cp, true
|
||||
}
|
||||
}
|
||||
return nil, false
|
||||
}
|
||||
|
||||
func (c *Store) ResetProviderKeyDailyCounts() {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
for _, k := range c.providerKeys {
|
||||
k.RequestsToday = 0
|
||||
k.TokensToday = 0
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Store) ListIPRules() []model.IPRule {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
out := make([]model.IPRule, len(c.ipRules))
|
||||
copy(out, c.ipRules)
|
||||
return out
|
||||
}
|
||||
|
||||
func (c *Store) SetIPRules(rules []model.IPRule) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.ipRules = rules
|
||||
}
|
||||
|
||||
func (c *Store) SetHealthStatus(providerID uint, rec model.HealthCheckRecord) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.healthStatus[providerID] = rec
|
||||
if p, ok := c.providers[providerID]; ok {
|
||||
p.Status = rec.Status
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Store) GetHealthStatus() map[uint]model.HealthCheckRecord {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
out := make(map[uint]model.HealthCheckRecord, len(c.healthStatus))
|
||||
for k, v := range c.healthStatus {
|
||||
out[k] = v
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (c *Store) RecordUsage(delta UsageDelta) {
|
||||
c.mu.Lock()
|
||||
if k, ok := c.ingressKeys[delta.IngressKeyID]; ok {
|
||||
k.BudgetUsed += delta.BudgetUsed
|
||||
k.RequestCount += delta.Requests
|
||||
k.TokenCount += delta.Tokens
|
||||
}
|
||||
if pk, ok := c.providerKeys[delta.ProviderKeyID]; ok {
|
||||
pk.RequestsToday += int(delta.Requests)
|
||||
pk.TokensToday += delta.Tokens
|
||||
}
|
||||
c.mu.Unlock()
|
||||
|
||||
c.usageMu.Lock()
|
||||
c.usageDeltas = append(c.usageDeltas, delta)
|
||||
c.usageMu.Unlock()
|
||||
}
|
||||
|
||||
func (c *Store) DrainUsageDeltas() []UsageDelta {
|
||||
c.usageMu.Lock()
|
||||
defer c.usageMu.Unlock()
|
||||
if len(c.usageDeltas) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := c.usageDeltas
|
||||
c.usageDeltas = nil
|
||||
return out
|
||||
}
|
||||
|
||||
func (c *Store) CheckRateLimit(ingressKeyID uint, rpm, tpm int, tokens int) bool {
|
||||
if rpm <= 0 && tpm <= 0 {
|
||||
return true
|
||||
}
|
||||
c.rateMu.Lock()
|
||||
defer c.rateMu.Unlock()
|
||||
now := time.Now()
|
||||
w, ok := c.rateWindows[ingressKeyID]
|
||||
if !ok || now.Sub(w.start) >= time.Minute {
|
||||
c.rateWindows[ingressKeyID] = &rateWindow{start: now, requests: 1, tokens: tokens}
|
||||
return true
|
||||
}
|
||||
if rpm > 0 && w.requests >= rpm {
|
||||
return false
|
||||
}
|
||||
if tpm > 0 && w.tokens+tokens > tpm {
|
||||
return false
|
||||
}
|
||||
w.requests++
|
||||
w.tokens += tokens
|
||||
return true
|
||||
}
|
||||
|
||||
func ParseJSONList(s string) []string {
|
||||
var out []string
|
||||
if s == "" {
|
||||
return []string{"*"}
|
||||
}
|
||||
_ = json.Unmarshal([]byte(s), &out)
|
||||
if len(out) == 0 {
|
||||
return []string{"*"}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func MatchesList(list []string, value string) bool {
|
||||
for _, item := range list {
|
||||
if item == "*" || item == value {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func ModelsToJSON(models []string) string {
|
||||
b, _ := json.Marshal(models)
|
||||
return string(b)
|
||||
}
|
||||
|
||||
func (c *Store) LoadRoutingRules(rules []model.RoutingRule) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.routingRules = make(map[uint][]model.RoutingRule)
|
||||
for _, r := range rules {
|
||||
c.routingRules[r.IngressKeyID] = append(c.routingRules[r.IngressKeyID], r)
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Store) ListRoutingRules(ingressKeyID uint) []model.RoutingRule {
|
||||
c.mu.RLock()
|
||||
defer c.mu.RUnlock()
|
||||
out := c.routingRules[ingressKeyID]
|
||||
cp := make([]model.RoutingRule, len(out))
|
||||
copy(cp, out)
|
||||
return cp
|
||||
}
|
||||
|
||||
func (c *Store) SetRoutingRules(ingressKeyID uint, rules []model.RoutingRule) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.routingRules[ingressKeyID] = rules
|
||||
}
|
||||
|
||||
func (c *Store) DeleteRoutingRulesForIngress(ingressKeyID uint) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
delete(c.routingRules, ingressKeyID)
|
||||
}
|
||||
|
||||
type Counter struct {
|
||||
v atomic.Int64
|
||||
}
|
||||
|
||||
func (c *Counter) Add(n int64) {
|
||||
c.v.Add(n)
|
||||
}
|
||||
|
||||
func (c *Counter) Load() int64 {
|
||||
return c.v.Load()
|
||||
}
|
||||
@@ -0,0 +1,136 @@
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/rose_cat707/luminary/internal/auth"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"github.com/rose_cat707/luminary/internal/settings"
|
||||
"github.com/rose_cat707/luminary/internal/store/cache"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
type Repository struct {
|
||||
db *gorm.DB
|
||||
cache *cache.Store
|
||||
}
|
||||
|
||||
func NewRepository(store *Store, c *cache.Store) *Repository {
|
||||
return &Repository{db: store.DB(), cache: c}
|
||||
}
|
||||
|
||||
func (r *Repository) BootstrapFromDB() error {
|
||||
var providers []model.Provider
|
||||
if err := r.db.Find(&providers).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
r.cache.LoadProviders(providers)
|
||||
|
||||
var keys []model.ProviderKey
|
||||
if err := r.db.Find(&keys).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
r.cache.LoadProviderKeys(keys)
|
||||
|
||||
var models []model.ModelEntry
|
||||
if err := r.db.Find(&models).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
r.cache.LoadModels(models)
|
||||
|
||||
var ingress []model.IngressKey
|
||||
if err := r.db.Find(&ingress).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
r.cache.LoadIngressKeys(ingress)
|
||||
|
||||
var rules []model.IPRule
|
||||
if err := r.db.Find(&rules).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
r.cache.LoadIPRules(rules)
|
||||
|
||||
var routing []model.RoutingRule
|
||||
if err := r.db.Find(&routing).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
r.cache.LoadRoutingRules(routing)
|
||||
r.PruneUsageRecords(model.MaxUsageLogRecords)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *Repository) EnsureAdmin() error {
|
||||
var count int64
|
||||
r.db.Model(&model.AdminUser{}).Count(&count)
|
||||
if count > 0 {
|
||||
return nil
|
||||
}
|
||||
username, password := settings.LegacyAdminCredentials()
|
||||
hash, err := auth.HashPassword(password)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return r.db.Create(&model.AdminUser{Username: username, PasswordHash: hash}).Error
|
||||
}
|
||||
|
||||
func (r *Repository) FlushUsage(deltas []cache.UsageDelta) {
|
||||
for _, d := range deltas {
|
||||
r.db.Create(&model.UsageRecord{
|
||||
IngressKeyID: d.IngressKeyID,
|
||||
ProviderID: d.ProviderID,
|
||||
ProviderKeyID: d.ProviderKeyID,
|
||||
ClientIP: d.ClientIP,
|
||||
Endpoint: d.Endpoint,
|
||||
Model: d.Model,
|
||||
TokensIn: d.TokensIn,
|
||||
TokensOut: d.TokensOut,
|
||||
Cost: d.Cost,
|
||||
LatencyMs: d.LatencyMs,
|
||||
Status: d.Status,
|
||||
Stream: d.Stream,
|
||||
ErrorMsg: d.ErrorMsg,
|
||||
RequestBody: d.RequestBody,
|
||||
ResponseBody: d.ResponseBody,
|
||||
CreatedAt: time.Now(),
|
||||
})
|
||||
if d.IngressKeyID > 0 {
|
||||
r.db.Model(&model.IngressKey{}).Where("id = ?", d.IngressKeyID).Updates(map[string]any{
|
||||
"budget_used": gorm.Expr("budget_used + ?", d.BudgetUsed),
|
||||
"request_count": gorm.Expr("request_count + ?", d.Requests),
|
||||
"token_count": gorm.Expr("token_count + ?", d.Tokens),
|
||||
})
|
||||
}
|
||||
if d.ProviderKeyID > 0 {
|
||||
r.db.Model(&model.ProviderKey{}).Where("id = ?", d.ProviderKeyID).Updates(map[string]any{
|
||||
"requests_today": gorm.Expr("requests_today + ?", d.Requests),
|
||||
"tokens_today": gorm.Expr("tokens_today + ?", d.Tokens),
|
||||
})
|
||||
}
|
||||
}
|
||||
r.PruneUsageRecords(model.MaxUsageLogRecords)
|
||||
}
|
||||
|
||||
func (r *Repository) RecordRequestLog(rec model.UsageRecord) {
|
||||
if rec.CreatedAt.IsZero() {
|
||||
rec.CreatedAt = time.Now()
|
||||
}
|
||||
r.db.Create(&rec)
|
||||
r.PruneUsageRecords(model.MaxUsageLogRecords)
|
||||
}
|
||||
|
||||
func (r *Repository) PruneUsageRecords(max int) {
|
||||
if max <= 0 {
|
||||
return
|
||||
}
|
||||
var count int64
|
||||
r.db.Model(&model.UsageRecord{}).Count(&count)
|
||||
if count <= int64(max) {
|
||||
return
|
||||
}
|
||||
var keepIDs []uint
|
||||
r.db.Model(&model.UsageRecord{}).Order("created_at desc").Limit(max).Pluck("id", &keepIDs)
|
||||
if len(keepIDs) == 0 {
|
||||
return
|
||||
}
|
||||
r.db.Where("id NOT IN ?", keepIDs).Delete(&model.UsageRecord{})
|
||||
}
|
||||
@@ -0,0 +1,109 @@
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
|
||||
"github.com/glebarez/sqlite"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/logger"
|
||||
)
|
||||
|
||||
type Store struct {
|
||||
db *gorm.DB
|
||||
}
|
||||
|
||||
func New(path string) (*Store, error) {
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||
return nil, fmt.Errorf("create data dir: %w", err)
|
||||
}
|
||||
db, err := gorm.Open(sqlite.Open(path+"?_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)"), &gorm.Config{
|
||||
Logger: logger.Default.LogMode(logger.Warn),
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open sqlite: %w", err)
|
||||
}
|
||||
sqlDB, err := db.DB()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sqlDB.SetMaxOpenConns(1)
|
||||
if err := ensureIPRulesSchema(db); err != nil {
|
||||
return nil, fmt.Errorf("migrate ip_rules: %w", err)
|
||||
}
|
||||
if err := db.AutoMigrate(
|
||||
&model.Provider{},
|
||||
&model.ProviderKey{},
|
||||
&model.ModelEntry{},
|
||||
&model.IngressKey{},
|
||||
&model.AdminUser{},
|
||||
&model.AdminSession{},
|
||||
&model.UsageRecord{},
|
||||
&model.HealthCheckRecord{},
|
||||
&model.RoutingRule{},
|
||||
&model.SystemSettings{},
|
||||
&model.Prediction{},
|
||||
&model.StoredImage{},
|
||||
); err != nil {
|
||||
return nil, fmt.Errorf("migrate: %w", err)
|
||||
}
|
||||
return &Store{db: db}, nil
|
||||
}
|
||||
|
||||
// ensureIPRulesSchema manages ip_rules without GORM AutoMigrate (avoids bad table rebuilds).
|
||||
func ensureIPRulesSchema(db *gorm.DB) error {
|
||||
if !tableExists(db, "ip_rules") {
|
||||
return db.Exec(`
|
||||
CREATE TABLE ip_rules (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
cidr TEXT NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
scope TEXT NOT NULL DEFAULT 'proxy',
|
||||
enabled NUMERIC DEFAULT 1,
|
||||
created_at DATETIME
|
||||
);
|
||||
`).Error
|
||||
}
|
||||
if !tableHasColumn(db, "ip_rules", "c_id_r") {
|
||||
return nil
|
||||
}
|
||||
|
||||
cidrExpr := "c_id_r"
|
||||
if tableHasColumn(db, "ip_rules", "cidr") {
|
||||
cidrExpr = "COALESCE(NULLIF(cidr, ''), c_id_r)"
|
||||
}
|
||||
|
||||
return db.Exec(`
|
||||
CREATE TABLE ip_rules_mig (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
cidr TEXT NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
scope TEXT NOT NULL DEFAULT 'proxy',
|
||||
enabled NUMERIC DEFAULT 1,
|
||||
created_at DATETIME
|
||||
);
|
||||
INSERT INTO ip_rules_mig (id, cidr, type, scope, enabled, created_at)
|
||||
SELECT id, ` + cidrExpr + `, type, scope, enabled, created_at
|
||||
FROM ip_rules;
|
||||
DROP TABLE ip_rules;
|
||||
ALTER TABLE ip_rules_mig RENAME TO ip_rules;
|
||||
`).Error
|
||||
}
|
||||
|
||||
func tableHasColumn(db *gorm.DB, table, column string) bool {
|
||||
var n int64
|
||||
db.Raw(`SELECT COUNT(*) FROM pragma_table_info(?) WHERE name = ?`, table, column).Scan(&n)
|
||||
return n > 0
|
||||
}
|
||||
|
||||
func tableExists(db *gorm.DB, table string) bool {
|
||||
var n int64
|
||||
db.Raw(`SELECT COUNT(*) FROM sqlite_master WHERE type='table' AND name=?`, table).Scan(&n)
|
||||
return n > 0
|
||||
}
|
||||
|
||||
func (s *Store) DB() *gorm.DB {
|
||||
return s.db
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
package sqlite
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/glebarez/sqlite"
|
||||
"github.com/rose_cat707/luminary/internal/model"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/logger"
|
||||
)
|
||||
|
||||
func TestPreMigrateIPRulesTable_LegacySchema(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "legacy.db")
|
||||
|
||||
legacy, err := gorm.Open(sqlite.Open(path), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := legacy.Exec(`
|
||||
CREATE TABLE ip_rules (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
c_id_r TEXT NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
scope TEXT NOT NULL DEFAULT 'all',
|
||||
enabled NUMERIC DEFAULT 1,
|
||||
created_at DATETIME
|
||||
);
|
||||
INSERT INTO ip_rules (c_id_r, type, scope, enabled) VALUES ('10.0.0.1', 'allow', 'proxy', 1);
|
||||
`).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sqlDB, _ := legacy.DB()
|
||||
sqlDB.Close()
|
||||
|
||||
store, err := New(path)
|
||||
if err != nil {
|
||||
t.Fatalf("New() with legacy schema: %v", err)
|
||||
}
|
||||
|
||||
var rules []model.IPRule
|
||||
if err := store.DB().Find(&rules).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(rules) != 1 || rules[0].CIDR != "10.0.0.1" {
|
||||
t.Fatalf("unexpected rules: %+v", rules)
|
||||
}
|
||||
if store.DB().Migrator().HasColumn(&model.IPRule{}, "c_id_r") {
|
||||
t.Fatal("legacy c_id_r column should be removed")
|
||||
}
|
||||
if tableHasColumn(store.DB(), "ip_rules", "c_id_r") {
|
||||
t.Fatal("legacy c_id_r column should be removed")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNew_FreshDatabase(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "fresh.db")
|
||||
store, err := New(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := store.DB().Create(&model.IPRule{CIDR: "127.0.0.1", Type: model.IPRuleAllow, Scope: model.IPScopeProxy, Enabled: true}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = os.Remove(path)
|
||||
}
|
||||
Executable
+23
@@ -0,0 +1,23 @@
|
||||
#!/bin/sh
|
||||
# Emergency recovery inside the container or local checkout.
|
||||
set -eu
|
||||
|
||||
DATA_DIR="${LUMINARY_DATA:-/data}"
|
||||
REPO_DIR="${LUMINARY_HOME:-/opt/luminary}"
|
||||
|
||||
if [ -x /usr/local/bin/luminary-recover ]; then
|
||||
exec /usr/local/bin/luminary-recover -data-dir "$DATA_DIR" "$@"
|
||||
fi
|
||||
|
||||
if [ -x "${REPO_DIR}/luminary-recover" ]; then
|
||||
exec "${REPO_DIR}/luminary-recover" -data-dir "$DATA_DIR" "$@"
|
||||
fi
|
||||
|
||||
if [ -f "${REPO_DIR}/go.mod" ]; then
|
||||
cd "$REPO_DIR"
|
||||
export CGO_ENABLED=0
|
||||
exec go run ./cmd/luminary-recover -data-dir "$DATA_DIR" "$@"
|
||||
fi
|
||||
|
||||
echo "luminary-recover binary not found" >&2
|
||||
exit 1
|
||||
@@ -0,0 +1 @@
|
||||
registry=https://registry.npmmirror.com
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user