-
Notifications
You must be signed in to change notification settings - Fork 0
fix(sql): 修复表结构同步忘记初始化新增语句的BUG,同步增加后端版本同步sql提醒 #64
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
a1889a9
fix(mysql): 修复 device_product 初始化数据列数不匹配
3db23ee
chore(tools): 新增初始化 SQL 列数校验脚本
b09dade
docs: 新增 CLAUDE.md 仓库指引
5e9c531
chore(review): 添加版本变更时同步检查 mysql8 初始化脚本的规则
e90dd5b
chore(review): 移除 Copilot 指令,仅保留 Claude 与 PR 模板规则
2f7751c
fix(PR): 修改变量
xiaorongnie 5b5e2f4
fix(PR): 修改变量2
xiaorongnie File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| # Copilot Review Instructions | ||
|
|
||
| When reviewing pull requests in this repository, apply the following rules. | ||
|
|
||
| ## Version bumps that require MySQL init data sync | ||
|
|
||
| This repository is a Docker Compose packaging repo for a fleet/vehicle tracking platform. Backend and frontend version variables in `default.env` / `default.en.env` map to application releases that may introduce new database migrations or seed-data changes. | ||
|
|
||
| If a PR changes either of these version variables, the author must also review and update the corresponding MySQL initialization scripts under `mysql8/initdb/`: | ||
|
|
||
| - `BUS_WEB_VERSION` — may affect the `cbus` database. Verify that `mysql8/initdb/03-cbus-init-table.sql` and `mysql8/initdb/04-cbus-init-data.sql` still match the expected schema and seed data for the new version. | ||
| - `TRACK_MAINTAIN_VERSION` — may affect the `maintain` database. Verify that `mysql8/initdb/01-maintain-init-table.sql` and `mysql8/initdb/02-maintain-init-data.sql` still match the expected schema and seed data for the new version. | ||
|
|
||
| Do not approve the PR until: | ||
|
|
||
| 1. The init SQL scripts are confirmed to be consistent with the new version, OR | ||
| 2. The author explicitly confirms that no schema/seed changes are required for this version bump. | ||
|
|
||
| ## General SQL init validation | ||
|
|
||
| If `mysql8/initdb/*.sql` is modified, run `node scripts/check-init-sql.js` to ensure every `INSERT INTO ... VALUES` statement has the same number of values as the corresponding `CREATE TABLE` has columns. | ||
|
|
||
| ## Compose validation | ||
|
|
||
| Any change to `compose.yml`, `compose*.yml`, or included compose fragments should be validated with `docker compose config` before merging. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| ## PR Checklist | ||
|
|
||
| - [ ] I have run `docker compose config` and the compose configuration resolves without errors. | ||
| - [ ] If I changed `BUS_WEB_VERSION`, I verified whether `mysql8/initdb/03-cbus-init-table.sql` and `mysql8/initdb/04-cbus-init-data.sql` need to be updated for the new version. | ||
| - [ ] If I changed `TRACK_MAINTAIN_VERSION`, I verified whether `mysql8/initdb/01-maintain-init-table.sql` and `mysql8/initdb/02-maintain-init-data.sql` need to be updated for the new version. | ||
| - [ ] If I changed `mysql8/initdb/*.sql`, I ran `node scripts/check-init-sql.js` and fixed any column/value mismatches. | ||
| - [ ] I exported `docker compose config > compose-stack.yaml` and reviewed the diff before submitting. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,117 @@ | ||
| # CLAUDE.md | ||
|
|
||
| This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. | ||
|
|
||
| ## Repository Overview | ||
|
|
||
| This is the Docker Compose configuration repository for TranscodeGroup's fleet/vehicle tracking platform. It does not contain application source code; it packages and orchestrates pre-built backend/frontend images and infrastructure services (MySQL, Redis, MongoDB, RabbitMQ, MinIO, nginx, SRS, etc.). | ||
|
|
||
| The repo is deployed to `/home/docker` on target servers. Runtime configuration and compose overrides live in `/home/docker-compose`, and persistent data is written to `/data`. | ||
|
|
||
| ## Common Commands | ||
|
|
||
| All compose commands are run from `/home/docker-compose`, where the top-level `compose.yaml` and `.env` files reside. | ||
|
|
||
| ```sh | ||
| # Validate compose configuration and required environment variables | ||
| docker compose config | ||
|
|
||
| # Export resolved compose config for diff review | ||
| docker compose config > compose-stack.yaml | ||
|
|
||
| # Start the stack in the background | ||
| docker compose up -d | ||
|
|
||
| # Recreate a specific service after changing its config/image | ||
| docker compose up -d --force-recreate <service> | ||
|
|
||
| # Follow logs for a service | ||
| docker compose logs -f <service> | ||
|
|
||
| # Validate that initdb INSERT statements match CREATE TABLE column counts | ||
| node /home/docker/scripts/check-init-sql.js | ||
| ``` | ||
|
|
||
| When modifying `mysql8/initdb/*.sql`, run `check-init-sql.js` before committing to catch column/value mismatches. | ||
|
|
||
| ## High-Level Architecture | ||
|
|
||
| ### Modular Compose Layout | ||
|
|
||
| Each top-level directory (`maintain/`, `jtt808/`, `bus/`, `video/`, `mysql8/`, `nginx/`, etc.) contains one or more focused `compose.yml` fragments. These fragments are assembled by the example deployments in `examples/` using Docker Compose `include`: | ||
|
|
||
| - `examples/track-http/compose.yaml` — Tracker V2 single-node HTTP deployment. | ||
| - `examples/track-https/compose.yaml` — Tracker V2 single-node HTTPS deployment. | ||
| - `examples/bus-http/compose.yaml` / `examples/bus-https/compose.yaml` — Bus platform single-node deployments. | ||
| - `examples/video-storage/compose.yaml` / `examples/video-stream/compose.yaml` — Distributed video storage/streaming nodes. | ||
|
|
||
| A deployment typically `include`s shared infrastructure (mysql8, redis, mongodb, rabbitmq, minio), one or more backend service fragments, nginx fragments, and optional video or web-downloader fragments. | ||
|
|
||
| ### Two Primary Products | ||
|
|
||
| 1. **Tracker V2 (track)** | ||
| - `maintain` — Spring Boot backend (`transcodegroup/maintain-server`), the main API and device registry. | ||
| - `jtt808` — JT/T 808 device gateway (`transcodegroup/jtt808-server`), handles terminal connections and forwards to `maintain`. | ||
| - `video` — SRS + RTP streaming stack for live video, playback, talk, and monitor. | ||
| - `nginx` — Reverse proxy and static frontend host for the `track` web app. | ||
|
|
||
| 2. **Bus** | ||
| - `gateway_808_2019` — JT/T 808 gateway for the Bus product (`transcodegroup/gateway-808-2019`). | ||
| - `gateway_web` — Spring Boot web backend (`transcodegroup/gateway-web`). | ||
| - `gateway_dispatch` — Spring Boot scheduler (`transcodegroup/gateway-dispatch`). | ||
| - `gateway_jsatl12` — Active-safety alarm file gateway (`transcodegroup/gateway-jsatl12`). | ||
| - `nginx` — Reverse proxy and static frontend host for the `bus` web app. | ||
|
|
||
| Both products share the same infrastructure services but use different MySQL databases (`maintain` vs `cbus`). | ||
|
|
||
| ### Configuration and Data Layout | ||
|
|
||
| - `/home/docker` — This repository. | ||
| - `/home/docker-compose/compose.yaml` — Deployment-specific composition, created from an `examples/` template. | ||
| - `/home/docker-compose/.env` — Runtime overrides, created from `default.env`. | ||
| - `/data` — All persistent runtime data (MySQL data, logs, uploaded files, nginx html, SRS recordings, etc.). | ||
| - `/home/docker-compose/bus-override/` and `/home/docker-compose/track-override/` — Frontend override directories. Files here are copied into `/data/nginx/html/` when nginx starts. Use these to override `_app.config.js`, `index-seo.html`, favicons, and logos instead of editing `/data/nginx/html/` directly. | ||
| - `/home/docker-compose/token/` — JWT RSA key pairs and `ip2region.xdb`. | ||
| - `/home/docker-compose/opt/` — Optional host binaries mounted into containers, e.g. `ffmpeg`, `ffprobe`, and `ifv2mp4` for the jtt808 video-conversion tools. | ||
|
|
||
| ### MySQL Initialization | ||
|
|
||
| `mysql8/compose.yml` mounts `mysql8/initdb/` into `/docker-entrypoint-initdb.d`. Scripts run once on the first container startup, before the application connects. The directory is split into table-definition and seed-data files per logical database (`maintain`, `cbus`, etc.). When altering these schemas, update the corresponding seed `INSERT`s and validate with `scripts/check-init-sql.js`. | ||
|
|
||
| ### Networking and Service Discovery | ||
|
|
||
| Services within the same compose project resolve each other by service name. Common aliases used in environment variables: | ||
|
|
||
| - `mysql8`, `redis`, `mongodb`, `rabbitmq`, `minio`, `nginx`, `jtt808`, `maintain`, `srs`, `rtp`. | ||
|
|
||
| External access is exposed through `nginx` (ports 80/443 by default) and, for device protocols, directly through `jtt808` / `gateway_808_2019` / `video` RTP ports. | ||
|
|
||
| ### Frontend Deployment | ||
|
|
||
| Frontend artifacts are not built here. They are downloaded from CI artifacts using `scripts/teamcity-download-artifact.sh` or deployed from tagged GitHub releases using `scripts/distar-beta-deploy.sh`. See `scripts/README.md` and `jtt808/README.md` for exact commands. | ||
|
|
||
| ## Review Rules | ||
|
|
||
| When a pull request changes the following version variables in `default.env` / `default.en.env`, the reviewer must confirm that the corresponding MySQL initialization scripts under `mysql8/initdb/` are still consistent with the new application version: | ||
|
|
||
| - `BUS_WEB_VERSION` — check `mysql8/initdb/03-cbus-init-table.sql` and `mysql8/initdb/04-cbus-init-data.sql`. | ||
| - `TRACK_MAINTAIN_VERSION` — check `mysql8/initdb/01-maintain-init-table.sql` and `mysql8/initdb/02-maintain-init-data.sql`. | ||
|
|
||
| If the SQL init files are modified, run `node scripts/check-init-sql.js` to catch column/value mismatches. Any compose change must pass `docker compose config`. | ||
|
|
||
| These rules are also encoded in `.github/copilot-instructions.md` and `.github/pull_request_template.md`. | ||
|
|
||
| ## Important Notes | ||
|
|
||
| - Most compose fragments rely on environment variables marked `{?required}` in their image tags or env vars. Always run `docker compose config` before `up` to catch missing values. | ||
| - MySQL init scripts only execute on a fresh data volume. To re-run them you must wipe `/data/mysql8/data`, which destroys all data. | ||
| - The `100-init-partition-table.sql` script uses `USE maintain;` and `USE cbus;` to create monthly partitions for large tables (position, alarm, depart_arrive). | ||
| - HTTPS deployments need a valid certificate path in `.env`. For testing, use the built-in placeholder cert at `${DOCKER_DIR}/nginx/ssl/placeholder`. | ||
| - GitHub Actions in `.github/workflows/` only manage project board automation and PR author assignment; they do not build or test this repo. | ||
|
|
||
| ## Related Documentation | ||
|
|
||
| - `README.md` / `README.en.md` — Deployment quick-start. | ||
| - `scripts/README.md` — Frontend artifact download and distar deployment scripts. | ||
| - `jtt808/README.md` — FFmpeg and IFV-to-MP4 tooling setup. | ||
| - `default.env` / `default.en.env` — Full list of configurable environment variables. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,186 @@ | ||
| #!/usr/bin/env node | ||
| /** | ||
| * 校验 mysql8/initdb 下所有初始化 SQL 的 INSERT VALUES 数量是否与对应 CREATE TABLE 的列数一致。 | ||
| * 用法:node scripts/check-init-sql.js | ||
| */ | ||
|
|
||
| const fs = require("fs"); | ||
| const path = require("path"); | ||
|
|
||
| const base = "mysql8/initdb"; | ||
| const files = fs.readdirSync(base).filter(f => f.endsWith(".sql")).sort().map(f => path.join(base, f)); | ||
|
|
||
| // schema.table -> column count | ||
| const tables = {}; | ||
| const inserts = []; | ||
|
|
||
| function norm(name) { | ||
| return name.replace(/`/g, "").toLowerCase(); | ||
| } | ||
|
|
||
| function getDb(line, currentDb) { | ||
| const m = line.match(/^USE\s+`?(\w+)`?\s*;/i); | ||
| return m ? norm(m[1]) : currentDb; | ||
| } | ||
|
|
||
| // Split body by top-level commas (not inside parentheses or quotes) | ||
| function splitTopLevel(text) { | ||
| const parts = []; | ||
| let current = ""; | ||
| let depth = 0; | ||
| for (let i = 0; i < text.length; i++) { | ||
| const ch = text[i]; | ||
| if (ch === "'") { | ||
| current += ch; | ||
| i++; | ||
| while (i < text.length && text[i] !== "'") { | ||
| if (text[i] === "\\" && i + 1 < text.length) { | ||
| current += text[i++]; | ||
| } | ||
| current += text[i++]; | ||
| } | ||
| if (i < text.length) current += text[i]; | ||
| } else if (ch === "(") { | ||
| depth++; | ||
| current += ch; | ||
| } else if (ch === ")") { | ||
| depth--; | ||
| current += ch; | ||
| } else if (ch === "," && depth === 0) { | ||
| parts.push(current.trim()); | ||
| current = ""; | ||
| } else { | ||
| current += ch; | ||
| } | ||
| } | ||
| if (current.trim()) parts.push(current.trim()); | ||
| return parts; | ||
| } | ||
|
|
||
| function defaultDb(file) { | ||
| const baseName = path.basename(file); | ||
| if (baseName.includes("maintain")) return "maintain"; | ||
| if (baseName.includes("cbus")) return "cbus"; | ||
| if (baseName.includes("gps")) return "gps"; | ||
| if (baseName.includes("analytics")) return "analytics"; | ||
| return null; | ||
| } | ||
|
|
||
| files.forEach(file => { | ||
| const text = fs.readFileSync(file, "utf-8"); | ||
| const lines = text.split(/\r?\n/); | ||
|
|
||
| let currentDb = defaultDb(file); | ||
|
|
||
| // Parse CREATE TABLE | ||
| const createRe = /CREATE\s+TABLE\s+(?:IF\s+NOT\s+EXISTS\s+)?([\w`.]+)\s*\((.*?)\)\s+ENGINE/sgi; | ||
| let m; | ||
| while ((m = createRe.exec(text)) !== null) { | ||
| const tableName = norm(m[1]); | ||
| const schema = tableName.includes(".") ? tableName.split(".")[0] : currentDb; | ||
| const shortName = tableName.includes(".") ? tableName.split(".")[1] : tableName; | ||
| const key = schema ? `${schema}.${shortName}` : shortName; | ||
| const body = m[2]; | ||
| const parts = splitTopLevel(body); | ||
| let colCount = 0; | ||
| for (const s of parts) { | ||
| if (!s) continue; | ||
| if (/^(PRIMARY\s+KEY|UNIQUE\s+KEY|UNIQUE\s+INDEX|KEY|INDEX|CONSTRAINT|FOREIGN\s+KEY)\b/i.test(s)) continue; | ||
| if (/^`\w+`\s+\w+/.test(s)) { | ||
| colCount++; | ||
| } | ||
| } | ||
| tables[key] = colCount; | ||
| } | ||
|
|
||
| // Parse INSERT INTO (only without explicit column list) | ||
| for (let i = 0; i < lines.length; i++) { | ||
| const line = lines[i]; | ||
| currentDb = getDb(line, currentDb); | ||
|
|
||
| const match = line.match(/^INSERT\s+INTO\s+([\w`.]+)\s+VALUES\s+(.*)/i); | ||
| if (!match) continue; | ||
| const tableName = norm(match[1]); | ||
| const schema = tableName.includes(".") ? tableName.split(".")[0] : currentDb; | ||
| const shortName = tableName.includes(".") ? tableName.split(".")[1] : tableName; | ||
| const key = schema ? `${schema}.${shortName}` : shortName; | ||
|
|
||
| let rest = match[2]; | ||
| let j = i; | ||
| while (!rest.trim().endsWith(";") && j + 1 < lines.length) { | ||
| j++; | ||
| rest += " " + lines[j]; | ||
| currentDb = getDb(lines[j], currentDb); | ||
| } | ||
| rest = rest.trim().replace(/;$/, ""); | ||
|
|
||
| // Extract top-level parentheses groups | ||
| const groups = []; | ||
| let start = -1; | ||
| let depth = 0; | ||
| for (let k = 0; k < rest.length; k++) { | ||
| const ch = rest[k]; | ||
| if (ch === "'") { | ||
| k++; | ||
| while (k < rest.length && rest[k] !== "'") { | ||
| if (rest[k] === "\\" && k + 1 < rest.length) k++; | ||
| k++; | ||
| } | ||
| } else if (ch === "(") { | ||
| if (depth === 0) start = k; | ||
| depth++; | ||
| } else if (ch === ")") { | ||
| depth--; | ||
| if (depth === 0 && start !== -1) { | ||
| groups.push(rest.slice(start + 1, k)); | ||
| start = -1; | ||
| } | ||
| } | ||
| } | ||
|
|
||
| for (const values of groups) { | ||
| let count = 1; | ||
| depth = 0; | ||
| for (let k = 0; k < values.length; k++) { | ||
| const ch = values[k]; | ||
| if (ch === "'") { | ||
| k++; | ||
| while (k < values.length && values[k] !== "'") { | ||
| if (values[k] === "\\" && k + 1 < values.length) k++; | ||
| k++; | ||
| } | ||
| } else if (ch === "(") { | ||
| depth++; | ||
| } else if (ch === ")") { | ||
| depth--; | ||
| } else if (ch === "," && depth === 0) { | ||
| count++; | ||
| } | ||
| } | ||
| inserts.push({file, line: j + 1, table: key, vcount: count}); | ||
| } | ||
| } | ||
| }); | ||
|
|
||
| const mismatches = inserts.filter(ins => tables[ins.table] && tables[ins.table] !== ins.vcount); | ||
| const grouped = {}; | ||
| mismatches.forEach(ins => { | ||
| const key = `${ins.file}:${ins.line}:${ins.table}`; | ||
| if (!grouped[key]) grouped[key] = ins; | ||
| }); | ||
|
|
||
| if (Object.keys(grouped).length) { | ||
| console.log("发现列数不匹配的 INSERT:"); | ||
| Object.values(grouped).sort((a,b)=>`${a.file}:${a.line}`.localeCompare(`${b.file}:${b.line}`)).forEach(ins => { | ||
| console.log(` ${ins.file}:${ins.line} 表=${ins.table} 列数=${tables[ins.table]} VALUES数=${ins.vcount}`); | ||
| }); | ||
| } else { | ||
| console.log("未发现列数不匹配的 INSERT。"); | ||
| } | ||
|
|
||
| const insertedTables = new Set(inserts.map(i => i.table)); | ||
| const unknown = [...insertedTables].filter(t => !tables[t]); | ||
| if (unknown.length) { | ||
| console.log("\n以下 INSERT 找不到对应 CREATE TABLE(可能跨库/表名解析问题):"); | ||
| unknown.sort().forEach(t => console.log(` ${t}`)); | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.