refactor: restore root layout and split mysql config
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
.git
|
||||
.DS_Store
|
||||
delivery/.env
|
||||
.env
|
||||
__pycache__
|
||||
*.pyc
|
||||
*.pyo
|
||||
|
||||
@@ -1,4 +1,8 @@
|
||||
MYSQL_HOST=your.mysql.host
|
||||
MYSQL_PORT=3306
|
||||
MYSQL_ROOT_USER=root
|
||||
MYSQL_ROOT_PASSWORD=mobilemodels_root_change_me
|
||||
MYSQL_DATABASE=mobilemodels
|
||||
MYSQL_READER_USER=mobilemodels_reader
|
||||
MYSQL_READER_PASSWORD=mobilemodels_reader_change_me
|
||||
MYSQL_AUTO_LOAD=0
|
||||
@@ -9,10 +9,7 @@ RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends git ca-certificates default-mysql-client \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY workspace /app/workspace
|
||||
COPY delivery /app/delivery
|
||||
|
||||
WORKDIR /app/delivery
|
||||
COPY . /app
|
||||
|
||||
EXPOSE 8123
|
||||
|
||||
46
README.md
46
README.md
@@ -1,37 +1,45 @@
|
||||
# MobileModels Workspace
|
||||
# 手机品牌型号汇总
|
||||
|
||||
当前仓库按两层结构组织:
|
||||
当前项目以根目录作为统一入口,支持通过 Docker Compose 直接启动设备查询、数据管理和 MySQL 服务。
|
||||
|
||||
- `workspace/`
|
||||
- 工作空间
|
||||
- 存放上游原始数据、补充资料和历史变更文件
|
||||
- `delivery/`
|
||||
- 交付物
|
||||
- 存放可直接运行的 Docker Compose 项目、Web 页面、MySQL schema 和交付文档
|
||||
|
||||
## 使用方式
|
||||
|
||||
进入交付目录启动:
|
||||
## 启动方式
|
||||
|
||||
```bash
|
||||
cd delivery
|
||||
docker compose up --build -d
|
||||
```
|
||||
|
||||
如需本地测试 MySQL,一起叠加测试配置启动:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.test.yml up --build -d
|
||||
```
|
||||
|
||||
页面入口:
|
||||
|
||||
- `http://127.0.0.1:8123/web/device_query.html`
|
||||
- `http://127.0.0.1:8123/web/brand_management.html`
|
||||
- `http://127.0.0.1:8123/web/device_query.html?view=docs`
|
||||
|
||||
## 目录说明
|
||||
## 目录结构
|
||||
|
||||
```text
|
||||
workspace/ 原始数据与工作空间
|
||||
delivery/ 可交付运行物
|
||||
workspace/ 上游原始数据、补充资料与历史文件
|
||||
dist/ 构建产物与 MySQL seed
|
||||
docs/ 项目文档
|
||||
sql/ MySQL schema
|
||||
tools/ 构建、同步、导入与服务脚本
|
||||
web/ 页面与静态资源
|
||||
```
|
||||
|
||||
更多交付说明见:
|
||||
## 说明
|
||||
|
||||
- [delivery/README.md](delivery/README.md)
|
||||
- [delivery/docs/README.md](delivery/docs/README.md)
|
||||
- `workspace/` 用于存放原始数据工作区
|
||||
- `docker-compose.yml`、`Dockerfile`、`tools/` 都位于项目主目录
|
||||
- 默认主配置面向远程 MySQL
|
||||
- `docker-compose.test.yml` 中的 MySQL 仅用于本地测试
|
||||
- 上游原始 git 同步、索引构建和 MySQL 刷新都在容器内完成
|
||||
|
||||
更多说明见:
|
||||
|
||||
- [docs/README.md](docs/README.md)
|
||||
- [docs/web-ui.md](docs/web-ui.md)
|
||||
|
||||
47
README_en.md
47
README_en.md
@@ -1,30 +1,45 @@
|
||||
# MobileModels Workspace
|
||||
# MobileModels
|
||||
|
||||
This repository is organized into two layers:
|
||||
The project now uses the repository root as the single runtime entry and can be started directly with Docker Compose.
|
||||
|
||||
- `workspace/`
|
||||
- source workspace
|
||||
- upstream raw data, notes, and historical files
|
||||
- `delivery/`
|
||||
- delivery artifact
|
||||
- a ready-to-run Docker Compose project with web UI, MySQL schema, and docs
|
||||
|
||||
## Usage
|
||||
|
||||
Run from the delivery directory:
|
||||
## Run
|
||||
|
||||
```bash
|
||||
cd delivery
|
||||
docker compose up --build -d
|
||||
```
|
||||
|
||||
If you want a local test MySQL together with the app:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.test.yml up --build -d
|
||||
```
|
||||
|
||||
Entry pages:
|
||||
|
||||
- `http://127.0.0.1:8123/web/device_query.html`
|
||||
- `http://127.0.0.1:8123/web/brand_management.html`
|
||||
- `http://127.0.0.1:8123/web/device_query.html?view=docs`
|
||||
|
||||
More delivery details:
|
||||
## Structure
|
||||
|
||||
- [delivery/README.md](delivery/README.md)
|
||||
- [delivery/docs/README.md](delivery/docs/README.md)
|
||||
```text
|
||||
workspace/ upstream raw data, notes, and history files
|
||||
dist/ build outputs and MySQL seed
|
||||
docs/ project docs
|
||||
sql/ MySQL schema
|
||||
tools/ build, sync, import, and service scripts
|
||||
web/ UI pages and static assets
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- `workspace/` stores the source workspace
|
||||
- `docker-compose.yml`, `Dockerfile`, and `tools/` live in the project root
|
||||
- the main compose file targets remote MySQL usage
|
||||
- `docker-compose.test.yml` provides a local MySQL only for testing
|
||||
- upstream git sync, index rebuild, and MySQL refresh run inside containers
|
||||
|
||||
More details:
|
||||
|
||||
- [docs/README.md](docs/README.md)
|
||||
- [docs/web-ui.md](docs/web-ui.md)
|
||||
|
||||
@@ -1,98 +0,0 @@
|
||||
# 手机品牌型号汇总
|
||||
[](https://creativecommons.org/licenses/by-nc-sa/4.0/)
|
||||
|
||||
这是交付目录,提供可直接运行的 Docker Compose 项目。
|
||||
|
||||
项目目标:
|
||||
|
||||
- 提供可直接交付的设备查询与数据管理系统
|
||||
- 通过容器内流程同步上游原始 git 数据
|
||||
- 在容器内重建索引并刷新 MySQL
|
||||
- 形成可独立部署的交付包
|
||||
|
||||
[English](README_en.md)
|
||||
|
||||
## 运行方式
|
||||
|
||||
在当前目录执行:
|
||||
|
||||
```bash
|
||||
docker compose up --build -d
|
||||
```
|
||||
|
||||
打开:
|
||||
|
||||
- `http://127.0.0.1:8123/web/device_query.html`
|
||||
- `http://127.0.0.1:8123/web/brand_management.html`
|
||||
|
||||
MySQL 也会一并启动:
|
||||
|
||||
- host: `127.0.0.1`
|
||||
- port: `3306`
|
||||
- database: `mobilemodels`
|
||||
- reader user: `mobilemodels_reader`
|
||||
|
||||
如需自定义 MySQL 账号密码,可先复制 `.env.example` 为 `.env` 后再启动。
|
||||
|
||||
## 结构说明
|
||||
|
||||
```text
|
||||
dist/ 构建产物与 MySQL seed
|
||||
docs/ 交付文档
|
||||
sql/ MySQL schema
|
||||
tools/ 容器内构建与同步脚本
|
||||
web/ 页面与静态资源
|
||||
```
|
||||
|
||||
上游原始数据不放在本目录,而是从仓库根目录下的 `workspace/` 读取。
|
||||
|
||||
容器启动后会自动完成:
|
||||
|
||||
- 初始化 `workspace/` 与 `dist/` 的运行期数据目录
|
||||
- 从 `workspace/brands` 构建设备索引
|
||||
- 导出 MySQL seed
|
||||
- 加载 MySQL schema 与 seed
|
||||
- 启动 Web 页面与 API
|
||||
|
||||
更多说明参见:
|
||||
|
||||
- [docs/README.md](docs/README.md)
|
||||
- [docs/web-ui.md](docs/web-ui.md)
|
||||
- [docs/mysql-query-design.md](docs/mysql-query-design.md)
|
||||
- [docs/device-mapper.md](docs/device-mapper.md)
|
||||
## 工作空间
|
||||
|
||||
原始数据与补充资料位于仓库根目录下的 `workspace/`:
|
||||
|
||||
- `../workspace/brands`
|
||||
- `../workspace/misc`
|
||||
- `../workspace/CHANGELOG.md`
|
||||
- `../workspace/CHANGELOG_en.md`
|
||||
|
||||
### 2016 年 3 月
|
||||
- 小米手机型号汇总发布至 [小米社区](http://bbs.xiaomi.cn/t-12641411)(帖子已失效)。
|
||||
|
||||
### 2016 年 2 月
|
||||
- 我开始汇总一些国内手机品牌的型号,「手机品牌型号汇总」的雏形诞生。
|
||||
|
||||
## 参考资料
|
||||
|
||||
- [电信设备终端网](http://zd.taf.org.cn)
|
||||
- [产品认证证书查询](http://webdata.cqccms.com.cn/webdata/query/CCCCerti.do)
|
||||
- [工业和信息化部政务服务平台](https://ythzxfw.miit.gov.cn/resultQuery)
|
||||
- [产品库-中国电信天翼终端信息平台](http://surfing.tydevice.com/)
|
||||
- [Google Play 支持的设备](http://storage.googleapis.com/play_public/supported_devices.html)
|
||||
- [Wi-Fi Alliance](https://www.wi-fi.org)
|
||||
- [Bluetooth Launch Studio](https://launchstudio.bluetooth.com/Listings/Search)
|
||||
- [Xiaomi Firmware Updater](https://xiaomifirmwareupdater.com/)
|
||||
- [Huawei Open Source Release Center](https://consumer.huawei.com/en/opensource/)
|
||||
- [ReaMEIZU](https://reameizu.com/)
|
||||
- [The Apple Wiki](https://theapplewiki.com/)
|
||||
- [ipsw.me](https://ipsw.me)
|
||||
- [XDA Developers](https://www.xda-developers.com)
|
||||
- [Huawei Firmware Database](https://pro-teammt.ru/en/online-firmware-database-ru/)
|
||||
- [XSMS IMEI 数据库](http://xsms.com.ua/phone/imei/all/1)
|
||||
- [Android Dumps](https://dumps.tadiphone.dev/dumps)
|
||||
- [Lenovo Android タブレット一覧](https://idomizu.dev/archives/20150)
|
||||
|
||||
以及各品牌官网、论坛、微博等,恕不一一列出
|
||||
@@ -1,61 +0,0 @@
|
||||
# Mobile Models
|
||||
[](https://creativecommons.org/licenses/by-nc-sa/4.0/)
|
||||
|
||||
This is the delivery directory and contains the ready-to-run Docker Compose project.
|
||||
|
||||
Project goals:
|
||||
|
||||
- provide a ready-to-deliver device query system
|
||||
- sync upstream raw git data inside containers
|
||||
- rebuild index and refresh MySQL inside containers
|
||||
- keep deployment content independent from workspace files
|
||||
|
||||
## Run
|
||||
|
||||
The project ships with device query and data management pages and now runs through `docker compose`:
|
||||
|
||||
```bash
|
||||
docker compose up --build -d
|
||||
```
|
||||
|
||||
Open:
|
||||
|
||||
- `http://127.0.0.1:8123/web/device_query.html`
|
||||
- `http://127.0.0.1:8123/web/brand_management.html`
|
||||
|
||||
MySQL is started together with the stack:
|
||||
|
||||
- host: `127.0.0.1`
|
||||
- port: `3306`
|
||||
- database: `mobilemodels`
|
||||
- reader user: `mobilemodels_reader`
|
||||
|
||||
If you want custom MySQL credentials, copy `.env.example` to `.env` before startup.
|
||||
|
||||
## Structure
|
||||
|
||||
```text
|
||||
dist/ build outputs and MySQL seed
|
||||
docs/ delivery docs
|
||||
sql/ MySQL schema
|
||||
tools/ container-side build and sync scripts
|
||||
web/ delivered web pages
|
||||
```
|
||||
|
||||
Raw source files are read from the repository root `workspace/` directory.
|
||||
|
||||
More details:
|
||||
|
||||
- [docs/README.md](docs/README.md)
|
||||
- [docs/web-ui.md](docs/web-ui.md)
|
||||
- [docs/mysql-query-design.md](docs/mysql-query-design.md)
|
||||
- [docs/device-mapper.md](docs/device-mapper.md)
|
||||
|
||||
## Workspace
|
||||
|
||||
Raw data and historical files live under the repository root `workspace/`:
|
||||
|
||||
- `../workspace/brands`
|
||||
- `../workspace/misc`
|
||||
- `../workspace/CHANGELOG.md`
|
||||
- `../workspace/CHANGELOG_en.md`
|
||||
@@ -1,11 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Shared path helpers for the workspace/delivery project layout."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
DELIVERY_ROOT = Path(__file__).resolve().parent.parent
|
||||
PROJECT_ROOT = DELIVERY_ROOT.parent
|
||||
WORKSPACE_ROOT = PROJECT_ROOT / "workspace"
|
||||
@@ -22,31 +22,14 @@ services:
|
||||
init: true
|
||||
|
||||
mobilemodels:
|
||||
build:
|
||||
context: ..
|
||||
dockerfile: delivery/Dockerfile
|
||||
container_name: mobilemodels-web
|
||||
working_dir: /app/delivery
|
||||
environment:
|
||||
MOBILEMODELS_DATA_ROOT: /data
|
||||
MYSQL_HOST: mysql
|
||||
MYSQL_PORT: 3306
|
||||
MYSQL_DATABASE: ${MYSQL_DATABASE:-mobilemodels}
|
||||
MYSQL_ROOT_USER: root
|
||||
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-mobilemodels_root}
|
||||
MYSQL_READER_USER: ${MYSQL_READER_USER:-mobilemodels_reader}
|
||||
MYSQL_READER_PASSWORD: ${MYSQL_READER_PASSWORD:-mobilemodels_reader_change_me}
|
||||
MYSQL_AUTO_LOAD: 1
|
||||
depends_on:
|
||||
mysql:
|
||||
condition: service_healthy
|
||||
command: ["sh", "tools/container_start.sh"]
|
||||
ports:
|
||||
- "8123:8123"
|
||||
volumes:
|
||||
- mobilemodels_app_data:/data
|
||||
restart: unless-stopped
|
||||
init: true
|
||||
|
||||
volumes:
|
||||
mobilemodels_app_data:
|
||||
mobilemodels_mysql_data:
|
||||
29
docker-compose.yml
Normal file
29
docker-compose.yml
Normal file
@@ -0,0 +1,29 @@
|
||||
services:
|
||||
mobilemodels:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
container_name: mobilemodels-web
|
||||
working_dir: /app
|
||||
environment:
|
||||
MOBILEMODELS_DATA_ROOT: /data
|
||||
MYSQL_HOST: ${MYSQL_HOST:-host.docker.internal}
|
||||
MYSQL_PORT: ${MYSQL_PORT:-3306}
|
||||
MYSQL_DATABASE: ${MYSQL_DATABASE:-mobilemodels}
|
||||
MYSQL_ROOT_USER: ${MYSQL_ROOT_USER:-root}
|
||||
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-mobilemodels_root}
|
||||
MYSQL_READER_USER: ${MYSQL_READER_USER:-mobilemodels_reader}
|
||||
MYSQL_READER_PASSWORD: ${MYSQL_READER_PASSWORD:-mobilemodels_reader_change_me}
|
||||
MYSQL_AUTO_LOAD: ${MYSQL_AUTO_LOAD:-0}
|
||||
command: ["sh", "tools/container_start.sh"]
|
||||
ports:
|
||||
- "8123:8123"
|
||||
volumes:
|
||||
- mobilemodels_app_data:/data
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
restart: unless-stopped
|
||||
init: true
|
||||
|
||||
volumes:
|
||||
mobilemodels_app_data:
|
||||
@@ -2,12 +2,18 @@
|
||||
|
||||
## 启动方式
|
||||
|
||||
在 `delivery/` 目录执行:
|
||||
在项目根目录执行:
|
||||
|
||||
```bash
|
||||
docker compose up --build -d
|
||||
```
|
||||
|
||||
如果要连本地测试 MySQL:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.test.yml up --build -d
|
||||
```
|
||||
|
||||
如需自定义环境变量:
|
||||
|
||||
```bash
|
||||
@@ -32,16 +38,16 @@ docker compose down -v
|
||||
- `http://127.0.0.1:8123/web/brand_management.html`:数据管理
|
||||
- `http://127.0.0.1:8123/web/device_query.html?view=docs`:相关文档
|
||||
|
||||
整个功能栈统一运行在 Docker Compose 中,不再依赖本地 Python 或本地 MySQL。
|
||||
整个功能栈统一运行在 Docker Compose 中,不再依赖本地 Python。
|
||||
|
||||
原始数据工作空间位于仓库根目录下的 `workspace/`,交付物位于 `delivery/`。
|
||||
原始数据工作空间位于项目内的 `workspace/` 目录。
|
||||
|
||||
## 启动后自动完成的动作
|
||||
|
||||
- 从 `workspace/brands` 构建设备索引
|
||||
- 生成 `dist/device_index.json`
|
||||
- 导出 MySQL seed 文件
|
||||
- 加载 MySQL schema 与 seed 数据
|
||||
- 如开启 `MYSQL_AUTO_LOAD=1`,则加载 MySQL schema 与 seed 数据
|
||||
- 启动 Web 页面与 API 服务
|
||||
|
||||
## MySQL 默认连接
|
||||
@@ -53,6 +59,15 @@ docker compose down -v
|
||||
|
||||
如需自定义账号密码,请使用 `.env` 覆盖默认值。
|
||||
|
||||
## MySQL 模式
|
||||
|
||||
- 主配置 `docker-compose.yml`
|
||||
- 面向远程 MySQL
|
||||
- 默认不自动装载 schema/seed
|
||||
- 测试配置 `docker-compose.test.yml`
|
||||
- 额外启动一个本地测试 MySQL
|
||||
- 应用容器会自动把数据加载进去
|
||||
|
||||
## 设备查询
|
||||
|
||||
页面顶部统一提供三个导航入口:
|
||||
@@ -1,12 +1,17 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
cd /app/delivery
|
||||
cd /app
|
||||
|
||||
sh tools/init_runtime_data.sh
|
||||
|
||||
python3 tools/device_mapper.py build
|
||||
python3 tools/export_mysql_seed.py
|
||||
python3 tools/load_mysql_seed.py
|
||||
|
||||
if [ "${MYSQL_AUTO_LOAD:-0}" = "1" ]; then
|
||||
python3 tools/load_mysql_seed.py
|
||||
else
|
||||
echo "Skipping MySQL load because MYSQL_AUTO_LOAD=${MYSQL_AUTO_LOAD:-0}"
|
||||
fi
|
||||
|
||||
exec python3 tools/web_server.py --host 0.0.0.0 --port 8123
|
||||
@@ -12,7 +12,7 @@ from datetime import date
|
||||
from pathlib import Path
|
||||
from typing import Dict, Iterable, List, Optional, Set
|
||||
|
||||
from project_layout import DELIVERY_ROOT, WORKSPACE_ROOT
|
||||
from project_layout import PROJECT_ROOT, WORKSPACE_ROOT
|
||||
|
||||
ENTRY_RE = re.compile(r"^\*\*(.+?)\*\*\s*$")
|
||||
VARIANT_RE = re.compile(r"^\s*((?:`[^`]+`\s*)+):\s*(.+?)\s*$")
|
||||
@@ -743,7 +743,7 @@ def main() -> None:
|
||||
if args.command == "build":
|
||||
output_path: Path = args.output
|
||||
if not output_path.is_absolute():
|
||||
output_path = DELIVERY_ROOT / output_path
|
||||
output_path = PROJECT_ROOT / output_path
|
||||
export_index(records, output_path)
|
||||
print(f"Built index: {output_path}")
|
||||
print(f"Total records: {len(records)}")
|
||||
@@ -16,7 +16,7 @@ from device_mapper import (
|
||||
normalize_text,
|
||||
resolve_parent_brand,
|
||||
)
|
||||
from project_layout import DELIVERY_ROOT, WORKSPACE_ROOT
|
||||
from project_layout import PROJECT_ROOT, WORKSPACE_ROOT
|
||||
|
||||
|
||||
LEGACY_CODE_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9,._/+\\-]{1,63}$")
|
||||
@@ -200,7 +200,7 @@ def parse_args() -> argparse.Namespace:
|
||||
def main() -> int:
|
||||
args = parse_args()
|
||||
repo_root = args.repo_root.resolve()
|
||||
output_path = args.output if args.output.is_absolute() else DELIVERY_ROOT / args.output
|
||||
output_path = args.output if args.output.is_absolute() else PROJECT_ROOT / args.output
|
||||
|
||||
records = build_records(repo_root)
|
||||
device_record_count = len(records)
|
||||
@@ -61,7 +61,7 @@ init_path() {
|
||||
|
||||
for rel_path in \
|
||||
workspace \
|
||||
delivery/dist
|
||||
dist
|
||||
do
|
||||
init_path "$rel_path"
|
||||
done
|
||||
@@ -10,7 +10,7 @@ import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
from project_layout import DELIVERY_ROOT
|
||||
from project_layout import PROJECT_ROOT
|
||||
|
||||
|
||||
def mysql_env(password: str) -> dict[str, str]:
|
||||
@@ -130,8 +130,8 @@ def parse_args() -> argparse.Namespace:
|
||||
|
||||
def main() -> int:
|
||||
args = parse_args()
|
||||
schema_path = args.schema if args.schema.is_absolute() else DELIVERY_ROOT / args.schema
|
||||
seed_path = args.seed if args.seed.is_absolute() else DELIVERY_ROOT / args.seed
|
||||
schema_path = args.schema if args.schema.is_absolute() else PROJECT_ROOT / args.schema
|
||||
seed_path = args.seed if args.seed.is_absolute() else PROJECT_ROOT / args.seed
|
||||
|
||||
wait_for_mysql(args.user, args.password, args.host, args.port, args.wait_timeout)
|
||||
|
||||
9
tools/project_layout.py
Normal file
9
tools/project_layout.py
Normal file
@@ -0,0 +1,9 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Shared path helpers for the project layout."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
||||
WORKSPACE_ROOT = PROJECT_ROOT / "workspace"
|
||||
@@ -11,7 +11,7 @@ import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from project_layout import DELIVERY_ROOT, PROJECT_ROOT, WORKSPACE_ROOT
|
||||
from project_layout import PROJECT_ROOT, WORKSPACE_ROOT
|
||||
|
||||
DEFAULT_REPO_URL = "https://github.com/KHwang9883/MobileModels.git"
|
||||
DEFAULT_BRANCH = "master"
|
||||
@@ -67,7 +67,7 @@ def build_index(output_path: str) -> None:
|
||||
run(
|
||||
[
|
||||
sys.executable,
|
||||
str(DELIVERY_ROOT / "tools/device_mapper.py"),
|
||||
str(PROJECT_ROOT / "tools/device_mapper.py"),
|
||||
"--repo-root",
|
||||
str(WORKSPACE_ROOT),
|
||||
"build",
|
||||
@@ -81,7 +81,7 @@ def export_mysql_seed(output_path: str) -> None:
|
||||
run(
|
||||
[
|
||||
sys.executable,
|
||||
str(DELIVERY_ROOT / "tools/export_mysql_seed.py"),
|
||||
str(PROJECT_ROOT / "tools/export_mysql_seed.py"),
|
||||
"--output",
|
||||
output_path,
|
||||
"--repo-root",
|
||||
@@ -94,7 +94,7 @@ def load_mysql_seed(seed_path: str) -> None:
|
||||
run(
|
||||
[
|
||||
sys.executable,
|
||||
str(DELIVERY_ROOT / "tools/load_mysql_seed.py"),
|
||||
str(PROJECT_ROOT / "tools/load_mysql_seed.py"),
|
||||
"--seed",
|
||||
seed_path,
|
||||
]
|
||||
@@ -14,20 +14,24 @@ from http import HTTPStatus
|
||||
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
|
||||
from project_layout import DELIVERY_ROOT, PROJECT_ROOT, WORKSPACE_ROOT
|
||||
from project_layout import PROJECT_ROOT, WORKSPACE_ROOT
|
||||
from sync_upstream_mobilemodels import DEFAULT_BRANCH, DEFAULT_REPO_URL
|
||||
|
||||
|
||||
SYNC_SCRIPT = DELIVERY_ROOT / "tools/sync_upstream_mobilemodels.py"
|
||||
INDEX_PATH = DELIVERY_ROOT / "dist/device_index.json"
|
||||
MYSQL_SEED_PATH = DELIVERY_ROOT / "dist/mobilemodels_mysql_seed.sql"
|
||||
MYSQL_LOADER = DELIVERY_ROOT / "tools/load_mysql_seed.py"
|
||||
SYNC_SCRIPT = PROJECT_ROOT / "tools/sync_upstream_mobilemodels.py"
|
||||
INDEX_PATH = PROJECT_ROOT / "dist/device_index.json"
|
||||
MYSQL_SEED_PATH = PROJECT_ROOT / "dist/mobilemodels_mysql_seed.sql"
|
||||
MYSQL_LOADER = PROJECT_ROOT / "tools/load_mysql_seed.py"
|
||||
DATA_ROOT = Path(os.environ.get("MOBILEMODELS_DATA_ROOT", "/data"))
|
||||
SYNC_METADATA_PATH = DATA_ROOT / "state/sync_status.json"
|
||||
SYNC_LOCK = threading.Lock()
|
||||
NORMALIZE_RE = re.compile(r"[^0-9a-z\u4e00-\u9fff]+")
|
||||
|
||||
|
||||
def mysql_auto_load_enabled() -> bool:
|
||||
return os.environ.get("MYSQL_AUTO_LOAD", "0").strip().lower() in {"1", "true", "yes", "on"}
|
||||
|
||||
|
||||
def run_command(args: list[str]) -> subprocess.CompletedProcess[str]:
|
||||
return subprocess.run(
|
||||
args,
|
||||
@@ -180,30 +184,34 @@ def get_status_payload() -> dict[str, object]:
|
||||
mysql_database = os.environ.get("MYSQL_DATABASE", "mobilemodels")
|
||||
mysql_reader_user = os.environ.get("MYSQL_READER_USER", "")
|
||||
mysql_reader_password = os.environ.get("MYSQL_READER_PASSWORD", "")
|
||||
mysql_auto_load = mysql_auto_load_enabled()
|
||||
mysql_ready = False
|
||||
mysql_status = ""
|
||||
sync_metadata = read_sync_metadata()
|
||||
mysql_proc = run_command(["python3", str(MYSQL_LOADER), "--check-only", "--wait-timeout", "5"])
|
||||
if mysql_proc.returncode == 0:
|
||||
mysql_ready = True
|
||||
mysql_status = mysql_proc.stdout.strip() or "MySQL ready"
|
||||
if mysql_auto_load:
|
||||
mysql_proc = run_command(["python3", str(MYSQL_LOADER), "--check-only", "--wait-timeout", "5"])
|
||||
if mysql_proc.returncode == 0:
|
||||
mysql_ready = True
|
||||
mysql_status = mysql_proc.stdout.strip() or "MySQL ready"
|
||||
else:
|
||||
mysql_status = mysql_proc.stderr.strip() or mysql_proc.stdout.strip() or "MySQL unavailable"
|
||||
else:
|
||||
mysql_status = mysql_proc.stderr.strip() or mysql_proc.stdout.strip() or "MySQL unavailable"
|
||||
mysql_status = "MySQL auto load disabled"
|
||||
|
||||
return {
|
||||
"supports_upstream_sync": True,
|
||||
"storage_mode": "docker_volume",
|
||||
"project_root": str(PROJECT_ROOT),
|
||||
"workspace_root": str(WORKSPACE_ROOT),
|
||||
"delivery_root": str(DELIVERY_ROOT),
|
||||
"data_root": str(DATA_ROOT),
|
||||
"mysql_auto_load": mysql_auto_load,
|
||||
"upstream_repo_url": DEFAULT_REPO_URL,
|
||||
"upstream_branch": DEFAULT_BRANCH,
|
||||
"last_sync_time": sync_metadata.get("last_sync_time"),
|
||||
"last_upstream_commit": sync_metadata.get("last_upstream_commit"),
|
||||
"index_file": str(INDEX_PATH.relative_to(DELIVERY_ROOT)),
|
||||
"index_file": str(INDEX_PATH.relative_to(PROJECT_ROOT)),
|
||||
"index_mtime": index_mtime,
|
||||
"mysql_seed_file": str(MYSQL_SEED_PATH.relative_to(DELIVERY_ROOT)),
|
||||
"mysql_seed_file": str(MYSQL_SEED_PATH.relative_to(PROJECT_ROOT)),
|
||||
"mysql_seed_mtime": mysql_seed_mtime,
|
||||
"mysql_host": mysql_host,
|
||||
"mysql_port": mysql_port,
|
||||
@@ -227,13 +235,15 @@ def run_upstream_sync() -> dict[str, object]:
|
||||
if upstream_proc.returncode == 0 and upstream_proc.stdout.strip():
|
||||
upstream_commit = upstream_proc.stdout.split()[0]
|
||||
|
||||
proc = run_command([
|
||||
command = [
|
||||
"python3",
|
||||
str(SYNC_SCRIPT),
|
||||
"--build-index",
|
||||
"--export-mysql-seed",
|
||||
"--load-mysql",
|
||||
])
|
||||
]
|
||||
if mysql_auto_load_enabled():
|
||||
command.append("--load-mysql")
|
||||
proc = run_command(command)
|
||||
output = "\n".join(
|
||||
part for part in [proc.stdout.strip(), proc.stderr.strip()] if part
|
||||
).strip()
|
||||
@@ -245,18 +255,17 @@ def run_upstream_sync() -> dict[str, object]:
|
||||
"storage_mode": "docker_volume",
|
||||
"project_root": str(PROJECT_ROOT),
|
||||
"workspace_root": str(WORKSPACE_ROOT),
|
||||
"delivery_root": str(DELIVERY_ROOT),
|
||||
"data_root": str(DATA_ROOT),
|
||||
"upstream_repo_url": DEFAULT_REPO_URL,
|
||||
"upstream_branch": DEFAULT_BRANCH,
|
||||
"upstream_commit": upstream_commit,
|
||||
"last_sync_time": datetime.now().isoformat(timespec="seconds"),
|
||||
"last_upstream_commit": upstream_commit,
|
||||
"index_file": str(INDEX_PATH.relative_to(DELIVERY_ROOT)),
|
||||
"index_file": str(INDEX_PATH.relative_to(PROJECT_ROOT)),
|
||||
"index_mtime": datetime.fromtimestamp(INDEX_PATH.stat().st_mtime).isoformat(timespec="seconds")
|
||||
if INDEX_PATH.exists()
|
||||
else None,
|
||||
"mysql_seed_file": str(MYSQL_SEED_PATH.relative_to(DELIVERY_ROOT)),
|
||||
"mysql_seed_file": str(MYSQL_SEED_PATH.relative_to(PROJECT_ROOT)),
|
||||
"mysql_seed_mtime": datetime.fromtimestamp(MYSQL_SEED_PATH.stat().st_mtime).isoformat(timespec="seconds")
|
||||
if MYSQL_SEED_PATH.exists()
|
||||
else None,
|
||||
@@ -275,7 +284,7 @@ def run_upstream_sync() -> dict[str, object]:
|
||||
|
||||
class MobileModelsHandler(SimpleHTTPRequestHandler):
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(*args, directory=str(DELIVERY_ROOT), **kwargs)
|
||||
super().__init__(*args, directory=str(PROJECT_ROOT), **kwargs)
|
||||
|
||||
def guess_type(self, path: str) -> str:
|
||||
content_type = super().guess_type(path)
|
||||
@@ -396,7 +396,7 @@
|
||||
|
||||
<section id="syncTabPanel" class="manage-panel hidden">
|
||||
<h3 class="title">原始数据同步</h3>
|
||||
<p class="sub">从上游 `KHwang9883/MobileModels` 拉取原始 markdown 数据,并重建 `dist/device_index.json`、刷新 MySQL。请先使用 `docker compose up --build -d` 启动完整服务。</p>
|
||||
<p class="sub">从上游 `KHwang9883/MobileModels` 拉取原始 markdown 数据,并重建 `dist/device_index.json`。如已开启 MySQL 自动装载,也会同步刷新 MySQL。请先启动完整服务。</p>
|
||||
<div class="btns">
|
||||
<button id="syncUpstreamBtn" type="button" class="primary">开始同步原始数据</button>
|
||||
<button id="refreshSyncStatusBtn" type="button">刷新同步状态</button>
|
||||
@@ -520,7 +520,6 @@
|
||||
if (data.data_root) lines.push(`数据目录: ${data.data_root}`);
|
||||
if (data.project_root) lines.push(`项目目录: ${data.project_root}`);
|
||||
if (data.workspace_root) lines.push(`工作空间目录: ${data.workspace_root}`);
|
||||
if (data.delivery_root) lines.push(`交付目录: ${data.delivery_root}`);
|
||||
if (data.storage_mode) lines.push(`存储模式: ${data.storage_mode}`);
|
||||
if (data.upstream_repo_url) lines.push(`上游仓库: ${data.upstream_repo_url}`);
|
||||
if (data.upstream_branch) lines.push(`上游分支: ${data.upstream_branch}`);
|
||||
@@ -533,6 +532,7 @@
|
||||
if (data.mysql_host && data.mysql_port) lines.push(`MySQL 地址: ${data.mysql_host}:${data.mysql_port}`);
|
||||
if (data.mysql_database) lines.push(`MySQL 数据库: ${data.mysql_database}`);
|
||||
if (data.mysql_reader_user) lines.push(`MySQL 只读账号: ${data.mysql_reader_user}`);
|
||||
if (typeof data.mysql_auto_load === "boolean") lines.push(`MySQL 自动装载: ${data.mysql_auto_load ? "enabled" : "disabled"}`);
|
||||
if (typeof data.mysql_ready === "boolean") lines.push(`MySQL 状态: ${data.mysql_ready ? "ready" : "not ready"}`);
|
||||
if (data.mysql_status) lines.push(`MySQL 详情: ${data.mysql_status}`);
|
||||
if (data.output) {
|
||||
Reference in New Issue
Block a user