refactor: restore root layout and split mysql config
This commit is contained in:
@@ -1,6 +1,6 @@
|
|||||||
.git
|
.git
|
||||||
.DS_Store
|
.DS_Store
|
||||||
delivery/.env
|
.env
|
||||||
__pycache__
|
__pycache__
|
||||||
*.pyc
|
*.pyc
|
||||||
*.pyo
|
*.pyo
|
||||||
|
|||||||
@@ -1,4 +1,8 @@
|
|||||||
|
MYSQL_HOST=your.mysql.host
|
||||||
|
MYSQL_PORT=3306
|
||||||
|
MYSQL_ROOT_USER=root
|
||||||
MYSQL_ROOT_PASSWORD=mobilemodels_root_change_me
|
MYSQL_ROOT_PASSWORD=mobilemodels_root_change_me
|
||||||
MYSQL_DATABASE=mobilemodels
|
MYSQL_DATABASE=mobilemodels
|
||||||
MYSQL_READER_USER=mobilemodels_reader
|
MYSQL_READER_USER=mobilemodels_reader
|
||||||
MYSQL_READER_PASSWORD=mobilemodels_reader_change_me
|
MYSQL_READER_PASSWORD=mobilemodels_reader_change_me
|
||||||
|
MYSQL_AUTO_LOAD=0
|
||||||
@@ -9,10 +9,7 @@ RUN apt-get update \
|
|||||||
&& apt-get install -y --no-install-recommends git ca-certificates default-mysql-client \
|
&& apt-get install -y --no-install-recommends git ca-certificates default-mysql-client \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
COPY workspace /app/workspace
|
COPY . /app
|
||||||
COPY delivery /app/delivery
|
|
||||||
|
|
||||||
WORKDIR /app/delivery
|
|
||||||
|
|
||||||
EXPOSE 8123
|
EXPOSE 8123
|
||||||
|
|
||||||
46
README.md
46
README.md
@@ -1,37 +1,45 @@
|
|||||||
# MobileModels Workspace
|
# 手机品牌型号汇总
|
||||||
|
|
||||||
当前仓库按两层结构组织:
|
当前项目以根目录作为统一入口,支持通过 Docker Compose 直接启动设备查询、数据管理和 MySQL 服务。
|
||||||
|
|
||||||
- `workspace/`
|
## 启动方式
|
||||||
- 工作空间
|
|
||||||
- 存放上游原始数据、补充资料和历史变更文件
|
|
||||||
- `delivery/`
|
|
||||||
- 交付物
|
|
||||||
- 存放可直接运行的 Docker Compose 项目、Web 页面、MySQL schema 和交付文档
|
|
||||||
|
|
||||||
## 使用方式
|
|
||||||
|
|
||||||
进入交付目录启动:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd delivery
|
|
||||||
docker compose up --build -d
|
docker compose up --build -d
|
||||||
```
|
```
|
||||||
|
|
||||||
|
如需本地测试 MySQL,一起叠加测试配置启动:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f docker-compose.yml -f docker-compose.test.yml up --build -d
|
||||||
|
```
|
||||||
|
|
||||||
页面入口:
|
页面入口:
|
||||||
|
|
||||||
- `http://127.0.0.1:8123/web/device_query.html`
|
- `http://127.0.0.1:8123/web/device_query.html`
|
||||||
- `http://127.0.0.1:8123/web/brand_management.html`
|
- `http://127.0.0.1:8123/web/brand_management.html`
|
||||||
- `http://127.0.0.1:8123/web/device_query.html?view=docs`
|
- `http://127.0.0.1:8123/web/device_query.html?view=docs`
|
||||||
|
|
||||||
## 目录说明
|
## 目录结构
|
||||||
|
|
||||||
```text
|
```text
|
||||||
workspace/ 原始数据与工作空间
|
workspace/ 上游原始数据、补充资料与历史文件
|
||||||
delivery/ 可交付运行物
|
dist/ 构建产物与 MySQL seed
|
||||||
|
docs/ 项目文档
|
||||||
|
sql/ MySQL schema
|
||||||
|
tools/ 构建、同步、导入与服务脚本
|
||||||
|
web/ 页面与静态资源
|
||||||
```
|
```
|
||||||
|
|
||||||
更多交付说明见:
|
## 说明
|
||||||
|
|
||||||
- [delivery/README.md](delivery/README.md)
|
- `workspace/` 用于存放原始数据工作区
|
||||||
- [delivery/docs/README.md](delivery/docs/README.md)
|
- `docker-compose.yml`、`Dockerfile`、`tools/` 都位于项目主目录
|
||||||
|
- 默认主配置面向远程 MySQL
|
||||||
|
- `docker-compose.test.yml` 中的 MySQL 仅用于本地测试
|
||||||
|
- 上游原始 git 同步、索引构建和 MySQL 刷新都在容器内完成
|
||||||
|
|
||||||
|
更多说明见:
|
||||||
|
|
||||||
|
- [docs/README.md](docs/README.md)
|
||||||
|
- [docs/web-ui.md](docs/web-ui.md)
|
||||||
|
|||||||
47
README_en.md
47
README_en.md
@@ -1,30 +1,45 @@
|
|||||||
# MobileModels Workspace
|
# MobileModels
|
||||||
|
|
||||||
This repository is organized into two layers:
|
The project now uses the repository root as the single runtime entry and can be started directly with Docker Compose.
|
||||||
|
|
||||||
- `workspace/`
|
## Run
|
||||||
- source workspace
|
|
||||||
- upstream raw data, notes, and historical files
|
|
||||||
- `delivery/`
|
|
||||||
- delivery artifact
|
|
||||||
- a ready-to-run Docker Compose project with web UI, MySQL schema, and docs
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
Run from the delivery directory:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd delivery
|
|
||||||
docker compose up --build -d
|
docker compose up --build -d
|
||||||
```
|
```
|
||||||
|
|
||||||
|
If you want a local test MySQL together with the app:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f docker-compose.yml -f docker-compose.test.yml up --build -d
|
||||||
|
```
|
||||||
|
|
||||||
Entry pages:
|
Entry pages:
|
||||||
|
|
||||||
- `http://127.0.0.1:8123/web/device_query.html`
|
- `http://127.0.0.1:8123/web/device_query.html`
|
||||||
- `http://127.0.0.1:8123/web/brand_management.html`
|
- `http://127.0.0.1:8123/web/brand_management.html`
|
||||||
- `http://127.0.0.1:8123/web/device_query.html?view=docs`
|
- `http://127.0.0.1:8123/web/device_query.html?view=docs`
|
||||||
|
|
||||||
More delivery details:
|
## Structure
|
||||||
|
|
||||||
- [delivery/README.md](delivery/README.md)
|
```text
|
||||||
- [delivery/docs/README.md](delivery/docs/README.md)
|
workspace/ upstream raw data, notes, and history files
|
||||||
|
dist/ build outputs and MySQL seed
|
||||||
|
docs/ project docs
|
||||||
|
sql/ MySQL schema
|
||||||
|
tools/ build, sync, import, and service scripts
|
||||||
|
web/ UI pages and static assets
|
||||||
|
```
|
||||||
|
|
||||||
|
Notes:
|
||||||
|
|
||||||
|
- `workspace/` stores the source workspace
|
||||||
|
- `docker-compose.yml`, `Dockerfile`, and `tools/` live in the project root
|
||||||
|
- the main compose file targets remote MySQL usage
|
||||||
|
- `docker-compose.test.yml` provides a local MySQL only for testing
|
||||||
|
- upstream git sync, index rebuild, and MySQL refresh run inside containers
|
||||||
|
|
||||||
|
More details:
|
||||||
|
|
||||||
|
- [docs/README.md](docs/README.md)
|
||||||
|
- [docs/web-ui.md](docs/web-ui.md)
|
||||||
|
|||||||
@@ -1,98 +0,0 @@
|
|||||||
# 手机品牌型号汇总
|
|
||||||
[](https://creativecommons.org/licenses/by-nc-sa/4.0/)
|
|
||||||
|
|
||||||
这是交付目录,提供可直接运行的 Docker Compose 项目。
|
|
||||||
|
|
||||||
项目目标:
|
|
||||||
|
|
||||||
- 提供可直接交付的设备查询与数据管理系统
|
|
||||||
- 通过容器内流程同步上游原始 git 数据
|
|
||||||
- 在容器内重建索引并刷新 MySQL
|
|
||||||
- 形成可独立部署的交付包
|
|
||||||
|
|
||||||
[English](README_en.md)
|
|
||||||
|
|
||||||
## 运行方式
|
|
||||||
|
|
||||||
在当前目录执行:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose up --build -d
|
|
||||||
```
|
|
||||||
|
|
||||||
打开:
|
|
||||||
|
|
||||||
- `http://127.0.0.1:8123/web/device_query.html`
|
|
||||||
- `http://127.0.0.1:8123/web/brand_management.html`
|
|
||||||
|
|
||||||
MySQL 也会一并启动:
|
|
||||||
|
|
||||||
- host: `127.0.0.1`
|
|
||||||
- port: `3306`
|
|
||||||
- database: `mobilemodels`
|
|
||||||
- reader user: `mobilemodels_reader`
|
|
||||||
|
|
||||||
如需自定义 MySQL 账号密码,可先复制 `.env.example` 为 `.env` 后再启动。
|
|
||||||
|
|
||||||
## 结构说明
|
|
||||||
|
|
||||||
```text
|
|
||||||
dist/ 构建产物与 MySQL seed
|
|
||||||
docs/ 交付文档
|
|
||||||
sql/ MySQL schema
|
|
||||||
tools/ 容器内构建与同步脚本
|
|
||||||
web/ 页面与静态资源
|
|
||||||
```
|
|
||||||
|
|
||||||
上游原始数据不放在本目录,而是从仓库根目录下的 `workspace/` 读取。
|
|
||||||
|
|
||||||
容器启动后会自动完成:
|
|
||||||
|
|
||||||
- 初始化 `workspace/` 与 `dist/` 的运行期数据目录
|
|
||||||
- 从 `workspace/brands` 构建设备索引
|
|
||||||
- 导出 MySQL seed
|
|
||||||
- 加载 MySQL schema 与 seed
|
|
||||||
- 启动 Web 页面与 API
|
|
||||||
|
|
||||||
更多说明参见:
|
|
||||||
|
|
||||||
- [docs/README.md](docs/README.md)
|
|
||||||
- [docs/web-ui.md](docs/web-ui.md)
|
|
||||||
- [docs/mysql-query-design.md](docs/mysql-query-design.md)
|
|
||||||
- [docs/device-mapper.md](docs/device-mapper.md)
|
|
||||||
## 工作空间
|
|
||||||
|
|
||||||
原始数据与补充资料位于仓库根目录下的 `workspace/`:
|
|
||||||
|
|
||||||
- `../workspace/brands`
|
|
||||||
- `../workspace/misc`
|
|
||||||
- `../workspace/CHANGELOG.md`
|
|
||||||
- `../workspace/CHANGELOG_en.md`
|
|
||||||
|
|
||||||
### 2016 年 3 月
|
|
||||||
- 小米手机型号汇总发布至 [小米社区](http://bbs.xiaomi.cn/t-12641411)(帖子已失效)。
|
|
||||||
|
|
||||||
### 2016 年 2 月
|
|
||||||
- 我开始汇总一些国内手机品牌的型号,「手机品牌型号汇总」的雏形诞生。
|
|
||||||
|
|
||||||
## 参考资料
|
|
||||||
|
|
||||||
- [电信设备终端网](http://zd.taf.org.cn)
|
|
||||||
- [产品认证证书查询](http://webdata.cqccms.com.cn/webdata/query/CCCCerti.do)
|
|
||||||
- [工业和信息化部政务服务平台](https://ythzxfw.miit.gov.cn/resultQuery)
|
|
||||||
- [产品库-中国电信天翼终端信息平台](http://surfing.tydevice.com/)
|
|
||||||
- [Google Play 支持的设备](http://storage.googleapis.com/play_public/supported_devices.html)
|
|
||||||
- [Wi-Fi Alliance](https://www.wi-fi.org)
|
|
||||||
- [Bluetooth Launch Studio](https://launchstudio.bluetooth.com/Listings/Search)
|
|
||||||
- [Xiaomi Firmware Updater](https://xiaomifirmwareupdater.com/)
|
|
||||||
- [Huawei Open Source Release Center](https://consumer.huawei.com/en/opensource/)
|
|
||||||
- [ReaMEIZU](https://reameizu.com/)
|
|
||||||
- [The Apple Wiki](https://theapplewiki.com/)
|
|
||||||
- [ipsw.me](https://ipsw.me)
|
|
||||||
- [XDA Developers](https://www.xda-developers.com)
|
|
||||||
- [Huawei Firmware Database](https://pro-teammt.ru/en/online-firmware-database-ru/)
|
|
||||||
- [XSMS IMEI 数据库](http://xsms.com.ua/phone/imei/all/1)
|
|
||||||
- [Android Dumps](https://dumps.tadiphone.dev/dumps)
|
|
||||||
- [Lenovo Android タブレット一覧](https://idomizu.dev/archives/20150)
|
|
||||||
|
|
||||||
以及各品牌官网、论坛、微博等,恕不一一列出
|
|
||||||
@@ -1,61 +0,0 @@
|
|||||||
# Mobile Models
|
|
||||||
[](https://creativecommons.org/licenses/by-nc-sa/4.0/)
|
|
||||||
|
|
||||||
This is the delivery directory and contains the ready-to-run Docker Compose project.
|
|
||||||
|
|
||||||
Project goals:
|
|
||||||
|
|
||||||
- provide a ready-to-deliver device query system
|
|
||||||
- sync upstream raw git data inside containers
|
|
||||||
- rebuild index and refresh MySQL inside containers
|
|
||||||
- keep deployment content independent from workspace files
|
|
||||||
|
|
||||||
## Run
|
|
||||||
|
|
||||||
The project ships with device query and data management pages and now runs through `docker compose`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose up --build -d
|
|
||||||
```
|
|
||||||
|
|
||||||
Open:
|
|
||||||
|
|
||||||
- `http://127.0.0.1:8123/web/device_query.html`
|
|
||||||
- `http://127.0.0.1:8123/web/brand_management.html`
|
|
||||||
|
|
||||||
MySQL is started together with the stack:
|
|
||||||
|
|
||||||
- host: `127.0.0.1`
|
|
||||||
- port: `3306`
|
|
||||||
- database: `mobilemodels`
|
|
||||||
- reader user: `mobilemodels_reader`
|
|
||||||
|
|
||||||
If you want custom MySQL credentials, copy `.env.example` to `.env` before startup.
|
|
||||||
|
|
||||||
## Structure
|
|
||||||
|
|
||||||
```text
|
|
||||||
dist/ build outputs and MySQL seed
|
|
||||||
docs/ delivery docs
|
|
||||||
sql/ MySQL schema
|
|
||||||
tools/ container-side build and sync scripts
|
|
||||||
web/ delivered web pages
|
|
||||||
```
|
|
||||||
|
|
||||||
Raw source files are read from the repository root `workspace/` directory.
|
|
||||||
|
|
||||||
More details:
|
|
||||||
|
|
||||||
- [docs/README.md](docs/README.md)
|
|
||||||
- [docs/web-ui.md](docs/web-ui.md)
|
|
||||||
- [docs/mysql-query-design.md](docs/mysql-query-design.md)
|
|
||||||
- [docs/device-mapper.md](docs/device-mapper.md)
|
|
||||||
|
|
||||||
## Workspace
|
|
||||||
|
|
||||||
Raw data and historical files live under the repository root `workspace/`:
|
|
||||||
|
|
||||||
- `../workspace/brands`
|
|
||||||
- `../workspace/misc`
|
|
||||||
- `../workspace/CHANGELOG.md`
|
|
||||||
- `../workspace/CHANGELOG_en.md`
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Shared path helpers for the workspace/delivery project layout."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
DELIVERY_ROOT = Path(__file__).resolve().parent.parent
|
|
||||||
PROJECT_ROOT = DELIVERY_ROOT.parent
|
|
||||||
WORKSPACE_ROOT = PROJECT_ROOT / "workspace"
|
|
||||||
@@ -22,31 +22,14 @@ services:
|
|||||||
init: true
|
init: true
|
||||||
|
|
||||||
mobilemodels:
|
mobilemodels:
|
||||||
build:
|
|
||||||
context: ..
|
|
||||||
dockerfile: delivery/Dockerfile
|
|
||||||
container_name: mobilemodels-web
|
|
||||||
working_dir: /app/delivery
|
|
||||||
environment:
|
environment:
|
||||||
MOBILEMODELS_DATA_ROOT: /data
|
|
||||||
MYSQL_HOST: mysql
|
MYSQL_HOST: mysql
|
||||||
MYSQL_PORT: 3306
|
MYSQL_PORT: 3306
|
||||||
MYSQL_DATABASE: ${MYSQL_DATABASE:-mobilemodels}
|
|
||||||
MYSQL_ROOT_USER: root
|
MYSQL_ROOT_USER: root
|
||||||
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-mobilemodels_root}
|
MYSQL_AUTO_LOAD: 1
|
||||||
MYSQL_READER_USER: ${MYSQL_READER_USER:-mobilemodels_reader}
|
|
||||||
MYSQL_READER_PASSWORD: ${MYSQL_READER_PASSWORD:-mobilemodels_reader_change_me}
|
|
||||||
depends_on:
|
depends_on:
|
||||||
mysql:
|
mysql:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
command: ["sh", "tools/container_start.sh"]
|
|
||||||
ports:
|
|
||||||
- "8123:8123"
|
|
||||||
volumes:
|
|
||||||
- mobilemodels_app_data:/data
|
|
||||||
restart: unless-stopped
|
|
||||||
init: true
|
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
mobilemodels_app_data:
|
|
||||||
mobilemodels_mysql_data:
|
mobilemodels_mysql_data:
|
||||||
29
docker-compose.yml
Normal file
29
docker-compose.yml
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
services:
|
||||||
|
mobilemodels:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
container_name: mobilemodels-web
|
||||||
|
working_dir: /app
|
||||||
|
environment:
|
||||||
|
MOBILEMODELS_DATA_ROOT: /data
|
||||||
|
MYSQL_HOST: ${MYSQL_HOST:-host.docker.internal}
|
||||||
|
MYSQL_PORT: ${MYSQL_PORT:-3306}
|
||||||
|
MYSQL_DATABASE: ${MYSQL_DATABASE:-mobilemodels}
|
||||||
|
MYSQL_ROOT_USER: ${MYSQL_ROOT_USER:-root}
|
||||||
|
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-mobilemodels_root}
|
||||||
|
MYSQL_READER_USER: ${MYSQL_READER_USER:-mobilemodels_reader}
|
||||||
|
MYSQL_READER_PASSWORD: ${MYSQL_READER_PASSWORD:-mobilemodels_reader_change_me}
|
||||||
|
MYSQL_AUTO_LOAD: ${MYSQL_AUTO_LOAD:-0}
|
||||||
|
command: ["sh", "tools/container_start.sh"]
|
||||||
|
ports:
|
||||||
|
- "8123:8123"
|
||||||
|
volumes:
|
||||||
|
- mobilemodels_app_data:/data
|
||||||
|
extra_hosts:
|
||||||
|
- "host.docker.internal:host-gateway"
|
||||||
|
restart: unless-stopped
|
||||||
|
init: true
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
mobilemodels_app_data:
|
||||||
@@ -2,12 +2,18 @@
|
|||||||
|
|
||||||
## 启动方式
|
## 启动方式
|
||||||
|
|
||||||
在 `delivery/` 目录执行:
|
在项目根目录执行:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker compose up --build -d
|
docker compose up --build -d
|
||||||
```
|
```
|
||||||
|
|
||||||
|
如果要连本地测试 MySQL:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f docker-compose.yml -f docker-compose.test.yml up --build -d
|
||||||
|
```
|
||||||
|
|
||||||
如需自定义环境变量:
|
如需自定义环境变量:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -32,16 +38,16 @@ docker compose down -v
|
|||||||
- `http://127.0.0.1:8123/web/brand_management.html`:数据管理
|
- `http://127.0.0.1:8123/web/brand_management.html`:数据管理
|
||||||
- `http://127.0.0.1:8123/web/device_query.html?view=docs`:相关文档
|
- `http://127.0.0.1:8123/web/device_query.html?view=docs`:相关文档
|
||||||
|
|
||||||
整个功能栈统一运行在 Docker Compose 中,不再依赖本地 Python 或本地 MySQL。
|
整个功能栈统一运行在 Docker Compose 中,不再依赖本地 Python。
|
||||||
|
|
||||||
原始数据工作空间位于仓库根目录下的 `workspace/`,交付物位于 `delivery/`。
|
原始数据工作空间位于项目内的 `workspace/` 目录。
|
||||||
|
|
||||||
## 启动后自动完成的动作
|
## 启动后自动完成的动作
|
||||||
|
|
||||||
- 从 `workspace/brands` 构建设备索引
|
- 从 `workspace/brands` 构建设备索引
|
||||||
- 生成 `dist/device_index.json`
|
- 生成 `dist/device_index.json`
|
||||||
- 导出 MySQL seed 文件
|
- 导出 MySQL seed 文件
|
||||||
- 加载 MySQL schema 与 seed 数据
|
- 如开启 `MYSQL_AUTO_LOAD=1`,则加载 MySQL schema 与 seed 数据
|
||||||
- 启动 Web 页面与 API 服务
|
- 启动 Web 页面与 API 服务
|
||||||
|
|
||||||
## MySQL 默认连接
|
## MySQL 默认连接
|
||||||
@@ -53,6 +59,15 @@ docker compose down -v
|
|||||||
|
|
||||||
如需自定义账号密码,请使用 `.env` 覆盖默认值。
|
如需自定义账号密码,请使用 `.env` 覆盖默认值。
|
||||||
|
|
||||||
|
## MySQL 模式
|
||||||
|
|
||||||
|
- 主配置 `docker-compose.yml`
|
||||||
|
- 面向远程 MySQL
|
||||||
|
- 默认不自动装载 schema/seed
|
||||||
|
- 测试配置 `docker-compose.test.yml`
|
||||||
|
- 额外启动一个本地测试 MySQL
|
||||||
|
- 应用容器会自动把数据加载进去
|
||||||
|
|
||||||
## 设备查询
|
## 设备查询
|
||||||
|
|
||||||
页面顶部统一提供三个导航入口:
|
页面顶部统一提供三个导航入口:
|
||||||
@@ -1,12 +1,17 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
cd /app/delivery
|
cd /app
|
||||||
|
|
||||||
sh tools/init_runtime_data.sh
|
sh tools/init_runtime_data.sh
|
||||||
|
|
||||||
python3 tools/device_mapper.py build
|
python3 tools/device_mapper.py build
|
||||||
python3 tools/export_mysql_seed.py
|
python3 tools/export_mysql_seed.py
|
||||||
python3 tools/load_mysql_seed.py
|
|
||||||
|
if [ "${MYSQL_AUTO_LOAD:-0}" = "1" ]; then
|
||||||
|
python3 tools/load_mysql_seed.py
|
||||||
|
else
|
||||||
|
echo "Skipping MySQL load because MYSQL_AUTO_LOAD=${MYSQL_AUTO_LOAD:-0}"
|
||||||
|
fi
|
||||||
|
|
||||||
exec python3 tools/web_server.py --host 0.0.0.0 --port 8123
|
exec python3 tools/web_server.py --host 0.0.0.0 --port 8123
|
||||||
@@ -12,7 +12,7 @@ from datetime import date
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Dict, Iterable, List, Optional, Set
|
from typing import Dict, Iterable, List, Optional, Set
|
||||||
|
|
||||||
from project_layout import DELIVERY_ROOT, WORKSPACE_ROOT
|
from project_layout import PROJECT_ROOT, WORKSPACE_ROOT
|
||||||
|
|
||||||
ENTRY_RE = re.compile(r"^\*\*(.+?)\*\*\s*$")
|
ENTRY_RE = re.compile(r"^\*\*(.+?)\*\*\s*$")
|
||||||
VARIANT_RE = re.compile(r"^\s*((?:`[^`]+`\s*)+):\s*(.+?)\s*$")
|
VARIANT_RE = re.compile(r"^\s*((?:`[^`]+`\s*)+):\s*(.+?)\s*$")
|
||||||
@@ -743,7 +743,7 @@ def main() -> None:
|
|||||||
if args.command == "build":
|
if args.command == "build":
|
||||||
output_path: Path = args.output
|
output_path: Path = args.output
|
||||||
if not output_path.is_absolute():
|
if not output_path.is_absolute():
|
||||||
output_path = DELIVERY_ROOT / output_path
|
output_path = PROJECT_ROOT / output_path
|
||||||
export_index(records, output_path)
|
export_index(records, output_path)
|
||||||
print(f"Built index: {output_path}")
|
print(f"Built index: {output_path}")
|
||||||
print(f"Total records: {len(records)}")
|
print(f"Total records: {len(records)}")
|
||||||
@@ -16,7 +16,7 @@ from device_mapper import (
|
|||||||
normalize_text,
|
normalize_text,
|
||||||
resolve_parent_brand,
|
resolve_parent_brand,
|
||||||
)
|
)
|
||||||
from project_layout import DELIVERY_ROOT, WORKSPACE_ROOT
|
from project_layout import PROJECT_ROOT, WORKSPACE_ROOT
|
||||||
|
|
||||||
|
|
||||||
LEGACY_CODE_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9,._/+\\-]{1,63}$")
|
LEGACY_CODE_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9,._/+\\-]{1,63}$")
|
||||||
@@ -200,7 +200,7 @@ def parse_args() -> argparse.Namespace:
|
|||||||
def main() -> int:
|
def main() -> int:
|
||||||
args = parse_args()
|
args = parse_args()
|
||||||
repo_root = args.repo_root.resolve()
|
repo_root = args.repo_root.resolve()
|
||||||
output_path = args.output if args.output.is_absolute() else DELIVERY_ROOT / args.output
|
output_path = args.output if args.output.is_absolute() else PROJECT_ROOT / args.output
|
||||||
|
|
||||||
records = build_records(repo_root)
|
records = build_records(repo_root)
|
||||||
device_record_count = len(records)
|
device_record_count = len(records)
|
||||||
@@ -61,7 +61,7 @@ init_path() {
|
|||||||
|
|
||||||
for rel_path in \
|
for rel_path in \
|
||||||
workspace \
|
workspace \
|
||||||
delivery/dist
|
dist
|
||||||
do
|
do
|
||||||
init_path "$rel_path"
|
init_path "$rel_path"
|
||||||
done
|
done
|
||||||
@@ -10,7 +10,7 @@ import sys
|
|||||||
import time
|
import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from project_layout import DELIVERY_ROOT
|
from project_layout import PROJECT_ROOT
|
||||||
|
|
||||||
|
|
||||||
def mysql_env(password: str) -> dict[str, str]:
|
def mysql_env(password: str) -> dict[str, str]:
|
||||||
@@ -130,8 +130,8 @@ def parse_args() -> argparse.Namespace:
|
|||||||
|
|
||||||
def main() -> int:
|
def main() -> int:
|
||||||
args = parse_args()
|
args = parse_args()
|
||||||
schema_path = args.schema if args.schema.is_absolute() else DELIVERY_ROOT / args.schema
|
schema_path = args.schema if args.schema.is_absolute() else PROJECT_ROOT / args.schema
|
||||||
seed_path = args.seed if args.seed.is_absolute() else DELIVERY_ROOT / args.seed
|
seed_path = args.seed if args.seed.is_absolute() else PROJECT_ROOT / args.seed
|
||||||
|
|
||||||
wait_for_mysql(args.user, args.password, args.host, args.port, args.wait_timeout)
|
wait_for_mysql(args.user, args.password, args.host, args.port, args.wait_timeout)
|
||||||
|
|
||||||
9
tools/project_layout.py
Normal file
9
tools/project_layout.py
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Shared path helpers for the project layout."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
||||||
|
WORKSPACE_ROOT = PROJECT_ROOT / "workspace"
|
||||||
@@ -11,7 +11,7 @@ import sys
|
|||||||
import tempfile
|
import tempfile
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from project_layout import DELIVERY_ROOT, PROJECT_ROOT, WORKSPACE_ROOT
|
from project_layout import PROJECT_ROOT, WORKSPACE_ROOT
|
||||||
|
|
||||||
DEFAULT_REPO_URL = "https://github.com/KHwang9883/MobileModels.git"
|
DEFAULT_REPO_URL = "https://github.com/KHwang9883/MobileModels.git"
|
||||||
DEFAULT_BRANCH = "master"
|
DEFAULT_BRANCH = "master"
|
||||||
@@ -67,7 +67,7 @@ def build_index(output_path: str) -> None:
|
|||||||
run(
|
run(
|
||||||
[
|
[
|
||||||
sys.executable,
|
sys.executable,
|
||||||
str(DELIVERY_ROOT / "tools/device_mapper.py"),
|
str(PROJECT_ROOT / "tools/device_mapper.py"),
|
||||||
"--repo-root",
|
"--repo-root",
|
||||||
str(WORKSPACE_ROOT),
|
str(WORKSPACE_ROOT),
|
||||||
"build",
|
"build",
|
||||||
@@ -81,7 +81,7 @@ def export_mysql_seed(output_path: str) -> None:
|
|||||||
run(
|
run(
|
||||||
[
|
[
|
||||||
sys.executable,
|
sys.executable,
|
||||||
str(DELIVERY_ROOT / "tools/export_mysql_seed.py"),
|
str(PROJECT_ROOT / "tools/export_mysql_seed.py"),
|
||||||
"--output",
|
"--output",
|
||||||
output_path,
|
output_path,
|
||||||
"--repo-root",
|
"--repo-root",
|
||||||
@@ -94,7 +94,7 @@ def load_mysql_seed(seed_path: str) -> None:
|
|||||||
run(
|
run(
|
||||||
[
|
[
|
||||||
sys.executable,
|
sys.executable,
|
||||||
str(DELIVERY_ROOT / "tools/load_mysql_seed.py"),
|
str(PROJECT_ROOT / "tools/load_mysql_seed.py"),
|
||||||
"--seed",
|
"--seed",
|
||||||
seed_path,
|
seed_path,
|
||||||
]
|
]
|
||||||
@@ -14,20 +14,24 @@ from http import HTTPStatus
|
|||||||
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
|
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from project_layout import DELIVERY_ROOT, PROJECT_ROOT, WORKSPACE_ROOT
|
from project_layout import PROJECT_ROOT, WORKSPACE_ROOT
|
||||||
from sync_upstream_mobilemodels import DEFAULT_BRANCH, DEFAULT_REPO_URL
|
from sync_upstream_mobilemodels import DEFAULT_BRANCH, DEFAULT_REPO_URL
|
||||||
|
|
||||||
|
|
||||||
SYNC_SCRIPT = DELIVERY_ROOT / "tools/sync_upstream_mobilemodels.py"
|
SYNC_SCRIPT = PROJECT_ROOT / "tools/sync_upstream_mobilemodels.py"
|
||||||
INDEX_PATH = DELIVERY_ROOT / "dist/device_index.json"
|
INDEX_PATH = PROJECT_ROOT / "dist/device_index.json"
|
||||||
MYSQL_SEED_PATH = DELIVERY_ROOT / "dist/mobilemodels_mysql_seed.sql"
|
MYSQL_SEED_PATH = PROJECT_ROOT / "dist/mobilemodels_mysql_seed.sql"
|
||||||
MYSQL_LOADER = DELIVERY_ROOT / "tools/load_mysql_seed.py"
|
MYSQL_LOADER = PROJECT_ROOT / "tools/load_mysql_seed.py"
|
||||||
DATA_ROOT = Path(os.environ.get("MOBILEMODELS_DATA_ROOT", "/data"))
|
DATA_ROOT = Path(os.environ.get("MOBILEMODELS_DATA_ROOT", "/data"))
|
||||||
SYNC_METADATA_PATH = DATA_ROOT / "state/sync_status.json"
|
SYNC_METADATA_PATH = DATA_ROOT / "state/sync_status.json"
|
||||||
SYNC_LOCK = threading.Lock()
|
SYNC_LOCK = threading.Lock()
|
||||||
NORMALIZE_RE = re.compile(r"[^0-9a-z\u4e00-\u9fff]+")
|
NORMALIZE_RE = re.compile(r"[^0-9a-z\u4e00-\u9fff]+")
|
||||||
|
|
||||||
|
|
||||||
|
def mysql_auto_load_enabled() -> bool:
|
||||||
|
return os.environ.get("MYSQL_AUTO_LOAD", "0").strip().lower() in {"1", "true", "yes", "on"}
|
||||||
|
|
||||||
|
|
||||||
def run_command(args: list[str]) -> subprocess.CompletedProcess[str]:
|
def run_command(args: list[str]) -> subprocess.CompletedProcess[str]:
|
||||||
return subprocess.run(
|
return subprocess.run(
|
||||||
args,
|
args,
|
||||||
@@ -180,30 +184,34 @@ def get_status_payload() -> dict[str, object]:
|
|||||||
mysql_database = os.environ.get("MYSQL_DATABASE", "mobilemodels")
|
mysql_database = os.environ.get("MYSQL_DATABASE", "mobilemodels")
|
||||||
mysql_reader_user = os.environ.get("MYSQL_READER_USER", "")
|
mysql_reader_user = os.environ.get("MYSQL_READER_USER", "")
|
||||||
mysql_reader_password = os.environ.get("MYSQL_READER_PASSWORD", "")
|
mysql_reader_password = os.environ.get("MYSQL_READER_PASSWORD", "")
|
||||||
|
mysql_auto_load = mysql_auto_load_enabled()
|
||||||
mysql_ready = False
|
mysql_ready = False
|
||||||
mysql_status = ""
|
mysql_status = ""
|
||||||
sync_metadata = read_sync_metadata()
|
sync_metadata = read_sync_metadata()
|
||||||
|
if mysql_auto_load:
|
||||||
mysql_proc = run_command(["python3", str(MYSQL_LOADER), "--check-only", "--wait-timeout", "5"])
|
mysql_proc = run_command(["python3", str(MYSQL_LOADER), "--check-only", "--wait-timeout", "5"])
|
||||||
if mysql_proc.returncode == 0:
|
if mysql_proc.returncode == 0:
|
||||||
mysql_ready = True
|
mysql_ready = True
|
||||||
mysql_status = mysql_proc.stdout.strip() or "MySQL ready"
|
mysql_status = mysql_proc.stdout.strip() or "MySQL ready"
|
||||||
else:
|
else:
|
||||||
mysql_status = mysql_proc.stderr.strip() or mysql_proc.stdout.strip() or "MySQL unavailable"
|
mysql_status = mysql_proc.stderr.strip() or mysql_proc.stdout.strip() or "MySQL unavailable"
|
||||||
|
else:
|
||||||
|
mysql_status = "MySQL auto load disabled"
|
||||||
|
|
||||||
return {
|
return {
|
||||||
"supports_upstream_sync": True,
|
"supports_upstream_sync": True,
|
||||||
"storage_mode": "docker_volume",
|
"storage_mode": "docker_volume",
|
||||||
"project_root": str(PROJECT_ROOT),
|
"project_root": str(PROJECT_ROOT),
|
||||||
"workspace_root": str(WORKSPACE_ROOT),
|
"workspace_root": str(WORKSPACE_ROOT),
|
||||||
"delivery_root": str(DELIVERY_ROOT),
|
|
||||||
"data_root": str(DATA_ROOT),
|
"data_root": str(DATA_ROOT),
|
||||||
|
"mysql_auto_load": mysql_auto_load,
|
||||||
"upstream_repo_url": DEFAULT_REPO_URL,
|
"upstream_repo_url": DEFAULT_REPO_URL,
|
||||||
"upstream_branch": DEFAULT_BRANCH,
|
"upstream_branch": DEFAULT_BRANCH,
|
||||||
"last_sync_time": sync_metadata.get("last_sync_time"),
|
"last_sync_time": sync_metadata.get("last_sync_time"),
|
||||||
"last_upstream_commit": sync_metadata.get("last_upstream_commit"),
|
"last_upstream_commit": sync_metadata.get("last_upstream_commit"),
|
||||||
"index_file": str(INDEX_PATH.relative_to(DELIVERY_ROOT)),
|
"index_file": str(INDEX_PATH.relative_to(PROJECT_ROOT)),
|
||||||
"index_mtime": index_mtime,
|
"index_mtime": index_mtime,
|
||||||
"mysql_seed_file": str(MYSQL_SEED_PATH.relative_to(DELIVERY_ROOT)),
|
"mysql_seed_file": str(MYSQL_SEED_PATH.relative_to(PROJECT_ROOT)),
|
||||||
"mysql_seed_mtime": mysql_seed_mtime,
|
"mysql_seed_mtime": mysql_seed_mtime,
|
||||||
"mysql_host": mysql_host,
|
"mysql_host": mysql_host,
|
||||||
"mysql_port": mysql_port,
|
"mysql_port": mysql_port,
|
||||||
@@ -227,13 +235,15 @@ def run_upstream_sync() -> dict[str, object]:
|
|||||||
if upstream_proc.returncode == 0 and upstream_proc.stdout.strip():
|
if upstream_proc.returncode == 0 and upstream_proc.stdout.strip():
|
||||||
upstream_commit = upstream_proc.stdout.split()[0]
|
upstream_commit = upstream_proc.stdout.split()[0]
|
||||||
|
|
||||||
proc = run_command([
|
command = [
|
||||||
"python3",
|
"python3",
|
||||||
str(SYNC_SCRIPT),
|
str(SYNC_SCRIPT),
|
||||||
"--build-index",
|
"--build-index",
|
||||||
"--export-mysql-seed",
|
"--export-mysql-seed",
|
||||||
"--load-mysql",
|
]
|
||||||
])
|
if mysql_auto_load_enabled():
|
||||||
|
command.append("--load-mysql")
|
||||||
|
proc = run_command(command)
|
||||||
output = "\n".join(
|
output = "\n".join(
|
||||||
part for part in [proc.stdout.strip(), proc.stderr.strip()] if part
|
part for part in [proc.stdout.strip(), proc.stderr.strip()] if part
|
||||||
).strip()
|
).strip()
|
||||||
@@ -245,18 +255,17 @@ def run_upstream_sync() -> dict[str, object]:
|
|||||||
"storage_mode": "docker_volume",
|
"storage_mode": "docker_volume",
|
||||||
"project_root": str(PROJECT_ROOT),
|
"project_root": str(PROJECT_ROOT),
|
||||||
"workspace_root": str(WORKSPACE_ROOT),
|
"workspace_root": str(WORKSPACE_ROOT),
|
||||||
"delivery_root": str(DELIVERY_ROOT),
|
|
||||||
"data_root": str(DATA_ROOT),
|
"data_root": str(DATA_ROOT),
|
||||||
"upstream_repo_url": DEFAULT_REPO_URL,
|
"upstream_repo_url": DEFAULT_REPO_URL,
|
||||||
"upstream_branch": DEFAULT_BRANCH,
|
"upstream_branch": DEFAULT_BRANCH,
|
||||||
"upstream_commit": upstream_commit,
|
"upstream_commit": upstream_commit,
|
||||||
"last_sync_time": datetime.now().isoformat(timespec="seconds"),
|
"last_sync_time": datetime.now().isoformat(timespec="seconds"),
|
||||||
"last_upstream_commit": upstream_commit,
|
"last_upstream_commit": upstream_commit,
|
||||||
"index_file": str(INDEX_PATH.relative_to(DELIVERY_ROOT)),
|
"index_file": str(INDEX_PATH.relative_to(PROJECT_ROOT)),
|
||||||
"index_mtime": datetime.fromtimestamp(INDEX_PATH.stat().st_mtime).isoformat(timespec="seconds")
|
"index_mtime": datetime.fromtimestamp(INDEX_PATH.stat().st_mtime).isoformat(timespec="seconds")
|
||||||
if INDEX_PATH.exists()
|
if INDEX_PATH.exists()
|
||||||
else None,
|
else None,
|
||||||
"mysql_seed_file": str(MYSQL_SEED_PATH.relative_to(DELIVERY_ROOT)),
|
"mysql_seed_file": str(MYSQL_SEED_PATH.relative_to(PROJECT_ROOT)),
|
||||||
"mysql_seed_mtime": datetime.fromtimestamp(MYSQL_SEED_PATH.stat().st_mtime).isoformat(timespec="seconds")
|
"mysql_seed_mtime": datetime.fromtimestamp(MYSQL_SEED_PATH.stat().st_mtime).isoformat(timespec="seconds")
|
||||||
if MYSQL_SEED_PATH.exists()
|
if MYSQL_SEED_PATH.exists()
|
||||||
else None,
|
else None,
|
||||||
@@ -275,7 +284,7 @@ def run_upstream_sync() -> dict[str, object]:
|
|||||||
|
|
||||||
class MobileModelsHandler(SimpleHTTPRequestHandler):
|
class MobileModelsHandler(SimpleHTTPRequestHandler):
|
||||||
def __init__(self, *args, **kwargs):
|
def __init__(self, *args, **kwargs):
|
||||||
super().__init__(*args, directory=str(DELIVERY_ROOT), **kwargs)
|
super().__init__(*args, directory=str(PROJECT_ROOT), **kwargs)
|
||||||
|
|
||||||
def guess_type(self, path: str) -> str:
|
def guess_type(self, path: str) -> str:
|
||||||
content_type = super().guess_type(path)
|
content_type = super().guess_type(path)
|
||||||
@@ -396,7 +396,7 @@
|
|||||||
|
|
||||||
<section id="syncTabPanel" class="manage-panel hidden">
|
<section id="syncTabPanel" class="manage-panel hidden">
|
||||||
<h3 class="title">原始数据同步</h3>
|
<h3 class="title">原始数据同步</h3>
|
||||||
<p class="sub">从上游 `KHwang9883/MobileModels` 拉取原始 markdown 数据,并重建 `dist/device_index.json`、刷新 MySQL。请先使用 `docker compose up --build -d` 启动完整服务。</p>
|
<p class="sub">从上游 `KHwang9883/MobileModels` 拉取原始 markdown 数据,并重建 `dist/device_index.json`。如已开启 MySQL 自动装载,也会同步刷新 MySQL。请先启动完整服务。</p>
|
||||||
<div class="btns">
|
<div class="btns">
|
||||||
<button id="syncUpstreamBtn" type="button" class="primary">开始同步原始数据</button>
|
<button id="syncUpstreamBtn" type="button" class="primary">开始同步原始数据</button>
|
||||||
<button id="refreshSyncStatusBtn" type="button">刷新同步状态</button>
|
<button id="refreshSyncStatusBtn" type="button">刷新同步状态</button>
|
||||||
@@ -520,7 +520,6 @@
|
|||||||
if (data.data_root) lines.push(`数据目录: ${data.data_root}`);
|
if (data.data_root) lines.push(`数据目录: ${data.data_root}`);
|
||||||
if (data.project_root) lines.push(`项目目录: ${data.project_root}`);
|
if (data.project_root) lines.push(`项目目录: ${data.project_root}`);
|
||||||
if (data.workspace_root) lines.push(`工作空间目录: ${data.workspace_root}`);
|
if (data.workspace_root) lines.push(`工作空间目录: ${data.workspace_root}`);
|
||||||
if (data.delivery_root) lines.push(`交付目录: ${data.delivery_root}`);
|
|
||||||
if (data.storage_mode) lines.push(`存储模式: ${data.storage_mode}`);
|
if (data.storage_mode) lines.push(`存储模式: ${data.storage_mode}`);
|
||||||
if (data.upstream_repo_url) lines.push(`上游仓库: ${data.upstream_repo_url}`);
|
if (data.upstream_repo_url) lines.push(`上游仓库: ${data.upstream_repo_url}`);
|
||||||
if (data.upstream_branch) lines.push(`上游分支: ${data.upstream_branch}`);
|
if (data.upstream_branch) lines.push(`上游分支: ${data.upstream_branch}`);
|
||||||
@@ -533,6 +532,7 @@
|
|||||||
if (data.mysql_host && data.mysql_port) lines.push(`MySQL 地址: ${data.mysql_host}:${data.mysql_port}`);
|
if (data.mysql_host && data.mysql_port) lines.push(`MySQL 地址: ${data.mysql_host}:${data.mysql_port}`);
|
||||||
if (data.mysql_database) lines.push(`MySQL 数据库: ${data.mysql_database}`);
|
if (data.mysql_database) lines.push(`MySQL 数据库: ${data.mysql_database}`);
|
||||||
if (data.mysql_reader_user) lines.push(`MySQL 只读账号: ${data.mysql_reader_user}`);
|
if (data.mysql_reader_user) lines.push(`MySQL 只读账号: ${data.mysql_reader_user}`);
|
||||||
|
if (typeof data.mysql_auto_load === "boolean") lines.push(`MySQL 自动装载: ${data.mysql_auto_load ? "enabled" : "disabled"}`);
|
||||||
if (typeof data.mysql_ready === "boolean") lines.push(`MySQL 状态: ${data.mysql_ready ? "ready" : "not ready"}`);
|
if (typeof data.mysql_ready === "boolean") lines.push(`MySQL 状态: ${data.mysql_ready ? "ready" : "not ready"}`);
|
||||||
if (data.mysql_status) lines.push(`MySQL 详情: ${data.mysql_status}`);
|
if (data.mysql_status) lines.push(`MySQL 详情: ${data.mysql_status}`);
|
||||||
if (data.output) {
|
if (data.output) {
|
||||||
Reference in New Issue
Block a user