feat: 添加付费用户数据迁移脚本、报告及相关管理逻辑调整。
Build docker and publish / build (20.15.1) (push) Successful in 7m49s
Build docker and publish / build (20.15.1) (push) Successful in 7m49s
This commit is contained in:
@@ -0,0 +1,319 @@
|
||||
# 付费用户数据迁移报告
|
||||
|
||||
> 生成时间: 2026-03-13
|
||||
> 脚本: `scripts/export_paid_users.sh`
|
||||
> 输出: `scripts/output/paid_users_migration.sql` (3189 行)
|
||||
|
||||
---
|
||||
|
||||
## 1. 数据总览
|
||||
|
||||
### 源库统计
|
||||
|
||||
| 指标 | 全量 | 付费用户筛选后 | 丢弃 |
|
||||
|------|-----:|---------------:|-----:|
|
||||
| 用户 (user) | 1,864 | **482** | 1,382 (74%) |
|
||||
| 登录方式 (user_auth_methods) | 2,041 | **598** | 1,443 |
|
||||
| 设备 (user_device) | 1,415 | **496** | 919 |
|
||||
| 订单 (order) | — | **1,669** | — |
|
||||
| 已完成订单 (status=3) | 1,806 | — | — |
|
||||
| 订阅 (user_subscribe) | 1,588 | **526** | 1,062 |
|
||||
| IAP 交易 (apple_iap_transactions) | 168 | **29** | 139 |
|
||||
| 系统日志 (system_logs) | 20,264 | **4,830** | 15,434 |
|
||||
| 套餐定义 (subscribe) | 1 | **1** (全量) | — |
|
||||
| 支付方式 (payment) | 4 | **4** (全量) | — |
|
||||
| 系统配置 (system) | 53 | **53** (全量) | — |
|
||||
|
||||
### 新增数据(迁移脚本自动生成)
|
||||
|
||||
| 指标 | 数量 | 说明 |
|
||||
|------|-----:|------|
|
||||
| 家庭组 (user_family) | **482** | 每个付费用户 1 个 |
|
||||
| 家庭成员 (user_family_members) | **482 + 24** | 482 家主 + 24 拆分设备 |
|
||||
| 新用户 (拆分设备) | **24** | 多设备用户第 2 个设备→独立用户 |
|
||||
|
||||
### 幽灵用户
|
||||
|
||||
| 类型 | 数量 | 处理 |
|
||||
|------|-----:|------|
|
||||
| 有订单/IAP 但 user 表不存在 | **82** | **已排除**(INNER JOIN user) |
|
||||
| user_id=0 的脏数据 | 若干 | **已排除**(WHERE user_id > 0) |
|
||||
|
||||
---
|
||||
|
||||
## 2. 付费用户定义
|
||||
|
||||
```sql
|
||||
SELECT DISTINCT t.uid FROM (
|
||||
SELECT user_id AS uid FROM `order` WHERE status=3 AND user_id > 0
|
||||
UNION
|
||||
SELECT user_id AS uid FROM apple_iap_transactions WHERE user_id > 0
|
||||
) t
|
||||
INNER JOIN user u ON u.id = t.uid
|
||||
ORDER BY t.uid;
|
||||
```
|
||||
|
||||
**逻辑分析:**
|
||||
|
||||
1. **子查询 1** — `order WHERE status=3`:查找所有已完成支付的订单(status=3 = 支付完成),提取 `user_id`
|
||||
2. **子查询 2** — `apple_iap_transactions`:查找所有 Apple IAP 交易记录的 `user_id`
|
||||
3. **UNION**:合并去重,满足**任一条件**即为付费用户
|
||||
4. **WHERE user_id > 0**:排除 `user_id=0` 的脏数据
|
||||
5. **INNER JOIN user**:只保留在 `user` 表中**实际存在**的用户(排除 82 个幽灵用户)
|
||||
|
||||
**结果**:564 个候选 → 排除 82 幽灵 → **482 个有效付费用户**
|
||||
|
||||
---
|
||||
|
||||
## 3. 脚本分步 SQL 逻辑分析
|
||||
|
||||
### Step 1: 查询付费用户 ID
|
||||
|
||||
见上方"付费用户定义"。输出为换行分隔的 ID 列表,转为逗号分隔用于后续 WHERE IN。
|
||||
|
||||
### Step 2: SQL 文件头
|
||||
|
||||
```sql
|
||||
SET NAMES utf8mb4; -- 确保中文字符正确
|
||||
SET FOREIGN_KEY_CHECKS = 0; -- 禁用外键检查,允许无序插入
|
||||
SET UNIQUE_CHECKS = 0; -- 禁用唯一键检查,加速批量插入
|
||||
SET AUTOCOMMIT = 0; -- 开启事务模式
|
||||
CREATE DATABASE IF NOT EXISTS `ppanel` ...;
|
||||
USE `ppanel`;
|
||||
```
|
||||
|
||||
**目的**:创建安全的导入环境,避免外键/唯一键冲突导致中断。
|
||||
|
||||
### Step 3: 导出表结构(DDL)
|
||||
|
||||
```bash
|
||||
mysqldump --no-data --skip-add-drop-table \
|
||||
user user_auth_methods user_device \
|
||||
order user_subscribe apple_iap_transactions \
|
||||
subscribe payment system system_logs
|
||||
```
|
||||
|
||||
**逻辑**:
|
||||
- `--no-data`:只导出 CREATE TABLE 语句,不含数据
|
||||
- `--skip-add-drop-table`:不生成 `DROP TABLE IF EXISTS`,避免误删新库已有表
|
||||
- 后处理 `sed` 将 `CREATE TABLE` 改为 `CREATE TABLE IF NOT EXISTS`
|
||||
- **手动追加** `user_family` 和 `user_family_members` DDL(新系统表,源库可能没有)
|
||||
|
||||
**涉及 12 张表**:
|
||||
| 表 | 类型 |
|
||||
|----|------|
|
||||
| user | 用户主表 |
|
||||
| user_auth_methods | 登录方式(email/device/telephone) |
|
||||
| user_device | 设备记录 |
|
||||
| order | 订单 |
|
||||
| user_subscribe | 用户订阅 |
|
||||
| apple_iap_transactions | Apple IAP 交易 |
|
||||
| subscribe | 套餐定义(全量配置表) |
|
||||
| payment | 支付方式(全量配置表) |
|
||||
| system | 系统配置(全量配置表) |
|
||||
| system_logs | 系统日志 |
|
||||
| user_family | 家庭组(新表,手动 DDL) |
|
||||
| user_family_members | 家庭成员(新表,手动 DDL) |
|
||||
|
||||
### Step 4: 全量配置表数据
|
||||
|
||||
```bash
|
||||
for TBL in subscribe payment system; do
|
||||
mysqldump --no-create-info --complete-insert --skip-extended-insert "${TBL}"
|
||||
done
|
||||
```
|
||||
|
||||
**逻辑**:
|
||||
- `--no-create-info`:只导出 INSERT,不重复 DDL
|
||||
- `--complete-insert`:生成包含列名的完整 INSERT(兼容性更好)
|
||||
- `--skip-extended-insert`:每行一条 INSERT(便于阅读和调试)
|
||||
- 这三张表**不按用户过滤**,全量导出
|
||||
|
||||
**数据量**:subscribe 1 条 + payment 4 条 + system 53 条 = 58 条
|
||||
|
||||
### Step 5: 付费用户关联数据
|
||||
|
||||
```bash
|
||||
export_table_by_user_ids() {
|
||||
mysqldump --no-create-info --complete-insert \
|
||||
--where="${COL} IN (${PAID_ID_LIST})" "${TBL}"
|
||||
}
|
||||
```
|
||||
|
||||
逐表使用 `--where` 子句过滤:
|
||||
|
||||
| 表 | 过滤列 | 导出数量 | SQL 逻辑 |
|
||||
|----|--------|---------|---------|
|
||||
| `user` | `id` | 482 | `WHERE id IN (1,5,7,...)` — 只导出付费用户的用户记录 |
|
||||
| `user_auth_methods` | `user_id` | 598 | `WHERE user_id IN (...)` — 付费用户的所有登录方式 |
|
||||
| `user_device` | `user_id` | 496 | `WHERE user_id IN (...)` — 付费用户的所有设备 |
|
||||
| `order` | `user_id` | 1,669 | `WHERE user_id IN (...)` — 付费用户的**所有**订单(含未完成) |
|
||||
| `user_subscribe` | `user_id` | 526 | `WHERE user_id IN (...)` — 付费用户的订阅记录 |
|
||||
| `apple_iap_transactions` | `user_id` | 29 | `WHERE user_id IN (...)` — 付费用户的 IAP 交易 |
|
||||
|
||||
**注意**:`order` 表导出的是付费用户的**全部订单**(1,669 条),不仅仅是 status=3 的。这是合理的——保留用户完整的订单历史。
|
||||
|
||||
### Step 6: 系统日志
|
||||
|
||||
```sql
|
||||
mysqldump --where="object_id IN (${PAID_ID_LIST})" system_logs
|
||||
```
|
||||
|
||||
**逻辑**:`system_logs.object_id` 记录的是操作对象 ID,通常是 user_id。按付费用户 ID 过滤。
|
||||
|
||||
**注意**:`object_id` 不一定都是 user_id(不同 type 含义不同),可能多导或少导少量记录,影响不大。
|
||||
|
||||
**数据量**:4,830 条
|
||||
|
||||
### Step 7: 家庭组初始化
|
||||
|
||||
```sql
|
||||
-- 对每个付费用户执行:
|
||||
INSERT INTO user_family (owner_user_id, max_members, status, created_at, updated_at)
|
||||
VALUES ({user_id}, 2, 1, NOW(), NOW());
|
||||
|
||||
INSERT INTO user_family_members (family_id, user_id, role, status, join_source, joined_at, ...)
|
||||
VALUES (LAST_INSERT_ID(), {user_id}, 1, 1, 'migration', NOW(), NOW(), NOW());
|
||||
```
|
||||
|
||||
**逻辑分析**:
|
||||
|
||||
1. 遍历 482 个付费用户 ID
|
||||
2. 为每个用户创建 **1 个家庭组**(`user_family`)
|
||||
- `owner_user_id` = 该用户 ID
|
||||
- `max_members = 2`(默认最多 2 人)
|
||||
- `status = 1`(活跃)
|
||||
3. 将该用户添加为**家主**(`user_family_members`)
|
||||
- `family_id = LAST_INSERT_ID()` — 引用刚插入的家庭组 ID
|
||||
- `role = 1`(家主)
|
||||
- `status = 1`(活跃)
|
||||
- `join_source = 'migration'`(标记来源为迁移)
|
||||
|
||||
**LAST_INSERT_ID() 链式调用**:MySQL 保证 `LAST_INSERT_ID()` 返回同一连接中最后一次 AUTO_INCREMENT 的值,在顺序执行的 SQL 中是安全的。
|
||||
|
||||
### Step 8: 多设备用户拆分
|
||||
|
||||
**背景**:旧系统中同一 user_id 可以有多个设备。新系统要求每个设备 = 独立用户,通过家庭组关联。
|
||||
|
||||
**查询多设备用户的第二个设备**:
|
||||
|
||||
```sql
|
||||
SELECT ud.user_id, ud.id, ud.Identifier, ud.user_agent, ud.created_at, ua.id
|
||||
FROM user_device ud
|
||||
INNER JOIN user_auth_methods ua
|
||||
ON ua.user_id = ud.user_id
|
||||
AND ua.auth_type = 'device'
|
||||
AND ua.auth_identifier = ud.Identifier
|
||||
WHERE ud.user_id IN (
|
||||
-- 找到有 >1 个设备的付费用户
|
||||
SELECT user_id FROM user_device
|
||||
WHERE user_id IN ({paid_ids})
|
||||
GROUP BY user_id HAVING COUNT(*) > 1
|
||||
)
|
||||
AND ud.id NOT IN (
|
||||
-- 排除每个用户的第一个设备(MIN(id) = 最早注册的设备)
|
||||
SELECT MIN(id) FROM user_device
|
||||
WHERE user_id IN (...多设备用户...)
|
||||
GROUP BY user_id
|
||||
)
|
||||
```
|
||||
|
||||
**逻辑分析**:
|
||||
|
||||
1. **识别多设备用户**:`GROUP BY user_id HAVING COUNT(*) > 1` → 找到 24 个用户
|
||||
2. **保留第一个设备**:`MIN(id)` = 最早注册的设备,保留在原 user 上
|
||||
3. **INNER JOIN user_auth_methods**:通过 `auth_type='device'` + `auth_identifier=Identifier` 关联设备的登录方式记录
|
||||
4. **输出**:每个需要拆分的设备的完整信息
|
||||
|
||||
**对每个需要拆分的设备生成 SQL**:
|
||||
|
||||
```sql
|
||||
-- 1. 创建新用户(无密码无邮箱的纯设备用户)
|
||||
INSERT INTO user (password, algo, salt, enable, is_admin, created_at, updated_at)
|
||||
VALUES ('', 'default', 'default', 1, 0, '{device_created}', NOW());
|
||||
SET @new_user_id = LAST_INSERT_ID();
|
||||
|
||||
-- 2. 将设备记录转移到新用户
|
||||
UPDATE user_device SET user_id = @new_user_id WHERE id = {device_id};
|
||||
|
||||
-- 3. 将设备的 auth_method 转移到新用户
|
||||
UPDATE user_auth_methods SET user_id = @new_user_id WHERE id = {auth_method_id};
|
||||
|
||||
-- 4. 将新用户加入原用户的家庭组
|
||||
INSERT INTO user_family_members (family_id, user_id, role, status, join_source, ...)
|
||||
VALUES (
|
||||
(SELECT id FROM user_family WHERE owner_user_id = {owner_uid}),
|
||||
@new_user_id, 2, 1, 'migration_split', ...
|
||||
);
|
||||
```
|
||||
|
||||
**处理流程**:
|
||||
```
|
||||
原 user(id=100, 2 个设备)
|
||||
├─ device_1 (id=50, MIN) → 保留在 user 100 上(已是家主)
|
||||
└─ device_2 (id=51) → 创建新 user(id=NEW)
|
||||
→ UPDATE user_device SET user_id=NEW WHERE id=51
|
||||
→ UPDATE user_auth_methods SET user_id=NEW WHERE id=...
|
||||
→ INSERT user_family_members(family_id=..., user_id=NEW, role=2)
|
||||
```
|
||||
|
||||
**结果**:24 个设备被拆分为独立用户,并加入原用户的家庭组作为 member。
|
||||
|
||||
### 文件尾
|
||||
|
||||
```sql
|
||||
SET FOREIGN_KEY_CHECKS = 1; -- 恢复外键检查
|
||||
SET UNIQUE_CHECKS = 1; -- 恢复唯一键检查
|
||||
COMMIT; -- 提交事务
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. 数据完整性校验点
|
||||
|
||||
| 校验项 | 预期值 | 说明 |
|
||||
|--------|--------|------|
|
||||
| 导入后 user 数 | 482 + 24 = **506** | 482 原始 + 24 拆分 |
|
||||
| user_family 数 | **482** | 每个付费用户 1 个家庭组 |
|
||||
| user_family_members 数 | **506** | 482 家主 + 24 成员 |
|
||||
| 每个 family 的成员数 | 1 或 2 | 无拆分=1,有拆分=2 |
|
||||
| role=1 的成员数 | **482** | 每个家庭只有 1 个家主 |
|
||||
| role=2 的成员数 | **24** | 拆分设备的新用户 |
|
||||
| user_device.user_id 无孤儿 | 全部指向存在的 user | 拆分后 device 指向新 user |
|
||||
| user_auth_methods.user_id 无孤儿 | 全部指向存在的 user | 拆分后 auth 指向新 user |
|
||||
|
||||
---
|
||||
|
||||
## 5. 风险与缓解
|
||||
|
||||
| 风险 | 级别 | 缓解措施 |
|
||||
|------|------|----------|
|
||||
| 新库已有数据,ID 冲突 | 高 | 新库应为空库;或改用 `INSERT IGNORE` |
|
||||
| `refer_code` 唯一键冲突 | 中 | 迁移用户保留原值,新库确保无重复 |
|
||||
| `LAST_INSERT_ID()` 链断裂 | 低 | SQL 文件必须**顺序执行**,不可并行 |
|
||||
| 设备拆分后原用户订阅归属 | 低 | 订阅保留在原 user 上,新 user 通过家庭组共享 |
|
||||
| `system_logs.object_id` 语义不一致 | 低 | 不同 type 的 object_id 含义不同,可能多导 |
|
||||
|
||||
---
|
||||
|
||||
## 6. 导入命令
|
||||
|
||||
```bash
|
||||
docker exec -i <新容器> mysql -uroot -p<密码> < scripts/output/paid_users_migration.sql
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. 不导出的表(已丢弃)
|
||||
|
||||
| 表 | 原因 |
|
||||
|----|------|
|
||||
| traffic_log | 体积大,非必要 |
|
||||
| ads / announcement / coupon / document | 0 条或非用户数据 |
|
||||
| nodes / servers / server / server_group | 节点配置,不随用户迁移 |
|
||||
| ticket / ticket_follow | 工单数据 |
|
||||
| task | 0 条 |
|
||||
| schema_migrations | 迁移记录 |
|
||||
| log_message / application_versions | 0 条 |
|
||||
| subscribe_application | 应用配置 |
|
||||
| user_device_online_record | 0 条 |
|
||||
@@ -0,0 +1,566 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"log"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"gorm.io/driver/mysql"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/logger"
|
||||
)
|
||||
|
||||
// ── 模型定义(与项目一致) ──
|
||||
|
||||
type User struct {
|
||||
Id int64 `gorm:"primaryKey"`
|
||||
Password string `gorm:"type:varchar(255)"`
|
||||
Algo string `gorm:"type:varchar(255)"`
|
||||
Salt string `gorm:"type:varchar(255)"`
|
||||
Avatar string `gorm:"type:varchar(255)"`
|
||||
Balance int64 `gorm:"type:int"`
|
||||
ReferCode string `gorm:"type:varchar(255)"`
|
||||
RefererId int64 `gorm:"type:bigint"`
|
||||
Commission int64 `gorm:"type:int"`
|
||||
ReferralPercentage int64 `gorm:"type:int"`
|
||||
OnlyFirstPurchase *bool `gorm:"type:tinyint(1)"`
|
||||
GiftAmount int64 `gorm:"type:int"`
|
||||
Enable *bool `gorm:"type:tinyint(1)"`
|
||||
IsAdmin *bool `gorm:"type:tinyint(1)"`
|
||||
EnableBalanceNotify *bool `gorm:"type:tinyint(1)"`
|
||||
EnableLoginNotify *bool `gorm:"type:tinyint(1)"`
|
||||
EnableSubscribeNotify *bool `gorm:"type:tinyint(1)"`
|
||||
EnableTradeNotify *bool `gorm:"type:tinyint(1)"`
|
||||
LastLoginTime *time.Time
|
||||
MemberStatus string `gorm:"type:varchar(255)"`
|
||||
Remark string `gorm:"type:text"`
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
DeletedAt gorm.DeletedAt `gorm:"index"`
|
||||
}
|
||||
|
||||
func (User) TableName() string { return "user" }
|
||||
|
||||
type AuthMethod struct {
|
||||
Id int64 `gorm:"primaryKey"`
|
||||
UserId int64 `gorm:"type:bigint"`
|
||||
AuthType string `gorm:"type:varchar(50)"`
|
||||
AuthIdentifier string `gorm:"type:varchar(255)"`
|
||||
Verified *bool `gorm:"type:tinyint(1)"`
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
func (AuthMethod) TableName() string { return "user_auth_methods" }
|
||||
|
||||
type Device struct {
|
||||
Id int64 `gorm:"primaryKey"`
|
||||
Ip string `gorm:"type:varchar(255)"`
|
||||
UserId int64 `gorm:"type:bigint"`
|
||||
UserAgent string `gorm:"type:text"`
|
||||
Identifier string `gorm:"type:varchar(255)"`
|
||||
ShortCode string `gorm:"type:varchar(50)"`
|
||||
Online *bool `gorm:"type:tinyint(1)"`
|
||||
Enabled *bool `gorm:"type:tinyint(1)"`
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
func (Device) TableName() string { return "user_device" }
|
||||
|
||||
type UserSubscribe struct {
|
||||
Id int64 `gorm:"primaryKey"`
|
||||
UserId int64 `gorm:"type:bigint"`
|
||||
OrderId int64 `gorm:"type:bigint"`
|
||||
SubscribeId int64 `gorm:"type:bigint"`
|
||||
StartTime time.Time `gorm:"type:datetime(3)"`
|
||||
ExpireTime *time.Time `gorm:"type:datetime(3)"`
|
||||
FinishedAt *time.Time `gorm:"type:datetime"`
|
||||
Traffic int64 `gorm:"type:bigint"`
|
||||
Download int64 `gorm:"type:bigint"`
|
||||
Upload int64 `gorm:"type:bigint"`
|
||||
Token string `gorm:"type:varchar(255)"`
|
||||
UUID string `gorm:"type:varchar(255)"`
|
||||
Status uint8 `gorm:"type:tinyint(1)"`
|
||||
Note string `gorm:"type:varchar(500)"`
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
func (UserSubscribe) TableName() string { return "user_subscribe" }
|
||||
|
||||
type Order struct {
|
||||
Id int64 `gorm:"primaryKey"`
|
||||
ParentId int64 `gorm:"type:bigint"`
|
||||
UserId int64 `gorm:"type:bigint"`
|
||||
SubscriptionUserId int64 `gorm:"type:bigint"`
|
||||
OrderNo string `gorm:"type:varchar(255)"`
|
||||
Type uint8 `gorm:"type:tinyint(1)"`
|
||||
Quantity int64 `gorm:"type:bigint"`
|
||||
Price int64 `gorm:"type:int"`
|
||||
Amount int64 `gorm:"type:int"`
|
||||
GiftAmount int64 `gorm:"type:int"`
|
||||
Discount int64 `gorm:"type:int"`
|
||||
Coupon string `gorm:"type:varchar(255)"`
|
||||
CouponDiscount int64 `gorm:"type:int"`
|
||||
Commission int64 `gorm:"type:int"`
|
||||
PaymentId int64 `gorm:"type:bigint"`
|
||||
Method string `gorm:"type:varchar(255)"`
|
||||
FeeAmount int64 `gorm:"type:int"`
|
||||
TradeNo string `gorm:"type:varchar(255)"`
|
||||
Status uint8 `gorm:"type:tinyint(1)"`
|
||||
SubscribeId int64 `gorm:"type:bigint"`
|
||||
SubscribeToken string `gorm:"type:varchar(255)"`
|
||||
AppAccountToken string `gorm:"type:varchar(36)"`
|
||||
IsNew bool `gorm:"type:tinyint(1)"`
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
func (Order) TableName() string { return "order" }
|
||||
|
||||
type IAPTransaction struct {
|
||||
Id int64 `gorm:"primaryKey"`
|
||||
UserId int64 `gorm:"type:bigint"`
|
||||
OriginalTransactionId string `gorm:"type:varchar(255)"`
|
||||
TransactionId string `gorm:"type:varchar(255)"`
|
||||
ProductId string `gorm:"type:varchar(255)"`
|
||||
PurchaseAt *time.Time
|
||||
RevocationAt *time.Time
|
||||
JWSHash string `gorm:"type:varchar(255)"`
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
func (IAPTransaction) TableName() string { return "apple_iap_transactions" }
|
||||
|
||||
type Subscribe struct {
|
||||
Id int64 `gorm:"primaryKey"`
|
||||
Name string `gorm:"type:varchar(255)"`
|
||||
Language string `gorm:"type:varchar(255)"`
|
||||
Description string `gorm:"type:text"`
|
||||
UnitPrice int64 `gorm:"type:int"`
|
||||
UnitTime string `gorm:"type:varchar(255)"`
|
||||
Discount string `gorm:"type:text"`
|
||||
Replacement int64 `gorm:"type:int"`
|
||||
Inventory int64 `gorm:"type:int"`
|
||||
Traffic int64 `gorm:"type:int"`
|
||||
SpeedLimit int64 `gorm:"type:int"`
|
||||
DeviceLimit int64 `gorm:"type:int"`
|
||||
Quota int64 `gorm:"type:int"`
|
||||
NewUserOnly *bool `gorm:"type:tinyint(1)"`
|
||||
Nodes string `gorm:"type:varchar(255)"`
|
||||
NodeTags string `gorm:"type:varchar(255)"`
|
||||
Show *bool `gorm:"type:tinyint(1)"`
|
||||
Sell *bool `gorm:"type:tinyint(1)"`
|
||||
Sort int64 `gorm:"type:int"`
|
||||
DeductionRatio int64 `gorm:"type:int"`
|
||||
AllowDeduction *bool `gorm:"type:tinyint(1)"`
|
||||
ResetCycle int64 `gorm:"type:int"`
|
||||
RenewalReset *bool `gorm:"type:tinyint(1)"`
|
||||
ShowOriginalPrice bool `gorm:"type:tinyint(1)"`
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
func (Subscribe) TableName() string { return "subscribe" }
|
||||
|
||||
type Payment struct {
|
||||
Id int64 `gorm:"primaryKey"`
|
||||
Name string `gorm:"type:varchar(100)"`
|
||||
Platform string `gorm:"type:varchar(100)"`
|
||||
Icon string `gorm:"type:varchar(255)"`
|
||||
Domain string `gorm:"type:varchar(255)"`
|
||||
Config string `gorm:"type:text"`
|
||||
Description string `gorm:"type:text"`
|
||||
FeeMode uint `gorm:"type:tinyint(1)"`
|
||||
FeePercent int64 `gorm:"type:int"`
|
||||
FeeAmount int64 `gorm:"type:int"`
|
||||
Enable *bool `gorm:"type:tinyint(1)"`
|
||||
Token string `gorm:"type:varchar(255)"`
|
||||
}
|
||||
|
||||
func (Payment) TableName() string { return "payment" }
|
||||
|
||||
type UserFamily struct {
|
||||
Id int64 `gorm:"primaryKey"`
|
||||
OwnerUserId int64 `gorm:"uniqueIndex"`
|
||||
MaxMembers int64 `gorm:"default:3"`
|
||||
Status uint8 `gorm:"type:tinyint(1);default:1"`
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
DeletedAt gorm.DeletedAt `gorm:"index"`
|
||||
}
|
||||
|
||||
func (UserFamily) TableName() string { return "user_family" }
|
||||
|
||||
type UserFamilyMember struct {
|
||||
Id int64 `gorm:"primaryKey"`
|
||||
FamilyId int64
|
||||
UserId int64 `gorm:"uniqueIndex"`
|
||||
Role uint8 `gorm:"type:tinyint(1);default:2"`
|
||||
Status uint8 `gorm:"type:tinyint(1);default:1"`
|
||||
JoinSource string `gorm:"type:varchar(32)"`
|
||||
JoinedAt time.Time
|
||||
LeftAt *time.Time
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
DeletedAt gorm.DeletedAt `gorm:"index"`
|
||||
}
|
||||
|
||||
func (UserFamilyMember) TableName() string { return "user_family_member" }
|
||||
|
||||
// ── 主程序 ──
|
||||
|
||||
const (
|
||||
defaultFamilyMaxSize = 3
|
||||
orderStatusCompleted = 3
|
||||
familyRoleOwner = 1
|
||||
familyRoleMember = 2
|
||||
batchSize = 100
|
||||
)
|
||||
|
||||
func main() {
|
||||
srcDSN := flag.String("src", "", "源数据库 DSN,格式: user:password@tcp(host:port)/dbname")
|
||||
dstDSN := flag.String("dst", "", "目标数据库 DSN,格式: user:password@tcp(host:port)/dbname")
|
||||
clean := flag.Bool("clean", false, "导入前清空目标库相关表")
|
||||
dryRun := flag.Bool("dry-run", false, "仅分析不写入,打印统计信息")
|
||||
flag.Parse()
|
||||
|
||||
if *srcDSN == "" || *dstDSN == "" {
|
||||
fmt.Println("付费用户数据迁移工具")
|
||||
fmt.Println()
|
||||
fmt.Println("用法:")
|
||||
fmt.Println(" go run scripts/migrate_paid_users.go \\")
|
||||
fmt.Println(" -src 'root:rootpassword@tcp(127.0.0.1:3306)/ppanel?charset=utf8mb4&parseTime=True&loc=Local' \\")
|
||||
fmt.Println(" -dst 'root:jpcV41ppanel@tcp(154.12.35.103:3306)/ppanel?charset=utf8mb4&parseTime=True&loc=Local' \\")
|
||||
fmt.Println(" -clean")
|
||||
fmt.Println()
|
||||
fmt.Println("参数:")
|
||||
fmt.Println(" -src 源数据库 DSN(旧备份库)")
|
||||
fmt.Println(" -dst 目标数据库 DSN(新线上库)")
|
||||
fmt.Println(" -clean 导入前清空目标库的用户/订单等表(保留表结构)")
|
||||
fmt.Println(" -dry-run 仅分析不写入")
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
gormCfg := &gorm.Config{
|
||||
Logger: logger.Default.LogMode(logger.Warn),
|
||||
}
|
||||
|
||||
// 连接源库
|
||||
fmt.Println("=== 付费用户数据迁移 ===")
|
||||
fmt.Println()
|
||||
fmt.Print("[1/9] 连接源数据库... ")
|
||||
srcDB, err := gorm.Open(mysql.Open(*srcDSN), gormCfg)
|
||||
if err != nil {
|
||||
log.Fatalf("源库连接失败: %v", err)
|
||||
}
|
||||
fmt.Println("OK")
|
||||
|
||||
// 连接目标库
|
||||
fmt.Print("[2/9] 连接目标数据库... ")
|
||||
dstDB, err := gorm.Open(mysql.Open(*dstDSN), gormCfg)
|
||||
if err != nil {
|
||||
log.Fatalf("目标库连接失败: %v", err)
|
||||
}
|
||||
fmt.Println("OK")
|
||||
|
||||
// ── Step 3: 查询付费用户 ID ──
|
||||
fmt.Print("[3/9] 查询付费用户... ")
|
||||
var paidIDs []int64
|
||||
err = srcDB.Raw(`
|
||||
SELECT DISTINCT t.uid FROM (
|
||||
SELECT user_id AS uid FROM ` + "`order`" + ` WHERE status = ? AND user_id > 0
|
||||
UNION
|
||||
SELECT user_id AS uid FROM apple_iap_transactions WHERE user_id > 0
|
||||
) t
|
||||
INNER JOIN user u ON u.id = t.uid
|
||||
ORDER BY t.uid
|
||||
`, orderStatusCompleted).Scan(&paidIDs).Error
|
||||
if err != nil {
|
||||
log.Fatalf("查询付费用户失败: %v", err)
|
||||
}
|
||||
fmt.Printf("%d 个付费用户\n", len(paidIDs))
|
||||
|
||||
if len(paidIDs) == 0 {
|
||||
fmt.Println("没有找到付费用户,退出")
|
||||
return
|
||||
}
|
||||
|
||||
// ── Step 4: 读取源库数据 ──
|
||||
fmt.Print("[4/9] 读取源库数据... ")
|
||||
var (
|
||||
users []User
|
||||
auths []AuthMethod
|
||||
devices []Device
|
||||
orders []Order
|
||||
subscribes []UserSubscribe
|
||||
iaps []IAPTransaction
|
||||
// 全量表
|
||||
subPlans []Subscribe
|
||||
payments []Payment
|
||||
)
|
||||
|
||||
srcDB.Where("id IN ?", paidIDs).Find(&users)
|
||||
srcDB.Where("user_id IN ?", paidIDs).Find(&auths)
|
||||
srcDB.Where("user_id IN ?", paidIDs).Find(&devices)
|
||||
srcDB.Where("user_id IN ? AND status = ?", paidIDs, orderStatusCompleted).Find(&orders)
|
||||
srcDB.Where("user_id IN ?", paidIDs).Find(&subscribes)
|
||||
srcDB.Where("user_id IN ?", paidIDs).Find(&iaps)
|
||||
srcDB.Find(&subPlans)
|
||||
srcDB.Find(&payments)
|
||||
|
||||
fmt.Println("OK")
|
||||
fmt.Println()
|
||||
fmt.Println(" 数据统计:")
|
||||
fmt.Printf(" user: %d\n", len(users))
|
||||
fmt.Printf(" user_auth_methods: %d\n", len(auths))
|
||||
fmt.Printf(" user_device: %d\n", len(devices))
|
||||
fmt.Printf(" order: %d\n", len(orders))
|
||||
fmt.Printf(" user_subscribe: %d\n", len(subscribes))
|
||||
fmt.Printf(" apple_iap: %d\n", len(iaps))
|
||||
fmt.Printf(" subscribe(全量): %d\n", len(subPlans))
|
||||
fmt.Printf(" payment(全量): %d\n", len(payments))
|
||||
|
||||
// ── 识别多设备用户 ──
|
||||
deviceByUser := make(map[int64][]Device)
|
||||
for _, d := range devices {
|
||||
deviceByUser[d.UserId] = append(deviceByUser[d.UserId], d)
|
||||
}
|
||||
authByUserDevice := make(map[string]*AuthMethod) // key: "userId:identifier"
|
||||
for i := range auths {
|
||||
a := &auths[i]
|
||||
if a.AuthType == "device" {
|
||||
key := fmt.Sprintf("%d:%s", a.UserId, a.AuthIdentifier)
|
||||
authByUserDevice[key] = a
|
||||
}
|
||||
}
|
||||
|
||||
type splitInfo struct {
|
||||
OwnerUID int64
|
||||
Device Device
|
||||
AuthMethod *AuthMethod
|
||||
}
|
||||
var splits []splitInfo
|
||||
|
||||
for uid, devs := range deviceByUser {
|
||||
if len(devs) <= 1 {
|
||||
continue
|
||||
}
|
||||
// 按 ID 排序:最小的保留,其余拆分
|
||||
minID := devs[0].Id
|
||||
for _, d := range devs[1:] {
|
||||
if d.Id < minID {
|
||||
minID = d.Id
|
||||
}
|
||||
}
|
||||
for _, d := range devs {
|
||||
if d.Id == minID {
|
||||
continue
|
||||
}
|
||||
key := fmt.Sprintf("%d:%s", uid, d.Identifier)
|
||||
auth := authByUserDevice[key]
|
||||
splits = append(splits, splitInfo{
|
||||
OwnerUID: uid,
|
||||
Device: d,
|
||||
AuthMethod: auth,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
fmt.Printf("\n 多设备拆分: %d 个设备 → 独立用户\n", len(splits))
|
||||
fmt.Println()
|
||||
|
||||
if *dryRun {
|
||||
fmt.Println("[DRY-RUN] 仅分析,不写入目标库")
|
||||
return
|
||||
}
|
||||
|
||||
// ── Step 5: 清空目标库(可选) ──
|
||||
if *clean {
|
||||
fmt.Print("[5/9] 清空目标库... ")
|
||||
dstDB.Exec("SET FOREIGN_KEY_CHECKS = 0")
|
||||
for _, tbl := range []string{
|
||||
"user", "user_auth_methods", "user_device",
|
||||
"`order`", "user_subscribe", "apple_iap_transactions",
|
||||
"user_family", "user_family_member",
|
||||
} {
|
||||
dstDB.Exec(fmt.Sprintf("TRUNCATE TABLE %s", tbl))
|
||||
}
|
||||
dstDB.Exec("SET FOREIGN_KEY_CHECKS = 1")
|
||||
fmt.Println("OK")
|
||||
} else {
|
||||
fmt.Println("[5/9] 跳过清空(未指定 -clean)")
|
||||
}
|
||||
|
||||
// ── Step 6: 写入全量配置表 ──
|
||||
fmt.Print("[6/9] 写入全量配置表... ")
|
||||
if len(subPlans) > 0 {
|
||||
dstDB.Exec("DELETE FROM subscribe") // 先清再插
|
||||
for _, s := range subPlans {
|
||||
dstDB.Create(&s)
|
||||
}
|
||||
}
|
||||
if len(payments) > 0 {
|
||||
dstDB.Exec("DELETE FROM payment")
|
||||
for _, p := range payments {
|
||||
dstDB.Create(&p)
|
||||
}
|
||||
}
|
||||
fmt.Println("OK")
|
||||
|
||||
// ── Step 7: 写入付费用户数据(事务) ──
|
||||
fmt.Print("[7/9] 写入付费用户数据... ")
|
||||
err = dstDB.Transaction(func(tx *gorm.DB) error {
|
||||
tx.Exec("SET FOREIGN_KEY_CHECKS = 0")
|
||||
|
||||
if err := tx.CreateInBatches(&users, batchSize).Error; err != nil {
|
||||
return fmt.Errorf("写入 user 失败: %w", err)
|
||||
}
|
||||
if err := tx.CreateInBatches(&auths, batchSize).Error; err != nil {
|
||||
return fmt.Errorf("写入 auth_methods 失败: %w", err)
|
||||
}
|
||||
if err := tx.CreateInBatches(&devices, batchSize).Error; err != nil {
|
||||
return fmt.Errorf("写入 device 失败: %w", err)
|
||||
}
|
||||
if err := tx.CreateInBatches(&orders, batchSize).Error; err != nil {
|
||||
return fmt.Errorf("写入 order 失败: %w", err)
|
||||
}
|
||||
if err := tx.CreateInBatches(&subscribes, batchSize).Error; err != nil {
|
||||
return fmt.Errorf("写入 subscribe 失败: %w", err)
|
||||
}
|
||||
if err := tx.CreateInBatches(&iaps, batchSize).Error; err != nil {
|
||||
return fmt.Errorf("写入 iap 失败: %w", err)
|
||||
}
|
||||
|
||||
tx.Exec("SET FOREIGN_KEY_CHECKS = 1")
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
log.Fatalf("写入失败: %v", err)
|
||||
}
|
||||
fmt.Println("OK")
|
||||
|
||||
// ── Step 8: 创建家庭组 ──
|
||||
fmt.Print("[8/9] 创建家庭组... ")
|
||||
now := time.Now()
|
||||
familyCount := 0
|
||||
|
||||
err = dstDB.Transaction(func(tx *gorm.DB) error {
|
||||
for _, uid := range paidIDs {
|
||||
family := UserFamily{
|
||||
OwnerUserId: uid,
|
||||
MaxMembers: defaultFamilyMaxSize,
|
||||
Status: 1,
|
||||
CreatedAt: now,
|
||||
UpdatedAt: now,
|
||||
}
|
||||
if err := tx.Create(&family).Error; err != nil {
|
||||
return fmt.Errorf("创建家庭组(uid=%d)失败: %w", uid, err)
|
||||
}
|
||||
|
||||
member := UserFamilyMember{
|
||||
FamilyId: family.Id,
|
||||
UserId: uid,
|
||||
Role: familyRoleOwner,
|
||||
Status: 1,
|
||||
JoinSource: "migration",
|
||||
JoinedAt: now,
|
||||
CreatedAt: now,
|
||||
UpdatedAt: now,
|
||||
}
|
||||
if err := tx.Create(&member).Error; err != nil {
|
||||
return fmt.Errorf("创建家主成员(uid=%d)失败: %w", uid, err)
|
||||
}
|
||||
familyCount++
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
log.Fatalf("家庭组创建失败: %v", err)
|
||||
}
|
||||
fmt.Printf("%d 个家庭组\n", familyCount)
|
||||
|
||||
// ── Step 9: 多设备拆分 ──
|
||||
fmt.Print("[9/9] 多设备拆分... ")
|
||||
splitCount := 0
|
||||
|
||||
err = dstDB.Transaction(func(tx *gorm.DB) error {
|
||||
for _, s := range splits {
|
||||
// 1. 创建新用户
|
||||
newUser := User{
|
||||
Password: "",
|
||||
Algo: "default",
|
||||
Salt: "default",
|
||||
Enable: boolPtr(true),
|
||||
IsAdmin: boolPtr(false),
|
||||
OnlyFirstPurchase: boolPtr(true),
|
||||
EnableBalanceNotify: boolPtr(false),
|
||||
EnableLoginNotify: boolPtr(false),
|
||||
EnableSubscribeNotify: boolPtr(false),
|
||||
EnableTradeNotify: boolPtr(false),
|
||||
CreatedAt: s.Device.CreatedAt,
|
||||
UpdatedAt: now,
|
||||
}
|
||||
if err := tx.Create(&newUser).Error; err != nil {
|
||||
return fmt.Errorf("创建拆分用户失败(owner=%d, device=%d): %w", s.OwnerUID, s.Device.Id, err)
|
||||
}
|
||||
|
||||
// 2. 转移设备到新用户
|
||||
if err := tx.Model(&Device{}).Where("id = ?", s.Device.Id).
|
||||
Update("user_id", newUser.Id).Error; err != nil {
|
||||
return fmt.Errorf("转移设备失败: %w", err)
|
||||
}
|
||||
|
||||
// 3. 转移 auth_method 到新用户
|
||||
if s.AuthMethod != nil {
|
||||
if err := tx.Model(&AuthMethod{}).Where("id = ?", s.AuthMethod.Id).
|
||||
Update("user_id", newUser.Id).Error; err != nil {
|
||||
return fmt.Errorf("转移 auth_method 失败: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
// 4. 查找原用户的家庭组
|
||||
var family UserFamily
|
||||
if err := tx.Where("owner_user_id = ?", s.OwnerUID).First(&family).Error; err != nil {
|
||||
return fmt.Errorf("查找家庭组失败(owner=%d): %w", s.OwnerUID, err)
|
||||
}
|
||||
|
||||
// 5. 加入家庭组
|
||||
member := UserFamilyMember{
|
||||
FamilyId: family.Id,
|
||||
UserId: newUser.Id,
|
||||
Role: familyRoleMember,
|
||||
Status: 1,
|
||||
JoinSource: "migration_split",
|
||||
JoinedAt: s.Device.CreatedAt,
|
||||
CreatedAt: now,
|
||||
UpdatedAt: now,
|
||||
}
|
||||
if err := tx.Create(&member).Error; err != nil {
|
||||
return fmt.Errorf("添加家庭成员失败: %w", err)
|
||||
}
|
||||
|
||||
splitCount++
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
log.Fatalf("设备拆分失败: %v", err)
|
||||
}
|
||||
fmt.Printf("%d 个设备\n", splitCount)
|
||||
|
||||
// ── 结果 ──
|
||||
fmt.Println()
|
||||
fmt.Println("=== 迁移完成 ===")
|
||||
fmt.Printf(" 用户: %d (原始) + %d (拆分) = %d\n", len(users), splitCount, len(users)+splitCount)
|
||||
fmt.Printf(" 家庭组: %d\n", familyCount)
|
||||
fmt.Printf(" 家庭成员: %d (家主) + %d (拆分) = %d\n", familyCount, splitCount, familyCount+splitCount)
|
||||
fmt.Printf(" 订单: %d\n", len(orders))
|
||||
fmt.Printf(" 订阅: %d\n", len(subscribes))
|
||||
}
|
||||
|
||||
func boolPtr(b bool) *bool { return &b }
|
||||
Executable
BIN
Binary file not shown.
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user