fix(cms): 整站重建后自动清扫孤儿静态产物

iwangbowen · Copilot · iwangbowen · commit 1156cd57c326 · 2026-07-25T23:59:30.000+08:00
问题
静态产物路径由栏目路径、详情页归档规则、内容 slug/staticPath 共同决定，任一变更都会
让旧路径下的文件失去归属。此前只有内容级增量刷新会按快照 deletePaths 精确删除旧路径，
栏目改路径/改归档规则、站点改主题等触发的整站重建只写新文件、不清旧文件，孤儿会持续
堆积；在 hybrid/static 模式下旧 URL 还会一直返回过期内容。clearSiteStatic() 虽已存在
但从未被调用，说明该问题长期无人兜底。

方案（mark &amp; sweep）
- mark：buildSiteStatic 用 AsyncLocalStorage 收集本次写入的全部相对路径。选择
  AsyncLocalStorage 而非层层透传，是因为写文件散落在 writeRenderedPath /
  regenerateChannelPages / refreshHomeStaticForChannel 等多个 helper 中，透传易漏，
  漏一处就会把有效产物误判成孤儿删掉；该模式与 cms-publish-artifact-tracker 一致，
  且天然按调用栈隔离，多站点串行重建互不串扰
- sweep：构建成功结束时递归遍历站点静态目录，删除不在写入集合中的文件，
  并自底向上回收空目录。删除走 deleteStaticFile，因此受发布围栏保护，
  且每个被删文件都会落一条 deleted 产物记录可供审计
- 清扫数量回填任务结果 prunedArtifacts

安全边界
- 断点续跑（resumeAfterKey 非空）跳过清扫：续跑只写了后半程，集合不完整
- 取消的构建一律 pruned: 0
- 增量刷新路径不受影响，仍按快照精确删除

路径归一化复用 pathToStaticFile：写入侧记的是 URL 形态（`news/`、``），磁盘上是
`news/index.html`、`index.html`。实测中两侧映射不一致会导致首页与全部栏目页被误判成
孤儿删光，已加回归用例锁定；另加接线断言防止后人移除续跑守卫。

Co-authored-by: Copilot App &lt;223556219+Copilot@users.noreply.github.com&gt;
diff --git a/docs/cms/static-and-render.md b/docs/cms/static-and-render.md
@@ -69,7 +69,29 @@
 - 日期类规则取 `publishedAt`，未发布时回退 `createdAt`；两者都缺失则退化为不归档，不会产生 `undefined` 目录
 - 规则由 `contentUrl()` 统一计算，静态化写文件与模板生成链接共用同一函数，不存在两侧算不一致的可能
 
-> ⚠️ **改动规则会使旧产物成为孤儿**。栏目保存会触发整站重建，新路径产物即时生成；旧路径下的文件不会自动删除（与「修改栏目 slug/path」的既有行为一致），需要时用 `cms_redirects` 配 301 或手工清理静态目录。
+> ⚠️ **改动规则会使旧产物成为孤儿**。栏目保存会触发整站重建，新路径产物即时生成；旧路径下的文件由重建结束时的**孤儿清扫**自动删除（见下），无需手工清理。若需保留旧 URL 可访问，请在「SEO 管理 → 301 重定向」中配置跳转。
+
+### 孤儿产物清扫
+
+静态产物的路径由栏目路径、归档规则、内容 slug/staticPath 等共同决定，任何一项变更都会让旧路径下的文件失去归属。**整站重建结束时会自动清扫这些孤儿**：
+
+- **mark**：构建过程中用 AsyncLocalStorage 收集本次写入的全部相对路径（含各发布通道的 `__{code}/` 子树）
+- **sweep**：递归遍历站点静态目录，删除不在写入集合中的文件，并自底向上回收空目录
+- 删除走 `deleteStaticFile`，因此同样受发布围栏保护，且每个被删文件都会落一条 `deleted` 产物记录，可在发布中心审计
+- 清扫数量回填任务结果 `prunedArtifacts`
+
+**触发时机**：任意整站重建。栏目改路径/改归档规则、站点改主题等操作本身就会触发整站重建，因此保存后即自愈，无需额外动作。
+
+**安全边界**：
+
+| 情形 | 行为 |
+| --- | --- |
+| 断点续跑的构建（`resumeAfterKey` 非空） | **跳过清扫** —— 续跑只写了后半程，集合不完整，清扫会误删前半程的有效产物 |
+| 被取消的构建 | **跳过清扫**（提前返回一律 `pruned: 0`） |
+| 内容级 / 栏目级增量刷新 | 不清扫，仅按快照 `deletePaths` 精确删除旧路径 |
+| hybrid 按需回写的深分页（如标签页第 2 页） | 会被清扫；下次访问自动重新生成。`static` 模式下这类页面本就不生成，无影响 |
+
+> 路径归一化复用 `pathToStaticFile`：写入侧记录的是 URL 形态（`news/`、``），磁盘上是 `news/index.html`、`index.html`。两侧若不用同一套映射，首页与全部栏目页会被误判成孤儿。
 
 ### 页面区块展示条件的静态安全策略
 
diff --git a/packages/server/src/services/cms/cms-publishing.service.ts b/packages/server/src/services/cms/cms-publishing.service.ts
@@ -620,6 +620,7 @@ export function registerCmsPublishingTaskHandler(): void {
         eq(cmsPublishArtifacts.path, `@target/${input.targetType}`),
       ));
       const tracked = await trackingContext(input, ctx.taskId, ctx);
+      let prunedArtifacts = 0;
       try {
         if (input.targetType === 'content' || input.targetType === 'contents') {
           const snapshots = [...(input.contentSnapshots ?? [])].sort((a, b) => a.contentId - b.contentId);
@@ -667,7 +668,7 @@ export function registerCmsPublishingTaskHandler(): void {
           }));
           await ctx.progress({ processed: 1, failed: tracked.artifactProgress.failed, total: 1, note: '搭建页面重建完成', checkpoint: { phase: 'page', lastId: input.pageId ?? null, pageSlug: slug } });
         } else {
-          await withCmsPublishArtifactTracking(tracked.context, () => buildSiteStatic(input.siteId, async (progress) => {
+          const build = await withCmsPublishArtifactTracking(tracked.context, () => buildSiteStatic(input.siteId, async (progress) => {
             const state = await ctx.progress({
               processed: progress.processed,
               failed: tracked.artifactProgress.failed,
@@ -677,6 +678,7 @@ export function registerCmsPublishingTaskHandler(): void {
             });
             return state.cancelRequested;
           }, { resumeAfterKey: typeof ctx.checkpoint?.lastKey === 'string' ? ctx.checkpoint.lastKey : null }));
+          prunedArtifacts = build.pruned;
         }
       } catch (error) {
         if (error instanceof TaskCancelledError) throw error;
@@ -690,6 +692,7 @@ export function registerCmsPublishingTaskHandler(): void {
       return {
         artifacts: tracked.artifactProgress.count,
         failedArtifacts: tracked.artifactProgress.failed,
+        prunedArtifacts,
         targetType: input.targetType,
       };
       }).catch(async (error) => {
diff --git a/packages/server/src/services/cms/cms-static-prune.test.ts b/packages/server/src/services/cms/cms-static-prune.test.ts
@@ -0,0 +1,127 @@
+import fs from 'node:fs/promises';
+import { readFile } from 'node:fs/promises';
+import path from 'node:path';
+import { afterEach, describe, expect, it, vi } from 'vitest';
+
+const { recordCmsPublishArtifact } = vi.hoisted(() => ({ recordCmsPublishArtifact: vi.fn() }));
+vi.mock('./cms-publish-artifact-tracker', () => ({ recordCmsPublishArtifact }));
+
+import { pruneOrphanStaticFiles, resolveStaticFile, siteStaticDir } from './cms-static.service';
+
+const SITE_CODE = 'prune-orphan-test';
+
+async function seed(files: string[]): Promise<void> {
+  for (const rel of files) {
+    const abs = resolveStaticFile(SITE_CODE, rel)!;
+    await fs.mkdir(path.dirname(abs), { recursive: true });
+    await fs.writeFile(abs, `<html>${rel}</html>`);
+  }
+}
+
+async function listAll(): Promise<string[]> {
+  const dir = siteStaticDir(SITE_CODE);
+  const walk = async (d: string, prefix = ''): Promise<string[]> => {
+    const entries = await fs.readdir(d, { withFileTypes: true }).catch(() => []);
+    const out: string[] = [];
+    for (const e of entries) {
+      const rel = prefix ? `${prefix}/${e.name}` : e.name;
+      if (e.isDirectory()) out.push(...await walk(path.join(d, e.name), rel));
+      else out.push(rel);
+    }
+    return out;
+  };
+  return (await walk(dir)).sort();
+}
+
+afterEach(async () => {
+  vi.clearAllMocks();
+  await fs.rm(siteStaticDir(SITE_CODE), { recursive: true, force: true });
+});
+
+describe('CMS 孤儿静态产物清扫', () => {
+  it('删除本次未写入的文件，保留写入集合内的文件', async () => {
+    await seed(['index.html', 'news/2024/1.html', 'news/1.html', 'news/index.html']);
+
+    // 改归档规则后：新产物在 news/2024/，旧的 news/1.html 成为孤儿
+    const kept = new Set(['index.html', 'news/2024/1.html', 'news/index.html']);
+    await expect(pruneOrphanStaticFiles(SITE_CODE, kept)).resolves.toBe(1);
+
+    expect(await listAll()).toEqual(['index.html', 'news/2024/1.html', 'news/index.html']);
+  });
+
+  it('清扫后回收空目录，不残留空壳', async () => {
+    await seed(['index.html', 'news/2023/9.html']);
+
+    await expect(pruneOrphanStaticFiles(SITE_CODE, new Set(['index.html']))).resolves.toBe(1);
+
+    expect(await listAll()).toEqual(['index.html']);
+    // 2023/ 与 news/ 均已随之回收
+    await expect(fs.readdir(path.join(siteStaticDir(SITE_CODE), 'news'))).rejects.toThrow();
+  });
+
+  it('非默认通道子树同样纳入清扫范围', async () => {
+    await seed(['__h5/news/1.html', '__h5/news/2024/1.html']);
+
+    await expect(pruneOrphanStaticFiles(SITE_CODE, new Set(['__h5/news/2024/1.html']))).resolves.toBe(1);
+
+    expect(await listAll()).toEqual(['__h5/news/2024/1.html']);
+  });
+
+  it('写入集合覆盖全部文件时不做任何删除', async () => {
+    const files = ['index.html', 'news/index.html'];
+    await seed(files);
+
+    await expect(pruneOrphanStaticFiles(SITE_CODE, new Set(files))).resolves.toBe(0);
+
+    expect(await listAll()).toEqual(files.sort());
+    expect(recordCmsPublishArtifact).not.toHaveBeenCalled();
+  });
+
+  it('每个被删文件都落一条 deleted 产物记录，保证可审计', async () => {
+    await seed(['a.html', 'b.html']);
+
+    await expect(pruneOrphanStaticFiles(SITE_CODE, new Set())).resolves.toBe(2);
+
+    expect(recordCmsPublishArtifact).toHaveBeenCalledTimes(2);
+    expect(recordCmsPublishArtifact).toHaveBeenCalledWith({ relPath: 'a.html', status: 'deleted' });
+    expect(recordCmsPublishArtifact).toHaveBeenCalledWith({ relPath: 'b.html', status: 'deleted' });
+  });
+
+  it('目录不存在时安全返回 0（站点从未发布过）', async () => {
+    await expect(pruneOrphanStaticFiles(SITE_CODE, new Set(['index.html']))).resolves.toBe(0);
+  });
+
+  // 回归：写入侧记的是 URL 形态（`news/`、``），磁盘上却是 `news/index.html`、`index.html`。
+  // 归一化若不复用 pathToStaticFile，首页与所有栏目页会被误判成孤儿删光。
+  it('URL 形态的目录路径与磁盘 index.html 视为同一文件', async () => {
+    await seed(['index.html', 'news/index.html', 'tag/release/index.html', 'news/1.html']);
+
+    const kept = new Set(['', 'news/', 'tag/release/']);
+    await expect(pruneOrphanStaticFiles(SITE_CODE, kept)).resolves.toBe(1);
+
+    expect(await listAll()).toEqual(['index.html', 'news/index.html', 'tag/release/index.html']);
+  });
+
+  it('非默认通道的目录形态路径同样正确归一', async () => {
+    await seed(['__h5/index.html', '__h5/news/index.html', '__h5/stale.html']);
+
+    await expect(pruneOrphanStaticFiles(SITE_CODE, new Set(['__h5/', '__h5/news/']))).resolves.toBe(1);
+
+    expect(await listAll()).toEqual(['__h5/index.html', '__h5/news/index.html']);
+  });
+});
+
+describe('CMS 孤儿清扫的接线约束', () => {
+  it('仅在未断点续跑时清扫（续跑集合不完整，误删将丢失有效产物）', async () => {
+    const text = await readFile(new URL('./cms-static.service.ts', import.meta.url), 'utf8');
+    expect(text).toMatch(/if \(resumeAfterKey == null\) \{[\s\S]{0,200}?pruneOrphanStaticFiles\(site\.code, written\)/);
+  });
+
+  it('取消构建的提前返回一律不清扫', async () => {
+    const text = await readFile(new URL('./cms-static.service.ts', import.meta.url), 'utf8');
+    // 所有 report(...) 触发的提前返回都必须是 pruned: 0
+    const earlyReturns = text.match(/\)\) return \{ pages[^}]*\}/g) ?? [];
+    expect(earlyReturns.length).toBeGreaterThan(0);
+    for (const ret of earlyReturns) expect(ret).toContain('pruned: 0');
+  });
+});
diff --git a/packages/server/src/services/cms/cms-static.service.ts b/packages/server/src/services/cms/cms-static.service.ts
@@ -1,5 +1,6 @@
 import path from 'node:path';
 import fs from 'node:fs/promises';
+import { AsyncLocalStorage } from 'node:async_hooks';
 import { eq, and, isNull, asc } from 'drizzle-orm';
 import { db } from '../../db';
 import { cmsChannels, cmsContents } from '../../db/schema';
@@ -17,7 +18,7 @@ import { triggerCdnPurge, triggerCdnPurgeAll } from './cms-cdn.service';
 
 // ─── 静态目录 ─────────────────────────────────────────────────────────────────
 import {
-  CMS_STATIC_ROOT, isStrictlyWithin, resolveStaticFile, siteStaticDir,
+  CMS_STATIC_ROOT, isStrictlyWithin, pathToStaticFile, resolveStaticFile, siteStaticDir,
 } from './cms-static-path';
 import { assertSiteAccess } from './cms-sites.service';
 import { resolveEffectiveCmsSiteRow } from './cms-site-inheritance.service';
@@ -57,6 +58,7 @@ export async function writeStaticFile(siteCode: string, relPath: string, html: s
     await fs.writeFile(tmp, html, 'utf8');
     await assertCmsStaticWriteFence();
     await fs.rename(tmp, abs);
+    buildWriteCollector.getStore()?.add(normalizeStaticRelPath(relPath));
     await recordCmsPublishArtifact({ relPath, status: 'generated', content: html });
   } catch (error) {
     await fs.rm(tmp, { force: true }).catch(() => undefined);
@@ -101,6 +103,75 @@ export async function clearSiteStatic(siteCode: string): Promise<void> {
   await fs.rm(dir, { recursive: true, force: true });
 }
 
+// ─── 孤儿产物清扫（全量重建的 mark & sweep）─────────────────────────────────────
+/**
+ * 本次构建写入的相对路径集合。
+ *
+ * 用 AsyncLocalStorage 而非层层透传：写文件散落在 writeRenderedPath / regenerateChannelPages /
+ * refreshHomeStaticForChannel 等多个 helper 中，透传参数既啰嗦又容易漏，漏一处就会把有效产物
+ * 误判成孤儿删掉。与 cms-publish-artifact-tracker 采用同一模式，且天然按调用栈隔离，
+ * 多站点/并发构建互不串扰。
+ */
+const buildWriteCollector = new AsyncLocalStorage<Set<string>>();
+
+/**
+ * 统一成磁盘上的实际相对路径，供集合比对。
+ *
+ * 必须复用 `pathToStaticFile`：写入侧记的是 URL 形态（`news/`、``），
+ * 磁盘上却是 `news/index.html`、`index.html`。只做斜杠归一会让首页/栏目页
+ * 在集合里查不到，被当成孤儿全部删掉。
+ */
+function normalizeStaticRelPath(relPath: string): string {
+  try {
+    return pathToStaticFile(relPath).replaceAll('\\', '/');
+  } catch {
+    return relPath.replaceAll('\\', '/').replace(/^\/+/, '');
+  }
+}
+
+/** 递归列出站点静态目录下的全部文件（返回归一化相对路径） */
+async function listSiteStaticFiles(dir: string, prefix = ''): Promise<string[]> {
+  const entries = await fs.readdir(dir, { withFileTypes: true }).catch(() => []);
+  const files: string[] = [];
+  for (const entry of entries) {
+    const rel = prefix ? `${prefix}/${entry.name}` : entry.name;
+    if (entry.isDirectory()) files.push(...await listSiteStaticFiles(path.join(dir, entry.name), rel));
+    else if (entry.isFile()) files.push(rel);
+  }
+  return files;
+}
+
+/** 自底向上删除空目录（清扫后残留的空壳目录，如改归档规则后的旧年份目录） */
+async function removeEmptyDirs(dir: string): Promise<void> {
+  const entries = await fs.readdir(dir, { withFileTypes: true }).catch(() => []);
+  for (const entry of entries) {
+    if (entry.isDirectory()) await removeEmptyDirs(path.join(dir, entry.name));
+  }
+  const rest = await fs.readdir(dir).catch(() => ['keep']);
+  if (rest.length === 0) await fs.rmdir(dir).catch(() => undefined);
+}
+
+/**
+ * 清扫孤儿产物：删除站点静态目录下本次全量构建未写入的文件。
+ *
+ * 只在「未断点续跑、且完整跑完」的整站重建后调用 —— 续跑/取消时 kept 集合不完整，
+ * 清扫会误删有效产物。删除走 deleteStaticFile，因此同样受发布围栏保护并留下 deleted 产物记录。
+ */
+export async function pruneOrphanStaticFiles(siteCode: string, kept: ReadonlySet<string>): Promise<number> {
+  const dir = siteStaticDir(siteCode);
+  if (!isStrictlyWithin(CMS_STATIC_ROOT, dir)) throw new Error('CMS 站点静态目录越界');
+  // 入参可能混用 URL 形态（`news/`）与磁盘形态（`news/index.html`），统一归一后再比对
+  const keptFiles = new Set([...kept].map(normalizeStaticRelPath));
+  const existing = await listSiteStaticFiles(dir);
+  let removed = 0;
+  for (const relPath of existing) {
+    if (keptFiles.has(relPath)) continue;
+    if (await deleteStaticFile(siteCode, relPath)) removed += 1;
+  }
+  if (removed > 0) await removeEmptyDirs(dir);
+  return removed;
+}
+
 // ─── sitemap / robots ─────────────────────────────────────────────────────────
 function xmlEscape(s: string): string {
   return s.replaceAll('&', '&amp;').replaceAll('<', '&lt;').replaceAll('>', '&gt;').replaceAll('"', '&quot;').replaceAll("'", '&apos;');
@@ -460,7 +531,17 @@ export async function buildSiteStatic(
   siteId: number,
   onProgress?: (p: FullBuildProgress) => Promise<boolean | void>,
   options?: { resumeAfterKey?: string | null },
-): Promise<{ pages: number }> {
+): Promise<{ pages: number; pruned: number }> {
+  const written = new Set<string>();
+  return buildWriteCollector.run(written, () => buildSiteStaticInner(siteId, written, onProgress, options));
+}
+
+async function buildSiteStaticInner(
+  siteId: number,
+  written: Set<string>,
+  onProgress?: (p: FullBuildProgress) => Promise<boolean | void>,
+  options?: { resumeAfterKey?: string | null },
+): Promise<{ pages: number; pruned: number }> {
   const site = await resolveEffectiveCmsSiteRow(siteId).catch(() => null);
   if (!site) throw new Error(`站点不存在（id=${siteId}）`);
 
@@ -508,7 +589,7 @@ export async function buildSiteStatic(
       if (await refreshHomeStaticForChannel(site, publishChannel)) pages += 1;
       if (await report(`${channelLabel}首页已生成`, {
         phase: 'home', lastKey: homeKey, lastId: null, publishChannelCode: publishChannel.code,
-      })) return { pages };
+      })) return { pages, pruned: 0 };
     }
 
     for (const channel of channels) {
@@ -517,7 +598,7 @@ export async function buildSiteStatic(
       pages += await regenerateChannelPages(site, channel, publishChannel);
       if (await report(`${channelLabel}栏目「${channel.name}」已生成`, {
         phase: 'channel', lastKey: key, lastId: channel.id, publishChannelCode: publishChannel.code,
-      })) return { pages };
+      })) return { pages, pruned: 0 };
     }
 
     for (const row of contents) {
@@ -533,7 +614,7 @@ export async function buildSiteStatic(
       }
       if (await report(`${channelLabel}内容 ${row.id} 已生成`, {
         phase: 'content', lastKey: key, lastId: row.id, publishChannelCode: publishChannel.code,
-      })) return { pages };
+      })) return { pages, pruned: 0 };
     }
 
     // 标签聚合页（仅首屏分页；深分页访问时由 hybrid 模式按需回写）
@@ -547,7 +628,7 @@ export async function buildSiteStatic(
       }
       if (await report(`${channelLabel}标签「${tag.name}」已生成`, {
         phase: 'tag', lastKey: key, lastId: tag.id, publishChannelCode: publishChannel.code,
-      })) return { pages };
+      })) return { pages, pruned: 0 };
     }
 
     // 可视化搭建页面 /p/{slug}/
@@ -561,25 +642,32 @@ export async function buildSiteStatic(
       }
       if (await report(`${channelLabel}搭建页「${page.name}」已生成`, {
         phase: 'page', lastKey: key, lastId: page.id, publishChannelCode: publishChannel.code,
-      })) return { pages };
+      })) return { pages, pruned: 0 };
     }
   }
 
   const sitemapKey = cmsStaticTargetKey('~meta', 0, 1);
   if (!skipCompleted(sitemapKey)) {
     await writeStaticFile(site.code, 'sitemap.xml', await generateSitemapXml(site));
-    if (await report('sitemap.xml 已生成', { phase: 'meta', lastKey: sitemapKey, lastId: 1, publishChannelCode: null })) return { pages };
+    if (await report('sitemap.xml 已生成', { phase: 'meta', lastKey: sitemapKey, lastId: 1, publishChannelCode: null })) return { pages, pruned: 0 };
   }
   const rssKey = cmsStaticTargetKey('~meta', 0, 2);
   if (!skipCompleted(rssKey)) {
     await writeStaticFile(site.code, 'rss.xml', await generateRssXml(site));
-    if (await report('rss.xml 已生成', { phase: 'meta', lastKey: rssKey, lastId: 2, publishChannelCode: null })) return { pages };
+    if (await report('rss.xml 已生成', { phase: 'meta', lastKey: rssKey, lastId: 2, publishChannelCode: null })) return { pages, pruned: 0 };
   }
   const robotsKey = cmsStaticTargetKey('~meta', 0, 3);
   if (!skipCompleted(robotsKey)) {
     await writeStaticFile(site.code, 'robots.txt', buildRobotsTxt(site));
-    if (await report('robots.txt 已生成', { phase: 'meta', lastKey: robotsKey, lastId: 3, publishChannelCode: null })) return { pages };
+    if (await report('robots.txt 已生成', { phase: 'meta', lastKey: robotsKey, lastId: 3, publishChannelCode: null })) return { pages, pruned: 0 };
   }
   triggerCdnPurgeAll(site);
-  return { pages };
+  // 孤儿清扫：仅在「从头完整跑完」时执行。断点续跑的 written 集合只含续跑段，
+  // 直接清扫会把前半程的有效产物当成孤儿删掉，故明确跳过。
+  let pruned = 0;
+  if (resumeAfterKey == null) {
+    pruned = await pruneOrphanStaticFiles(site.code, written);
+    if (pruned > 0) logger.info(`[CMS] 站点 ${site.code} 全量重建清理孤儿产物 ${pruned} 个`);
+  }
+  return { pages, pruned };
 }
diff --git a/packages/server/src/services/cms/cms-tasks.ts b/packages/server/src/services/cms/cms-tasks.ts
