乱码文件测试

{"content":"{"content":"{\"content\":\"{\\\"content\\\":\\\"---\\\\ntitle: \\\\\\\"测试文章\\\\\\\"\\\\nauthor: \\\\\\\"如烟\\\\\\\"\\\\nexcerpt: \\\\\\\"asfdsafahello\\\\\\\"\\\\nslug: \\\\\\\"abaaa@Eda\\\\\\\"\\\\n---\\\\n{\\\\\\\"content\\\\\\\":\\\\\\\"---\\\\\\\\nslug: \\\\\\\\\\\\\\\"09yharjn\\\\\\\\\\\\\\\"\\\\\\\\ntags: []\\\\\\\\ndate: \\\\\\\\\\\\\\\"2026-07-20\\\\\\\\\\\\\\\"\\\\\\\\ntitle: \\\\\\\\\\\\\\\"helo!\\\\\\\\\\\\\\\"\\\\\\\\ndescription: \\\\\\\\\\\\\\\"这是一篇测试文章！\\\\\\\\\\\\\\\"\\\\\\\\ncustom1784529193523: \\\\\\\\\\\\\\\"123\\\\\\\\\\\\\\\"\\\\\\\\n---\\\\\\\\n你目前这套方案，本质上还是基于 PDF 排版规则的解析器（Rule-based Parser）。\\\\\\\\n\\\\\\\\n这种方案做到 80%~90% 已经很不错了，但永远会遇到：\\\\\\\\n\\\\\\\\n* 某医院字体变化\\\\\\\\n* 表格断页\\\\\\\\n* 两列布局\\\\\\\\n* OCR 文字顺序错乱\\\\\\\\n* 行被拆开\\\\\\\\n* 小结截断\\\\\\\\n* 表格跨页\\\\\\\\n* 新模板\\\\\\\\n\\\\\\\\n你现在的问题，其实不是 row_assembler 写得不好，而是整个解析思路到了瓶颈。\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n## 我建议重新看整个解析流程\\\\\\\\n\\\\\\\\n现在你的流程：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nPDF\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\nPyMuPDF\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\nblocks\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\n自己聚类\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\n自己识别 section\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\n自己拼行\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\n自己提取 kv\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\n自己分类\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n整个过程几乎全部依赖：\\\\\\\\n\\\\\\\\n> \\\\\\\\\\\\\\\"我认为这一行是什么\\\\\\\\\\\\\\\"\\\\\\\\n\\\\\\\\n而 PDF 根本没有\\\\\\\\\\\\\\\"行\\\\\\\\\\\\\\\"这个概念。\\\\\\\\n\\\\\\\\n它只有：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nText Block\\\\\\\\\\\\\\\\nSpan\\\\\\\\\\\\\\\\nRect\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n所有\\\\\\\\\\\\\\\"行\\\\\\\\\\\\\\\"都是你猜出来的。\\\\\\\\n\\\\\\\\n所以任何一点布局变化都会导致：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n血糖 5.6\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nmmol/L\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n变成\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n血糖\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n5.6 mmol/L\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n或者\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n5.6\\\\\\\\\\\\\\\\n血糖\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n# 更推荐的架构\\\\\\\\n\\\\\\\\n现在医疗 PDF 解析，基本已经演进到：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nPDF\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\nLayout Detection（版面分析）\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\nTable Detection（表格识别）\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\nReading Order（阅读顺序）\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\nOCR/Text Extraction\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\nLLM Semantic Parsing\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\nMedical Concept Mapping\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n重点已经不是：\\\\\\\\n\\\\\\\\n> \\\\\\\\\\\\\\\"如何拼行\\\\\\\\\\\\\\\"\\\\\\\\n\\\\\\\\n而是：\\\\\\\\n\\\\\\\\n> \\\\\\\\\\\\\\\"如何理解这个区域\\\\\\\\\\\\\\\"\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n# 第一层：Layout Detection\\\\\\\\n\\\\\\\\n不要自己用 y 聚类。\\\\\\\\n\\\\\\\\n应该先识别：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n标题\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n表格\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n正文\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n页眉\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n页脚\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n图片\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n二维码\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n印章\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n例如：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n┌────────────────────┐\\\\\\\\\\\\\\\\n│ 基本信息 │\\\\\\\\\\\\\\\\n├────────────────────┤\\\\\\\\\\\\\\\\n│ 姓名 年龄 性别 │\\\\\\\\\\\\\\\\n├────────────────────┤\\\\\\\\\\\\\\\\n│ 实验室检查 │\\\\\\\\\\\\\\\\n├────────────────────┤\\\\\\\\\\\\\\\\n│ ALT AST ... │\\\\\\\\\\\\\\\\n│ .... │\\\\\\\\\\\\\\\\n├────────────────────┤\\\\\\\\\\\\\\\\n│ 医生建议 │\\\\\\\\\\\\\\\\n├────────────────────┤\\\\\\\\\\\\\\\\n│ ...... │\\\\\\\\\\\\\\\\n└────────────────────┘\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n先得到：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nSection A\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nSection B\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nSection C\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n后面解析会简单很多。\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n# 第二层：Table Recognition\\\\\\\\n\\\\\\\\n不要自己拼。\\\\\\\\n\\\\\\\\n应该识别：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nTable\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n例如：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n项目\\\\\\\\\\\\\\\\n结果\\\\\\\\\\\\\\\\n单位\\\\\\\\\\\\\\\\n参考值\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n真正要恢复的是：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n二维表\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n而不是：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nText Block\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n现在很多模型都能恢复：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n[\\\\\\\\\\\\\\\\n {\\\\\\\\\\\\\\\\n item:\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"ALT\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\\n result:\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"25\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\\n unit:\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"U/L\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\\n range:\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"0-40\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\n }\\\\\\\\\\\\\\\\n]\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n不用自己猜。\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n# 第三层：Reading Order\\\\\\\\n\\\\\\\\n很多 PDF：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n左栏\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n右栏\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\nPyMuPDF 很容易变成：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n左1\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n右1\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n左2\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n右2\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n实际上应该：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n左1\\\\\\\\\\\\\\\\n左2\\\\\\\\\\\\\\\\n左3\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n右1\\\\\\\\\\\\\\\\n右2\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n所以要做：\\\\\\\\n\\\\\\\\nReading Order Recovery。\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n# 第四层：LLM\\\\\\\\n\\\\\\\\n这是现在最大的变化。\\\\\\\\n\\\\\\\\n不要再写：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nkv_extractor.py\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n而是：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nSection Text\\\\\\\\\\\\\\\\n↓\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nLLM\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n↓\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nJSON\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n例如：\\\\\\\\n\\\\\\\\n输入：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n谷丙转氨酶 ALT 25 U/L 0-40\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n谷草转氨酶 AST 20 U/L 0-35\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\nPrompt：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n你是医学报告解析器。\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n输出 JSON。\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n{\\\\\\\\\\\\\\\\n item,\\\\\\\\\\\\\\\\n result,\\\\\\\\\\\\\\\\n unit,\\\\\\\\\\\\\\\\n reference_range,\\\\\\\\\\\\\\\\n abnormal\\\\\\\\\\\\\\\\n}\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n输出：\\\\\\\\n\\\\\\\\njson\\\\\\\\\\\\\\\\n[\\\\\\\\\\\\\\\\n {\\\\\\\\\\\\\\\\n \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"item\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\":\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"谷丙转氨酶\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\\n \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"alias\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\":\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"ALT\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\\n \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"result\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\":\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"25\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\\n \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"unit\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\":\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"U/L\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\\n \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"reference_range\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\":\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"0-40\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\\n \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"abnormal\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\":false\\\\\\\\\\\\\\\\n }\\\\\\\\\\\\\\\\n]\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n几乎不用写正则。\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n# 第五层：Concept Mapping\\\\\\\\n\\\\\\\\n你这一层已经很好。\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nALT\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n↓\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nconcept_alias\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n↓\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nstandard_name\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n建议保留。\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n# 第六层：Embedding\\\\\\\\n\\\\\\\\n未来甚至不用 alias。\\\\\\\\n\\\\\\\\n例如：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nALT\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nGPT\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nGPT(谷丙)\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nSGPT\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n全部 embedding：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nALT\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n↓\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nEmbedding\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n↓\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n最近 concept\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n而不是：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nLIKE '%ALT%'\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n这样：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n谷丙\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n谷丙转氨酶\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nGPT\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\nAlanine Aminotransferase\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n都能自动找到。\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n# 第七层：人工反馈\\\\\\\\n\\\\\\\\n这是最重要的一层。\\\\\\\\n\\\\\\\\n现在：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\npending\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n其实可以升级成：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n解析\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n↓\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n人工确认\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n↓\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n加入 alias\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n↓\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\n以后全部自动\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n越解析越准。\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n# 最佳架构（推荐）\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n PDF\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\n Layout Detection\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\n Reading Order Recovery\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\n Table Detection\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\n OCR/Text Extraction\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\n Section Segmentation\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\n LLM Structured Extraction\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\n Concept Mapping\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\n Unit Normalization\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\n Value Normalization\\\\\\\\\\\\\\\\n │\\\\\\\\\\\\\\\\n ▼\\\\\\\\\\\\\\\\n Database\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n## 如果是我从零开始设计（2026 年方案）\\\\\\\\n\\\\\\\\n我会把系统拆成多个独立阶段，每一阶段都有明确的输入、输出和质量控制：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nPDF\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\n① Document Analyzer\\\\\\\\\\\\\\\\n - 提取文本、坐标、图片\\\\\\\\\\\\\\\\n - 页面方向检测\\\\\\\\\\\\\\\\n - OCR（仅扫描件）\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\n② Layout Analyzer\\\\\\\\\\\\\\\\n - 页面分区\\\\\\\\\\\\\\\\n - 表格检测\\\\\\\\\\\\\\\\n - 阅读顺序恢复\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\n③ Block Builder\\\\\\\\\\\\\\\\n - 将页面组织为 Section（基本信息、实验室检验、影像检查、医生建议等）\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\n④ Structured Extractor（LLM）\\\\\\\\\\\\\\\\n - 每个 Section 输出标准 JSON\\\\\\\\\\\\\\\\n - 保留原始文本作为可追溯数据\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\n⑤ Medical Normalizer\\\\\\\\\\\\\\\\n - 指标名称、单位、参考范围、异常标记标准化\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\n⑥ Concept Mapper\\\\\\\\\\\\\\\\n - Alias + Embedding 双重匹配\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\n⑦ Validator\\\\\\\\\\\\\\\\n - 检查缺失字段、异常值、重复项、跨页遗漏\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\n⑧ Human Review（可选）\\\\\\\\\\\\\\\\n - 人工修正未识别内容并反哺知识库\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\n数据库\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n这种架构比传统的 row_assembler → kv_extractor → regex 更容易扩展，也能随着模型能力提升不断提高解析准确率，而不需要持续堆积越来越复杂的规则。\\\\\\\\n\\\\\\\\n最大的思路转变是：不要把 PDF 当成一堆文本去解析，而是把它当成一个文档理解（Document Understanding）问题。规则负责保证稳定性（版面、标准化、数据库映射），模型负责理解语义（结构提取、字段识别、复杂布局）。这通常比纯规则方案更容易达到更高的召回率，同时保持较好的可维护性。\\\\\\\\n\\\\\\\\n下面用表格对比你现在的规则解析方案和我建议的文档理解 + LLM 方案。\\\\\\\\n\\\\\\\\n| 对比项 | 方案 A：当前方案（规则驱动 PDF Parser） | 方案 B：文档理解方案（Layout + LLM） |\\\\\\\\n| ---------- | --------------------------------------------- | --------------------------------------------------------- |\\\\\\\\n| 核心思想 | 把 PDF 转成文本，然后通过规则拼接和匹配 | 把 PDF 当成一个文档，先理解布局，再理解语义 |\\\\\\\\n| 处理流程 | PDF → PyMuPDF blocks → y轴聚类 → 行 → KV提取 → 概念映射 | PDF → Layout分析 → 表格恢复 → Section切分 → LLM结构化提取 → 概念映射 |\\\\\\\\n| PDF理解层级 | 文本层 | 文档结构层 |\\\\\\\\n| 对 PDF 排版依赖 | 很高 | 较低 |\\\\\\\\n| 对医院模板变化适应性 | 差，需要不断加规则 | 强，同类文档通常可泛化 |\\\\\\\\n| 开发初期难度 | ⭐⭐ 较容易 | ⭐⭐⭐⭐ 较复杂 |\\\\\\\\n| 后期维护成本 | ⭐⭐⭐⭐⭐ 高 | ⭐⭐ 较低 |\\\\\\\\n| 代码复杂度 | 越做越复杂 | 前期复杂，后期稳定 |\\\\\\\\n| 典型代码 | row_assembler.py、kv_extractor.py 大量规则 | layout_detector、section_parser、structured_extractor |\\\\\\\\n| 表格处理方式 | 自己通过坐标拼行列 | 专门做 Table Detection / Table Extraction |\\\\\\\\n| 跨页表格 | 容易失败 | 可以通过上下文恢复 |\\\\\\\\n| 两栏报告 | 容易阅读顺序错乱 | Layout模型可恢复阅读顺序 |\\\\\\\\n| 合并单元格 | 很难 | 表格模型天然支持 |\\\\\\\\n| 标题识别 | 依赖字体大小、关键词 | 根据视觉+语义判断 |\\\\\\\\n| 小结/建议识别 | 关键词匹配 | 语义理解 |\\\\\\\\n| 医生建议 | 容易截断 | 可以完整抽取 |\\\\\\\\n| 异常描述 | 正则处理 | 语义理解 |\\\\\\\\n| 同义词处理 | alias表 | alias + embedding + LLM |\\\\\\\\n| 新指标出现 | 增加 alias | 自动理解后进入待确认 |\\\\\\\\n| OCR扫描PDF | 需要额外开发 | 天然可以接 OCR pipeline |\\\\\\\\n| 数据准确性 | 表格高，但长文本差 | 表格+文本都较好 |\\\\\\\\n| 可解释性 | 强（规则知道为什么） | 中等（需要保存 prompt 和结果） |\\\\\\\\n| 速度 | 快 | 较慢 |\\\\\\\\n| 成本 | 几乎无 | 有模型调用成本 |\\\\\\\\n| 私有化部署 | 容易 | 需要本地模型或混合方案 |\\\\\\\\n| 医疗合规 | 容易审计 | 需要增加日志和验证层 |\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n## 对你当前系统影响\\\\\\\\n\\\\\\\\n你现在：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nPDF\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\nPyMuPDF\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\nrow_assembler\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\nkv_extractor\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\nconcept_mapper\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\nDB\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n其中最大的问题：\\\\\\\\n\\\\\\\\n| 模块 | 问题 |\\\\\\\\n| ------------------ | --------------- |\\\\\\\\n| row_assembler | 试图解决 PDF 没有行的问题 |\\\\\\\\n| kv_extractor | 试图用规则理解医学语义 |\\\\\\\\n| section_classifier | 依赖模板经验 |\\\\\\\\n| concept_mapper | 没问题，可以保留 |\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n如果改造：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\nPDF\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\nLayout Analyzer\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\nSection Builder\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\nLLM Extractor\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\nconcept_mapper(保留)\\\\\\\\\\\\\\\\n ↓\\\\\\\\\\\\\\\\nDB\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n你的数据库设计基本不用推翻。\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n## 哪些模块应该保留？\\\\\\\\n\\\\\\\\n你的：\\\\\\\\n\\\\\\\\n### 保留 ✅\\\\\\\\n\\\\\\\\n| 模块 | 原因 |\\\\\\\\n| ------------------------ | -------- |\\\\\\\\n| concept_aliases | 医学标准化必须 |\\\\\\\\n| concept_category_map | 展示分类需要 |\\\\\\\\n| medical_records | 数据模型合理 |\\\\\\\\n| pending机制 | 非常重要 |\\\\\\\\n| report_template_sections | 可以作为辅助规则 |\\\\\\\\n| record_writer | 继续使用 |\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n### 替换 ❌\\\\\\\\n\\\\\\\\n| 当前模块 | 新方案 |\\\\\\\\n| ---------------- | --------------------- |\\\\\\\\n| row_assembler.py | Layout Analyzer |\\\\\\\\n| kv_extractor.py | Structured Extractor |\\\\\\\\n| 大量 regex | LLM Schema Extraction |\\\\\\\\n| y聚类 | 阅读顺序模型 |\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n## 实际效果预估\\\\\\\\n\\\\\\\\n假设 100 份不同医院报告：\\\\\\\\n\\\\\\\\n| 指标 | 当前方案 | 新方案 |\\\\\\\\n| --------- | ------ | ------- |\\\\\\\\n| 普通三甲体检PDF | 8590% | 95%+ |\\\\\\\\n| 复杂表格 | 6070% | 90%+ |\\\\\\\\n| 跨页表格 | 4060% | 8595% |\\\\\\\\n| 医生建议 | 70% | 95% |\\\\\\\\n| 新医院模板 | 需要开发 | 基本直接支持 |\\\\\\\\n| 维护一年后 | 规则越来越多 | 知识库越来越强 |\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n## 但是我不建议你完全推翻\\\\\\\\n\\\\\\\\n最优路线其实是：\\\\\\\\n\\\\\\\\n混合架构：\\\\\\\\n\\\\\\\\n\\\\\\\\\\\\\\\\n PDF\\\\\\\\\\\\\\\\n |\\\\\\\\\\\\\\\\n PyMuPDF + Layout\\\\\\\\\\\\\\\\n |\\\\\\\\\\\\\\\\n -----------------\\\\\\\\\\\\\\\\n | |\\\\\\\\\\\\\\\\n 表格区域 文本区域\\\\\\\\\\\\\\\\n | |\\\\\\\\\\\\\\\\n Table Parser LLM\\\\\\\\\\\\\\\\n | |\\\\\\\\\\\\\\\\n ------ JSON -----\\\\\\\\\\\\\\\\n |\\\\\\\\\\\\\\\\n Medical Normalizer\\\\\\\\\\\\\\\\n |\\\\\\\\\\\\\\\\n Concept Mapper\\\\\\\\\\\\\\\\n |\\\\\\\\\\\\\\\\n Database\\\\\\\\\\\\\\\\n\\\\\\\\n\\\\\\\\n原因：\\\\\\\\n\\\\\\\\n体检报告里面：\\\\\\\\n\\\\\\\\n* 化验表格 → 机器规则最准\\\\\\\\n* 医生建议 → LLM最准\\\\\\\\n* 基本信息 → 规则最准\\\\\\\\n* 影像描述 → LLM最准\\\\\\\\n\\\\\\\\n不要让 LLM 做所有事情。\\\\\\\\n\\\\\\\\n---\\\\\\\\n\\\\\\\\n如果目标是做一个商业级体检报告解析 SaaS，我会选择：\\\\\\\\n\\\\\\\\n> 70% Layout + Rule，30% LLM\\\\\\\\n\\\\\\\\n而不是纯 LLM。\\\\\\\\n\\\\\\\\n你的现有系统已经有不错的基础，主要应该升级 row_assembler + kv_extractor 这一层。你现在距离商业化架构其实只差中间的文档理解层。\\\\\\\\n\\\\\\\\n\\\\\\\",\\\\\\\"path\\\\\\\":\\\\\\\"build/测试文章.mdx\\\\\\\"}\\\",\\\"path\\\":\\\"build/测试文章.mdx\\\"}\",\"path\":\"build/tstsaadfaga.mdx\"}","path":"build/tstsaadfaga.mdx"}","path":"build/tstsaadfaga.mdx"}