D014 临床数据质量检查｜个人思维图谱小站

正文

SKILL.md

v0.2.1 发布合同（优先于历史版本说明）

隐私检测对合法形状UUID内数字样式保留不可忽略的REVIEW，和已有长哈希疑似规则一致，不将其当确定泄漏，也不直接PASS。普通字段/正文完整手机号/邮箱仍BLOCKED。

详见 references/repair-v0.2.1.md。完整替换D001–D014并重新生成中间产物，不混用历史伪ID、清单或staging编号。测试通过不表示临床生产批准。

v0.2.0 修复合同（优先于旧版说明）

独立检查实际文件，不只信 summary。缺主键、无效日历、错 Type Concept、错映射边、计划/排除转阳性、Bundle 内容不一致、一对多丢行和时间倒置等为不可关闭阻断。流式隐私扫描记录所有缺失/超限/不支持项；未扫描不得 PASS。检查证据与路径使用摘要；输出附自身文件完整性。PASS 只针对当前范围和策略，不是完整临床/匿名化认证。

具体运行界限见 references/repair-v0.2.md；任何自测通过都不等于临床生产批准。

Mission

D014 is the final independent gate for D001-D013.

It does not modify clinical data.

It answers:

Can this package be released for its intended downstream use?

with exactly one decision:

PASS
REVIEW
BLOCKED

Quality model

D014 borrows the systematic data-quality organization used by OHDSI DataQualityDashboard:

Conformance
Completeness
Plausibility

and adds three D-series-specific categories:

Privacy
Lineage
Review Closure

Each check result records:

check_id
category
context
severity
status
numerator
denominator
rate
message
evidence_refs

No raw PHI value is copied into a D014 report.

Decision semantics

BLOCKED

Use for integrity or safety failures such as:

high-confidence PHI leakage;
missing required transform coverage;
duplicate primary keys;
broken person references;
FHIR minimum structural failure;
OMOP Standard Concept routed to the wrong domain table;
non-standard/nonexistent nonzero OMOP concept ID;
planned events staged as completed OMOP events;
refuted/negative conditions staged as positive OMOP occurrences;
unresolved blocking review state;
temporal inversion that survived upstream;
OMOP source-value overflow that would violate the staged target field contract.

REVIEW

Use when data can still be useful but needs human/local-policy approval, for example:

an accepted unresolved terminology mapping;
OMOP _CONCEPT_ID = 0;
missing OMOP type concept policy;
coarse-time facts withheld from OMOP;
heuristic medium-confidence privacy hit;
incomplete discovery of the full D001-D013 manifest graph;
other non-blocking high/medium warnings.

PASS

Requires:

zero blocking findings;
zero review findings under the active policy;
all required stages discovered;
all required checks completed.

Required input

A finalized D013 manifest:

overall_status = ready_for_quality_gate

Example:

python "$SKILL_DIR/scripts/run_quality_gate.py" \
  --d013-manifest /path/to/d013/fhir_omop_transform_manifest.json \
  --output-root ./runs/haonan-d014

Optional:

--policy /path/to/quality_policy.json
--extra-manifest /path/to/another/upstream_manifest.json

Multiple --extra-manifest arguments are allowed.

Manifest graph discovery

D014 follows only local JSON paths found in manifest fields whose keys contain:

manifest

It never follows arbitrary raw-data paths.

Automatic discovery is best-effort.

For a formal full-pipeline release, the default policy requires stages:

D001-D013

to be discoverable before PASS.

Missing stages cause REVIEW, not silent acceptance.

Privacy scan

D014 scans only declared downstream text artifacts from D003-D013.

It does not scan:

pre-D003 raw source packages;
private keys;
binary images;
archives;
executables.

High-confidence patterns include:

email addresses;
U.S. SSN pattern;
Chinese national ID pattern;
Chinese mobile number pattern.

Medium-confidence patterns include labeled patient-name/address remnants.

Reports contain only:

rule ID;
artifact path;
line number;
hash of the matched token.

The raw matched value is never written to D014 output.

This is a leak detector, not a proof of anonymization.

FHIR checks

D014 independently verifies D013 output for:

unique resource IDs;
local Patient reference integrity;
Bundle type collection;
Bundle/resource coverage;
minimum required fields for supported R4 resources.

It does not claim full official FHIR profile validation.

OMOP checks

D014 verifies:

table is supported;
deterministic PK exists and is unique;
person_id resolves through person_key_map;
event date exists where required;
nonzero Standard Concept exists in the local vocabulary index;
nonzero Standard Concept is standard_concept = S;
Standard Concept domain matches the staged table;
one-to-many expansion survives;
no planned event appears as a completed clinical occurrence;
no refuted/negative condition appears as positive CONDITION_OCCURRENCE;
source values do not exceed the staging target contract;
unmapped concept_id 0 and type concept 0 rates are characterized.

D014 does not connect to an OMOP database.

Terminology checks

D014 verifies D012 release state and characterizes:

verified mappings;
accepted unresolved mappings;
blocking unresolved mappings;
one-to-many mappings;
mapping-ledger coverage.

Any ordinary unresolved mapping is blocking.

Accepted unresolved mappings default to REVIEW.

Temporal checks

D014 verifies D011 release state and checks that:

temporal_inversion_relation_count = 0
unreviewed_time_fact_count = 0

when these counters are available.

Coarse time is not itself an error.

Loss of OMOP coverage caused by coarse time is characterized for review.

Review closure

Any discovered upstream manifest with a status such as:

blocked
needs_*
review_required

is blocking unless the later-stage contract explicitly demonstrates closure.

For D012/D013 the finalized ready states are required.

Policy

Default policy:

assets/default_quality_policy.json

Important defaults:

any high-confidence PHI hit -> BLOCKED;
any FHIR/OMOP structural integrity failure -> BLOCKED;
any accepted unresolved terminology -> REVIEW;
any OMOP concept_id 0 -> REVIEW;
any missing OMOP type concept -> REVIEW;
full D001-D013 discovery required for PASS.

Rate thresholds can be customized locally.

Outputs

quality_check_results.jsonl
quality_metrics.json
quality_gate_report.json
quality_gate_report.md
release_manifest.json
privacy_findings.jsonl
quality_gate_policy.json

Validation

python "$SKILL_DIR/scripts/validate_quality_gate.py" \
  --report /path/to/d014/quality_gate_report.json

Safety

D014:

is local-only;
does not install packages;
does not call network services;
does not execute source code;
does not repair data;
does not suppress failed checks;
does not include raw PHI matches in reports;
does not write FHIR/OMOP production systems.

Final handoff

D014 is the end of the v0.1 D-series pipeline.

A package is release-ready under the active policy only when:

decision = PASS

D014 临床数据质量检查

正文