Research question determines data set structure
The design of a longitudinal data set cannot start with the folder structure, but must first clarify what questions the model or study wants to answer. Early identification of disease requires clear baseline time points and follow-up observation windows; treatment response analysis requires the sequence between drug or surgical exposure, test results, and outcomes; prediction of recurrence or readmission requires definition of index date, follow-up duration, outcome events, and loss-to-follow-up rules. Without these definitions, "three-year time span" is just a broad collection requirement and does not guarantee that the sample is sufficient to support specific analysis.
It is recommended to clarify four things for each case: what is the starting point, how to calculate the observation period, how to determine the end point, and which records belong to this trajectory. The starting point may be the date of first diagnosis, pretreatment evaluation, admission, or a test; the end point may be recurrence, progression, death, readmission, remission, or reaching a specified follow-up deadline. For patients with "unobserved events", it is necessary to distinguish between "follow-up is complete but did not occur", "lost to follow-up early" and "no follow-up data in the recording system". Writing these statuses as "no outcome" will cause labeling errors and follow-up bias.
The Delphi-2M study published in "Nature" in 2025 used disease timelines to model the future risk of multiple diseases, and was externally validated with Danish registry data after training in the British cohort, demonstrating the potential of time series data for disease process modeling. The study also pointed out that the model may learn selection bias and missing patterns in the data set, so the prediction performance cannot be explained in isolation from the cohort composition and data generation mechanism.[1] This has direct implications for data construction: timestamps, reasons for termination of follow-up, source institutions, and missing mechanisms are all contexts that the model needs to know.
Patients, visits, events and files need to be hierarchically related
recommends splitting longitudinal data into four interrelated levels. The patient layer uses anonymous and stable patient identifiers and is responsible for stringing together multiple visits of the same person; the visit layer records outpatient, hospitalization, emergency department, transfer or follow-up events; the clinical event layer records diagnosis, symptoms, examinations, tests, medications, operations and outcomes; the file layer stores original carriers such as disease course documents, examination reports, DICOM, pathological slides and so on. Each layer should have unique keys and be related by explicit foreign keys, rather than relying on guessing relationships by file names or directory locations.
Patient-level stable ID is not meant to preserve reversible identity information. It should be a pseudonymized key within the project scope, the real identity mapping table is kept separately according to the authorization requirements, and the delivery data does not carry the mapping. The scope of treatment ID, report ID, imaging Study/Series ID and document ID also need to be clearly defined to prevent two different hospitals or different batches from being mistakenly merged after generating the same number. Cross-system merging should record matching rules, confidence and manual review status; when reliable matching cannot be achieved, retaining the unassociated status is better than forcibly splicing.
The time field must at least distinguish event occurrence time, collection time, report generation time, entry time, modification time and export time. They are often different in medical records. Test specimens may be collected first and results will be released later; disease course records may review and describe previous events; system export time cannot replace the time of clinical events. Preserving the time zone, precision, source fields and missing reasons allows the algorithm team to determine which point in time should be used as visible information to avoid leaking data after the result is generated into the model input. The source system, time zone, precision, and conversion rules should also be described for each type of time field. When there is only a date without time, date-level accuracy should be retained and the specific time should not be made up; when there is cross-hospital transfer, subsequent supplementary recording, or retrospective sorting, the uncertainty of the event time should be marked. Missingness should also be differentiated between not occurring, not acquired, not recorded, unable to be associated, and missed delivery, as these states are not identical for clinical interpretation and model bias analysis. For the prediction task, the data dictionary must clearly define the baseline time point, feature visibility cutoff time, outcome observation window, and reason for termination of follow-up, and through patient-level timeline sampling verification, confirm that the model has not read information after the target event occurs.
The complete text of the document, document type, recording time, signature or revision status and corresponding medical treatment should be saved. If the business requires structured events, the extraction results must also establish position mapping with the original text paragraphs or page numbers. In this way, clinical experts can go back to the original evidence for verification, algorithm engineers can filter samples by document type, and data engineers can check text truncation and duplication. For key records, the focus of acceptance should be on text integrity, classification accuracy, time logic and patient association; over-processing of non-key fields may increase errors and obscure the original content.
Multimodality does not mean placing files in the same directory
A patient's data may include disease course text, test values, medication orders, radiology reports, DICOM images, pathology reports, and full-field slides. Only when the temporal and semantic relationships between them are clear can a multimodal case be constituted. Each image must at least be able to find the corresponding patient, examination event, modality, body part, acquisition date and report; when the report mentions "comparison with the previous film", the previous image and its report should also be included in the traceability chain. Pathological sections need to be related to the sampling site, specimen, surgical or biopsy event, and pathological conclusion.
DICOM sequence ontology and key header fields carry not only image pixels, but also technical information such as modality, date, voxel spacing, layer thickness, sequence, etc. Desensitization should cover standard tags, private tags, file names, and pixel-embedded information, while trying to preserve the technical context needed to complete the research task. It should be clear whether the sequence is complete, whether it has been resampled or compressed, the export tool and the processed version. If a screenshot is provided, it should be marked which information it cannot replace the original DICOM to prevent downstream teams from mistaking the thumbnail for complete image data.
"Patient packages" are an organization method that facilitates delivery and sampling: grouping structured tables, PDF documents, imaging and pathology files into the same anonymous patient directory, and using a list to index each file and its associated events. But the catalog itself is just a delivery shell. The real data model still needs to have a clear table structure, primary keys, foreign keys, date fields and dictionaries. The checklist should answer: What visits did each patient have, what paperwork and tests were there for each visit, what was missing, and whether the lack was a structural non-existence or a delivery omission. For the "if there is any inspection" modal, the applicable conditions and counting caliber should be provided at the same time to avoid treating the absence of a certain inspection as a missing payment.
The patient journey study published in "Nature Medicine" in 2026 proposed a method to extract clinical entities from structured and unstructured electronic medical records, integrate them with coded data, and organize them into queryable patient journeys. It illustrates research directions that place encounters, medications, tests, vital signs, and symptoms in the same temporal context.[2] For the data supply chain, this does not mean that all projects should first build a knowledge map, but it means that a traceable mapping needs to be retained between original documents and structured data, so that a timeline or graph structure can be constructed later based on specific tasks.
Clinical variables retain their original context and change process
Structured timing tables should not separate the "value" from its clinical condition. The test results must at least retain the project name, original results, units, reference intervals, abnormal identification, specimens and collection time; the drug data must distinguish medical orders, start and stop time, dosage, route, frequency and execution status. The modification or discontinuation of medical orders may themselves reflect changes in the course of the disease, and if only the final prescription is retained, the treatment trajectory will be lost. The scale should save the scale name, version, total score and item score to facilitate recalculation and verification. If age is generalized or binning, the generalization rules and usage restrictions must be retained, and the processed values must not be mistaken for the original measurement.
diagnostics also have time semantics. Preliminary diagnosis, revised diagnosis, discharge diagnosis and pathological diagnosis may complement or change each other. The data structure should retain the time of diagnosis, record type, coding version and source instrument, and cannot leave only one final code. If a unified label is generated for a certain model task, it is necessary to create a separate task label field and retain its judgment basis. This allows for the study of diagnostic evolution and prevents subsequent researchers from conflating clinical records and processing labels.
Multimodal connections should further fall from “same patient” to “same clinical event”. The same patient may undergo multiple examinations of different parts on the same day, or scans may be repeated within a short period of time; association only by patient ID or date will stitch together unrelated images and reports. The association can be established through the treatment ID, examination application number, report ID, Study/Series UID or internal event key. After the sensitive identifier is desensitized, it will be replaced by a stable anonymous key. Before delivery, the time, location, and comparative references in the report can be extracted, and the index list can be automatically checked, and then medical personnel can review mismatches, missing layers, or unreasonable time sequences.
Disease cohorts and outcome labels need to be jointly defined clinically When constructing the
cohort, disease inclusion criteria should be implemented into coding, diagnostic text and clinical evidence. Relying solely on a single diagnosis code may include cases that do not meet the target definition; only looking at free text may also lead to misjudgments due to synonyms, negatives, or historical diagnoses. Coding screening can be used as a preliminary screening, and then require critical baseline evidence support, such as corresponding imaging, pathology, laboratory or specialist records, and have medical review to handle conflicting samples. Encoding versions, mapping rules, and exceptions must be entered into the data dictionary and acceptance records.
Multi-disease and multi-department cohorts also need to examine the population structure. Factors such as age, gender, disease stage, disease severity, treatment type, institution and time period will affect model performance. If a certain type of samples are concentrated in a single institution, a single device, or a certain diagnosis and treatment path, the model may learn institutional habits rather than disease patterns. For rare diseases or complex cases, appropriately adding difficult cases can help training and testing, but it must be consistent with the target use and clearly explain how the samples are screened to avoid mistaking the artificially enriched cohort for the true clinical prevalence distribution.
The outcome field must have judgment rules, observation windows, evidence sources and review processes. A time boundary should be set between model input and outcome labels. For example, to predict future recurrence, recurrence documents after the baseline time point cannot be entered into the input features; imaging or pathology reports may be generated after clinical events, and it is also necessary to avoid traveling before the prediction time point. It is best to split the training, validation, and test data by patient group; when the goal is to verify cross-institutional applicability, the entire institution should be retained for external testing; when the goal is time migration, a later time period should be used for testing. Random splitting by report can leak near-duplicate records for the same patient into multiple collections.
Advances in imaging AI have further increased data requirements
CTPA is an important imaging test for the evaluation of pulmonary embolism. A study published in "npj Digital Medicine" in 2025 established a multi-modal model around CTPA imaging, report generation and outcome prediction. The data used covered multiple medical sources and patient cohorts, reflecting that research is moving from single positive discrimination to joint analysis of images, reports and clinical results.[3] The data required for this type of task is not just "having a CTPA image": it also requires the ability to align the entire set of images, report details, clinical context and follow-up outcomes, so that we can evaluate whether the model can summarize the lesions and support subsequent analysis.
As of October 2026, another multi-institutional real-world study reported on the actual performance of a commercial pulmonary embolism AI in different examination types and institutions, noting that there may be a gap in sensitivity under real-world deployment and registration study benchmarks.[4] This type of evidence reminds data teams that a model’s performance once it goes live will be affected by patient composition, exam protocol, disease stage, and institutional workflow. The training set only covers a certain scanning protocol or a single hospital and cannot be automatically extrapolated to other hospitals; the research report must describe case screening, equipment and protocols, data segmentation and external validation methods.
The imaging data set resources of the same year also began to provide finer-grained information such as slice-level labels, embolization locations and types.[5] Fine-grained labels are helpful for localization and subtyping tasks, but the finer the labels, the higher the requirements for annotation rules, image range, and expert agreement. Algorithmic metrics need to be linked to clinical use: case-level screening, lesion localization, severity stratification, and report generation should not share a single accuracy rate as the final answer. For small lesions or borderline cases, sensitivity, specificity, stratification indicators, confidence intervals, and error types all need to be analyzed separately.
Langhui can connect data engineering and clinical review into a closed loop
For long-sequence multi-modal projects, Langhui’s professional presentation can focus on “requirements definition, patient-level modeling, cross-modal correlation, quality review and reviewable delivery”. When the project is started, the target disease type, time starting point, outcome definition and data range are jointly confirmed; in the sample stage, patient associations, time fields, document integrity and image sequences are checked; in the formal batch, defects are identified through document lists, automatic rules and medical sampling inspections; versions and re-inspection records are retained after rework. Each step should output checkable fields, logs or acceptance criteria. The specific data source and authorization status should be confirmed according to the project file.
This method advances "whether the data can be submitted" to "whether this batch of data can answer the target question."Customers can verify association rules and data gaps at the time of POC, algorithm teams can evaluate whether model inputs can reconstruct the timeline, medical experts can go back to documents and imaging verification evidence, and data governance personnel can audit sources, redaction and change processes. For items that need to be repurchased, keeping the patient package structure, field dictionary and acceptance version consistent will save costs more than reinterpreting the catalog and fields each time.
FAQ
How many years does the long-series medical data need to be at least? It is not enough. You cannot answer with just one number of years. The observation window, baseline, number of follow-up visits and outcome determination should be defined according to the research objectives. The construction specifications in the attachment set more than three years as a collection threshold, but specific model tasks still need to set interpretable time start and end rules.
Why do inspections and images need to retain accurate time? Because the reference in the course record, test result generation, and image acquisition may occur at different times. If only the date or export date is retained, it may not be possible to determine whether an indicator is available at the time of decision-making, nor to check the sequential relationship between reports.
How should representative POC samples be selected? Cross-collection of the same patient should generally be avoided. By further using time segmentation or external testing of the organization according to target tasks, and checking for near-duplicate files and report template leaks, we can obtain a generalized assessment that is closer to actual deployment.
References
[1] Kaplan A, et al. Learning the natural history of human disease with generative transformers. Nature, 2025. https://doi.org/10.1038/s41586-025-09529-3
[2] Computable longitudinal patient journeys from structured and unstructured EHR data. Nature Medicine, 2026. https://www.nature.com/articles/s41591-026-04695-x
[3] Vision-language model for report generation and outcome prediction in CT pulmonary angiogram. npj Digital Medicine, 2025. https://www.nature.com/articles/s41746-025-01807-8
[4] Multi-site real-world performance of commercial AI for pulmonary and incidental pulmonary embolism detection. npj Digital Medicine, 2026. https://www.nature.com/articles/s41746-026-03418-3
[5] Stanford AIMI. CT Pulmonary Angiography dataset. https://aimi.stanford.edu/datasets/CT-Pulmonary-Angiography
