Task specification · Project verification required
Browse data specifications / eval-medical-gate
Medical Gate private evaluation (EVAL-MED)
This page scopes procurement and production. Quantities, samples, permissions, quality and timing are confirmed in the data card and contract after verification. Entries are not distinct assets or available stock.
Candidate formats and task scope
These are scoping options, subject to source verification, sample review and project agreement.
Fields to agree for delivery
task_idholdout_versionmodel_versionrubric_versionrun_idfailure_categoryreview_statusaccess_scope
Counting and quality
Agree one counting unit: case, study, document or slide. File and slice counts are not case counts. Report metric scope, denominator, version and review status.
What makes a deliverable verifiable
Agree the content, access level and timing of these materials before delivery.
- Task and counting
- Define modality, inclusion rules, deduplication keys and units. Cases, studies, series, slices and annotations are distinct.
- Source and permissions
- Check provenance, lawful processing basis, training use, retention and redistribution. De-identification does not replace authorization.
- Samples and distributions
- Review samples, fields, strata and missingness. Sample approval is not batch acceptance.
- Expertise and quality
- Specify expert fit, review, sample denominators, disagreement resolution, rework and acceptance records.
- Files and versions
- Agree paths, checksums, versions, format checks and image–text pairing. Settle against accepted, deduplicated units.
- Evaluation separation
- Freeze tasks and rubrics; agree access, contamination checks, retest conditions and role disclosure.
How evaluation informs the next production cycle
Use failure categories and new task specifications as feedback. Do not automatically return private holdout questions or answers to training. Where one provider performs production and evaluation, disclose roles and agree personnel, access and test-set separation, including independence limitations. Evaluation supports development decisions; it does not replace clinical validation or regulatory approval.
Bring a task; start with verifiable outputs
Share the objective, modality, data readiness, acceptance needs and timing. Do not submit patient information or raw medical data through public forms.