Identify theoretical weaknesses in your protocol before a technical review.

Five questions target common failure points: data splitting, external validation, subgroup analysis and metric selection.

Self-assessment

A useful result without pretending to automatically audit your AI.

This diagnostic only assesses the theoretical consistency of your answers. It does not analyse your code, data or the model’s actual performance.

Inspired by PRISM work

This diagnostic covers selected methodological points. It does not represent the full PRISM framework.

QSTOM-IT · Pre-flight diagnostic Local processing

Methodological robustness diagnostic

Answer according to the documentation currently available. “I don’t know” is useful information.

DataWhat is the primary type of data processed by the AI system?This answer adapts the consistency checks applied to the following questions.

AMedical imaging or histology slidesBTime series or physiological signalsECG, EEG, vital signs, longitudinal dataCTabular data or electronic health recordsDMedical text, literature or generative AILLMs, reports, document retrieval

Dataset splitHow were the training and evaluation data separated?Choose the primary rule used to construct the final test set.

ARandom split by sample, image or measurementBStrict patient-level splitCTemporal splitOne period for training, a later period for testingDSplit by hospital or clinical centreEUnknown or undocumented method

External validationWhere do the data used to evaluate performance come from?Centre diversity helps assess sensitivity to local practices, equipment and populations.

AA single hospital or care centreBSeveral centres mixed across training and test setsCOne or more strictly external test centresDA generic public dataset onlyEUnknown or undocumented origin

SubgroupsHow was performance examined across patient subgroups?A global average can hide material differences between patients, equipment or clinical settings.

AA single global metric across the entire datasetBAn analysis by age and sex onlyCDetailed clinical and technical stratificationSeverity, subtypes, equipment, centres or acquisition conditionsDNo subgroup analysisEI do not know or it is undocumented

Primary metricWhich measure is primarily used to support the performance claim?Select the metric highlighted in presentations or decision documents.

AOverall accuracyBGlobal ROC-AUC, PR-AUC or F1-scoreCSensitivity and specificity at a defined operational thresholdDText metric or human evaluationFor generative or text-based systemsEUnknown or undocumented metric

Select an answer to continue.

Your answers remain in this browser and are not sent to QSTOM-IT.

Methodological vigilance sheet

Result based on 5 declarations, without access to code or data.

0Important risks

0Points to verify

0Favourable points

What this result does - and does not - demonstrate

The diagnostic flags theoretical risks based on your declarations. Only a review of cohorts, pipeline and results can establish whether these weaknesses actually exist.

QSTOM-IT

Quantitative & Scientific Testing of Operational Models - Intelligent Technologies

Design, assessment and robustness of artificial intelligence systems for health.

Lille - Hauts-de-France - France

Services

Health AI project design Health AI audit Stress testing Training

Method and resources

PRISM Robustness diagnostic Publications EU AI Act

Company

About Contactcontact [at] qstom-it.com