Helmholtz Gemeinschaft

Search
Browse
Statistics
Feeds

Verity plots: a novel method of visualizing reliability assessments of artificial intelligence methods in quantitative cardiovascular magnetic resonance.

[thumbnail of Original Article]
Preview
PDF (Original Article) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
1MB
[thumbnail of Supporting Information] Other (Supporting Information)
10MB

Item Type:Article
Title:Verity plots: a novel method of visualizing reliability assessments of artificial intelligence methods in quantitative cardiovascular magnetic resonance.
Creators Name:Hadler, T., Ammann, C., Saad, H., Grassow, L., Reisdorf, P., Lange, S., Däuber, S. and Schulz-Menger, J.
Abstract:BACKGROUND: Artificial intelligence (AI) methods have established themselves in cardiovascular magnetic resonance (CMR) as automated quantification tools for ventricular volumes, function, and myocardial tissue characterization. Quality assurance approaches focus on measuring and controlling AI-expert differences but there is a need for tools that better communicate reliability and agreement. This study introduces the Verity plot, a novel statistical visualization that communicates the reliability of quantitative parameters (QP) with clear agreement criteria and descriptive statistics. METHODS: Tolerance ranges for the acceptability of the bias and variance of AI-expert differences were derived from intra- and interreader evaluations. AI-expert agreement was defined by bias confidence and variance tolerance intervals being within bias and variance tolerance ranges. A reliability plot was designed to communicate this statistical test for agreement. Verity plots merge reliability plots with density and a scatter plot to illustrate AI-expert differences. Their utility was compared against Correlation, Box and Bland-Altman plots. RESULTS: Bias and variance tolerance ranges were established for volume, function, and myocardial tissue characterization QPs. Verity plots provided insights into statstistcal properties, outlier detection, and parametric test assumptions, outperforming Correlation, Box and Bland-Altman plots. Additionally, they offered a framework for determining the acceptability of AI-expert bias and variance. CONCLUSION: Verity plots offer markers for bias, variance, trends and outliers, in addition to deciding AI quantification acceptability. The plots were successfully applied to various AI methods in CMR and decisively communicated AI-expert agreement.
Keywords:Artificial Intelligence, Magnetic Resonance Imaging, Observer Variation, Reproducibility of Results
Source:PLoS ONE
ISSN:1932-6203
Publisher:Public Library of Science
Volume:20
Number:5
Page Range:e0323371
Date:16 May 2025
Official Publication:https://doi.org/10.1371/journal.pone.0323371
PubMed:View item in PubMed

Repository Staff Only: item control page

Downloads

Downloads per month over past year

Open Access
MDC Library