Artificial intelligence (AI) methods have attracted widespread interest in the field of medical imaging, and the increasing number of AI publications in radiological journals reflects this growing attention from researchers and journals alike. As the old saying goes, “interest is the best teacher,” yet interest in AI does not automatically translate into proper use and adequate reporting of AI methods. Promising results in published articles do not necessarily ensure high methodological quality. More often than not, incomplete reporting of methodology and the lack of data and code sharing hinder study evaluation and model replication. To address this issue, the Checklist for AI in Medical Imaging (CLAIM) was developed as a guide for the complete reporting of AI studies in medical imaging.¹ CLAIM has been widely adopted, with more than 800 articles citing the guideline as of March 14, 2025, and has also been used as a tool for quality assessment in systematic reviews of AI studies.² However, these systematic reviews, which typically focus on medical imaging studies using AI, highlight the limited quality of current studies. This raises the question of whether CLAIM has been used appropriately and how the reporting and methodological quality of AI studies in medical imaging can be improved.

In this issue of Diagnostic and Interventional Radiology, the study by Koçak et al.³ not only reveals a substantial gap between the current state of reporting and the ideal reporting of AI studies in medical imaging but also identifies factors influencing adherence to CLAIM. The study finds that CLAIM adherence is associated with the journal impact factor quartile, publication year, and specific radiology subfields. Not surprisingly, CLAIM adherence improved after CLAIM’s publication, likely because authors became more familiar with standard practices. Higher adherence to CLAIM was observed in cardiovascular studies, suggesting a more mature use of AI methods in this subfield, from automated reconstruction tools for coronary computed tomography angiography to analysis software for cardiac magnetic resonance. Despite this progress, improving CLAIM adherence remains more important than identifying the sources of high adherence. High-impact journals might promote more transparent reporting practices through more rigorous peer review processes and encourage authors to follow AI guidelines and include them in submission requirements.⁴As the mandatory use of reporting guidelines has been shown to improve study quality,⁵ the current study provides a clear and actionable recommendation to enhance the quality of AI studies: increase journal support for CLAIM use.⁶

In this study, Koçak et al.³ follow a two-level analysis to address concerns regarding common CLAIM critiques. The study summarizes comments from systematic reviews and identifies two main critiques: concerns about the inapplicability of certain items to all study types and the subjective nature of reporting decisions. The concern regarding inapplicability has been addressed with the update of CLAIM 2024, which includes a “not applicable” option for item evaluation,⁷ but the issue of subjectivity remains. These factors may all contribute to the unreliability of CLAIM evaluation, such as unclear item descriptions, subjective comprehension, and the complexity of AI methods.⁸ When researchers use CLAIM for future systematic reviews, a greater focus on reproducibility may be necessary. CLAIM still needs updates, including more explanations and elaborations with examples, so that users can apply the tool with a better understanding.⁹Additionally, developing user-friendly online tools would enhance convenience.¹⁰ The introduction of automatic tools, such as large language models, may also aid in optimizing the reproducibility of CLAIM evaluation. Furthermore, translated versions of the tools endorsed by the original authors may increase visibility and adaptability to local cultures.

Beyond the CLAIM tool itself, the quality of individual studies remains crucial. A previous study by Kocak et al.⁹ investigated the use of CLAIM in individual studies. The study found that only a small percentage of publications used CLAIM along with a supplementary filled-out checklist, and many of the completed checklists contained errors. CLAIM is a useful tool for post-publication evaluation,² but it is not currently required before submission. It remains unclear whether the endorsement of CLAIM and other AI-specific guidelines can improve reporting and methodological quality. Furthermore, it is uncertain whether and how these AI-specific guidelines are used during the editorial process, as only a limited number of journals practice open peer review or publish articles with filled-out checklists. Instead of solely critiquing the adherence of published studies to CLAIM, it would be more valuable to investigate the influence of CLAIM on scientific publication practices. The primary intention of developing a checklist is not to evaluate existing studies retrospectively with strict standards but to guide ongoing research. The checklist can also serve as guidance for peer review before publication and as a tool for study design prior to submission.

In conclusion, the work by Koçak et al.³ draws the community’s attention to the lack of quality in reporting and methodology in AI studies in medical imaging. Although checklists may not resolve this problem overnight, they pave the way for a future of transparent reporting and high-quality methodology. Therefore, the use of reporting checklists is recommended before submission, during evaluation, and after publication.