Comparing cardiovascular magnetic resonance strain software packages by their abilities to discriminate outcomes in patients with heart failure with preserved ejection fraction – Journal of Cardiovasc

We illustrated a framework by which CMR FT postprocessing strain software packages can be compared by evaluating its ability to discriminate HFpEF from non-HFpEF controls and further differentiate patients with adverse outcome. For the three software packages examined, all were able to discriminate HFpEF patients from controls. SuiteHeart 4-chamber GLS was associated with adverse outcomes of HFpEF patients in this small cohort of patients. cvi42 and DRA-Trufistrain were limited by the small sample size, but showed trends toward discrimination for adverse outcomes. The intra-observer and inter-observer reproducibilities in all three software packages were excellent. These findings may have significant implications for the clinical usage of different software packages in myocardial strain analyses.

Myocardial strain imaging techniques and analysis provide a valuable diagnostic and prognostic tool for assessing cardiac function [14]. CMR myocardial tagging (MT) serves as the gold standard technique for measuring myocardial strain and validating other strain measurement techniques [3, 15]. CMR MT utilizes spatial modulation of magnetization to create tags that move with the myocardium. This grid of myocardial tags can then be used to track cardiac deformation and measure strain [16]. Additional methods such as displacement encoding with stimulated echoes (DENSE) sequence can also be used to assess strain [17]. However, while CMR MT and DENSE are possibly the more accurate non-invasive techniques for measuring strain, they require acquisition of specialized images and complex post-processing, limiting their wide-spread use in clinical practice [11, 16].

CMR FT, on the other hand, is based on pattern-matching techniques of tracking “features” across multiple images in a cardiac cycle [16]. A pixel is identified in one frame and followed in the next successive frames, leading to tracking of myocardial deformations [18]. Different software packages use different often proprietary algorithms to perform tracking and thus result in different numerical values. These numerical values provided by CMR FT are also different from those derived from the CMR MT or DENSE and may not be as sensitive in disease detection [19]. Although there has been relatively good levels of agreement between FT and MT for globally measured strain, some FT software packages have been shown to consistently and systemically overestimate strain values [20]. Furthermore, intra- and inter-observer agreement of segmental strain by FT is lower than MT [21]. The fact that there are differences between the values produced by different FT software packages and from strains analyzed from tagging and DENSE is well known [22]. However, because FT does not require additional image acquisition and can estimate regional deformation using clinical bSSFP cine images, we will continue to see the growing use of FT in research and clinical settings. Thus, understanding the performance of different software packages and choosing appropriate software packages become important issues.

In addition to cvi42 and SuiteHeart, we included DRA—Trufistrain in our comparison. DRA stands for deformable registration algorithms. DRA also measures strain values from bSSFP cine images, but unlike most of the FT software which utilize optical flow methods and track endocardial features, DRA-Trufistrain method tracks the myocardium and produces layer-specific information [23]. DRA has been found to provide a reliable measurement of segmental and peak systolic strains with better accuracy and reproducibility than FT [23, 24].

In addition to CMR, 2D speckle-tracking echocardiography (STE) has also been widely used to assess myocardial strain due to its ease of use and availability. STE has been shown to have good correlation with the strain calculated by CMR FT and MT, although the agreement is not optimal in myocardial deformation analysis [25]. STE relies heavily on image quality and acoustic window for strain analysis, which can be challenging to consistently acquire [26]. There is currently no standardization in the calculation of myocardial strain in STE, leading to inter-vendor differences [14, 27]. Two vendors (General Electric Healthcare, Chicago, Illinois, USA, and Phillips Healthcare, Best, the Netherlands) and vendor-independent TomTec (TomTec Imaging Systems, Munich, Germany) all produce different strain values. Significant inter-vendor variability for 2D GLS measurements have led the European Association of Cardiovascular Imaging (EAVCI) and the American Society of Echocardiography (ASE) to set up a task force to assess the source of STE measurement variability in partnership with industry vendors [28]. Although inter-vendor agreements for STE have improved over time, the variability remains problematic when GLS is being used clinically across different vendor platforms.

Like echocardiography, strain values differ by vendor packages in CMR-FT. Worse than the problem in STE, where three main vendors differ and TomTec is the only vendor independent software, more than 10 vendor independent CMR FT software packages have emerged, which has amplified the problem of FT analysis in CMR. Each software utilizes different techniques to derive their strain measurements and because some methodologies are proprietary, it is difficult to directly compare these software methodologies. Furthermore, the selective use of different post-processing software may affect the significance of strain measurements. Although CMR is the gold standard for strain analysis using MT or DENSE, one study has found that 2D-STE provided stronger prognostic value to predict overall and CV mortality in HFpEF patients compared to CMR methods [29]. This discrepancy in the prognostic strain value obtained from CMR may be due to the specific post-processing software used. In our study, manually contoured 4-chamber longitudinal strain measurements from both SuiteHeart and cvi42 did not differ significantly from automatically contoured strain measurements. Therefore, we suspect that the performance differences among these two vendors is based on proprietary tracking algorithms, rather than the difference in contour segmentation. This underscores the need for standardization of CMR software for strain measurement [30]. One possible future solution is to make large datasets with outcomes available for software vendors to benchmark their algorithms.

Our study addresses one important concept in evaluating diagnostic and prognostic parameters or methodologies by directly examining the software package’s ability to detect and differentiate disease. Since it would be impossible to standardize individual algorithms, we propose to standardize methodology to evaluate output parameters from the different software packages. The goal is to find parameters from the analysis to differentiate disease from control and to detect more severe disease with worse prognosis from the less severe. In this particular study cohort, we have found that all three software packages investigated were able to fulfill the first requirement of differentiating disease from control, but only one was able to differentiate the more severe from the less severe disease in this small cohort of patients as evidenced by adverse outcome events. Both cvi42 and Trufistrain will likely be able to inform outcome if the sample size were larger, as evidenced by the p-value trends (Table 4). In fact, cvi42 had been shown  to predict outcomes in a larger cohort of patients [31].

Table 4 4-chamber GLS receiver operating curve and Kaplan Meier curve analysis based on area under the curve derived cut-off values and follow-up time for death or heart failure hospitalization in HFpEF patients.

Full size table

Limitations

Our study has a number of limitations. We only studied one disease to illustrate the concept. Our myocardial strain analysis also only focused on two parameters, GLS from 2-chamber and 4-chamber views. 3-chamber views were not used due to inconsistencies in slice selection in acquired images. We choose not to compare GCS on the short-axis view because when we compared GLS and GCS between HFpEF patients and controls, we found that the GLS had greater sensitivity for separation of patients and controls and thus chose to use GLS as an illustrative example. We did not perform global radial strain analysis due to its inferior reproducibility compared to GLS and GCS. Our goal is to provide a demonstration of the method of comparison of strain software using HFpEF as an illustrative example, rather than making a firm statement about strain and outcomes in HFpEF patients. Another limitation to our study is that we only included a single version of each of three software packages. We considered that these software packages to be representative to illustrate the concept as both SuiteHeart and cvi42 have artificial intelligence assisted contour tracking algorithms and are currently widely used. We also included DRA-Trufistrain as it represents a different tracking algorithm.