Consistency and diagnostic accuracy of 4 assays in the detection of the total and free prostate-specific antigen

Introduction

Prostate-specific antigen (PSA) is produced by prostate epithelial cells and found mostly in prostatic fluid, with very small amounts released into the bloodstream. PSA can exist in the blood in 3 forms; the first binds to α2 macroglobulin and lacks immune reactivity; the second binds α1-antichymotrypsin (ACT), forming the most abundant conjugated type of PSA (i.e., PSA-ACT) in the blood; and the third is the free PSA (fPSA), which does not bind to protease inhibitors. Total PSA (tPSA) is the sum of PSA-ACT and fPSA (1). Since its introduction into clinical settings in the late 1980s, PSA has been widely used in the screening, monitoring, response assessment, and active monitoring of prostate cancer (PCa) (2).

However, PSA is not a highly specific tumor marker. Indeed, there are a large number of healthy individuals or patients with benign prostate diseases with PSA levels that will fall within the “grey zone” (of 4–10 ng/mL). Research has reported that the PCa detection rate by needle biopsy is only 26% in this zone (3). A variety of attempts have been made to further assess the risk of PCa in patients with PSA values between 4.0–10.0 ng/mL. It has been found that the free-to-total prostate-specific antigen ratio (%fPSA) can improve specificity while maintaining sensitivity (4).

At present, a major problem in tPSA and fPSA management is the inconsistent results yielded by different assays, which may lead to confusion in the interpretation of results. The main reasons for assay variability were the non-equimolar detection of tPSA and the non-uniform assay calibration (5). To minimize the variation among different assays, the World Health Organization (WHO) developed the 96/670 tPSA calibration in 1999, which is based on 90% PSA-ACT and 10% fPSA, and a separate standard for fPSA (the WHO 96/668 calibration) (6). However, the tPSA cut-off value of 4 ng/mL, which is currently widely used worldwide, was derived from a large, multicenter prospective study and was based on the Hybritech calibration rather than the WHO calibration (7). Studies have shown that the tPSA level was 20–25% lower after calibration with the WHO reference than after calibration with the Hybritech reference (8,9). The increasing adoption of the WHO calibration by reagent manufacturers has significantly reduced biases among different assays; however, the consistency of the tPSA/fPSA results among different assays remains controversial (10,11). Further, while the differences of commonly-used assays have been compared in foreign studies, no Chinese study has compared the differences in PSA measurements among Chinese assays or the effects of different assays on PCa screening in Chinese populations. China’s PCa diagnosis and treatment guidelines recommend that a prostate biopsy should be recommended if the tPSA is 4–10 ng/mL and the %fPSA is <16% (12), and the European guidelines recommend a PSA cut-off value of 2 ng/mL (13). However, despite inherent variations among different assays, these guidelines do not indicate to which assay the cut-off value applies.

In this study, we compared 4 commonly-used assays (i.e., the Beckman, Roche, Abbott, and Mindray assays), which use 2 different calibration standards (i.e., the Hybritech calibration and WHO calibration) to determine the consistency of the tPSA and fPSA results and explore their potential effects on PCa screening in Chinese patients with prostate disease. This will be the case to explore whether the cut-off values recommended in the Chinese guidelines are appropriate for these different platforms. We present the following article in accordance with the STARD reporting checklist (available at https://tau.amegroups.com/article/view/10.21037/tau-23-29/rc).

Methods

Subjects

Eligible consecutive patients who visited the Department of Urology, Tongji Hospital, Tongji Medical College of Huazhong University of Science and Technology from April 2021 to June 2022 were enrolled in this study. To be eligible for inclusion in this study, the patients had to meet the following inclusion criteria: (I) present with a prostate disease; (II) voluntarily undergo a prostate biopsy; and (III) have a tPSA value within the grey zone (2–10 ng/mL). Patients were excluded from the study if they met any of the following exclusion criteria: (I) had a previous history of prostate biopsy or surgery; (II) used 5α-reductase inhibitors and androgens; (III) had a urinary tract infection or acute prostatitis; (IV) had a history of blood transfusion or chronic renal failure; and/or (V) had other tumors. A total of 163 patients with prostate diseases, aged 66.7±7.8 years (range, 47–89 years), entered the final analysis. There were 32 PCa patients (19.6%) and 131 patients with benign prostate disease (80.4%) as confirmed by the pathological findings. Prostate biopsy with at least 12 cores and the pathological reports had been finished by the experienced urological surgeons and pathologists in Tongji Hospital, respectively. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study was approved by the Ethics Committee of Tongji Hospital of Tongji Medical College of Huazhong University of Science and Technology (No. TJ-IRB20220627). Because of using the residual serum, the requirement for informed consent was waived.

Blood sample collection and assays

Venous blood (3–5 mL) was collected by a vacuum blood collection tube containing separation gel, and the serum was separated at a relative centrifugal force of 1,912 ×g for 10 min. Next, the tPSA and fPSA were measured on the Abbott ARCHITECT i2000SR immunoassay analyzer within 3 h of blood sample collection. The remaining serum was divided into 4 aliquots within 8 h of blood collection and frozen at −80 ℃. After blood collection, the levels of tPSA and fPSA were also detected by the Beckman Coulter Access DXI800 (using the Hybritech calibration), Roche Cobas e801 (using the WHO 96/670 calibration for tPSA and WHO 96/668 calibration for fPSA), Abbott Architect i2000sr (using the WHO 96/670 calibration for tPSA and WHO 96/670 calibration for fPSA), and Mindray CL6000I chemiluminescence system (a Chinese platform using the WHO 96/670 calibration for tPSA and WHO 96/668 calibration for fPSA).

The quality control measure

Precision was evaluted according to the EP15-A2 of the Clinical and Laboratory Standards Institute (CLSI, USA), 2 concentration levels of commercial quality control solutions (Bio-Rad Tumor Marker Control, lot numbers 54,691 and 54,693) were applied. The experiments were performed in 5 days, with 1 batch per day. At least 3 measurements were performed at each level per batch, and the repeatability and total imprecision were then calculated.

Previously to the study samples, 2 levels of the Bio-Rad Tumor Marker Control were assayed for each of the four assays. Because there are no target values for the Mindray assays of the Bio-Rad control, 2 levels of the Mindray Tumor Marker Multi Control were tested for the Mindray tPSA and fPSA, additionally. All values of the quality control tests were within the range of one standard deviation below or above the mean.

Methodological comparisons

The methodological comparison and bias were performed according to CLSI EP9-A3. The tPSA, fPSA, and %fPSA detected by the Hybritech-calibrated Beckman chemiluminescence system were used as the reference methods because the Beckman assay was the first method developed for tPSA detection and the first Food Drug Administration–approved strategy for PCa screening (7). Passing-Bablok regression and Bland-Altman difference plots were used to analyze the consistency of the results obtained from the other assays with those obtained from the Beckman assay (using the Hybritech calibration). A Pearson’s correlation analysis was used to evaluate the correlation of the results of the other assays with those of the Beckman assay (using the Hybritech calibration). The medical decision levels were set at tPSA levels of 2, 4, and 10 ng/mL and %fPSA of 16%, and the expected biases at these medical decision levels were calculated.

The predictive accuracy of the biomarkers was quantified as the area under the receiver operating characteristic curve (AUC) for each of the four methods, with the pathological findings as the gold standard. The clinical diagnostic performances (sensitivity, specificity, negative predictive value, positive predictive value, and missed diagnosis rate) of these detection platforms were calculated and compared.

Statistical analysis

A Kolmogorov-Smirnov test was performed to assess the data normality, and the normally distributed measurement data are presented as the mean ± standard deviation (SD). The statistical analysis was performed using SPSS 16.0 software and MedCalc software. A two-sided P value of <0.05 was considered statistically significant.

Results

Precision of individual assays

Both the repeatability and the total imprecision of these 4 assays were <5% (Table 1), which was less than the required thresholds in the manufacturer’s instructions. Thus, the precision of these 4 assays was clinically acceptable.

Methodological comparisons of the measurement results among the 4 assays

The tPSA and fPSA levels and the calculated %fPSA of the 163 patients with prostate disease are detailed in Table 2.

The tPSA levels measured by the Roche, Abbott, and Mindray assays correlated well with the results obtained by the Beckman assay (using the Hybritech calibration). All the Pearson’s correlation coefficients were ≥0.97 (Roche: 0.983, Abbott: 0.974, and Abbott: 0.970). As shown by the Passing-Bablok regression analysis, the tPSA test results were similar among these 4 assays, with slopes close to 1 (Roche: 1.03, Abbott: 1.05, and Mindray: 1.07) and intercepts close to 0 (Roche: −0.14, Abbott: −0.19, and Mindray: −0.12) compared to the Beckman assay (Figure 1). The Bland-Altman plots showed that compared to the tPSA level detected by the Beckman assay, the measured tPSA values were 1.1% higher for the Roche assay, 2.1% higher for the Abbott assay, and 6.7% higher for the Mindray assay; all the average relative biases were below 7% (Figure 2). A further analysis showed that the biases of the other 3 assays relative to the Beckman assay were <6% when the medical decision levels were set at 2, 4, and 10 ng/mL. Notably, when the medical decision level was 4 ng/mL, the relative bias was only −0.50% for the Roche assay, 0.25% for the Abbott assay, and 4% for the Mindray assay (Table 3). Thus, the tPSA test results of these 4 assays had good correlation and consistency.

Figure 1 Passing-Bablok regression analysis. tPSA, total prostate-specific antigen; fPSA, free prostate-specific antigen; %fPSA, percentage of free prostate-specific antigen.

Figure 2 Bland-Altman’s relative bias and precision plots for the tPSA, fPSA, and %fPSA by the 4 assays. tPSA, total prostate-specific antigen; fPSA, free prostate-specific antigen; %fPSA, percentage of free prostate-specific antigen.

The fPSA levels measured by the Roche, Abbott, and Mindray assays correlated well with the results obtained by the Beckman assay (using the Hybritech calibration). All the Pearson’s correlation coefficients were >0.975 (Roche: 0.983, Abbott: 0.979, and Abbott: 0.988). However, the results of the Passing-Bablok regression analysis were quite diverse with slopes as follows: Roche: 0.89; Abbott: 1.23; and Mindray: 1.03. The intercepts were close to 0 (Roche: 0.05, Abbott: −0.02, and Mindray: 0.004), which suggests that the fPSA levels detected by the Roche and Mindray assays were similar to that detected by the Beckman assay, but the fPSA level was 20% higher on the Abbott assay than the Beckman assay. The Bland-Altman plots confirmed the above finding; that is, the fPSA levels measured by the Roche, Abbott, and Mindray assays were 5.4% lower, 22.1% higher, and 4.6% higher than that measured by the Beckman assay, respectively (Figures 1,2).

In relation to the %fPSA, the Roche and Abbott assays had magnified differences in the %fPSA compared to the Beckman assay due to the difference in the fPSA, but this difference was narrowed between the Mindray and Beckman assays. The Bland-Altman analysis further revealed that the difference in the %fPSA was 1.4% between the Beckman and Mindray assays, 6.1% between the Beckman and Roche assays, and about 20% between the Beckman and Abbott assays (Figure 2). Further, when the medical decision level was set at 16%, the converted levels for the other 3 assays were as follows: Roche: 14.69%, Abbott: 19.08%, and Mindray: 15.66%, of which the difference was the smallest between the Mindray and Beckman assays (Table 3).

Comparison of the clinical performance of the test results obtained by the 4 assays

With the pathological diagnosis as the gold standard, when the tPSA was 2–10 ng/mL, the ROC curves were drawn to evaluate the precision of the tPSA and %fPSA detected by these 4 assays in predicting PCa. The AUCs (0.593–0.605) of the tPSA were not high, and no significant difference was found among these 4 assays. With 4 ng/mL as the cut-off, the sensitivity, specificity, positive predictive value, and negative predictive value were similar, and the missed diagnosis rate was the same (all 6.25%) among the 4 assays. The AUC of the %fPSA was larger than that of the tPSA. Comparisons of the AUCs of the %fPSA showed that there was no significant difference between the Roche and Beckman assays (P=0.563) or between the Mindray and Beckman assays (P=0.094), but that of the Abbott assay was significantly higher than that of the Beckman assay (P=0.013). However, when the cut-off value of %fPSA was set at 16%, the Abbott assay had the highest missed diagnosis rate, reaching 37.5%, while the Mindray assay had a sensitivity of 81.25%, the highest negative predictive value (93.88%), and the lowest missed diagnosis rate (18.75%) (Table 4).

Discussion

In our current study, the tPSA of the Roche, Abbott, and Mindray assays were all traced to the WHO 96/670 standard, while that of the Beckman assay was traced to the Hybritech standard. All the 4 evaluated tPSA assays had good consistency both numerically and in terms of their clinical diagnostic performance. In the present study, the mean relative biases were <7% for all of the 3 assays that used the WHO calibration when compared to the Beckman assay that used the Hybritech calibration. Our results differ greatly to previous findings that reported when compared to Hybritech calibration, the PSA levels were 20–25% lower after the calibration of PSA assays to the WHO reference (8,9). It has been suggested that the widely used cut-off value of 4 ng/mL should be lowered to 3.1 ng/mL when using WHO-standardized assays (8). However, the present study showed that in Chinese populations, the Roche Cobas, Abbott Architect, and Mindray CL6000I had a converted relative bias of <4% when the cut-off value of tPSA was 4 ng/mL for the Beckman assay (using the Hybritech calibration). In addition, the positive predictive value, sensitivity, specificity, and missed diagnosis rates were quite similar among these 4 assays, and thus there is no need to lower the cut-off value of tPSA. The smaller difference in the assay results between these 2 calibration standards may be due to the fact that manufacturers who adopt the WHO 96/670 standard have introduced some factors in the calibration curve to promote consistency with Hybritech-calibrated PSA results (14). However, there is a limitation of this study, that is the Beckman Coulter WHO-calibrated PSA assays were not evaluated.

We also found that differences in the tPSA measurements existed among different assays, but were quite small. In our current study, the Roche tPSA values were only about 1% higher than the Beckman tPSA values (using the Hybritech calibration), which is consistent with the findings of Stephan et al. (10) and Garrido et al. (11). In our present study, the Abbott tPSA value was 2.1% higher than that of the Beckman tPSA value (with Hybritech calibration), which differs to the results of previous studies that reported that the Abbott tPSA value was about 5–13% lower. A possible explanation for the difference in the results is the model of the instrument used. For example, Stephan et al. (10) used the earlier Abbott AxSYM^®PSA assay, while both our team and Garrido et al. (11) used the Abbott Architect PSA assay. Thus, our findings were more consistent with those of Garrido et al. In addition, we also found that the Mindray tPSA values were approximately 6.7% higher than the Beckman tPSA values (using the Hybritech calibration).

A variety of factors may lead to the large disparities among different assays. First, the calibration standards traced are different. Second, there are matrix differences between serum samples and the WHO buffer-based reference material used to assign values to the assay calibrators, and anti-PSA and tracer antibodies show different reactivities and affinities between in buffer- and in serum-based samples (15). Third, PSA contains 6 major epitope regions, and different reagent manufacturers may use anti-PSA antibodies with different epitope specificity and affinity for several forms of the fPSA and cPSA (16). Fourth, anti-PSA antibodies can also bind to human kallikrein (HK2), as HK2 is about 80% homologous to PSA at the amino acid level (16).

In relation to the fPSA, both the Passing-Bablok regression analysis and Bland-Altman analysis found large inconsistencies in the results of the 4 assays. Among them, the Beckman (which used the Hybritech calibration), Roche, and Mindray fPSA values were relatively consistent, but not the Abbott assay. The Beckman (which used the Hybritech calibration) fPSA value was about 22.1% lower than that of the Abbott fPSA value and about 5.4% higher than that of the Roche fPSA value, which is consistent with the findings of previous studies (10,11). The relative difference was close to Garrido et al.’s study (the Beckman Coulter fPSA values were 3% higher than the Roche fPSA values and 17% lower than the Abbott fPSA values) (11). In particular, the slope (1.23) of the Abbott fPSA versus Beckman Coulter fPSA was almost identical to those reported in Foj et al.’s study (1.245) (17). Notably, the calibration standard used for the Abbott fPSA values was the WHO 96/670, while that used for the Roche and Mindray fPSA values was the WHO 96/668. The inconsistency in the calibration standards may be one of the reasons for the large difference in the fPSA values.

Despite the use of different standards (WHO96/670 and Hybritech), the tPSA values detected by these 4 assays were almost interchangeable when the tPSA values were within the range of 2–10 ng/mL. Conversely, the fPSA values differed remarkably among different assays, which also affected the %fPSA results. Comparisons of the AUCs confirmed that the %fPSA was useful in improving the accuracy of PCa screening, and the Abbott assay had the highest AUC for the %fPSA. However, when the cut-off value was set at 16% (as recommended by the Chinese guidelines), the Abbott assay had the highest missed diagnosis rate, reaching 37.5%, the Mindray %fPSA value was associated with the lowest missed diagnosis rate (18.75%), and the Mindray %fPSA value also had the highest sensitivity and positive predictive value. Thus, the fPSA detection needs to be further standardized, and the cut-off value of the %fPSA recommended by the Chinese guidelines cannot be applied to the Abbott %fPSA.

Conclusions

In summary, when the tPSA is in the range of 2–10 ng/mL, the tPSA values measured by the Abbott, Roche, Mindray, and Beckman assays (using the Hybritech calibration) have good consistency, and the results are almost interchangeable. However, the consistency of the fPSA measurements is unsatisfactory, and the cut-off value of %fPSA (i.e., 16%) is not applicable to every assay. Each assay must be carefully evaluated to establish its own cut-off value.

Acknowledgments

The tPSA and fPSA kits used in this study were kindly provided by the manufacturers; those are, Beckman Coulter, Roche, Abbott and Mindray.

Funding: None.

Footnote

Reporting Checklist: The authors have completed the STARD reporting checklist. Available at https://tau.amegroups.com/article/view/10.21037/tau-23-29/rc

Data Sharing Statement: Available at https://tau.amegroups.com/article/view/10.21037/tau-23-29/dss

Peer Review File: Available at https://tau.amegroups.com/article/view/10.21037/tau-23-29/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://tau.amegroups.com/article/view/10.21037/tau-23-29/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study was approved by the Ethics Committee of Tongji Hospital of Tongji Medical College of Huazhong University of Science and Technology (No. TJ-IRB20220627). Because of using the residual serum, the requirement for informed consent was waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Ward AM, Catto JW, Hamdy FC. Prostate specific antigen: biology, biochemistry and available commercial assays. Ann Clin Biochem 2001;38:633-51. [Crossref] [PubMed]
2. Balk SP, Ko YJ, Bubley GJ. Biology of prostate-specific antigen. J Clin Oncol 2003;21:383-91. [Crossref] [PubMed]
3. Chen R, Xie LP, Zhou LQ, et al. Current status of prostate biopsy in Chinese Prostate Cancer Consortium member hospitals. Chinese Journal of Urology 2015;36:342-5.
4. Catalona WJ, Partin AW, Slawin KM, et al. Use of the percentage of free prostate-specific antigen to enhance differentiation of prostate cancer from benign prostatic disease: a prospective multicenter clinical trial. JAMA 1998;279:1542-7. [Crossref] [PubMed]
5. Blanchet J, Brinkmann T. The clinical impact of WHO standardization of PSA assays. Journal of Medical Biochemistry 2008;27:161-8. [Crossref]
6. Stamey TA. Second Stanford Conference on International Standardization of Prostate-Specific Antigen Immunoassays: September 1 and 2, 1994. Urology 1995;45:173-84. [Crossref] [PubMed]
7. Catalona WJ, Richie JP, Ahmann FR, et al. Comparison of Digital Rectal Examination and Serum Prostate Specific Antigen in the Early Detection of Prostate Cancer: Results of a Multicenter Clinical Trial of 6,630 Men. J Urol 2017;197:S200-7. [Crossref] [PubMed]
8. Stephan C, Bangma C, Vignati G, et al. 20-25% lower concentrations of total and free prostate-specific antigen (PSA) after calibration of PSA assays to the WHO reference materials--analysis of 1098 patients in four centers. Int J Biol Markers 2009;24:65-9. [PubMed]
9. Stephan C, Köpke T, Semjonow A, et al. Discordant total and free prostate-specific antigen (PSA) assays: does calibration with WHO reference materials diminish the problem? Clin Chem Lab Med 2009;47:1325-31. [Crossref] [PubMed]
10. Stephan C, Klaas M, Müller C, et al. Interchangeability of measurements of total and free prostate-specific antigen in serum with 5 frequently used assay combinations: an update. Clin Chem 2006;52:59-64. [Crossref] [PubMed]
11. Garrido MM, Marta JC, Ribeiro RM, et al. Comparison of Three Assays for Total and Free PSA Using Hybritech and WHO Calibrations. In Vivo 2021;35:3431-9. [Crossref] [PubMed]
12. Gu WJ, Zhu Y. Interpretation of the 2022 edition of the CSCO Guidelines for the Di-agnosis and Treatment of Prostate Cancer. Chinese Journal of Surgical Oncology 2022;14:224-32.
13. Parker C, Castro E, Fizazi K, et al. Prostate cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol 2020;31:1119-34. [Crossref] [PubMed]
14. Kort SA, Martens F, Vanpoucke H, et al. Comparison of 6 automated assays for total and free prostate-specific antigen with special reference to their reactivity toward the WHO 96/670 reference preparation. Clin Chem 2006;52:1568-74. [Crossref] [PubMed]
15. Ishibashi M. Standardization of prostate-specific antigen (PSA) assays: can interchangeability of PSA measurements be improved? Clin Chem 2006;52:1-2. [Crossref] [PubMed]
16. Leinonen J, Wu P, Stenman UH. Epitope mapping of antibodies against prostate-specific antigen with use of peptide libraries. Clin Chem 2002;48:2208-16. [Crossref] [PubMed]
17. Foj L, Filella X, Alcover J, et al. Variability of assay methods for total and free PSA after WHO standardization. Tumour Biol 2014;35:1867-73. [Crossref] [PubMed]

(English Language Editor: L. Huleatt)