Automatic segmentation of clear cell renal cell carcinoma based on deep learning and a preliminary exploration of the tumor microenvironment

Introduction

There were approximately 403,000 new renal cell carcinoma (RCC) cases and 175,000 RCC-related deaths worldwide in 2018 (1). RCC is the second most common urogenital cancer in China (2). Clear cell RCC (ccRCC) is the most common type of RCC, and accounts for approximately 70–80% of all RCC cases (3). Generally, ccRCC patients have a 5-year survival rate of 90%; however, when the tumor spreads locally or metastasizes, the 5-year survival rate may drop as low as 12% (4). The emergence of artificial intelligence (AI) technology has led to great developments in medical care (5,6). The application of AI technology to ccRCC may assist with diagnosis and uncover disease characteristics that may benefit more patients.

At the end of the 19^th century, William Coley pioneered the joint research of immunology and oncology (7). Since then, immunotherapy has become a prominent method of treatment in a variety of tumors (8-11). The tumor microenvironment (TME) (12,13) refers to the environment surrounding the tumor, including blood vessels, lymphocytes, and fibrocytes, which plays a critical role in tumor occurrence, growth, metastasis, and apoptosis. Over the past 20 years, research on the TME and immunotherapy of ccRCC has achieved great breakthroughs, but characterization based on fluorescence-activated cell sorting (14-16) and multiple immunohistochemistry (IHC) (17-19) technologies has not been implemented in hospitals. Hematoxylin and eosin (H&E) (20) and IHC staining, and whole-slide imaging (WSI) are commonly used to diagnose tumors in general hospitals. It is essential to visualize the TME of ccRCC on WSI to determine the potential of spatial TME patterns in guiding immunotherapy.

Advances in AI (21-24) technology have provided new directions for tumor recognition and research on the TME. Wang et al. (25) used deep learning to identify gastric cancer lymph node metastasis, while Shi et al. (26) used convolutional neural network methods to successfully identify different components in liver cancer tissues. The Automatic Cancer Detection and Classification in Whole-slide Lung Histopathology (ACDC@LungHP) challenge focused on the segmentation of cancer tissue in WSI for accurate lung cancer diagnosis (27). Wang et al. developed an annotation-free instance boosting deep-learning ensemble framework based on the IHC of pyruvate kinase M2 (PKM2) and angiopoietin-2 (Ang-2) to predict the treatment response to bevacizumab in ovarian cancer (28). Furthermore, Khalil et al. developed a deep learning-based system for the fast segmentation of metastatic foci in WSI images of breast cancer and created a reliable and objective reference standard based on IHC (29). However, research on kidney tumors is limited.

Early research on RCC mainly relied on machine-learning methods to classify benign and malignant renal tumors based on computed tomography images (30-33) or to determine the types of RCC (34,35). In recent years, studies have increasingly implemented pathomics and AI within RCC. For example, Cheng et al. (36) successfully distinguished between transcription factor E3 (TFE3) RCC and ccRCC based on the extraction of relevant nuclei features using pathological image patches. Ohe et al. (37) established a classification method based on vascular structure, revealing the regulatory effect of blood vessels in the TME of ccRCC. Myers et al. (38) developed an AI-based RCC subtype classification technique that could classify clear cell, papillary, chromophobe, and benign oncocytoma. Baxi et al. (39) analyzed IHC images with AI technology, and demonstrated the utility of AI-powered analysis tools for drug development and clinical trial design.

Although previous research has considered AI and radiomics within RCC (40), the application of AI analysis of the TME based on WSI and its implications on immunotherapy in ccRCC is under examined. Therefore, this study sought to characterize the TME of ccRCC using deep learning and digital image processing. The accurate identification of tumor cells and complex components in the TME may improve the objectivity and accuracy of pathological diagnosis, which would not only effectively reduce the workload of pathologists, but would also lay a good foundation for the individualized management and precise treatment of ccRCC patients. We present this article in accordance with the TRIPOD reporting checklist (available at https://tau.amegroups.com/article/view/10.21037/tau-2025-400/rc).

Methods

Datasets

Initially, 708 H&E pathology slides of 151 ccRCC (no other types) patients treated at Wuxi No. 2 Hospital (Jiangnan University Medical Center) between 2015 and 2021 were included in our study. However, 189 slides were subsequently removed from the dataset due to poor imaging quality related to image content redundancy, fuzzy image, or image overlap. Thus, ultimately, the data of 132 patients were included in the study. To better study the effect of lymphocytes and fibers in the TME, paraffin-embedded tissue samples for each of the 132 patients were cut into serial sections and prepared for H&E, cluster of differentiation (CD)3, CD8, and Masson’s trichrome staining. A total of 915 WSI images, including 519 H&E slides, 132 slides with CD3 staining, 132 slides with CD8 staining, and 132 slides with Masson’ trichrome staining, were included in this study. These slides were scanned at 400× by Jiangfeng KF-Pro-120 (Konfoong Bioinformation Tech Co., Ltd., Ningbo, China). The 132 ccRCC patients comprised 83 males (62.9%) and 49 females (37.1%), with stage distribution of 67 (50.8%) stage I, 1 (0.8%) stage II, 57 (43.2%) stage III, and 7 (5.3%) stage IV cases. Patients with stage I–II disease were classified as early-stage, whereas those with stage III–IV were categorized as advanced-stage ccRCC. In addition, 72 pathological slices from 60 ccRCC patients treated at the Chinese PLA Joint Logistics Support Force No. 904 Hospital (Wuxi No. 904 Hospital) from 2010 to 2021 were used to verify our classification network.

The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the Wuxi No. 2 Hospital (Jiangnan University Medical Center) Ethics Committee (No. 2021-Y-10) and the Chinese PLA Joint Logistics Support Force No. 904 Hospital Ethics Committee (No. 2021-08-003). Written informed consent was obtained from the individual participants or their guardians. After signed the informed consent forms and obtaining ethical approval, the relevant clinical data of all patients, including tumor-node-metastasis (TNM) staging information, were collected. Limited follow-up duration precluded assessment of long-term survival outcomes (e.g., 3-/5-year rates) in our cohort.

Image patch classification

All the data were pre-processed before using a deep-learning method to segment the tumors in the WSI images. The tumor and non-tumor regions in each WSI image were annotated using ASAP software (Breault Research Organization, Tucson, AZ, USA) under the guidance of pathologists. The tumor areas (TAs) were characterized by clear cells with different degrees of differentiation, different locations, and different degrees of density, while the non-TAs mainly included renal tissue, fat tissue, blood vessels, stroma adrenal tissue, hemorrhages, and lymphocytes. Next, image patches sized 512×512 in the labeled area were randomly intercepted. The collected data were divided into training, validation, and test datasets, ensuring that slices from the same patient only appeared in the same dataset. The tumor classification network was trained on 9,892 image patches, validated on 930 image patches, and tested on 1,288 image patches.

The Camelyon16 algorithm has been used successfully in the tumor segmentation of breast cancer. We used Camelyon16, but we replaced the original ResNet18 (39) with SeNet (36) to introduce an attention mechanism to classify the tumor and non-tumor images of ccRCC. The structure of SeNet (including the number of network layers, layer settings, and activation functions) is detailed in Table 1. Finally, the classification network was applied to the WSI images to segment the tumor regions. When training the classification network, the number of image batches sent to the network was 16 for each batch, and the images underwent random vertical flipping and normalization to improve the convergence speed and robustness of the model. After a round of training, the cumulative data were called an epoch, and a total of 70 epochs were run. The Adam optimizer was used to optimize the parameters in the model, and the initial value of the learning rate was set as 10⁻⁴. In the experiment, binary cross entropy was used as the loss function to measure the degree of fit of the model. The original label value of the image patch was denoted as y_label, and the predicted value of the model was denoted y_pred. The loss function was then calculated as follows:

loss=−[ylabellogypred+(1−ylabel)log(ypred)][1]

Tumor segmentation

The size of a normal WSI image is about 10,000×10,000. Due to computer hardware limitations, it was difficult to predict the TA on a WSI image. Therefore, we scaled the WSI image to a small thumbnail for which the longest edge was 512, and obtained the tissue region in the thumbnail image by transforming the image into the hue, saturation, and value (HSV) color space and using Optical Density Threshold Segmentation (OUTS) (41) threshold segmentation. Later, we used the dilate operation of 3×3 structural elements to obtain a more accurate tissue mask. Every pixel in the foreground of the mask was traversed and transformed to the corresponding coordinate of the original WSI image through coordinate transformation. The 512×512 image patch centered on the coordinate was sent to the tumor classification network, and the classification result was used as the gray value of the pixel in the mask. The preliminary segmentation results of the tumor were obtained after traversing all foreground pixels in the mask. The tumor segmentation result was obtained by filtering out noise with an area less than 200; the segmentation workflow can be seen in the supplementary material (Figure S1).

Segmentation of the lymphocyte and fibrous regions

In the WSI images with CD3 or CD8 staining, the lymphocytes were usually stained dark brown, while the background or other components were white or stained light blue. Threshold segmentation can quickly segment positive regions. Due to the large size of the WSI images, we applied a sliding window method with a window size of 2,048×2,048 pixels to extract tissue patches from WSIs. The blue channel in the image patch was extracted, and the initial lymphocyte segmentation result was obtained by inverse threshold segmentation with a threshold of 90. Next, some small segmentation noises were eliminated by morphological operations (including erosion and a dilate operation). We randomly selected 100 512×512 image patches from 20 WSI segmentation images, and each patch contained 1–20 non-adhesive lymphocytes. By dividing the total area by the total number, the average area of approximately one lymphocyte was calculated to be 565 pixels.

In Masson’s trichrome-stained images, the fibrous tissues were dyed blue. Similar to the lymphocyte segmentation, Mason’s trichrome-stained WSI also obtained image patches through a sliding window. The image blocks were first converted into the HSV color space, extracting H [100–126], S [43–255], and V [46–255] values, the noise was then removed by morphology, and connected regions with areas below a threshold (e.g., <500 pixels) were eliminated to retain significant domains. To further investigate the mechanisms in the TME, we scaled the segmentation results of both lymphocytes and fibers to a thumbnail for which the longest edge was 512.

Image registration and related metrics

WSI image registration has very high requirements in terms of the computer hardware and standardization of pathological slides; thus, slides with different staining methods were registering as thumbnails. Registration included the following steps: (I) the preprocessing of different stained WSI thumbnails. The WSI images obtained by different staining methods had different prominent subjects and large differences in the gray image levels, but the shape of the entire tissue was roughly similar. By addressing the difference in the pixel values of the differently stained images, thumbnail-level registration was able to be achieved. Through the separation of the red, green, and blue (RGB) color channels, the gray images of the red channel of the H&E and Masson’s trichrome-stained images had the brightness information, while the green channel of the CD3 and CD8 IHC images had less noise and more information. Subsequently, Contrast Limited Adaptive Histogram Equalization (CLAHE) (42) was applied to gray-scale images of different colors. The algorithm suppressed noise as much as possible while redistributing brightness information to improve the contrast of the image. (II) The Scale Invariant Feature Transform (SIFT) (43) algorithm was used to enable the feature points of the two images to be registered. The transformation matrix H was calculated from the closest feature points, and the transformation image under the source image was obtained by applying H to the target image. (III) Step II was repeated to register the segmentation results of the CD3, CD8, and Masson’s trichrome-stained images with the segmentation results of the H&E images to obtain the final TME.

At 40× magnification, the registration time of the two WSI images ranged from 1 to 4 minutes, which was positively correlated with the tissue area and the number of feature points in the WSI image. CLAHE and the SIFT algorithm were selected after several experiments and comparisons. The CLAHE was completed on the thumbnail of the WSI image (for which the longest edge was only 1,024), which improved the image quality as much as possible, and did not take too much memory or time. In relation to the extraction of feature points in the image registration, the SIFI operator is a classic and effective algorithm. In this experiment, we compared a variety of feature extraction operators (speeded up robust features) and (oriented FAST and rotated BRIEF). Other feature extraction operators are superior in terms of speed but have registration errors. Some secondary matching patches did not fit well. Thus, to improve the registration accuracy, the SIFT operator was adopted, which has high registration accuracy and a reasonable time. The registration algorithm proposed by Wang et al. (44) has a fast speed and was used as a reference direction to improve the registration accuracy of our algorithm in the later stage. However, in our subsequent research, we focused on nuclear research at higher resolution to ensure a high registration accuracy.

Statistical analysis

Continuous variables were presented as medians to reflect the central tendency of the data. The Mann-Whitney U test was employed to assess for significant differences between early-stage (TNM stages I–II) and advanced-stage (TNM stages III–IV) patient groups for variables including age, lymphocyte counts, and fiber area (FA). A two-sided P value of <0.05 was established as the threshold for statistical significance. The diagnostic performance of the deep learning model was quantified using accuracy, sensitivity, specificity, and the area under the receiver operating characteristic (ROC) curve (AUC).

Results

Workflow of the ccRCC analysis system

The workflow of our ccRCC analysis system is shown in Figure 1. Patients found to have a renal mass concerning for malignancy and underwent either radical or partial nephrectomy. After the operation, the resected tissues underwent H&E, IHC for CD3 and CD8, and Masson’s trichrome staining for further diagnosis and treatment. After the WSI images were collected, deep-learning and traditional image processing algorithms were used to characterize the tumor, lymphocyte, and fibers. Finally, the differently stained images were merged via the registration process to analyze the TME of ccRCC to gain novel insights into the diagnosis and prognosis of patients, promote basic research on ccRCC, and develop individualized treatment strategies.

Figure 1 Workflow of ccRCC analysis system. The magnification of the whole slides: 10×; CD3 and CD8 stained using IHC. ccRCC, clear cell renal cell carcinoma; CD, cluster of differentiation; H&E, hematoxylin and eosin; IHC, immunohistochemistry; TME, tumor microenvironment.

Tumor segmentation with deep learning

As Figure 2 shows, the segmentation of each ccRCC tumor included the following steps: the collection of image patch data sets (Figure 2A); the training and testing of the classification network (Figure 2B); and the acquisition of the tumor segmentation results based on the classification network (Figure 2C-2E). In the tumor image classification task, 10,822 image patches from 196 WSI images collected at the Wuxi No. 2 Hospital from 2020 to 2021 were used to train a classification network, and the remaining WSIs were employed for verification. The number of specific picture patches is detailed in the supplementary material (Table S1). Due to the introduction of the attention mechanism, SeNet (45) has higher recognition accuracy than VGG16, ResNet18, InceptionV3, and MobileNet. When tested on the Wuxi No. 2 Hospital test set, our model achieved an accuracy of 96.67%, a sensitivity of 94.29%, a specificity of 99.06% (Figure 2C), and an AUC of 96.68% (Figure 2D). Application of this model to the external validation cohort from Wuxi No. 904 Hospital—exhibiting variations in staining protocols and section preservation—demonstrated a modest yet statistically significant performance degradation. Nevertheless, the model maintained clinically excellent discrimination in this technically distinct setting (accuracy: 90.54%, sensitivity: 84.22%, specificity: 97.94%, AUC: 91.08%), notably surpassing comparative networks (Table 2). Figure S2 confusion matrices further confirm robust classification consistency across both tumor and non-tumor classes.

Figure 2 Visualization of ccRCC tumor segmentation based on deep learning. (A) Collection of image patches. H&E staining for all WSIs. The magnification of the whole slides: 10×, of the small slides: 100×. The tumor (orange) and non-tumor (blue) regions were marked in the WSI images, and the respective areas were randomly selected to generate image patches. (B) Training of an image patch classification network based on SeNet. (C) Confusion matrix of the classification results. (D) ROC curve for SeNet. (E) Visualization of the ccRCC segmentation results. H&E staining; magnification of the whole slides: 10×, of the small slides: 100×. ccRCC, clear cell renal cell carcinoma; H&E, hematoxylin and eosin; ROC, receiver operating characteristic; WSI, whole-slide imaging.

After the classification network was trained based on the image patches, some image processing algorithms were used to obtain the final segmentation results. The specific implementation steps are outlined in the “Method” section. The tumor segmentation results are shown in Figure 2E.

Recognition of lymphocytes and fibers in IHC and Masson’s trichrome-stained images

In the TME, lymphocytes and fibers play an important role in tumor development. Among them, CD3 is a pan-T cell marker expressed on all T lymphocytes, while CD8⁺ T cells differentiate into cytotoxic effectors that eliminate tumor cells and infected host cells (51). The activity of tumor-associated fibroblasts is regulated by tumor cells and further promotes tumor growth, angiogenesis, and metastasis. Unlike other solid tumors, the nuclei of tumor cells in ccRCC are very similar in size and morphology to lymphocytes (50). IHC staining is therefore essential to differentiate tumor-infiltrating lymphocytes from neoplastic cells based on their distinct immunophenotypic profiles.

In the IHC images of CD3⁺ and CD8⁺ in ccRCC, only the cells expressing the corresponding antigenic body were stained dark brown (i.e., were positive). The threshold segmentation algorithm can be used to segment images in which there is a large difference between the foreground and background. We used a 2,048×2,048 window slide to obtain the image patches of raw WSI images. The blue channel was extracted from each RGB image patch through color channel separation, as it provides more distinct nuclear features with reduced background interference in H&E-stained histology images. The positive region in the image patches was obtained by setting a threshold value of 90. Finally, all the patches were spliced back to the same position on the raw WSI images to obtain the segmentation result region for lymphocytes in the IHC slides (Figure 3A). In addition to tumor cells and lymphocytes, fibers are another component of the TME. After Masson’s trichrome staining, the collagen fibers in the tissue appear blue. We used roughly the same procedure as that used for lymphocyte segmentation to obtain the fiber region. Specifically, RGB images were converted to HSV color space, with collagen-defined regions selected using H values between 100 and 126, S between 43 and 255, and V between 46 and 255. Morphological operations subsequently eliminated noise (Figure 3B). Final lymphocyte and fiber distributions are visualized in the composite heatmap (Figure 3C).

Figure 3 CD3⁺ and CD8⁺ T lymphocyte and fiber extraction pipeline. (A) The CD3⁺ and CD8⁺ T lymphocytes stained by IHC are automatically segmented. Magnification: 100×. (B) The extraction method of fibrous tissue stained by Masson staining. Magnification: 100×. (C) Heatmaps for lymphocytes stained by IHC and fibers stained by Masson staining. Magnification for large images: 10×; magnification for small images: 100×. CD, cluster of differentiation; HSV, hue, saturation, and value; IHC, immunohistochemistry.

Registration and related metrics

Based on the registered images, we calculated the ccRCC metrics, such as the TA, FA, the number of CD3⁺ lymphocytes in the tumor (CD3-T), the number of CD8⁺ lymphocytes in the tumor (CD8-T), the number of CD3⁺ lymphocytes in fiber (CD3-F), and the number of CD8⁺ lymphocytes in fiber (CD8-F). The median value statistics were not affected by extreme values and better reflected the intermediate level of the variables. As demonstrated in Table 3, significant differences in tumor characteristics were observed between early-stage (T1–2N0M0) and advanced-stage (T3–4 or N1 or M1) ccRCC patients. Early-stage patients exhibited smaller TA (median TA: 0.6856 vs. 0.7644 cm²) and lower fiber content (median FA: 0.5216 vs. 0.5851 cm²) compared to their advanced-stage counterparts. indicating that tumor development was positively correlated with tumor and fiber size. The abundance of immune cells in the TME is of great clinical significance. We roughly estimated the number of lymphocytes in the tumor and peripheral fibers. The median cell values of CD3⁺ and CD8⁺ lymphocytes in the tumors of the early-stage ccRCC patients were approximately 25,280 and 4,980, respectively, which were significantly lower than the median cell values of 50,620 and 9,060, respectively, in the advanced-stage ccRCC patients. In terms of the fibrous tissue around the tumor, no significant difference in the number of CD3⁺ and CD8⁺ lymphocytes was found.

In addition to calculating the relevant ccRCC evaluation characteristics, we also depicted the distribution of key clinical, demographic, or molecular characteristics across all the patients in this research (Figure 4A,4B). Consistent with known epidemiological patterns, our cohort revealed a male predominance in ccRCC cases (male:female ratio =1.7:1). Figure 4C-4G shows the distribution of fibers and lymphocytes in the patients. Specifically, the figures show that the early-stage patients had less fibers, CD3-T, and CD8-T lymphocytes than the advanced-stage patients, while no significant differences were found in relation to CD3-F and CD8-F. Figure 4H shows the statistical differences in the ccRCC patients in relation to the distributions of gender and age. The abundance of lymphocytes in the tumor was negatively correlated with the patients’ early-disease stage, while no such statistically significant correlation was found in relation to the lymphocytes around the primary tumor.

Figure 4 Related distribution characteristics of ccRCC. (A) Pie chart of the gender ratio in the ccRCC patients. (B) Visualization of the proportions of patients at different TNM stages. (C) Distribution of fibers at different TNM stages. (D) Distribution of CD3-T at different TNM stages. (E) Distribution of CD8-T at different TNM stages. (F) Distribution of CD3-F at different TNM stages. (G) Distribution of CD8-F at different TNM stages. (H) The Mann-Whitney U test was used to compare differences between the early and advanced-stage patients. *, P<0.05; ***, P<0.001; ns, not significant. ccRCC, clear cell renal cell carcinoma; CD, cluster of differentiation; CD3-F, number of CD3⁺ lymphocytes in fiber; CD3-T, number of CD3⁺ lymphocytes in the tumor; CD8-F, number of CD8⁺ lymphocytes in fiber; CD8-T, number of CD8⁺ lymphocytes in the tumor; TNM, tumor-node-metastasis; y, years.

Discussion

Pathological diagnosis is the gold standard for tumor diagnosis; however, the morphological diversity of ccRCC and the interference of other cell types may complicate diagnosis and cause disagreement amongst pathologists (46). In this study, we developed a SeNet model that integrates an attention mechanism and residual network to classify ccRCC. This ccRCC tumor segmentation network efficiently segments the TA, which will help liberate pathologists from the tedious and repetitive work of image reading and increase objectivity. Non-tumor tissues, such as degenerated renal tubular cells and adrenal cortex cells, are not clearly distinguishable from tumor cells, leading to misdiagnoses of ccRCC (47). To address this issue, we added indistinguishable image patches to the training set. Figure S3 demonstrates that our method achieves superior performance in distinguishing tumor cells from ambiguous non-tumor tissues (e.g., degenerated renal tubular cells, adrenal cortex cells) in WSIs, compared to conventional diagnostic approaches that lack explicit training on indistinguishable image patches.

Immunotherapy has increasingly been implemented and demonstrated effectiveness in the treatment of tumors, and its role in ccRCC requires continues to be explored (48,49,52,53). Existing research has classified tumors into cold tumors and hot tumors according to the degree of tumor-infiltrating lymphocytes (54). Research has also shown that an abundance of immune cells is associated with better survival in a large number of cancers (55-57). However, in ccRCC, recent clinical trials suggest that the abundance of immune cells is negatively correlated with the survival rate (58-61). Similarly, our experiments showed that the T cells of advanced-stage patients had higher density than those of early-stage patients. The mechanisms that prevent immune cells in ccRCC patients from identifying and killing tumor cells require subsequent research. Cancer-associated fibroblasts (CAFs) constitute pivotal regulators within the ccRCC TME. Their tumor-promoting functions are mediated through four principal mechanisms: extracellular matrix remodeling via matrix metalloproteinase (MMP)-mediated collagen fiber realignment, creating invasive tracks for tumor cells (62); paracrine signaling through vascular endothelial growth factor (VEGF)/hepatocyte growth factor (HGF) secretion inducing angiogenesis and epithelial-mesenchymal transition (63); metabolic reprogramming of tumor cells via lactate shuttle-mediated Warburg effect potentiation (64); immune evasion facilitation through C-X-C motif chemokine 12 (CXCL12)-mediated suppression of P62-dependent autophagic degradation of programmed death-ligand 1 (PD-L1) in bladder cancer (65). In our study, although not reaching statistical significance, advanced-stage ccRCC patients did demonstrate higher median fibrotic tissue area compared to early-stage patients, indicating a potential role CAFs in ccRCC development.

Recently, Bagaev et al. (66) conducted a transcriptional analysis of more than 10,000 patients with malignancy, and classified the TME into the following four different TME subtypes: immune-depleted, fibrotic, immune-enriched and non-fibrotic, and immune-enriched and fibrotic. They showed that these four TME subtypes are common in more than 20 cancers and are closely related to patients’ responses to immunotherapy. Patients with a high immunological activity of the TME subtypes benefit the most from immunotherapy. Interestingly, as Figure 5 shows, we also found these four TME subtypes in ccRCC, and we further demonstrated the specific metrics of T lymphocytes and fibers. Due to the limited number of ccRCC patients and lack of follow-up information, the exact role of these four TME types is not yet known. A study with a large number of ccRCC patients should be conducted to determine the potential value of these four TME subtypes in predicting patient prognosis and treatment efficiency, especially in relation to immunotherapy.

Figure 5 Visualization of the four ccRCC TME subtypes. “A”, “B”, “C”, and “D” represent the TMEs of four patients. The first column represents the original H&E image, magnification: 10×; the second column shows the segmentation results of CD3⁺ and CD8⁺ lymphocytes; the third column shows the overlay of lymphocytes and tumors; and the last column shows the overall segmentation results. The table shows the distribution of T lymphocytes and fibers from corresponding patients. ccRCC, clear cell renal cell carcinoma; CD, cluster of differentiation; CD3-F, number of CD3⁺ lymphocytes in fiber; CD3-T, number of CD3⁺ lymphocytes in the tumor; CD8-F, number of CD8⁺ lymphocytes in fiber; CD8-T, number of CD8⁺ lymphocytes in the tumor; D, immune-depleted; F, fibrotic; FA, fiber area; H&E, hematoxylin and eosin; IE/F, immune-enriched, fibrotic; IE, immune-enriched, non-fibrotic; TA, tumor area; TME, tumor microenvironment; TNM, tumor-node-metastasis.

In our ccRCC TME analysis, all four co-registered stained whole-slide images exhibited registration errors within clinically acceptable thresholds, enabling reliable downstream analysis. However, the functional significance of collagen fibers in ccRCC progression remains incompletely defined. We identified a potential confounding factor: 87% of histopathological sections contained adjacent non-tumorous renal tissue. Masson’s trichrome staining characteristically highlights intrinsic collagen fibers in these kidney regions as distinct blue structures, which may inadvertently obscure tumor-specific stromal changes during TME evaluation. A deep learning-based digital pathology pipeline will be necessary to discriminate tumor-associated collagen fibers from non-neoplastic renal stromal fibers through multi-scale topological analysis of Masson’s trichrome-stained WSIs.

This study has certain limitations that warrant emphasis: (I) the grouping of stage III and IV patients under the “advanced-stage” category may obscure critical heterogeneity in TME characteristics, particularly given the distinct metastatic potentials between these stages; (II) current quantitative analyses of CD3⁺/CD8⁺ T lymphocytes and collagen fibers did not systematically differentiate density metrics (cells/mm²) from area occupancy ratios (% area), potentially introducing collinearity bias in spatial distribution interpretations. To address these, future work will implement stage-specific analytical frameworks and employ multivariate regression models to isolate independent effects of density vs. area parameters. Additionally, we plan to validate these findings through multi-center cohorts and integrate multi-omics data to refine prognostic stratification models for ccRCC immunotherapy response. The other main limitation of this study is the lack of sufficient WSI images and related therapeutic and/or prognostic information for the patients due to the low incidence of recurrence and death. Future studies will aim to increase the sample size and extend the follow-up duration to better investigate these outcomes.

Conclusions

The methods we proposed provide quantitative and qualitative results, which could aid pathologists in the segmentation of WSI images of ccRCC tissues. The quantitative results and spatial distributions of tumor-infiltrating lymphocytes in WSI images may help in delivering suitable and personalized targeted therapy to ccRCC patients. The precise segmentation of tumor cells and the identification of various components in the TME have yielded promising results (29,67), and integrating the AI-discovered features with clinical characteristics can uncover biologically or clinically meaningful patterns (26). In addition, the production of high-quality continuous pathological sections (i.e., images without folding and of good staining quality) is important to support this work. In subsequent studies, we intend to collect more high-quality WSI images with detailed clinical information, and provide effective suggestions for investigating the growth mechanism of ccRCC and the choice of treatment methods.

Acknowledgments

We would like to thank Dr. Ying Chen and Canrong Ni for processing the whole pathological slides and their helpful insights.

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://tau.amegroups.com/article/view/10.21037/tau-2025-400/rc

Data Sharing Statement: Available at https://tau.amegroups.com/article/view/10.21037/tau-2025-400/dss

Peer Review File: Available at https://tau.amegroups.com/article/view/10.21037/tau-2025-400/prf

Funding: This research was supported by funding from the National Natural Science Foundation of China (No. 8210101340), the Key R&D (Social Development) Projects of Jiangsu Provincial Department of Science and Technology (No. BE2018629), the Wuxi “Taihu Talents Program” Medical and Health High-Level Talents Project (No. THRCJH20200406), and the Wuxi “Key Medical Discipline Construction” Municipal Clinical Medical Center (Municipal Public Health Center) Project (No. ZD2021002).

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://tau.amegroups.com/article/view/10.21037/tau-2025-400/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. This study was approved by the Wuxi No. 2 Hospital (Jiangnan University Medical Center) Ethics Committee (No. 2021-Y-10) and the Chinese PLA Joint Logistics Support Force No. 904 Hospital Ethics Committee (No. 2021-08-003). Written informed consent was obtained from the individual participants or their guardians.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Bray F, Laversanne M, Sung H, et al. Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 2024;74:229-63. [Crossref] [PubMed]
2. Bai T, Wang L, Wang D, et al. Clinicopathologica Epidemiological Characteristics and Change Tendencies of Renal Cell Carcinoma in Shanxi Province of China from 2005 to 2014. PLoS One 2015;10:e0144246. [Crossref] [PubMed]
3. Sato Y, Yoshizato T, Shiraishi Y, et al. Integrated molecular analysis of clear-cell renal cell carcinoma. Nat Genet 2013;45:860-7. [Crossref] [PubMed]
4. Serzan MT, Atkins MB. Current and emerging therapies for first line treatment of metastatic clear cell renal cell carcinoma. J Cancer Metastasis Treat 2021;7:39. [Crossref] [PubMed]
5. Distante A, Marandino L, Bertolo R, et al. Artificial Intelligence in Renal Cell Carcinoma Histopathology: Current Applications and Future Perspectives. Diagnostics (Basel) 2023;13:2294. [Crossref] [PubMed]
6. Raman AG, Fisher D, Yap F, et al. Radiomics and Artificial Intelligence: Renal Cell Carcinoma. Urol Clin North Am 2024;51:35-45. [Crossref] [PubMed]
7. Dobosz P, Dzieciątkowski T. The Intriguing History of Cancer Immunotherapy. Front Immunol 2019;10:2965. [Crossref] [PubMed]
8. Esfahani K, Roudaia L, Buhlaiga N, et al. A review of cancer immunotherapy: from the past, to the present, to the future. Curr Oncol 2020;27:S87-97. [Crossref] [PubMed]
9. Riley RS, June CH, Langer R, et al. Delivery technologies for cancer immunotherapy. Nat Rev Drug Discov 2019;18:175-96. [Crossref] [PubMed]
10. Kwak Y, Seo AN, Lee HE, et al. Tumor immune response and immunotherapy in gastric cancer. J Pathol Transl Med 2020;54:20-33. [Crossref] [PubMed]
11. Xu W, Atkins MB, McDermott DF. Checkpoint inhibitor immunotherapy in kidney cancer. Nat Rev Urol 2020;17:137-50. [Crossref] [PubMed]
12. Giraldo NA, Sanchez-Salas R, Peske JD, et al. The clinical role of the TME in solid cancer. Br J Cancer 2019;120:45-53. [Crossref] [PubMed]
13. Drake CG, Stein MN. The Immunobiology of Kidney Cancer. J Clin Oncol 2018; Epub ahead of print. [Crossref]
14. Jolly LA, Massoll N, Franco AT. Immune Suppression Mediated by Myeloid and Lymphoid Derived Immune Cells in the Tumor Microenvironment Facilitates Progression of Thyroid Cancers Driven by Hras(G12V) and Pten Loss. J Clin Cell Immunol 2016;7:451. [Crossref] [PubMed]
15. Aydin O, Chandran P, Lorsung RR, et al. The Proteomic Effects of Pulsed Focused Ultrasound on Tumor Microenvironments of Murine Melanoma and Breast Cancer Models. Ultrasound Med Biol 2019;45:3232-45. [Crossref] [PubMed]
16. Hou P, Kapoor A, Zhang Q, et al. Tumor Microenvironment Remodeling Enables Bypass of Oncogenic KRAS Dependency in Pancreatic Cancer. Cancer Discov 2020;10:1058-77. [Crossref] [PubMed]
17. Park SM, Chen CJ, Mathy JE, et al. Seven-colour multiplex immunochemistry/immunofluorescence and whole slide imaging of frozen sections. J Immunol Methods 2023;518:113490. [Crossref] [PubMed]
18. Yeong J, Tan T, Chow ZL, et al. Multiplex immunohistochemistry/immunofluorescence (mIHC/IF) for PD-L1 testing in triple-negative breast cancer: a translational assay compared with conventional IHC. J Clin Pathol 2020;73:557-62. [Crossref] [PubMed]
19. Lu S, Stein JE, Rimm DL, et al. Comparison of Biomarker Modalities for Predicting Response to PD-1/PD-L1 Checkpoint Blockade: A Systematic Review and Meta-analysis. JAMA Oncol 2019;5:1195-204. [Crossref] [PubMed]
20. Fischer AH, Jacobson KA, Rose J, et al. Hematoxylin and eosin staining of tissue and cell sections. CSH Protoc 2008;2008:pdb.prot4986.
21. Xing Fuyong, Xie Yuanpu, Su Hai, et al. Deep Learning in Microscopy Image Analysis: A Survey. IEEE Trans Neural Netw Learn Syst 2018;29:4550-68. [Crossref] [PubMed]
22. Kermany DS, Goldbaum M, Cai W, et al. Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning. Cell 2018;172:1122-1131.e9. [Crossref] [PubMed]
23. Wang S, Yang DM, Rong R, et al. Pathology Image Analysis Using Segmentation Deep Learning Algorithms. Am J Pathol 2019;189:1686-98. [Crossref] [PubMed]
24. Guo H, Diao L, Zhou X, et al. Artificial intelligence-based analysis for immunohistochemistry staining of immune checkpoints to predict resected non-small cell lung cancer survival and relapse. Transl Lung Cancer Res 2021;10:2452-74. [Crossref] [PubMed]
25. Wang X, Chen Y, Gao Y, et al. Predicting gastric cancer outcome from resected lymph node histopathology images using deep learning. Nat Commun 2021;12:1637. [Crossref] [PubMed]
26. Shi JY, Wang X, Ding GY, et al. Exploring prognostic indicators in the pathological images of hepatocellular carcinoma based on deep learning. Gut 2021;70:951-61. [Crossref] [PubMed]
27. Li Z, Zhang J, Tan T, et al. Deep Learning Methods for Lung Cancer Segmentation in Whole-Slide Histopathology Images-The ACDC@LungHP Challenge 2019. IEEE J Biomed Health Inform 2021;25:429-40. [Crossref] [PubMed]
28. Wang CW, Lee YC, Lin YJ, et al. Ensemble biomarkers for guiding anti-angiogenesis therapy for ovarian cancer using deep learning. Clin Transl Med 2023;13:e1162. [Crossref] [PubMed]
29. Khalil MA, Lee YC, Lien HC, et al. Fast Segmentation of Metastatic Foci in H&E Whole-Slide Images for Breast Cancer Diagnosis. Diagnostics (Basel) 2022;12:990. [Crossref] [PubMed]
30. Lee H, Hong H, Kim J, et al. Deep feature classification of angiomyolipoma without visible fat and renal cell carcinoma in abdominal contrast-enhanced CT images with texture image patches and hand-crafted feature concatenation. Med Phys 2018;45:1550-61. [Crossref] [PubMed]
31. Feng Z, Rong P, Cao P, et al. Machine learning-based quantitative texture analysis of CT images of small renal masses: Differentiation of angiomyolipoma without visible fat from renal cell carcinoma. Eur Radiol 2018;28:1625-33. [Crossref] [PubMed]
32. Erdim C, Yardimci AH, Bektas CT, et al. Prediction of Benign and Malignant Solid Renal Masses: Machine Learning-Based CT Texture Analysis. Acad Radiol 2020;27:1422-9. [Crossref] [PubMed]
33. Yang R, Wu J, Sun L, et al. Radiomics of small renal masses on multiphasic CT: accuracy of machine learning-based classification models for the differentiation of renal cell carcinoma and angiomyolipoma without visible fat. Eur Radiol 2020;30:1254-63. [Crossref] [PubMed]
34. Li ZC, Zhai G, Zhang J, et al. Differentiation of clear cell and non-clear cell renal cell carcinomas by all-relevant radiomics features from multiphase CT: a VHL mutation perspective. Eur Radiol 2019;29:3996-4007. [Crossref] [PubMed]
35. Kocak B, Yardimci AH, Bektas CT, et al. Textural differences between renal cell carcinoma subtypes: Machine learning-based quantitative computed tomography texture analysis with independent external validation. Eur J Radiol 2018;107:149-57. [Crossref] [PubMed]
36. Cheng J, Han Z, Mehra R, et al. Computational analysis of pathological images enables a better diagnosis of TFE3 Xp11.2 translocation renal cell carcinoma. Nat Commun 2020;11:1778. [Crossref] [PubMed]
37. Ohe C, Yoshida T, Amin MB, et al. Development and validation of a vascularity-based architectural classification for clear cell renal cell carcinoma: correlation with conventional pathological prognostic factors, gene expression patterns, and clinical outcomes. Mod Pathol 2022;35:816-24. [Crossref] [PubMed]
38. Myers MR, Ravipati C, Thangam V. Artificial Intelligence-Based Non-invasive Differentiation of Distinct Histologic Subtypes of Renal Tumors With Multiphasic Multidetector Computed Tomography. Cureus 2024;16:e57959. [Crossref] [PubMed]
39. Baxi V, Edwards R, Montalto M, et al. Digital pathology and artificial intelligence in translational medicine and clinical practice. Mod Pathol 2022;35:23-32. [Crossref] [PubMed]
40. Khene ZE, Tachibana I, Bertail T, et al. Clinical application of radiomics for the prediction of treatment outcome and survival in patients with renal cell carcinoma: a systematic review. World J Urol 2024;42:541. [Crossref] [PubMed]
41. Otsu N. A threshold selection method from gray-level histograms. Automatica 1975;11:23-7.
42. Zuiderveld KJ. Contrast limited adaptive histogram equalization. Graphics Gems 1994;4:474-85.
43. Lowe DG. Distinctive image features from scale-invariant keypoints. Int J Comput Vis 2004;60:91-110.
44. Wang CW, Lee YC, Khalil MA, et al. Fast cross-staining alignment of gigapixel whole slide images with application to prostate cancer and breast cancer analysis. Sci Rep 2022;12:11623. [Crossref] [PubMed]
45. Hu J, Shen L, Albanie S, et al. Squeeze-and-Excitation Networks. IEEE Trans Pattern Anal Mach Intell 2020;42:2011-23. [Crossref] [PubMed]
46. Loghin A, Raicea A, Popelea MC, et al. Histopathological and immunohistochemical characteristics of adult renal tumors: a five-year retrospective study in Mureş County, Romania. Rom J Morphol Embryol 2024;65:457-65. [Crossref] [PubMed]
47. Jia JD, Wang CF, Yang XQ. Uncommon histological morphology and diagnostic ideas of clear cell renal cell carcinoma. Chinese Journal of Clinical and Experimental Pathology 2024;40:354-7, 362.
48. Chevrier S, Levine JH, Zanotelli VRT, et al. An Immune Atlas of Clear Cell Renal Cell Carcinoma. Cell 2017;169:736-749.e18. [Crossref] [PubMed]
49. Du B, Zhou Y, Yi X, et al. Identification of Immune-Related Cells and Genes in Tumor Microenvironment of Clear Cell Renal Cell Carcinoma. Front Oncol 2020;10:1770. [Crossref] [PubMed]
50. Zheng Q, Mei H, Weng X, et al. Artificial intelligence-based multimodal prediction for nuclear grading status and prognosis of clear cell renal cell carcinoma: a multicenter cohort study. Int J Surg 2025;111:3722-30. [Crossref] [PubMed]
51. Marivaux L, Vélez-Juarbe J, Merzeraud G, et al. Early Oligocene chinchilloid caviomorphs from Puerto Rico and the initial rodent colonization of the West Indies. Proc Biol Sci 2020;287:20192806. [Crossref] [PubMed]
52. Saliby RM, Labaki C, Jammihal TR, et al. Impact of renal cell carcinoma molecular subtypes on immunotherapy and targeted therapy outcomes. Cancer Cell 2024;42:732-5. [Crossref] [PubMed]
53. Grigolo S, Filgueira L. Immunotherapy of Clear-Cell Renal-Cell Carcinoma. Cancers (Basel) 2024;16:2092. [Crossref] [PubMed]
54. Bonaventura P, Shekarian T, Alcazer V, et al. Cold Tumors: A Therapeutic Challenge for Immunotherapy. Front Immunol 2019;10:168. [Crossref] [PubMed]
55. Galon J, Fox BA, Bifulco CB, et al. Immunoscore and Immunoprofiling in cancer: an update from the melanoma and immunotherapy bridge 2015. J Transl Med 2016;14:273. [Crossref] [PubMed]
56. Shimizu S, Hiratsuka H, Koike K, et al. Tumor-infiltrating CD8(+) T-cell density is an independent prognostic marker for oral squamous cell carcinoma. Cancer Med 2019;8:80-93. [Crossref] [PubMed]
57. Borcherding N, Kolb R, Gullicksrud J, et al. Keeping Tumors in Check: A Mechanistic Review of Clinical Response and Resistance to Immune Checkpoint Blockade in Cancer. J Mol Biol 2018;430:2014-29. [Crossref] [PubMed]
58. Borcherding N, Vishwakarma A, Voigt AP, et al. Mapping the immune environment in clear cell renal carcinoma by single-cell genomics. Commun Biol 2021;4:122. [Crossref] [PubMed]
59. Qi Y, Xia Y, Lin Z, et al. Tumor-infiltrating CD39(+)CD8(+) T cells determine poor prognosis and immune evasion in clear cell renal cell carcinoma patients. Cancer Immunol Immunother 2020;69:1565-76. [Crossref] [PubMed]
60. Dai S, Zeng H, Liu Z, et al. Intratumoral CXCL13(+)CD8(+)T cell infiltration determines poor clinical outcomes and immunoevasive contexture in patients with clear cell renal cell carcinoma. J Immunother Cancer 2021;9:e001823. [Crossref] [PubMed]
61. Braun DA, Hou Y, Bakouny Z, et al. Interplay of somatic alterations and immune infiltration modulates response to PD-1 blockade in advanced clear cell renal cell carcinoma. Nat Med 2020;26:909-18. [Crossref] [PubMed]
62. Fullár A, Dudás J, Oláh L, et al. Remodeling of extracellular matrix by normal and tumor-associated fibroblasts promotes cervical cancer progression. BMC Cancer 2015;15:256. [Crossref] [PubMed]
63. Guinn S, Kinny-Köster B, Tandurella JA, et al. Transfer Learning Reveals Cancer-Associated Fibroblasts Are Associated with Epithelial-Mesenchymal Transition and Inflammation in Cancer Cells in Pancreatic Ductal Adenocarcinoma. Cancer Res 2024;84:1517-33. [Crossref] [PubMed]
64. Ahuja S, Sureka N, Zaheer S. Unraveling the intricacies of cancer-associated fibroblasts: a comprehensive review on metabolic reprogramming and tumor microenvironment crosstalk. APMIS 2024;132:906-27. [Crossref] [PubMed]
65. Zhang Z, Yu Y, Zhang Z, et al. Cancer-associated fibroblasts-derived CXCL12 enhances immune escape of bladder cancer through inhibiting P62-mediated autophagic degradation of PDL1. J Exp Clin Cancer Res 2023;42:316. [Crossref] [PubMed]
66. Bagaev A, Kotlov N, Nomie K, et al. Conserved pan-cancer microenvironment subtypes predict response to immunotherapy. Cancer Cell 2021;39:845-865.e7. [Crossref] [PubMed]
67. Liang Q, Nan Y, Coppola G, et al. Weakly Supervised Biomedical Image Segmentation by Reiterative Learning. IEEE J Biomed Health Inform 2019;23:1205-14. [Crossref] [PubMed]

(English Language Editor: L. Huleatt)