In this tutorial we will consider colorectal histology tissues classification using ResNet architecture and PyTorch framework.
Introduction
Recently machine learning (ML) applications became widespread in the healthcare industry: omics field (genomics, transcriptomics, proteomics), drug investigation, radiology and digital histology. Deep learning based image analysis studies in histopathology include different tasks (e.g., classification, semantic segmentation, detection, and instance segmentation). The main goal of ML in this field is automatic detection, grading and prognosis of cancer.
However, there are several challenges in digital pathology. Usually histology slides are large sized hematoxylin and eosin (H&E) stained images with color variations and artifacts; different levels of magnification result in different levels of information extraction. One Whole Slide Image (WSI) is a multi-gigabyte image with typical resolution 100,000 × 100,000 pixels.
In a supervised classification scenario, WSIs are divided into patches with some stride, then a CNN architecture extracts feature vectors from patches which can be passed into traditional ML algorithms (SVM, gradient boosting) for further operations.
In this article we apply CNN ResNet architecture to classify tissue types of colon. We won't use transfer learning — weights from ImageNet are not related to histology and won't help convergence.
Dataset
The collection of textures in colorectal cancer histology — a "MNIST for biologists". Available at:
Two folders:
- 5000 image tiles: 150 × 150 px each (74 × 74 µm). Eight tissue categories.
- 10 larger images: 5000 × 5000 px each. Multiple tissue types per image.
All images are RGB, 0.495 µm/pixel, digitized with Aperio ScanScope, magnification 20×. Histological samples are fully anonymized images of formalin-fixed paraffin-embedded human colorectal adenocarcinomas from the University Medical Center Mannheim, Germany.
Colorectal MNIST Classification with ResNet
Setup
Data Exploration
Sample tiles from each class:
PyTorch Dataset and DataLoaders
Example batch image (denormalised):
Train Loop
Model Setup and Training
ResNet-50 with the final linear layer replaced to output 8 classes. StepLR reduces the Adam learning rate by 10× every 7 epochs.
Results
Train/validation loss curves:
Conclusion
We trained ResNet-50 for 15 epochs achieving 92% accuracy on the test set. Tumor and Empty classes are the most recognisable (F1 = 0.98). The most confusable label is Complex, which likely represents combinations of other tissue types.