DeepHCS++: Bright-field to fluorescence microscopy image conversion using multi-task learning with adversarial losses for label-free high-content screening

Abstract

In this paper, we propose a novel microscopy image translation method for transforming a bright-field microscopy image into three different fluorescence images to observe the apoptosis, nuclei, and cytoplasm of cells, which visualize dead cells, nuclei of cells, and cytoplasm of cells, respectively. These biomarkers are commonly used in high-content drug screening to analyze drug response. The main contribution of the proposed work is the automatic generation of three fluorescence images from a conventional bright-field image; this can greatly reduce the time-consuming and laborious tissue preparation process and improve throughput of the screening process. Our proposed method uses only a single bright-field image and the corresponding fluorescence images as a set of image pairs for training an end-to-end deep convolutional neural network. By leveraging deep convolutional neural networks with a set of image pairs of bright-field and corresponding fluorescence images, our proposed method can produce synthetic fluorescence images comparable to real fluorescence microscopy images with high accuracy. Our proposed model uses multi-task learning with adversarial losses to generate more accurate and realistic microscopy images. We assess the efficacy of the proposed method using real bright-field and fluorescence microscopy image datasets from patient-driven samples of a glioblastoma, and validate the method's accuracy with various quality metrics including cell number correlation (CNC), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), cell viability correlation (CVC), error maps, and R2 correlation. © 2021 Elsevier B.V.