Segmentation and classification of dot and non-dot-like fluorescence in situ hybridization signals for automated detection of cytogenetic abnormalities.

Signal segmentation and classification of fluorescence in situ hybridization (FISH) images are essential for the detection of cytogenetic abnormalities. Since current methods are limited to dot-like signal analysis, we propose a methodology for segmentation and classification of dot and non-dot-like signals. First, nuclei are segmented from their background and from each other in order to associate signals with specific isolated nuclei. Second, subsignals composing non-dot-like signals are detected and clustered to signals. Features are measured to the signals and a subset of these features is selected representing the signals to a multiclass classifier. Classification using a naive Bayesian classifier (NBC) or a multilayer perceptron is accomplished. When applied to a FISH image database, dot and non-dot-like signals were segmented almost perfectly and then classified with accuracy of approximately 80% by either of the classifiers.