Entropy‐assisted image segmentation for nano‐ and micro‐sized networks

Image analysis is an important tool for characterizing nano/micro network structures. To understand the connection, organization and proper alignment of network structures, the knowledge of the segments that represent the materials inside the image is very necessary. Image segmentation is generally carried out using statistical methods. In this study, we developed a simple and reliable masking method that improves the performance of the indicator kriging method by using entropy. This method selectively chooses important pixels in an image (optical or electron microscopy image) depending on the degree of information required to assist the thresholding step. Reasonable threshold values can be obtained by selectively choosing important pixels in a complex network image composed of extremely large numbers of thin and narrow objects. Thus, the overall image segmentation can be improved as the number of disconnected objects in the network is minimized. Moreover, we also proposed a new method for analyzing high‐pixel resolution images on a large scale and optimized the time‐consuming steps such as covariance estimation of low‐pixel resolution image, which is rescaled by performing the affine transformation on high‐pixel resolution images. Herein, image segmentation is executed in the original high‐pixel resolution image. This entropy‐based masking method of low‐pixel resolution significantly decreases the analysis time without sacrificing accuracy.     

Minimizing scanning electron microscope artefacts by filter design

A new type of non‐linear filter for digital images has been developed. By using distance transforms we estimate the average point spread function for a set of fibre cross‐sectional images. Then a fast filter technique, based on lookup tables for distance layers, attenuates the uneven background response from the scanning electron microscope. Compared to the convolution‐based techniques that we tried, this approach caused less blurring effects on our fibre images and also made the background pixels more homogeneous. The only assumption we make is that we can roughly segment the background pixels by using a pixel‐wise classifier. Although the assumption that the uneven background response can be described by a circular point spread function is only approximately true in the case discussed here, this method greatly attenuates the effect and provides a fast and general filtering method that can also be of use for other applications.     

Single particle raster image analysis of diffusion for particle mixtures

Recently we complemented the raster image correlation spectroscopy (RICS) method of analysing raster images via estimation of the image correlation function with the method single particle raster image analysis (SPRIA). In SPRIA, individual particles are identified and the diffusion coefficient of each particle is estimated by a maximum likelihood method. In this paper, we extend the SPRIA method to analyse mixtures of particles with a finite set of diffusion coefficients in a homogeneous medium. In examples with simulated and experimental data with two and three different diffusion coefficients, we show that SPRIA gives accurate estimates of the diffusion coefficients and their proportions. A simple technique for finding the number of different diffusion coefficients is also suggested. Further, we study the use of RICS for mixtures with two different diffusion coefficents and investigate, by plotting level curves of the correlation function, how large the quotient between diffusion coefficients needs to be in order to allow discrimination between models with one and two diffusion coefficients. We also describe a minor correction (compared to published papers) of the RICS autocorrelation function.     

Performance of a gradient‐based shift estimator in a spatially sparse data environment: tracking the sub‐pixel motion of fluorescent particles

Through a series of numerical simulations, we investigate the suitability of a relatively new gradient‐based particle‐tracking algorithm for efficiently quantifying sub‐pixel shifts of fluorescently labelled cells or particles from a sequence of video microscopy images. The algorithm excels at estimating sub‐0.5 pixel per frame shifts in both data‐dense (e.g. laser speckle imaging) and data‐sparse (e.g. fluorescence imaging) applications. No upsampling (i.e. interpolation) is required to achieve the sub‐pixel shift resolution, and thus the approach avoids the complexity and potential errors associated with the interpolation process. An efficient matlab sub‐routine is provided for implementing the algorithm.     

Simultaneous four-color imaging of single molecule fluorophores using dichroic mirrors and four charge-coupled devices

We developed a total-internal-reflection (TIR) fluorescence microscopy using three dichroic mirrors and four charge-coupled devices (CCDs) to detect simultaneously four colors of single-molecule (SM) fluorophores. Four spectrally distinct species of fluorophores (Alexa 488, Cy3, Cy5, or Cy5.5) were each immobilized on a different fused silica slide. A species of fluorophores on the slide was irradiated simultaneously, by two excitation beams from an Ar ion laser (488 and 514.5 nm) and a diode laser (642 nm) through TIR on the slide surface. Fluorescence emitted from the fluorophores was spectrally resolved into four components by the dichroic mirrors, and four images were generated from them simultaneously and continuously, with the four CCDs at a rate of 10 Hz. A series of images was thus obtained with each CCD. Fluorescence spots for a species were observed mainly in the series of images recorded by its respective-color CCD. In the first image in the series, we picked out the spots as continuous pixel regions that had the values greater than a threshold. Then we selected only those spots that exhibited single-step photobleaching and regarded them as SM fluorescence spots. Pixel values of SM fluorescence spots widely differed. Some SM fluorophores had pixel values smaller than the threshold, and were left unpicked. Assuming the pixel values of SM fluorescence spots differed with a Gaussian profile, we estimated the ratios of unpicked fluorophores to be less than 20% for all the species. Because of the spectral overlaps between species, we also observed cross-talk spots into CCDs other than the respective-color CCDs. These cross-talk SM fluorescence spots can be mistaken for correct species. We thus introduced the classification method and classified SM fluorescence spots into correct species in accordance with two kinds of four-dimensional signal vectors. The error rates of fluorophore classification were estimated to be less than 3.2% for all the species. Our system is suitable for the biological studies that desire to simultaneously monitor the four colors of SM fluorophores.     

Segmenting time‐lapse phase contrast images of adjacent NIH 3T3 cells

We present a new method for segmenting phase contrast images of NIH 3T3 fibroblast cells that is accurate even when cells are physically in contact with each other. The problem of segmentation, when cells are in contact, poses a challenge to the accurate automation of cell counting, tracking and lineage modelling in cell biology. The segmentation method presented in this paper consists of (1) background reconstruction to obtain noise‐free foreground pixels and (2) incorporation of biological insight about dividing and nondividing cells into the segmentation process to achieve reliable separation of foreground pixels defined as pixels associated with individual cells. The segmentation results for a time‐lapse image stack were compared against 238 manually segmented images (8219 cells) provided by experts, which we consider as reference data. We chose two metrics to measure the accuracy of segmentation: the ‘Adjusted Rand Index’ which compares similarities at a pixel level between masks resulting from manual and automated segmentation, and the ‘Number of Cells per Field’ (NCF) which compares the number of cells identified in the field by manual versus automated analysis. Our results show that the automated segmentation compared to manual segmentation has an average adjusted rand index of 0.96 (1 being a perfect match), with a standard deviation of 0.03, and an average difference of the two numbers of cells per field equal to 5.39% with a standard deviation of 4.6%.     

Multicolour analysis and local image correlation in confocal microscopy

Multiparameter fluorescence microscopy is often used to identify cell types and subcellular organelles according to their differential labelling. For thick objects, the quantitative comparison of different multiply labelled specimens requires the three-dimensional (3-D) sampling capacity of confocal laser scanning microscopy, which can be used to generate pseudocolour images. To analyse such 3-D data sets, we have created pixel fluorogram representations, which are estimates of the joint probability densities linking multiple fluorescence distributions. Such pixel fluorograms also provide a powerful means of analysing image acquisition noise, fluorescence cross-talk, fluorescence photobleaching and cell movements. To identify true fluorescence co-localization, we have developed a novel approach based on local image correlation maps. These maps discriminate the coincident fluorescence distributions from the superimposition of noncorrelated fluorescence profiles on a local basis, by correcting for contrast and local variations in background intensity in each fluorescence channel. We believe that the pixel fluorograms are best suited to the quality control of multifluorescence image acquisition. The local image correlation methods are more appropriate for identifying co-localized structures at the cellular or subcellular level. The thresholding of these correlation maps can further be used to recognize and classify biological structures according to multifluorescence attributes.     

A new algorithm to reduce noise in microscopy images implemented with a simple program in python

All microscopical images contain noise, increasing when (e.g., transmission electron microscope or light microscope) approaching the resolution limit. Many methods are available to reduce noise. One of the most commonly used is image averaging. We propose here to use the mode of pixel values. Simple Python programs process a given number of images, recorded consecutively from the same subject. The programs calculate the mode of the pixel values in a given position (a, b). The result is a new image containing in (a, b) the mode of the values. Therefore, the final pixel value corresponds to that read in at least two of the pixels in position (a, b). The application of the program on a set of images obtained by applying salt and pepper noise and GIMP hurl noise with 10-90% standard deviation showed that the mode performs better than averaging with three-eight images. The data suggest that the mode would be more efficient (in the sense of a lower number of recorded images to process to reduce noise below a given limit) for lower number of total noisy pixels and high standard deviation (as impulse noise and salt and pepper noise), while averaging would be more efficient when the number of varying pixels is high, and the standard deviation is low, as in many cases of Gaussian noise affected images. The two methods may be used serially.     

Application of convolutional neural network to acquisition of clear images for objects with large vertical size in stereo light microscope vision system

For an object with large vertical size that exceeds the certain depth of a stereo light microscope (SLM), its image will be blurred. To obtain clear images, we proposed an image fusion method based on the convolutional neural network (CNN) for the microscopic image sequence. The CNN was designed to discriminate clear and blurred pixels in the source images according to the neighborhood information. To train the CNN, a training set that contained correctly labeled clear and blurred images was created from an open‐access database. The image sequence to be fused was aligned at first. The trained CNN was then used to measure the activity level of each pixel in the aligned source images. The fused image was obtained by taking the pixels with the highest activity levels in the source image sequence. The performance was evaluated using five microscopic image sequences. Compared with other two fusion methods, the proposed method obtained better performance in terms of both visual quality and objective assessment. It is suitable for fusion of the SLM image sequence.     

Fourier transform multipixel spectroscopy for quantitative cytology

A Fourier transform multipixel spectroscopy system was set up and applied to fluorescence microscopy of single living cells. Continuous fluorescence spectra for all pixels of the cell image were recorded simultaneously by the system. Multiple frames of data were first acquired and stored as a set of interferograms for each pixel of the image; they were then Fourier transformed and used as a spatially organized set of fluorescence spectra. Practical spectral resolution of 5 nm was achieved, typically, for 10⁴ pixels in a single cell. The net result was I ( x y ,λ), the fluorescence intensity ( I ) for each pixel of the image ( x y ), as function of wavelength (λ). The present study demonstrates that multipixel spectroscopy can reveal dynamic processes of the food-digestive cycle in the unicellular Paramecium vulgaris fed with algae. Spectral variability of fluorescence intensity at different cytoplasmic sites pinpointed the location of cellular deposits of chlorophyll (630 nm) and of pheophytin (695 nm), a digestive product of the chlorophyll. Localization of compartmental spectral changes was achieved using a 'similarity mapping' algorithm, followed by enhanced image construction. Similarity mapping based on the fluorescence spectrum of native chlorophyll revealed a highlighted image of the cell cytopharynx structure where algae were ingested. Phagolysosomes, migrating vacuoles and the cytoproct, each containing different ratios of pheophytin, were similarly imaged.     

Effects of signal diffusion on x-ray phase contrast images

We discuss the problem of signal diffusion among neighbouring pixels in x-ray phase contrast imaging (XPCi) specifically for coded-aperture (CA) XPCi, but many of the discussed observations are directly transferable to other XPCi modalities. CA XPCi exploits the principle of pixel edge illumination by means of two CA masks. The first mask, placed in contact with the detector, creates insensitive regions between adjacent pixels; the second one, placed immediately before the sample, creates individual beams impinging on the boundaries between sensitive and insensitive regions on the detector, as created by the detector mask. In this way, edge illumination is achieved for all pixels of an area detector illuminated by a divergent and polychromatic beam generated by a conventional source. As the detector mask redefines the resolution properties of the detector, sample dithering can be used to effectively increase the system spatial resolution, without having to apply any post-processing procedure (e.g., deconvolution). This however creates artifacts in the form of secondary fringes (which have nothing to do with phase-related secondary fringes) if there is signal diffusion between adjacent pixels. In non-dithered images, signal diffusion between adjacent pixels causes a reduction in image contrast. This effect is investigated both theoretically and experimentally, and its direct implications on image quality are discussed. The interplay with the sample positioning with respect to the detector pixel matrix, which also has an effect on the obtained image contrast, is also discussed.     

Modulated CMOS camera for fluorescence lifetime microscopy

Widefield frequency‐domain fluorescence lifetime imaging microscopy (FD‐FLIM) is a fast and accurate method to measure the fluorescence lifetime of entire images. However, the complexity and high costs involved in construction of such a system limit the extensive use of this technique. PCO AG recently released the first luminescence lifetime imaging camera based on a high frequency modulated CMOS image sensor, QMFLIM2. Here we tested and provide operational procedures to calibrate the camera and to improve the accuracy using corrections necessary for image analysis. With its flexible input/output options, we are able to use a modulated laser diode or a 20 MHz pulsed white supercontinuum laser as the light source. The output of the camera consists of a stack of modulated images that can be analyzed by the SimFCS software using the phasor approach. The nonuniform system response across the image sensor must be calibrated at the pixel level. This pixel calibration is crucial and needed for every camera settings, e.g. modulation frequency and exposure time. A significant dependency of the modulation signal on the intensity was also observed and hence an additional calibration is needed for each pixel depending on the pixel intensity level. These corrections are important not only for the fundamental frequency, but also for the higher harmonics when using the pulsed supercontinuum laser. With these post data acquisition corrections, the PCO CMOS‐FLIM camera can be used for various biomedical applications requiring a large frame and high speed acquisition. Microsc. Res. Tech. 78:1075–1081, 2015. © 2015 Wiley Periodicals, Inc.     

Correlation-based methods of automatic particle detection in electron microscopy images with smoothing by anisotropic diffusion

Two methods of correlation‐based automatic particle detection in electron microscopy images are compared – computing a cross‐correlation function or a local correlation coefficient vs. azimuthally averaged reference projections (either from a model or from experimental particle images). The ability of smoothing images by anisotropic diffusion to improve the performance of particle detection is also considered. Anisotropic diffusion is an effective method of preprocessing that enhances the edges and overall shape of particles while reducing noise. It is found that anisotropic diffusion improves particle detection by a local correlation coefficient when projections from a high‐resolution reconstruction are used as references. When references from experimental particle images are used, a cross‐correlation function shows a more marked improvement in particle detection in images smoothed by anisotropic diffusion.     

Inherent bias in correlation averaged images

The correlation averaging algorithm frequently used to enhance micrographs of repeating structures contains an inherent bias that favours images with larger pixel values or positive noise levels. This bias not only skews the composite image toward higher pixel values, but also distorts the image by increasing the value of high-valued pixels more than that of low-valued pixels. These errors are especially important in scanning probe microscopy images where the pixel value reflects a distinct height. A similar algorithm that uses a structure function in place of the correlation function eliminates this bias.     

Comparative evaluation of performance measures for shading correction in time‐lapse fluorescence microscopy

Time‐lapse fluorescence microscopy is a valuable technology in cell biology, but it suffers from the inherent problem of intensity inhomogeneity due to uneven illumination or camera nonlinearity, known as shading artefacts. This will lead to inaccurate estimates of single‐cell features such as average and total intensity. Numerous shading correction methods have been proposed to remove this effect. In order to compare the performance of different methods, many quantitative performance measures have been developed. However, there is little discussion about which performance measure should be generally applied for evaluation on real data, where the ground truth is absent. In this paper, the state‐of‐the‐art shading correction methods and performance evaluation methods are reviewed. We implement 10 popular shading correction methods on two artificial datasets and four real ones. In order to make an objective comparison between those methods, we employ a number of quantitative performance measures. Extensive validation demonstrates that the coefficient of joint variation (CJV) is the most applicable measure in time‐lapse fluorescence images. Based on this measure, we have proposed a novel shading correction method that performs better compared to well‐established methods for a range of real data tested.     

一种改进的Harris角点提取算法

针对广泛使用的Harris角点提取算法在对T型和斜T型角点存在定位不准确以及运算速度慢的特点,提出了一种改进算法。改进算法以目标像素点的8-领域范围内与之灰度相似的点的数目为基础,并将该值与目标像素点附近的其他像素点进行比较,分析局部范围内的像素点的灰度值分布,根据比较结果,从中遴选出部分像素点作为下一步Harris角点检测算法的计算对象,并根据角点函数响应值的大小,最终判断该像素点是否为角点。实验结果表明改进算法大大提高了角点检测速度,并对T、斜T型角点提高了其角点检测的定位精度。     

A comparison of different intensity gradient formulae for orientation analysis of electron micrographs

A detailed study of different intensity gradient formulae was conducted using 91 separate images covering a wide range of microfabric. It was found that a formula based on the 3 × 3 local pixel array gives almost identical results over all images with the standard formula based on 20 pixels forming a circular grouping within the 5 × 5 local array. Other formulae based on other groupings of pixels within the same pixel arrays give markedly different results. Computation times are reduced up to 45% when the 8,5 formula is used in place of the 20,14 one, and would thus seem to have advantages when large numbers of images are to be processed. While all formulae give somewhat similar domain-segmented images when viewed visually, differences are noted both in the number of segmented domains as well as in the area of domains in each orientation sector. It is possible that the radius of the modal filter selected for the domain segmentation should be varied according to the formula type used.     

Maximum pixel spectrum: a new tool for detecting and recovering rare, unanticipated features from spectrum image data cubes

A new software tool, the maximum pixel spectrum, detects rare events within a spectrum image data cube, such as that generated with electron‐excited energy‐dispersive X‐ray spectrometry in a scanning electron microscope. The maximum pixel spectrum is a member of a class of ‘derived spectra’ that are constructed from the spectrum image data cube. Similar to a conventional spectrum, a derived spectrum is a linear array of intensity vs. channel index that corresponds to photon energy. A derived spectrum has the principal characteristics of a real spectrum so that X‐ray peaks can be recognized. A common example of a derived spectrum is the summation spectrum, which is a linear array in which the summation of all pixels within each energy plane gives the intensity value for that channel. The summation spectrum is sensitive to the dominant features of the data cube. The maximum pixel spectrum is constructed by selecting the maximum pixel value within each X‐ray energy plane, ignoring the remaining pixels. Peaks corresponding to highly localized trace constituents or foreign contaminants, even those that are confined to one pixel of the image, can be seen at a glance when the maximum pixel spectrum is compared with the summation spectrum.     

Localizing and extracting filament distributions from microscopy images

Detailed quantitative measurements of biological filament networks represent a crucial step in understanding architecture and structure of cells and tissues, which in turn explain important biological events such as wound healing and cancer metastases. Microscopic images of biological specimens marked for different structural proteins constitute an important source for observing and measuring meaningful parameters of biological networks. Unfortunately, current efforts at quantitative estimation of architecture and orientation of biological filament networks from microscopy images are predominantly limited to visual estimation and indirect experimental inference. Here, we describe a new method for localizing and extracting filament distributions from 2D microscopy images of different modalities. The method combines a filter‐based detection of pixels likely to contain a filament with a constrained reverse diffusion‐based approach for localizing the filaments centrelines. We show with qualitative and quantitative experiments, using both simulated and real data, that the new method can provide more accurate centreline estimates of filament in comparison to other approaches currently available. In addition, we show the algorithm is more robust with respect to variations in the initial filter‐based filament detection step often used. We demonstrate the application of the method in extracting quantitative parameters from confocal microscopy images of actin filaments and atomic force microscopy images of DNA fragments.