Face-off: automatic alteration of facial features

Pursuing or maintaining beautifulness nowadays has become a trend in modern society, especially among the celebrity community. In some cases, one may choose to adopt drastic procedures to alter his or her facial or body features to achieve the desired beauty, thus the blossom of industry on cosmetic plastic surgeries. In addition, people whose faces got damaged due to accidental burns or wounds may also find these surgeries necessary. However, as performing the related surgeries are still considered intrusive and costly, it is better to “preview” the result before a surgery is actually carried out. As many believe that facial appearance matters most, we have developed a system that allows a user to input a photo and changes the associated individual facial feature in an automatic and user-friendly manner. Overall speaking, our system makes contributions in the following four aspects. First, our system not only offers the previewing functionality, but also allows users to interactively fine-tune the desired results, thus making it a useful companion tool for facial cosmetic surgeries. Second, instead of exchanging the overall look of a face, as being done by some existing approaches, our system offers much finer granularity by allowing each and every facial feature to be changed individually and independently, thus achieving higher face-off flexibility. Third, while existing tools generally entail manual effort to locate or align facial features, our system, through the help of Active Shape Model or ASM for short, characterized by a scheme of automatic feature extraction, eliminates most of the needs of user assistance. Finally, for convenience, we have constructed a database of facial features to facilitate the facial feature alteration process. To justify our claims, we have rendered results and compared them with those from existing approaches to demonstrate the effectiveness of our system. We have also conducted a user study to further confirm the usefulness of such a system.     

Wireless Face Interface: Using voluntary gaze direction and facial muscle activations for human-computer interaction

The present aim was to investigate the functionality of a new wireless prototype called Face Interface. The prototype combines the use of voluntary gaze direction and facial muscle activations, for pointing and selecting objects on a computer screen, respectively. The subjective and objective functionality of the prototype was evaluated with a series of pointing tasks using either frowning (i.e., frowning technique) or raising the eyebrows (i.e., raising technique) as the selection technique. Pointing task times and accuracies were measured using three target diameters (i.e., 25, 30, 40 mm), seven pointing distances (i.e., 60, 120, 180, 240, 260, 450, and 520 mm), and eight pointing angles (0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°). The results showed that the raising technique was faster selection technique than the frowning technique for the objects that were presented in the pointing distances from 60 mm to 260 mm. For those pointing distances the overall pointing task times were 2.4 s for the frowning technique, and 1.6 s for the raising technique. Fitts’ law computations showed that the correlations for the Fitts’ law model were r = 0.77 for the frowning technique and r = 0.51 for the raising technique. Further, the index of performance (IP) value was 1.9 bits/s for the frowning technique and 5.4 bits/s for raising the eyebrows technique. Based on the results, the prototype functioned well and was adjustable so that two different facial activations can be used in combination with gaze direction for pointing and selecting objects on a computer screen.     

Correction to: Transmission of correct gaze direction in video conferencing using screen-embedded cameras

Multimedia Tools and Applications -     

基于局部特征和整体特征融合的面部表情识别

提出融合局部特征和整体特征的方法实现人脸面部表情特征的提取。在每一个人脸图像上测量10个距离,把这些距离标准化后作为局部表情特征,用Fisher线性判别提取面部表情的整体特征;为了解决小样本问题,采取“PCA+FLD”的策略,先通过PCA把人脸图像向量投影到一个较低维的空间,再通过标准的FLD提取表情特征。融合后的特征输入到基于反向传播的前向型神经网络进行分类。在耶鲁大学yaleface数据库和日本ART建立的日本女性表情数据库(JAFFE)上实验,得到令人满意的结果。     

Inland waterbody mapping: towards improving discrimination and extraction of inland surface water features

Surface waterbodies in arid and semi-arid environments are threatened by both natural and anthropogenic pressures. Mapping the distribution of surface waterbodies is crucial for managing their dwindling quantities and quality. In this study, a fast and reliable method of water extraction has been introduced. A remote-sensing index called the simple water index (SWI) was formulated to differentiate waterbodies from vegetation class automatically, and to differentiate waterbodies from shadows or built-up areas (water-like features). Its performance was compared with the automated water extraction index (AWEI) and the modified normalized difference water index (MNDWI) on Landsat 8 Operational Land Imager (OLI) image of South Africa. The robustness of the algorithm was tested on images in Madagascar and the Democratic Republic of Congo (DRC) with different biomes. The overall accuracies and kappa coefficient (κ) were used to compare the performance of each index. The McNemar test was performed to assess the significance of the output map and the validation data set. The SWI showed the highest overall accuracy of 91.9% (κ = 0.83), whereas the AWEI and MNDWI yielded overall accuracies of 83.8% (κ = 0.65) and 78.4% (κ = 0.53), respectively. The McNemar test showed that there was no significant difference between the SWI map (p = 0.248), whereas both AWEI and MNDWI maps were significantly different from the validation data set at p = 0.041 and p = 0.013, respectively. The SWI approach reduces the thresholding problem by 50% over the conventional MNDWI and AWEI. It is expected that the SWI will also be useful for the accurate quantification of waterbodies for large areas.     

信息系统不确定性的粗糙边界熵度量方法

不确定性度量是粗糙集理论研究的重要内容之一。分析了目前粗糙集不确定性度量主要方法的不足,给出了基于边界域的粗糙集粗糙边界熵的定义。证明了这种粗糙边界熵随着知识粒度的减小而单调减小,而且当负域的知识颗粒被细分时,粗糙边界熵不变。给出了粗糙边界熵的两条性质。     

Distraction detection of lectures in e-learning using machine learning based on human facial features and postural information

Artificial Life and Robotics - While e-learning lectures allow students to learn at their own pace, it is difficult to manage students’ concentration, which prevents them from receiving...     

On a new coordinate system for improved discrimination of vegetation and burned areas using MIR/NIR information

A transformation defined on the near-infrared (NIR) and middle-infrared (MIR) space is presented that allows deriving a new coordinate system appropriate for vegetation and burned area discrimination. The transformation is based on the difference between MIR and NIR in conjunction with the distance from a convergence point in the MIR/NIR space, representative of a totally burnt surface. One of the derived coordinates presents a small scatter for pixels associated to vegetated surfaces (strict scale) whereas the other one covers a wide range of values (large scale) that suggest its use as a proxy of water content of vegetation. The strict scale character of the first coordinate together with the large scale character of the second one make the coordinate system especially adequate to discriminate vegetated surfaces and rank them according to the water content, from green and dry to burned vegetation. The performance of the new coordinate system is then assessed against than traditional ratio or modified ratio indices (namely the Vegetation Index, the Burned Area Index and the Global Environmental Monitoring Index, modified to the MIR/NIR space) and it is shown that the new coordinate system provides better information than traditional indices, opening interesting perspectives for burned area discrimination and other applications like drought monitoring.     

Real-time hands,face and facial features detection and tracking: Application to cognitive rehabilitation tests monitoring

In this paper, a marker-free computer vision system for cognitive rehabilitation tests monitoring is presented. The system monitors and analyzes the correct and incorrect realization of a set of psicomotricity exercises in which a hand has to touch a facial feature. The monitoring requires different human body parts detection and tracking. Detection of face, eyes, nose, and hands is achieved with a set of classifiers built independently based on the AdaBoost algorithm. Comparisons with other detection approaches, regarding performance and applicability to the monitoring system, are presented. Face and hands tracking is accomplished through the CAMShift algorithm with independent and adaptive two-dimensional histograms of the chromaticity components of the TSL color space for the pixels inside these three regions. The TSL color space was selected after a study of five color spaces regarding skin color characterization. The system is easily implemented with a consumer-grade computer and a camera, unconstrained background and illumination and runs at more than 23 frames per second. The system was tested and achieved a successful monitoring percentage of 97.62%. The automation of the human body parts motion monitoring, its analysis in relation to the psicomotricity exercise indicated to the patient and the storage of the result of the realization of a set of exercises free the rehabilitation experts of doing such demanding tasks. The vision-based system is potentially applicable to other human–computer interface tasks with minor changes.     

A two-step approach for automatic microscopic image segmentation using fuzzy clustering and neural discrimination

The early diagnosis of lymphatic system tumors heavily relies on the computerized morphological analysis of blood cells in microscopic specimen images. Automating this analysis necessarily requires an accurate segmentation of the cells themselves. In this paper, we propose a robust method for the automatic segmentation of microscopic images. Cell segmentation is achieved following a coarse-to-fine approach, which primarily consists in the rough identification of the blood cell and, then, in the refinement of the nucleus contours by means of a neural model. The method proposed has been applied to different case studies, revealing its actual feasibility. This article was submitted by the authors in English. Sara Colantonio, M. Sc. honors degree in Computer Science from the University of Pisa in 2004, PhD student in Information Engineering at the Dept. of Information Engineering, Pisa University, is a research fellow at the Institute of Information Science and Technologies of the Italian National Research Council, in Pisa. She has a grant from Finmeccanica for studies in the field of image categorization with applications in medicine and quality control. Her main interests include neural networks, machine learning, industrial diagnostics, and medical imaging. She is a coauthor of more than fifteen scientific papers. At present, she is involved in a number of European research projects regarding image mining, information technology, and medical decision support systems. Ovidio Salvetti, director of research at the Institute of Information Science and Technologies (ISTI) of the Italian National Research Council (CNR), in Pisa, is working in the field of theoretical and applied computer vision. His fields of research are image analysis and understanding, pictorial information systems, spatial modeling, and intelligent processes in computer vision. He is a coauthor of four books and monographs and more than three hundred technical and scientific articles; he also possesses ten patents regarding systems and software tools for image processing. He has been a scientific coordinator of several national and European research and industrial projects, in collaboration with Italian and foreign research groups, in the fields of computer vision and high-performance computing for diagnostic imaging. He is member of the editorial boards of the international journals Pattern Recognition and Image Analysis and G. Ronchi Foundation Acts. He is at present the CNR contact person in ERCIM (the European Research Consortium for Informatics and Mathematics) for the Working Group on Vision and Image Understanding, member of IEEE and of the steering committee of a number of EU projects. He is head of the ISTI Signals and Images Laboratory. Igor B. Gurevich. Born 1938. Dr. Eng. [Diploma Engineer (Automatic Control and Electrical Engineering), 1961, Moscow Power Engineering Institute, Moscow, USSR]; Dr. (Theoretical Computer Science/Mathematical Cybernetics), 1975, Moscow Institute of Physics and Technology, Moscow, USSR. Head of department at the Dorodnicyn Computing Center of the Russian Academy of Sciences, Moscow; assistant professor at the Computer Science Faculty, Moscow State University. He has worked from 1960 to present as an engineer and researcher in industry, medicine, and universities and in the Russian Academy of Sciences. Area of expertise: image analysis, image understanding, mathematical theory of pattern recognition, theoretical computer science, pattern recognition and image analysis techniques for applications in medicine, nondestructive testing, process control, knowledge bases, knowledge-based systems. Two monographs (in coauthorship), 135 papers on pattern recognition, image analysis, theoretical computer science and applications in peer reviewed international and Russian journals, conference and workshop proceedings; one patent of the USSR, four patents of the RF Executive Secretary of the Russian Federation Association for Pattern Recognition and Image Analysis, member of the International Association for Pattern Recognition Governing Board (representative from the Russian Federation), IAPR fellow. He has been the PI of many research and development projects as part of national research (applied and basic research) programs of the Russian Academy of Sciences, of the Ministry of Education and Science of the Russian Federation, of the Russian Foundation for Basic Research, of the Soros Foundation, and of INTAS. Vice Editor-in-Chief of Pattern Recognition and Image Analysis, International Academic Publishing Company “Nauka/Interperiodica” Pleiades Publishing.     

气体流速流向传感器

设计并研制了一种气体流速流向测量传感器 ,流速测量范围为 3～ 30m/s,流向测量范围为± 45°。实验结果表明 ,流速测量的非线性度为 0 .5 %FS ,流向测量分辨率为 0 .45°。     

Real-time classification of evoked emotions using facial feature tracking and physiological responses

We present automated, real-time models built with machine learning algorithms which use videotapes of subjects’ faces in conjunction with physiological measurements to predict rated emotion (trained coders’ second-by-second assessments of sadness or amusement). Input consisted of videotapes of 41 subjects watching emotionally evocative films along with measures of their cardiovascular activity, somatic activity, and electrodermal responding. We built algorithms based on extracted points from the subjects’ faces as well as their physiological responses. Strengths of the current approach are (1) we are assessing real behavior of subjects watching emotional videos instead of actors making facial poses, (2) the training data allow us to predict both emotion type (amusement versus sadness) as well as the intensity level of each emotion, (3) we provide a direct comparison between person-specific, gender-specific, and general models. Results demonstrated good fits for the models overall, with better performance for emotion categories than for emotion intensity, for amusement ratings than sadness ratings, for a full model using both physiological measures and facial tracking than for either cue alone, and for person-specific models than for gender-specific or general models.     

Design and implementation of an augmented reality system using gaze interaction

In this paper, we discuss an interactive optical see-through head-mounted device (HMD) which makes use of a user’s gaze for an augmented reality (AR) interface. In particular, we propose a method to employ a user’s half-blink information for more efficient interaction. Since the interaction is achieved using a user’s gaze and half-blinks, the proposed system can create a more efficient computing environment. In addition, the proposed system can be quite helpful to those who have difficulties in using their hands for conventional interaction methods. The experimental results present the robustness and efficiency of the proposed system.     

STAVIES: a system for information extraction from unknown Web data sources through automatic Web wrapper generation using clustering techniques

A fully automated wrapper for information extraction from Web pages is presented. The motivation behind such systems lies in the emerging need for going beyond the concept of "human browsing". The World Wide Web is today the main "all kind of information" repository and has been so far very successful in disseminating information to humans. By automating the process of information retrieval, further utilization by targeted applications is enabled. The key idea in our novel system is to exploit the format of the Web pages to discover the underlying structure in order to finally infer and extract pieces of information from the Web page. Our system first identifies the section of the Web page that contains the information to be extracted and then extracts it by using clustering techniques and other tools of statistical origin. STAVIES can operate without human intervention and does not require any training. The main innovation and contribution of the proposed system consists of introducing a signal-wise treatment of the tag structural hierarchy and using hierarchical clustering techniques to segment the Web pages. The importance of such a treatment is significant since it permits abstracting away from the raw tag-manipulating approach. Experimental results and comparisons with other state of the art systems are presented and discussed in the paper, indicating the high performance of the proposed algorithm.     

Person recognition using facial video information: A state of the art

In this article we propose a detailed state of the art on person recognition using facial video information. We classify the existing approaches present in the scientific literature between those that neglect the temporal information, and those that exploit it even partially. Concerning the first category, we detail the extensions to video data of: eigenfaces, fisherfaces, active appearance models (AAMs), radial basis function neural networks (RBFNNs), elastic graph matching (EGM), hierarchical discriminative regression trees (HDRTs) and pairwise clustering methods. After that, we focus on the strategies exploiting the temporal information, in particular those analysing: the facial motion with optical flow, the evolution of facial appearance over time with hidden Markov models (HMMs) or with various probabilistic tracking and recognition approaches, and the head motion with Gaussian mixture models.     

Multilingual disaster information system: information delivery using graphic text for mobile phones

A multilingual disaster information system (MLDI) has been developed to overcome the language barrier during times of natural disaster. MLDI is a web-based system that includes templates in nine languages so that translated texts can be made available immediately. Mobile phone e-mail with graphic text is a useful tool for delivering multilingual disaster information. The visibility of graphic text on mobile phones was measured and found to be equivalent to the built-in font. However, visibility deteriorates as the character size becomes smaller, especially, on displays with poor resolution. This article also discusses the necessity of multilingual information and measures for a safe and barrier-free society.     

Automatic localization and annotation of facial features using machine learning techniques

Content-based image retrieval (CBIR) systems traditionally find images within a database that are similar to query image using low level features, such as colour histograms. However, this requires a user to provide an image to the system. It is easier for a user to query the CBIR system using search terms which requires the image content to be described by semantic labels. However, finding a relationship between the image features and semantic labels is a challenging problem to solve. This paper aims to discover semantic labels for facial features for use in a face image retrieval system. Face image retrieval traditionally uses global face-image information to determine similarity between images. However little has been done in the field of face image retrieval to use local face-features and semantic labelling. Our work aims to develop a clustering method for the discovery of semantic labels of face-features. We also present a machine learning based face-feature localization mechanism which we show has promise in providing accurate localization.     

Optimization and improvement of a robotics gaze control system using LSTM networks

Multimedia Tools and Applications - Gaze control represents an important issue in the interaction between a robot and humans. Specifically, deciding who to pay attention to in a multi-party...     

Face localization for facial features extraction using a symmetrical filter and linear Hough transform

In this paper, face localization for facial feature extraction is presented. The method consists of three steps: (1) facial features enhancement using symmetrical filter, and then the morphological process is applied to examine the edge, peaks, and valley fields; (2) line construction using linear Hough transform; (3) localization of the face region based on the constructed lines and the elimination of excess areas outside the face boundary. This work was presented in part at the 12th International Symposium on Artificial Life and Robotics, Oita, Japan, January 25–27, 2007