Recognizing multiple emotion from ambiguous facial expressions on mobile platforms

Extracting and understanding of emotion is of high importance for the interaction between human and machine communication systems. The most expressive way to display the human’s emotion is through facial expression analysis. This paper proposes a multiple emotion recognition system that can recognize combinations of up to a maximum of three different emotions using an active appearance model (AAM), the proposed classification standard, and a k-nearest neighbor (k-NN) classifier in mobile environments. AAM can take the expression of variations that are calculated by the proposed classification standard according to changes in human expressions in real time. The proposed k-NN can classify basic emotions (normal, happy, sad, angry, surprise) as well as more ambiguous emotions by combining the basic emotions in real time, and each recognized emotion that can be subdivided has strength. Whereas most previous methods of emotion recognition recognize various kind of a single emotion, this paper recognizes various emotions with a combination of the five basic emotions. To be easily understood, the recognized result is presented in three ways on a mobile camera screen. The result of the experiment was an average 85 % recognition rate and a 40 % performance showed optimized emotions. The implemented system can be represented by one of the example for augmented reality on displaying combination of real face video and virtual animation with user’s avatar.     

Affective content analysis of music emotion through EEG

Emotion recognition of music objects is a promising and important research issues in the field of music information retrieval. Usually, music emotion recognition could be considered as a training/classification problem. However, even given a benchmark (a training data with ground truth) and using effective classification algorithms, music emotion recognition remains a challenging problem. Most previous relevant work focuses only on acoustic music content without considering individual difference (i.e., personalization issues). In addition, assessment of emotions is usually self-reported (e.g., emotion tags) which might introduce inaccuracy and inconsistency. Electroencephalography (EEG) is a non-invasive brain-machine interface which allows external machines to sense neurophysiological signals from the brain without surgery. Such unintrusive EEG signals, captured from the central nervous system, have been utilized for exploring emotions. This paper proposes an evidence-based and personalized model for music emotion recognition. In the training phase for model construction and personalized adaption, based on the IADS (the International Affective Digitized Sound system, a set of acoustic emotional stimuli for experimental investigations of emotion and attention), we construct two predictive and generic models \(AN\!N_1\) (“EEG recordings of standardized group vs. emotions”) and \(AN\!N_2\) (“music audio content vs. emotion”). Both models are trained by an artificial neural network. We then collect a subject’s EEG recordings when listening the selected IADS samples, and apply the \(AN\!N_1\) to determine the subject’s emotion vector. With the generic model and the corresponding individual differences, we construct the personalized model H by the projective transformation. In the testing phase, given a music object, the processing steps are: (1) to extract features from the music audio content, (2) to apply \(AN\!N_2\) to calculate the vector in the arousal-valence emotion space, and (3) to apply the transformation matrix H to determine the personalized emotion vector. Moreover, with respect to a moderate music object, we apply a sliding window on the music object to obtain a sequence of personalized emotion vectors, in which those predicted vectors will be fitted and organized as an emotion trail for revealing dynamics in the affective content of music object. Experimental results suggest the proposed approach is effective.     

Simulation and fabrication of a branch-channel rhombic micromixer for low pressure drop and short mixing length

The planar micromixers merit easy integration with the microsystem but encounter the problems of high pressure drop and long mixing length for high mixing efficiency over 90 %. In this article, an advanced branched rhombic micromixer (BRM) have been designed and investigated by 3D numerical simulations and experiments. Polydimethylsiloxane (PDMS) molding process was used for chip fabrication of the experiment. The CFD-ACE⁺ software was applied for modeling simulations. Simulation results showed that this optimum geometry design of four-mixing-unit BRM with the branch position at 700 μm high and 100 μm wide can effectively improve the mixing efficiency over 95 % at Reynolds number (Re) 120 with a rather low pressure drop about 9,000 Pa and a short mixing length of 5.5 mm as well. The high mixing efficiency at low pressure drop is attributed to the greatly increased contact interface area and chaotic-convection vortices as well as the low flow resistance with branched channels. The mixing verification is performed by the flow visualization system via the popular rhodamine B dye (10 mM) added DI method for PDMS BRM chips.     

Group improved enhanced dynamic frame slotted ALOHA anti-collision algorithm

With the development of information technology and declining in the cost of tags, radio frequency identification (RFID) system has become more and more popular, which has been widely used in a lot of areas, such as logistics tracking, animals identification, medicine, electronic toll collection, inventory, asset management, manufacturing, etc. However, when we use RFID technology to identify the objects, tag collision is one of the important factors to influence the identification efficiency. Currently, Aloha-based algorithm is one of the popular anti-collision algorithms which performs well when the number of tags is small. But it is not very efficient for cases with large number of tags and some areas which tags’ number can be estimated, such as warehouse, supermarket, the production lines of smart factory and so on. So in this paper, we proposed a new anti-collision algorithm called group improved enhanced dynamic frame slotted ALOHA (GroupIEDFSA) by estimating the number of unread tags first, comparing the maximum frame size and dividing tags into groups when the number of tags which are activated is large. What is more, compared with enhanced dynamic frame slotted ALOHA (EDFSA) algorithm in the process of identification, GroupIEDFSA algorithm will combine new group based on the unread tags’ number. Simulation results show that the efficiency of GroupIEDFSA algorithm system improves by 20 % in time and over 50 % in rounds than EDFSA algorithm in the standard mode, and increases by 1 % in time when we used fast mode.     

Parallelizing Complex Streaming Applications on Distributed Scratchpad Memory Multicore Architecture

Multicore processors can provide sufficient computing power and flexibility for complex streaming applications, such as high-definition video processing. For less hardware complexity and power consumption, the distributed scratchpad memory architecture is considered, instead of the cache memory architecture. However, the distributed design poses new challenges to programming. It is difficult to exploit all available capabilities and achieve maximal throughput, due to the combined complexity of inter-processor communication, synchronization, and workload balancing. In this study, we developed an efficient design flow for parallelizing multimedia applications on a distributed scratchpad memory multicore architecture. An application is first partitioned into streaming components and then mapped onto multicore processors. Various hardware-dependent factors and application-specific characteristics are involved in generating efficient task partitions and allocating resources appropriately. To test and verify the proposed design flow, three popular multimedia applications were implemented: a full-HD motion JPEG decoder, an object detector, and a full-HD H.264/AVC decoder. For demonstration purposes, SONY PlayStation \(^{\circledR }\) 3 was selected as the target platform. Simulation results show that, on PS3, the full-HD motion JPEG decoder with the proposed design flow can decode about 108.9 frames per second (fps) in the 1080p format. The object detection application can perform real-time object detection at 2.84 fps at \(1280 \times 960\) resolution, 11.75 fps at \(640 \times 480\) resolution, and 62.52 fps at \(320 \times 240\) resolution. The full-HD H.264/AVC decoder applications can achieve nearly 50 fps.     

Demand look-ahead memory access scheduling for 3D graphics processing units

With the rapid growing complexity of 3D applications, the memory subsystem has become the most bandwidth-exhausting bottleneck in a Graphics Processing Unit (GPU). To produce realistic images, tens to hundreds of thousands of primitives are used. Furthermore, each primitive generates thousands of pixels, and these pixels are computed by shaders with special effects, even to blend multiple texture pixels from external memory to obtain a final color. To hide the long latency texture operations, the shaders are usually highly multithreaded to increase its throughput. However, conventional memory scheduling mechanisms are unaware of the producer-consumer relationship between primitives and pixels. The conventional scheduling mechanisms neither assume that all initiators are independent nor that they use a fixed priority scheme. This paper proposes Demand Look-Ahead (DLA) memory access scheduling based on the statuses of each unit in the GPU, and dynamically generates priority for the memory request scheduler. By considering the producer-consumer relationship, the proposed mechanism reschedules most urgent requests to be serviced first. Experimental results show that the proposed DLA improves 1.47 % and 1.44 % in FPS and IPC, respectively, than First-Ready First-Come-First-Serve (FR-FCFS). By integrating DLA with Bank-level Parallelism Awareness (BPA), DLA-BPA improves FPS and IPC by 7.28 % and 6.55 %, respectively. Furthermore, shader thread performance is improved by 22.06 % and increases the attainable bandwidth by 5.91 % with DLA-BPA.     

A method of DDoS attack detection using HTTP packet pattern and rule engine in cloud computing environment

Cloud computing is a more advanced technology for distributed processing, e.g., a thin client and grid computing, which is implemented by means of virtualization technology for servers and storages, and advanced network functionalities. However, this technology has certain disadvantages such as monotonous routing for attacks, easy attack method, and tools. This means that all network resources and operations are blocked all at once in the worst case. Various studies such as pattern analyses and network-based access control for infringement response based on Infrastructure as a Service, Platform as a Service and Software as a Service in cloud computing services have therefore been recently conducted. This study proposes a method of integration between HTTP GET flooding among Distributed Denial-of-Service attacks and MapReduce processing for fast attack detection in a cloud computing environment. In addition, experiments on the processing time were conducted to compare the performance with a pattern detection of the attack features using Snort detection based on HTTP packet patterns and log data from a Web server. The experimental results show that the proposed method is better than Snort detection because the processing time of the former is shorter with increasing congestion.     

An energy-efficient storage for video surveillance

With the rapid growth of the video surveillance applications, the storage energy consumption of video surveillance is more noticeable, but existed energy-saving methods for massive storage system most concentrate on the data centers mainly with random accesses. The storage of video surveillance has inherent access pattern, and requires special energy-saving approach to save more energy. An energy-efficient data layout for video surveillance, Semi-RAID is proposed. It adopts partial-parallelism strategy, which partitions disk data into different groups, and implements parallel accesses in each group. Grouping benefits to realize only partial disks working and the rest ones idle, and inner-group parallelism provides the performance guarantee. In addition, greedy strategy for address allocation is adopted to effectively prolong the idle period of the disks; particular Cache strategies are used to filter the small amount of random accesses. The energy-saving efficiency of Semi-RAID is verified by a simulated video surveillance consisting of 32 cameras with D1 resolution. The experiment shows: Semi-RAID can save 45 % energy than Hibernator; 80 % energy than PARAID; 33 % energy than MAID; 79 % energy than eRAID-5, while providing single disk fault tolerance and meeting the performance requirement, such as throughput.     

Significance of incorporating excitation source parameters for improved emotion recognition from speech and electroglottographic signals

The work presented in this paper explores the effectiveness of incorporating the excitation source parameters such as strength of excitation and instantaneous fundamental frequency (\(F_0\)) for emotion recognition task from speech and electroglottographic (EGG) signals. The strength of excitation (SoE) is an important parameter indicating the pressure with which glottis closes at the glottal closure instants (GCIs). The SoE is computed by the popular zero frequency filtering (ZFF) method which accurately estimates the glottal signal characteristics by attenuating or removing the high frequency vocaltract interactions in speech. The arbitrary impulse sequence, obtained from the estimated GCIs, is used to derive the instantaneous \(F_0\). The SoE and the instantaneous \(F_0\) parameters are combined with the conventional mel frequency cepstral coefficients (MFCC) to improve the recognition rates of distinct emotions (Anger, Happy and Sad) using Gaussian mixture models as classifier. The performances of the proposed combination of SoE and instantaneous \(F_0\) and their dynamic features with MFCC coefficients are compared with the emotion utterances (4 emotions and neutral) from classical German full blown emotion speech database (EmoDb) having simultaneous speech and EGG signals and Surrey Audio Visual Expressed Emotion database (3 emotions and neutral) for both speaker dependent and speaker independent emotion recognition scenarios. To reinforce the effectiveness of the proposed features and for better statistical consistency of the emotion analysis, a fairly large emotion speech database of 220 utterances per emotion in Tamil language with simultaneous EGG recordings, is used in addition to EmoDb. The effectiveness of SoE and instantaneous \(F_0\) in characterizing different emotions is also confirmed by the improved emotion recognition performance in Tamil speech-EGG emotion database.     

Emotion modeling from speech signal based on wavelet packet transform

The recognition of emotion in human speech has gained increasing attention in recent years due to the wide variety of applications that benefit from such technology. Detecting emotion from speech can be viewed as a classification task. It consists of assigning, out of a fixed set, an emotion category e.g. happiness, anger, to a speech utterance. In this paper, we have tackled two emotions namely happiness and anger. The parameters extracted from speech signal depend on speaker, spoken word as well as emotion. To detect the emotion, we have kept the spoken utterance and the speaker constant and only the emotion is changed. Different features are extracted to identify the parameters responsible for emotion. Wavelet packet transform (WPT) is found to be emotion specific. We have performed the experiments using three methods. Method uses WPT and compares the number of coefficients greater than threshold in different bands. Second method uses energy ratios of different bands using WPT and compares the energy ratios in different bands. The third method is a conventional method using MFCC. The results obtained using WPT for angry, happy and neutral mode are 85 %, 65 % and 80 % respectively as compared to results obtained using MFCC i.e. 75 %, 45 % and 60 % respectively for the three emotions. Based on WPT features a model is proposed for emotion conversion namely neutral to angry and neutral to happy emotion.     

A Distributed Address Configuration Scheme for a MANET

This paper proposes a distributed address configuration scheme for a Mobile Ad hoc Network (MANET). The architecture for a MANET, the algorithm of constructing a MANET, and the algorithm for calculating a unique address space for the assignment are proposed. In the architecture, a common node has a unique address space for the assignment, and an isolated node can acquire a unique address from a neighboring common node without performing duplicate address detection. In this way, the address configuration task is distributed around common nodes. In this scheme, the control packets used for address configuration are exchanged within a one-hop scope, so the scalability is enhanced. This paper also presents an address recovery algorithm that can effectively retrieve the address resources released by failed nodes and the MANET merging/partitioning algorithm that can ensure a node’s address uniqueness. This paper compares the performance parameters of the proposed scheme and the existing schemes, including strong duplicate address detection and prime dynamic host configuration protocol, and the comparative results show that the address configuration cost of the proposed scheme is lower and the delay is shorter.     

U-biofeedback: a multimedia-based reference model for ubiquitous biofeedback systems

Biofeedback is a well-accepted approach in preventative and alternative healthcare. It is known to promote wellbeing and help prevent and treat a wide variety of disorders related to the human physiology and psychology. With the exceptional growth of wearable sensor technologies, the potential for devising biofeedback systems that blend into everyday living is immense. Therefore, we present our vision for U-Biofeedback, a reference model for systems designed to continuously monitor our physiology and convey to us important messages regarding our status. Also, we present a case study for an application that implements our reference model. The application is designed to monitor the stress of individuals working in an office setting and provide an assistive response whenever stress reaches elevated levels. By devising an algorithm for stress detection that makes use of Heart Rate Variability (HRV) measures, we were able to identify negative stress situations with an accuracy of 89.63 % and a false positive detection rate of 5.55 % during our evaluation.     

Fuzzy model of dominance emotions in affective computing

To date, most of the human emotion recognition systems are intended to sense the emotions and their dominance individually. This paper discusses a fuzzy model for multilevel affective computing based on the dominance dimensional model of emotions. This model can detect any other possible emotions simultaneously at the time of recognition. One hundred and thirty volunteers from various countries with different cultural backgrounds were selected to record their emotional states. These volunteers have been selected from various races and different geographical locations. Twenty-seven different emotions with their strengths in a scale of 5 were questioned through a survey. Recorded emotions were analyzed with the other possible emotions and their levels of dominance to build the fuzzy model. Then this model was integrated into a fuzzy emotion recognition system using three input devices of mouse, keyboard and the touch screen display. Support vector machine classifier detected the other possible emotions of the users along with the directly sensed emotion. The binary system (non-fuzzy) sensed emotions with an incredible accuracy of 93 %. However, it only could sense limited emotions. By integrating this model, the system was able to detect more possible emotions at a time with slightly lower recognition accuracy of 86 %. The recorded false positive rates of this model for four emotions were measured at 16.7 %. The resulted accuracy and its false positive rate are among the top three accurate human emotion recognition (affective computing) systems.     

Designing RBF Networks Using the Agent-Based Population Learning Algorithm

Radial Basis Function Neural Networks (RBFNs) are nowadays quite popular due to their ability to discover and approximate complex nonlinear dependencies within the data under analysis. Performance of the RBF network depends on numerous factors related to its initialization and training. The paper proposes an approach to the radial basis function networks design, where initial parameters of the network, output weights and parameters of the transfer function are set using the proposed agent-based population learning algorithm (PLA). The algorithm is validated experimentally. Advantages and main features of the PLA-based RBF designs are discussed basing on results of the computational experiment.     

Replication of sub-100 nm structures using h- and s-PDMS composite stamps

Soft-UV-NIL as replication technique was used to replicate sub-100 nm structures. The aim of this work is the stamp production and the replication of structures with dimensions smaller than 100 nm in a simple manner. Composite stamps composed of two layers, a thin hard PDMS layer supported by a thick soft PDMS (s-PDMS) layer are compared to common s-PDMS stamps regarding the resolution by using a Siemens star (star burst pattern) as test structure. The master is fabricated by electron beam lithography in a 140 nm thick PMMA resist layer. The stamp is molded directly from the structured resist, without any additional anti sticking treatment. Therefore the resist thickness determines the aspect ratio, which is 1.5 at the resolution limit. The replication is done in a UV-curing cycloaliphatic epoxy material. The employed test structure provides good comparability, the resolution limit at a glance, and it integrates a smooth transition from micro- to nanostructures. Therefore it is a capable structure to characterize the UV-NIL.     

Adaptive image data hiding using transformation and error replacement

Watermarking is popular method for effective copyright protection of digital data. The invisible image watermarking is a type of watermarking which is used to conceal secret information in a cover image. In any watermarking system, there is a trade-off between imperceptibility and payload of the data to be hidden in the cover medium. Therefore, only a limited amount of data can be concealed in the cover image while watermark remain imperceptible. This paper proposes a new method of adaptive blind digital image data hiding in Discrete Cosine Transform (DCT) domain using Minimum Error Lease Significant Bit Replacement (MELSBR) method and Genetic Algorithm (GA). In the proposed method, the secret image is embedded into the DCT transformed cover image. Initially, the host and the secret image both are partitioned into equal sized blocks. Then for each secret image block, host image block is intelligently selected through GA and is embedded using MELSBR method. GA helps in identifying the target cover image blocks such that with the LSB embedding, both visual quality of the cover image and imperceptibility of the secret image remain least affected. In watermark extraction process, the watermarked image is decomposed into equal sized blocks and then using the Jigsaw Puzzle Solver (JPS) the secret image is reconstructed. In spite of doubling the payload of the data to hide, the experimental results have shown better imperceptibility and robustness results of the proposed method as compared to the current approaches in this domain.     

WTrack: HMM-based walk pattern recognition and indoor pedestrian tracking using phone inertial sensors

Indoor tracking systems have become very popular, wherein pedestrian movement is analyzed in a variety of commercial and secure spaces. The inertial sensor-based method makes great contributions to continuous and seamless indoor pedestrian tracking. However, such a system is vulnerable to the cumulative locating errors when moving distance increases. Inaccurate heading values caused by the interference of body swing of natural walking and the geomagnetic disturbances are the main sources of the accumulative errors. To reduce such errors, additional infrastructure or highly accurate sensors have been used by previous works that considerably raise the complexity of the architecture. This paper presents an indoor pedestrian tracking system called WTrack, using only geomagnetic sensors and acceleration sensors that are commonly carried by smartphones. A fine-grained walk pattern of indoor pedestrians is modeled through Hidden Markov Model. With this model, WTrack can track indoor pedestrians by continuously recognizing the pre-defined pedestrians’ walk pattern. More importantly, WTrack is able to resist both the interference of body swing of natural walking and the geomagnetic disturbances of nearby objects. Our experimental results reveal that the location error is <2 m, which is considered adequate for indoor location-based-service applications. The adaptive sample rate adjustment mode further reduces the energy consumption by 52 % in comparison, as opposed to the constant sampling mode.     

Lightweight graphics instrumentation for game state-specific power management in Android

Battery life is a major concern on portable devices like smartphones and tablet PCs. On these devices, games constitute the class of most popular applications and are at the same time highly compute-intensive. Every game consists of several states like the loading, main menu and the gaming state. Each of those states has its own workload characteristics, e.g., the loading phase is likely to be memory bound and the main menu state is less interactive than the gaming state. We propose an interception technique that allows to profile the game and detect its current state based on the game’s communication with the underlying OS. Current power management governors are unaware of the running applications and scale the processor’s voltage and frequency merely based on the system’s utilization. We provide the game’s state information and workload profile to our governor which selects the processing frequency such that the desired frame rate of the current state is ensured. This leads to an optimal choice of processing frequencies and thereby significantly reduces power consumption. We have implemented the scheme on an Android-based Samsung Galaxy Nexus smartphone using popular games like Jetpack Joyride and Temple Run. We reduced the CPU’s power consumption by up to 43.2 % compared to the Android interactive governor without impacting the gaming experience. Motivated by these results we propose a power management API that would allow game developers to significantly reduce the power consumption of their game using simple API calls.