Human–robot interaction based on gesture and movement recognition

Human–robot interaction (HRI) has become a research hotspot in computer vision and robotics due to its wide application in human–computer interaction (HCI) domain. Based on the explored algorithms of gesture recognition and limb movement recognition in somatosensory interaction, an HRI model of a robotic arm is proposed for robot arm manipulation. More specifically, 3D SSD architecture is used for the location and identification of gesture and arm movement. Then, DTW template matching algorithm is adopted to trace the dynamic gestures. The interactive scenarios and interactive modes are designed for experiment and implementation. Virtual interactive experimental results have demonstrated the usefulness of our method.     

A model and framework for human–environment interaction

This paper describes our approach for studying and prototyping human–environment interaction (HEI) within a pervasive space applied to ambient assistive living context. The objective of our approach consists in developing and implementing an HEI framework to modelling the human–machine interaction. This framework allows a customization facility for designers, developers and even end-users in defining and processing multimodal interaction. We underline the fact that the design of interaction for users with special needs do not have to be considered as orthogonal to the application but rather as a validation context which is the heart of our research laboratory activities. We have integrated our framework in demonstrator dedicated to people with disabilities to validate the concept. This paper will present the framework, the developed components of our HEI model and the prototype realised. The outcome of this research activity is to provide a multimodal processing framework to favour natural human pervasive environment by decreasing the cognitive workload necessary in a complex environment. Thus, this will contribute to achieve ubiquitous accessible space.     

Deep learning and RGB-D based human action,human–human and human–object interaction recognition: A survey

Human activity recognition is one of the most studied topics in the field of computer vision. In recent years, with the availability of RGB-D sensors and powerful deep learning techniques, research on human activity recognition has gained momentum. From simple human atomic actions, the research has advanced towards recognizing more complex human activities using RGB-D data. This paper presents a comprehensive survey of the advanced deep learning based recognition methods and categorizes them in human atomic action, human–human interaction, human–object interaction. The reviewed methods are further classified based on the individual modality used for recognition i.e. RGB based, depth based, skeleton based, and hybrid. We also review and categorize recent challenging RGB-D datasets for the same. In addition, the paper also briefly reviews RGB-D datasets and methods for online activity recognition. The paper concludes with a discussion on limitations, challenges, and recent trends for promising future directions.     

Brain–computer interface: The next frontier of telemedicine in human–computer interaction

The study proposes a novel brain–computer interface scheme for the next frontier of telemedicine in human–computer interaction, where the goal is to improve the interactions between users and computers in telemedicine. The system consists of discriminative area selection, feature extraction and classification. Discriminative area selection is proposed to obtain the optimal discriminative area, which can decrease the time length of event-related area to achieve more efficient computation and higher accuracy. A fuzzy Hopfield neural network is used to classify the features extracted by means of wavelet-fractal approach. Experimental results show that the proposed system is robust and performs better than several previous methods. It is also suggested being suitable for the applications of telemedicine in human–computer interaction.     

Gesture recognition for human–machine interaction in table tennis video based on deep semantic understanding

The analysis of moving objects in videos, especially the recognition of human motions and gestures, is attracting increasing emphasis in computer vision area. However, most existing video analysis methods do not take into account the effect of video semantic information. The topological information of the video image plays an important role in describing the association relationship of the image content, which will help to improve the discriminability of the video feature expression. Based on the above considerations, we propose a video semantic feature learning method that integrates image topological sparse coding with dynamic time warping algorithm to improve the gesture recognition in videos. This method divides video feature learning into two phases: semi-supervised video image feature learning and supervised optimization of video sequence features. Next, a distance weighting based dynamic time warping algorithm and K-nearest neighbor algorithm is leveraged to recognize gestures. We conduct comparative experiments on table tennis video dataset. The experimental results show that the proposed method is more discriminative to the expression of video features and can effectively improve the recognition rate of gestures in sports video.     

Dempster–Shafer theory-based human activity recognition in smart home environments

Context awareness and activity recognition are becoming a hot research topic in ambient intelligence (AmI) and ubiquitous robotics, due to the latest advances in wireless sensor network research which provides a richer set of context data and allows a wide coverage of AmI environments. However, using raw sensor data for activity recognition is subject to different constraints and makes activity recognition inaccurate and uncertain. The Dempster–Shafer evidence theory, known as belief functions, gives a convenient mathematical framework to handle uncertainty issues in sensor information fusion and facilitates decision making for the activity recognition process. Dempster–Shafer theory is more and more applied to represent and manipulate contextual information under uncertainty in a wide range of activity-aware systems. However, using this theory needs to solve the mapping issue of sensor data into high-level activity knowledge. The present paper contributes new ways to apply the Dempster–Shafer theory using binary discrete sensor information for activity recognition under uncertainty. We propose an efficient mapping technique that allows converting and aggregating the raw data captured, using a wireless senor network, into high-level activity knowledge. In addition, we propose a conflict resolution technique to optimize decision making in the presence of conflicting activities. For the validation of our approach, we have used a real dataset captured using sensors deployed in a smart home. Our results demonstrate that the improvement of activity recognition provided by our approaches is up to of 79 %. These results demonstrate also that the accuracy of activity recognition using the Dempster–Shafer theory with the proposed mappings outperforms both naïve Bayes classifier and J48 decision tree.     

A fast test technique for life time estimation of ultrasonically welded Cu–Cu interconnects

In this research the quality of the interconnects of the ultrasonically welded Cu terminals to the Cu substrate in the IGBT-module has been investigated. An ultrasonic resonance fatigue system in combination with a laser Doppler vibrometer and a special specimen design was used for shear fatigue testing of these large ultrasonic Cu–Cu welds (about 0.5 cm²). Fatigue life curves up to 10⁹ loading cycles were obtained in a very short period of time. Using this technique it was possible to evaluate the fatigue strength of these interconnects for the first time. The microstructural features of the interconnects were characterized and their crack growth behaviour was studied. Fracture analysis of the fatigued specimen shows that failure occur due to the propagation of the crack beneath the welding interface into the copper substrate. Additionally performed finite element simulations offer an insight into the stress and strain concentrations during the mechanical fatigue tests. As this method is not restricted to the welding geometry, material joints with larger interconnects can be tested likewise. Thus this new technique can be used as a practical and valid fatigue testing method for evaluation of various interconnects.     

A pre‐serialization transaction management technique for mobile multidatabases

Rapid advances in hardware and wireless communication technology have made the concept of mobile computing a reality. Thus, evolving database technology needs to address the requirements of the future mobile user. The frequent disconnection and migration of the mobile user violate underlying presumptions about connectivity that exist in wired database systems and introduce new issues that affect transaction management. In this paper, we present the PreSerialization (PS) transaction management technique for the mobile multidatabase environment. This technique addresses disconnection and migration and enforces a range of atomicity and isolation criteria. We also develop an analytical model to compare the performance of the PS technique to that of the Kangaroo model.     

Modelling of spatio–temporal interaction for video quality assessment

Video services have appeared in the recent years due to advances in video coding and convergence to IP networks. As these emerging services mature, the ability to deliver adequate quality to end-users becomes increasingly important. However, the transmission of digital video over error-prone and bandwidth-limited networks may produce spatial and temporal visual distortions in the decoded video. Both types of impairments affect the perceived video quality. In this paper, we examine the impact of spatio–temporal artefacts in video and especially how both types of errors interact to affect the overall perceived video quality. We show that the impact of the spatial quality on overall video quality is dependent on the temporal quality and vice-versa. We observe that the introduction of a degradation in one modality affects the quality perception in the other modality, and this change is larger for high-quality conditions than for low-quality conditions. The contribution of the spatial quality to the overall quality is found to be greater than the contribution of the temporal quality. Our results also indicate that low-motion talking-head content can be more negatively affected by temporal frame freezing artefacts than other general type of content with higher motion. Based on the results of a subjective experiment, we propose an objective model to predict overall video quality by integrating the contributions of a spatial quality and a temporal quality. The non-linear model shows a very high linear correlation with subjective data.     

Intermediate deep feature coding for human–machine vision collaboration

Traditional image coding are mainly designed for human vision. While for collaborative intelligence, deep feature coding is specific for machine vision, which includes feature extraction and compression. Actually, deep features can build a bridge between human and machine vision. Therefore, we focus on generalized deep feature extraction and compression for multitask, which includes image reconstruction task for human vision and computer visual tasks for machine vision. After analyzing correlation among multitask, a reconstruction guided feature extraction strategy and feature fusion based network are proposed to get more generalized intermediate deep feature, which contains sufficient information friendly for human and machine vision. Besides, a non-uniform quantization method based on importance and a compact representation method for feature distribution information protection are proposed for high efficiency feature coding. Eventually, we come up with an entire intermediate deep feature coding framework including feature extraction and compression. Experimental results indicate the performance gains with our framework.     

A 0.5 V LC-VCO with improved varactor tuning technique for WSN

A 0.5 V LC-VCO implemented in 0.18 μm CMOS technology for wireless sensor network is described in this paper. An improved varactor tuning technique is proposed to decrease low frequency noise up-conversion and AM–FM phase noise of VCO, also it can increase Q of LC tank and reduce power consumption of VCO. For coarse tuning of VCO, it can increase the varactor control voltage variation range. For fine tuning of VCO, it can reduce the varactor nonlinearity. The measured tuning range is 4.58–5.26 GHz and power consumption is 2.2 mW. The measured phase noise is ?114 dBc/Hz at 1 MHz frequency offset from a 4.8 GHz carrier.     

A 0.1–1 GHz low power RF receiver front-end with noise cancellation technique for WSN applications

A low power 0.1–1 GHz RF receiver front-end composed of noise-cancelling trans-conductor stage and I/Q switch stage was presented in this paper. The RF receiver front-end chip was fabricated in 0.18 µm RF CMOS. Measurement results show the receiver front-end has a conversion gain of 28.1 dB at high gain mode, and the single-sideband (SSB) noise figure is 6.2 dB. In the low gain mode, the conversion gain of the receiver front-end is 15.5 dB and the IP1dB is −12 dBm. In this design, low power consumption and low cost is achieved by current-reuse and inductor-less topology. The receiver front-end consumes only 5.2 mW from a 1.8 V DC supply and the chip size of the core circuit is 0.12 mm².     

High-accuracy current sensing circuit with current compensation technique for buck–boost converter

A novel on-chip current sensing circuit with current compensation technique suitable for buck–boost converter is presented in this article. The proposed technique can sense the full-range inductor current with high accuracy and high speed. It is mainly based on matched current mirror and does not require a large proportion of aspect ratio between the powerFET and the senseFET, thus it reduces the complexity of circuit design and the layout mismatch issue without decreasing the power efficiency. The circuit is fabricated with TSMC 0.25 µm 2P5M mixed-signal process. Simulation results show that the buck-boost converter can be operated at 200 kHz to 4 MHz switching frequency with an input voltage from 2.8 to 4.7 V. The output voltage is 3.6 V, and the maximum accuracy for both high and low side sensing current reaches 99% within the load current ranging from 200 to 600 mA.     

Excess-loop-delay compensation technique for CT ΔΣ modulator with hybrid active–passive loop-filters

The design and optimization methodology for CT ΣΔ modulators with hybrid Active–Passive (AP) loop-filters is indicated in this work. From the discussion, by appropriately scaling the passive filter gain and cooperating with a single-bit quantizer, the hybrid AP loop filtering can achieve an approximated noise-shaping function as a fully active ΔΣ modulator with the same order. The ELD effect in the hybrid AP CT ΔΣ modulator which influences the poles and zeros locations of the Noise Transfer Function (NTF) in the modulator is depicted. This paper also investigates the feasibility of applying the ELD compensation techniques that were used to be implemented in the active integrator’s case to the hybrid AP CT ΔΣ modulator; however, some of them cannot be practically applied since the passive loop-filter cannot perform proportional feedback signal summation. After the discussion and analysis, the technique similar to Vadipour et al. (In: Symposium on VLSI circuits digest of technical papers, 2008) can be easily implemented at circuit-level and after applying it, there is one additional zero to compensate the peak in the NTF. With the help of this technique, the maximum quantizer delay tolerance can be a full clock period. The mentioned ELD compensation technique was applied in a 2nd order CT ΔΣ modulator with an active-RC integrator as the 1st stage and a passive RC filter as the 2nd stage, which was verified by transistor-level simulations in 65 nm CMOS. The circuit exhibits either 67.3 dB or 65.3 of SNDR, under the effect of half clock period or one clock period ELD, respectively; by contrast, without compensation, the system is unstable with both half or one clock period ELD effect. The designed hybrid CT ΔΣ modulator achieves 2 MHz signal bandwidth and consumes 2.54 mW of power.     

A technique to improve the linearization of frequency–voltage characteristic of LC-VCO

This paper presents a very low-power linearization technique to improve the linearity of frequency-voltage characteristic of LC-VCO (voltage controlled oscillator) using MOS varactor. This reduces the VCO gain (K _VCO) variation and its required value over the tuning voltage range. Low K _VCO improves noise and reference spur performances at the output of phase lock loop/frequency synthesizer (FS). Low K _VCO variation reduces FS loop stability problem. Using this VCO circuit, a fully on-chip integer-N frequency synthesizer has been fabricated in 0.18 μm epi-digital CMOS technology for 2.45 GHz ZigBee application. The measured VCO phase noise is ?115.76 and ?125.23 dBc/Hz at 1 and 3 MHz offset frequencies, respectively from 2.445 GHz carrier and the reference spur of the frequency synthesizer is ?68.62 dBc. The used supply voltage is 1.5 V.     

A new model-based technique for enhanced small-vessel measurements in X-ray ciné-angiograms

Arterial diameter estimation from X-ray ciné angiograms is important for quantifying coronary artery disease (CAD) and for evaluating therapy. However, diameter measurement in vessel cross sections < or =1.0 mm is associated with large measurement errors. We present a novel diameter estimator which reduces both magnitude and variability of measurement error. We use a parametric nonlinear imaging model for X-ray ciné angiography and estimate unknown model parameters directly from the image data. Our technique allows us to exploit additional diameter information contained within the intensity profile amplitude, a feature which is overlooked by existing methods. This method uses a two-step procedure: the first step estimates the imaging model parameters directly from the angiographic frame and the second step uses these measurements to estimate the diameter of vessels in the same image. In Monte-Carlo simulation over a range of imaging conditions, our approach consistently produced lower estimation error and variability than conventional methods. With actual X-ray images, our estimator is also better than existing methods for the diameters examined (0.4-4.0 mm). These improvements are most significant in the range of narrow vessel widths associated with severe coronary artery disease.     

Constrained haptic-guided shared control for collaborative human–robot percutaneous nephrolithotomy training

Percutaneous nephrolithotomy is a procedure used to treat patients with large or irregularly shaped kidney stones. Surgical instruments are inserted through a small incision to access the kidney and remove the calculi. Surgeons who have less experience with the procedure manifest significantly higher rates of complications due to the procedure’s steep learning curve. This issue is further exacerbated by a lack of training opportunities in clinical settings. This paper introduces a teleoperative framework that can provide training to surgeons as well as assistance during procedures, based on two main components. Firstly, a type of constrained inverse kinematics that decouples the tooltip position from its orientation using a remote centre of motion, and incorporates the joint limits analytically. This reduces the workload of the procedure by having the surgeon control only the tooltip position rather than the position and the orientation while preventing the inverse kinematics from returning joint angles outside of the robot’s abilities. This kinematic framework also allows a three-degrees-of-freedom haptic device to control a six-degrees-of-freedom manipulator. Secondly, haptic feedback is provided to help guide and teach the surgeon during the procedure. Haptic feedback allows the surgeon to remain in full control during the procedure while still receiving haptic cues and assistance.Experimental results indicate that the haptic cues improved user’s accuracy, and they had shorter and smoother paths. This leads to a shorter procedure time overall. The results also indicate that the haptic assistance helped teach users the ideal trajectory of the procedure and that users who were taught with haptic feedback performed better than those who never experienced any haptic feedback.     

Recent fluid–structure interaction modeling challenges in IC encapsulation – A review

The rapid development of computing software has facilitated multifarious research in integrated circuit (IC) packaging. Complicated and complex processes can be visualized via simulation modeling with this software. The applications of aided software enhance the fundamental physicochemical understanding and visualization of the IC encapsulation process. In this article, fluid–structure interaction (FSI) during IC encapsulation through computer-aided simulation is reviewed based on the amount of substantial work conducted from the past decades to the present. FSI phenomena in various IC encapsulations, such as wire sweep, paddle shift, lead frame deformation, IC chip, and through-silicon via (TSV) deformation, is considered in the review. The significance and challenges of FSI analysis are also highlighted in this article.