View-invariant gesture recognition using 3D optical flow and harmonic motion context

This paper presents an approach for view-invariant gesture recognition. The approach is based on 3D data captured by a SwissRanger SR4000 camera. This camera produces both a depth map as well as an intensity image of a scene. Since the two information types are aligned, we can use the intensity image to define a region of interest for the relevant 3D data. This data fusion improves the quality of the motion detection and hence results in better recognition. The gesture recognition is based on finding motion primitives (temporal instances) in the 3D data. Motion is detected by a 3D version of optical flow and results in velocity annotated point clouds. The 3D motion primitives are represented efficiently by introducing motion context. The motion context is transformed into a view-invariant representation using spherical harmonic basis functions, yielding a harmonic motion context representation. A probabilistic Edit Distance classifier is applied to identify which gesture best describes a string of primitives. The approach is trained on data from one viewpoint and tested on data from a very different viewpoint. The recognition rate is 94.4% which is similar to the recognition rate when training and testing on gestures from the same viewpoint, hence the approach is indeed view-invariant.     

Real-time construction worker posture analysis for ergonomics training

Construction activities performed by workers are usually repetitive and physically demanding. Execution of such tasks in awkward postures can strain their body parts and can result in fatigue, injuries or in severe cases permanent disabilities. In view of this, it is essential to train workers, before the commencement of any construction activity. Furthermore, traditional worker monitoring methods are tedious, inefficient and are carried out manually whereas, an automated approach, apart from monitoring, can yield valuable information concerning work-related behavior of worker that can be beneficial for worker training in a virtual reality world. Our research work focuses on developing an automated approach for posture estimation and classification using a range camera for posture analysis and categorizing it as ergonomic or non-ergonomic. Using a range camera, first we classify worker’s pose to determine whether a worker is ‘standing’, ‘bending’, ‘sitting’, or ‘crawling’ and then estimate the posture of the worker using OpenNI middleware to get the body joint angles and spatial locations. A predefined set of rules is then formulated to use this body posture information to categorize tasks as ergonomic or non-ergonomic.     

Ergonomic assessment of office worker postures using 3D automated joint angle assessment

Sedentary activity and static postures are associated with work-related musculoskeletal disorders (WMSDs) and worker discomfort. Ergonomic evaluation for office workers is commonly performed by experts using tools such as the Rapid Upper Limb Assessment (RULA), but there is limited evidence suggesting sustained compliance with expert’s recommendations. Assessing postural shifts across a day and identifying poor postures would benefit from automation by means of real-time, continuous feedback. Automated postural assessment methods exist; however, they are usually based on ideal conditions that may restrict users’ postures, clothing, and hair styles, or may require unobstructed views of the participants. Using a Microsoft Kinect camera and open-source computer vision algorithms, we propose an automated ergonomic assessment algorithm to monitor office worker postures, the 3D Automated Joint Angle Assessment, 3D-AJA. The validity of the 3D-AJA was tested by comparing algorithm-calculated joint angles to the angles obtained from manual goniometry and the Kinect Software Development Kit (SDK) for 20 participants in an office space. The results of the assessment show that the 3D-AJA has mean absolute errors ranging from 5.6° ± 5.1° to 8.5° ± 8.1° for shoulder flexion, shoulder abduction, and elbow flexion relative to joint angle measurements from goniometry. Additionally, the 3D-AJA showed relatively good performance on the classification of RULA score A using a Random Forest model (micro averages F1-score = 0.759, G-mean = 0.811), even at high levels of occlusion on the subjects’ lower limbs. The results of the study provide a basis for the development of a full-body ergonomic assessment for office workers, which can support personalized behavior change and help office workers to adjust their postures, thus reducing their risks of WMSDs.     

Vision-based Classification of Driving Postures by Efficient Feature Extraction and Bayesian Approach

Reports of traffic accidents show that a considerable percentage of the accidents are caused by human factors. Human-centric driver assistance systems, with integrated sensing, processing and networking, aim to find solutions to this problem and other relevant issues. The key technology in such systems is the capability to automatically understand and characterize driver behaviors. In this paper, we propose a novel, efficient feature extraction approach for driving postures from a video camera, which consists of Homomorphic filter, skin-like regions segmentation, canny edge detection, connected regions detection, small connected regions deletion and spatial scale ratio calculation. With features extracted from a driving posture dataset we created at Southeast University (SEU), holdout and cross-validation experiments on driving posture classification are then conducted using Bayes classifier. Compared with a number of commonly used classification methods including naive Bayes classifier, subspace classifier, linear perception classifier and Parzen classifier, the holdout and cross-validation experiments show that the Bayes classifier offers better classification performance than the other four classifiers. Among the four predefined classes, i.e., grasping the steering wheel, operating the shift gear, eating a cake and talking on a cellular phone, the class of talking on a cellular phone is the most difficult to classify. With Bayes classifier, the classification accuracies of talking on a cellular phone are over 90 % in holdout and cross-validation experiments, which shows the effectiveness of the proposed feature extraction method and the importance of Bayes classifier in automatically understanding and characterizing driver behaviors towards human-centric driver assistance systems.     

Personalized method for self-management of trunk postural ergonomic hazards in construction rebar ironwork

Construction rebar workers face postural ergonomic hazards that can lead to work-related Lower Back Disorders (LBDs), primarily due to their prolonged awkward working postures required by the job. In a previous study, Wearable Inertial Measurement Units (WIMUs)-based Personal Protective Equipment (PPE) was developed to alert workers when their trunk inclination holding time exceeded acceptable thresholds as defined in ISO standard 11226:2000. However, subsequent field testing identified PPE was ineffective for some workers because the adopted ISO thresholds were not personalized and did not consider differences in individual’s response to postural ergonomic hazards. To address this problem, this paper introduces a worker-centric method to assist in the self-management of work-related ergonomic hazards, based on data-driven personalized healthcare intervention. Firstly, personalized information is gathered by providing each rebar ironworker a WIMU-based personalized mobile health (mHealth) system to capture their trunk inclination angle and holding time data. Then, the captured individual trunk inclination holding times are analyzed by a Gaussian-like probability density function, where abnormal holding time thresholds can be generated and updated in response to incoming trunk inclination records of an individual during work time. These abnormal holding time thresholds are then adapted to be used as personalized trunk inclination holding time recommendations for an individual worker to self-manage their working postures, based on their own trunk inclination records. The proposed worker-centric method to assist in the self-management of ergonomic postural hazards leading to LBDs was field tested on a construction site over a three-month duration. The results of the paired t-tests indicate that posture scores evaluated by the Ovako Working Posture Analysis System (OWAS) significantly decrease when the personalized recommendation is applied, while increase again when the personalized recommendation is removed. Based on data-driven personalized healthcare intervention, the results demonstrate the significant potential of the proposed worker-centric self-management method for rebar workers in preventing and controlling postural ergonomic hazards during construction rebar ironwork.     

Recognition of driving postures by multiwavelet transform and multilayer perceptron classifier

To develop Human-centric Driver Assistance Systems (HDAS) for automatic understanding and charactering of driver behaviors, an efficient feature extraction of driving postures based on Geronimo–Hardin–Massopust (GHM) multiwavelet transform is proposed, and Multilayer Perceptron (MLP) classifiers with three layers are then exploited in order to recognize four pre-defined classes of driving postures. With features extracted from a driving posture dataset created at Southeast University (SEU), the holdout and cross-validation experiments on driving posture classification are conducted by MLP classifiers, compared with the Intersection Kernel Support Vector Machines (IKSVMs), the k-Nearest Neighbor (kNN) classifier and the Parzen classifier. The experimental results show that feature extraction based on GHM multwavelet transform and MLP classifier, using softmax activation function in the output layer and hyperbolic tangent activation function in the hidden layer, offer the best classification performance compared to IKSVMs, kNN and Parzen classifiers. The experimental results also show that talking on a cellular phone is the most difficult one to classify among four predefined classes, which are 83.01% and 84.04% in the holdout and cross-validation experiments respectively. These results show the effectiveness of the feature extraction approach using GHM multiwavelet transform and MLP classifier in automatically understanding and characterizing driver behaviors towards Human-centric Driver Assistance Systems (HDAS).     

A software‐based method for ergonomic posture assessment in automotive preproduction planning: Concordance and difference in using software and personal observation for assessments

This article describes the pilot study for testing a system for ergonomic posture assessment. 1 The software‐based approach for ergonomic assessment is based on motion capturing work tasks using realistic mock‐ups of the assembly environment. The approach for ergonomic assessment reduces the expenditure of time for evaluation. This may lead to considerable ergonomic aspects during the early stages of production planning. A comparative test between human observers and the software system was carried out. The test of the system focuses on correct recognition of postures from the motion data and compares the computer‐based evaluation with the results of a manual analysis carried out by a human observer. The tests indicate sufficient results for the automated assessment, especially for the arm and leg postures. But there is still some potential for further improvement of the recognition accuracy of the back postures. © 2011 Wiley Periodicals, Inc.     

Applying incremental Deep Neural Networks-based posture recognition model for ergonomics risk assessment in construction

Monitoring and assessing awkward postures is a proactive approach for Musculoskeletal Disorders (MSDs) prevention in construction. Machine Learning models have shown promising results when used in recognition of workers’ posture from Wearable Sensors. However, there is a need to further investigate: i) how to enable Incremental Learning, where trained recognition models continuously learn new postures from incoming subjects while controlling the forgetting of learned postures; ii) the validity of ergonomics risk assessment with recognized postures. The research discussed in this paper seeks to address this need through an adaptive posture recognition model– the incremental Convolutional Long Short-Term Memory (CLN) model. The paper discusses the methodology used to develop and validate this model’s use as an effective Incremental Learning strategy. The evaluation was based on real construction workers’ natural postures during their daily tasks. The CLN model with “shallow” (up to two) convolutional layers achieved high recognition performance (Macro F1 Score) under personalized (0.87) and generalized (0.84) modeling. Generalized CLN model, with one convolutional layer, using the “Many-to-One” Incremental Learning scheme can potentially balance the performance of adaptation and controlling forgetting. Applying the ergonomics rules on recognized and ground truth postures yielded comparable risk assessment results. These findings support that the proposed incremental Deep Neural Networks model has a high potential for adaptive posture recognition. They can be deployed alongside ergonomics rules for effective MSDs risk assessment.     

Physical ergonomic hazards in highway tunnel construction: overview from the Construction Occupational Health Program

This report provides an overview of physical ergonomic exposures in highway construction work across trades and major operations. For each operation, the observational method “PATH” (Posture, Activity, Tools and Handling) was used to estimate the percentage of time that workers spent in specific tasks and with exposure to awkward postures and load handling. The observations were carried out on 73 different days, typically for about 4 h per day, covering 120 construction workers in 5 different trades: laborers, carpenters, ironworkers, plasterers, and tilers. Non-neutral trunk postures (forward or sideways flexion or twisting) were frequently observed, representing over 40% of observations for all trades except laborers (28%). Kneeling and squatting were common in all operations, especially tiling and underground utility relocation work. Handling loads was frequent, especially for plasterers and tilers, with a range of load weights but most often under 15 pounds. The results of this study provide quantitative evidence that workers in highway tunnel construction operations are exposed to ergonomic factors known to present significant health hazards. Numerous opportunities exist for the development and implementation of ergonomic interventions to protect the health and safety of construction workers.     

基于姿态校正与姿态融合的2D/3D骨架动作识别方法

针对现有的人体骨架动作识别方法对肢体信息挖掘不足以及时间特征提取不足的问题,提出了一种基于姿态校正模块与姿态融合模块的模型PTF-SGN,实现了对骨架图关键时空信息的充分利用。首先,对骨架图数据进行预处理,挖掘肢体和关节点的位移信息并提取特征;然后,姿态校正模块通过无监督学习的方式获取姿态调整因子,并对人体姿态进行自适应调整,增强了模型在不同环境下的鲁棒性;其次,提出一种基于时间注意力机制的姿态融合模块,学习骨架图中的短时刻特征与长时刻特征并融合长短时刻特征,加强了对时间特征的表征能力;最后,将骨架图的全局时空特征输入到分类网络中得到动作识别结果。在NTU60 RGB+D、NTU120 RGB+D两个3D骨架数据集和Penn-Action、HARPET两个2D骨架数据集上的实验结果表明,该模型能够有效地识别骨架时序数据的动作。     

Reliability of assessing upper limb postures among workers performing manufacturing tasks

The purpose of this study was to determine the inter- and intra-rater reliability of assessing upper limb postures of workers performing manufacturing tasks. Assessment of neck, shoulder, and wrist postures of 20 manufacturing employees was conducted by two raters observing digital video files using Multimedia Video Task Analysis (MVTA). Generalizability theory was used to estimate the inter- and intra-rater reliability. The results demonstrated good to excellent inter-rater reliability for neck and shoulder postures and fair to excellent inter-rater reliability for wrist postures. Intra-rater posture assessment demonstrated good to excellent reliability for both raters in all postures of the neck, shoulder, and wrist. This study demonstrated that posture assessment of manufacturing workers using MVTA is a reliable method.     

Perception and visualization of human posture information for computer-aided ergonomic analysis

This article reports three experiments that examined the effects of photographic method, computerized visualization scheme, and posture complexity on posture perception and specification for computer-aided ergonomic analysis. The subjects were presented with photographs of working postures, and were required to manipulate human forms generated by an ergonomics software program to match the postures in the photographs. The first experiment showed a clear advantage of using a three-dimensional (3-D) human form graphic with two photographs when complex, asymmetric postures were analysed. However, the use of a 3-D human graphic display and two photographs jeopardized the subjects' posture specification performance when simple, symmetric postures were analysed. The results of the second and the third experiment demonstrated the importance of achieving congruency between photographic and computer display perspectives in improving posture specification performance. Implications for ergonomic job analysis and ergonomics software design are discussed.     

Development of a fully automated RULA assessment system based on computer vision

The purpose of this study was to develop an automated, RULA-based posture assessment system using a deep learning algorithm to estimate RULA scores, including scores for wrist posture, based on images of workplace postures. The proposed posture estimation system reported a mean absolute error (MAE) of 2.86 on the validation dataset obtained by randomly splitting 20% of the original training dataset before data augmentation. The results of the proposed system were compared with those of two experts’ manual evaluation by computing the intraclass correlation coefficient (ICC), which yielded index values greater than 0.75, thereby confirming good agreement between manual raters and the proposed system. This system will reduce the time required for postural evaluation while producing highly reliable RULA scores that are consistent with those generated by manual approach. Thus, we expect that this study will aid ergonomic experts in conducting RULA-based surveys of occupational postures in workplace conditions.     

Long-duration assessment of upper arm posture and motion and their association with perceived symptoms among bakery workers

Awkward shoulder postures have been suggested to be a cause of shoulder discomfort among bakery workers. This study aimed at long-duration assessment of upper arms posture and motion and their association with perceived symptoms among bakery workers. Among workers in three types of bread systems, fifty-seven bakers were randomly selected. The self-reported rates for the perceived severity and frequency of shoulder discomfort were collected through questionnaires. Working postures and movements of the shoulders during work were continuously recorded with inclinometry measurements for three hours. Percentage of time spent with the upper arm right elevated more than 60° was significantly correlated with the perceived discomfort rate in the right shoulder for all bakery workers (r = 0.48 to 0.63, p ≤ 0.05). A significant correlation was found between angular velocity with the perceived discomfort intensity for right upper arm.(r = 0.60 to 0.87, ≤0.005) of all workers in the three baking systems. Physical exposure in bakers was characterized by significantly more awkward postures and the percentage of time spent with the upper arms elevated more than 60°. The current findings can facilitate more informed decisions with respect to both engineering (e.g.ergonomic workstation and tool design) and administration (e.g. work organization) control strategies.     

Assessing the ergonomic exposures for drywall workers

Objective

The study was conducted to assess the ergonomic exposures to risk factors that may lead to the reported musculoskeletal injuries (especially back, neck and wrist injuries) of drywall workers.

Methods

A hierarchical taxonomy for construction of drywall panel hanging (drywall panel fitting and installation) was developed with activities defined within the interior wall systems tasks (drywall panel, studs and insulation). Exposures were characterized for the drywall panel work with the PATH (Posture, Activity, Tools, and Handling) work-sampling observation method. Data on working postures were collected for three main body parts: legs, arms and trunk. Activities performed for each task, tools used, and manually handled loads were also recorded for each observation.

Results

The study identified several ergonomic exposures in interior systems construction. Several risk factors were especially prevalent in the drywall panel installation task: awkward body postures such as overhead arm posture, trunk flexion, and handling of heavy drywall panels. Some tasks were observed to have combinations of these musculoskeletal risk factors, such as drywall panel installation, where the workers lifted heavy drywall panels in awkward body postures. In addition, a safety hazard frequently resulted when a worker's foot was poorly supported on a ladder while lifting heavy drywall panels to hang them on the ceiling or upper wall.

Conclusion

The drywall panel installation task poses a severe threat to the safety and musculoskeletal health of the drywall workers. Much of this could be eliminated by reducing the burden of handling heavy and bulky drywall panels.

Relevance to industry

The construction industry is well-documented to have high rates of injury and musculoskeletal disorders. Design of appropriate interventions requires specific knowledge of which tasks and activities involve the highest levels of exposure to relevant factors. Assessment of such factors in drywall panel hanging has provided data that will be useful to evaluate the ergonomics efficacy of future changes in task processes or tools. Feasible solutions appear to exist; effectiveness trials and worker input are needed in order to evaluate whether they could eliminate the observed exposures.     

The effect of viewing angle on wrist posture estimation from photographic images using novice raters

Observational assessment of wrist posture using photographic methods is theoretically affected by camera view angle. A study was conducted to investigate whether wrist flexion/extension and radial/ulnar deviation postures were estimated differently by raters depending on the viewing angle and compared to predictions using a quantitative 2D model of parallax. Novice raters (n = 26) estimated joint angles from images of wrist postures photographed from ten different viewing angles. Results indicated that ideal views, orthogonal to the plane of motion, produced more accurate estimates of posture compared to non-ideal views. The neutral (0°) posture was estimated the most accurately even at different viewing angles. Raters were more accurate than model predictions. Findings demonstrate a need for more systematic methods for collecting and analyzing photographic data for observational studies of posture. Renewed caution in interpreting existing studies of wrist posture where viewing angle was not controlled is advised.     

Ergonomic stress levels, personal characteristics, accident occurrence and sickness absence among factory workers

This study was designed to evaluate the association of a single, integrated measure of simultaneous exposure to a number of adverse work and environmental conditions, termed the Ergonomic Stress Level (E-S-L), on workers' accident and sickness absence rates. The factors determining the E-S-L were body motion and posture, physical effort, active hazards and environmental stressors. E-S-L evaluation was based on 'walk-through' hazard inventories, direct observations, measurements and interviews. Workers were assigned to one of four stress levels ranging from low (A) to high (D). Subjects were 729 males, aged 20-67 years, employed in five factories in Israel. A linear relationship between E-S-L AND accident incidence was found, increasing from the lowest to the highest E-S-L. Moreover, workers more sensitive to environmental stressors, as indicated by their reported subjective annoyance, showed higher accident rates across all the ergonomic stress levels, a tendency which was statistically significant at levels C and D. On the other hand, sickness absence was significantly related to the overall subjective stress experienced, as manifested by reported job dissatisfaction and somatic complaints, and not directly to E-S-L. These findings highlight the role of aggregate work stress, coupled with individual sensitivity to environmental stressors, in increasing the risk of accidents.     

A comparison of different postures for scaffold end-frame disassembly

Overexertion and fall injuries comprise the largest category of nonfatal injuries among scaffold workers. This study was conducted to identify the most favourable scaffold end-frame disassembly techniques and evaluate the associated slip potential by measuring whole-body isometric strength capability and required coefficient of friction (RCOF) to reduce the incidence of injury. Forty-six male construction workers were used to study seven typical postures associated with scaffold end-frame disassembly. An analysis of variance (ANOVA) showed that the isometric forces (334.4-676.3 N) resulting from the seven postures were significantly different (p < 0.05). Three of the disassembly postures resulted in considerable biomechanical stress to workers. The symmetric front-lift method with hand locations at knuckle height would be the most favourable posture; at least 93% of the male construction worker population could handle the end frame with minimum overexertion risk. The static RCOF value resulting from this posture during the disassembly phase was less than 0.2, thus the likelihood of a slip should be low.     

The influence of posture on power output generated in single pulling movements

The power generated in single pulling movements in seated and standing postures was measured by means of novel dynamometers. The power available is strongly dependent upon the resistance to movement, as well as the posture. Some of the results have been applied to an assessment of the matching of outboard motors to their users, illustrating an ergonomic value of the methodology.     

融合双目视觉和2D激光雷达的室外定位

高精度的定位对于自动驾驶至关重要. 2D激光雷达作为一种高精度的传感器被广泛应用于各种室内定位系统.然而在室外环境下,大量动态目标的存在使得相邻点云的匹配变得尤为困难,且2D激光雷达的点云数据存在稀疏性的问题,导致2D激光雷达在室外环境下的定位精度极低甚至无法实现定位.为此,提出一种融合双目视觉和2D激光雷达的室外定位算法.首先,利用双目视觉作为里程计提供相对位姿,将一个局部时间窗口内多个时刻得到的2D激光雷达数据融合成一个局部子图;然后,采用DS证据理论融合局部子图中的时态信息,以消除动态目标带来的噪声;最后,利用基于ICA的图像匹配方法将局部子图与预先构建的全局先验地图进行匹配,消除里程计的累积误差,实现高精度定位.在KITTI数据集上的实验结果表明,仅利用低成本的双目相机和2D激光雷达便可实现较高精度的定位,所提出算法的定位精度相比于ORB-SLAM2里程计最高可提升37.9%.