Towards a customizable immersive virtual reality serious game for earthquake emergency training

Earthquake emergencies require a variety of behavioral responses in order to ensure the safety of occupants, which is different from simply exiting a building in fire emergencies. This makes it more complex to train building occupants in order to acquire skills that align to best practices for immediate earthquake response and post-earthquake evacuation. In recent years, Immersive Virtual Reality (IVR) and Serious Games (SGs) have become popular as training tools for earthquake emergencies. IVR SGs have been introduced to train individuals for specific building layouts or settings with fixed training objectives. However, the lack of flexibility in existing IVR SGs makes it challenging to have widespread uptake as trainees require different training objectives, pedagogical strategies, context, and content. As a result, the effectiveness of IVR SGs training is jeopardized if the customization ability is limited. To overcome this limitation, this paper presents a customization framework for IVR SGs suited to earthquake emergency training, using the concept of adaptive game-based learning. Trainees can receive training in context by customizing virtual environments, storylines, and teaching methods. A case study was undertaken to validate the proposed framework. Results showed the potential to carry out the customization process with ease, to generate a customized training experience, and to deliver the customized training for optimum learning.     

Engineering knowledge formalization and proposition for informatics development towards a CAD-integrated DfX system for product design

Target design methodologies (DfX) were developed to cope with specific engineering design issues such as cost-effectiveness, manufacturability, assemblability, maintainability, among others. However, DfX methodologies are undergoing the lack of real integration with 3D CAD systems. Their principles are currently applied downstream of the 3D modelling by following the well-known rules available from the literature and engineers’ know-how (tacit internal knowledge).This paper provides a method to formalize complex DfX engineering knowledge into explicit knowledge that can be reused for Advanced Engineering Informatics to aid designers and engineers in developing mechanical products. This research work wants to define a general method (ontology) able to couple DfX design guidelines (engineering knowledge) with geometrical product features of a product 3D model (engineering parametric data). A common layer for all DfX methods (horizontal) and dedicated layers for each DfX method (vertical) allow creating the suitable ontology for the systematic collection of the DfX rules considering each target. Moreover, the proposed framework is the first step for developing (future work) a software tool to assist engineers and designers during product development (3D CAD modelling).A design for assembly (DfA) case study shows how to collect assembly rules in the given framework. It demonstrates the applicability of the CAD-integrated DfX system in the mechanical design of a jig-crane. Several benefits are recognized: (i) systematic collection of DfA rules for informatics development, (ii) identification of assembly issues in the product development process, and (iii) reduction of effort and time during the design review.     

Counterfactual inference to predict causal knowledge graph for relational transfer learning by assimilating expert knowledge --Relational feature transfer learning algorithm

Transfer learning (TL) is a machine learning (ML) method in which knowledge is transferred from the existing models of related problems to the model for solving the problem at hand. Relational TL enables the ML models to transfer the relationship networks from one domain to another. However, it has two critical issues. One is determining the proper way of extracting and expressing relationships among data features in the source domain such that the relationships can be transferred to the target domain. The other is how to do the transfer procedure. Knowledge graphs (KGs) are knowledge bases that use data and logic to graph-structured information; they are helpful tools for dealing with the first issue. The proposed relational feature transfer learning algorithm (RF-TL) embodies an extended structural equation modelling (SEM) as a method for constructing KGs. Additionally, in fields such as medicine, economics, and law related to people’s lives and property safety and security, the knowledge of domain experts is a gold standard. This paper introduces the causal analysis and counterfactual inference in the TL domain that directs the transfer procedure. Different from traditional feature-based TL algorithms like transfer component analysis (TCA) and CORelation Alignment (CORAL), RF-TL not only considers relations between feature items but also utilizes causality knowledge, enabling it to perform well in practical cases. The algorithm was tested on two different healthcare-related datasets — sleep apnea questionnaire study data and COVID-19 case data on ICU admission — and compared its performance with TCA and CORAL. The experimental results show that RF-TL can generate better transferred models that give more accurate predictions with fewer input features.     

Production logistics digital twins: Research profiling,application, challenges and opportunities

In the era of Industry 4.0, Production Logistic Digital Twins (PLDTs) have garnered remarkable attention from both academic and industrial communities. This is evident from the growing number of research publications on PLDTs in international scientific journals and conferences. However, given the diversity and complexity of production logistics activities, there is a pressing need for systematic literature review to chart past research and identify potential directions for future endeavors. Therefore, this study primarily focuses on the application of Digital Twins (DTs) in Production Logistics (PL). Firstly, an analysis of PLDTs research profiling is carried out based on general trends, keywords, application scenarios, and basic functions. Secondly, the functional characteristics of PLDTs are examined while summarizing their advantages and limitations across various application scenarios such as transportation, packaging, warehousing, material distribution, and information processing. And the roles played by smart technologies such as Internet of Things (IoT) in PLDTs system are discussed. Finally, possible challenges and future directions of PLDTs in industrial application are presented, accompanied by appropriate classification and extensive recommendations.     

EarlyData knowledge base for material decisions in building design

To make use of the great opportunities for emission reduction in early building design, future emissions need to be calculated when only geometric, but no detailed material information about a building is available. Currently, early design phase life cycle assessments (LCAs) are heavily reliant on assumptions of specific material choices, leading to single point emission values which suggest a precision not representative for an early design stage. By adding knowledge about possible locations and functions of materials within a building to life cycle inventory (LCI) data, the EarlyData knowledge base makes LCA data sets accessible and more transparent. Additionally, “generic building parts” are defined, which describe building parts independently of precise material choices as a combination of layers with specific functions. During evaluation, enriched LCI data and generic building parts enable assessment of a vast number of possible material combinations at once. Thus, instead of single value results for a particular material combination, ranges of results are displayed revealing the building parts with the greatest emission reduction potential. The application of the EarlyData tool is illustrated on a use case comparing a wood building and a concrete building. The database is developed with extensibility in mind, to include other criteria, such as (life cycle) costs.     

Non-invasive embedded system hardware/firmware anomaly detection based on the electric current signature

Quality control is a critical aspect of the modern electronic circuit industry. In addition to being a pre-requisite to proper functioning, circuit quality is closely related to safety, security, and economic issues. Quality control has been reached through system testing. Meanwhile, device miniaturization and multilayer Printed Circuit Boards have increased the electronic circuit test complexity considerably. Hence, traditional test processes based on manual inspections have become outdated and inefficient. More recently, the concept of Advanced Manufacturing or Industry 4.0 has enabled the manufacturing of customized products, tailored to the changing customers’ demands. This scenario points out additional requirements for electronic system testing: it demands a high degree of flexibility in production processes, short design and manufacturing cycles, and cost control. Thus, there is a demand for circuit testing systems that present effectiveness and accessibility without placing numerous test points. This work is focused on automated test solutions based on machine learning, which are becoming popular with advances in computational tools. We present a new testing approach that uses autoencoders to detect firmware or hardware anomalies based on the electric current signature. We built a test set-up using an embedded system development board to evaluate the proposed approach. We implemented six firmware versions that can run independently on the test board – one of them is considered anomaly-free. In order to obtain a reference frame to our results, two other classification techniques (a computer vision algorithm and a random forest classification model) were employed to detect anomalies on the same development board. The outcomes of the experiments demonstrated that the proposed test method is highly effective. For several test scenarios, the correct detection rate was above 99%. Test results showed that autoencoder and random forest approaches are effective. However, random forests require all data classes to be trained. Training an autoencoder, on the other hand, only requires the reference (anomaly-free) class.     

Design of data feature-driven 1D/2D convolutional neural networks classifier for recycling black plastic wastes through laser spectroscopy

In this study, two types of convolutional neural network (CNN) classifiers are designed to handle the problem of classifying black plastic wastes. In particular, the black plastic wastes have the property of absorbing laser light coming from spectrometer. Therefore, the classification of black plastic wastes remains still a challenging problem compared to classifying other colored plastic wastes using existing spectroscopy (i.e., NIR). When it comes the classification problem of black plastic wastes, effective classification techniques by the laser spectroscopy of Fourier Transform-Infrared Radiation (FT-IR) with Attenuated Total Reflectance (ATR) and Raman to analyze the classification problem of black plastic wastes are introduced. Due to the strong ability of extracting spatial features and remarkable performance in image classification, 1D and 2D CNN through data features are designed as classifiers. The technique of chemical peak points selection is considered to reduce data redundancy. Furthermore, through the selection of data features based on the extracted 1D data with peak points is introduced. Experimental results demonstrate that 2DCNN classifier designed with the help of 2D data feature selection as well as 1DCNN classifier shows the best performance compared with other reported methods for classifying black plastic wastes.     

A digital twin-based multidisciplinary collaborative design approach for complex engineering product development

With the ever-increasing demand for personalized product functions, product structure becomes more and more complex. To design a complex engineering product, it involves mechanical, electrical, automation and other relevant fields, which requires a closer multidisciplinary collaborative design (MCD) and integration. However, the traditional design method lacks multidisciplinary coordination, which leads to interaction barriers between design stages and disconnection between product design and prototype manufacturing. To bridge the gap, a novel digital twin-enabled MCD approach is proposed. Firstly, the paper explores how to converge the MCD into the digital design process of complex engineering products in a cyber-physical system manner. The multidisciplinary collaborative design is divided into three parts: multidisciplinary knowledge collaboration, multidisciplinary collaborative modeling and multidisciplinary collaborative simulation, and the realization methods are proposed for each part. To be able to describe the complex product in a virtual environment, a systematic MCD framework based on the digital twin is further constructed. Integrate multidisciplinary collaboration into three stages: conceptual design, detailed design and virtual verification. The ability to verify and revise problems arising from multidisciplinary fusions in real-time minimizes the number of iterations and costs in the design process. Meanwhile, it provides a reference value for complex product design. Finally, a design case of an automatic cutting machine is conducted to reveal the feasibility and effectiveness of the proposed approach.     

Discovery and prediction capabilities in metal-based nanomaterials: An overview of the application of machine learning techniques and some recent advances

The application of machine learning (ML) techniques to metal-based nanomaterials has contributed greatly to understanding the interaction of nanoparticles, properties prediction, and new materials discovery. However, the prediction accuracy and efficiency of distinctive ML algorithms differ with different metal-based nanomaterials problems. This, alongside the high dimensionality and nonlinearity of available datasets in metal-based nanomaterials problems, makes it imperative to review recent advances in the implementation of ML techniques for these kinds of problems. In addition to understanding the applicability of different ML algorithms to various kinds of metal-based nanomaterials problems, it is hoped that this work will help facilitate understanding and promote interest in this emerging and less explored area of materials informatics. The scope of this review covers the introduction of metal-based nanomaterials, several techniques used in generating datasets for training ML models, feature engineering techniques used in nanomaterials-machine learning applications, and commonly applied ML algorithms. Then, we present the recent advances in ML applications to metal-based nanomaterials, with emphasis on the procedure and efficiency of algorithms used for such applications. In the concluding section, we identify the most common and efficient algorithms for distinctive property predictions. The common problems encountered in ML applications for metal-based nanoinformatics were mentioned. Finally, we propose suitable solutions and future outlooks for various challenges in metal-based nanoinformatics research.     

A design method for an intelligent manufacturing and service system for rehabilitation assistive devices and special groups

The maturity of Industrial 4.0 technologies (smart wearable sensors, Internet of things [IoT], cloud computing, etc.) has facilitated the iteration and digitization of rehabilitation assistive devices (RADs) and the innovative development of intelligent manufacturing systems of RADs, expanding the value-added component of smart healthcare services. The intelligent manufacturing service mode, based on the concept of the product life cycle, completes the multi-source data production process analysis and the optimization of manufacturing, operation, and maintenance through intelligent industrial Internet of things and other means and improves the product life cycle management and operation mechanism. The smart product-service system (PSS) realizes the value-added of products by providing users with personalized products and value-added services, service efficiency, and sustainable development and gradually forms an Internet-product-service ecosystem. However, research on the PSS of RADs for special populations is relatively limited. Thus, this paper provides an overview of an IoT-based production model for RADs and a smart PSS-based development method of multimodal healthcare value-added services for special people. Taking the hand rehabilitation training devices for autistic children as a case, this paper verifies the effectiveness and availability of the proposed method. Compared with the traditional framework, the method used in this paper primarily helps evaluate rehabilitation efficacy, personalizes schemes for patients, provides auxiliary intelligent manufacturing service data and digital rehabilitation data for RAD manufacturers, and optimizes the product iteration development procedures by combining user-centered product interaction, multimodal evaluation, and value-added design. This study incorporates the iterative design of RADs into the process of smart PSS to provide some guidance to the RADs design manufacturers.     

Energy consumption mode identification and monitoring method of process industry system under unstable working conditions

Process industry systems under unstable working conditions are prone to potential anomalies, deviating from the original transition trajectory, and taking longer than expected to return to stability due to persistent disturbances from uncertainties and experience-based regulation errors. The energy waste caused by this situation has not received sufficient attention, and cannot be addressed by existing energy consumption monitoring methods. Herein, an energy consumption mode (ECM) identification and monitoring method under unstable working conditions is proposed, consisting of ECM identification model and multi-mode dynamic monitoring model, focusing on the variation rules of the correlation between energy consumption and other states of the system. In the ECM identification stage, the ECM correlation parameters that reflect the comprehensive production information are selected. Then, given the transfer characteristics of ECM, a Hidden Semi-Markov Model (HSMM) is constructed to fit the migration between modes and the duration within modes. The Variational Bayesian Gaussian Mixture Model is introduced to improve the HSMM, which solves the problem of lacking prior knowledge of ECM and achieves the automatic classification and online identification of ECM. In the dynamic monitoring stage of multi-ECMs, a series of dynamic kernel principle component analysis models are established, and the corresponding monitoring thresholds are set for each ECM. By calculating the maximum of the posteriori probability and the mode thresholds, the ECMs under unstable conditions can be accurately identified and automatically monitored. Compared with previous methods, the proposed method reduces the false detection rate and missed detection rate of abnormal ECM identification to 1.04% and 1.31% in the actual slag grinding production process, which proves its effectiveness.     

An enhanced 3D model and generative adversarial network for automated generation of horizontal building mask images and cloudless aerial photographs

Information extracted from aerial photographs is widely used in the fields of urban planning and design. An effective method for detecting buildings in aerial photographs is to use deep learning to understand the current state of a target region. However, the building mask images used to train the deep learning model must be manually generated in many cases. To overcome this challenge, a method has been proposed for automatically generating mask images by using textured three-dimensional (3D) virtual models with aerial photographs. Some aerial photographs include clouds, which degrade image quality. These clouds can be removed by using a generative adversarial network (GAN), which leads to improvements in training quality. Therefore, the objective of this research was to propose a method for automatically generating building mask images by using 3D virtual models with textured aerial photographs. In this study, using GAN to remove clouds in aerial photographs improved training quality. A model trained on datasets generated by the proposed method was able to detect buildings in aerial photographs with IoU = 0.651.     

A parameter-extended case-based reasoning method based on a functional basis for automated experiential reasoning in mechanical product designs

The mechanical product design process involves much experiential reasoning which relies extensively on accumulated experience knowledge and ambiguous synthetic decision of experts (ASDE). This makes it hard to achieve the automated, intelligent and rapid design of mechanical products. Furthermore, due to the lack of consideration of experts' cognition of product functions and structures in the application of the current case-based reasoning (CBR) method in the field of automated experiential reasoning (AER), the parameter solving process is separated from ASDE. Aiming at improving the accuracy and intelligence level of AER in mechanical product design, this paper proposed a parameter-extended CBR (PECBR) method based on a functional basis by integrating ASDE into AER. The PECBR method mainly contains two parts: firstly, in order to acquire and quantitatively describe expert experiential knowledge to provide an effective basis for AER, a knowledge representation method integrating a function-flow-parameter matrix set (FFP-MS) using functional bases and a parameter experiential correlation matrix (PEC-M) extracted from FFP-MS were presented for mechanical products, where the FFP-MS characterized the operation of function and energy flow during the working process of products. An acquisition rule for FFP-MS was designed to extract the degree of correlation between each two parameters, in which the implicit knowledge hiding among functions, flows and parameters was mined to form PEC-M; secondly, to cope with the difficulty in integrating ASDE into AER, a feature-weighted case adaptation (FCA) method was proposed by adopting a presented weighted kernel support vector machine (WK-SVM) and dynamic particle swarm optimization (DPSO). The FCA method can achieve the intelligent and automated solving of product parameters through identifying PEC-M during the case adaptation process. Two case studies on two-stage reducers and corn huskers were carried out to demonstrate the validity of the PECBR method. Compared with other conventional CBR methods, PECBR method can derive a more accurate value of parameters in mechanical product designs especially in the case of limited similar cases.     

Instructional mechanisms in immersive virtual reality serious games: Earthquake emergency training for children

Children are vulnerable in earthquakes, but they are also essential to foster earthquake-resilient communities. It is critical to enhance the preparedness of children against earthquakes through effective education and training. Immersive virtual reality (IVR) and serious games (SGs) are innovative digital technologies that enable realistic and engaging training environments. However, little research has been made on the applications of IVR SGs for earthquake training targeting children. In order to fill this gap, this paper presents an IVR SG training system based on a problem-based gaming framework. Three instructional mechanisms within the training system, namely prior instruction, immediate feedback, and post-game assessment, were investigated to promote learning through effective reflection. A controlled experiment involving 125 secondary school students aged from 11 to 15 years old was undertaken, using leaflets as a traditional training approach for the control group and the IVR SG training system as the main intervention. Results revealed that the IVR SG training system with post-game assessment was the most effective way to train children, with greater knowledge acquisition and self-efficacy improvement observed. Possible improvements, such as increasing the time for reflection and differentiating the stimulation between positive and negative feedback, are suggested for further research.     

Convolutional long short-term memory model for recognizing construction workers’ postures from wearable inertial measurement units

This paper proposes using Deep Neural Networks (DNN) models for recognizing construction workers’ postures from motion data captured by wearable Inertial Measurement Units (IMUs) sensors. The recognized awkward postures can be linked to known risks of Musculoskeletal Disorders among workers. Applying conventional Machine Learning (ML)-based models has shown promising results in recognizing workers’ postures. ML models are limited – they reply on heuristic feature engineering when constructing discriminative features for characterizing postures. This makes further improving the model performance regarding recognition accuracy challenging. In this paper, the authors investigate the feasibility of addressing this problem using a DNN model that, through integrating Convolutional Neural Networks (CNN) with Long Short-Term Memory (LSTM) layers, automates feature engineering and sequential pattern detection. The model’s recognition performance was evaluated using datasets collected from four workers on construction sites. The DNN model integrating one convolutional and two LSTM layers resulted in the best performance (measured by F1 Score). The proposed model outperformed baseline CNN and LSTM models suggesting that it leveraged the advantages of the two baseline models for effective feature learning. It improved benchmark ML models’ recognition performance by an average of 11% under personalized modelling. The recognition performance was also improved by 3% when the proposed model was applied to 8 types of postures across three subjects. These results support that the proposed DNN model has a high potential in addressing challenges for improving the recognition performance that was observed when using ML models.     

Risk assessment of China-Pakistan Fiber Optic Project (CPFOP) in the light of Multi-Criteria Decision Making (MCDM)

The China-Pakistan Economic Corridor (CPEC) is considered as an excellent breakthrough for improving the economic and security situation in the region. The estimated worth of CPEC is 62$ billion which is comprising of 49 developmental projects. China-Pakistan Fiber Optic Project (CPFOP) is one of the core projects among these, which will deliver safe route of voice traffic between both countries. CPFOP is greatly beneficial in terms of enhanced security and revenue generation. Currently, Pakistan’s international connectivity is via submarine cables. CPFOP will provide an alternative route for international telecom traffic and also assist in achieving the rapidly growing internet traffic demand in Pakistan. It is estimated that 17 million people will get benefit from this project. However, every project has some undesirable impacts. The aim of this research paper is twofold; 1st to trace out the pros and cons of CPFOP. 2ndly, performing a risk assessment of CPFOP by using Fuzzy VIKOR technique. This approach will help in prioritizing a list of failure modes of Fiber Optic Cable (FOC). Lastly, this paper will help authorities for optimizing and safeguarding national interest in the wake of CPFOP.     

Novel thoraco-lumbo-sacral corset design increases Biering-Sorensen back endurance and alters knee and ankle angles during a box lifting task

Research investigating lumbosacral corset designs and their effects are limited and conflicting. The objective was to compare thoraco-lumbo-sacral support corsets (polyester/nylon: TLSSC-poly and neoprene: TLSSC-neo) with a traditional model (TRAD) and Control. Twenty male, university-aged, healthy, recreationally active, participants performed Biering-Sorensen back endurance (BS) test and box lifting tasks (BL:30 repetitions using 20% body mass). Lower and upper erector spinae and hamstrings electromyography (EMG); trunk-hip, knee, and ankle kinematics as well as endurance time were monitored. With BL, the TLSSC-poly (121.4°±17.9) exhibited 1.9% (p = 0.01), 2.7% (p = 0.003), and 3.7% (p = 0.0003) greater knee flexion than TRAD (119.1°±17.5), TLSSC-neo (116.8°±17.4) and Control (120.1°±17.6) respectively. The TLSSC-poly (101.9°± 8.9) demonstrated significant 3.5% (p = 0.005), 2.2% (p = 0.002) and 1.4% (p = 0.01) greater dorsiflexion than TRAD (103.4°±8.7), TLSSC-neo (104.2°±9.8) and Control (105.7°±7.2) respectively. With BS, TLSSC-poly (137.4-s±31.2, 9.7%, p = 0.018) and TLSSC-neo (133.8-s±32.3, 9.2%, p = 0.006) exhibited significantly longer durations than Control (124.8-s±29.8). Relevance to industry: The TLSSC increased BS endurance and TLSSC-poly increased BL knee and ankle angles, possibly providing benefits for workers, with repeated actions over a full work day.