A digital twin-based multidisciplinary collaborative design approach for complex engineering product development

With the ever-increasing demand for personalized product functions, product structure becomes more and more complex. To design a complex engineering product, it involves mechanical, electrical, automation and other relevant fields, which requires a closer multidisciplinary collaborative design (MCD) and integration. However, the traditional design method lacks multidisciplinary coordination, which leads to interaction barriers between design stages and disconnection between product design and prototype manufacturing. To bridge the gap, a novel digital twin-enabled MCD approach is proposed. Firstly, the paper explores how to converge the MCD into the digital design process of complex engineering products in a cyber-physical system manner. The multidisciplinary collaborative design is divided into three parts: multidisciplinary knowledge collaboration, multidisciplinary collaborative modeling and multidisciplinary collaborative simulation, and the realization methods are proposed for each part. To be able to describe the complex product in a virtual environment, a systematic MCD framework based on the digital twin is further constructed. Integrate multidisciplinary collaboration into three stages: conceptual design, detailed design and virtual verification. The ability to verify and revise problems arising from multidisciplinary fusions in real-time minimizes the number of iterations and costs in the design process. Meanwhile, it provides a reference value for complex product design. Finally, a design case of an automatic cutting machine is conducted to reveal the feasibility and effectiveness of the proposed approach.     

Counterfactual inference to predict causal knowledge graph for relational transfer learning by assimilating expert knowledge --Relational feature transfer learning algorithm

Transfer learning (TL) is a machine learning (ML) method in which knowledge is transferred from the existing models of related problems to the model for solving the problem at hand. Relational TL enables the ML models to transfer the relationship networks from one domain to another. However, it has two critical issues. One is determining the proper way of extracting and expressing relationships among data features in the source domain such that the relationships can be transferred to the target domain. The other is how to do the transfer procedure. Knowledge graphs (KGs) are knowledge bases that use data and logic to graph-structured information; they are helpful tools for dealing with the first issue. The proposed relational feature transfer learning algorithm (RF-TL) embodies an extended structural equation modelling (SEM) as a method for constructing KGs. Additionally, in fields such as medicine, economics, and law related to people’s lives and property safety and security, the knowledge of domain experts is a gold standard. This paper introduces the causal analysis and counterfactual inference in the TL domain that directs the transfer procedure. Different from traditional feature-based TL algorithms like transfer component analysis (TCA) and CORelation Alignment (CORAL), RF-TL not only considers relations between feature items but also utilizes causality knowledge, enabling it to perform well in practical cases. The algorithm was tested on two different healthcare-related datasets — sleep apnea questionnaire study data and COVID-19 case data on ICU admission — and compared its performance with TCA and CORAL. The experimental results show that RF-TL can generate better transferred models that give more accurate predictions with fewer input features.     

A parameter-extended case-based reasoning method based on a functional basis for automated experiential reasoning in mechanical product designs

The mechanical product design process involves much experiential reasoning which relies extensively on accumulated experience knowledge and ambiguous synthetic decision of experts (ASDE). This makes it hard to achieve the automated, intelligent and rapid design of mechanical products. Furthermore, due to the lack of consideration of experts' cognition of product functions and structures in the application of the current case-based reasoning (CBR) method in the field of automated experiential reasoning (AER), the parameter solving process is separated from ASDE. Aiming at improving the accuracy and intelligence level of AER in mechanical product design, this paper proposed a parameter-extended CBR (PECBR) method based on a functional basis by integrating ASDE into AER. The PECBR method mainly contains two parts: firstly, in order to acquire and quantitatively describe expert experiential knowledge to provide an effective basis for AER, a knowledge representation method integrating a function-flow-parameter matrix set (FFP-MS) using functional bases and a parameter experiential correlation matrix (PEC-M) extracted from FFP-MS were presented for mechanical products, where the FFP-MS characterized the operation of function and energy flow during the working process of products. An acquisition rule for FFP-MS was designed to extract the degree of correlation between each two parameters, in which the implicit knowledge hiding among functions, flows and parameters was mined to form PEC-M; secondly, to cope with the difficulty in integrating ASDE into AER, a feature-weighted case adaptation (FCA) method was proposed by adopting a presented weighted kernel support vector machine (WK-SVM) and dynamic particle swarm optimization (DPSO). The FCA method can achieve the intelligent and automated solving of product parameters through identifying PEC-M during the case adaptation process. Two case studies on two-stage reducers and corn huskers were carried out to demonstrate the validity of the PECBR method. Compared with other conventional CBR methods, PECBR method can derive a more accurate value of parameters in mechanical product designs especially in the case of limited similar cases.     

Member combination selection for product collaborative design under the open innovation model

A temporary product collaborative design team (PCDT) formed by customers and candidate service providers is the main organization form required to complete the task of product collaborative design (PCD) under the open innovation model. Therefore, the aim of this study was to implement synergy effect-based member combination selection (SE-MCS) while ensuring customer participation in the PCD. First, the conceptual framework of SE-MCS method was developed to characterise the SE-MCS process that includes the customer. Second, SE-MCS indicators were determined by analysing the characteristics of PCD under the open innovation model, and the quantitative calculation methods for these indicators were provided. Subsequently, the mathematical model for SE-MCS considering customer participation was established, and a multi-objective optimisation algorithm was adopted to identify the optimal scheme. Finally, the formation of a design team for a beach waste collection vehicle was performed to verify the proposed method. The results showed that the proposed method is more suitable to implement SE-MCS of PCD under the open innovation model. It can facilitate the smooth operation of PCD tasks and improve the quality and efficiency of teamwork, thereby increasing customer satisfaction.     

A design method for an intelligent manufacturing and service system for rehabilitation assistive devices and special groups

The maturity of Industrial 4.0 technologies (smart wearable sensors, Internet of things [IoT], cloud computing, etc.) has facilitated the iteration and digitization of rehabilitation assistive devices (RADs) and the innovative development of intelligent manufacturing systems of RADs, expanding the value-added component of smart healthcare services. The intelligent manufacturing service mode, based on the concept of the product life cycle, completes the multi-source data production process analysis and the optimization of manufacturing, operation, and maintenance through intelligent industrial Internet of things and other means and improves the product life cycle management and operation mechanism. The smart product-service system (PSS) realizes the value-added of products by providing users with personalized products and value-added services, service efficiency, and sustainable development and gradually forms an Internet-product-service ecosystem. However, research on the PSS of RADs for special populations is relatively limited. Thus, this paper provides an overview of an IoT-based production model for RADs and a smart PSS-based development method of multimodal healthcare value-added services for special people. Taking the hand rehabilitation training devices for autistic children as a case, this paper verifies the effectiveness and availability of the proposed method. Compared with the traditional framework, the method used in this paper primarily helps evaluate rehabilitation efficacy, personalizes schemes for patients, provides auxiliary intelligent manufacturing service data and digital rehabilitation data for RAD manufacturers, and optimizes the product iteration development procedures by combining user-centered product interaction, multimodal evaluation, and value-added design. This study incorporates the iterative design of RADs into the process of smart PSS to provide some guidance to the RADs design manufacturers.     

A review of digital twin in product design and development

In the era of digitalization, there are many emerging technologies, such as the Internet of Things (IoT), Digital Twin (DT), Cloud Computing and Artificial Intelligence (AI), which are quickly developped and used in product design and development. Among those technologies, DT is one promising technology which has been widely used in different industries, especially manufacturing, to monitor the performance, optimize the progresses, simulate the results and predict the potential errors. DT also plays various roles within the whole product lifecycle from design, manufacturing, delivery, use and end-of-life. With the growing demands of individualized products and implementation of Industry 4.0, DT can provide an effective solution for future product design, development and innovation. This paper aims to figure out the current states of DT research focusing on product design and development through summarizing typical industrial cases. Challenges and potential applications of DT in product design and development are also discussed to inspire future studies.     

Convolutional long short-term memory model for recognizing construction workers’ postures from wearable inertial measurement units

This paper proposes using Deep Neural Networks (DNN) models for recognizing construction workers’ postures from motion data captured by wearable Inertial Measurement Units (IMUs) sensors. The recognized awkward postures can be linked to known risks of Musculoskeletal Disorders among workers. Applying conventional Machine Learning (ML)-based models has shown promising results in recognizing workers’ postures. ML models are limited – they reply on heuristic feature engineering when constructing discriminative features for characterizing postures. This makes further improving the model performance regarding recognition accuracy challenging. In this paper, the authors investigate the feasibility of addressing this problem using a DNN model that, through integrating Convolutional Neural Networks (CNN) with Long Short-Term Memory (LSTM) layers, automates feature engineering and sequential pattern detection. The model’s recognition performance was evaluated using datasets collected from four workers on construction sites. The DNN model integrating one convolutional and two LSTM layers resulted in the best performance (measured by F1 Score). The proposed model outperformed baseline CNN and LSTM models suggesting that it leveraged the advantages of the two baseline models for effective feature learning. It improved benchmark ML models’ recognition performance by an average of 11% under personalized modelling. The recognition performance was also improved by 3% when the proposed model was applied to 8 types of postures across three subjects. These results support that the proposed DNN model has a high potential in addressing challenges for improving the recognition performance that was observed when using ML models.     

EarlyData knowledge base for material decisions in building design

To make use of the great opportunities for emission reduction in early building design, future emissions need to be calculated when only geometric, but no detailed material information about a building is available. Currently, early design phase life cycle assessments (LCAs) are heavily reliant on assumptions of specific material choices, leading to single point emission values which suggest a precision not representative for an early design stage. By adding knowledge about possible locations and functions of materials within a building to life cycle inventory (LCI) data, the EarlyData knowledge base makes LCA data sets accessible and more transparent. Additionally, “generic building parts” are defined, which describe building parts independently of precise material choices as a combination of layers with specific functions. During evaluation, enriched LCI data and generic building parts enable assessment of a vast number of possible material combinations at once. Thus, instead of single value results for a particular material combination, ranges of results are displayed revealing the building parts with the greatest emission reduction potential. The application of the EarlyData tool is illustrated on a use case comparing a wood building and a concrete building. The database is developed with extensibility in mind, to include other criteria, such as (life cycle) costs.     

An enhanced 3D model and generative adversarial network for automated generation of horizontal building mask images and cloudless aerial photographs

Information extracted from aerial photographs is widely used in the fields of urban planning and design. An effective method for detecting buildings in aerial photographs is to use deep learning to understand the current state of a target region. However, the building mask images used to train the deep learning model must be manually generated in many cases. To overcome this challenge, a method has been proposed for automatically generating mask images by using textured three-dimensional (3D) virtual models with aerial photographs. Some aerial photographs include clouds, which degrade image quality. These clouds can be removed by using a generative adversarial network (GAN), which leads to improvements in training quality. Therefore, the objective of this research was to propose a method for automatically generating building mask images by using 3D virtual models with textured aerial photographs. In this study, using GAN to remove clouds in aerial photographs improved training quality. A model trained on datasets generated by the proposed method was able to detect buildings in aerial photographs with IoU = 0.651.     

The use of PCA and signal processing techniques for processing time-based construction settlement data of road embankments

Instrumentation is beneficial in civil engineering for monitoring structures during their construction and operation. The data collected can be used to observe real-time response and develop data-driven models for predicting future behaviour. However, a limited number of sensors are usually used for on-site civil engineering construction due to cost restrictions and practicalities. This results in relatively small raw datasets, which often contain errors and anomalies. Interpreting and making judicious use of the available dataset for developing reliable predictive model represents a significant challenge. Therefore, it is essential to pre-process and clean the data for improving their quality. To date, little investigation has been performed in the application of such data cleaning methods to geotechnical engineering datasets collected from full-scale sites. The purpose of this study is to apply simple and effective data pre-processing techniques to site-data collected from a highway embankment constructed on a sequence of soil layers of different physical make-up and non-linear consolidation characteristics. Various cleaning methods were applied to magnetic extensometer data collected for monitoring settlement within foundation soils beneath the embankment. PCA was used to explore raw data, identify and remove outliers. Numerous filtering and smoothing methods were used to clean noise in the data and their results were further compared using RMSE and NMSE. The methods adopted for data pre-processing and cleaning proved very effective for capturing the raw settlement behaviour on site. The findings from this study would be useful to site engineers regarding complex decision-making relating to ground response due to embankment construction. This also has positive prospects for developing dynamic prediction models for embankment settlement.     

Palette designer: A color-code design tool

The US Federal Aviation Administration (FAA) has developed a standard set of colors for coding information on air traffic control (ATC) displays. A significant complication was that the air traffic controller population includes people who have color-vision deficiencies (CVDs). We wrote a software tool to assist the FAA in selecting a preliminary color set. It accepts a set of luminances and chromaticity coordinates as input and: (1) Draws graphics and calculates color-related figures of merit to predict whether the set will be acceptable for color-normal and CVD users; (2) Flags colors and pairings that violate human factors criteria; and (3) Allows designers to adjust the colors and see the resulting changes immediately. The tool has been used to perform a pilot study for the FAA’s color-set development project and should be useful for designing other color-coding sets, also.     

An IIoT-driven and AI-enabled framework for smart manufacturing system based on three-terminal collaborative platform

Smart manufacturing has great potential in the development of network collaboration, mass personalised customisation, sustainability and flexibility. Customised production can better meet the dynamic user needs, and network collaboration can significantly improve production efficiency. Industrial internet of things (IIoT) and artificial intelligence (AI) have penetrated the manufacturing environment, improving production efficiency and facilitating customised and collaborative production. However, these technologies are isolated and dispersed in the applications of machine design and manufacturing processes. It is a challenge to integrate AI and IIoT technologies based on the platform, to develop autonomous connect manufacturing machines (ACMMs), matching with smart manufacturing and to facilitate the smart manufacturing services (SMSs) from the overall product life cycle. This paper firstly proposes a three-terminal collaborative platform (TTCP) consisting of cloud servers, embedded controllers and mobile terminals to integrate AI and IIoT technologies for the ACMM design. Then, based on the ACMMs, a framework for SMS to generate more IIoT-driven and AI-enabled services is presented. Finally, as an illustrative case, a more autonomous engraving machine and a smart manufacturing scenario are designed through the above-mentioned method. This case implements basic engraving functions along with AI-enabled automatic detection of broken tool service for collaborative production, remote human-machine interface service for customised production and network collaboration, and energy consumption analysis service for production optimisation. The systematic method proposed can provide some inspirations for the manufacturing industry to generate SMSs and facilitate the optimisation production and customised and collaborative production.     

Design of data feature-driven 1D/2D convolutional neural networks classifier for recycling black plastic wastes through laser spectroscopy

In this study, two types of convolutional neural network (CNN) classifiers are designed to handle the problem of classifying black plastic wastes. In particular, the black plastic wastes have the property of absorbing laser light coming from spectrometer. Therefore, the classification of black plastic wastes remains still a challenging problem compared to classifying other colored plastic wastes using existing spectroscopy (i.e., NIR). When it comes the classification problem of black plastic wastes, effective classification techniques by the laser spectroscopy of Fourier Transform-Infrared Radiation (FT-IR) with Attenuated Total Reflectance (ATR) and Raman to analyze the classification problem of black plastic wastes are introduced. Due to the strong ability of extracting spatial features and remarkable performance in image classification, 1D and 2D CNN through data features are designed as classifiers. The technique of chemical peak points selection is considered to reduce data redundancy. Furthermore, through the selection of data features based on the extracted 1D data with peak points is introduced. Experimental results demonstrate that 2DCNN classifier designed with the help of 2D data feature selection as well as 1DCNN classifier shows the best performance compared with other reported methods for classifying black plastic wastes.     

A rough set-based interval-valued intuitionistic fuzzy conceptual design decision approach with considering diverse customer preference distribution

Conceptual design evaluation plays a crucial role in new product development (NPD) and determines the quality of downstream design activities. Currently, most existing methods focus on fuzzy quantitative the evaluation information of multi-objectives in conceptual schemes selection. However, the above process ignores the various customers' preferences for each scheme under the evaluation objective, causing inconsistent preference weights in the various schemes, which cannot guarantee the market value of the optimal scheme. Furthermore, the ambiguous attitude from experts in the early design stage is not well taken into account. To this end, a conceptual scheme decision model with considering diverse customer preference distribution based on interval-valued intuitionistic fuzzy set (IVIFS) is proposed. The model is divided into three parts. Firstly, the initial decision matrix of multi-experts concerning the qualitative and quantitative design attributes is constructed based on intuitionistic fuzzy sets, and then the IFS decision matrix with interval boundaries is formed by using rough set technology. Secondly, the mapping model of design attribute to customer preference is constructed, and then the demand preference strategy implied by design attribute is judged. Thirdly, based on the demand preference strategy, the preferences’ weights for each scheme are calculated. Next, integrating the evaluation data with the same preference in the scheme, the comprehensive satisfaction of the scheme is obtained through IVIFS weighted aggregation operator, and then the optimal scheme is decided. Eventually, a case study of mobile phone form feature schemes is further employed to verify the proposed decision model, and results are sensitivity analyzed and compared.     

Energy consumption mode identification and monitoring method of process industry system under unstable working conditions

Process industry systems under unstable working conditions are prone to potential anomalies, deviating from the original transition trajectory, and taking longer than expected to return to stability due to persistent disturbances from uncertainties and experience-based regulation errors. The energy waste caused by this situation has not received sufficient attention, and cannot be addressed by existing energy consumption monitoring methods. Herein, an energy consumption mode (ECM) identification and monitoring method under unstable working conditions is proposed, consisting of ECM identification model and multi-mode dynamic monitoring model, focusing on the variation rules of the correlation between energy consumption and other states of the system. In the ECM identification stage, the ECM correlation parameters that reflect the comprehensive production information are selected. Then, given the transfer characteristics of ECM, a Hidden Semi-Markov Model (HSMM) is constructed to fit the migration between modes and the duration within modes. The Variational Bayesian Gaussian Mixture Model is introduced to improve the HSMM, which solves the problem of lacking prior knowledge of ECM and achieves the automatic classification and online identification of ECM. In the dynamic monitoring stage of multi-ECMs, a series of dynamic kernel principle component analysis models are established, and the corresponding monitoring thresholds are set for each ECM. By calculating the maximum of the posteriori probability and the mode thresholds, the ECMs under unstable conditions can be accurately identified and automatically monitored. Compared with previous methods, the proposed method reduces the false detection rate and missed detection rate of abnormal ECM identification to 1.04% and 1.31% in the actual slag grinding production process, which proves its effectiveness.     

A cooperative game theory based user-centered medical device design decision approach under uncertainty

The medical device conceptual design decision-making is a process of coordinating pertinent stakeholders, which will significantly affect the quality of follow-up market competitiveness. However, as the most challenging parts of user-centered design, traditional methods are mainly focusing on determining the priorities of the evaluation criteria and forming the comprehensive value (utility) of the conceptual scheme, may not fully deal with the interaction and interdependent between the conflicts of interest among stakeholders and weigh the ambiguous influence on the overall design expectations, which results in the unstable decision-making results. To overcome this drawback, this paper proposes a cooperative game theory based decision model for device conceptual scheme under uncertainty. The proposed approach consists of three parts: first part is to collect and classify needs of end users and professional users based on predefined evaluation criteria; second part is using rough set theory technique to create criteria correlation diagram and scheme value matrix from users; and third part is developing the fuzzy coalition utility model to maximize the overall desirability through the criteria correlation diagram with the conflict of interests of end and professional users considered, and then selecting the optimal scheme. A case study of blood pressure meter is used to illustrate the proposed approach and the result shows that this approach is more robust compared with the widely used the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) approach.     

Discovery and prediction capabilities in metal-based nanomaterials: An overview of the application of machine learning techniques and some recent advances

The application of machine learning (ML) techniques to metal-based nanomaterials has contributed greatly to understanding the interaction of nanoparticles, properties prediction, and new materials discovery. However, the prediction accuracy and efficiency of distinctive ML algorithms differ with different metal-based nanomaterials problems. This, alongside the high dimensionality and nonlinearity of available datasets in metal-based nanomaterials problems, makes it imperative to review recent advances in the implementation of ML techniques for these kinds of problems. In addition to understanding the applicability of different ML algorithms to various kinds of metal-based nanomaterials problems, it is hoped that this work will help facilitate understanding and promote interest in this emerging and less explored area of materials informatics. The scope of this review covers the introduction of metal-based nanomaterials, several techniques used in generating datasets for training ML models, feature engineering techniques used in nanomaterials-machine learning applications, and commonly applied ML algorithms. Then, we present the recent advances in ML applications to metal-based nanomaterials, with emphasis on the procedure and efficiency of algorithms used for such applications. In the concluding section, we identify the most common and efficient algorithms for distinctive property predictions. The common problems encountered in ML applications for metal-based nanoinformatics were mentioned. Finally, we propose suitable solutions and future outlooks for various challenges in metal-based nanoinformatics research.     

Assessing future landscapes using enhanced mixed reality with semantic segmentation by deep learning

Architecture, engineering, and construction projects need to be promoted in harmony with the natural environment and with the aim of preserving people’s living environment. At the planning and design stage, decision-makers and stakeholders share and assess landscape images during and after construction in order to avoid as much uncertainty as possible when performing environmental impact assessment. Given the lack of a standard visualization method for future landscapes that do not yet exist, mixed reality (MR), which overlays virtual content onto a real scene, has attracted attention in the field of landscape design. One challenge in MR is occlusion, which occurs when virtual objects obscure physical objects that should be rendered in the foreground. In MR-based landscape visualization, the distance between the MR camera and real objects located in front of the virtual objects might vary and might be large, causing difficulty for existing occlusion handling methods. In the process of landscape design, an evidence-based approach has also become important. Landscape index estimation using semantic segmentation by deep learning, which can recognize the surrounding environment, has been actively studied for landscape assessment. In this study, semantic segmentation by deep learning was integrated into an MR system to enable dynamic occlusion handling and landscape index estimation for both existing and designed landscape assessment. This system can be operated on a mobile device with video communication over the internet by connecting to real-time semantic segmentation on a high-performance personal computer. The applicability of the developed system is demonstrated through accuracy verification and case studies.