An IIoT-driven and AI-enabled framework for smart manufacturing system based on three-terminal collaborative platform

Smart manufacturing has great potential in the development of network collaboration, mass personalised customisation, sustainability and flexibility. Customised production can better meet the dynamic user needs, and network collaboration can significantly improve production efficiency. Industrial internet of things (IIoT) and artificial intelligence (AI) have penetrated the manufacturing environment, improving production efficiency and facilitating customised and collaborative production. However, these technologies are isolated and dispersed in the applications of machine design and manufacturing processes. It is a challenge to integrate AI and IIoT technologies based on the platform, to develop autonomous connect manufacturing machines (ACMMs), matching with smart manufacturing and to facilitate the smart manufacturing services (SMSs) from the overall product life cycle. This paper firstly proposes a three-terminal collaborative platform (TTCP) consisting of cloud servers, embedded controllers and mobile terminals to integrate AI and IIoT technologies for the ACMM design. Then, based on the ACMMs, a framework for SMS to generate more IIoT-driven and AI-enabled services is presented. Finally, as an illustrative case, a more autonomous engraving machine and a smart manufacturing scenario are designed through the above-mentioned method. This case implements basic engraving functions along with AI-enabled automatic detection of broken tool service for collaborative production, remote human-machine interface service for customised production and network collaboration, and energy consumption analysis service for production optimisation. The systematic method proposed can provide some inspirations for the manufacturing industry to generate SMSs and facilitate the optimisation production and customised and collaborative production.     

Engineering knowledge formalization and proposition for informatics development towards a CAD-integrated DfX system for product design

Target design methodologies (DfX) were developed to cope with specific engineering design issues such as cost-effectiveness, manufacturability, assemblability, maintainability, among others. However, DfX methodologies are undergoing the lack of real integration with 3D CAD systems. Their principles are currently applied downstream of the 3D modelling by following the well-known rules available from the literature and engineers’ know-how (tacit internal knowledge).This paper provides a method to formalize complex DfX engineering knowledge into explicit knowledge that can be reused for Advanced Engineering Informatics to aid designers and engineers in developing mechanical products. This research work wants to define a general method (ontology) able to couple DfX design guidelines (engineering knowledge) with geometrical product features of a product 3D model (engineering parametric data). A common layer for all DfX methods (horizontal) and dedicated layers for each DfX method (vertical) allow creating the suitable ontology for the systematic collection of the DfX rules considering each target. Moreover, the proposed framework is the first step for developing (future work) a software tool to assist engineers and designers during product development (3D CAD modelling).A design for assembly (DfA) case study shows how to collect assembly rules in the given framework. It demonstrates the applicability of the CAD-integrated DfX system in the mechanical design of a jig-crane. Several benefits are recognized: (i) systematic collection of DfA rules for informatics development, (ii) identification of assembly issues in the product development process, and (iii) reduction of effort and time during the design review.     

Digital twin-driven clamping force control for thin-walled parts

Clamping quality is one of the main factors that will affect the deformation of thin-walled parts during their processing, which can then directly affect parts’ performance. However, traditional clamping force settings are based on manual experience, which is a random and inaccurate manner. In addition, dynamic clamping force adjustment according to clamping deformation is rarely considered in clamping force control process, which easily causes large clamping deformation and low machining accuracy. To address these issues, this study proposes a digital twin-driven clamping force control approach to improve the machining accuracy of thin-walled parts. The total factor information model of clamping system is built to integrate the dynamic information of the clamping process. The virtual space model is constructed based on finite element simulation and deep neural network algorithm. To ensure bidirectional mapping of physical-virtual space, the workflow of clamping force control and interoperability method between digital twin models are elaborated. Finally, a case study is used to verify the effectiveness and feasibility of the proposed method.     

Member combination selection for product collaborative design under the open innovation model

A temporary product collaborative design team (PCDT) formed by customers and candidate service providers is the main organization form required to complete the task of product collaborative design (PCD) under the open innovation model. Therefore, the aim of this study was to implement synergy effect-based member combination selection (SE-MCS) while ensuring customer participation in the PCD. First, the conceptual framework of SE-MCS method was developed to characterise the SE-MCS process that includes the customer. Second, SE-MCS indicators were determined by analysing the characteristics of PCD under the open innovation model, and the quantitative calculation methods for these indicators were provided. Subsequently, the mathematical model for SE-MCS considering customer participation was established, and a multi-objective optimisation algorithm was adopted to identify the optimal scheme. Finally, the formation of a design team for a beach waste collection vehicle was performed to verify the proposed method. The results showed that the proposed method is more suitable to implement SE-MCS of PCD under the open innovation model. It can facilitate the smooth operation of PCD tasks and improve the quality and efficiency of teamwork, thereby increasing customer satisfaction.     

A cooperative game theory based user-centered medical device design decision approach under uncertainty

The medical device conceptual design decision-making is a process of coordinating pertinent stakeholders, which will significantly affect the quality of follow-up market competitiveness. However, as the most challenging parts of user-centered design, traditional methods are mainly focusing on determining the priorities of the evaluation criteria and forming the comprehensive value (utility) of the conceptual scheme, may not fully deal with the interaction and interdependent between the conflicts of interest among stakeholders and weigh the ambiguous influence on the overall design expectations, which results in the unstable decision-making results. To overcome this drawback, this paper proposes a cooperative game theory based decision model for device conceptual scheme under uncertainty. The proposed approach consists of three parts: first part is to collect and classify needs of end users and professional users based on predefined evaluation criteria; second part is using rough set theory technique to create criteria correlation diagram and scheme value matrix from users; and third part is developing the fuzzy coalition utility model to maximize the overall desirability through the criteria correlation diagram with the conflict of interests of end and professional users considered, and then selecting the optimal scheme. A case study of blood pressure meter is used to illustrate the proposed approach and the result shows that this approach is more robust compared with the widely used the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) approach.     

Assessing future landscapes using enhanced mixed reality with semantic segmentation by deep learning

Architecture, engineering, and construction projects need to be promoted in harmony with the natural environment and with the aim of preserving people’s living environment. At the planning and design stage, decision-makers and stakeholders share and assess landscape images during and after construction in order to avoid as much uncertainty as possible when performing environmental impact assessment. Given the lack of a standard visualization method for future landscapes that do not yet exist, mixed reality (MR), which overlays virtual content onto a real scene, has attracted attention in the field of landscape design. One challenge in MR is occlusion, which occurs when virtual objects obscure physical objects that should be rendered in the foreground. In MR-based landscape visualization, the distance between the MR camera and real objects located in front of the virtual objects might vary and might be large, causing difficulty for existing occlusion handling methods. In the process of landscape design, an evidence-based approach has also become important. Landscape index estimation using semantic segmentation by deep learning, which can recognize the surrounding environment, has been actively studied for landscape assessment. In this study, semantic segmentation by deep learning was integrated into an MR system to enable dynamic occlusion handling and landscape index estimation for both existing and designed landscape assessment. This system can be operated on a mobile device with video communication over the internet by connecting to real-time semantic segmentation on a high-performance personal computer. The applicability of the developed system is demonstrated through accuracy verification and case studies.     

A rough set-based interval-valued intuitionistic fuzzy conceptual design decision approach with considering diverse customer preference distribution

Conceptual design evaluation plays a crucial role in new product development (NPD) and determines the quality of downstream design activities. Currently, most existing methods focus on fuzzy quantitative the evaluation information of multi-objectives in conceptual schemes selection. However, the above process ignores the various customers' preferences for each scheme under the evaluation objective, causing inconsistent preference weights in the various schemes, which cannot guarantee the market value of the optimal scheme. Furthermore, the ambiguous attitude from experts in the early design stage is not well taken into account. To this end, a conceptual scheme decision model with considering diverse customer preference distribution based on interval-valued intuitionistic fuzzy set (IVIFS) is proposed. The model is divided into three parts. Firstly, the initial decision matrix of multi-experts concerning the qualitative and quantitative design attributes is constructed based on intuitionistic fuzzy sets, and then the IFS decision matrix with interval boundaries is formed by using rough set technology. Secondly, the mapping model of design attribute to customer preference is constructed, and then the demand preference strategy implied by design attribute is judged. Thirdly, based on the demand preference strategy, the preferences’ weights for each scheme are calculated. Next, integrating the evaluation data with the same preference in the scheme, the comprehensive satisfaction of the scheme is obtained through IVIFS weighted aggregation operator, and then the optimal scheme is decided. Eventually, a case study of mobile phone form feature schemes is further employed to verify the proposed decision model, and results are sensitivity analyzed and compared.     

A design method for an intelligent manufacturing and service system for rehabilitation assistive devices and special groups

The maturity of Industrial 4.0 technologies (smart wearable sensors, Internet of things [IoT], cloud computing, etc.) has facilitated the iteration and digitization of rehabilitation assistive devices (RADs) and the innovative development of intelligent manufacturing systems of RADs, expanding the value-added component of smart healthcare services. The intelligent manufacturing service mode, based on the concept of the product life cycle, completes the multi-source data production process analysis and the optimization of manufacturing, operation, and maintenance through intelligent industrial Internet of things and other means and improves the product life cycle management and operation mechanism. The smart product-service system (PSS) realizes the value-added of products by providing users with personalized products and value-added services, service efficiency, and sustainable development and gradually forms an Internet-product-service ecosystem. However, research on the PSS of RADs for special populations is relatively limited. Thus, this paper provides an overview of an IoT-based production model for RADs and a smart PSS-based development method of multimodal healthcare value-added services for special people. Taking the hand rehabilitation training devices for autistic children as a case, this paper verifies the effectiveness and availability of the proposed method. Compared with the traditional framework, the method used in this paper primarily helps evaluate rehabilitation efficacy, personalizes schemes for patients, provides auxiliary intelligent manufacturing service data and digital rehabilitation data for RAD manufacturers, and optimizes the product iteration development procedures by combining user-centered product interaction, multimodal evaluation, and value-added design. This study incorporates the iterative design of RADs into the process of smart PSS to provide some guidance to the RADs design manufacturers.     

A parameter-extended case-based reasoning method based on a functional basis for automated experiential reasoning in mechanical product designs

The mechanical product design process involves much experiential reasoning which relies extensively on accumulated experience knowledge and ambiguous synthetic decision of experts (ASDE). This makes it hard to achieve the automated, intelligent and rapid design of mechanical products. Furthermore, due to the lack of consideration of experts' cognition of product functions and structures in the application of the current case-based reasoning (CBR) method in the field of automated experiential reasoning (AER), the parameter solving process is separated from ASDE. Aiming at improving the accuracy and intelligence level of AER in mechanical product design, this paper proposed a parameter-extended CBR (PECBR) method based on a functional basis by integrating ASDE into AER. The PECBR method mainly contains two parts: firstly, in order to acquire and quantitatively describe expert experiential knowledge to provide an effective basis for AER, a knowledge representation method integrating a function-flow-parameter matrix set (FFP-MS) using functional bases and a parameter experiential correlation matrix (PEC-M) extracted from FFP-MS were presented for mechanical products, where the FFP-MS characterized the operation of function and energy flow during the working process of products. An acquisition rule for FFP-MS was designed to extract the degree of correlation between each two parameters, in which the implicit knowledge hiding among functions, flows and parameters was mined to form PEC-M; secondly, to cope with the difficulty in integrating ASDE into AER, a feature-weighted case adaptation (FCA) method was proposed by adopting a presented weighted kernel support vector machine (WK-SVM) and dynamic particle swarm optimization (DPSO). The FCA method can achieve the intelligent and automated solving of product parameters through identifying PEC-M during the case adaptation process. Two case studies on two-stage reducers and corn huskers were carried out to demonstrate the validity of the PECBR method. Compared with other conventional CBR methods, PECBR method can derive a more accurate value of parameters in mechanical product designs especially in the case of limited similar cases.     

A three-stage multi-objective heterogeneous integrated model with decomposition-reconstruction mechanism and adaptive segmentation error correction method for ship motion multi-step prediction

Reliable and accurate ship motion prediction is essential for ship navigation at sea and marine operations. Although previous studies have yielded rich results in the field of ship motion prediction, most of them have ignored the importance of the dynamic characteristics of ship motion for constructing forecasting models. Besides, the limitations of the single model and the autocorrelation characteristics of the residual series are also unfavorable factors that hinder the forecasting performance. To fill these gaps, a multi-objective heterogeneous integration model based on decomposition-reconstruction mechanism and adaptive segmentation error correction method is proposed in this paper for ship motion multi-step prediction. Specifically, the proposed model is divided into three stages, which are decomposition-reconstruction mechanism, multi-objective heterogeneous integration model and adaptive segmentation error correction method. The effectiveness of the proposed model is verified using four sets of real ship motion data collected from two sites in the South China Sea. The evaluation results show that the proposed model can effectively improve the prediction performance and outperforms other traditional models and state-of-the-art models in the field of ship motion prediction. Prospectively, the model proposed in this study can be used as an effective aid to ship warning systems and has the potential for practical application in ship marine operations.     

An enhanced 3D model and generative adversarial network for automated generation of horizontal building mask images and cloudless aerial photographs

Information extracted from aerial photographs is widely used in the fields of urban planning and design. An effective method for detecting buildings in aerial photographs is to use deep learning to understand the current state of a target region. However, the building mask images used to train the deep learning model must be manually generated in many cases. To overcome this challenge, a method has been proposed for automatically generating mask images by using textured three-dimensional (3D) virtual models with aerial photographs. Some aerial photographs include clouds, which degrade image quality. These clouds can be removed by using a generative adversarial network (GAN), which leads to improvements in training quality. Therefore, the objective of this research was to propose a method for automatically generating building mask images by using 3D virtual models with textured aerial photographs. In this study, using GAN to remove clouds in aerial photographs improved training quality. A model trained on datasets generated by the proposed method was able to detect buildings in aerial photographs with IoU = 0.651.     

A review of digital twin in product design and development

In the era of digitalization, there are many emerging technologies, such as the Internet of Things (IoT), Digital Twin (DT), Cloud Computing and Artificial Intelligence (AI), which are quickly developped and used in product design and development. Among those technologies, DT is one promising technology which has been widely used in different industries, especially manufacturing, to monitor the performance, optimize the progresses, simulate the results and predict the potential errors. DT also plays various roles within the whole product lifecycle from design, manufacturing, delivery, use and end-of-life. With the growing demands of individualized products and implementation of Industry 4.0, DT can provide an effective solution for future product design, development and innovation. This paper aims to figure out the current states of DT research focusing on product design and development through summarizing typical industrial cases. Challenges and potential applications of DT in product design and development are also discussed to inspire future studies.     

Convolutional long short-term memory model for recognizing construction workers’ postures from wearable inertial measurement units

This paper proposes using Deep Neural Networks (DNN) models for recognizing construction workers’ postures from motion data captured by wearable Inertial Measurement Units (IMUs) sensors. The recognized awkward postures can be linked to known risks of Musculoskeletal Disorders among workers. Applying conventional Machine Learning (ML)-based models has shown promising results in recognizing workers’ postures. ML models are limited – they reply on heuristic feature engineering when constructing discriminative features for characterizing postures. This makes further improving the model performance regarding recognition accuracy challenging. In this paper, the authors investigate the feasibility of addressing this problem using a DNN model that, through integrating Convolutional Neural Networks (CNN) with Long Short-Term Memory (LSTM) layers, automates feature engineering and sequential pattern detection. The model’s recognition performance was evaluated using datasets collected from four workers on construction sites. The DNN model integrating one convolutional and two LSTM layers resulted in the best performance (measured by F1 Score). The proposed model outperformed baseline CNN and LSTM models suggesting that it leveraged the advantages of the two baseline models for effective feature learning. It improved benchmark ML models’ recognition performance by an average of 11% under personalized modelling. The recognition performance was also improved by 3% when the proposed model was applied to 8 types of postures across three subjects. These results support that the proposed DNN model has a high potential in addressing challenges for improving the recognition performance that was observed when using ML models.     

Ergonomics demands associated with combinations of manual and powered emergency medical service cots and ambulance loading systems: A work simulation study

Digital human models were used to perform a virtual ergonomics assessment of manual and powered emergency medical services cots combined with both manual and powered ambulance loading systems. Simulations were run with all combinations of emergency medical technicians (EMTs), at 50 kg (female), 72 kg (female) and 125 kg (male) with cots containing no patient and patients of 125 kg and 159 kg. There was a substantial decrease in low back and upper extremity demands with the use of a powered cot, and a further decrease with the additional use of a powered loading system, even though it only required one EMT. The benefits of a fully powered system were magnified with the simulation of both heavier EMTs and patients. Additionally, this study demonstrates the utility of digital human models and work simulation to evaluate product designs that impact occupational demands and injury risk.     

Conceptual design evaluation considering the ambiguity semantic variables fusion with conflict beliefs: An integrated Dempster-Shafer evidence theory and intuitionistic fuzzy -VIKOR

Quantifying the uncertain linguistic evaluation from decision-makers (DMs) is one of the most challenging parts in the conceptual design decision. Although fuzzy decision models have been widely used to capture potential uncertainty by assigning a fuzzy term with the certain belief, the ambiguity subjective evaluation of semantic variables with conflict beliefs derived from DMs have not been well addressed. To solve this drawback, a concept decision model based on Dempster-Shafer (DS) evidence theory and intuitionistic fuzzy -Vlsekriterijumska Optimizacija I Kompromisno Resenje (VIKOR) considering the ambiguity semantic variables fusion is proposed. Firstly, by incorporating semantic variables of intuitionistic fuzzy sets (IFSs), the diversified semantic judgments and its belief will be taken into account to form an ambiguity semantic initial decision matrix; secondly, the DS combination rule will be used to fuse the different semantic variables of multi-DMs in each scheme, update the belief of each semantic variable, and then the semantic fusion value matrix of the scheme will be constructed; finally, the weight of each evaluation objective will be calculated based on the value matrix and information entropy model, IFS-VIKOR model will be constructed to rank the concepts. A case study of the tree climbing and trimming machine will be employed to verify the proposed decision model. This decision model considering diversifying semantic variables and the conflict belief is proven to be effective compared with the IFS-SAW and ISF-TOPSIS.     

The use of PCA and signal processing techniques for processing time-based construction settlement data of road embankments

Instrumentation is beneficial in civil engineering for monitoring structures during their construction and operation. The data collected can be used to observe real-time response and develop data-driven models for predicting future behaviour. However, a limited number of sensors are usually used for on-site civil engineering construction due to cost restrictions and practicalities. This results in relatively small raw datasets, which often contain errors and anomalies. Interpreting and making judicious use of the available dataset for developing reliable predictive model represents a significant challenge. Therefore, it is essential to pre-process and clean the data for improving their quality. To date, little investigation has been performed in the application of such data cleaning methods to geotechnical engineering datasets collected from full-scale sites. The purpose of this study is to apply simple and effective data pre-processing techniques to site-data collected from a highway embankment constructed on a sequence of soil layers of different physical make-up and non-linear consolidation characteristics. Various cleaning methods were applied to magnetic extensometer data collected for monitoring settlement within foundation soils beneath the embankment. PCA was used to explore raw data, identify and remove outliers. Numerous filtering and smoothing methods were used to clean noise in the data and their results were further compared using RMSE and NMSE. The methods adopted for data pre-processing and cleaning proved very effective for capturing the raw settlement behaviour on site. The findings from this study would be useful to site engineers regarding complex decision-making relating to ground response due to embankment construction. This also has positive prospects for developing dynamic prediction models for embankment settlement.     

Deterministic and probabilistic ship pitch prediction using a multi-predictor integration model based on hybrid data preprocessing,reinforcement learning and improved QRNN

The deterministic and probabilistic prediction of ship motion is important for safe navigation and stable real-time operational control of ships at sea. However, the volatility and randomness of ship motion, the non-adaptive nature of single predictors and the poor coverage of quantile regression pose serious challenges to uncertainty prediction, making research in this field limited. In this paper, a multi-predictor integration model based on hybrid data preprocessing, reinforcement learning and improved quantile regression neural network (QRNN) is proposed to explore the deterministic and probabilistic prediction of ship pitch motion. To validate the performance of the proposed multi-predictor integrated prediction model, an experimental study is conducted with three sets of actual ship longitudinal motions during sea trials in the South China Sea. The experimental results indicate that the root mean square errors (RMSEs) of the proposed model of deterministic prediction are 0.0254°, 0.0359°, and 0.0188°, respectively. Taking series #2 as an example, the prediction interval coverage probabilities (PICPs) of the proposed model of probability predictions at 90%, 95%, and 99% confidence levels (CLs) are 0.9400, 0.9800, and 1.0000, respectively. This study signifies that the proposed model can provide trusted deterministic predictions and can effectively quantify the uncertainty of ship pitch motion, which has the potential to provide practical support for ship early warning systems.     

Non-invasive embedded system hardware/firmware anomaly detection based on the electric current signature

Quality control is a critical aspect of the modern electronic circuit industry. In addition to being a pre-requisite to proper functioning, circuit quality is closely related to safety, security, and economic issues. Quality control has been reached through system testing. Meanwhile, device miniaturization and multilayer Printed Circuit Boards have increased the electronic circuit test complexity considerably. Hence, traditional test processes based on manual inspections have become outdated and inefficient. More recently, the concept of Advanced Manufacturing or Industry 4.0 has enabled the manufacturing of customized products, tailored to the changing customers’ demands. This scenario points out additional requirements for electronic system testing: it demands a high degree of flexibility in production processes, short design and manufacturing cycles, and cost control. Thus, there is a demand for circuit testing systems that present effectiveness and accessibility without placing numerous test points. This work is focused on automated test solutions based on machine learning, which are becoming popular with advances in computational tools. We present a new testing approach that uses autoencoders to detect firmware or hardware anomalies based on the electric current signature. We built a test set-up using an embedded system development board to evaluate the proposed approach. We implemented six firmware versions that can run independently on the test board – one of them is considered anomaly-free. In order to obtain a reference frame to our results, two other classification techniques (a computer vision algorithm and a random forest classification model) were employed to detect anomalies on the same development board. The outcomes of the experiments demonstrated that the proposed test method is highly effective. For several test scenarios, the correct detection rate was above 99%. Test results showed that autoencoder and random forest approaches are effective. However, random forests require all data classes to be trained. Training an autoencoder, on the other hand, only requires the reference (anomaly-free) class.