Performance Evaluation Indicator (PEI): A new paradigm to evaluate the competence of machine learning classifiers in predicting rockmass conditions |
| |
| Affiliation: | 1. Department of Geotechnical Engineering, School of Civil Engineering, Tongji University, 1239 Siping Road, Yangpu District, Shanghai, PR China;2. Civil and Environmental Engineering, Colorado School of Mines, 1610 Illinois St., Golden, CO 80401, USA;3. Department of Civil and Environmental Engineering, School of Engineering and Applied Science, University of California, Los Angeles, 405 Hilgard Ave., Los Angeles, CA 90095, USA;1. Department of Information Science, Faculty of Sciences, Toho University, 2-2-1 Miyama, 274-8510 Funabashi, Japan;2. Department of Applied Mathematics and Computational Sciences, E.T.S.I. Caminos, Canales y Puertos, University of Cantabria, Avda. de los Castros, s/n, 39005 Santander, Spain;3. School of Civil Engineering, Universidad de Cantabria, Avda. de los Castros 44, E-39005 Santander, Spain;4. Faculty of Electrical Engineering and Computer Science, University of Maribor, Maribor SI-2000, Slovenia;5. R&D EgiCAD, School of Civil Engineering, Universidad de Cantabria, Avda. de los Castros 44, 39005 Santander, Spain;1. Faculty of Science, Agriculture, and Engineering, Newcastle University, Singapore 599493, Singapore;2. Xylem Inc, USA;3. Department of Civil, Architectural and Environmental Engineering, University of Texas at Austin, TX 78712, USA;1. School of Urban Economics and Management, Beijing University of Civil Engineering and Architecture, Beijing 100044, China;2. Department of Construction Management, Tsinghua University, Beijing 100084, China;1. Symbiosis Institute of Tehnology and Symbiosis Centre for Applied Artificial Intelligence, Symbiosis International (Deemed) University, Pune, Maharashtra, India;2. Innovation Division, Technical University of Denmark, Denmark Faculty of Technology, Denmark;3. Computer Science Department Linnaeus University, Vaxjo, Sweden;4. Building Realization and Robotics, Technical University of Munich, Germany |
| |
| Abstract: | To illustrate an unprejudiced comparison among machine learning classifiers established on proprietary databases, and to guarantee the validity and robustness of these classifiers, a Performance Evaluation Indicator (PEI) and the corresponding failure criterion are proposed in this study. Three types of machine learning classifiers, including the strictly binary classifier, the normal multiclass classifier and the misclassification cost-sensitive classifier, are trained on four datasets recorded from a water drainage TBM project. The results indicate that: (1) the PEI successfully compares the competence of classifiers under different scenarios by isolating the effects of different overlapping-degree of rockmass classes, and (2) the cost-sensitive algorithm is warranted to classify rockmasses when the ratio of inter-class classes is more than 8:1. The contributions of this research are to fill the gap in performance evaluations of a classifier for imbalanced training data, and to identify the best situation to apply this classifier. |
| |
| Keywords: | Performance Evaluation Indicator (PEI) Rockmass classification TBM projects Machine learning classifiers Imbalanced database |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
