Data preprocessing issues for incomplete medical datasets |
| |
Authors: | Min‐Wei Huang Wei‐Chao Lin Chih‐Wen Chen Shih‐Wen Ke Chih‐Fong Tsai William Eberle |
| |
Affiliation: | 1. Department of Psychiatry, Chiayi Branch, Taichung Veterans General Hospital, Chiayi, Taiwan;2. Department of Computer Science and Information Engineering, Asia University, Taichung, Taiwan;3. Department of Pharmacy, Kaohsiung Municipal Chinese Medical Hospital, Kaohsiung, Taiwan;4. Graduate Institute of Natural Products, Kaohsiung Medical University, Kaohsiung, Taiwan;5. Department of Information and Computer Engineering, Chung Yuan Christian University, Taoyuan City, Taiwan;6. Department of Information Management, National Central University, Taoyuan City, Taiwan;7. Department of Computer Science, Tennessee Technological University, Cookeville, TN, USA |
| |
Abstract: | While there is an ample amount of medical information available for data mining, many of the datasets are unfortunately incomplete – missing relevant values needed by many machine learning algorithms. Several approaches have been proposed for the imputation of missing values, using various reasoning steps to provide estimations from the observed data. One of the important steps in data mining is data preprocessing, where unrepresentative data is filtered out of the data to be mined. However, none of the related studies about missing value imputation consider performing a data preprocessing step before imputation. Therefore, the aim of this study is to examine the effect of two preprocessing steps, feature and instance selection, on missing value imputation. Specifically, eight different medical‐related datasets are used, containing categorical, numerical and mixed types of data. Our experimental results show that imputation after instance selection can produce better classification performance than imputation alone. In addition, we will demonstrate that imputation after feature selection does not have a positive impact on the imputation result. |
| |
Keywords: | missing value imputation feature selection instance selection incomplete medical datasets |
|
|