Maximum Value Attribute-Based Extra Trees, XGBoost, and LightGBM for Early Disease Prediction

Abstract

The evolution of intelligent healthcare systems has increased the importance of machine learning techniques in timely prediction of heart disease by using clinical and medical data. In recent years machine Learning techniques have been used widely in disease prediction systems due to their ability to examine complex medical patterns and also for better diagnostic accuracy. Many machine learning classifiers achieve strong predictive performance but still acquire high computational power and longer execution time during their training and testing phases. The ensemble learning algorithms such as Extra Trees, Extreme Gradient boosting and LightGBM have excellent performance in prediction of data due to their high classification accuracy and robustness. The main problem is that these models often suffer from high iteration counts, longer execution time and the inclusion of irrelevant or redundant features which may affect prediction efficiency and the overall performance of a model. To address these limitations, this research paper introduces a Rough Set Theory which is totally based on hybrid maximum value attribute (MVA). Hybrid MVA is a feature selection method that combines cardinality based ranking for categorical attributes and variance based ranking for continuous attributes in order to choose the most relevant features before training the models and predict heart disease in an efficient manner.The proposed models evaluate  three techniques that are hybrid maximum value attribute Extra trees (MVA ET), hybrid maximum value Attribute XGBoost (MVA XGBoost) and hybrid maximum value attribute LightGBM (MVA LightGBM). This hybrid MVA method improves the performance of the model and improves computational efficiency by selecting the most relevant categorical and continuous features and removes irrelevant ones. The models are implemented using Python in Jupyter Notebook. The dataset is obtained from Kaggle platform which is titled as “Synthetic Heart Disease Prediction Dataset” that contains 50000 records and 20 features. The standard performance evaluation metrics including accuracy, precision, recall, F1 score and execution time are used to measure the success of the proposed approach. This report compares standard models against hybrid MVA across various train and test data distributions. The experimental evaluation is performed using accuracy, precision, recall, F1-score, execution time and iteration analysis to measure model effectiveness comprehensively and Experimental results show that the hybrid MVA based models outperforms the traditional classifiers in terms of prediction accuracy, computational efficiency and feature optimization. It reduces the feature dimensionality from 20 to 13 features, i.e., a 35% reduction in the input feature space before model training. For the Extra Trees classifier, the average iteration steps decreased from 46,787 to 31,040, corresponding to a 33.7% reduction in computational complexity. Similarly, training and execution time were also reduced in Hybrid MVA Extra Trees. The Hybrid MVA XGBoost model and Hybrid MVA LightGBM reduced the average iteration steps from 4,560 to 2,964, corresponding to a 35% reduction, while also reducing training and execution times. The Hybrid MVA Extra Trees model achieved an average accuracy of 99.07%, while Hybrid MVA XGBoost and Hybrid MVA LightGBM maintained average accuracies of 99.73% and 99.57%, respectively. Among the proposed approaches, Hybrid MVA XGBoost provided the best balance between predictive performance and computational efficiency.