Improved behavior-based malware detection algorithm with AdaBoost

We present a new algorithm for abstracting features of a program from its API calls, network packages and static analysis characteristics. API calls are aggregated by a low level data dependence analysis to form the abstract behaviors．Network packages and static analysis characteristics are directly utilized as discrete value features．All of these abstract features are then embedded in a high dimension vector space. Besides, we further design a new behavior-based malware classification algorithm, which advances the AdaBoost boosted decision tree algorithm. Firstly, the new algorithm optimizes an anti-noise loss function to lower the probability of the noise data to train the next classifier, and thus improves the anti-noise ability of the AdaBoost algorithm. Secondly, to improve the algorithm's performance in multi-class classif bication problem, a vote vector is adopted to combine base classifiers, which discriminates the accuracy with which a classifier classifies samples from different classes.