MD5算法在农业数据消重中的应用

垂直搜索引擎的数据来源于各大相关网站。随着互联网用户越来越多，相互转载，重复发布的数据也越来越多，由于这些重复及相似数据的存在，严重影响了搜索引擎的检索结果。针对这一问题本文提出了一种解决方法，即利用MD5算法在数据处理阶段有效的消除冗余数据。结果表明该方法很好的提高了检索质量。

Application of MD5 in Agricultural Data Cleaning

The performance and membrane fouling of submerged membrane bioreactor were studied in the case of active sludge bulking under low temperature. The factors contributing to membrane fouling were discussed from the microorganism aspect. The results showed that COD removal efficiencies of supernatant and permeate were 85% and 92% respectively and filamentous sludge bulking had little impact on them. The sludge settleability became bad and the filament index (FI) increased from 2 to 5 during the formation of filamentous sludge bulking under low temperature. The filamentous bacteria extending from the sludge flocs formed net structure. Membrane fouling changed with time in linear under low temperature and the operation period of MBR was 15 d. However, membrane fouling was more serious in the condition of filamentous sludge bulking at low temperature, shortening the operation period of MBR to 7 d. The extracellular polymeric substances (EPS) content of bulking sludge was three times as that of normal sludge and the relative hydrophobicity (RH) of sludge flocs was decreased as FI increased. The increase of EPS and RH may cause more materials to deposit on the membrane surface, thus the membrane fouling rate improved and the operation period of MBR became short. Further analysis indicated that the mixed liquid viscosity, Zeta potential and sludge floc structure were all important factors of membrane fouling.