基于网络日志的用户行为检测和画像构建系统

用户画像可以帮助企业更多地了解用户,从而更好地制定决策。目前,大多数用户画像构建模型均依赖于应用厂商提的内部数据,而数据源的局限性可能难以保证用户画像的精准性。文章利用网络设备中的多源访问日志,基于知识数据库构建和指纹匹配技术,设计了一种新的用户画像构建架构,并借助Pyspider和Hadoop分布式框架提高数据采集和画像生成的计算效率。大量实验表明,该方法可以比现有模型构建出更全面、更准确的用户画像。

User behavior detection and portrait construction system based on network log

User portrait can help enterprises know more about users and do better decision-making.At present,most user portrait building models rely on the internal data provided by application vendors,and the limitations of data sources may be difficult to ensure the accuracy of user portrait.In this paper,by using the multi-source access logs recorded in the network node equipments,new user portrait architecture based on knowledge database construction and fingerprint matching is designed,meanwhile the computing efficiency of data acquisition and portrait generation is improved with the help of Pyspider and Hadoop distributed framework.A large number of experiments show that this method can build more comprehensive and accurate user portraits than that of the off-the-shelf models.