基于异常特征的钓鱼网站URL检测技术

典型的网络钓鱼是采用群发垃圾邮件,欺骗用户点击钓鱼网站URL地址,登录并输入个人机密信息的一种攻击手段。文章通过分析钓鱼网站URL地址的结构和词汇特征,提出一种基于异常特征的钓鱼网站URL检测方法。抽取钓鱼网站URL地址中4个结构特征、8个词汇特征,组成12个特征的特征向量,用SVM进行训练和分类。对PhishTank上7291条钓鱼网站URL分类实验,检测出7134条钓鱼网站URL,准确率达到97.85%。     

Deep belief network based detection and categorization of malicious URLs

The Internet, web consumers and computing systems have become more vulnerable to cyber-attacks. Malicious uniform resource locator (URL) is a prominent cyber-attack broadly used with the intention of data, money or personal information stealing. Malicious URLs comprise phishing URLs, spamming URLs, and malware URLs. Detection of malicious URL and identification of their attack type are important to thwart such attacks and to adopt required countermeasures. The proposed methodology for detection and categorization of malicious URLs uses stacked restricted Boltzmann machine for feature selection with deep neural network for binary classification. For multiple classes, IBK-kNN, Binary Relevance, and Label Powerset with SVM are used for classification. The approach is tested with 27700 URL samples and the results demonstrate that the deep learning-based feature selection and classification techniques are able to quickly train the network and detect with reduced false positives.     

基于集成学习的钓鱼网页深度检测系统

网络钓鱼是一种在线欺诈行为,它利用钓鱼网页仿冒正常合法的网页,窃取用户敏感信息从而达到非法目的.提出了基于集成学习的钓鱼网页深度检测方法,采用网页渲染来应对常见的页面伪装手段,提取渲染后网页的URL信息特征、链接信息特征以及页面文本特征,利用集成学习的方法,针对不同的特征信息构造并训练不同的基础分类器模型,最后利用分类集成策略综合多个基础分类器生成最终的结果.针对PhishTank钓鱼网页的检测实验表明,本文提出的检测方法具有较好的准确率与召回率.     

基于深度学习的恶意URL识别

网络攻击日益成为一个严重的问题.在这些攻击中,恶意URLs经常扮演着重要角色,并被广泛应用到各种类型的攻击,比如钓鱼、垃圾邮件以及恶意软件中.检测恶意链接对于阻止这些攻击具有重要意义.多种技术被应用于恶意URLs的检测,而近年来基于机器学习的方法得到越来越多的重视.但传统的机器学习算法需要大量的特征预处理工作,非常耗时耗力.在本文中,我们提出了一个完全基于词法特征的检测方法.首先,我们训练一个2层的神经网络,得到URLs中的字符的分布表示,然后训练对URL的分布表示生成的特征图像进行分类.在我们的试验中,使用真实数据,取得了精度为0.973和F1为0.918的结果.     

基于AdaCostBoost算法的网络钓鱼检测

针对日益严重的网络钓鱼攻击, 提出机器学习的方法进行钓鱼网站的检测和判断. 首先, 根据URL提取敏感特征, 然后, 采用AdaBoost算法进行训练出分类器, 再用训练好的分类器对未知URL检测识别. 最后, 针对非平衡代价问题, 采用了改进后的AdaBoost算法--AdaCostBoost, 加入代价因子的计算. 实验结果表明, 文中提出的网络钓鱼检测方法, 具有较优的检测性能.     

An antiphishing strategy based on visual similarity assessment

The authors' proposed antiphishing strategy uses visual characteristics to identify potential phishing sites and measure suspicious pages similarity to actual sites registered with the system. The first of two sequential processes in the SiteWatcher system runs on local email servers and monitors emails for keywords and suspicious URLs. The second process then compares the potential phishing pages against actual pages and assesses visual similarities between them in terms of key regions, page layouts, and overall styles. The approach is designed to be part of an enterprise antiphishing solution.     

一种基于集成学习的钓鱼网站检测方法

针对钓鱼攻击者常用的伪造HTTPS网站以及其他混淆技术,借鉴了目前主流基于机器学习以及规则匹配的检测钓鱼网站的方法RMLR和PhishDef,增加对网页文本关键字和网页子链接等信息进行特征提取的过程,提出了Nmap-RF分类方法。Nmap-RF是基于规则匹配和随机森林方法的集成钓鱼网站检测方法。根据网页协议对网站进行预过滤,若判定其为钓鱼网站则省略后续特征提取步骤。否则以文本关键字置信度,网页子链接置信度,钓鱼类词汇相似度以及网页PageRank作为关键特征,以常见URL、Whois、DNS信息和网页标签信息作为辅助特征,经过随机森林分类模型判断后给出最终的分类结果。实验证明,Nmap-RF集成方法可以在平均9~10 μs的时间内对钓鱼网页进行检测,且可以过滤掉98.4%的不合法页面,平均总精度可达99.6%。     

基于SVM主动学习算法的网络钓鱼检测系统

针对钓鱼式网络攻击,从URL入手,对网址URL和Web页面内容综合特征进行识别、分类,实现网络钓鱼检测并保证检测的效率和精度.用支持向量机主动学习算法和适合小样本集的分类模型提高分类性能.实验结果证明,网络钓鱼检测系统能达到较高的检测精度.     

Fraudulent and malicious sites on the web

Fraudulent and malicious web sites pose a significant threat to desktop security, integrity, and privacy. This paper examines the threat from different perspectives. We harvested URLs linking to web sites from different sources and corpora, and conducted a study to examine these URLs in-depth. For each URL, we extract its domain name, determine its frequency, IP address and geographic location, and check if the web site is accessible. Using 3 search engines (Google, Yahoo!, and Windows Live), we check if the domain name appears in the search results; and using McAfee SiteAdvisor, we determine the domain name’s safety rating. Our study shows that users can encounter URLs pointing to fraudulent and malicious web sites not only in spam and phishing messages but in legitimate email messages and the top search results returned by search engines. To provide better countermeasures against these threats, we present a proxy-based approach to dynamically block access to fraudulent and malicious web sites based on the safety ratings set by McAfee SiteAdvisor.     

基于内容提取的短链接生成算法研究

短网址服务通过将较长的目标网址缩短,来方便人们记忆并分享。社交网络尤其是近年来微博的盛行,使短网址服务获得广泛的应用。然而,现在短网址服务同时被不法分子盯上,他们利用短网址来伪装恶意链接,尤其是钓鱼网站链接,利用微博平台进行快速传播,最终窃取用户敏感信息甚至诈骗钱财,危害互联网安全。本文提出了一种新的短链接生成算法,通过对链接进行分析并在短网址中插入简短的目的网址内容,使用户在点击链接前能够辨识短网址的链接目标,以达到防范网络钓鱼的目的。     

Phish Block: A Blockchain Framework for Phish Detection in Cloud

The data in the cloud is protected by various mechanisms to ensure security aspects and user’s privacy. But, deceptive attacks like phishing might obtain the user’s data and use it for malicious purposes. In Spite of much technological advancement, phishing acts as the first step in a series of attacks. With technological advancements, availability and access to the phishing kits has improved drastically, thus making it an ideal tool for the hackers to execute the attacks. The phishing cases indicate use of foreign characters to disguise the original Uniform Resource Locator (URL), typosquatting the popular domain names, using reserved characters for re directions and multi-chain phishing. Such phishing URLs can be stored as a part of the document and uploaded in the cloud, providing a nudge to hackers in cloud storage. The cloud servers are becoming the trusted tool for executing these attacks. The prevailing software for blacklisting phishing URLs lacks the security for multi-level phishing and expects security from the client’s end (browser). At the same time, the avalanche effect and immutability of block-chain proves to be a strong source of security. Considering these trends in technology, a block-chain based filtering implementation for preserving the integrity of user data stored in the cloud is proposed. The proposed Phish Block detects the homographic phishing URLs with accuracy of 91% which assures the security in cloud storage.     

Semantic Based Greedy Levy Gradient Boosting Algorithm for Phishing Detection

The detection of phishing and legitimate websites is considered a great challenge for web service providers because the users of such websites are indistinguishable. Phishing websites also create traffic in the entire network. Another phishing issue is the broadening malware of the entire network, thus highlighting the demand for their detection while massive datasets (i.e., big data) are processed. Despite the application of boosting mechanisms in phishing detection, these methods are prone to significant errors in their output, specifically due to the combination of all website features in the training state. The upcoming big data system requires MapReduce, a popular parallel programming, to process massive datasets. To address these issues, a probabilistic latent semantic and greedy levy gradient boosting (PLS-GLGB) algorithm for website phishing detection using MapReduce is proposed. A feature selection-based model is provided using a probabilistic intersective latent semantic preprocessing model to minimize errors in website phishing detection. Here, the missing data in each URL are identified and discarded for further processing to ensure data quality. Subsequently, with the preprocessed features (URLs), feature vectors are updated by the greedy levy divergence gradient (model) that selects the optimal features in the URL and accurately detects the websites. Thus, greedy levy efficiently differentiates between phishing websites and legitimate websites. Experiments are conducted using one of the largest public corpora of a website phish tank dataset. Results show that the PLS-GLGB algorithm for website phishing detection outperforms state-of-the-art phishing detection methods. Significant amounts of phishing detection time and errors are also saved during the detection of website phishing.     

Malicious URL detection by dynamically mining patterns without pre-defined elements

Detecting malicious URLs is an essential task in network security intelligence. In this paper, we make two new contributions beyond the state-of-the-art methods on malicious URL detection. First, instead of using any pre-defined features or fixed delimiters for feature selection, we propose to dynamically extract lexical patterns from URLs. Our novel model of URL patterns provides new flexibility and capability on capturing malicious URLs algorithmically generated by malicious programs. Second, we develop a new method to mine our novel URL patterns, which are not assembled using any pre-defined items and thus cannot be mined using any existing frequent pattern mining methods. Our extensive empirical study using the real data sets from Fortinet, a leader in the network security industry, clearly shows the effectiveness and efficiency of our approach.     

基于域名信息的钓鱼URL探测

提出一种基于域名信息的钓鱼URL探测方法。使用编辑距离寻找与已知正常域名相似的域名,根据域名信息提取域名单词最大匹配特征、域名分割特征和URL分割特征,利用上述特征训练分类器,由此判断其他URL是否为钓鱼URL。在真实数据集上的实验结果表明,该方法钓鱼URL正确检测率达94%。     

Five-tier barrier anti-phishing scheme using hybrid approach

ABSTRACT

Though hoaxing people to make financial benefits is an old idea, phishers have realized that social engineering tools for web attacks are relatively easy to execute and are highly profitable over the Internet. One of the threatening criminal activities is phishing, in which the phishers trap users into revealing their identities and financial information to a fraudulent website. Researchers have proposed a number of anti-phishing techniques based on blacklist, whitelist, and visual similarity, but the major disadvantage with such approaches is that they are slow techniques with high false positive rates. For robust detection of phishing attacks, this article uses fundamentals of heuristic factors and a whitelist. The article proposes a safeguard scheme referred as the five-tier barrier hybrid approach. Input to the five-tier barrier is a uniform resource locator (URL), and output of the application is a status of the page (“Secure Connection” representing a legitimate URL, “Phishing Alert” representing phishing URL, and “Query Page” representing that the webpage needs to be processed further/failure of JSoup connection). In comparison to a blacklist, the five-tier barrier is competent in detecting zero-hour phishing attacks, and it is much faster than visual similarity–based anti-phishing techniques.     

Why do people get phished? Testing individual differences in phishing vulnerability within an integrated, information processing model

This research presents an integrated information processing model of phishing susceptibility grounded in the prior research in information process and interpersonal deception. We refine and validate the model using a sample of intended victims of an actual phishing attack. The data provides strong support for the model's theoretical structure and causative sequence. Overall, the model explains close to 50% of the variance in individual phishing susceptibility. The results indicate that most phishing emails are peripherally processed and individuals make decisions based on simple cues embedded in the email. Interestingly, urgency cues in the email stimulated increased information processing thereby short circuiting the resources available for attending to other cues that could potentially help detect the deception. Additionally, the findings suggest that habitual patterns of media use combined with high levels of email load have a strong and significant influence on individuals' likelihood to be phished. Consistent with social cognitive theory, computer self-efficacy was found to significantly influence elaboration, but its influence was diminished by domain specific-knowledge.     

基于用户交互的钓鱼邮件检测机制及其实现

针对国内外钓鱼邮件研究没有注重个体差异以及缺乏细粒度用户交互反馈的情况,提出一种集理论研究与实际应用相结合的框架,构建一种可视化邮件异常特征模型,设计一个chrome插件,使用分类器对邮件进行检测,将邮件异常特征详细信息和判定情况实时报告给用户。用户查看邮件后结合实际情况与分类器判定情况进行对照检查,出现判定不一致时对相应特征信息进行校正,将校正后的特征向量反馈给分类器进行修正,并将相应的特征信息组加入黑名单或白名单。实验证明,框架能根据个体反馈修正分类器,有效提高钓鱼邮件检出率和用户对钓鱼邮件细粒度感知体验。     

New rule-based phishing detection method

In this paper, we present a new rule-based method to detect phishing attacks in internet banking. Our rule-based method used two novel feature sets, which have been proposed to determine the webpage identity. Our proposed feature sets include four features to evaluate the page resources identity, and four features to identify the access protocol of page resource elements. We used approximate string matching algorithms to determine the relationship between the content and the URL of a page in our first proposed feature set. Our proposed features are independent from third-party services such as search engines result and/or web browser history. We employed support vector machine (SVM) algorithm to classify webpages. Our experiments indicate that the proposed model can detect phishing pages in internet banking with accuracy of 99.14% true positive and only 0.86% false negative alarm. Output of sensitivity analysis demonstrates the significant impact of our proposed features over traditional features. We extracted the hidden knowledge from the proposed SVM model by adopting a related method. We embedded the extracted rules into a browser extension named PhishDetector to make our proposed method more functional and easy to use. Evaluating of the implemented browser extension indicates that it can detect phishing attacks in internet banking with high accuracy and reliability. PhishDetector can detect zero-day phishing attacks too.     

基于URL混淆技术识别的钓鱼网页检测方法

针对钓鱼URL常用的混淆技术,提出一种基于规则匹配和逻辑回归的钓鱼网页检测方法（RMLR）。首先,使用针对违反URL命名标准及隐藏钓鱼目标词等混淆技术所构建的规则库对给定网页分类,若可判定其为钓鱼网址,则省略后续的特征提取及检测过程,以满足实时检测的需要。若未能直接判定为钓鱼网址,则提取该URL的相关特征,并使用逻辑回归分类器进行二次检测,以提升检测的适应性和准确率,并降低因规则库规模不足导致的误报率。同时,RMLR引入基于字符串相似度的Jaccard随机域名识别方法来辅助检测钓鱼URL。实验结果表明,RMLR准确率达到98.7%,具有良好的检测效果。     

Web安全测试中URL参数重写检测框架

Web站点中URL参数重写会对Web安全测试的准确性造成较大影响。针对该问题,设计URL参数重写检测框架,构造多个测试URL并提交请求,通过基于3种差异分析方法的随机URL取样验证策略,识别出URL中的伪路径,从而提取重写规则、并实现URL参数重写检测。应用C#语言实现的URL参数重写检测爬虫验证了该框架的有效性。