网络用户浏览行为的分析

近年来,Web使用挖掘成为数据挖掘领域中一个新的研究热点,Web使用挖掘是从记录了大量网络用户行为信息的Web日志中发现用户访问行为特征和潜在规律.本文结合某高校主页的真实运行数据,通过Web使用挖掘对于网站的运行日志文件进行全面的挖掘分析,分析用户对信息内容的兴趣度,并通过用户对网页的访问数据推算出各个页面受众的兴趣度高低,借此改良网站的内容和布局.     

A Movie Recommendation System – An Application of Voting Theory in User Modeling

Our research agenda focuses on building software agents that can employ user modeling techniques to facilitate information access and management tasks. Personal assistant agents embody a clearly beneficial application of intelligent agent technology. A particular kind of assistant agents, recommender systems, can be used to recommend items of interest to users. To be successful, such systems should be able to model and reason with user preferences for items in the application domain. Our primary concern is to develop a reasoning procedure that can meaningfully and systematically tradeoff between user preferences. We have adapted mechanisms from voting theory that have desirable guarantees regarding the recommendations generated from stored preferences. To demonstrate the applicability of our technique, we have developed a movie recommender system that caters to the interests of users. We present issues and initial results based on experimental data of our research that employs voting theory for user modeling, focusing on issues that are especially important in the context of user modeling. We provide multiple query modalities by which the user can pose unconstrained, constrained, or instance-based queries. Our interactive agent learns a user model by gaining feedback aboutits recommended movies from the user. We also provide pro-active information gathering to make user interaction more rewarding. In the paper, we outline the current status of our implementation with particular emphasis on the mechanisms used to provide robust and effective recommendations.     

Interactive Paper as a Mobile Client for a Multi-channel Web Information System

We describe how interactive paper can be used together with a multi-channel web information system to build a platform for experimenting with multi-modal context-aware mobile information services. As an application, we present a tourist guide for visitors to an international festival that was developed to investigate alternative modes of information delivery and interaction in mobile environments. The guide is based around a set of interactive paper documents—an event brochure, map and bookmark. The brochure and map are augmented with digital services by using a digital pen to activate links and a text-to-speech engine for information delivery. The digital pen is also used for data capture of event ratings and reviews. The bookmark provides access to advanced searches and ticket reservations. We describe the architecture and operation of the system, highlighting the challenges of extending a web information system to support both the generation of the paper documents and the interaction from these documents, alongside more traditional access channels. Finally, we discuss the range of context-aware interactions that is supported by our platform.     

面向用户互联网访问日志的异常点击分析

随着互联网用户人数的日益增长,用户行为分析已经成为互联网技术领域重要的研究方法之一。在日志中去除异常点击,对于准确挖掘用户行为的意图和习惯十分重要。该文采用某公司提供的真实用户互联网访问日志,对日志中的连续点击,单IP多用户以及单用户多IP等可能的异常点击,从访问集中度,用户平均访问量等方面进行了分析。我们认为对于连续点击,用户行为分析研究人员可以分情况滤去多余点击或该用户所有点击,而对于单IP多用户和单用户多 IP的点击,我们建议不做处理。     

Modeling user interests by conceptual clustering

As more information becomes available on the Web, there has been a crescent interest in effective personalization techniques. Personal agents providing assistance based on the content of Web documents and the user interests emerged as a viable alternative to this problem. Provided that these agents rely on having knowledge about users contained into user profiles, i.e., models of user preferences and interests gathered by observation of user behavior, the capacity of acquiring and modeling user interest categories has become a critical component in personal agent design. User profiles have to summarize categories corresponding to diverse user information interests at different levels of abstraction in order to allow agents to decide on the relevance of new pieces of information. In accomplishing this goal, document clustering offers the advantage that an a priori knowledge of categories is not needed, therefore the categorization is completely unsupervised. In this paper we present a document clustering algorithm, named WebDCC (Web Document Conceptual Clustering), that carries out incremental, unsupervised concept learning over Web documents in order to acquire user profiles. Unlike most user profiling approaches, this algorithm offers comprehensible clustering solutions that can be easily interpreted and explored by both users and other agents. By extracting semantics from Web pages, this algorithm also produces intermediate results that can be finally integrated in a machine-understandable format such as an ontology. Empirical results of using this algorithm in the context of an intelligent Web search agent proved it can reach high levels of accuracy in suggesting Web pages.     

Adaptive,intelligent presentation of information for the museum visitor in PEACH

The study of intelligent user interfaces and user modeling and adaptation is well suited for augmenting educational visits to museums. We have defined a novel integrated framework for museum visits and claim that such a framework is essential in such a vast domain that inherently implies complex interactivity. We found that it requires a significant investment in software and hardware infrastructure, design and implementation of intelligent interfaces, and a systematic and iterative evaluation of the design and functionality of user interfaces, involving actual visitors at every stage. We defined and built a suite of interactive and user-adaptive technologies for museum visitors, which was then evaluated at the Buonconsiglio Castle in Trento, Italy: (1) animated agents that help motivate visitors and focus their attention when necessary, (2) automatically generated, adaptive video documentaries on mobile devices, and (3) automatically generated post-visit summaries that reflect the individual interests of visitors as determined by their behavior and choices during their visit. These components are supported by underlying user modeling and inference mechanisms that allow for adaptivity and personalization. Novel software infrastructure allows for agent connectivity and fusion of multiple positioning data streams in the museum space. We conducted several experiments, focusing on various aspects of PEACH. In one, conducted with 110 visitors, we found evidence that even older users are comfortable interacting with a major component of the system.     

数据挖掘技术在Web预取中的应用研究

WWW以其多媒体的传输及良好的交互性而倍受青睐,虽然近几年来网络速度得到了很大的提高,但是由于接入Internet的用户数量剧增以及Web服务和网络固有的延迟,使得网络越来越拥护,用户的服务质量得不到很好的保证。为此文中提出了一种智能Web预取技术,它能够加快用户浏览Web页面时获取页面的速度。该技术通过简化的WWW数据模型表示用户浏览器缓冲器中的数据,在此基础上利用数据挖掘技术挖掘用户的兴趣关联规则,存放在兴趣关联知识库中,作为对用户行为进行预测的依据。在用户端,智能代理负责用户兴趣的挖掘及基于兴趣关联知识库的Web预取,从而对用户实现透明的浏览器加速。     

Using informative behavior to increase engagement while learning from human reward

In this work, we address a relatively unexplored aspect of designing agents that learn from human reward. We investigate how an agent’s non-task behavior can affect a human trainer’s training and agent learning. We use the TAMER framework, which facilitates the training of agents by human-generated reward signals, i.e., judgements of the quality of the agent’s actions, as the foundation for our investigation. Then, starting from the premise that the interaction between the agent and the trainer should be bi-directional, we propose two new training interfaces to increase a human trainer’s active involvement in the training process and thereby improve the agent’s task performance. One provides information on the agent’s uncertainty which is a metric calculated as data coverage, the other on its performance. Our results from a 51-subject user study show that these interfaces can induce the trainers to train longer and give more feedback. The agent’s performance, however, increases only in response to the addition of performance-oriented information, not by sharing uncertainty levels. These results suggest that the organizational maxim about human behavior, “you get what you measure”—i.e., sharing metrics with people causes them to focus on optimizing those metrics while de-emphasizing other objectives—also applies to the training of agents. Using principle component analysis, we show how trainers in the two conditions train agents differently. In addition, by simulating the influence of the agent’s uncertainty–informative behavior on a human’s training behavior, we show that trainers could be distracted by the agent sharing its uncertainty levels about its actions, giving poor feedback for the sake of reducing the agent’s uncertainty without improving the agent’s performance.     

使用去噪和相异度的电子商务网站用户访问聚类算法

电子商务网站包含相当大的用户访问信息,对用户信息的数据挖掘,可以加强网站对用户访问信息的准确了解,提高电子商务网站的点击率。为此将提取电子商务网站日志中记录的用户访问链接数据,利用去噪技术对用户访问链接日志记录数据进行过滤分析,将过滤后的用户访问数据利用相异度二元关系组成二元数组,通过对二元数组的相异度分析计算,可实现电商务网站用户的聚类,为网站页面的优化及访问用户的兴趣、爱好的掌握提供参考。     

Web挖掘在电子商务中的应用

如何从海量的Web数据中发现有用的知识是一个迫切需要研究的课题,因此,Web挖掘应运而生,成为一个全新的研究领域。Web挖掘就是从Web文档和Web活动中抽取潜在的有用模式和隐藏信息。随着电子商务的发展,Web挖掘进入了一个新的应用领域,介绍了Web挖掘技术在电子商务中的具体应用,运用Web挖掘技术对Web数据进行挖掘,了解客户的行为,从而调整站点结构、市场策略等,使电子商务活动具有针对性。     

Adaptive personal agents

Personal agents are an important advance in the management of electronic information. Accurate user profiling is critical to the personalisation of agent-based services. In this paper we describe a framework for personal agents, which provides a range of services using a common user profile. The user profile is automatically learned from observation of the electronic documents a user reads, and tracks user interests over time. We report on a large trial of this framework, which has given us important insights into the effectiveness of personal agent applications. In particular we discuss the benefits obtained from the use of a common profile and the interoperation of several personal agent applications.     

自适应Web站点：挑战与机遇

1 引言万维网(World Wide Web)已经成为信息传播、交流与共享的主要媒体。在全球Web站点数目迅速增长的同时,各个Web站点的信息量及其复杂度也在迅速上升,包含成千上万个网页与超链接是很平常的。由于以下的因素,数据密集型Web站点的设计与管理也变得越来越困难:     

一种基于智能体的Web文档预取模式

文章深入分析了用户对Ｉｎｔｅｒｎｅｔ资源的访问模式和ｗｅｂ文档自身的更新模式,并提出了一个新的基于智能体的ｗｅｂ文档预取系统结构。在这个系统结构基础上,通过用户存取日志及各种算法,发现特定用户感兴趣的主题,实现对兴趣文档的主动预取,从而提高分布式信息系统上信息的获取效率。     

A Knowledge Base for the maintenance of knowledge extracted from web data

By applying web mining tools, significant patterns about the visitor behavior can be extracted from data originated in web sites. Supported by a domain expert, the patterns are validated or rejected and rules about how to use the patterns are created. This results in discovering new knowledge about the visitor behavior to the web site. But, due to frequent changes in the visitor’s interests, as well as in the web site itself, the discovered knowledge may become obsolete in a short period of time. In this paper, we introduce a Knowledge Base (KB), which consists of a database-type repository for maintaining the patterns, and rules, as an independent program that consults the pattern repository. Using the proposed architecture, an artificial system or a human user can consult the KB in order to improve the relation between the web site and its visitors. The proposed structure was tested using data from a Chilean virtual bank, which proved the effectiveness of our approach.     

基于矩阵聚类的电子商务网站个性化推荐系统

提出一种基于“矩阵聚类”的电子商务网站个性化推荐系统,通过分析Web server日志文件中的访问页面序列行为数据,构建较高购买者的顾客行为的矩阵模型;并使用一种新型的“矩阵聚类”算法挖掘潜在购买者与较高购买者的相似特征,从而帮助顾客发现他所希望购买的产品信息,用于提高实际购买量．该技术特别适合于目前大型的电子商务网站,实验数据表明,该系统是高效并可广泛使用．     

Learning users' interests by quality classification in market-based recommender systems

Recommender systems are widely used to cope with the problem of information overload and, to date, many recommendation methods have been developed. However, no one technique is best for all users in all situations. To combat this, we have previously developed a market-based recommender system that allows multiple agents (each representing a different recommendation method or system) to compete with one another to present their best recommendations to the user. In our system, the marketplace encourages good recommendations by rewarding the corresponding agents who supplied them according to the users' ratings of their suggestions. Moreover, we have theoretically shown how our system incites the agents to bid in a manner that ensures only the best recommendations are presented. To do this effectively in practice, however, each agent needs to be able to classify its recommendations into different internal quality levels, learn the users' interests for these different levels, and then adapt its bidding behavior for the various levels accordingly. To this end, in this paper, we develop a reinforcement learning and Boltzmann exploration strategy that the recommending agents can exploit for these tasks. We then demonstrate that this strategy does indeed help the agents to effectively obtain information about the users' interests which, in turn, speeds up the market convergence and enables the system to rapidly highlight the best recommendations.     

Personalized blog content recommender system for mobile phone users

Compared to newspaper columnists and broadcast media commentators, bloggers do not have organizations actively promoting their content to users; instead, they rely on word-of-mouth or casual visits by web surfers. We believe the WAP Push service feature of mobile phones can help bridge the gap between internet and mobile services, and expand the number of potential blog readers. Since mobile phone screen size is very limited, content providers must be familiar with individual user preferences in order to recommend content that matches narrowly defined personal interests. To help identify popular blog topics, we have created (a) an information retrieval process that clusters blogs into groups based on keyword analyses, and (b) a mobile content recommender system (M-CRS) for calculating user preferences for new blog documents. Here we describe results from a case study involving 20,000 mobile phone users in which we examined the effects of personalized content recommendations. Browsing habits and user histories were recorded and analyzed to determine individual preferences for making content recommendations via the WAP Push feature. The evaluation results of our recommender system indicate significant increases in both blog-related push service click rates and user time spent reading personalized web pages. The process used in this study supports accurate recommendations of personalized mobile content according to user interests. This approach can be applied to other embedded systems with device limitations, since document subject lines are elaborated and more attractive to intended users.     

Feature evaluation for web crawler detection with data mining techniques

Distributed Denial of Service (DDoS) is one of the most damaging attacks on the Internet security today. Recently, malicious web crawlers have been used to execute automated DDoS attacks on web sites across the WWW. In this study we examine the effect of applying seven well-established data mining classification algorithms on static web server access logs in order to: (1) classify user sessions as belonging to either automated web crawlers or human visitors and (2) identify which of the automated web crawlers sessions exhibit ‘malicious’ behavior and are potentially participants in a DDoS attack. The classification performance is evaluated in terms of classification accuracy, recall, precision and F₁ score. Seven out of nine vector (i.e. web-session) features employed in our work are borrowed from earlier studies on classification of user sessions as belonging to web crawlers. However, we also introduce two novel web-session features: the consecutive sequential request ratio and standard deviation of page request depth. The effectiveness of the new features is evaluated in terms of the information gain and gain ratio metrics. The experimental results demonstrate the potential of the new features to improve the accuracy of data mining classifiers in identifying malicious and well-behaved web crawler sessions.     

An Analysis of the Interaction Between Intelligent Software Agents and Human Users

Interactions between an intelligent software agent (ISA) and a human user are ubiquitous in everyday situations such as access to information, entertainment, and purchases. In such interactions, the ISA mediates the user’s access to the content, or controls some other aspect of the user experience, and is not designed to be neutral about outcomes of user choices. Like human users, ISAs are driven by goals, make autonomous decisions, and can learn from experience. Using ideas from bounded rationality (and deploying concepts from artificial intelligence, behavioural economics, control theory, and game theory), we frame these interactions as instances of an ISA whose reward depends on actions performed by the user. Such agents benefit by steering the user’s behaviour towards outcomes that maximise the ISA’s utility, which may or may not be aligned with that of the user. Video games, news recommendation aggregation engines, and fitness trackers can all be instances of this general case. Our analysis facilitates distinguishing various subcases of interaction (i.e. deception, coercion, trading, and nudging), as well as second-order effects that might include the possibility for adaptive interfaces to induce behavioural addiction, and/or change in user belief. We present these types of interaction within a conceptual framework, and review current examples of persuasive technologies and the issues that arise from their use. We argue that the nature of the feedback commonly used by learning agents to update their models and subsequent decisions could steer the behaviour of human users away from what benefits them, and in a direction that can undermine autonomy and cause further disparity between actions and goals as exemplified by addictive and compulsive behaviour. We discuss some of the ethical, social and legal implications of this technology and argue that it can sometimes exploit and reinforce weaknesses in human beings.     

Exploiting semantic annotation to supporting user browsing on the web

The aim of this paper is to support user browsing on semantically heterogeneous information spaces. In advance of a user’s explicit actions, his search context should be predicted by the locally annotated resources in his access histories. We thus exploit semantic transcoding method and measure the relevance between the estimated model of user intention and the candidate resources in web spaces. For these experiments, we simulated the scenario of comparison-shopping systems on the testing bed organized by twelve online stores in which images are annotated with semantically heterogeneous metadata.