Modelling user preferences and mediating agents in electronic commerce

An important ingredient in agent-mediated electronic commerce is the presence of intelligent mediating agents that assist electronic commerce participants (e.g. individual users, other agents, organisations). These mediating agents are in principle autonomous agents that interact with their environments (e.g. other agents and web-servers) on behalf of participants who have delegated tasks to them. For mediating agents a (preference) model of participants is indispensable. In this paper, a generic mediating agent architecture is introduced. Furthermore, we discuss our view of user preference modelling and its need in agent-mediated electronic commerce. We survey the state of the art in the field of preference modelling and suggest that the preferences of electronic commerce participants can be modelled by learning from their behaviour. In particular, we employ an existing machine learning method called inductive logic programming (ILP). We argue that this method can be used by mediating agents to detect regularities in the behaviour of the involved participants and induce hypotheses about their preferences automatically. Finally, we discuss some advantages and disadvantages of using inductive logic programming as a method for learning user preferences and compare this method with other approaches.     

Consensus and disturbance attenuation in multi‐agent chains with nonlinear control and time delays

In this paper, we investigate consensus and disturbance attenuation in a chain of mobile agents, which include non‐autonomous agents, semi‐autonomous agents and autonomous agents. In particular, the nonlinear dynamics of non‐autonomous agents is given and cannot be designed, while the dynamics of semi‐autonomous and autonomous agents can be partially and fully designed, respectively. To improve the robustness of multi‐agent chains against disturbances, we propose a nonlinear control framework for semi‐autonomous and autonomous agents such that they mimic the behavior of non‐autonomous agents for compatibility while also exploiting long‐range connections with distant agents. This framework ensures the existence of a unique consensus equilibrium, which is independent of the network size, connectivity topologies, control gains and information delays. Robustness of multi‐agent chains against disturbances is investigated by evaluating the frequency response at the nonlinear level. For infinitely long multi‐agent chains with recurrent patterns, we also derive a condition that ensures the disturbance attenuation but only requires the analysis of the linearized model. A case study is conducted for a connected vehicle system where numerical simulations are used to validate the analytical results. Copyright © 2016 John Wiley & Sons, Ltd.     

A Computational Framework for Practical Social Reasoning

This article describes a framework for practical social reasoning designed to be used for analysis, specification, and implementation of the social layer of agent reasoning in multiagent systems. Our framework, called the expectation strategy behavior (ESB) framework, is based on (i) using sets of update rules for social beliefs tied to observations (so‐called expectations), (ii) bounding the amount of reasoning to be performed over these rules by defining a reasoning strategy, and (iii) influencing the agent's decision‐making logic by means of behaviors conditioned on the truth status of current and future social beliefs. We introduce the foundations of ESB conceptually and present a formal framework and an actual implementation of a reasoning engine, which is specifically combined with a general (belief–desire–intention‐based) practical reasoning programming system. We illustrate the generality of ESB through select case studies, which show that it is able to represent and implement different typical styles of social reasoning. The broad coverage of existing social reasoning methods, the modularity that derives from its declarative nature, and its focus on practical implementation make ESB a useful tool for building advanced socially reasoning agents.     

Time-bound negotiation framework for electronic commerce agents

For efficient and informative coordination of agents especially in electronic commerce environment, a time-bound agent negotiation framework is proposed utilizing a time-based commitment scheme. By attaching commitment duration to agent messages, the traditional contract net protocol is extended to a time-bound negotiation framework (TBNF). The proposed negotiation framework has a new message type which allows for parties to agree upon the extension of a commitment duration, and a novel commitment concept in the form of negative commitment. The semantics of the messages with the commitment duration are interpreted, and then the three typical negotiation protocols are formally defined and compared — nothing-guaranteed protocol, acceptance-guaranteed protocol, and finite-time guarantee protocol — which can be incorporated into TBNF. The TBNF should provide a background for efficient and effective electronic commerce negotiation while accommodating each agent's adaptive negotiation strategy.     

EFFICIENT PERSONALIZATION OF E‐LEARNING ACTIVITIES USING A MULTI‐DEVICE DECENTRALIZED RECOMMENDER SYSTEM

Personalization is becoming a key issue in designing effective e‐learning systems and, in this context, a promising solution is represented by software agents. Usually, these systems provide the student with a student agent that interacts with a site agent associated with each e‐learning site. However, in presence of a large number of students and of e‐learning sites, the tasks of the agents are often onerous, even more if the student agents run on devices with limited resources. To face this problem, we propose a new multiagent learning system, called ISABEL. Our system provides each student, that are using a specific device, with a device agent able to autonomously monitor the student's behavior when accessing e‐learning Web sites. Each site is associated, in its turn, with a teacher agent. When a student visits an e‐learning site, the teacher agent collaborates with some tutor agents associated with the student, to provide him with useful recommendations. We present both theoretical and experimental results to show that this distributed approach introduces significant advantages in quality and efficiency of the recommendation activity with respect to the performances of other past recommenders.     

DART: A DISTRIBUTED ANALYSIS OF REPUTATION AND TRUST FRAMEWORK

Artificial societies—distributed systems of autonomous agents—are becoming increasingly important in open distributed environments, especially in e‐commerce. Agents require trust and reputation concepts to identify communities of agents with which to interact reliably. We have noted in real environments that adversaries tend to focus on exploitation of the trust and reputation model. These vulnerabilities reinforce the need for new evaluation criteria for trust and reputation models called exploitation resistance which reflects the ability of a trust model to be unaffected by agents who try to manipulate the trust model. To examine whether a given trust and reputation model is exploitation‐resistant, the researchers require a flexible, easy‐to‐use, and general framework. This framework should provide the facility to specify heterogeneous agents with different trust models and behaviors. This paper introduces a Distributed Analysis of Reputation and Trust (DART) framework. The environment of DART is decentralized and game‐theoretic. Not only is the proposed environment model compatible with the characteristics of open distributed systems, but it also allows agents to have different types of interactions in this environment model. Besides direct, witness, and introduction interactions, agents in our environment model can have a type of interaction called a reporting interaction, which represents a decentralized reporting mechanism in distributed environments. The proposed environment model provides various metrics at both micro and macro levels for analyzing the implemented trust and reputation models. Using DART, researchers have empirically demonstrated the vulnerability of well‐known trust models against both individual and group attacks.     

An agent‐based approach for managing e‐commerce activities

In this article, we propose an agent‐based approach for managing e‐commerce activities. In our approach, an agent is present in each e‐commerce site, managing the information stored there. In addition, another agent is associated with each customer, handling his/her profile. The proposed approach is based on the use of a particular conceptual model called the Behaviour‐Semantic Distance and Relevance (B‐SDR) network, which is capable of uniformly representing and handling information stored in e‐commerce sites and customer profiles. The capabilities of the B‐SDR network model are exploited to let customer and site agents cooperate in such a way in order to support a customer in identifying, whenever he/she accesses an e‐commerce site, those products and services present in the site itself and for better matching his/her interests. The approach has been implemented in a prototype in which its functionalities are discussed here also. © 2004 Wiley Periodicals, Inc.     

Hierarchical multi-agent reinforcement learning

In this paper, we investigate the use of hierarchical reinforcement learning (HRL) to speed up the acquisition of cooperative multi-agent tasks. We introduce a hierarchical multi-agent reinforcement learning (RL) framework, and propose a hierarchical multi-agent RL algorithm called Cooperative HRL. In this framework, agents are cooperative and homogeneous (use the same task decomposition). Learning is decentralized, with each agent learning three interrelated skills: how to perform each individual subtask, the order in which to carry them out, and how to coordinate with other agents. We define cooperative subtasks to be those subtasks in which coordination among agents significantly improves the performance of the overall task. Those levels of the hierarchy which include cooperative subtasks are called cooperation levels. A fundamental property of the proposed approach is that it allows agents to learn coordination faster by sharing information at the level of cooperative subtasks, rather than attempting to learn coordination at the level of primitive actions. We study the empirical performance of the Cooperative HRL algorithm using two testbeds: a simulated two-robot trash collection task, and a larger four-agent automated guided vehicle (AGV) scheduling problem. We compare the performance and speed of Cooperative HRL with other learning algorithms, as well as several well-known industrial AGV heuristics. We also address the issue of rational communication behavior among autonomous agents in this paper. The goal is for agents to learn both action and communication policies that together optimize the task given a communication cost. We extend the multi-agent HRL framework to include communication decisions and propose a cooperative multi-agent HRL algorithm called COM-Cooperative HRL. In this algorithm, we add a communication level to the hierarchical decomposition of the problem below each cooperation level. Before an agent makes a decision at a cooperative subtask, it decides if it is worthwhile to perform a communication action. A communication action has a certain cost and provides the agent with the actions selected by the other agents at a cooperation level. We demonstrate the efficiency of the COM-Cooperative HRL algorithm as well as the relation between the communication cost and the learned communication policy using a multi-agent taxi problem.     

GOAL‐DRIVEN AUTONOMY FOR RESPONDING TO UNEXPECTED EVENTS IN STRATEGY SIMULATIONS

To operate autonomously in complex environments, an agent must monitor its environment and determine how to respond to new situations. To be considered intelligent, an agent should select actions in pursuit of its goals, and adapt accordingly when its goals need revision. However, most agents assume that their goals are given to them; they cannot recognize when their goals should change. Thus, they have difficulty coping with the complex environments of strategy simulations that are continuous, partially observable, dynamic, and open with respect to new objects. To increase intelligent agent autonomy, we are investigating a conceptual model for goal reasoning called Goal‐Driven Autonomy (GDA), which allows agents to generate and reason about their goals in response to environment changes. Our hypothesis is that GDA enables an agent to respond more effectively to unexpected events in complex environments. We instantiate the GDA model in ARTUE (A utonomous R esponse t o U nexpected E vents), a domain‐independent autonomous agent. We evaluate ARTUE on scenarios from two complex strategy simulations, and report on its comparative benefits and limitations. By employing goal reasoning, ARTUE outperforms an off‐line planner and a discrepancy‐based replanner on scenarios requiring reasoning about unobserved objects and facts and on scenarios presenting opportunities outside the scope of its current mission.     

A combinatorial testing strategy for concurrent programs

One approach to testing concurrent programs is called reachability testing, which derives test sequences automatically and on‐the‐fly, without constructing a static model. Existing reachability testing algorithms are exhaustive in that they are intended to exercise all possible synchronization sequences of a concurrent program with a given input. In this paper, we present a new testing strategy, called t‐way reachability testing, that adopts the dynamic framework of reachability testing but selectively exercises a subset of synchronization sequences. The selection of the synchronization sequences is based on a combinatorial testing strategy called t‐way testing. We present an algorithm that implements t‐way reachability testing, and report the results of several case studies that were conducted to evaluate its effectiveness. The results indicate that t‐way reachability testing can substantially reduce the number of synchronization sequences exercised during reachability testing while still effectively detecting faults. Copyright © 2007 John Wiley & Sons, Ltd.     

UCov: a user‐defined coverage criterion for test case intent verification

The goal of regression testing is to ensure that the behaviour of existing code, believed correct by previous testing, is not altered by new program changes. This paper argues that the primary focus of regression testing should be on code associated with (1) earlier bug fixes and (2) particular application scenarios considered to be important by the developer or tester. Existing coverage criteria do not enable such focus, for example, 100% branch coverage does not guarantee that a given bug fix is exercised or a given application scenario is tested. Therefore, there is a need for a new and complementary coverage criterion in which the user can definea test requirement characterizing a given behaviour to be covered as opposed to choosing from a pool of pre‐defined and generic program elements. This paper proposes this new methodology and calls it UCov, a user‐defined coverage criterion wherein a test requirement is an execution pattern of program elements, and possibly predicates, that a test case must satisfy. The proposed criterion is not meant to replace existing criteria, but to complement them as it focuses the testing on important code patterns that could go untested otherwise. UCov supports test case intent verification. For example, following a bug fix, the testing team may augment the regression suite with the test case that revealed the bug. However, this test case might become obsolete due to code modifications not related to the bug. But if a test requirement characterizing the bug was defined by the user, UCov would determine that test case intent verification failed. The UCov methodology was implemented for the Java platform, was successfully applied onto 10 real‐life case studies and was shown to have advantages over JUnit. The implementation comprises the following tools: (1) TRSpec: allows the user to easily specify complex test requirements; (2) TRCheck: checks whether user‐defined test requirements were satisfied, that is, supports test case intent verification; and (3) TRMigrate: migrates user‐defined test requirements to subsequent versions of a given program. Copyright © 2016 John Wiley & Sons, Ltd.     

On the robustness to multiple agent losses in 2D and 3D formations

Multi‐agent formations have been recently the subject of many studies. An important operational challenge, largely unaddressed in the literature, is to ensure that functionality of the formation is retained should agents be lost through misadventure, mission reassignment, and so on. In the context of sensor networks, it is also important to allow for the loss of multiple sensors as the low quality of sensor hardware, common unattended implementations, and so on makes it a common issue. In this paper, we address these issues by proposing information structures that are tolerant to the loss of multiple agents. Using graph theory (and more specifically rigidity theory), we characterize several properties of such formations/networks in a unified framework: We characterize the k‐vertex rigidity property of the underling graph of such formations. This is performed by deriving a set of useful conditions that can form a guideline for designing agent‐loss‐tolerant formations. We elaborate the study by characterizing robust formations with the optimal number of control links. We also propose a set of operations preserving the tolerance to multiple agent losses in such formations. These operations provide flexibility in designing the formations in terms of several designing parameters (e.g., geometry, diameter, and max degree). Especially in the case of formations, the ability to handle controller adjustments in a distributed way is important, and the paper addresses this issue for a number of the robustness problems considered. Copyright © 2014 John Wiley & Sons, Ltd.     

Toward Self‐Motivated,Cognitive, Continually Planning Agents

We present a flexible initial framework for defining self‐motivated, self‐aware agents in simulated worlds, planning continuously so as to maximize long‐term rewards. While such agents employ reasoned exploration of feasible sequences of actions and corresponding states, they also behave opportunistically and recover from failure, thanks to their continual plan updates and quest for rewards. Our framework allows for both specific and general (quantified) knowledge and for epistemic predicates such as knowing‐that and knowing‐whether. Because realistic agents have only partial knowledge of their world, the reasoning of the proposed agents uses a weakened closed‐world assumption; this has consequences for epistemic reasoning, in particular introspection. The planning operators allow for quantitative, gradual change and side effects such as the passage of time, changes in distances and rewards, and language production, using a uniform procedural attachment method. Question answering (involving introspection) and experimental runs are shown for our particular agent ME in a simple world, demonstrating the value of continual deliberate, reward‐driven planning. Though the primary merit of agents definable in our framework is that they combine all of the aforementioned features, they can also be configured as single or multiple goal‐seeking agents and as such perform comparably with some recent experimental agents.     

An internal model approach to autonomous leader/follower trailing for non‐holonomic vehicles

The focus of this paper is on the design of a control architecture of decentralized type for controlling a leader/follower pair of autonomous non‐holonomic vehicles. A fundamental constraint in this trailing control requires that each agent employs local sensor information to process data on the relative position and velocity between its neighbouring vehicles, without relying on global communication with mission control. This constraint poses a challenge in the design of the control system because the reference trajectory to be tracked, which in the case considered in this paper is related to the motion of the leader, is not known a priori. It is shown in the paper that this specific control problem can be approached from the point of view of the internal model paradigm. In particular, once models of the autonomous dynamics of the leader are embedded in a decentralized dynamic controller, the design of the controller can be completed with a robust stabilizer, obtained by using ISS‐gain‐assignment techniques. It is shown that asymptotic convergence of the follower to an arbitrarily small neighbourhood of the desired steady‐state configuration is achieved, despite the presence of possibly large parameter uncertainties, while the motion of each agent remains confined into specified ‘sectors’ to avoid possible collision between neighbouring vehicles during transients. Simulation results are presented to illustrate the design methodology. Copyright © 2006 John Wiley & Sons, Ltd.     

IPOG/IPOG‐D: efficient test generation for multi‐way combinatorial testing

This paper presents two strategies for multi‐way testing (i.e. t‐way testing with t>2). The first strategy generalizes an existing strategy, called in‐parameter‐order, from pairwise testing to multi‐way testing. This strategy requires all multi‐way combinations to be explicitly enumerated. When the number of multi‐way combinations is large, however, explicit enumeration can be prohibitive in terms of both the space for storing these combinations and the time needed to enumerate them. To alleviate this problem, the second strategy combines the first strategy with a recursive construction procedure to reduce the number of multi‐way combinations that have to be enumerated. Both strategies are deterministic, i.e. they always produce the same test set for the same system configuration. This paper reports a multi‐way testing tool called FireEye, and provides an analytic and experimental evaluation of the two strategies. Copyright © 2007 John Wiley & Sons, Ltd.     

A Study of Building Internet Marketplaces on the Basis of Mobile Agents for Parallel Processing

In this paper, we propose a framework of Internet marketplaces on the basis of mobile agents. It not only simulates real commercial activities by consumers, agents and merchants, but also provides an environment for parallel processing. The latter is particularly important as more shops (sites) can be searched in real time to provide consumers with better choices. Meanwhile, if the number of mobile agents is very large and the dispatch is processed in a serial way, it can become a bottleneck that impacts the efficiency as a whole. In this paper, we also present and discuss several hierarchical dispatch models where the dispatch of multiple mobile agents can be processed in parallel over different hosts. We study these models analytically and empirically. The conducted experiments show that, in comparison with several serial mobile agent models, parallel mobile agent models can improve the performance significantly. In addition, in the best case for the parallel dispatch model, the time complexity for dispatching n mobile agents is O(log₂ n).     

Distributed robust control of uncertain linear multi‐agent systems

In this paper, the distributed H _∞ robust control problem synthesized with transient performance is investigated for a group of autonomous agents governed by uncertain general linear node dynamics. Based on the relative information between neighboring agents and some information of other agents, distributed state‐feedback and observer‐type output‐feedback control protocols are designed and analyzed, respectively. By using tools from robust control theory, conditions for the existence of controllers for solving such a problem are established. It is shown that the problem of distributed H _∞ robust control synthesized with transient performance can be converted to the H _∞ control problem synthesized with transient performance for decoupled linear systems of the same low dimensions. Finally, simulation examples are provided to illustrate the effectiveness of the design. Copyright © 2014 John Wiley & Sons, Ltd.     

Rule-based agents for the semantic web

Artificial agents, subsuming both robots and software agents, represent a new paradigm in software engineering and artificial intelligence. Depending on the technologies used in their implementation, they may exhibit various skills; in particular, they may act more or less autonomously, they may be able to learn and to adapt to a changing environment, and they may be able to pursue their goals pro-actively. An artificial agent is called rule-based, if its behaviour and/or its knowledge is expressed by means of rules. In this paper, we discuss a general architecture for rule-based agents and how it can be realized with the help of semantic web languages. We also show how such agents can go live on the web by presenting an implementation in Mandarax, a Java rule platform. The concept and implementation are complemented by a running example, the portfolio agent.