Adaptive learning algorithm of self-organizing teams

In order to improve the ability of achieving good performance in self-organizing teams, this paper presents a self-adaptive learning algorithm for team members. Members of the self-organizing teams are simulated by agents. In the virtual self-organizing team, agents adapt their knowledge according to cooperative principles. The self-adaptive learning algorithm is approached to learn from other agents with minimal costs and improve the performance of the self-organizing team. In the algorithm, agents learn how to behave (choose different game strategies) and how much to think about how to behave (choose the learning radius). The virtual team is self-adaptively improved according to the strategies’ ability of generating better quality solutions in the past generations. Six basic experiments are manipulated to prove the validity of the adaptive learning algorithm. It is found that the adaptive learning algorithm often causes agents to converge to optimal actions, based on agents’ continually updated cognitive maps of how actions influence the performance of the virtual self-organizing team. This paper considered the influence of relationships in self-organizing teams over existing works. It is illustrated that the adaptive learning algorithm is beneficial to both the development of self-organizing teams and the performance of the individual agent.     

Breaking the habit: Measuring and predicting departures from routine in individual human mobility

Researchers studying daily life mobility patterns have recently shown that humans are typically highly predictable in their movements. However, no existing work has examined the boundaries of this predictability, where human behaviour transitions temporarily from routine patterns to highly unpredictable states. To address this shortcoming, we tackle two interrelated challenges. First, we develop a novel information-theoretic metric, called instantaneous entropy, to analyse an individual’s mobility patterns and identify temporary departures from routine. Second, to predict such departures in the future, we propose the first Bayesian framework that explicitly models breaks from routine, showing that it outperforms current state-of-the-art predictors.     

Cooperative learning neural network output feedback control of uncertain nonlinear multi-agent systems under directed topologies

Without assuming that the communication topologies among the neural network (NN) weights are to be undirected and the states of each agent are measurable, the cooperative learning NN output feedback control is addressed for uncertain nonlinear multi-agent systems with identical structures in strict-feedback form. By establishing directed communication topologies among NN weights to share their learned knowledge, NNs with cooperative learning laws are employed to identify the uncertainties. By designing NN-based κ-filter observers to estimate the unmeasurable states, a new cooperative learning output feedback control scheme is proposed to guarantee that the system outputs can track nonidentical reference signals with bounded tracking errors. A simulation example is given to demonstrate the effectiveness of the theoretical results.     

A Distributed Approach for Coordination of Traffic Signal Agents

Innovative control strategies are needed to cope with the increasing urban traffic chaos. In most cases, the currently used strategies are based on a central traffic-responsive control system which can be demanding to implement and maintain. Therefore, a functional and spatial decentralization is desired. For this purpose, distributed artificial intelligence and multi-agent systems have come out with a series of techniques which allow coordination and cooperation. However, in many cases these are reached by means of communication and centrally controlled coordination processes, giving little room for decentralized management. Consequently, there is a lack of decision-support tools at managerial level (traffic control centers) capable of dealing with decentralized policies of control and actually profiting from them. In the present work a coordination concept is used, which overcomes some disadvantages of the existing methods. This concept makes use of techniques of evolutionary game theory: intersections in an arterial are modeled as individually-motivated agents or players taking part in a dynamic process in which not only their own local goals but also a global one has to be taken into account. The role of the traffic manager is facilitated since s/he has to deal only with tactical ones, leaving the operational issues to the agents. Thus the system ultimately provides support for the traffic manager to decide on traffic control policies. Some application in traffic scenarios are discussed in order to evaluate the feasibility of transferring the responsibility of traffic signal coordination to agents. The results show different performances of the decentralized coordination process in different scenarios (e.g. the flow of vehicles is nearly equal in both opposing directions, one direction has a clearly higher flow, etc.). Therefore, the task of the manager is facilitate once s/he recognizes the scenario and acts accordingly.This revised version was published online in August 2005 with a corrected cover date.     

Providing adaptation and guidance for design learning by problem solving: The design planning approach in DomoSim-TPC environment

Experimental learning environments based on simulation usually require monitoring and adaptation to the actions the users carry out. Some systems provide this functionality, but they do so in a way which is static or cannot be applied to problem solving tasks. In response to this problem, we propose a method based on the use of intermediate languages to provide adaptation in design learning scenarios. Although we use some approaches which are familiar from other domains (e.g., programming tutors) they are novel as regards their application to a very different domain and as a result we have incorporated new strategies. The purpose of our proposal is to provide monitoring, guidance and adaptive features for PlanEdit, a tool for the learning of integral automation methods in buildings and housing by design. This tool is part of a collaborative environment, called DomoSim-TPC, which supports distance learning of domotical design. We have carried out an experiment to obtain some data which confirm that our position can be effective for group learning of domotical design, studying the relationship between the quantity of model work carried out and the errors made.     

Enhancing the performance of an agent-based manufacturing system through learning and forecasting

Agent-based technology has been identified as an important approach for developing next generation manufacturing systems. One of the key techniques needed for implementing such advanced systems will be learning. This paper first discusses learning issues in agent-based manufacturing systems and reviews related approaches, then describes how to enhance the performance of an agent-based manufacturing system through learning from history (based on distributed case-based learning and reasoning) and learning from the future (through system forecasting simulation). Learning from history is used to enhance coordination capabilities by minimizing communication and processing overheads. Learning from the future is used to adjust promissory schedules through forecasting simulation, by taking into account the shop floor interactions, production and transportation time. Detailed learning and reasoning mechanisms are described and partial experimental results are presented.     

Learning from demonstration in robots: Experimental comparison of neural architectures

Robots have played an important role in the automation of computer aided manufacturing. The classical robot control implementation involves an expensive key step of model-based programming. An intuitive way to reduce this expensive exercise is to replace programming with machine learning of robot actions from demonstration where a (learner) robot learns an action by observing a demonstrator robot performing the same. To achieve this learning from demonstration (LFD) different machine learning techniques such as Artificial Neural Networks (ANN), Genetic Algorithms, Hidden Markov Models, Support Vector Machines, etc. can be used. This piece of work focuses exclusively on ANNs. Since ANNs have many standard architectural variations divided into two basic computational categories namely the recurrent networks and feed-forward networks, representative networks from each have been selected for study, i.e. Feed Forward Multilayer Perceptron (FF) network for feed-forward networks category and Elman (EL), and Nonlinear Autoregressive Exogenous Model (NARX) networks for the recurrent networks category. The main objective of this work is to identify the most suitable neural architecture for application of LFD in learning different robot actions. The sensor and actuator streams of demonstrated action are used as training data for ANN learning. Consequently, the learning capability is measured by comparing the error between demonstrator and corresponding learner streams. To achieve fairness in comparison three steps have been taken. First, Dynamic Time Warping is used to measure the error between demonstrator and learner streams, which gives resilience against translation in time. Second, comparison statistics are drawn between the best, instead of weight-equal, configurations of competing architectures so that learning capability of any architecture is not forced handicap. Third, each configuration's error is calculated as the average of ten trials of all possible learning sequences with random weight initialization so that the error value is independent of a particular sequence of learning or a particular set of initial weights. Six experiments are conducted to get a performance pattern of each architecture. In each experiment, a total of nine different robot actions were tested. Error statistics thus obtained have shown that NARX architecture is most suitable for this learning problem whereas Elman architecture has shown the worst suitability. Interestingly the computationally lesser MLP gives much lower and slightly higher error statistics compared to the computationally superior Elman and NARX neural architectures, respectively.     

Supporting teachers in adaptive educational systems through predictive models: A proof of concept

Adaptive educational systems (AESs) guide students through the course materials in order to improve the effectiveness of the learning process. However, AES cannot replace the teacher. Instead, teachers can also benefit from the use of adaptive educational systems enabling them to detect situations in which students experience problems (when working with the AES). To this end the teacher needs to monitor, understand and evaluate the students’ activity within the AES. In fact, these systems can be enhanced if tools for supporting teachers in this task are provided. In this paper, we present the experiences with predictive models that have been undertaken to assist the teacher in PDinamet, a web-based adaptive educational system for teaching Physics in secondary education. Although the obtained models are still very simple, our findings suggest the feasibility of predictive modeling in the area of supporting teachers in adaptive educational systems.     

Fostering collaborative learning in Second Life: Metaphors and affordances

In this paper we examine the transferability of the Jigsaw and Fishbowl collaborative learning techniques to the Second Life platform. Our aim is to assess the applicability of Second Life for collaborative learning by developing virtual tools and metaphors and exploiting the representational richness of this novel medium. In order to enhance the existing metaphors and affordances of SL, our research team implemented educational spaces, avatar clothing, and tools for non-verbal communication and visualisation. By implementing a blended learning evaluation approach we attempted to answer three research questions focusing on student collaboration, avatar representation and learning space awareness. We can conclude that SL can supplement and/or augment face to face interactions, improving upon previous approaches in distance collaboration and communication. Furthermore, although our team augmented SL’s ability to support collaborative learning, avatar representation does not seem to scale well. Finally, the majority of the implemented affordances and metaphors seem to have enhanced collaboration and learning space awareness.     

Stable Flocking Motion of Mobile Agents Following a Leader in Fixed and Switching Networks

Multiple mobile agents with double integrator dynamics, following a leader to achieve a flocking motion formation, are studied in this paper. A class of local control laws for a group of mobile agents is proposed. Prom a theoretical proof, the following conclusions are reached: (i) agents globally align their velocity vectors with a leader, (ii) they converge their velocities to the leaders velocity, (iii) collisions among interconnected agents are avoided, and (iv) agent's artificial potential functions are minimized. We model the interaction and/or communication relationship between agents by algebraic graph theory. Stability analysis is achieved by using classical Lyapunov theory in a fixed network topology, and differential inclusions and nonsmooth analysis in a switching network topology respectively. Simulation examples are provided.     

Supporting teachers in guiding collaborating students: Effects of learning analytics in CSCL

The aim of this study was to examine the effects of teacher supporting tools that present summaries, visualizations, and analyses of student participation and discussion on the way teachers guide collaborating groups of students in a digital learning environment. An experimental set-up was used in which authentic student data was converted to simulation vignettes that participants could interact with, enabling them to act as the teacher. The vignettes contained groups that had a problem concerning participation or discussion. When presented with the supporting tools, teachers and student teachers were better able to spot the problems regarding participation, intervened more often in problematic groups as time progressed, and displayed more specific explanations of their actions.     

The impact of media on collaborative learning in virtual settings: The perspective of social construction

This study examines an alternative function of information sharing – social construction of meaning. Drawing on social construction, social interaction, and task closure theories, we explored the influence of both the media environment in which students are situated and the medium that group members choose to communicate with one another on the intricate relationships among breadth of information sharing, depth of information sharing, and performance of computer-supported collaborative learning (CSCL). A total of 126 students participated in the experiment – including 63 students (15 groups of four students and one group of three students) in the control and experimental groups respectively. Our findings show that most of the proposed hypotheses are supported. Intersubjective interpretation underlies groups information sharing and plays a key role in student learning performance. Evidence shows that when facing a relatively complex task in multimedia environments, students who choose to utilize a medium lower in social presence (i.e., electronic information sharing) are more likely to achieve task closure than a medium higher in social presence (i.e., verbal information sharing). This in turn leads to higher learning performance. The implications for both theory and pedagogy are also discussed.     

Role of personality in computer based learning

With the rapid growth of technology enhanced learning, new mediums for learning have emerged. One of these mediums is computer based learning where the main concern is how to design a computer based learning system which takes into consideration the learners' differences. Personality is considered as one of the most critical sources of individual differences. This study investigates how personality differences within learners can affect computer based learning, through a comprehensive review of the literature. The highlighted results from the obtained nineteen studies are: (a) the most referred to personality model in computer based learning is MBTI; (b) personality traits affect how learners prefer learning content and learning approach like collecting information, communicating with instructor and peer, study behavior, acting and performing; (c) a new model of personality variables should be considered in computer based learning by taking all interested researchers and practitioners into accounts; and (d) the traditional questionnaire approach which is still the pre-dominant method to identify the learner's personality; and this needs to be changed with new potential of big data and learning analytics. Furthermore, this study presents a new implicit approach using learning analytics instead of questionnaire-based approach to identify the learner's personality.     

Opportunities for multiagent systems and multiagent reinforcement learning in traffic control

The increasing demand for mobility in our society poses various challenges to traffic engineering, computer science in general, and artificial intelligence and multiagent systems in particular. As it is often the case, it is not possible to provide additional capacity, so that a more efficient use of the available transportation infrastructure is necessary. This relates closely to multiagent systems as many problems in traffic management and control are inherently distributed. Also, many actors in a transportation system fit very well the concept of autonomous agents: the driver, the pedestrian, the traffic expert; in some cases, also the intersection and the traffic signal controller can be regarded as an autonomous agent. However, the “agentification” of a transportation system is associated with some challenging issues: the number of agents is high, typically agents are highly adaptive, they react to changes in the environment at individual level but cause an unpredictable collective pattern, and act in a highly coupled environment. Therefore, this domain poses many challenges for standard techniques from multiagent systems such as coordination and learning. This paper has two main objectives: (i) to present problems, methods, approaches and practices in traffic engineering (especially regarding traffic signal control); and (ii) to highlight open problems and challenges so that future research in multiagent systems can address them.     

Differences in learning processes between successful and less successful students in computer-supported collaborative learning in the field of human nutrition and health

This study aimed to investigate the differences in learning processes between successful and less successful pairs of students in computer-supported collaborative learning (CSCL) in the field of human nutrition and health. As part of their regular MSc (and optional BSc) course “Exposure assessment in nutrition and health research” at Wageningen University, 44 students were asked (as an individual pretest) to design and analyze a study which evaluates a certain dietary assessment method. Subsequently, they were asked to discuss their evaluation studies in randomized pairs using a CSCL platform. As an individual posttest, students had to re-design and re-analyze the same evaluation study. The quality of students’ knowledge construction in both tests and characteristics of their learning processes in the CSCL environment were assessed through two coding schemes. Based on their learning outcomes (quality of knowledge construction), pairs of students were divided into two subgroups: successful and less successful students. Next, the learning processes of these subgroups were compared. This study revealed that the learning processes of successful and less successful students in the CSCL environment differed in terms of relevance, width and depth of discussion and justification and reasoning. Based on these findings, recommendations for further research and educational practice are formulated.     

Acceptance of competency-based workplace e-learning systems: Effects of individual and peer learning support

Current endeavors to integrate competency-based learning approaches with e-learning systems designed for delivery of training to adult learners in the workplace are growing. However, academic efforts in examining learners’ perceptions of, and reactions toward, this technology-delivered pedagogical innovation are limited. Drawing together perspectives from the literature on training and instructional design and technology acceptance, this research proposed a conceptual model to examine the influences of perceived individual and social learning support on employees’ acceptance of competency-based e-learning systems. Structural equation modeling and multi-group structural equation modeling techniques were applied to sample data collected from work settings in Mainland China. The results show the positive effects of perceived individual learning support and perceived support for promoting a norm of cooperation on employees’ intention to use the competency-based e-learning system. It was also found that perceived support for enhancing social ties had a negative effect on employees’ behavioral intention. The gender, age, prior experience, and work experience differences in the main effects were also investigated. Besides, qualitative data collected via semi-structured in-depth interview provide complementary evidence to the findings.     

Persistency of excitation and performance of deterministic learning

Recently, a deterministic learning theory was proposed for locally-accurate identification of nonlinear systems. In this paper, we investigate the performance of deterministic learning, including the learning speed and learning accuracy. By analyzing the convergence properties of a class of linear time-varying (LTV) systems, explicit relations between the persistency of excitation (PE) condition (especially the level of excitation) and the convergence properties of the LTV systems are derived. It is shown that the learning speed increases with the level of excitation and decreases with the upper bound of PE. An optimal learning speed is shown to exist. The learning accuracy also increases with the level of excitation, in particular, when the level of excitation is large enough, locally-accurate learning can be achieved to the desired accuracy, whereas low level of PE may result in the deterioration of the learning performance. This paper reveals that the performance analysis of deterministic learning can be established on the basis of classical results on stability and convergence of adaptive control. Simulation studies are included to illustrate the results.     

Group differences in computer supported collaborative learning: Evidence from patterns of Taiwanese students’ online communication

This study explored the differences among online elementary school student groups based on their communication features. Two hundred and ninety-one Taiwanese students, ranging in age from 11 to 12 years old, participated in this study. The students were randomly arranged within-class into three-member groups. Each group was asked to use a collaborative learning system to accomplish a group task generating a shared concept map. The textual discussions in each group during collaboration were collected, coded, categorized, and quantified to profile their communication characteristics. Cluster analysis on the resulting communication characteristics resulted in four types of small student groups, including passive or reticent, frequently off-task, actively participating, and knowledge emphasizing. Most student groups (56%) were found to be relatively passive or reticent. Frequently off-task student groups made a protrusive amount of messages for off-task social purposes. The actively participating student groups were characterized by abundant discussion, particularly for continuing task, managing procedure and coordinating efforts. The distinctive feature of knowledge emphasizing student groups was that they devoted particular attention to task related knowledge. In addition, they performed better in task accomplishment.     

Self-adaptation of learning rate in XCS working in noisy and dynamic environments

An extended classifier system (XCS) is an adaptive rule-based technique that uses evolutionary search and reinforcement learning to evolve complete, accurate, and maximally general payoff map of an environment. The payoff map is represented by a set of condition-action rules called classifiers. Despite this insight, till now parameter-setting problem associated with LCS/XCS has important drawbacks. Moreover, the optimal values of some parameters are strongly influenced by properties of the environment like its complexity, changeability, and the level of noise. The aim of this paper is to overcome some of these difficulties by a self-adaptation of a learning rate parameter, which plays a key role in reinforcement learning, since it is used for updates of classifier parameters: prediction, prediction error, fitness, and action set estimation. Self-adaptive control of prediction learning rate is investigated in the XCS, whereas the fitness and error learning rates remain fixed. Simultaneous self-adaptation of prediction learning rate and mutation rate also undergo experiments. Self-adaptive XCS solves one-step problems in noisy and dynamic environments.