Driving as a human: a track learning based adaptable architecture for a car racing controller

We present the evolution and current state of the Mr. Racer car racing controller that excelled at the corresponding TORCS competitions of the last years. Although several heuristics and black-box optimization methods are employed, the basic idea of the controller architecture has been to take over many of the mechanisms human racing drivers apply. They learn the track geometry, plan ahead, and wherever necessary, adapt their plans to the current circumstances quickly. Mr. Racer consists of several modules that have partly been adapted and optimized separately, where the final tuning is usually done with respect to a certain racing track during the warmup phase of the TORCS competitions. We also undertake an experimental evaluation that investigates how the controller profits from adding some of the modules to a basic configuration and which modules are most important for reaching the best possible performance.     

An evolutionary tuned driving system for virtual car racing games: The AUTOPIA driver

This work presents a driving system designed for virtual racing situations. It is based on a complete modular architecture capable of automatically driving a car along a track with or without opponents. The architecture is composed of intuitive modules, with each one being responsible for a basic aspect of car driving. Moreover, this modularity of the architecture will allow us to replace or add modules in the future as a way to enhance particular features of particular situations. In the present work, some of the modules are implemented by means of hand‐designed driving heuristics, whereas modules responsible for adapting the speed and direction of the vehicle to the track's shape, both critical aspects of driving a vehicle, are optimized by means of a genetic algorithm that evaluates the performance of the controller in four different tracks to obtain the best controller in a large number of situations; the algorithm also penalizes controllers that go out of the track, lose control, or get damaged. The evaluation of the performance is done in two ways. First, in runs with and without adversaries over several tracks. And second, the architecture was submitted as a participant to the 2010 Simulated Car Racing Competition, which in end won laurels. © 2012 Wiley Periodicals, Inc.     

An embedded testbed architecture to evaluate autonomous car driving

The progress in autonomous car driving or also called Advanced Driving Assistance Systems (ADAS) has recently received an increasing attention in car industries. Thus, many autonomous car models are recently developed and some are even commercialized. However, they are still not ready to enter our daily life since they lack efficiency and reliability. In this sense, to obtain efficient and reliable systems, the evaluation process plays an important role. Nowadays, the evaluation is strongly related to the number of kilometers of drive. However, in order to make a comprehensive evaluation of such systems, in addition to the number of kilometers, each component should also be tested. In this study, we focused on developing a system that is able to evaluate each component of ADAS. We defined a versatile architecture that simplifies the evaluation of different types of ADAS. Several evaluation scenarios are shown and discussed.     

Autonomous car driving by a humanoid robot

Enabling a humanoid robot to drive a car requires the development of a set of basic primitive actions. These include walking to the vehicle, manually controlling its commands (e.g., ignition, gas pedal, and steering) and moving with the whole body to ingress/egress the car. We present a sensor‐based reactive framework for realizing the central part of the complete task, consisting of driving the car along unknown roads. The proposed framework provides three driving strategies by which a human supervisor can teleoperate the car or give the robot full or partial control of the car. A visual servoing scheme uses features of the road image to provide the reference angle for the steering wheel to drive the car at the center of the road. Simultaneously, a Kalman filter merges optical flow and accelerometer measurements to estimate the car linear velocity and correspondingly compute the gas pedal command for driving at a desired speed. The steering wheel and gas pedal reference are sent to the robot control to achieve the driving task with the humanoid. We present results from a driving experience with a real car and the humanoid robot HRP‐2Kai. Part of the framework has been used to perform the driving task at the DARPA Robotics Challenge.     

Advanced overtaking behaviors for blocking opponents in racing games using a fuzzy architecture

In car racing, blocking refers to maneuvers that can prevent, disturb or completely block an overtaking action by an incoming car. In this paper, we present an advanced overtaking behavior that is able to deal with opponents implementing blocking strategies of various difficulty level. The behavior we developed has been integrated in an existing fuzzy-based architecture for driving simulated cars and tested using The Open Car Racing Simulator (TORCS). We compared a driver implementing our overtaking strategy against four of the bots available in the TORCS distribution and Simplix, a state-of-the-art driver which won several competitions. The comparison was carried out against opponents implementing three blocking strategies of increasing difficulty and two different scenarios: (i) a basic scenario with one opponent on a straight stretch to overtake as quickly as possible; (ii) an advanced scenario involving a race on a non-trivial track against several opponents. The results from the basic scenario show that our strategy can always overtake the opponent car; in particular, our strategy is slightly more risky than the other ones and may result in a little damage, however, all the other controllers show a more careful and safe policy that often prevents them to complete an overtaking maneuver. When racing against several opponents on complex tracks, our strategy results in the best trade-off between the time spent being blocked by an opponent ahead and the number of overtaking maneuvers completed.     

An integrated architecture for future car generations

The DECOS architecture is an integrated architecture that builds upon the validated services of a time-triggered network, which serves as a shared resource for the communication activities of more than one application subsystem. In addition, encapsulated partitions are used to share the computational resources of Electronic Control Units (ECUs) among software modules of multiple application subsystems. This paper investigates the benefits of the DECOS architecture as an electronic infrastructure for future car generations. The shift to an integrated architecture will result in quantifiable cost reductions in the areas of system hardware cost and system development. In the paper we present a current federated Fiat car E/E architecture and discuss a possible mapping to an integrated solution based on the DECOS architecture. The proposed architecture provides a foundation for mixed criticality integration with both safety-critical and non safety-critical subsystems. In particular, this architecture supports applications up to the highest criticality classes (10^?9 failures per hour), thereby taking into account the emerging dependability requirements of by-wire functionality in the automotive industry.     

A portable AWT/Swing architecture for Java game development

Recently, the performance of Java platforms has been greatly improved to satisfy the requirements for game development. However, the rendering performance of Java 1.1, which is still used by about one‐third of current Web browser users, is not sufficient for high‐profile games. Therefore, practically, Java game developers, especially those who use applets, have to take this into consideration in most environments. In order to solve the above problems, this paper proposes a portable window toolkit architecture called the CYC Window Toolkit (CWT) with the ability to: (1) reach high rendering performance particularly in Java 1.1 applications and applets when using DirectX to render widgets in CWT; (2) support AWT/Swing compatible widgets, so hence the CWT can be easily applied to existing Java games; (3) define a general architecture that supports multiple graphics libraries such as AWT, DirectX and OpenGL, multiple virtual machines such as Java VM and .NET CLR, and multiple operating systems (OSs) such as Microsoft Windows, Mac OS and UNIX‐based OSs; (4) provide programmers with one‐to‐one mapping APIs to directly manipulate DirectX objects and other game‐related properties. The CWT has also been applied to an online Java game system to demonstrate the proposed architecture. Copyright © 2006 John Wiley & Sons, Ltd.     

Time perception as a workload measure in simulated car driving

In experimental studies using flight simulations subjects’ duration estimates have shown to be an effective indicator of cognitive task demands. In this study we wanted to find out whether subjective time perception could serve as a measure of cognitive workload during simulated car driving. Participants drove on a round course of a driving simulator consisting of three different environments with different levels of task demands. Drivers were required to perform a time-production task while driving the vehicle. Electrodermal activity and subjective ratings of mental workload (SWAT) were recorded simultaneously. The length of produced intervals increased significantly in more complex driving situations, as did electrodermal activity and subjective ratings of mental workload. Thus, time production is a valid indicator of cognitive involvement in simulated driving and could become a valid method to measure the current mental workload of car drivers in various traffic situations.     

基于经验迁移的赛车学习预测控制研究

为提高车辆控制算法对不同道路的适应能力,在原有学习预测控制算法的基础上,本文提出一种基于经验迁移的赛车学习预测控制策略.基于所建立的赛车曲线坐标系模型,记录小车在历史赛道上的行驶轨迹,将其作为采样安全集.采样安全集蕴含了车辆行驶的经验信息.在新赛道上,通过与采样安全集内曲率相近的轨迹进行特征匹配,找出新赛道的虚拟路径跟踪轨迹.然后,对虚拟路径跟踪轨迹附近的采样点进行坐标变换,将历史轨迹转换为新赛道的虚拟采样轨迹,实现对历史赛道上的行驶经验的迁移.构造了迁移学习预测控制(TLMPC),使小车在新的赛道上能够通过学习预测控制器以更快的速度行驶.本文在4个典型赛道上进行了仿真,结果表明所设计的控制策略控制效果有明显提升.与LMPC相比, 10次迭代结果中单圈耗时至少减少了1.2 s.     

Evolutionary multiobjective industrial design: the case of a racing car tire-suspension system

When dealing with multiobjective optimization (MO) of the tire-suspension system of a racing car, a large number of design variables and a large number of objectives have to be taken into account. Two different models have been used, both validated on data coming from an instrumented car, a differential equation-based physical model, and a neural network purely numerical model. Up to 23 objective functions have been defined, at least 14 of which are in strict conflict of each other. The equivalent scalar function based and the objective-as-constraint formulations are intentionally avoided due to their well-known limitations. A fuzzy definition of optima, being a generalization of Pareto optimality, is applied to the problem. The result of such an approach is that subsets of Pareto optimal solutions (on such a problem, a big portion of the entire search space) can be properly selected as a consequence of input from the designer. The obtained optimal solutions are compared with the reference vehicle and with the optima previously obtained with design of experiment techniques and different MO optimization strategies. The proposed strategy improves both the reference (actual) car and previously obtained optima (scalar preference function) in the majority of objectives with technically significant improvements. Moreover, the strategy offers an univoque criterion for the choice among tradeoff solutions in the 14-dimensional objective space. The problem is used as a test of a proposed optimal design strategy for industrial problems, integrating differential equation and neural networks modeling, design of experiments, MO, and fuzzy optimal-based decision making. Such a linked approach gives also a unified view of where to concentrate the computational effort.     

Occlusion-free animation of driving routes for car navigation systems

This paper presents a method for occlusion-free animation of geographical landmarks, and its application to a new type of car navigation system in which driving routes of interest are always visible. This is achieved by animating a nonperspective image where geographical landmarks such as mountain tops and roads are rendered as if they are seen from different viewpoints. The technical contribution of this paper lies in formulating the nonperspective terrain navigation as an inverse problem of continuously deforming a 3D terrain surface from the 2D screen arrangement of its associated geographical landmarks. The present approach provides a perceptually reasonable compromise between the navigation clarity and visual realism where the corresponding nonperspective view is fully augmented by assigning appropriate textures and shading effects to the terrain surface according to its geometry. An eye tracking experiment is conducted to prove that the present approach actually exhibits visually-pleasing navigation frames while users can clearly recognize the shape of the driving route without occlusion, together with the spatial configuration of geographical landmarks in its neighborhood.     

A rule-based support system for dissonance discovery and control applied to car driving

This paper is based on the concept of dissonance, that is, gaps or conflicts existing in a specific knowledge base or among different knowledge bases. It presents a rule-based system that assists human operators in dissonance discovery and control by taking into account two kinds of dissonance, i.e., affordance to study conflicts of use, and inconsistencies to study conflicts of intention and action, through the analysis of cognitive behavior implemented in knowledge bases. This system elaborates the knowledge base composed of rules, and analyzes the knowledge content to discover new knowledge by creating additional rules, or to identify inconsistencies when conflicts between rules occur. The affordance discovery control process uses a deductive and an inductive reasoning algorithm of which the aim is to establish new rules using existing ones. The inconsistency discovery control process applies an abductive reasoning algorithm in order to determine contradictory rules when existing rules may result in opposite intentions being accomplished. Two groups of inconsistencies are addressed: interferences involving several decision makers, and contradictions involving the same decision maker. A knowledge acquisition control process facilitates the creation of the initial rules that contain parameters such as intentions relating to the goals to be achieved, actions to be performed to achieve these intentions, objects used to carry out these actions and the decision makers who execute these actions using the corresponding objects. A feasibility study taking into account five rule bases relating to the manual use of an Automated Speed Control System (ASCS), the automated control of the car speed by the ASCS, the manual control of aquaplaning, the manual control of the car speed, and the manual control of car fuel consumption is proposed to validate the rule-based support system.     

Adaptive user modelling in car racing games using behavioural and physiological data

Personalised content adaptation has great potential to increase user engagement in video games. Procedural generation of user-tailored content increases the self-motivation of players as they immerse themselves in the virtual world. An adaptive user model is needed to capture the skills of the player and enable automatic game content altering algorithms to fit the individual user. We propose an adaptive user modelling approach using a combination of unobtrusive physiological data to identify strengths and weaknesses in user performance in car racing games. Our system creates user-tailored tracks to improve driving habits and user experience, and to keep engagement at high levels. The user modelling approach adopts concepts from the Trace Theory framework; it uses machine learning to extract features from the user’s physiological data and game-related actions, and cluster them into low level primitives. These primitives are transformed and evaluated into higher level abstractions such as experience, exploration and attention. These abstractions are subsequently used to provide track alteration decisions for the player. Collection of data and feedback from 52 users allowed us to associate key model variables and outcomes to user responses, and to verify that the model provides statistically significant decisions personalised to the individual player. Tailored game content variations between users in our experiments, as well as the correlations with user satisfaction demonstrate that our algorithm is able to automatically incorporate user feedback in subsequent procedural content generation.     

基于博弈论的多车智能驾驶交互决策综述

智能驾驶是交通和汽车领域未来发展的重要方向,决策规划作为智能驾驶系统中的关键模块,一直是其重点研究领域之一.当前的研究热点正在从单车智能驾驶决策向混行交通场景下的多车智能驾驶决策进行拓展,因此,需要在复杂动态场景和多并行任务下生成符合车辆动力学且不与道路边界和其他交通参与者发生碰撞的高质量轨迹.多车混行驾驶是对道路时空资源的竞争性使用,博弈论可为多车交互决策提供重要的理论与技术手段.对此,应用博弈论方法进行智能驾驶决策研究的综述,基于滚动时域、微分博弈和马尔科夫博弈这3类常用的博弈模型,对现有相关研究进行归类总结和分析.首先简要介绍博弈论基础知识;其次,总结常见的智能驾驶场景并分析各场景下交互决策的核心问题;然后,通过3种不同的博弈模型对多车交互决策进行建模,分别介绍它们的求解算法和思路及相关的研究工作;最后,介绍相关的仿真实验和测试方法,同时也对未来的技术发展和挑战给出见解.     

Data‐driven detailed hair animation for game characters

We propose a data‐driven method to realize high‐quality detailed hair animations in interactive applications like games. By devising an error metric method to evaluate hair animation similarities, we take hair features into consideration as much as possible. We also propose a novel database construction algorithm based on Secondary Motion Graph. Our algorithm can improve the efficiency of such graphs to reduce redundant data and also achieves visually smooth connection of two animation clips while taking into consideration their future motions. The costs for the run‐time process using our Secondary Motion Graph are relatively low, allowing real‐time interactive operations. Copyright © 2016 John Wiley & Sons, Ltd.     

A novel modular Q-learning architecture to improve performance under incomplete learning in a grid soccer game

Multi-agent reinforcement learning methods suffer from several deficiencies that are rooted in the large state space of multi-agent environments. This paper tackles two deficiencies of multi-agent reinforcement learning methods: their slow learning rate, and low quality decision-making in early stages of learning. The proposed methods are applied in a grid-world soccer game. In the proposed approach, modular reinforcement learning is applied to reduce the state space of the learning agents from exponential to linear in terms of the number of agents. The modular model proposed here includes two new modules, a partial-module and a single-module. These two new modules are effective for increasing the speed of learning in a soccer game. We also apply the instance-based learning concepts, to choose proper actions in states that are not experienced adequately during learning. The key idea is to use neighbouring states that have been explored sufficiently during the learning phase. The results of experiments in a grid-soccer game environment show that our proposed methods produce a higher average reward compared to the situation where the proposed method is not applied to the modular structure.