Random Sampling of States in Dynamic Programming

We combine three threads of research on approximate dynamic programming: sparse random sampling of states, value function and policy approximation using local models, and using local trajectory optimizers to globally optimize a policy and associated value function. Our focus is on finding steady-state policies for deterministic time-invariant discrete time control problems with continuous states and actions often found in robotics. In this paper, we describe our approach and provide initial results on several simulated robotics problems.      

A longitudinal examination of fear reactions in victims of rape.

Investigated fear reactions in rape victims for 1 yr following their assaults. 150 female victims, over 15 yrs of age, seen approximately 2 wks after the assault and at 1, 2, 4, 8, and 12 mo postrape, were compared with a matched control group of nonvictims seen at the same intervals. To control for the effects of repeated testing, 3 additional groups of victims were assessed only once at either 2, 4, or 8 mo postrape. All participants completed the Modified Fear Survey Schedule (MFS), which yielded a total fearfulness index as well as 6 subscale scores: rape fears, animal fears, classical fears, social–interpersonal fears, tissue-damage fears, and miscellaneous fears. Following the assault, victims were significantly more fearful than nonvictim controls as indicated by their overall MFS score and most of the subscale scores. Although their overall fearfulness declined somewhat and stabilized by 2 mo postassault, victims remained significantly more fearful than nonvictim controls at 12 mo postassault. The rape fears and classical fears subscales seemed to contribute most to this elevation. Results from the single-testing victim groups indicated that repeated assessment had no effect on participants' scores. (23 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Victims of rape: Repeated assessment of depressive symptoms.

115 rape victims (aged 15–71 yrs) were examined at 2 wks and at 1, 2, 4, 8, and 12 mo after the assault. A matched control group of 87 nonvictims was seen at the same intervals. To control for the effects of repeated testing, 3 additional groups of 22–26 victims were assessed only once at 2, 4, or 8 mo postrape. All Ss completed the Beck Depression Inventory and the Hamilton Psychiatric Rating Scale for Depression. Depressive symptoms were significantly higher in victims of rape than in nonvictim controls following the assault. By 4 mo postrape, depressive symptoms in the victim group had diminished to the level shown by the nonvictim control group. Results from the single-testing victim groups indicated that repeated assessment affected the self-report measure, but not the interviewer-rated measure. Variables reflecting pre-rape functioning were more predictive of continued problems with depressive symptoms than were demographic variables or variables associated with the rape and its aftermath. (17 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Locally Weighted Learning

This paper surveys locally weighted learning, a form of lazy learning and memory-based learning, and focuses on locally weighted linear regression. The survey discusses distance functions, smoothing parameters, weighting functions, local model structures, regularization of the estimates and bias, assessing predictions, handling noisy data and outliers, improving the quality of predictions by tuning fit parameters, interference between old and new data, implementing locally weighted learning efficiently, and applications of locally weighted learning. A companion paper surveys how locally weighted learning can be used in robot learning and control.     

The Parti-game Algorithm for Variable Resolution Reinforcement Learning in Multidimensional State-spaces

Parti-game is a new algorithm for learning feasible trajectories to goal regions in high dimensional continuous state-spaces. In high dimensions it is essential that neither planning nor exploration occurs uniformly over a state-space. Parti-game maintains a decision-tree partitioning of state-space and applies techniques from game-theory and computational geometry to efficiently and adaptively concentrate high resolution only on critical areas. The current version of the algorithm is designed to find feasible paths or trajectories to goal regions in high dimensional spaces. Future versions will be designed to find a solution that optimizes a real-valued criterion. Many simulated problems have been rested, ranging from two-dimensional to nine-dimensional state-spaces, including mazes, path planning, non-linear dynamics, and planar snake robots in restricted spaces. In all cases, a good solution is found in less than ten trials and a few minutes.     

Social adjustment in victims of sexual assault.

93 rape victims (aged 15–71 yrs) were seen approximately 2 wks after the assault and at 1, 2, 4, 8, and 12 mo postrape. A matched control group of nonvictims was seen at the same intervals. To control for the effects of repeated testing, 3 additional groups with 72 victims were assessed only once at either 2, 4, or 8 mo postrape. All Ss completed the Social Adjustment Scale–Self Report, which yielded a total adjustment score as well as subscale scores for 5 areas of adjustment: Work; Economic; Social and Leisure; Marital, Parental, and Family Unit; and Extended Family. Ss exhibited disruption in overall social adjustment and most of the subscale roles for the first few months following their assaults. By 4 mo postrape, most of the subscales had stabilized at levels similar to the nonvictims'. Work adjustment continued to be affected through 8 mo postrape. Single-testing control victims appeared to have more problems in social adjustment, particularly with regard to relationships with relatives. (14 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Locally Weighted Learning for Control

Lazy learning methods provide useful representations and training algorithms for learning about complex phenomena during autonomous adaptive control of complex systems. This paper surveys ways in which locally weighted learning, a type of lazy learning, has been applied by us to control tasks. We explain various forms that control tasks can take, and how this affects the choice of learning paradigm. The discussion section explores the interesting impact that explicitly remembering all previous experiences has on the problem of learning to control.     

An assessment of long-term reaction to rape.

Examined reactions to rape by interviewing 27 female rape victims at least 1 yr after the assault and assessing their current functioning through such measures as the Beck Depression Inventory and Profile of Mood States. Victims were significantly more depressed and reported less pleasure in daily activities than 26 matched nonvictim controls. Ss who had been victims of sudden violent attacks by complete strangers showed the most severe reactions, being even more depressed, fatigued, and fearful and getting less satisfaction from activities than victims of other types of assaults. (14 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Implementing projection pursuit learning

This paper examines the implementation of projection pursuit regression (PPR) in the context of machine learning and neural networks. We propose a parametric PPR with direct training which achieves improved training speed and accuracy when compared with nonparametric PPR. Analysis and simulations are done for heuristics to choose good initial projection directions. A comparison of a projection pursuit learning network with a single hidden-layer sigmoidal neural network shows why grouping hidden units in a projection pursuit learning network is useful. Learning robot arm inverse dynamics is used as an example problem.     

An examination of differences between multiple- and single-incident victims of sexual assault.

Examined differences between 25 multiple- and 25 single-incident victims of sexual assault using demographic data and a 117-item interview that assessed functioning prior to the current rape. Multiple-incident Ss were poorer and more transient than were single-incident Ss. Multiple-incident Ss also had a history of more frequent victimization other than rape and were significantly more dysfunctional in their personal and interpersonal adjustment. They reported more problems with depression, and 52% had attempted suicide. Comparisons of events leading up to and surrounding the current rape for these 2 groups were not significant. (10 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)