A time aggregation approach to Markov decision processes

We propose a time aggregation approach for the solution of infinite horizon average cost Markov decision processes via policy iteration. In this approach, policy update is only carried out when the process visits a subset of the state space. As in state aggregation, this approach leads to a reduced state space, which may lead to a substantial reduction in computational and storage requirements, especially for problems with certain structural properties. However, in contrast to state aggregation, which generally results in an approximate model due to the loss of Markov property, time aggregation suffers no loss of accuracy, because the Markov property is preserved. Single sample path-based estimation algorithms are developed that allow the time aggregation approach to be implemented on-line for practical systems. Some numerical and simulation examples are presented to illustrate the ideas and potential computational savings.     

Algorithm for finding all k nearest neighbors

A very simple algorithm for computing all k nearest neighbors in 2-D is presented. The method does not rely on complicated forms of tessellation, it only requires simple data binning for fast range searching. Its applications range from scattered data interpolation to reverse engineering.     

Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system

This paper describes how SWoRD (scaffolded writing and rewriting in the discipline), a web-based reciprocal peer review system, supports writing practice, particularly for large content courses in which writing is considered critical but not feasibly included. To help students gain content knowledge as well as writing and reviewing skills, SWoRD supports the whole cycle of writing, reviews, back-reviews, and rewriting by scaffolding the journal publication process as its authentic practice model. In addition, SWoRD includes algorithms that compute individual reviewer’s review accuracy, which is in turn used to support the various drawbacks of reciprocal peer reviews (e.g., variation in motivation or ability of reviewers). Finally, this paper describes an empirical evaluation showing that the SWoRD approach is effective in improving writing quality in content classes.     

Sexualities, technologies, and the teaching of writing: A critical overview

In this introduction to a special issue of Computers and Composition, the authors critically review current literature on computer-assisted writing pedagogies that grapple with issues of sexuality. Although this body of work is small, it points to provocative ways to develop our students’ critical and rhetorical sensibilities about the constructions of sexuality in our culture. Further, such work innovatively addresses the place of networked communication technologies in the interrogation of such constructions. The authors conclude with both an introduction of the work in this special issue that addresses the intersection of sexuality studies and computer-assisted writing studies and with a call for additional work in this field.     

A parametric interpolator with confined chord errors, acceleration and deceleration for NC machining

Parametric interpolation has many advantages over linear interpolation in machining curves. Real time parametric interpolation research so far has addressed achieving a uniform feed rate, confined chord errors and jerk limited trajectory planning. However, simultaneous consideration of confined chord errors that respect the acceleration and deceleration capabilities of the machine has not been attempted. In this paper, the offline detection of feed rate sensitive corners is proposed. The velocity profile in these zones is planned so that chord errors are satisfied while simultaneously accommodating the machine's acceleration and deceleration limits. Outside the zone of the feed rate sensitive corners, the feed rate is planned using the Taylor approximation. Simulation results indicate that the offline detection of feed rate sensitive corners improves parametric interpolation. For real time interpolation, the parametric curve information can be augmented with the detected feed rate sensitive corners that are stored in 2×2 matrices.     

A quantitative analysis of the effectiveness of laparascopy and endoscopy virtual reality simulators

The increasing use of virtual reality (VR) simulators in surgical training makes it imperative that definitive studies be performed to assess their training effectiveness. Indeed in this paper we report the meta-analysis of the efficacy of virtual reality simulators in (1) the transference of skills from the simulator training environment to the operating room and (2) their ability to discriminate between the experience levels of its users. The task completion time and the error score were the two study outcomes collated and analyzed in this meta-analysis. Sixteen studies were identified from a computer-based literature search (1996-2004). The meta-analysis of the random-effects model (because of the heterogeneity of the data) revealed that training on virtual reality simulators did lessen the time taken to complete a given surgical task as also clearly differentiate between the experienced and the novice trainees. Meta-analytic studies such as the one reported here would be very helpful in the planning and setting up of surgical training programs and for the establishment of reference ‘learning curves’ for a specific simulator and surgical task. If any such programs already exist they can then indicate the improvements to be made in the simulator used such as providing for more variety in their case scenarios based on the state and/or rate of learning of the trainee.     

Determination of optimal feedback terminal controllers for general boundary conditions using generating functions

Given a nonlinear system and a performance index to be minimized, we present a general approach to expressing the finite time optimal feedback control law applicable to different types of boundary conditions. Starting from the necessary conditions for optimality represented by a Hamiltonian system, we solve the Hamilton-Jacobi equation for a generating function for a specific canonical transformation. This enables us to obtain the optimal feedback control for fundamentally different sets of boundary conditions only using a series of algebraic manipulations and partial differentiations. Furthermore, the proposed approach reveals an insight that the optimal cost functions for a given dynamical system can be decomposed into a single generating function that is only a function of the dynamics plus a term representing the boundary conditions. This result is formalized as a theorem. The whole procedure provides an advantage over methods rooted in dynamic programming, which require one to solve the Hamilton-Jacobi-Bellman equation repetitively for each type of boundary condition. The cost of this favorable versatility is doubling the dimension of the partial differential equation to be solved.