A time aggregation approach to Markov decision processes

We propose a time aggregation approach for the solution of infinite horizon average cost Markov decision processes via policy iteration. In this approach, policy update is only carried out when the process visits a subset of the state space. As in state aggregation, this approach leads to a reduced state space, which may lead to a substantial reduction in computational and storage requirements, especially for problems with certain structural properties. However, in contrast to state aggregation, which generally results in an approximate model due to the loss of Markov property, time aggregation suffers no loss of accuracy, because the Markov property is preserved. Single sample path-based estimation algorithms are developed that allow the time aggregation approach to be implemented on-line for practical systems. Some numerical and simulation examples are presented to illustrate the ideas and potential computational savings.     

Algorithm for finding all k nearest neighbors

A very simple algorithm for computing all k nearest neighbors in 2-D is presented. The method does not rely on complicated forms of tessellation, it only requires simple data binning for fast range searching. Its applications range from scattered data interpolation to reverse engineering.     

Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system

This paper describes how SWoRD (scaffolded writing and rewriting in the discipline), a web-based reciprocal peer review system, supports writing practice, particularly for large content courses in which writing is considered critical but not feasibly included. To help students gain content knowledge as well as writing and reviewing skills, SWoRD supports the whole cycle of writing, reviews, back-reviews, and rewriting by scaffolding the journal publication process as its authentic practice model. In addition, SWoRD includes algorithms that compute individual reviewer’s review accuracy, which is in turn used to support the various drawbacks of reciprocal peer reviews (e.g., variation in motivation or ability of reviewers). Finally, this paper describes an empirical evaluation showing that the SWoRD approach is effective in improving writing quality in content classes.     

Use of pooled time series in the study of naturally occurring clinical events and problem behavior in a foster care setting.

Pooled time series is an underused analytic technique with the potential to increase researchers' ability to exploit clinical data. This article demonstrates the value of pooled time series by analyzing the behavior of youths in a specialized foster care treatment setting in response to a naturally occurring clinical event: changes in the number of youths living together in a treatment foster care setting. Pooled time series moves beyond typical clinical analyses with an increased capability of controlling statistically for complex within-S effects and with a clinically useful measure of effect size. The complexity of the intra-S data made it virtually impossible to determine the relevant significance (i.e., clinical meaning) of the clinical event by the use of standard n?=?1 visual analysis procedures or standard statistical methods (e.g., chi square). After things such as autocorrelation and individual time trends were statistically controlled, each additional youth increased the number of problematic behaviors by one behavior per youth per day on the Parent Daily Report. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Saturation of subjective reward magnitude as a function of current and pulse frequency.

In 6 rats with electrodes in the medial forebrain bundle, the upper portion of the function relating the experienced magnitude of the reward to pulse frequency was determined at currents ranging from 100 to 1,000 μA. The pulse frequency required to produce an asymptotic level of reward was inversely proportional to current except at the lowest currents and highest pulse frequencies. At a given current, the subjective reward magnitude functions decelerated to an asymptote over an interval in which the pulse frequency doubled or tripled. The asymptotic level of reward was approximately constant for currents between 200 and 1,000 μA but declined substantially at currents at or below 100 μA and pulse frequencies at or above 250 to 400 pulses per second. Results are consistent with the hypothesis that the magnitude of the experienced reward depends only on the number of action potentials generated by the train of pulses in the bundle of reward-relevant axons. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Optoelectronic integration of a GaAs/AlGaAs bistable field effecttransistor (BISFET) and LED

The bistable field effect transistor (BISFET) is a novel inversion-channel switching device exhibiting abrupt current transitions and hysteresis in its output characteristics. The semiconductor structure of the BISFET is compatible with a range of electronic and optoelectronic devices. In this work, integration of a BISFET with an LED is reported. Both devices have been implemented on a single semiconductor substrate using a single fabrication sequence. The BISFET is used to current drive the LED. Abrupt transitions and hysteresis are seen in the optical output from the circuit in the range of gate voltage from 1.75 V to 1.9 V     

A strategy for designing error detection schemes for general datanetworks

The author presents a five-step methodology that provides insight into the order in which errors should be considered when designing an error detection scheme, which types of error detection mechanisms are most effective, and which layer should be responsible for detecting a given type of error     

Two studies of the long-term follow-up of minimal therapist contact treatments of vascular and tension headache.

Tension and vascular headache patients, initially treated with biofeedback and/or relaxation training in either a minimal therapist contact protocol (3 visits) or an intensive individual protocol (10 or 16 visits) were followed-up prospectively for 2 years. In the first study, for the first 6 months of follow-up, half of all patients continued to keep headache diaries and were seen monthly and the other half had only minimal contact. The results at 1-year follow-up, based on 4 weeks of daily headache diaries, revealed equally good maintenance from both treatment protocols and from both follow-up conditions. In Study 2, we found that patients remained improved over pretreatment baseline levels at the 2-year follow-up regardless of initial treatment intensity. Approximately three quarters of vascular patients who were initially improved at posttreatment remained improved at 2 years. (PsycINFO Database Record (c) 2010 APA, all rights reserved)