Linear Least-Squares Algorithms for Temporal Difference Learning

We introduce two new temporal difference (TD) algorithms based on the theory of linear least-squares function approximation. We define an algorithm we call Least-Squares TD (LS TD) for which we prove probability-one convergence when it is used with a function approximator linear in the adjustable parameters. We then define a recursive version of this algorithm, Recursive Least-Square TD (RLS TD). Although these new TD algorithms require more computation per time-step than do Suttons TD() algorithms, they are more efficient in a statistical sense because they extract more information from training experiences. We describe a simulation experiment showing the substantial improvement in learning rate achieved by RLS TD in an example Markov prediction problem. To quantify this improvement, we introduce the TD error variance of a Markov chain, TD, and experimentally conclude that the convergence rate of a TD algorithm depends linearly on TD. In addition to converging more rapidly, LS TD and RLS TD do not have control parameters, such as a learning rate parameter, thus eliminating the possibility of achieving poor performance by an unlucky choice of parameters.     

Analysis of sperm concentration and motility in a microfluidic device

A home-use device that allows rapid and quantitative sperm quality analysis is desirable but not yet fully realized. To aid this effort, this article presents a microfluidic device capable of quantifying sperm quality in terms of two critical fertility-related parameters—motile sperm concentration and motility. The microdevice produces flow field for sperms to swim against, and sperms that overcame the flow within a specified time are propelled along in a separate channel and counted via resistive pulse technique. Data are compared to two control methods clinically utilized for sperm quality exam—hemocytometer and the sperm quality analyzer. Results reveal the numbers of pulses generated by passage of sperms correlates strongly with the two control methods: pulse number from 0 to 335 corresponds to progressively motile sperm concentrations from 0 to 19 × 10⁶/ml (hemocytometer) and Sperm Motility Index from 0 to 204 (sperm quality analyzer). The microdevice should be applicable to facilitate self-assessment of sperm quality at home.     

African Clouds over the Oslo Opera

Expressive mobile narratives are in the early stages of design and participative formation. We ask what relations and articulations can be brought to play in the design and enactment of a mobile fictional space in which identity is a core motivation and design feature. Through a design case, we share experiences in the building and trial process of an experimental, mobile “social fiction” called NarraHand. We introduce a new category of expressive mobile narrative, the “reflexive place-space narrative,” along with the term Markers of Identity (MOI).     

Greedy algorithms for generalized k-rankings of paths

A k-ranking of a graph is a labeling of the vertices with positive integers 1,2,…,k so that every path connecting two vertices with the same label contains a vertex of larger label. An optimal ranking is one in which k is minimized. Let Pn be a path with n vertices. A greedy algorithm can be used to successively label each vertex with the smallest possible label that preserves the ranking property. We seek to show that when a greedy algorithm is used to label the vertices successively from left to right, we obtain an optimal ranking. A greedy algorithm of this type was given by Bodlaender et al. in 1998 [1] which generates an optimal k-ranking of Pn. In this paper we investigate two generalizations of rankings. We first consider bounded (k,s)-rankings in which the number of times a label can be used is bounded by a predetermined integer s. We then consider kt-rankings where any path connecting two vertices with the same label contains t vertices with larger labels. We show for both generalizations that when G is a path, the analogous greedy algorithms generate optimal k-rankings. We then proceed to quantify the minimum number of labels that can be used in these rankings. We define the bounded rank number to be the smallest number of labels that can be used in a (k,s)-ranking and show for n?2, where i=⌊log₂(s)⌋+1. We define the kt-rank number, to be the smallest number of labels that can be used in a kt-ranking. We present a recursive formula that gives the kt-rank numbers for paths, showing for all an−1<j?an where {an} is defined as follows: a₁=1 and an=⌊((t+1)/t)an−1⌋+1.     

Remote sensing of photosynthetic light-use efficiency across two forested biomes: Spatial scaling

Eddy covariance (EC) measurements have greatly advanced our knowledge of carbon exchange in terrestrial ecosystems. However, appropriate techniques are required to upscale these spatially discrete findings globally. Satellite remote sensing provides unique opportunities in this respect, but remote sensing of the photosynthetic light-use efficiency (ε), one of the key components of Gross Primary Production, is challenging. Some progress has been made in recent years using the photochemical reflectance index, a narrow waveband index centered at 531 and 570 nm. The high sensitivity of this index to various extraneous effects such as canopy structure, and the view observer geometry has so far prevented its use at landscape and global scales. One critical aspect of upscaling PRI is the development of generic algorithms to account for structural differences in vegetation. Building on previous work, this study compares the differences in the PRI: ? relationship between a coastal Douglas-fir forest located on Vancouver Island, British Columbia, and a mature Aspen stand located in central Saskatchewan, Canada. Using continuous, tower-based observations acquired from an automated multi-angular spectro-radiometer (AMSPEC II) installed at each site, we demonstrate that PRI can be used to measure ? throughout the vegetation season at the DF-49 stand (r² = 0.91, p < 0.00) as well as the deciduous site (r² = 0.88, p < 0.00). It is further shown that this PRI signal can be also observed from space at both sites using daily observations from the Moderate Resolution Imaging Spectro-radiometer (MODIS) and a multi-angular implementation of atmospheric correction (MAIAC) (r² = 0.54 DF-49; r² = 0.63 SOA; p < 0.00). By implementing a simple hillshade model derived from airborne light detection and ranging (LiDAR) to approximate canopy shadow fractions (α_s), it is further demonstrated that the differences observed in the relationship between PRI and ε at DF-49 and SOA can be attributed largely to differences in α_s. The findings of this study suggest that algorithms used to separate physiological from extraneous effects in PRI reflectance may be more broadly applicable and portable across these two climatically and structurally different biome types, when the differences in canopy structure are known.     

Animatronic shader lamps avatars

Applications such as telepresence and training involve the display of real or synthetic humans to multiple viewers. When attempting to render the humans with conventional displays, non-verbal cues such as head pose, gaze direction, body posture, and facial expression are difficult to convey correctly to all viewers. In addition, a framed image of a human conveys only a limited physical sense of presence—primarily through the display’s location. While progress continues on articulated robots that mimic humans, the focus has been on the motion and behavior of the robots rather than on their appearance. We introduce a new approach for robotic avatars of real people: the use of cameras and projectors to capture and map both the dynamic motion and the appearance of a real person onto a humanoid animatronic model. We call these devices animatronic Shader Lamps Avatars (SLA). We present a proof-of-concept prototype comprised of a camera, a tracking system, a digital projector, and a life-sized styrofoam head mounted on a pan-tilt unit. The system captures imagery of a moving, talking user and maps the appearance and motion onto the animatronic SLA, delivering a dynamic, real-time representation of the user to multiple viewers.