Reinforcement learning, spike-time-dependent plasticity, and the BCM rule

Learning agents, whether natural or artificial, must update their internal parameters in order to improve their behavior over time. In reinforcement learning, this plasticity is influenced by an environmental signal, termed a reward, that directs the changes in appropriate directions. We apply a recently introduced policy learning algorithm from machine learning to networks of spiking neurons and derive a spike-time-dependent plasticity rule that ensures convergence to a local optimum of the expected average reward. The approach is applicable to a broad class of neuronal models, including the Hodgkin-Huxley model. We demonstrate the effectiveness of the derived rule in several toy problems. Finally, through statistical analysis, we show that the synaptic plasticity rule established is closely related to the widely used BCM rule, for which good biological evidence exists.     

A Cut-Based Algorithm for the Nonlinear Dual of the Minimum Cost Network Flow Problem

We consider a convex, or nonlinear, separable minimization problem with constraints that are dual to the minimum cost network flow problem. We show how to reduce this problem to a polynomial number of minimum s,t-cut problems. The solution of the reduced problem utilizes the technique for solving integer programs on monotone inequalities in three variables, and a so-called proximity-scaling technique that reduces a convex problem to its linear objective counterpart. The problem is solved in this case in a logarithmic number of calls, O(log U), to a minimum cut procedure, where U is the range of the variables. For a convex problem on n variables the minimum cut is solved on a graph with O(n²) nodes. Among the consequences of this result is a new cut-based scaling algorithm for the minimum cost network flow problem. When the objective function is an arbitrary nonlinear function we demonstrate that this constrained problem is solved in pseudopolynomial time by applying a minimum cut procedure to a graph on O(nU) nodes.     

How to allocate review tasks for robust ranking

In the process of reviewing and ranking projects by a group of reviewers, the allocation of the subset of projects to each reviewer has major impact on the robustness of the outcome ranking. We address here this problem where each reviewer is assigned, out of the list of all projects, a subset of up to k projects. Each individual reviewer then ranks and compares all pairs of k projects. The k-allocation problem is to determine an allocation of up to k projects to each reviewer, that lie within the expertise set of the reviewer, so that the resulting union of reviewed projects has certain desirable properties. The k-complete problem is a k-allocation with the property that all pairs of projects have been compared by at least one reviewer. A k-complete allocation is desirable as otherwise there may be projects that were not compared by any reviewer, leading to possible adverse properties in the outcome ranking. When a k-complete allocation cannot be achieved, one might settle for other properties. One basic requirement is that each pair of projects is comparable via a ranking path which is a sequence of pairwise rankings of projects implying a comparison of all pairs on the path. A k-allocation with a ranking path between each pair is the connectivity-k-aloc. Since the robustness of relative comparisons deteriorates with increased length of the ranking path, another goal is that between each pair of projects there will be at least one ranking path that has at most two hops or q hops for fixed values of q. An alternative means for increasing the robustness of the ranking is to use a k-allocation with at least p disjoint ranking paths between each pair. We model all these problems as graph problems. We demonstrate that the connectivity-k-aloc problem is polynomially solvable, using matroid intersection; we prove that the k-complete problem is NP-hard unless k = 2; and we provide approximation algorithms for a related optimization problem. All other variants are shown to be NP-complete for all values of k ≥ 2.     

New Contact Sex Pheromone Components of the German Cockroach, Blattella germanica, Predicted from the Proposed Biosynthetic Pathway

Upon contacting the cuticle of a sexually mature female, a male German cockroach exhibits a characteristic courtship behavior: he turns away from the female and raises his wings, thereby exposing tergal glands. The glandular secretion stimulates the female to mount the male and feed, thus positioning her appropriately for copulation. A multi-component contact sex pheromone produced by females is responsible for eliciting courtship behavior. The most abundant pheromone components are 3,11-dimethylnonacosan-2-one and 3,11-dimethylheptacosan-2-one, oxidation products of the abundant hydrocarbon analogs 3,11-dimethylnonacosane and 3,11-dimethylheptacosane, respectively. The C₂₉-dimethyl ketone is thought to be further metabolized to two less abundant pheromone components, 29-hydroxy-3,11-dimethylnonacosan-2-one and 29-oxo-3,11-dimethylnonacosan-2-one. Based on this proposed biosynthetic pathway of pheromone production, we hypothesized that 3,11-dimethylheptacosan-2-one also would be oxidized to give two candidate pheromone components, 27-hydroxy-3,11-dimethylheptacosan-2-one, and 27-oxo-3,11-dimethylheptacosan-2-one. By using bioassay-guided fractionation and chemical analyses of cuticular extracts of virgin females and synthesis of the (3S,11S)-isomer of each of the two predicted pheromone components, we showed that the epicuticle of the German cockroach does indeed contain these two compounds. The contact sex pheromone of the female German cockroach, thus may consist of at least six biosynthetically related components.