Practical Issues in Temporal Difference Learning

This paper examines whether temporal difference methods for training connectionist networks, such as Sutton's TD() algorithm, can be successfully applied to complex real-world problems. A number of important practical issues are identified and discussed from a general theoretical perspective. These practical issues are then examined in the context of a case study in which TD() is applied to learning the game of backgammon from the outcome of self-play. This is apparently the first application of this algorithm to a complex non-trivial task. It is found that, with zero knowledge built in, the network is able to learn from scratch to play the entire game at a fairly strong intermediate level of performance, which is clearly better than conventional commercial programs, and which in fact surpasses comparable networks trained on a massive human expert data set. This indicates that TD learning may work better in practice than one would expect based on current theory, and it suggests that further analysis of TD methods, as well as applications in other complex domains, may be worth investigating.     

A cyclic CCK8 analogue selective for the cholecystokinin type A receptor: design, synthesis, NMR structure and binding measurements

A cyclic CCK8 analogue, cyclo(29,34)[Dpr(29),Lys(34)]-CCK8 (Dpr=L-2,3-diaminopropionic acid), has been designed on the basis of the NMR structure of the bimolecular complex between the N-terminal fragment of the CCK(A) receptor and its natural ligand CCK8. The conformational features of cyclo(29,34)[Dpr(29),Lys(34)]-CCK8 have been determined by NMR spectroscopy in aqueous solution and in water containing DPC-d(38) micelles (DPC=dodecylphosphocholine). The structure of the cyclic peptide in aqueous solution is found to be in a relaxed conformation, with the backbone and Dpr29 side chain atoms making a planar ring and the N-terminal tripeptide extending approximately along the plane of this ring. In DPC/water, the cyclic peptide adopts a "boat-shaped" conformation, which is more compact than that found in aqueous solution. The cyclic constraint between the Dpr29 side chain and the CCK8 carboxyl terminus (Lys34) introduces a restriction in the backbone conformational freedom. However, the interaction of cyclo(29,34)[Dpr(29),Lys(34)]-CCK8 with the micelles still plays an important role in the stabilisation of the bioactive conformation. A careful comparison of the NMR structure of the cyclic peptide in a DPC micelle aqueous solution with the structure of the rationally designed model underlines that the turn-like conformation in the Trp30-Met31 region is preserved, such that the Trp30 and Met31 side chains can adopt the proper spatial orientation to interact with the CCK(A) receptor. The binding properties of cyclo(29,34)[Dpr(29),Lys(34)]-CCK8 to the N-terminal receptor fragment have been investigated by fluorescence spectroscopy in a micellar environment. Estimates of the apparent dissociation constant, K(d), were in the range of 70-150 nM, with a mean value of 120+/-27 nM. Preliminary nuclear medicine studies on cell lines transfected with the CCK(A) receptor indicate that the sulfated-Tyr derivative of cyclo(29,34)[Dpr(29),Lys(34)]-CCK8 displaces the natural ligand with an IC(50) value of 15 microM.     

Performance analysis of a novel time-domain real-time SAR processor

We test the performance of a new phase-preserving time-domain signum coded SAR processor (SCSP) aimed at real time operations. Raw signal interferometric pairs relevant to the Shuttle Radar Topography X-band Mission are simulated. A full result comparison between SCSP and conventional interferometric products is presented by using simulated canonical and real scenarios. The simulated canonical scene consists of a pyramid and three corner reflectors. The simulated real scene refers to the Mt Etna area in Sicily, Italy. Raw data are simulated both for ascending and descending orbits. Results show that SCSP combined with an iterative phase unwrapping algorithm can generate digital elevation models with good accuracy in spite of the higher phase noise level.     

Peptide-containing aggregates as selective nanocarriers for therapeutics

New nanocarriers are obtained by assembling two amphiphilic monomers: one containing the bioactive peptide CCK8 spaced, by a polydisperse poly(ethylene glycol), from two hydrophobic tails ((C18)2PEG2000CCK8), and the other containing a chelating agent able to give stable radiolabeled indium-111 complexes linked to the same hydrophobic moiety ((C18)2DTPAGlu). The size and shape of the supramolecular aggregates were structurally characterized by dynamic light scattering, small-angle neutron scattering, and cryogenic transmission electronic microscopy. Under the experimental conditions we investigated (pH 7.4 and molar ratio between monomers 30:70), there is the presence of high polydisperse aggregates: rod-like micelles with a radius of approximately 40 A and length >700 A, open bilayer fragments with thickness approximately 65 A, and probably vesicles. The presence of the bioactive peptide well exposed on the external surface of the aggregate allows selective targeting of nanocarriers towards the cholecystokinin receptors overexpressed by the cancerous cells. In vitro binding assays and in vivo biodistribution studies by nuclear medicine experiments using indium-111 are reported. Moreover, preliminary data concerning the drug loading capability of the aggregates and their drug efficiency on the target cells is reported by using the cytotoxic drug doxorubicin. Incubation of receptor-positive and control cells with peptide-containing aggregates filled with doxorubicin shows significantly lower cell survival in receptor-expressing cells relative to the control, for samples incubated in the presence of doxorubicin.     

Rare complications of biliary sepsis

The surgical approach to the acute biliary pathologies also today is often controversial. The choice of the right time to operate an acute patient is based either on personal clinical experiences, either under the statement that waiting for the resolution of the acute process could be preferable in the aim of reduce the surgical risk. This is the almost general tendency. Recently, some interesting articles issued by Swedish and German schools conducted as controlled trials on a great base of cases, try to emphasize the advantages of an early surgical therapy, particularly in the elderly patients. In these ones, in fact, the concomitance of cardiovascular, metabolic and immunodepressive pathologies makes more serious the complications too. In this article, the Authors refer on three clinical cases, all of which were quite different, and in which it was possible to identify a former septic hepato-biliary pathology. All the patients, upon hospital admission showed an acute pattern. In two cases it was an hepatic abscess, accompanied in one case by a "satellite" pulmonary abscess on the right lung. These two were treated conservatively, although by a TC-guided drainage of the liver abscesses. The third case, a localized choleperitoneum (biloma saccatus), underwent an operation. The accurate investigation on the clinical records of Authors' Department since 1980 to 1995 and in particular on the three referred cases seems to confirm that the importance of some complications after acute biliary pathology and their great morbidity must stimulate the surgeons to investigate always on the real causes of all clinical patterns, even if uncommon.     

Spotlight SAR data focusing based on a two-step processing approach

The authors present a new spotlight SAR data-focusing algorithm based on a two-step processing strategy that combines the advantages of two commonly adopted processing approaches: the efficiency of SPECAN algorithms and the precision of stripmap focusing techniques. The first step of the proposed algorithm implements a linear and space-invariant azimuth filtering that is carried out via a deramping-based technique representing a simplified version of the SPECAN approach. This operation allows the authors to perform a bulk azimuth raw data compression and to achieve a pixel spacing smaller than (or equal to) the expected azimuth resolution of the fully focused image. Thus, the azimuth spectral folding phenomenon, typically affecting the spotlight data, is overcome, and the space-variant characteristics of the stripmap system transfer function are preserved. Accordingly, the residual and precise focusing of the SAR data is achieved by applying a conventional stripmap processing procedure requiring a minor modification and implemented in the frequency domain. The extension of the proposed technique to the case of high bandwidth transmitted chirp signals is also discussed. Experiments carried out on real and simulated data confirm the validity of the presented approach, which is mainly focused on spaceborne systems     

Surface and structural disorder in MBE and sputtering deposited Cu thin films revealed by X-ray measurements

Copper thin films have been deposited on Si substrates by molecular beam epitaxy (MBE) at different deposition rates varying from 1 up to 22 Å/s. X-ray reflectivity and θ-2θ measurements have shown that the surface roughness correlation length, the structural disorder and the grain dimensions are strongly affected by the deposition rate. Comparing these results with those obtained for sputtered deposited thin films with a low deposition rate (2.5 Å/s), a clear similarity between the MBE samples deposited with the highest deposition rate and the sputtering Cu films is observed. This result has been interpreted considering the different energies of the particles that approach the substrate in the two deposition techniques.     

Target-selective drug delivery through liposomes labeled with oligobranched neurotensin peptides

The structure and the in vitro behavior of liposomes filled with the cytotoxic drug doxorubicin (Doxo) and functionalized on the external surface with a branched moiety containing four copies of the 8-13 neurotensin (NT) peptide is reported. The new functionalized liposomes, DOPC-NT?Lys(C??)?, are obtained by co-aggregation of the DOPC phospholipid with a new synthetic amphiphilic molecule, NT? Lys(C??)?, which contains a lysine scaffold derivatized with a lipophilic moiety and a tetrabranched hydrophilic peptide, NT8-13, a neurotensin peptide fragment well known for its ability to mimic the neurotensin peptide in receptor binding ability. Dynamic light scattering measurements indicate a value for the hydrodynamic radius (RH) of 88.3±4.4?nm. The selective internalization and cytotoxicity of DOPC-NT? Lys(C??)? liposomes containing Doxo, as compared to pure DOPC liposomes, were tested in HT29 human colon adenocarcinoma and TE671 human rhabdomyosarcoma cells, both of which express neurotensin receptors. Peptide-functionalized liposomes show a clear advantage in comparison to pure DOPC liposomes with regard to drug internalization in both HT29 and TE671 tumor cells: FACS analysis indicates an increase in fluorescence signal of the NT?-liposomes, compared to the DOPC pure analogues, in both cell lines; cytotoxicity of DOPC-NT? Lys(C??)?-Doxo liposomes is increased four-fold with respect to DOPC-Doxo liposomes in both HT29 and TE671 cell lines. These effects could to be ascribed to the higher rate of internalization for DOPC-NT? Lys(C??)?-Doxo liposomes, due to stronger binding driven by a lower dissociation constant of the NT?-liposomes that bind the membrane onto a specific protein, in contrast to DOPC liposomes, which approach the plasma membrane unselectively.     

Comments on “Co-Evolution in the Successful Learning of Backgammon Strategy”

The results obtained by Pollack and Blair substantially underperform my 1992 TD Learning results. This is shown by directly benchmarking the 1992 TD nets against Pubeval. A plausible hypothesis for this underperformance is that, unlike TD learning, the hillclimbing algorithm fails to capture nonlinear structure inherent in the problem, and despite the presence of hidden units, only obtains a linear approximation to the optimal policy for backgammon. Two lines of evidence supporting this hypothesis are discussed, the first coming from the structure of the Pubeval benchmark program, and the second coming from experiments replicating the Pollack and Blair results.     

Digital elevation model generation using ascending and descending ERS-1/ERS-2 tandem data

In this study we show how ascending and descending ERS-1/ERS-2 tandem data can be used to generate a precise digital elevation model (DEM). In particular we present the advantages, for DEM generation, of combining ascending and descending interferometric data pairs with large baselines; unwrapping problems caused by the high fringe rates of these data are minimized by applying a flattening technique based on the use of a data pair with smaller baseline. The presented experiments carried out on the data of an area with steep topography (Etna volcano, Italy) confirm the potentiality of the proposed technique.