Selecting and assessing challenge problems

Abstract

Organisations conducting research programs often focus the work of their scientists and technologists on challenge problems (CPs). These challenges are designed to ensure that progress is measurable and relevant to the goals of the program sponsor. Generating and selecting pertinent CPs is difficult, as is assessing their value. We describe a method of generating and selecting CPs and its application in a highly collaborative, multi-organisation research program. Thirty-eight biologists, chemists, mathematicians and computer scientists across academic, commercial and government organisations generated and ranked their top choices from among 12 richly described candidate challenge problems. A ranked-choice voting formula was applied. Five CPs were highly scored; the remaining seven were distributed across a lower range of scores. The program sponsor subsequently directed researchers to address six CPs, including the elected five. Analysis of the rationales that participants offered for their CP rankings revealed four domain-independent dimensions of value: capability, speed, impact and synergy. These dimensions of value can help managers of interdisciplinary research programs systematically select a portfolio of CPs that will efficiently apply utilise resources towards program goals and facilitate measurement of scientific progress.     

Introduction To Probability and Statistical Decision Theory

A nonparametric sequential rank correlation test for independence consisting of a generalized sequential probability ratio test is investigated through the Monte Carlo method. The test is found to be satisfactory in terms of the power function. Comparison of the expected sample size of the test and three other tests including the fixed sample size test suggests that the sequential rank test is also efficient.     

The Role of Cholesterol on Triterpenoid Saponin-Induced Endolysosomal Escape of a Saporin-Based Immunotoxin

Cholesterol seems to play a central role in the augmentation of saporin-based immunotoxin (IT) cytotoxicity by triterpenoid saponins. Endolysosomal escape has been proposed as one mechanism for the saponin-mediated enhancement of targeted toxins. We investigated the effects of lipid depletion followed by repletion on Saponinum album (SA)-induced endolysosomal escape of Alexa Fluor labelled saporin and the saporin-based immunotoxin OKT10-SAP, directed against CD38, in Daudi lymphoma cells. Lipid deprived cells showed reduced SA-induced endolysosomal escape at two concentrations of SA, as determined by a flow cytometric method. The repletion of membrane cholesterol by low density lipoprotein (LDL) restored SA-induced endolysosomal escape at a concentration of 5 µg/mL SA but not at 1 µg/mL SA. When LDL was used to restore the cholesterol levels in lipid deprived cells, the SA augmentation of OKT10-SAP cytotoxicity was partially restored at 1 µg/mL SA and fully restored at 5 µg/mL SA. These results suggest that different mechanisms of action might be involved for the two different concentrations of SA and that endosomal escape may not be the main mechanism for the augmentation of saporin IT cytotoxicity by SA at the sub-lytic concentration of 1 µg/mL SA.     

How verified (or tested) is my code? Falsification-driven verification and testing

Formal verification has advanced to the point that developers can verify the correctness of small, critical modules. Unfortunately, despite considerable efforts, determining if a “verification” verifies what the author intends is still difficult. Previous approaches are difficult to understand and often limited in applicability. Developers need verification coverage in terms of the software they are verifying, not model checking diagnostics. We propose a methodology to allow developers to determine (and correct) what it is that they have verified, and tools to support that methodology. Our basic approach is based on a novel variation of mutation analysis and the idea of verification driven by falsification. We use the CBMC model checker to show that this approach is applicable not only to simple data structures and sorting routines, and verification of a routine in Mozilla’s JavaScript engine, but to understanding an ongoing effort to verify the Linux kernel read-copy-update mechanism. Moreover, we show that despite the probabilistic nature of random testing and the tendency to incompleteness of testing as opposed to verification, the same techniques, with suitable modifications, apply to automated test generation as well as to formal verification. In essence, it is the number of surviving mutants that drives the scalability of our methods, not the underlying method for detecting faults in a program. From the point of view of a Popperian analysis where an unkilled mutant is a weakness (in terms of its falsifiability) in a “scientific theory” of program behavior, it is only the number of weaknesses to be examined by a user that is important.     

TSTL: the template scripting testing language

International Journal on Software Tools for Technology Transfer - A test harness, in automated test generation, defines the set of valid tests for a system, as well as their correctness properties....     

Pesticide residues in foodstuffs in Great Britain: XII.—Organochlorine insecticide residues in hens' eggs from battery,deep-Utter and free-range systems and from houses containing insecticide thermal vaporisers

A study has been made of the organochlorine pesticide residues in hens' eggs produced in battery, deep-litter and free-range systems. Low residues (less than 0.05 ppm) were found except where the birds or their houses had been deliberately treated with pesticides to eradicate fleas or lice. A separate study was also made of eggs produced in houses which had insecticide thermal vaporisers installed in them. The results were variable, depending on a number of factors, but relatively high levels of BHC and DDT were found regularly in the eggs from some of these houses.     

Statewide investigation of the role of pyrethroid pesticides in sediment toxicity in California's urban waterways

A statewide investigation of urban creek sediment toxicity was conducted in California in recognition of increased incidences of toxicity linked to pyrethroid pesticides. The goals were to examine the spatial occurrence and magnitude of sediment toxicity in California urban creeks, and to examine the role of pyrethroids in toxic urban creek sediment samples. After a preliminary screening of 90 sites, 30 creeks were sampled in eight geographical regions. Sediment toxicity was assessed using 10 day bioassays with the resident amphipod Hyalella azteca. Bioassays were conducted at two test temperatures of 23 degrees C and at 15 degrees C to provide evidence of the cause of toxicity, and to more accurately reflect ambient environmental temperatures. Twenty-five of 30 samples were toxic when tested at 23 degrees C, and all 30 samples were toxic when tested at 15 degrees C. The magnitude of toxicity increased in samples tested at 15 degrees C suggesting the influence of pyrethroids, which are more toxic at colder temperatures. Pyrethroids were present in all sediment samples and were the only compounds detected at concentrations toxic to H. azteca. Bifenthrin was the pyrethroid of greatest toxicological concern, occurring in all 30 samples at concentrations up to 219 ng/g. Pyrethroid contamination of urban creeks was most severe in the Los Angeles, Central Valley, and San Diego regions, respectively. However, pyrethroids were also linked to urban creek aquatic toxicity in all regions sampled, including the less urbanized areas of the North Coast and Lake Tahoe.