Correlations of perceived post-editing effort with measurements of actual effort

Human rating of predicted post-editing effort is a common activity and has been used to train confidence estimation models. However, the correlation between human ratings and actual post-editing effort is under-measured. Moreover, the impact of presenting effort indicators in a post-editing user interface on actual post-editing effort has hardly been researched. In this study, ratings of perceived post-editing effort are tested for correlations with actual temporal, technical and cognitive post-editing effort. In addition, the impact on post-editing effort of the presentation of post-editing effort indicators in the user interface is also tested. The language pair involved in this study is English-Brazilian Portuguese. Our findings, based on a small sample, suggest that there is little agreement between raters for predicted post-editing effort and that the correlations between actual post-editing effort and predicted effort are only moderate, and thus an inefficient basis for MT confidence estimation. Moreover, the presentation of post-editing effort indicators in the user interface appears not to impact on actual post-editing effort.     

Measuring and predicting software productivity: A systematic map and review

Context

Software productivity measurement is essential in order to control and improve the performance of software development. For example, by identifying role models (e.g. projects, individuals, tasks) when comparing productivity data. The prediction is of relevance to determine whether corrective actions are needed, and to discover which alternative improvement action would yield the best results.

Objective

In this study we identify studies for software productivity prediction and measurement. Based on the identified studies we first create a classification scheme and map the studies into the scheme (systematic map). Thereafter, a detailed analysis and synthesis of the studies is conducted.

Method

As a research method for systematically identifying and aggregating the evidence of productivity measurement and prediction approaches systematic mapping and systematic review have been used.

Results

In total 38 studies have been identified, resulting in a classification scheme for empirical research on software productivity. The mapping allowed to identify the rigor of the evidence with respect to the different productivity approaches. In the detailed analysis the results were tabulated and synthesized to provide recommendations to practitioners.

Conclusion

Risks with simple ratio-based measurement approaches were shown. In response to the problems data envelopment analysis seems to be a strong approach to capture multivariate productivity measures, and allows to identify reference projects to which inefficient projects should be compared. Regarding simulation no general prediction model can be identified. Simulation and statistical process control are promising methods for software productivity prediction. Overall, further evidence is needed to make stronger claims and recommendations. In particular, the discussion of validity threats should become standard, and models need to be compared with each other.     

Evolution of Gaussian Process kernels for machine translation post-editing effort estimation

In many Natural Language Processing problems the combination of machine learning and optimization techniques is essential. One of these problems is the estimation of the human effort needed to improve a text that has been translated using a machine translation method. Recent advances in this area have shown that Gaussian Processes can be effective in post-editing effort prediction. However, Gaussian Processes require a kernel function to be defined, the choice of which highly influences the quality of the prediction. On the other hand, the extraction of features from the text can be very labor-intensive, although recent advances in sentence embedding have shown that this process can be automated. In this paper, we use a Genetic Programming algorithm to evolve kernels for Gaussian Processes to predict post-editing effort based on sentence embeddings. We show that the combination of evolutionary optimization and Gaussian Processes removes the need for a-priori specification of the kernel choice, and, by using a multi-objective variant of the Genetic Programming approach, kernels that are suitable for predicting several metrics can be learned. We also investigate the effect that the choice of the sentence embedding method has on the kernel learning process.

     

Total productivity management

In June 1989, Georgetown University Hospital (GUH) implemented an internally developed, Model 204 DBMS-based comprehensive Decision Support tool, the Financial Performance Report (FPR). FPR was designed to bring order out of management reporting chaos by integrating labor productivity reporting and cost accounting in a progressively detailed display that provides the basis for “Total Productivity Management”. This paper will describe the philosophy that lead to developing the tool, the objectives, the system components, and the principal features of the report.     

Towards communityware

This paper summarizes our recent activities to support people to communicate with each other using public computer network systems. Unlike conventional teleconferencing systems, which are mainly for business meetings, we focus on informal communication in open orgnizations. So far, three different systems have been developed and actually tested.

Debunking the productivity paradox

The productivity paradox is not responsible for all the problems with computers. When all the evidence is in, there probably is almost no productivity paradox at all. The author discusses the arguments against the productivity paradox     

Benchmarking software development productivity

The article examines a statistical analysis of a productivity variation, involving a unique database containing 206 business software projects from 26 Finnish companies. The authors examine differences in the factors, explaining productivity in the banking, insurance, manufacturing, wholesale/retail, and public administration sectors. The authors provide productivity benchmarking equations that are useful both for estimating expected productivity at the start of a new project and for benchmarking a completed project for each business sector     

Lazy productivity via termination

We present a procedure for transforming strongly sequential constructor-based term rewriting systems (TRSs) into context-sensitive TRSs in such a way that productivity of the input system is equivalent to termination of the output system. Thereby automated termination provers become available for proving productivity. A TRS is called productive if all its finite ground terms are constructor normalizing, and all ‘inductive constructor paths’ through the resulting (possibly non-wellfounded) constructor normal form are finite. To our knowledge, this is the first complete transformation from productivity to termination.The transformation proceeds in two steps: (i) The strongly sequential TRS is converted into a shallow TRS, where patterns do not have nested constructors. (ii) The shallow TRS is transformed into a context-sensitive TRS, where rewriting below constructors and in arguments not ‘consumed from’ is disallowed.Furthermore, we show how lazy evaluation can be encoded by strong sequentiality, thus extending our transformation to, e.g., Haskell programs.Finally, we present a simple, but fruitful extension of matrix interpretations to make them applicable for proving termination of context-sensitive TRSs.     

National productivity and computers

We are the first generation of Americans who think that our children will not live substantially better than we do. This article takes a hard look at US productivity, education, technology, and the prospects of improving national output. The problem of national productivity is not well understood, yet it matters enormously. It may be that the answer cannot be obtained from those who look on from a great distance. Perhaps the causes of productivity are too detail-sensitive for that. Computer professionals more than any other group are contributing to the improvement of productivity. Perhaps they can also contribute to its understanding     

Increasing productivity at Saturn

To remain competitive, manufacturing enterprises must increase and simultaneously reduce costs. This requires daily and long term examination and analysis of a plant's functions and operations. Using this data, an enterprise can identify production flow bottlenecks and analyze capacity and other factors, which in turn helps identify improvement opportunities. While such information is critical, it is often a challenge to obtain in a complex manufacturing environment. The automotive industry provides some prime examples of complex manufacturing environments. At Saturn Corp., we turned to academic research-work involving model integrated computing (MIC)-as a framework to organize the diverse types of data our information system must deal with. Using MIC and working with the Vanderbilt University researchers, Saturn built the Saturn Site Production Flow system. This system provides real time information-culled from 80000 data points-to Saturn personnel across the plant. Using SSPF provided information helped Saturn improve throughput at its plant by 10 percent. Through models, we can readily explain a complex data structure in a virtual view of a manufacturing plant floor. Diverse services are required for applications such as process monitoring and control, process simulation, statistical analysis packages, and other data manipulation tools. MIC integrates these tools into a common problem solving environment     

Towards coalgebraic behaviourism

In this paper we show that it is possible to model observable behaviour of coalgebras independently from their internal dynamics, but within the general framework of representing behaviour by a map into a “final” coalgebra.In the first part of the paper we characterise Set-endofunctors F with the property that bisimilarity of elements of F-coalgebras coincides with having the same observable behaviour. We show that such functors have the final coalgebra of a rather simple nature, and preserve some weak pullbacks. We also show that this is the case if and only if F-bisimilarity corresponds to logical equivalence in the finitary fragment of the coalgebraic logic.In the second part of the paper, we present a construction of a “final” coalgebra that captures the observable behaviour of F-coalgebras. We keep the word “final” quoted since the object we are going to construct need not belong to the original category. The construction is carried out for arbitrary Set-endofunctor F, throughout the construction we remain in Set, but the price to pay is the introduction of new morphisms. The paper concludes with a hint to a possible application to modelling weak bisimilarity for coalgebras.     

走进数字图书馆

本文针对数字图书馆的概念,特征及数字图书馆的业务工作和对馆员的知识结构要求等问题进行了探讨。同时阐述了建设中国数字图书馆对社会产生积极影响。