Using meta-learning to predict performance metrics in machine learning problems

Machine learning has been facing significant challenges over the last years, much of which stem from the new characteristics of machine learning problems, such as learning from streaming data or incorporating human feedback into existing datasets and models. In these dynamic scenarios, data change over time and models must adapt. However, new data do not necessarily mean new patterns. The main goal of this paper is to devise a method to predict a model's performance metrics before it is trained, in order to decide whether it is worth it to train it or not. That is, will the model hold significantly better results than the current one? To address this issue, we propose the use of meta-learning. Specifically, we evaluate two different meta-models, one built for a specific machine learning problem, and another built based on many different problems, meant to be a generic meta-model, applicable to virtually any problem. In this paper, we focus only on the prediction of the root mean square error (RMSE). Results show that it is possible to accurately predict the RMSE of future models, event in streaming scenarios. Moreover, results also show that it is possible to reduce the need for re-training models between 60% and 98%, depending on the problem and on the threshold used.     

Multi‐scale Internet traffic forecasting using neural networks and time series methods

This article presents three methods to forecast accurately the amount of traffic in TCP/IP based networks: a novel neural network ensemble approach and two important adapted time series methods (ARIMA and Holt‐Winters). In order to assess their accuracy, several experiments were held using real‐world data from two large Internet service providers. In addition, different time scales (5 min, 1 h and 1 day) and distinct forecasting lookaheads were analysed. The experiments with the neural ensemble achieved the best results for 5 min and hourly data, while the Holt‐Winters is the best option for the daily forecasts. This research opens possibilities for the development of more efficient traffic engineering and anomaly detection tools, which will result in financial gains from better network resource management.     

Employing resilience engineering in eliciting software requirements for complex systems: experiments with the functional resonance analysis method (FRAM)

Cognition, Technology & Work - Resilience engineering provides concepts and methods for assessing the ability of socio-technical systems to adjust their functioning before, during, or after...     

Histochemical evidence of osteoclastic degradation of extracellular matrix in osteolytic metastasis originating from human lung small carcinoma (SBC-5) cells

The aim of this study was to assess the dynamics of osteoclast migration and the degradation of unmineralized extracellular matrix in an osteolytic metastasis by examining a well-standardized lung cancer metastasis model of nude mice. SBC-5 human lung small carcinoma cells were injected into the left cardiac ventricle of 6-week-old BALB/c nu/nu mice under anesthesia. At 25-30 days after injection, the animals were sacrificed and their femora and/or tibiae were removed for histochemical analyses. Metastatic lesions were shown to occupy a considerable area extending from the metaphyses to the bone marrow region. Tartrate resistant acid phosphatase (TRAPase)-positive osteoclasts were found in association with an alkaline phosphatase (ALPase)-positive osteoblastic layer lining the bone surface, but could also be localized in the ALPase-negative stromal tissues that border the tumor nodules. These stromal tissues were markedly positive for osteopontin, and contained a significant number of TRAPase-positive osteoclasts expressing immunoreactivity for CD44. We thus speculated that, mediating its affinity for CD44, osteopontin may serve to facilitate osteoclastic migration after their formation associated with ALPase-positive osteoblasts. We next examined the localization of cathepsin K and matrix metallo-proteinase-9 (MMP-9) in osteoclasts. Osteoclasts adjacent to the bone surfaces were positive for both proteins, whereas those in the stromal tissues in the tumor nests showed only MMP-9 immunoreactivity. Immunoelectron microscopy disclosed the presence of MMP-9 in the Golgi apparatus and in vesicular structures at the baso-lateral cytoplasmic region of the osteoclasts found in the stromal tissue. MMP-9-positive vesicular structures also contained fragmented extracellular materials. Thus, osteoclasts appear to either select an optimized function, namely secreting proteolytic enzymes from ruffled borders during bone resorption, or recognize the surrounding extracellular matrix by mediating osteopontin/CD44 interaction, and internalize the extracellular matrices. Microsc.     

Mixed H2/H∞ multi-channel linear parameter-varying control in discrete time

This paper develops a new method for the synthesis of linear parameter-varying (LPV) controllers in discrete time. LPV plants under consideration have a linear fractional transformation (LFT) representation. In contrast to earlier results which are restricted to single-objective LPV problems, the proposed method can handle a set of H₂/H_∞ specifications that can be defined channel-wise. This practically attractive extension is derived by using specific transformations of both the Lyapunov and scaling/multiplier variables in tandem with appropriate linearizing transformations of the controller data and of the controller scheduling function. It is shown that the controller gain-scheduling function can be constructed as an affine matrix-valued function in the polytopic coordinates of the scheduled parameter, hence is easily implemented on line. Finally, these manipulations give rise to a tractable and practical LMI formulation of the multi-objective LPV control problem.     

On the impact of disproportional samples in credit scoring models: An application to a Brazilian bank data

Statistical methods have been widely employed to assess the capabilities of credit scoring classification models in order to reduce the risk of wrong decisions when granting credit facilities to clients. The predictive quality of a classification model can be evaluated based on measures such as sensitivity, specificity, predictive values, accuracy, correlation coefficients and information theoretical measures, such as relative entropy and mutual information. In this paper we analyze the performance of a naive logistic regression model (Hosmer & Lemeshow, 1989) and a logistic regression with state-dependent sample selection model (Cramer, 2004) applied to simulated data. Also, as a case study, the methodology is illustrated on a data set extracted from a Brazilian bank portfolio. Our simulation results so far revealed that there is no statistically significant difference in terms of predictive capacity between the naive logistic regression models and the logistic regression with state-dependent sample selection models. However, there is strong difference between the distributions of the estimated default probabilities from these two statistical modeling techniques, with the naive logistic regression models always underestimating such probabilities, particularly in the presence of balanced samples.     

Modeling interactive storytelling genres as application domains

In this paper, we introduce a formalism to specify interactive storytelling genres in the context of digital entertainment, adopting an information systems approach. We view a genre as a set of plots, where a plot is a partially ordered sequence of events, taken from a fixed repertoire. In general, the specification of a genre should allow to determine whether a plot is a legitimate representative of the genre, and also to generate all plots belonging to the genre. The formalism divides the specification of a genre into static, dynamic and behavioral schemas, that reflect a plan recognition/plan generation paradigm. It leads to executable specifications, supported by LOGTELL, a prototype tool that helps users generate, modify and reuse plots that follow a genre specification. To illustrate the use of the formalism, we specify a simple Swords & Dragons genre and show plots generated by the tool.