A rapid-prototyping framework for extracting small-scale incident-related information in microblogs: Application of multi-label classification on tweets

Small scale-incidents such as car crashes or fires occur with high frequency and in sum involve more people and consume more money than large and infrequent incidents. Therefore, the support of small-scale incident management is of high importance.Microblogs are an important source of information to support incident management as important situational information is shared, both by citizens and official sources. While microblogs are already used to address large-scale incidents detecting small-scale incident-related information was not satisfyingly possible so far.In this paper we investigate small-scale incident reporting behavior with microblogs. Based on our findings, we present an easily extensible rapid prototyping framework for information extraction of incident-related tweets. The framework enables the precise identification and extraction of information relevant for emergency management. We evaluate the rapid prototyping capabilities and usefulness of the framework by implementing the multi-label classification of tweets related to small-scale incidents. An evaluation shows that our approach is applicable for detecting multiple labels with an match rate of 84.35%.     

A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model

Sensitivity analysis (SA) is a commonly used approach for identifying important parameters that dominate model behaviors. We use a newly developed software package, a Problem Solving environment for Uncertainty Analysis and Design Exploration (PSUADE), to evaluate the effectiveness and efficiency of ten widely used SA methods, including seven qualitative and three quantitative ones. All SA methods are tested using a variety of sampling techniques to screen out the most sensitive (i.e., important) parameters from the insensitive ones. The Sacramento Soil Moisture Accounting (SAC-SMA) model, which has thirteen tunable parameters, is used for illustration. The South Branch Potomac River basin near Springfield, West Virginia in the U.S. is chosen as the study area. The key findings from this study are: (1) For qualitative SA methods, Correlation Analysis (CA), Regression Analysis (RA), and Gaussian Process (GP) screening methods are shown to be not effective in this example. Morris One-At-a-Time (MOAT) screening is the most efficient, needing only 280 samples to identify the most important parameters, but it is the least robust method. Multivariate Adaptive Regression Splines (MARS), Delta Test (DT) and Sum-Of-Trees (SOT) screening methods need about 400–600 samples for the same purpose. Monte Carlo (MC), Orthogonal Array (OA) and Orthogonal Array based Latin Hypercube (OALH) are appropriate sampling techniques for them; (2) For quantitative SA methods, at least 2777 samples are needed for Fourier Amplitude Sensitivity Test (FAST) to identity parameter main effect. McKay method needs about 360 samples to evaluate the main effect, more than 1000 samples to assess the two-way interaction effect. OALH and LP_τ (LPTAU) sampling techniques are more appropriate for McKay method. For the Sobol' method, the minimum samples needed are 1050 to compute the first-order and total sensitivity indices correctly. These comparisons show that qualitative SA methods are more efficient but less accurate and robust than quantitative ones.     

Bayesian uncertainty quantification in the evaluation of alloy properties with the cluster expansion method

Parametrized surrogate models are used in alloy modeling to quickly obtain otherwise expensive properties such as quantum mechanical energies, and thereafter used to optimize, or simply compute, some alloy quantity of interest, e.g., a phase transition, subject to given constraints. Once learned on a data set, the surrogate can compute alloy properties fast, but with an increased uncertainty compared to the computer code. This uncertainty propagates to the quantity of interest and in this work we seek to quantify it. Furthermore, since the alloy property is expensive to compute, we only have available a limited amount of data from which the surrogate is to be learned. Thus, limited data further increases the uncertainties in the quantity of interest, and we show how to capture this as well. We cannot, and should not, trust the surrogate before we quantify the uncertainties in the application at hand. Therefore, in this work we develop a fully Bayesian framework for quantifying the uncertainties in alloy quantities of interest, originating from replacing the expensive computer code with the fast surrogate, and from limited data. We consider a particular surrogate popular in alloy modeling, the cluster expansion, and aim to quantify how well it captures quantum mechanical energies. Our framework is applicable to other surrogates and alloy properties.     

Towards optimal allocation of computer resources: Trade-offs between uncertainty quantification,discretization and model reduction

The computational complexity of numerical models can be broken down into contributions ranging from spatial, temporal and stochastic resolution, e.g., spatial grid resolution, time step size and number of repeated simulations dedicated to quantify uncertainty. Controlling these resolutions allows keeping the computational cost at a tractable level whilst still aiming at accurate and robust predictions. The objective of this work is to introduce a framework that optimally allocates the available computational resources in order to achieve highest accuracy associated with a given prediction goal. Our analysis is based on the idea to jointly consider the discretization errors and computational costs of all individual model dimensions (physical space, time, parameter space). This yields a cost-to-error surface which serves to aid modelers in finding an optimal allocation of the computational resources (ORA). As a pragmatic way to proceed, we propose running small cost-efficient pre-investigations in order to estimate the joint cost-to-error surface, then fit underlying complexity and error models, decide upon a computational design for the full simulation, and finally to perform the designed simulation at near-optimal costs-to-accuracy ratio. We illustrate our approach with three examples from subsurface hydrogeology and show that the computational costs can be substantially reduced when allocating computational resources wisely and in a situation-specific and task-specific manner. We conclude that the ORA depends on a multitude of parameters, assumptions and problem-specific features and, hence, ORA needs to be determined carefully prior to each investigation.     

A modular framework for the automatic classification of chromosomes in Q-band images

The manual analysis of the karyogram is a complex and time-consuming operation, as it requires meticulous attention to details and well-trained personnel. Routine Q-band laboratory images show chromosomes that are randomly rotated, blurred or corrupted by overlapping and dye stains. We address here the problem of robust automatic classification, which is still an open issue. The proposed method starts with an improved estimation of the chromosome medial axis, along which an established set of features is then extracted. The following novel polarization stage estimates the chromosome orientation and makes this feature set independent on the reading direction along the axis. Feature rescaling and normalizing techniques take full advantage of the results of the polarization step, reducing the intra-class and increasing the inter-class variances. After a standard neural network based classification, a novel class reassignment algorithm is employed to maximize the probability of correct classification, by exploiting the constrained composition of the human karyotype. An average 94% of correct classification was achieved by the proposed method on 5474 chromosomes, whose images were acquired during laboratory routine and comprise karyotypes belonging to slightly different prometaphase stages. In order to provide the scientific community with a public dataset, all the data we used are publicly available for download.     

Handling uncertainty in economic nonlinear model predictive control: A comparative case study

In the last years, the use of an economic cost function for model predictive control (MPC) has been widely discussed in the literature. The main motivation for this choice is that often the real goal of control is to maximize the profit or the efficiency of a certain system, rather than tracking a predefined set-point as done in the typical MPC approaches, which can be even counter-productive. Since the economic optimal operation of a system resulting from the application of an economic model predictive control approach drives the system to the constraints, the explicit consideration of the uncertainties becomes crucial in order to avoid constraint violations. Although robust MPC has been studied during the past years, little attention has yet been devoted to this topic in the context of economic nonlinear model predictive control, especially when analyzing the performance of the different MPC approaches. In this work, we present the use of multi-stage scenario-based nonlinear model predictive control as a promising strategy to deal with uncertainties in the context of economic NMPC. We make a comparison based on simulations of the advantages of the proposed approach with an open-loop NMPC controller in which no feedback is introduced in the prediction and with an NMPC controller which optimizes over affine control policies. The approach is efficiently implemented using CasADi, which makes it possible to achieve real-time computations for an industrial batch polymerization reactor model provided by BASF SE. Finally, a novel algorithm inspired by tube-based MPC is proposed in order to achieve a trade-off between the variability of the controlled system and the economic performance under uncertainty. Simulations results show that a closed-loop approach for robust NMPC increases the performance and that enforcing low variability under uncertainty of the controlled system might result in a big performance loss.     

Thermodynamic modeling of the Nb-Ni system with uncertainty quantification using PyCalphad and ESPEI

The Nb–Ni system is remodeled with uncertainty quantification (UQ) using software tools of PyCalphad and ESPEI (the Extensible, Self-optimizing Phase Equilibria Infrastructure) with the presently implemented capability of modeling site fraction based on Wyckoff positions. The five- and three-sublattice models are used to model the topologically close pack (TCP) μ-Nb₇Ni₆ and δ-NbNi₃ phases according to their Wyckoff positions. The inputs for CALPHAD-based thermodynamic modeling include the thermochemical data as a function of temperature predicted by first-principles and phonon calculations based on density functional theory (DFT), ab initio molecular dynamics (AIMD) simulations, together with phase equilibrium and site fraction data in the literature. In addition to phase diagram and thermodynamic properties, the CALPHAD-based predictions of site fractions of Nb in μ-Nb₇Ni₆ agree well with experimental data. Furthermore, the UQ estimation using the Markov Chain Monte Carlo (MCMC) method as implemented in ESPEI is applied to study the uncertainty of site fraction in μ-Nb₇Ni₆ and enthalpy of mixing (ΔH_mix) in liquid.     

Identifying uncertainty states during wayfinding in indoor environments: An EEG classification study

The researchers used a machine-learning classification approach to better understand neurological features associated with periods of wayfinding uncertainty. The participants (n = 30) were asked to complete wayfinding tasks of varying difficulty in a virtual reality (VR) hospital environment. Time segments when participants experienced navigational uncertainty were first identified using a combination of objective measurements (frequency of inputs into the VR controller) and behavioral annotations from two independent observers. Uncertainty time-segments during navigation were ranked on a scale from 1 (low) to 5 (high). The machine-learning model, a Random Forest classifier implemented using scikit-learn in Python, was used to evaluate common spatial patterns of EEG spectral power across the theta, alpha, and beta bands associated with the researcher-identified uncertainty states. The overall predictive power of the resulting model was 0.70 in terms of the area under the Receiver Operating Characteristics curve (ROC-AUC). These findings indicate that EEG data can potentially be used as a metric for identifying navigational uncertainty states, which may provide greater rigor and efficiency in studies of human responses to architectural design variables and wayfinding cues.     

A comparative study of uncertainty propagation methods for black-box-type problems

A wide variety of uncertainty propagation methods exist in literature; however, there is a lack of good understanding of their relative merits. In this paper, a comparative study on the performances of several representative uncertainty propagation methods, including a few newly developed methods that have received growing attention, is performed. The full factorial numerical integration, the univariate dimension reduction method, and the polynomial chaos expansion method are implemented and applied to several test problems. They are tested under different settings of the performance nonlinearity, distribution types of input random variables, and the magnitude of input uncertainty. The performances of those methods are compared in moment estimation, tail probability calculation, and the probability density function construction, corresponding to a wide variety of scenarios of design under uncertainty, such as robust design, and reliability-based design optimization. The insights gained are expected to direct designers for choosing the most applicable uncertainty propagation technique in design under uncertainty.     

Evaluating pedagogical classification frameworks for learning objects: A case study

The use of toolkits and reference frameworks for the design and evaluation of learning activities enables the systematic application of pedagogical criteria in the elaboration of learning resources and learning designs. Pedagogical classification as described in such frameworks is a major criterion for the retrieval of learning objects, since it serves to partition the space of available learning resources depending either on the pedagogical standpoint that was used to create them, or on the interpreted pedagogical orientation of their constituent learning contents and activities. However, pedagogical classification systems need to be evaluated to assess their quality with regards to providing a degree of inter-subjective agreement on the meaning of the classification dimensions they provide. Without such evaluation, classification metadata, which is typically provided by a variety of contributors, is at risk of being fuzzy in reflecting the actual pedagogical orientations, thus hampering the effective retrieval of resources. This paper describes a case study that evaluates the general pedagogical dimensions proposed by Conole et al. to classify learning resources. Rater agreement techniques are used for the assessment, which is proposed as a general technique for the evaluation of such kind of classification schemas. The case study evaluates the degree of coherence of the pedagogical dimensions proposed by Conole et al. as an objective instrument to classify pedagogical resources. In addition, the technical details on how to integrate such classifications in learning object metadata are provided.     

Evaluation of feature selection methods for text classification with small datasets using multiple criteria decision-making methods

The evaluation of feature selection methods for text classification with small sample datasets must consider classification performance, stability, and efficiency. It is, thus, a multiple criteria decision-making (MCDM) problem. Yet there has been few research in feature selection evaluation using MCDM methods which considering multiple criteria. Therefore, we use MCDM-based methods for evaluating feature selection methods for text classification with small sample datasets. An experimental study is designed to compare five MCDM methods to validate the proposed approach with 10 feature selection methods, nine evaluation measures for binary classification, seven evaluation measures for multi-class classification, and three classifiers with 10 small datasets. Based on the ranked results of the five MCDM methods, we make recommendations concerning feature selection methods. The results demonstrate the effectiveness of the used MCDM-based method in evaluating feature selection methods.     

Spatio-temporal uncertainty in Spatial Decision Support Systems: A case study of changing land availability for bioenergy crops in Mozambique

Spatial Decision Support Systems (SDSSs) often include models that can be used to assess the impact of possible decisions. These models usually simulate complex spatio-temporal phenomena, with input variables and parameters that are often hard to measure. The resulting model uncertainty is, however, rarely communicated to the user, so that current SDSSs yield clear, but therefore sometimes deceptively precise outputs. Inclusion of uncertainty in SDSSs requires modeling methods to calculate uncertainty and tools to visualize indicators of uncertainty that can be understood by its users, having mostly limited knowledge of spatial statistics. This research makes an important step towards a solution of this issue. It illustrates the construction of the PCRaster Land Use Change model (PLUC) that integrates simulation, uncertainty analysis and visualization. It uses the PCRaster Python framework, which comprises both a spatio-temporal modeling framework and a Monte Carlo analysis framework that together produce stochastic maps, which can be visualized with the Aguila software, included in the PCRaster Python distribution package. This is illustrated by a case study for Mozambique in which it is evaluated where bioenergy crops can be cultivated without endangering nature areas and food production now and in the near future, when population and food intake per capita will increase and thus arable land and pasture areas are likely to expand. It is shown how the uncertainty of the input variables and model parameters effects the model outcomes. Evaluation of spatio-temporal uncertainty patterns has provided new insights in the modeled land use system about, e.g., the shape of concentric rings around cities. In addition, the visualization modes give uncertainty information in an comprehensible way for users without specialist knowledge of statistics, for example by means of confidence intervals for potential bioenergy crop yields. The coupling of spatio-temporal uncertainty analysis to the simulation model is considered a major step forward in the exposure of uncertainty in SDSSs.     

Cloud-aided online EEG classification system for brain healthcare: A case study of depression evaluation with a lightweight CNN

Brain healthcare, when supported by Internet of Things, can perform online and accurate analysis of brain big data for the classification of multivariate Electroencephalogram (EEG), which is a prerequisite for the recent boom in neurofeedback applications and clinical practices. However, it remains a grand research challenge due to (1) the embedded intensive noises and the intrinsic nonstationarity determined by the evolution of brain states; and (2) the lack of a user-friendly computing platform to sustain the complicated analytics. This study presents the design of an online EEG classification system aided by Cloud centering on a lightweight Convolutional Neural Network (CNN). The system incrementally trains the CNN on Cloud and enables hot deployment of the trained classifier without the need to restart the gateway to adapt to the users' needs. The classifier maintains a High Convolutional Layer to gain the ability of processing high-dimensional EEG segments. The number of hidden layers is minimized to ensure the efficiency of training. The lightweight CNN adopts an “hourglass” block of fully connected layers to reduce the number of neurons quickly toward the output end. A case study of depression evaluation has been performed against raw EEG datasets to distinguish between (1) Healthy and Major Depression Disorder with an accuracy, sensitivity, and specificity of [98.59% ± 0.28%], [97.77% ± 0.63%], and [99.51% ± 0.19%], respectively; and (2) Effective and Noneffective treatment outcome with an accuracy, sensitivity, and specificity of [99.53% ± 0.002%], [99.50% ± 0.01%], and [99.58% ± 0.02%], respectively. The results show that the classification can be completed several magnitudes faster when EEG is collected on the gateway (several milliseconds vs. 4 seconds).     

System identification applied to stiction quantification in industrial control loops: A comparative study

A comparative study of different models and identification techniques applied to the quantification of valve stiction in industrial control loops is presented in this paper, with the objective of taking into account for the presence of external disturbances. A Hammerstein system is used to model the controlled process (linear block) and the sticky valve (nonlinear block): five different candidates for the linear block and two different candidates for the nonlinear block are evaluated and compared. Two of the five linear models include a nonstationary disturbance term that is estimated along with the input-to-output model, and these extended models are meant to cope with situations in which significant nonzero mean disturbances affect the collected data. The comparison of the different models and identification methods is carried out thoroughly in three steps: simulation, application to pilot plant data and application to industrial loops. In the first two cases (simulation and pilot plant) the specific source of fault (stiction with/without external disturbances) is known and hence a validation of each candidate can be carried out more easily. Nonetheless, each fault case considered in the previous two steps has been found in the application to a large number of datasets collected from industrial loops, and hence the merits and limitations of each candidate have been confirmed. As a result of this study, extended models are proved to be effective when large, time varying disturbances affect the system, whereas conventional (stationary) noise models are more effective elsewhere.     

A comparative study of ranking methods, similarity measures and uncertainty measures for interval type-2 fuzzy sets

Ranking methods, similarity measures and uncertainty measures are very important concepts for interval type-2 fuzzy sets (IT2 FSs). So far, there is only one ranking method for such sets, whereas there are many similarity and uncertainty measures. A new ranking method and a new similarity measure for IT2 FSs are proposed in this paper. All these ranking methods, similarity measures and uncertainty measures are compared based on real survey data and then the most suitable ranking method, similarity measure and uncertainty measure that can be used in the computing with words paradigm are suggested. The results are useful in understanding the uncertainties associated with linguistic terms and hence how to use them effectively in survey design and linguistic information processing.     

Resource oriented selection of rule-based classification models: An empirical case study

The amount of resources allocated for software quality improvements is often not enough to achieve the desired software quality. Software quality classification models that yield a risk-based quality estimation of program modules, such as fault-prone (fp) and not fault-prone (nfp), are useful as software quality assurance techniques. Their usefulness is largely dependent on whether enough resources are available for inspecting the fp modules. Since a given development project has its own budget and time limitations, a resource-based software quality improvement seems more appropriate for achieving its quality goals. A classification model should provide quality improvement guidance so as to maximize resource-utilization. We present a procedure for building software quality classification models from the limited resources perspective. The essence of the procedure is the use of our recently proposed Modified Expected Cost of Misclassification (MECM) measure for developing resource-oriented software quality classification models. The measure penalizes a model, in terms of costs of misclassifications, if the model predicts more number of fp modules than the number that can be inspected with the allotted resources. Our analysis is presented in the context of our Rule-Based Classification Modeling (RBCM) technique. An empirical case study of a large-scale software system demonstrates the promising results of using the MECM measure to select an appropriate resource-based rule-based classification model. Taghi M. Khoshgoftaar is a professor of the Department of Computer Science and Engineering, Florida Atlantic University and the Director of the graduate programs and research. His research interests are in software engineering, software metrics, software reliability and quality engineering, computational intelligence applications, computer security, computer performance evaluation, data mining, machine learning, statistical modeling, and intelligent data analysis. He has published more than 300 refereed papers in these areas. He is a member of the IEEE, IEEE Computer Society, and IEEE Reliability Society. He was the general chair of the IEEE International Conference on Tools with Artificial Intelligence 2005. Naeem Seliya is an Assistant Professor of Computer and Information Science at the University of Michigan - Dearborn. He recieved his Ph.D. in Computer Engineering from Florida Atlantic University, Boca Raton, FL, USA in 2005. His research interests include software engineering, data mining and machine learnring, application and data security, bioinformatics and computational intelligence. He is a member of IEEE and ACM.     

Medical image classification based on artificial intelligence approaches: A practical study on normal and abnormal confocal corneal images

Corneal images can be acquired using confocal microscopes which provide detailed views of the different layers inside a human cornea. Some corneal problems and diseases can occur in one or more of the main corneal layers: the epithelium, stroma and endothelium. Consequently, for automatically extracting clinical information associated with corneal diseases, identifying abnormality or evaluating the normal cornea, it is important to be able to automatically recognise these layers reliably. Artificial intelligence (AI) approaches can provide improved accuracy over the conventional processing techniques and save a useful amount of time over the manual analysis time required by clinical experts. Artificial neural networks (ANNs), adaptive neuro fuzzy inference systems (ANFIS) and a committee machine (CM) have been investigated and tested to improve the recognition accuracy of the main corneal layers and identify abnormality in these layers. The performance of the CM, formed from ANN and ANFIS, achieves an accuracy of 100% for some classes in the processed data sets. Three normal corneal data sets and seven abnormal corneal images associated with diseases in the main corneal layers have been investigated with the proposed system. Statistical analysis for these data sets is performed to track any change in the processed images. This system is able to pre-process (quality enhancement, noise removal), classify corneal images, identify abnormalities in the analysed data sets and visualise corneal stroma images as well as each individual keratocyte cell in a 3D volume for further clinical analysis.     

PAT: A pattern classification approach to automatic reference oracles for the testing of mesh simplification programs

Graphics applications often need to manipulate numerous graphical objects stored as polygonal models. Mesh simplification is an approach to vary the levels of visual details as appropriate, thereby improving on the overall performance of the applications. Different mesh simplification algorithms may cater for different needs, producing diversified types of simplified polygonal model as a result. Testing mesh simplification implementations is essential to assure the quality of the graphics applications. However, it is very difficult to determine the oracles (or expected outcomes) of mesh simplification for the verification of test results.A reference model is an implementation closely related to the program under test. Is it possible to use such reference models as pseudo-oracles for testing mesh simplification programs? If so, how effective are they?This paper presents a fault-based pattern classification methodology called PAT, to address the questions. In PAT, we train the C4.5 classifier using black-box features of samples from a reference model and its fault-based versions, in order to test samples from the subject program. We evaluate PAT using four implementations of mesh simplification algorithms as reference models applied to 44 open-source three-dimensional polygonal models. Empirical results reveal that the use of a reference model as a pseudo-oracle is effective for testing the implementations of resembling mesh simplification algorithms. However, the results also show a tradeoff: When compared with a simple reference model, the use of a resembling but sophisticated reference model is more effective and accurate but less robust.     

Accounting for surface–groundwater interactions and their uncertainty in river and groundwater models: A case study in the Namoi River,Australia

Surface–groundwater (SW–GW) interactions constitute a critical proportion of the surface and groundwater balance especially during dry conditions. Conjunctive management of surface and groundwater requires an explicit account of the exchange flux between surface and groundwater when modelling the two systems. This paper presents a case study in the predominantly gaining Boggabri–Narrabri reach of the Namoi River located in eastern Australia. The first component of the study uses the Upper Namoi numerical groundwater model to demonstrate the importance of incorporating SW–GW interactions into river management models. The second component demonstrates the advantages of incorporating groundwater processes in the Namoi River model.Results of the numerical groundwater modelling component highlighted the contrasting groundwater dynamics close to, and away from the Namoi River where lower declines were noted in a near-field well due to water replenishment sourced from river depletion. The contribution of pumping activities to river depletion was highlighted in the results of the uncertainty analysis, which showed that the SW–GW exchange flux is the most sensitive to pumping rate during dry conditions. The uncertainty analysis also showed that after a drought period, the 95% prediction interval becomes larger than the simulated flux, which implies an increasing probability of losing river conditions. The future prospect of a gaining Boggabri–Narrabri reach turning into losing was confirmed with a hypothetical extended drought scenario during which persistent expansion of groundwater pumping was assumed. The river modelling component showed that accounting for SW–GW interactions improved the predictions of low flows, and resulted in a more realistic calibration of the Namoi River model.Incorporating SW–GW interactions into river models allows explicit representation of groundwater processes that provides a mechanism to account for the impacts of additional aquifer stresses that may be introduced beyond the calibration period of the river model. Conventional river models that neglect the effects of such future stresses suffer from the phenomenon of non-stationarity and hence have inferior low flow predictions past the calibration period of the river model. The collective knowledge acquired from the two modelling exercises conducted in this study leads to a better understanding of SW–GW interactions in the Namoi River thus leading to improved water management especially during low flow conditions.