Validation of scientific topic models using graph analysis and corpus metadata

Probabilistic topic modeling algorithms like Latent Dirichlet Allocation (LDA) have become powerful tools for the analysis of large collections of documents (such as papers, projects, or funding applications) in science, technology an innovation (STI) policy design and monitoring. However, selecting an appropriate and stable topic model for a specific application (by adjusting the hyperparameters of the algorithm) is not a trivial problem. Common validation metrics like coherence or perplexity, which are focused on the quality of topics, are not a good fit in applications where the quality of the document similarity relations inferred from the topic model is especially relevant. Relying on graph analysis techniques, the aim of our work is to state a new methodology for the selection of hyperparameters which is specifically oriented to optimize the similarity metrics emanating from the topic model. In order to do this, we propose two graph metrics: the first measures the variability of the similarity graphs that result from different runs of the algorithm for a fixed value of the hyperparameters, while the second metric measures the alignment between the graph derived from the LDA model and another obtained using metadata available for the corresponding corpus. Through experiments on various corpora related to STI, it is shown that the proposed metrics provide relevant indicators to select the number of topics and build persistent topic models that are consistent with the metadata. Their use, which can be extended to other topic models beyond LDA, could facilitate the systematic adoption of this kind of techniques in STI policy analysis and design.

     

Modernization of Air Supply Unit for Oxidation Towers Used in Bitumen Production

Chemical and Petroleum Engineering - The hydrodynamics of oxidation towers used for bitumen production were studied using a model tower having transparent walls. Experiments were carried out in two...     

Modeling the Destruction of an Anisotropic Composite Barrier in Interaction with Slender Strikers at an Angle

Journal of Engineering Physics and Thermophysics - A numerical investigation has been made into the interaction of slender (5–20 calibers) steel cylinder strikers with orthotropic compost...     

Influence of the Corrosion Process on Thermotechnical Characteristics of Heat Pipes of Low-Temperature Range

Journal of Engineering Physics and Thermophysics - Results of investigation into the influence of a corrosion process on thermotechnical characteristics of heat pipes of low-temperature range are...     

Magnetically Modified Waste Bark of Aegle marmelos Tree as a Promising Biosorbent for Uranium(VI) Sorption

Radiochemistry - Magnetically modified waste bark of the Aegle marmelos tree was prepared by using green synthesis method and was used in a batch system for U(VI) removal from aqueous solution. The...     

Wall Rock Alteration,Uranium Mineralization,and Potentiality of Uranium and Iron Dissolution from Mineralized Abu Rusheid Gneiss Rock,South Eastern Desert,Egypt

Radiochemistry - Ferrugination, sericitization, desulfitization, chloritization, and kaolinitization of hydrothermal origin are the main wall rock alterations occurring within the NNW shear zones...     

Justification for the Use of Cohen’s Kappa Statistic in Experimental Studies of NLP and Text Mining

Modern metrics for evaluating agreement coefficients between the experimental results and expert opinion are compared, and the possibility of using these metrics in experimental research in automatic text processing by machine learning methods is assessed. The choice of Cohen’s kappa coefficient as a measure of expert opinion agreement in the NLP and Text Mining problems is justified. An example of using Cohen’s kappa coefficient for evaluating the level of agreement between the opinion of an expert and the results of ML classification and the measure of agreement of expert opinions in the alignment of sentences of the Kazakh-Russian parallel corpus is given. Based on this analysis, it is proved that Cohen’s kappa coefficient is one of the best statistical methods for determining the level of agreement in experimental studies due to its ease of use, computing simplicity, and high accuracy of the results.

     

An Efficient Modified Dragonfly Optimization Based MIMO-OFDM for Enhancing QoS in Wireless Multimedia Communication

Wireless Personal Communications - Multiple inputs multiple outputs (MIMO) is a reliable technique which can manage the increased wireless data traffic in the future generation of wireless...     

Organizing Production at High-Tech Enterprises in the Digital Era

Russian Engineering Research - The modernization of manufacturing by the comprehensive application of digital technology—in both production and management—is considered. Principles are...