Journal relatedness: An actor-actor and actor-objectives case study

Using the MACTOR (Matrix of Alliances and Conflicts: Tactics, Objectives and Recommendations) method, a set of 13 related journals covering the subject category “Chemistry, Multidisciplinary” was analyzed in terms of direct and indirect reciprocal influences (measured by relatedness indexes Rji), their positions towards a generic set of common objectives (total cites; impact factor; immediacy index; number of published articles; cited half life) and the convergences (Actors x Actors and Actors x Objectives) existing in the above-mentioned relatedness network. The study identified 4 types of actors: dominant (3), independent (8), relay (1) and dominated (1). Maps of: influences and dependences between actors; convergence between actors; net distances between actors and actors-objectives relationships are presented, together with short interpretations. Defining scientific journals as actors on a specific “knowledge market”, identifying influences and dependences between them and positioning these journals towards a set of measurable objectives creates an interesting possibility to define “relationships of power” of a strategic nature and enables the introduction of more complex future-oriented scientometric analyses than those based solely on standard bibliometric indicators such as the impact factor.     

A New Approach to the SCI Journal Citation Reports,a System for Evaluating Scientific Journals

The Science Citation Index, Journal Citation Reports (JCR), published by the Institute for Scientific Information (ISI) and designed to rank, evaluate, categorize and compare journals, is used in a wide scientific context as a tool for evaluating researchers and research work, through the use of just one of its indicators, the impact factor. With the aim of obtaining an overall and synthetic perspective of impact factor values, we studied the frequency distributions of this indicator using the box-plot method. Using this method we divided the journals listed in the JCR into five groups (low, lower central, upper central, high and extreme). These groups position the journal in relation to its competitors. Thus, the group designated as extreme contains the journals with high impact factors which are deemed to be prestigious by the scientific community. We used the JCR data from 1996 to determine these groups, firstly for all subject categories combined (all 4779 journals) and then for each of the 183 ISI subject categories. We then substituted the indicator value for each journal by the name of the group in which it was classified. The journal group may differ from one subject category to another. In this article, we present a guide for evaluating journals constructed as described above. It provides a comprehensive and synthetic view of two of the most used sections of the JCR. It makes it possible to make more accurate and complete judgements on and through the journals, and avoids an oversimplified view of the complex reality of the world of journals. It immediately reveals the scientific subject category where the journal is best positioned. Also, whereas it used to be difficult to make intra- and interdisciplinary comparisons, this is now possible without having to consult the different sections of the JCR. We construct this guide each year using indicators published in the JCR by the ISI.     

Citation mechanics in Journals covered by the journal citation reports

Citations from 1980 to 1988, obtained from fifty biomedical journals covered by theJournal Citation Reports (JCR) are studied. In purely numerical terms, the evolution of each citation (journal citation), including its impact factor (IF), would depend essentially on three variables for each journal: (i) the yearly rate of increase of items that could be cited (citable items), (ii) the relative yearly increment of the citing journals, (iii) the relative yearly increment of citations. The mechanics of this give rise to the three standard patterns for journal citations, namely: (i) annual impact factors increase each year (ascending evolution), (ii) annual impact factors remain the same each year (constant evolution), (iii) annual impact factors decrease each year (descending evolution). The reason why some journal citation profiles do not fit into the standard patterns is presumably that forces are at work able to alter the numerical mechanics described. The concepts of saturation/unsaturation of the demand for scientific information are introduced, showing how they are reflected in the impact factor figures for the journals cited.     

A modified method for calculating the Impact Factors of journals in ISI Journal Citation Reports: Polymer Science Category in 1997–2001

This article introduces a new modified method for calculating the impact factor of journals based on the current ISI practice in generating journal impact factor values. The impact factor value for a journal calculated by the proposed method, the so-called Cited Half-Life Impact Factor (CHAL) method, which is based on the ratio of the number of current year citations of articles from the previous X years to that of articles published in the previous X years, the X value being equal to the value of the cited half-life of the journal in the current year. Thirty-four journals in the Polymer Science Category from the ISI Subject Heading Categories were selected and examined. Total citations, impact factors and cited half-life of the 34 journals during the last five years (1997-2001) were retrieved from the ISI Journal Citation Reports and were used as the data source for the calculations in this work, the impact factor values from ISI and CHAL methods then being compared. The positions of the journals ranked by impact factors obtained from the ISI method were different from those from the CHAL method. It was concluded that the CHAL method was more suitable for calculating the impact factor of the journals than the existing ISI method. This revised version was published online in June 2006 with corrections to the Cover Date.     

Reliability-based citation impact factor and the manipulation of impact factor

According to the definition of reliability-based citation impact factor (R-impact factor) proposed by KUO & RUPE and the cumulative citation age distribution model, a mathematical expression of the relationship between R-impact factor and impact factor is established in this paper. By simulation of the change processes of the R-impact factor and impact factor in the manipulation process of the impact factor, it is found that the effect of manipulation can be partly corrected by the R-impact factor in some cases. Based on the Journal Citation Report database, impact factors of 4 normal journals and 4 manipulated journals were collected. The journals’ R-impact factors and self-cited rates in the previous two years were calculated for each year during the period 2000 to 2007, and various characteristics influenced by the manipulation were analyzed. We find that the R-impact factor has greater fairness than the impact factor for journals with relatively short cited half-lives. Finally, some issues about using the R-impact factor as a measure for evaluating scientific journals are discussed.     

Evaluation of recent scientific research output by a bibliometric method

Describes a new method of evaluation of scientific output by laboratories engaged in diverse fields of research. This method helps to evaluate those outputs which are quite recent and not amenable to citation analysis. For the purpose of analysis, impact factor of journals in which papers are published are considered. A method for normalisation of impact factor of journals has been described and, normalised impact factors have also been used for the purpose of analysis. It is found that in such analysis normalised impact factor tends to show better results compared to simple impact factor. The analysis helps us to generate numerous performance indicators such as average impact factor and normalised impact factor for each laboratory and the research complex such as CSIR as a whole; average impact factor and normalised impact factor for each scientist of a laboratory and the research complex; spectral distribution of papers falling within various ranges of impact factors and normalised impact factors. By comparing the performances over several years the trend of research activity of each laboratory can also be obtained.Paper presented at the International Conference on Science Indicators for Developing Countries, Paris, 15–19 October, 1990.     

Topic-based Pagerank: toward a topic-level scientific evaluation

Within the same research field, different subfields and topics may exhibit varied citation behaviors and scholarly communication patterns. For a more effect scientific evaluation at the topic level, this study proposes a topic-based PageRank approach. This approach aims to evaluate the scientific impact of research entities (e.g., papers, authors, journals, and institutions) at the topic-level. The proposed topic-based PageRank, when applied to a data set on library and information science publications, has effectively detected a variety of research topics and identified authors, papers, and journals of the highest impact from each topic. Evaluation results show that compared with the standard PageRank and a topic modeling technique, the proposed topic-based PageRank has the best performance on relevance and impact. Different perspectives of organizing scientific literature are also discussed and this study recommends the mode of organization that integrates stable research domains and dynamic topics.     

Six years of Spanish scientific activity in physics and engineering through INSPEC and COMPENDEX

This study is an analysis of six years of Spanish bibliography retrieved from INSPEC and COMPENDEX. The quantitative evolution of the scientific activity by years and Institutions, the recent tendencies to publish in foreign journals, as well as to have the papers signed by more authors are followed. The most frequently used journals are ranked according to their impact factor and subject. Some hypothesis are formulated and tested, trying to find a relationship between the growth of the Spanish scientific activity and its quality.     

The SCI Journal Citation Reports: A potential tool for studying journals?

In this paper, we analysed six indicators of the SCI Journal Citation Reports (JCR) over a 19-year period: number of total citations, number of citations to the two previous years, number of source items, impact factor, immediacy index and cited half-life. The JCR seems to have become more or less an authority for evaluating scientific and technical journals, essentially through its impact factor. However it is difficult to find one's way about in the impressive mass of quantitative data that JCR provides each year. We proposed the box plot method to aggregate the values of each indicator so as to obtain, at a glance, portrayals of the JCR population from 1974 to 1993. These images reflected the distribution of the journals into 4 groups designated low, central, high and extreme. The limits of the groups became a reference system with which, for example, it was rapidly possible to situate visually a given journal within the overall JCR population. Moreover, the box plot method, which gives a zoom effect, made it possible to visualize a large sub-population of the JCR usually overshadowed by the journals at the top of the rankings. These top level journals implicitly play the role of reference in evaluation processes. This often incites categorical judgements when the journals to be evaluated are not part of the top level. Our «rereading» of the JCR, which presented the JCR product differently, made it possible to qualify these judgements and bring a new light on journals.     

Impact of open access on citation of scholarly publications in the field of civil engineering

The development of science is accompanied by growth of scholarly publications, primarily in the form of articles in peer-reviewed journals. Scientific work is often evaluated through the number of scientific publications in international journals and their citations. This article discusses the impact of open access (OA) on the number of citations for an institution from the field of civil engineering. We analyzed articles, published in 2007 in 14 international journals with impact factor, which are included in the Journal Citation Reports subject category “Civil Engineering”. The influence of open access on the number of citations was analyzed. The aim of our research was to determine if open access articles from the field of civil engineering receive more citations than non-open access articles. Based on the value of impact factor and ranking in quartiles, we also looked at the influence of the rank of journals on the number of citations, separately for OA and Non OA articles, in databases Web of Science (WOS), Scopus and Google Scholar. For 2,026 studied articles we found out that 22 % of them were published as OA articles. They received 29 % of all citations in the observed period. We can conclude by the significance level 5 % or less that in the databases WOS and Scopus the articles from top ranked journals (first quartile) achieved more citations than Non OA articles. This argument can be confirmed for some other journals from second quartile as well, while for the journals ranked into the third quartile it can’t be confirmed. This could be confirmed only partly for journals from the second quartile, and would not be confirmed for journals ranked into the third quartile. This shows that open access is not a sufficient condition for citation, but increases the number of citations for articles published in journals with high impact.     

The evaluation of scientific production: Towards a neutral impact factor

Measurement of research activity still remains a controversial question. The use of the impact factor from the Institute for Scientific Information (ISI) is quite widespread nowadays to carry out evaluations of all kinds; however, the calculation formula employed by ISI in order to construct its impact factors biases the results in favour of knowledge fields which are better represented in the sample, cite more in average and whose citations are concentrated in the early years of the articles. In the present work, we put forward a theoretical proposal regarding how aggregated normalization should be carried out with these biases, which allows comparing scientific production between fields, institutions and/or authors in a neutral manner. The technical complexity of such work, together with data limitations, lead us to propose some adjustments on the impact factor proposed by ISI which — although they do not completely solve the problem — reduce it and allow glimpsing the path towards more neutral evaluations. The proposal is empirically applied to three analysis levels: single journals, knowledge fields and the set of journals from the Journal Citation Report.     

Online determination of the journal impact factor and its international properties

The article describes the method for the online determination of the journal impact factor (JIF). The method is very simple and can be used both for the ISI defined journal impact factor and for the calculation of other generalised journal impact factors. But the direct online method fails for non-ISI journals i.e. journals not indexed by ISI to the three citation databases. For such journals only the “External Cited Impact Factor” associated with citations from ISI journals (ECIFisi) can be determined online by the common method. As an extra benefit the online method makes available the determination of the geographical distribution of citations and citable units in relation to any given JIF, i.e. the international impact for a particular journal in a given year. The method is illustrated by calculating the generalised JIF, self-citations and ECIF(isi) as well as the international impact for Journal of Documentation and Scientometrics.     

Journal impact measures in bibliometric research

The Impact Factor introduced by Eugene Garfield is a fundamental citation-based measure for significance and performance of scientific journals. It is perhaps the most popular bibliometric product used in bibliometrics itself, as well as outside the scientific community. First, a concise review of the background and history of the ISI impact factor and the basic ideas underlying it are given. A cross-citation matrix is used to visualise the construction of the Impact Factor and several related journal citation measures.Both strengths and flaws of the impact factor are discussed. Several attempts made by different authors to introduce more sophisticated journal citation measures and the reasons why many indicators aiming at a correction of methodological limitations of the Impact Factor were not successful are described. The next section is devoted to the analysis of basic technical and methodological aspects. In this context, the most important sources of possible biases and distortions for calculation and use of journal citation measures are studied. Thereafter, main characteristics of application contexts are summarised. The last section is concerned with questions of statistical reliability of journal citation measures. It is shown that in contrast to a common misbelief statistical methods can be applied to discrete "skewed" distributions, and that the statistical reliability of these statistics can be used as a basis for application of journal impact measures in comparative analyses. Finally, the question of sufficiency or insufficiency of a single, howsoever complex measure for characterising the citation impact of scientific journals is discussed. This revised version was published online in June 2006 with corrections to the Cover Date.     

Advantages and limitations in the use of impact factor measures for the assessment of research performance

Impact factor is a quasi-qualitative indicator, which provides a measurement of the prestige and international visibility of journals. Although the use of impact factor-based indicators for science policy purposes has increased over the last two decades, several limitations have been pointed out and should be borne in mind. The use of impact factor should be treated carefully when applied to the analysis of peripheral countries, whose national journals are hardly covered by ISI databases. Our experience in the use of impact factor based indicators for the analysis of the Spanish scientific production is shown. The usefulness of the impact factor measures in macro, meso and micro analyses is displayed. In addition, the main advantages, such as the great accessibility of impact factor and its ready-to-use nature are pointed out. Several limitations such as the need to avoid inter-field comparisons or the convenience of using a fixed journal set for international comparisons are also stressed. It is worth noting that the use of impact factor in the research evaluation process has influenced strongly the publication strategy of scientists. This revised version was published online in June 2006 with corrections to the Cover Date.     

Research productivity of scientists in consolidated vs. non-consolidated teams: The case of Spanish university geologists

We present some results of an evaluation of research performance of Spanish senior university researchers in Geology. We analyse to what extent productivity of individual researchers is influenced by the level of consolidation of the team they belong to. Methodology is based on the combination of a mail survey carried out among a defined set of researchers, and a bibliometric study of their scientific output. Differences among researchers have been investigated with regard to team size and composition, patterns of publication in domestic and foreign journals, productivity, co-authorship of papers, and impact of publications. Results indicate that not belonging to a research team represents a handicap at the time of publishing in top international journals. Researchers belonging to consolidated teams are more productive than their colleagues in non-consolidated teams, and these in turn more than individuals without team. Team size does not appear to be as important for scientific productivity as the number of researchers within the team that reached a stable job position. Analysis of the impact factor of journals has not revealed differences among researchers with regard to the visibility of their papers. This revised version was published online in June 2006 with corrections to the Cover Date.     

Renormalized Impact Factor

Many aspects determine the quality of scientific journals. The impact factor is one of these quantitative parameters. However, the impact factor has a strong dependence on the journal discipline. This dependence forbids a direct comparison between different journals without introducing external considerations. In this paper, a renormalized impact factor, F_r, inspired in the definition of dimensionless physical parameters, is proposed. F_r allows a direct comparison among journals classified into different categories and, furthermore, the time evolution analysis of the journal's role in its field.     

Bibliometric positioning of scientific manufacturing journals: a comparative analysis

This article analyzes some of the most popular scientific journals in the Manufacturing field from the point of view of four bibliometric indicators: the ISI impact factor (ISI-IF), the Hirsch (h) index-for-journal, the total number of citations and the h-spectrum. h-spectrum is a novel tool based on h, making it possible to (i) identify a reference profile of the typical authors of a journal, (ii) compare different journals and (iii) provide a rough indication of their ??bibliometric positioning?? in the scientific community. Results of this analysis can be helpful for guiding potential authors and members of the scientific community in the Manufacturing area. Of particular interest is the construction of maps based on h-spectrum and ISI-IF to compare journals and monitor their bibliometric positioning over time. A large amount of empirical data are presented and discussed.     

The value of an overseas research trip

As an essential part of the academic environment, international scientific mobility draws considerable attention from researchers. Previous studies have indicated a strong relationship between scientific mobility and scientific output. However, few researchers have addressed the causality between them. The research questions in this study focused on how the international scientific mobilization of the researchers affects their number of international collaborations, their ability to get published at higher impact factor journals, the number of citations that they get. Based on the SCOPUS database of English language scientific journal articles, this paper revealed the causal effects of international scientific mobility of the researchers on their scientific productivity, collaborations, and impact on science using the synthetic control method. The author’s affiliation on their articles provided the geographical location that can be tracked in time to infer the international scientific mobility of each author. A sample of more than 79,000 immobile scientists was used to create the synthetic versions of over 1500 internationally mobile scientists, so that, the synthetic version of each mobile author best resembled the academic ability of her/his counterpart mobile author in the pre-mobilization period. This allowed investigating the effects of the international mobilization on their publications by comparing the post-mobilization publication characteristics of the mobile authors and their immobile synthetic controls.The findings show strong evidence of a substantial positive effect of scientific mobility on the ability to get published in more prestigious journals, the number of citations received in total and from overseas, and international collaborations. The magnitude of the effect is conditional on the duration of scientific mobility.

     

What makes a great journal great in the sciences? Which came first,the chicken or the egg?

The paper is concerned with analysing what makes a great journal great in the sciences, based on quantifiable Research Assessment Measures (RAM). Alternative RAM are discussed, with an emphasis on the Thomson Reuters ISI Web of Science database (hereafter ISI). Various ISI RAM that are calculated annually or updated daily are defined and analysed, including the classic 2-year impact factor (2YIF), 5-year impact factor (5YIF), Immediacy (or 0-year impact factor (0YIF)), Eigenfactor, Article Influence, C3PO (Citation Performance Per Paper Online), h-index, Zinfluence, PI-BETA (Papers Ignored—By Even The Authors), Impact Factor Inflation (IFI), and three new RAM, namely Historical Self-citation Threshold Approval Rating (H-STAR), 2 Year Self-citation Threshold Approval Rating (2Y-STAR), and Cited Article Influence (CAI). The RAM data are analysed for the 6 most highly cited journals in 20 highly-varied and well-known ISI categories in the sciences, where the journals are chosen on the basis of 2YIF. The application to these 20 ISI categories could be used as a template for other ISI categories in the sciences and social sciences, and as a benchmark for newer journals in a range of ISI disciplines. In addition to evaluating the 6 most highly cited journals in each of 20 ISI categories, the paper also highlights the similarities and differences in alternative RAM, finds that several RAM capture similar performance characteristics for the most highly cited scientific journals, determines that PI-BETA is not highly correlated with the other RAM, and hence conveys additional information regarding research performance. In order to provide a meta analysis summary of the RAM, which are predominantly ratios, harmonic mean rankings are presented of the 13 RAM for the 6 most highly cited journals in each of the 20 ISI categories. It is shown that emphasizing THE impact factor, specifically the 2-year impact factor, of a journal to the exclusion of other informative RAM can lead to a distorted evaluation of journal performance and influence on different disciplines, especially in view of inflated journal self citations.