A New Approach to the SCI Journal Citation Reports,a System for Evaluating Scientific Journals

The Science Citation Index, Journal Citation Reports (JCR), published by the Institute for Scientific Information (ISI) and designed to rank, evaluate, categorize and compare journals, is used in a wide scientific context as a tool for evaluating researchers and research work, through the use of just one of its indicators, the impact factor. With the aim of obtaining an overall and synthetic perspective of impact factor values, we studied the frequency distributions of this indicator using the box-plot method. Using this method we divided the journals listed in the JCR into five groups (low, lower central, upper central, high and extreme). These groups position the journal in relation to its competitors. Thus, the group designated as extreme contains the journals with high impact factors which are deemed to be prestigious by the scientific community. We used the JCR data from 1996 to determine these groups, firstly for all subject categories combined (all 4779 journals) and then for each of the 183 ISI subject categories. We then substituted the indicator value for each journal by the name of the group in which it was classified. The journal group may differ from one subject category to another. In this article, we present a guide for evaluating journals constructed as described above. It provides a comprehensive and synthetic view of two of the most used sections of the JCR. It makes it possible to make more accurate and complete judgements on and through the journals, and avoids an oversimplified view of the complex reality of the world of journals. It immediately reveals the scientific subject category where the journal is best positioned. Also, whereas it used to be difficult to make intra- and interdisciplinary comparisons, this is now possible without having to consult the different sections of the JCR. We construct this guide each year using indicators published in the JCR by the ISI.     

CAB: Citation-Assisted Background

Summary The assessment of scientific journals is of particular interest to South Africa's higher education institutions as their research is partly funded according to the number of publications of their members of staff. This article has two objectives. The first one is to identify the effects of the government's withdrawal of financial support on these journals' impact factors. The second objective is to provide an assessment of the visibility of the South African journals indexed in the Journal Citation Report (JCR) of the 2002. The findings indicate that the termination of the government interference in the affairs of the journals had on average a beneficial effect on the impact factors of the journals. South Africa is found to have a good representation in the JCR, similar or better to that of the scientifically small countries in Europe, and represents approximately 90% of the African continent journals in the JCR. The visible scientific disciplines are identified and the journals are assessed according to their impact factors, to the impact factors of journals citing them, and the self-citing and self-cited rates.     

The SCI Journal Citation Reports: A potential tool for studying journals?

In this paper, we analysed six indicators of the SCI Journal Citation Reports (JCR) over a 19-year period: number of total citations, number of citations to the two previous years, number of source items, impact factor, immediacy index and cited half-life. The JCR seems to have become more or less an authority for evaluating scientific and technical journals, essentially through its impact factor. However it is difficult to find one's way about in the impressive mass of quantitative data that JCR provides each year. We proposed the box plot method to aggregate the values of each indicator so as to obtain, at a glance, portrayals of the JCR population from 1974 to 1993. These images reflected the distribution of the journals into 4 groups designated low, central, high and extreme. The limits of the groups became a reference system with which, for example, it was rapidly possible to situate visually a given journal within the overall JCR population. Moreover, the box plot method, which gives a zoom effect, made it possible to visualize a large sub-population of the JCR usually overshadowed by the journals at the top of the rankings. These top level journals implicitly play the role of reference in evaluation processes. This often incites categorical judgements when the journals to be evaluated are not part of the top level. Our «rereading» of the JCR, which presented the JCR product differently, made it possible to qualify these judgements and bring a new light on journals.     

Citation analysis significance of scientific journals

The application of methods of quantitative analysis makes it possible to evaluate the impact of scientific journals on one another. These methods are used to determine the significance of similar scientific journals by their cross-citations, taking into account data from theJournal Citation Reports (JCR). They also help to improve theJournal Citation Reports structure and widen its uses for the evaluation of scientific journals. The above methods are applied to analyse critically the principles of ranking journals in package 1 and the tabular contents ofJCR's packages 2 and 3, as well as to study frequency distributions of the journals both in time and space.     

A new approach to measuring scientific production in JCR journals and its application to Spanish public universities

Scientific production has been evaluated from very different perspectives, the best known of which are essentially based on the impact factors of the journals included in the Journal Citation Reports (JCR). This has been no impediment to the simultaneous issuing of warnings regarding the dangers of their indiscriminate use when making comparisons. This is because the biases incorporated in the elaboration of these impact factors produce significant distortions, which may invalidate the results obtained. Notable among such biases are those generated by the differences in the propensity to cite of the different areas, journals and/or authors, by variations in the period of materialisation of the impact and by the varying presence of knowledge areas in the sample of reviews contained in the JCR. While the traditional evaluation method consists of standardisation by subject categories, recent studies have criticised this approach and offered new possibilities for making inter-area comparisons. In view of such developments, the present study proposes a novel approach to the measurement of scientific activity, in an attempt to lessen the aforementioned biases. This approach consists of combining the employment of a new impact factor, calculated for each journal, with the grouping of the institutions under evaluation into homogeneous groups. An empirical application is undertaken to evaluate the scientific production of Spanish public universities in the year 2000. This application considers both the articles published in the multidisciplinary databases of the Web of Science (WoS) and the data concerning the journals contained in the Sciences and Social Sciences Editions of the Journal Citation Report (JCR). All this information is provided by the Institute of Scientific Information (ISI), via its Web of Knowledge (WoK).     

Some citation-related characteristics of scientific journals published in individual countries

Relationships between publication language, impact factors and self-citations of journals published in individual countries, eight from Europe and one from South America (Brazil), are analyzed using bibliometric data from Thomson Reuters JCR Science Edition databases of ISI Web of Knowledge. It was found that: (1) English-language journals, as a rule, have higher impact factors than non-English-language journals, (2) all countries investigated in this study have journals with very high self-citations but the proportion of journals with high self-citations with reference to the total number of journals published in different countries varies enormously, (3) there are relatively high percentages of low self-citations in high subject-category journals published in English as well as non-English journals but national-language journals have higher self-citations than English-language journals, and (4) irrespective of the publication language, journals devoted to very specialized scientific disciplines, such as electrical and electronic engineering, metallurgy, environmental engineering, surgery, general and internal medicine, pharmacology and pharmacy, gynecology, entomology and multidisciplinary engineering, have high self-citations.     

Comparing two “thermometers”: Impact factors of 20 leading economic journals according to Journal Citation Reports and Scopus

Impact factors for 20 journals ranked first by Journal Citation Reports (JCR) were compared with the same indicator calculated on the basis of citation data obtained from Scopus database. A significant discrepancy was observed as Scopus, though results differed from title to title, found in general more citations than listed in JCR. This also affected ranking of the journals. More thorough examination of two selected titles proved that the divergence resulted mainly from difference in coverage of two products, although other important factors also play their part.     

The visibility of Italian journals

The main purpose of this study was to analyze the Italian journals indexed in the 2000 edition of the Journal Citation Reports (JCR) published by the Institute for Scientific Information (ISI) (Philadelphia, USA). The performance and the visibility of these journals were evaluated in terms of Impact Factor (IF), mean IF from citing journals and cited journals, and self-citing and self-cited rates. Seventy-three Italian journals were indexed in the JCR, 14 of which achieved an IF equal to or higher than one. Most citing journals were European and American, thus showing a fairly good visibility of the articles published in the 14 journals analyzed. The self-citing and self-cited rates showed a wide variation. The journal that appeared to perform best was theJournal of High Energy Physics, an electronic publication whose success seemingly confirms Internet circulation as an effective means to enhance the visibility and consequently the quality, in term of citations, of a journal. Italy's low overall expenditure on research & development (R&D) and low number of researchers compared to countries with longstanding high publishing standards and traditions are no doubt partly to blame for its poor performance in scientific publishing. This revised version was published online in August 2006 with corrections to the Cover Date.     

The effect of additional citations in the stability of Journal Citation Report categories

We use a new approach to study the ranking of journals in JCR categories. The objectives of this study were to empirically evaluate the effect of increases in citations on the computation of the journal impact factor (JIF) for a large set of journals as measured by changes in JIF, and to ascertain the influence of additional citations on the rank order of journals according their new JIFs within JCR groups. To do so, modified JIFs were computed by adding additional citations to the number used by Thomson-Reuters to compute the JIF of journals listed in the JCR for 2008. We considered the effect on rank order of a given journal of adding 1, 2, 3 or more citations to the number used to compute the JIF, keeping everything else equal (i.e., without changing the JIF of other journals in a given group). The effect of additional citations on the internal structure of rankings in JCR groups increased with the number of citations added. In about one third of JCR groups, about half the journals changed their rank order when 1–5 citations were added. However, in general the rank order tended to be relatively stable after small increases in citations.     

The peripheral terrorism literature: Bringing it closer to the core

Summary Core/periphery scientific communication is important for information transfer in terrorism literature. The mutual awareness between disciplinary journals contributors in the mainstream and those in the margins of the field enhances their social interaction. The usual case is that the mainstream of a discipline is visible through such indexes as the Web of Science (SCI) and the Journal Citation Report (JCR) the second of which assigns an impact factor to the most cited journals. In terrorism subject area, however, the reverse situation exists; only the peripheral journals in this field are indexed in JCR. From a scientific communication perspective, then, the core journals of terrorism writings are relatively invisible. This study attempts to identify the core and the periphery of journals dealing with terrorism, and suggests a way to bring them closer together. The assumption is that the quality and quantity of work in this field will increase as the distance between these two poles decreases.     

Visualizing the intellectual structure in urban studies: A journal co-citation analysis (1992-2002)

Summary Core/periphery scientific communication is important for information transfer in terrorism literature. The mutual awareness between disciplinary journals contributors in the mainstream and those in the margins of the field enhances their social interaction. The usual case is that the mainstream of a discipline is visible through such indexes as the Web of Science (SCI) and the Journal Citation Report (JCR) the second of which assigns an impact factor to the most cited journals. In terrorism subject area, however, the reverse situation exists; only the peripheral journals in this field are indexed in JCR. From a scientific communication perspective, then, the core journals of terrorism writings are relatively invisible. This study attempts to identify the core and the periphery of journals dealing with terrorism, and suggests a way to bring them closer together. The assumption is that the quality and quantity of work in this field will increase as the distance between these two poles decreases.     

The influence of references per paper in the SCI to Impact Factors and the Matthew Effect

All references data was extracted from the annual volumes of the CD-Edition of Science Citation Index (SCI) and the Web of Science of the Institute for Scientific Information (ISI), the journal citation and self-citation data extracted from the Journal Citation Report (JCR), the self-citing rate and self-cited rate calculated based on the JCR method. To determine the trend of mean value of references per paper throughout 1970–2005, a total number of 10,000 records were randomly chosen for each year of under study, and the mean value of references per paper was calculated. To determine the growth of journals IF a total number of 5,499 journals were chosen in the JCR in 2002 and the same set of journals in the year 2004. To show the trend of journals IF, all journals indexed in the JCR throughout 1999–2005 were extracted and the mean values of their IFs was calculated annually. The study showed that the number of references per paper from 1970 to 2005 has steady increased. It reached from 8.40 in 1970 to 34.63 in 2005, an increase of more than 4 times. The majority of publications (76.17%) were in the form of Journals Article. After articles, Meeting Abstracts (9.46%), Notes (3.90%) and Editorial Material (3.78%) are the most frequented publication forms, respectively. 94.57% of all publications were in English. After English, German (1.50%), Russian (1.48%) and French (1.37%) were the most frequented languages, respectively. The study furthermore showed that there is a significant correlation between the IF and total citation of journals in the JCR, and there is an important hidden correlation between IF and the self-citation of journals. This phenomena causes the elevation of journals IF. The more often a journal is citing other journals, the more often it is also cited (by a factor of 1.5) by others. In consequence the growing percentage of journal self-citation is followed by journal self-citedness, which can be considered as the Matthew Effect. There is a linear correlation between journal self-citing and journal self-cited value, the mean value of self-cited rate always stays higher than the self-citing rate. The mean value of self-cited rate in 2000 was 14% and the mean value of self-citing rate is 6.61%, whereas the mean value of self-cited rate in 2005 was 12% and the mean value of self-citing rate was 7.81%.     

Some aspects of citation indexes in Spain: A comparative analysis

This paper studies the main characteristics of the citation indexes currently developed in Spain. The paper compares the impact factors offered by Spanish citation indexes with the impact factor of Spanish journals also collected by the JCRs of the ISI (SCI and SSCI) over a five-year period (2001–2005). Spanish journals published in English have higher impact factor scores in the JCR databases of the ISI than in Spanish citation indexes.     

Disciplinarity and interdisciplinarity in citation and reference dimensions: knowledge importation and exportation taxonomy of journals

This work proposes an entropy-based disciplinarity indicator (EBDI) which allows the classification of scientific journals in four classes: knowledge importer, knowledge exporter, disciplinary and interdisciplinary with regards to the discipline(s) in which they are classified. Assuming that the set references in the papers published in a journal represent a significant part of their knowledge basis, the diversity (measured with Shannon’s entropy) and ratio between internal and external (to the discipline in which the journal is classified) references can provide a measure of the disciplinarity/interdisciplinarity of the journal in the reference dimension. The homologous analysis can be applied to the set of citations received by the papers published in the journal. In this article, an entropy-based indicator for the measurement of the disciplinarity of scientific journals is developed, applied (to the cited and citing dimensions) and discussed. The indicator can take finite values and it is found to be theoretically consistent when tested against two definitions for bibliometric indicators. The combinations of disciplinarity values in the citing and cited dimensions permits the classification of journals according to their knowledge importing/exporting profile (separately, with regards to the social sciences or the sciences), providing a taxonomy of the role of journals according to their importing, exporting, interdisciplinary or specialized profile with regards to the subject category in which they are classified. The indicator, EBDI and the resulting taxonomy is proposed and tested for the set of journals in LIS subject category in JCR 2013 and for the sets of journals in Andrology and Legal Medicine in JCR 2015. Evidence of concurrent validity with journal co-classification patterns is found in the three sets of journals.     

Fluctuation analysis of discipline development based on impact factor

Summary  Based on the impact factors of the journals recorded by JCR from 1998 to 2003, this paper established the fluctuation model for discipline development. According to the Fluctuation Strength Coefficient, then we gave analysis and evaluation of developing trends of the disciplines in recent years.     

Towards appropriate indicators of journal impact

This paper reviews a range of studies conducted by the authors on indicators reflecting scholarly journal impact. A critical examination of the journal impact data in theJournal Citation Reports (JCR), published by the Institute for Scientific Information (ISI) has shown that the JCR impact factor is inaccurate and biased towards journals revealing a rapid maturing or decline in impact. In addition, it was found that the JCR cited half life is an inappropriate measure of decline of journal impact. More appropriate impact measures of scholarly journals are proposed. A new classification system is explored, describing both maturing and decline of journal impact as measured through citations. Suggestions for future research are made, analysing in more detail the distribution of citations among papers in a journal.     

On the impact of Gold Open Access journals

Gold Open Access (=Open Access publishing) is for many the preferred route to achieve unrestricted and immediate access to research output. However, true Gold Open Access journals are still outnumbered by traditional journals. Moreover availability of Gold OA journals differs from discipline to discipline and often leaves scientists concerned about the impact of these existent titles. This study identified the current set of Gold Open Access journals featuring a Journal Impact Factor (JIF) by means of Ulrichsweb, Directory of Open Access Journals and Journal Citation Reports (JCR). The results were analyzed regarding disciplines, countries, quartiles of the JIF distribution in JCR and publishers. Furthermore the temporal impact evolution was studied for a Top 50 titles list (according to JIF) by means of Journal Impact Factor, SJR and SNIP in the time interval 2000–2010. The identified top Gold Open Access journals proved to be well-established and their impact is generally increasing for all the analyzed indicators. The majority of JCR-indexed OA journals can be assigned to Life Sciences and Medicine. The success-rate for JCR inclusion differs from country to country and is often inversely proportional to the number of national OA journal titles. Compiling a list of JCR-indexed OA journals is a cumbersome task that can only be achieved with non-Thomson Reuters data sources. A corresponding automated feature to produce current lists “on the fly” would be desirable in JCR in order to conveniently track the impact evolution of Gold OA journals.