Analyzing IS research productivity: an inclusive approach to global IS scholarship

An increasing number of studies have appeared that evaluate and rank journal quality and the productivity of IS scholars and their institutions. In this paper, we describe the results of one recent study identifying the ‘Top 30’ IS Researchers, revealing many unexamined assumptions about which IS publication outlets should be included in any definition of high-quality, scholarly IS journals. Drawing from the argument that all categories and classification schemes are grounded in politics, we critique the process by which the recent study in question (and several earlier studies) have derived the set of journals from which they count researcher publications. Based on a critical examination of the widespread inclusion of practitioner outlets, and the consistent exclusion of European scholarly IS journals, we develop our own arguments for which journals should be included in such evaluations of researcher productivity. We conduct our own analysis of IS researcher productivity for the period 1999–2003, based on articles published in a geographically balanced set of 12 IS journals, and then we compare our results with those from the recent study in question and their predecessors. Our results feature a more diverse set of scholars – both in terms of location (specifically, the country and continent in which the researchers are employed) and gender. We urge future studies of IS research productivity to follow our practice of including high-quality European journals, while eschewing practitioner-oriented publications (such as Harvard Business Review and Communications of the ACM). We also advocate that such studies count only research contributions (e.g., research articles), and that other genres of non-research articles – such as book reviews, ‘issues and opinions’ pieces and editorial introductions – not be conflated with counts of research contributions.     

Determinants of software quality: A survey of information systems project managers

Software quality is important for the success of any information systems (IS). In this research, we find the determinants of software quality. We used five attributes for software quality: system reliability, maintainability, ease of use, usefulness, and relevance. By surveying 112 IS project managers, we collected data about their perceptions on the software quality attributes and their determinants. We arrived at six factors through exploratory factor analysis. We determined the individual factors that impacted the software quality attributes; for example, reliability is associated with responsiveness of IS department; ease of use is influenced by the capabilities of users and attitude of management; and usefulness is impacted by capabilities of IS department and responsiveness of IS department. We show that organizational factors are more important than technical factors in impacting software quality in IS projects. We provide implications of our research to practice and to future research.     

Reconsidering deconstruction in information systems research

Deconstruction, a post-structuralist approach to examining language in texts, is most often associated with the philosophical works of Jacque Derrida. After a flurry of interest among management and information systems (IS) scholars, this qualitative approach to exploring organizational texts has received little attention in the IS literature. We suggest deconstruction could help our field explore how IS texts describe the social and technical past and also prescribe and circumscribe the future of IS practice. Thus, we suggest the IS field reconsider how deconstruction might contribute to language-based approaches in IS research and practice. In this paper, we discuss deconstruction in light of the linguistic turn in social science research and the support and criticism for its use in management research. We consider IS research publications that have employed deconstruction explicitly, examine a well-known IS publication as an example of the deconstruction of IS texts, and suggest ways in which deconstruction might be applied to various genres of IS texts to inspire insights and creativity. To conclude, we highlight considerations for researchers who might adopt this approach and for the editors and reviewers who would evaluate qualitative research papers that employ deconstruction.     

Information systems action research: Debunking myths and overcoming barriers

The relevance of action research as a research method in the information systems (IS) discipline is not disputed. Nevertheless, the extent to which action research is published in good journals is infrequent enough to indicate a serious problem. In this article, we explore the reasons underlying this situation and make recommendations aiming to increase both the practice and the publication of action research. To identify both the barriers to undertaking action research and potential ways of overcoming those barriers, we survey 218 authors of 120 articles demonstrating empirical action research published in 12 of our good journals during the period 1982–2016. We received 70 usable responses. We also surveyed 52 editors of selected IS journals and received 25 usable responses. Our findings are revealing as they indicate both genuine barriers associated with action research and some apparent barriers that are in reality misperceptions or myths. In reflecting on these, we emphasize the special qualities of action research. We also reflect on the critical role that action research plays in the IS field as a whole and its potential for further contributions to research and practice, given the strong and close connections with organizational problem contexts that action research requires. Finally, we make a number of recommendations that are designed to increase the incidence of action research in the IS discipline     

Organizational impact of system quality,information quality,and service quality

Increased organizational dependence on information systems drives management attention towards improving information systems’ quality. A recent survey shows that “Improve IT quality” is one of the top concerns facing IT executives. As IT quality is a multidimensional measure, it is important to determine what aspects of IT quality are critical to organizations to help Chief Information Officers (CIOs) to devise effective IT quality improvement strategies. In this research, we model the relationship between information systems’ (IS) quality and organizational impact. We hypothesize greater organizational impact in situations in which system quality, information quality and service quality are high. We also hypothesize a positive relationship between system quality and information quality. We test our hypotheses using survey data. Our structural equation model exhibits a good fit with the observed data. Our results show that IS service quality is the most influential variable in this model (followed by information quality and system quality), thus highlighting the importance of IS service quality for organizational performance. This paper contributes theoretically to IS success models through the system quality-to-information quality and IS quality-to-organizational impact links. Implications of our results for practice and research are discussed.     

Information systems investment decisions in business practice: the Spanish case

Previous research has reviewed the foundations of the most significant methods for the evaluation of investments in information systems (IS). However, in our view, it is also necessary to provide an updated view on the evaluation methods actually used in practice by organizations. This paper offers a view on the investment in IS in business practice in Spain, analysing: (i) the evaluation methods currently used by Spanish firms; (ii) how companies make decisions concerning investments in IS; (iii) the nature and quality of the decisions; and (iv) the consequences derived from them.     

Why one big picture is worth a thousand numbers: measuring intangible benefits of investments in augmented reality based assistive technology using utility effect chains and system dynamics

The body of literature in the area of IT/IS investment evaluation proposes a variety of methods and techniques for the ex-ante assessment of IT/IS investments. However, the financial evaluation of intangible benefits associated with IT/IS investments in general and investments in innovative IT/IS in particular still remains a problem area of high relevance to decision makers. As a consequence, investment decisions are still taken by practitioners as an “act of faith”. With this in mind, it is our primary aim to develop a quantification model for the financial assessment of intangible benefits concerning investments in innovative IT/IS. Based on an augmented reality and smart glasses application scenario from the construction domain, we demonstrate how intangible benefits of innovative IT/IS investments can be visualised and measured more effectively by means of utility effect chains and system dynamics prior to their incorporation into a cost–benefit analysis. Based on design science research, the quantification model is developed by means of a systematic literature review and evaluated using the augmented reality application scenario in construction. The model can serve as an aid to the managerial decision making process by providing an enhanced understanding of the various tangible and intangible benefits associated with the investment.     

Why security and privacy research lies at the centre of the information systems (IS) artefact: proposing a bold research agenda

In this essay, we outline some important concerns in the hope of improving the effectiveness of security and privacy research. We discuss the need to re-examine our understanding of information technology and information system (IS) artefacts and to expand the range of the latter to include those artificial phenomena that are crucial to information security and privacy research. We then briefly discuss some prevalent limitations in theory, methodology, and contributions that generally weaken security/privacy studies and jeopardise their chances of publication in a top IS journal. More importantly, we suggest remedies for these weaknesses, identifying specific improvements that can be made and offering a couple of illustrations of such improvements. In particular, we address the notion of loose re-contextualisation, using deterrence theory research as an example. We also provide an illustration of how the focus on intentions may have resulted in an underuse of powerful theories in security and privacy research, because such theories explain more than just intentions. We then outline three promising opportunities for IS research that should be particularly compelling to security and privacy researchers: online platforms, the Internet of things, and big data. All of these carry innate information security and privacy risks and vulnerabilities that can be addressed only by researching each link of the systems chain, that is, technologies–policies–processes–people–society–economy–legislature. We conclude by suggesting several specific opportunities for new research in these areas.     

Improving scale adaptation practices in information systems research: Development and validation of a cognitive validity assessment method

Scale adaptation, where authors alter the wording of an already published scale, is a deeply rooted social practice in IS research. This paper argues that the time is ripe to question this activity as well as the beliefs that have progressively formed around it. We identify and challenge five fallacious scale adaptation beliefs that hinder the development of more robust measure development norms. Contributing to this area of research, this paper offers a conceptual definition of the cognitive validity concept, defined as the extent to which a scale is free of problematic item characteristics (PICs) that bias the survey response process and subsequent empirical results. Building on this conceptualization effort, a new methodological process for assessing the cognitive validity of adapted IS measures is introduced. Through a series of three programmatic studies, we find converging evidence that the method can benefit the IS field by making the scale adaptation process more robust, transparent, and consistent. Along with the method, we introduce a new index that IS scholars can use to benchmark the cognitive quality of their scales against venerable IS measures. We discuss the implications of our work for IS research (including detailed implementation guidelines) and provide directions for future research on measurement in IS.     

The impact of IT outsourcing on information systems success

The objective of this research is to assess the impact of IT outsourcing on Information Systems’ success. We modeled the relationships among the extent of IT outsourcing, the ZOT (the Zone of Tolerance), and IS success. We justified our model using the expectancy–disconfirmation theory, the agency theory, and transaction cost economics, and we empirically tested it using structural equation modeling with responses from IS users. We found significant direct and indirect effects (through the service quality) of outsourcing on IS systems’ perceived usefulness and their users’ satisfaction. Whereas the extent of outsourcing is negatively related to the service quality and perceived usefulness, the ZOT-based IS service quality is positively related to the user satisfaction.     

Understanding strategy assessment in IS management

Information Systems and e-Business Management - In our research, we suggest a process theory for explaining the strategy assessment process and its effect in information systems (IS) planning. The...     

Epistemological perspectives on IS research: a framework for analysing and systematizing epistemological assumptions

Abstract.  Over the last three decades, a methodological pluralism has developed within information systems (IS) research. Various disciplines and many research communities as well, contribute to this discussion. However, working on the same research topic or studying the same phenomenon does not necessarily ensure mutual understanding. Especially within this multidisciplinary and international context, the epistemological assumptions made by different researchers may vary fundamentally. These assumptions exert a substantial impact on how concepts like validity, reliability, quality and rigour of research are understood. Thus, the extensive publication of epistemological assumptions is, in effect, almost mandatory. Hence, the aim of this paper is to develop an epistemological framework which can be used for systematically analysing the epistemological assumptions in IS research. Rather than attempting to identify and classify IS research paradigms, this research aims at a comprehensive discussion of epistemology within the context of IS. It seeks to contribute to building the basis for identifying similarities as well as differences between distinct IS approaches and methods. In order to demonstrate the epistemological framework, the consensus-oriented interpretivist approach to conceptual modelling is used as an example.     

Twenty years of the European information systems academy at ECIS: emergent trends and research topics

While the information systems (IS) community is increasingly international, it is reasonable to expect that different regions might display different research approaches, interests and publication orientations. This paper contributes to the growing number of historical accounts in the IS field by further developing the profile of European IS research that was reported on in EJIS following the first 10 years of the European Conference on Information Systems (ECIS). On the basis of an analysis of all papers published in ECIS proceedings during the 10-year period 2003–2012, the paper highlights three key characteristics of the developing European IS research profile: (1) continuation of the traditional European IS research profile as developed in the first decade; (2) convergence with aspects of the North American tradition and (3) development of a distinct approach to design science. We place these observed characteristics within broader historical and contextual features such as the changing European academic landscape, with increasing pressures to ‘publish or perish’ in order to be internationally competitive. Our contribution lies in providing a contemporaneous account of the dominant contextual factors influencing the European IS academy in recent years as well as our interpretation of the developing research profile, thus informing future understanding of European IS research and the choices facing individual IS researchers.     

Management of scoring sessions in alternative assessment: the computer-assisted scoring approach

We describe a computer-assisted scoring approach in educational assessment. In this approach, scores are captured and analyzed as scoring takes place; information on scoring quality is used to provide immediate feedback to raters and make timely re-calibration and dismissal decisions. We present a conceptual model for computer-assisted scoring and describe how we used this approach to manage the scoring sessions of an assessment for teacher certification. We found that computer-assisted scoring: (a) allowed us to provide immediate feedback to raters about their scoring quality and make accurate re-calibration and dismissal decisions; and (b) did not affect the dependability of the scores or the flow of the scoring sessions. We also confirmed that, with appropriate software, raters can be trained readily to score complex performance with the aid of computers even when they have no prior experience with computers. The conceptual model allowed us to identify how close our scoring sessions were to optimal efficiency.     

Visual quality assessment algorithms: what does the future hold?

Creating algorithms capable of predicting the perceived quality of a visual stimulus defines the field of objective visual quality assessment (QA). The field of objective QA has received tremendous attention in the recent past, with many successful algorithms being proposed for this purpose. Our concern here is not with the past however; in this paper we discuss our vision for the future of visual quality assessment research. We first introduce the area of quality assessment and state its relevance. We describe current standards for gauging algorithmic performance and define terms that we will use through this paper. We then journey through 2D image and video quality assessment. We summarize recent approaches to these problems and discuss in detail our vision for future research on the problems of full-reference and no-reference 2D image and video quality assessment. From there, we move on to the currently popular area of 3D QA. We discuss recent databases, algorithms and 3D quality of experience. This yet-nascent technology provides for tremendous scope in terms of research activities and we summarize each of them. We then move on to more esoteric topics such as algorithmic assessment of aesthetics in natural images and in art. We discuss current research and hypothesize about possible paths to tread. Towards the end of this article, we discuss some other areas of interest including high-definition (HD) quality assessment, immersive environments and so on before summarizing interesting avenues for future work in multimedia (i.e., audio-visual) quality assessment.     

Crowdsourcing of information systems research

This paper addresses how technology-mediated mass collaboration offers a dramatically innovative alternative for producing IS research. We refer to this emerging genre as the crowdsourced research genre and develop a framework to structure discourse on how it may affect the production of IS research. After systematically traversing the alternative genre’s landscape using the framework, we propose a research agenda of the most substantial and imminent issues for the successful development of the genre, including contributor incentives, scholarly contribution assessment, anonymity, governance, intellectual property ownership, and value propositions. In addressing this research agenda, we reflect on what might be learned from other areas in which crowdsourcing has been established with success.     

Information quality measurement of medical encoding support based on usability

Medical encoding support systems for diagnoses and medical procedures are an emerging technology that begins to play a key role in billing, reimbursement, and health policies decisions. A significant problem to exploit these systems is how to measure the appropriateness of any automatically generated list of codes, in terms of fitness for use, i.e. their quality. Until now, only information retrieval performance measurements have been applied to estimate the accuracy of codes lists as quality indicator. Such measurements do not give the value of codes lists for practical medical encoding, and cannot be used to globally compare the quality of multiple codes lists. This paper defines and validates a new encoding information quality measure that addresses the problem of measuring medical codes lists quality. It is based on a usability study of how expert coders and physicians apply computer-assisted medical encoding. The proposed measure, named ADN, evaluates codes Accuracy, Dispersion and Noise, and is adapted to the variable length and content of generated codes lists, coping with limitations of previous measures. According to the ADN measure, the information quality of a codes list is fully represented by a single point, within a suitably constrained feature space. Using one scheme, our approach is reliable to measure and compare the information quality of hundreds of codes lists, showing their practical value for medical encoding. Its pertinence is demonstrated by simulation and application to real data corresponding to 502 inpatient stays in four clinic departments. Results are compared to the consensus of three expert coders who also coded this anonymized database of discharge summaries, and to five information retrieval measures. Information quality assessment applying the ADN measure showed the degree of encoding-support system variability from one clinic department to another, providing a global evaluation of quality measurement trends.     

The mysterious case of the missing paradigm: a review of critical information systems research 1991–2001

Abstract.  In this journal, Chen and Hirschheim have provided a historical analysis of positivist and interpretivist research paradigms and methodologies in the 10 years following the much cited work of Orlikowski and Baroudi. In this paper, we investigate the mysterious case of the missing paradigm – that of the critical approach to information systems (IS) research. We take Chen and Hirschheim's survey as our starting point and aim to fill the gap left by the absence of the critical paradigm in their analysis and make some criticisms of their method. Recent years have seen the growth of IS research that consciously adopts a critical perspective. This paper charts the development of critical IS research over the period of 1991–2001, adding some comments on more recent developments. We conclude by critically reflecting on the current development of critical research in the field of IS.     

State of the academic field of IS in India

The field of information systems (IS) remains ill-defined, facing problems of recognition and identity crisis. Several papers and debates have been dedicated on this topic for over three decades. The focus of these debates have largely centered on the state of the IS field in Americas and Europe and with a limited study conducted in the Asia–Pacific context. This study attempts to assess the field in the Indian context and highlight the challenges and issues faced by IS faculties in India. Based on literature survey of the seminal papers in the field, we identified the salient topics of interest to IS researchers: identity of the IS field; diversity of the field; research approaches, methods and rigor; reference disciplines; journals and conferences; and IS course and curriculum. A two-phase study was carried out to explore the interest generated by the IS field in India. In the first phase, secondary data review of leading business schools in India was carried out to understand their focus on IS as an academic discipline. In the second phase, we conducted interviews of IS researchers from the premium business schools in India. In our study, we analyze the research focus of IS faculties in B-schools in India, analyze the composition of the faculty pool and the nature of research work undertaken by them. Our study points at the restricted focus of IS researchers in India around a few narrow areas, with a significant breadth of topics not being explored for research. We found the identity struggle of the field to be prominent, which was further compounded by the limited number of active, IS researchers. The growing importance of qualitative techniques in IS research, the challenges of developing a sound base in multiple reference disciplines and the need for making IS course interesting for business school students were some of the other significant observations. Our study also highlighted the paradox of the success of the IT industry in India leading to reduced enrollment of IS courses in B-Schools. We conclude by providing recommendations to B-school deans and directors.     

The scholarly influence of Heinz Klein: ideational and social measures of his impact on IS research and IS scholars

Heinz Klein was a fine scholar and mentor whose work and life have inspired us to explore the notion of ‘scholarly influence’ which we cast as ‘ideational’ and ‘social influence’. We adopt a portfolio of measures approach, using the Hirsch family of statistics to assess ideational influence and Social Network Analysis centrality measures for social influence to profile Heinz Klein's contribution to information systems (IS) research. The results show that Heinz was highly influential in both ideational terms (a significant body of citations) and social terms (he is close to the heart of the IS research community). Reflecting on the major research themes and scholarly values espoused by Klein we define a ‘Kleinian view of IS research’, grounded in Habermas’ Theory of Communicative Action, and use that to frame four affirmative propositions to address what we observe to be a distortion and attenuation of the academic discourse on the evaluation of scholarly production. This paper argues that focus should be shifted from the venue of publication of the research to the uptake of the ideas contained in it, thus increasing the openness of the discourse, participation in the discourse, truthfulness, and reduction of the inequities in power distribution within academia.