A systematic review on the relationship between user involvement and system success

ContextFor more than four decades it has been intuitively accepted that user involvement (UI) during system development lifecycle leads to system success. However when the researchers have evaluated the user involvement and system success (UI-SS) relationship empirically, the results were not always positive.ObjectiveOur objective was to explore the UI-SS relationship by synthesizing the results of all the studies that have empirically investigated this complex phenomenon.MethodWe performed a Systematic Literature Review (SLR) following the steps provided in the guidelines of Evidence Based Software Engineering. From the resulting studies we extracted data to answer our 9 research questions related to the UI-SS relationship, identification of users, perspectives of UI, benefits, problems and challenges of UI, degree and level of UI, relevance of stages of software development lifecycle (SDLC) and the research method employed on the UI-SS relationship.ResultsOur systematic review resulted in selecting 87 empirical studies published during the period 1980–2012. Among 87 studies reviewed, 52 reported that UI positively contributes to system success, 12 suggested a negative contribution and 23 were uncertain. The UI-SS relationship is neither direct nor binary, and there are various confounding factors that play their role. The identification of users, their degree/level of involvement, stage of SDLC for UI, and choice of research method have been claimed to have impact on the UI-SS relationship. However, there is not sufficient empirical evidence available to support these claims.ConclusionOur results have revealed that UI does contribute positively to system success. But it is a double edged sword and if not managed carefully it may cause more problems than benefits. Based on the analysis of 87 studies, we were able to identify factors for effective management of UI alluding to the causes for inconsistency in the results of published literature.     

Encouraging user participation in a course recommender system: An impact on user behavior

User participation emerged as a critical issue for collaborative and social recommender systems as well as for a range of other systems based on the power of user community. A range of mechanisms to encourage user participation in social systems has been proposed over the last few years; however, the impact of these mechanisms on users behavior in recommender systems has not been studied sufficiently. This paper investigates the impact of encouraging user participation in the context of CourseAgent, a community-based course recommender system. The recommendation power of CourseAgent is based on course ratings provided by a community of students. To increase the number of course ratings, CourseAgent applies an incentive mechanism which turns user feedback into a self-beneficial activity. In this paper, we describe the design and implementation of our course recommendation system and its incentive mechanism. We also report a dual impact of this mechanism on user behavior discovered in two user studies.     

Computer consultants' views of user participation in the system development process

The form of user participation adopted in any system development process is commonly argued to be an important determinant of its outcome. This study investigated computer consultants' experiences and views of different forms of user participation in the system development process. Thirty-four consultants from 14 Swedish companies of three different sizes filled out a questionnaire (response rate 47%). The results indicated that the consultants viewed both external factors and their own roles as guiding the degree of user participation. The consultants reported that reference groups, seminars/meetings and user testing were common forms of user participation in projects introducing existing programs into organizations and that seminars/meetings, users in project groups and reference groups, were the most common user participation forms in projects developing new programs for an organization. In both contexts, user testing was reported to be most common in the operation/administration phase of the project. The majority of consultants considered union participation to be fairly unimportant. Further, it was found that questionnaires were seldom used for data collection in projects. Finally, the consultants did not appear to categorize development projects by the type of user participation employed in the projects.     

User participation in the requirements engineering process: An empirical study

In the development of information systems, user participation in the requirements engineering (RE) process is hypothesised to be necessary for RE success. In this paper we develop a theoretical model which predicts that the interaction between user participation in the RE process and uncertainty has an impact on RE success. This theory is empirically tested using survey data. We develop instruments to measure user participation and uncertainty. An existing instrument for measuring RE success was used. This instrument covers two dimensions of RE success: (a) the quality of RE service, and (b) the quality of RE products. The results, indicate that as uncertainty increases, greater user participation alleviates the negative influence of uncertainty on the quality of RE service, and that as uncertainty decreases, the beneficial effects on the quality of RE service of increasing user participation diminish. Furthermore, we did not find that the interaction between user participation and uncertainty had an impact on the quality of RE products. Based on these results, we make recommendations for managing user participation in the RE process, and provide directions for future research.This work has been supported, in part, by the IT Macroscope Project and NSERC Canada.     

The impact of IT outsourcing on information systems success

The objective of this research is to assess the impact of IT outsourcing on Information Systems’ success. We modeled the relationships among the extent of IT outsourcing, the ZOT (the Zone of Tolerance), and IS success. We justified our model using the expectancy–disconfirmation theory, the agency theory, and transaction cost economics, and we empirically tested it using structural equation modeling with responses from IS users. We found significant direct and indirect effects (through the service quality) of outsourcing on IS systems’ perceived usefulness and their users’ satisfaction. Whereas the extent of outsourcing is negatively related to the service quality and perceived usefulness, the ZOT-based IS service quality is positively related to the user satisfaction.     

The relationship between perceived e-service quality and brand equity: A simultaneous equations system approach

Using the data collected from the online banking users in Taiwan, we build a hierarchical model of e-banking service quality and investigate the relationships among e-service quality, trust, satisfaction, loyalty, and brand equity. Moreover, the simultaneous equations system approach is also applied to transfer the traditional satisfaction-loyalty path into the simultaneous relationships between trust and loyalty, and between satisfaction and loyalty. As the structural form coefficients demonstrate the direct relations between our research constructs, the reduced form estimates further disclose the total impacts of the quality of e-banking service on trust, satisfaction, loyalty, and brand equity, respectively. The results indicate that the perceived quality formed through interaction with an online banking service positively affects customer trust and satisfaction, which in turn influence loyalty and brand equity. The significant simultaneous relationships between trust and loyalty, and between satisfaction and loyalty, are confirmed by our data, implying that these relationships must be determined simultaneously, rather than sequentially.     

Supporting user participation design using a fuzzy analytic hierarchy process approach

There are three fundamental problems that may occur in the process of user participation design: first, the participants/users may not be able to express their requirements clearly; second, they have little knowledge about design; and third, they are generally unfamiliar with the software that designers use. Based on this understanding, a method that considers design rationale is proposed in this work to support the process of user participation design. In addressing the user participation process, a fuzzy analytic hierarchy process (AHP) approach is applied to grasp people's ideas, in the initial design phase. A case study on creating house layout design is employed to illustrate the proposed approach. In this regard, to help participants/users create layout designs, it is proposed that a 3D generative system is used, which integrates navigational concepts, direct manipulation, and the design rationale theory. In a nutshell, this research proposes a system to implement a design rational model and improve design communication in the user participation process. To demonstrate the effectiveness of the proposed prototype system, a user test is performed and we put forward some findings and research questions for further research and industry practices.     

The priority factor model for customer relationship management system success

As the market competition becomes keen, constructing a customer relationship management system is coming to the front for winning over new customers, developing service and products for customer satisfaction and retaining existing customers. However, decisions for CRM implementation have been hampered by inconsistency between information technology and marketing strategies, and the lack of conceptual bases necessary to develop the success measures. Using a structural equation analysis, this study explores the CRM system success model that consists of CRM initiatives: process fit, customer information quality, and system support; intrinsic success: efficiency and customer satisfaction; and extrinsic success: profitability. These constructs underlie much of the existing literature on information system success and customer satisfaction perspectives. We found the empirical support for CRM implementation decision-making from 253 respondents of 14 companies which have implemented the CRM system. These findings should be of great interest to both researchers and practitioners.     

IT infrastructure services as a requirement for e-learning system success

Evaluation of e-learning systems success is an essential process in managing and developing these types of systems. There are still arguments about the factors can be used to evaluate the success of e-learning systems. Inadequate attention has been paid to investigate the role of IT infrastructure services as a foundation to create the success of e-learning systems. The focus of this study is the role of IT infrastructure services in terms of the success of e-learning systems. A model is proposed which includes five constructs: IT infrastructure services; perceived usefulness; user satisfaction; customer value; and organizational value. A quantitative study was conducted at an Australian University, with survey responses from 110 Academic staff members and 720 students who use the e-learning system. The empirical study confirmed that an IT infrastructure services is a valid and reliable construct to measure e-learning systems success. In addition, this study provides evidence of the critical role of IT infrastructure services in the success of e-learning systems via its significant effect on perceived usefulness, user satisfaction, customer value, and organizational value.     

Modelling electronic customer relationship management success: functional and temporal considerations

Previous information systems satisfaction research predominantly focused on generic technological attributes, failing to account for the specificity of the artefact. Furthermore, viewing satisfaction as a static evaluation state, the prevalent cross-sectional approach could not account for the dynamic nature of satisfaction. In this study, we address these gaps by following a functional approach and taking a temporal view in developing and testing a model explaining the effects of various types of electronic customer relationship management (eCRM) functions on customer satisfaction in the context of online shopping. A framework based on the transaction cycle is used to classify eCRM functions into pre-, at-, and post-purchase eCRM. Two distinct temporal phases, i.e. attraction and retention, are identified. The results of a longitudinal survey involving 670 customers of hardware retailers demonstrate the appropriateness of the functional approach in investigating eCRM success and the necessity of the temporal conceptualisation of customer satisfaction. The theoretical and practical implications of these results are discussed.     

Systems analysis in a complex environment: An interactive educational approach

Many approaches have been proposed for systems analysis but few researchers propose a continuous educational exchange between users and systems analysts. This type of exchange is critical to successful system implementation when the environment is complex and subject to change. Both users and systems analysts must be informed of the others' roles and the purposes and objectives of the system. This paper presents a conceptual model of providing education to both users and analysts in each phase of system development and an application and discussion of the model.     

The relationships between management styles,user participation,and system success over MIS growth stages

MIS growth can be viewed as an on-going process of organizational change. An MIS manager has to deal with people and lead them in different ways. Situations change as the MIS growth stages alter and thus each stage may require different management styles and participation-success relationships. This paper investigates the relationships between management styles, user participation, and system success over MIS growth stages. Surveying 172 information managers in different organizations, we found that the relationships between management styles, user participation, and system success are different over different MIS growth stages. Information managers are therefore assumed to be adaptable according to the requirements of MIS growth stages.     

The influence of user interaction and participation in social media on the consumption intention of niche products

Social media may be particularly adept at promoting niche products because of the tendency of consumers to participate in generating reviews and discussing such products, thereby raising interest in them. In this study, we investigate how patterns of user interaction in discussing a niche cultural product may influence participation levels, which in turn enhance consumption intentions. We show that higher levels of participation can indeed enhance consumption intention. Furthermore, interaction patterns with high inclusiveness and betweenness centralization may enhance participation levels, whereas out-degree centralization and core-periphery have a detrimental influence. Implications for research and practice are discussed.     

Investigating the relationship between success factors and student participation in online and blended learning in adult education

Online and blended learning (OBL) is valued, but it also offers challenges. Literature indicates that OBL can enhance access to education and increase flexibility for students. However, the reported dropout rates indicate that student participation in OBL programmes is a concern. Scientifically valid knowledge about how factors that help students participate in OBL are related to student participation is necessary for quality improvement of OBL. This knowledge can help professionals determine what they need to improve in their institution and how to prioritize those improvements. In this study, we report on the validation of a quality instrument with indicators related to quality dimensions present in quality management frameworks and important success factors that aid student participation in OBL. These success factors are as follows: credibility, transparency, flexibility, accessibility, interactivity, personalization, and productivity. The partial least squares structural equation modelling (PLS‐SEM) method is suitable for validating complex models in studies where predictive accuracy is important. According to adult students, all success factors, except flexibility, are important for aiding OBL participation. Adult students perceive that the quality dimensions of learning activities and student support (related to interactivity) deserve priority in improving participation in OBL in adult education.     

The mediating effect of rewarding on the relationship between employee involvement and job satisfaction

Employee support provides enormous benefits to help sustain a competitive advantage, respond to changes more quickly than competitors, and position the organization ahead of others. Awareness of this fact triggers organizations to prepare new motivational programs and practices. Employee involvement and rewards are among the many ways to achieve employee job satisfaction and motivational needs. Employee involvement entails giving employees an opportunity to influence decisions and actions regarding their jobs. Furthermore, rewards have the potential to prompt employees to act in line with organizational goals. This study aims to examine the relationship between employee involvement and job satisfaction based on the mediation effect of rewarding. Four hundred employees from the financial sector responded to a questionnaire. The relationship between employee involvement and job satisfaction was tested using hierarchical linear regression analysis. Results revealed that rewarding does mediate the relationship between employee involvement and job satisfaction. Organizations that give priority to employee needs and motivational processes should take both employee involvement and rewarding into consideration.     

Intranet satisfaction questionnaire: Development and validationof a questionnaire to measure user satisfaction with the Intranet

In recent years, Intranets have become increasingly important to their companies. Substantial investments have been made to provide crucial information and workflows to employees. In this context the question of quality assurance arises: how can user satisfaction with the Intranet be measured? This article presents the development of a questionnaire to measure user satisfaction with the Intranet. After a first validation of the instrument (18 items) in an international insurance company (_N1=881)$N_1=881)$, a final set of 13 items remained. These were tested with the Intranet of a national retail company (_N2=1350)$N_2=1350)$. The final version showed a high internal consistency (Cronbach α)$\alpha )$ of .89, good item difficulties (.36–.73) and discriminatory power coefficients (.48–.73), as well as a moderate average homogeneity of .44. An exploratory factor analysis revealed two factors, “Content Quality” and “Intranet Usability”, explaining 56.54% of the variance. Meanwhile, the survey was translated into 10 languages: Chinese, English, French, German, Italian, Japanese, Portuguese, Russian, Slovenian, and Spanish.     

Assessing e‐commerce systems success: a respecification and validation of the DeLone and McLean model of IS success

Abstract. DeLone & McLean (2003) propose an updated information systems (IS) success model and suggest that it can be extended to investigating e‐commerce systems success. However, the updated IS success model has not been empirically validated in the context of e‐commerce. Further, the existing IS/e‐commerce success models have been subject to considerable debate on the ‘IS Use’ and ‘Perceived Usefulness’ constructs, and the nomological structure of the updated DeLone and McLean model is somewhat inconsistent with the IS acceptance and marketing literature. Based on the IS and marketing literature, this paper respecifies and validates a multidimensional model for assessing e‐commerce systems success. The validated model consists of six dimensions: Information Quality, System Quality, Service Quality, Perceived Value, User Satisfaction and Intention to Reuse. Structural equation modelling techniques were applied to data collected by questionnaire from 240 users of e‐commerce systems in Taiwan. The empirical evidence suggests that Intention to Reuse is affected by Perceived Value and User Satisfaction, which, in turn, are influenced by Information Quality, System Quality and Service Quality. The nomological structure of the respecified e‐commerce systems success model is concurred with that of the technology acceptance model (TAM) in the IS field and the consumer behaviour models in the traditional business‐to‐business and retail contexts. The findings of this study provide several important implications for research and practice. This paper concludes by discussing the contributions of this study and the limitations that could be addressed in future studies.     

The ergonomics lab: a practical approach

Abstract

Abstract. This article is concerned with methods and experiences in usability testing of standard application business software. In order to achieve the multiple trade-off between scientific objectivity, practical applicability, and the cost-benefit ratio, a set of standard methods and the resulting testing environment in the ergonomics lab are described and demonstrated by examples. Stumbling blocks are discussed. Necessary additional prerequisites for a successful practical approach are stressed.     

Exploring MARS: developing indoor and outdoor user interfaces to a mobile augmented reality system

We describe an experimental mobile augmented reality system (MARS) testbed that employs different user interfaces to allow outdoor and indoor users to access and manage information that is spatially registered with the real world. Outdoor users can experience spatialized multimedia presentations that are presented on a head-tracked, see-through, head-worn display used in conjunction with a hand-held pen-based computer. Indoor users can get an overview of the outdoor scene and communicate with outdoor users through a desktop user interface or a head- and hand-tracked immersive augmented reality user interface.     

The design of civic technology: factors that influence public participation and impact

Civic technology needs to be better understood in terms of the factors that promote representative public participation and impact. This paper reports on a mixed-methods study of a civic tech platform that enabled the public to provide feedback on public transport to the service providers. The overall aim of this research was to investigate the public’s use of a leading civic tech platform, FixMyTransport. The key findings were that: an effective and easy-to-use civic technology platform enables broad participation; data and process complexity need to be removed; factual information can be captured in situ with impacts, consequences and opinions added later; emotions (if important) need to be explicitly elicited; feedback to, and a ‘conversation’ with, the users is important for engagement, as is a feeling of being part of a community. These findings can contribute to the future design of civic technology platforms.

Practitioner Summary: There is a lack of understanding of how ‘civic tech’ platforms are used and how they may be designed for maximum effectiveness. Multiple data collection methods were used to investigate a well-developed example of civic tech. Effective civic tech can enable broad democratic participation to improve public services.