Evaluating design proposals for complex systems with work domain analysis

In this paper we propose a new framework for evaluating designs based on work domain analysis, the first phase of cognitive work analysis. We develop a rationale for a new approach to evaluation by describing the unique characteristics of complex systems and by showing that systems engineering techniques only partially accommodate these characteristics. We then present work domain analysis as a complementary framework for evaluation. We explain this technique by example by showing how the Australian Defence Force used work domain analysis to evaluate design proposals for a new system called Airborne Early Warning and Control. This case study also demonstrates that work domain analysis is a useful and feasible approach that complements standard techniques for evaluation and that promotes a central role for human factors professionals early in the system design and development process. Actual or potential applications of this research include the evaluation of designs for complex systems.     

A socio-technical approach to improving the systems development process

Research on improving the systems development processes has primarily focused on mechanisms such as tools, software development methodologies, knowledge sharing and process capabilities. This research has yielded considerable insights into improving the systems development process, but the large majority of information systems development projects still continue to be over budget, late, and ineffective in meeting user needs. Together with the advent of software development moving offshore, or consisting of offshore team members, a more holistic approach is appropriate. Approached from a socio-technical perspective the software development process is viewed as a process embedded in a social and a technical subsystem. Drawing upon socio-technical work design principles, this paper suggests how capabilities of the development process can be improved. Data collected from a survey of software development practices in organizations indicates that organizations at different levels of process capabilities differ in work system characteristics as well as process performance. For example, the use of multi-skilled teams was found to be significantly related to the systems development process maturity level as well as significantly related to all the performance measures studied. This paper provides empirical support for the socio-technical approach and provides a theoretical foundation for designing software process initiatives in organizations.     

Modeling development process of skill-based system for human-like manipulation robot

The importance of system integration is widely recognized in robotics. This motivates the application of Model-Based Systems Engineering (MBSE) approaches to improve the development process of robot systems. This paper models a development process to achieve given task goals using a human-like upper body robot based on MBSE approach. For this purpose, we focus on the domain knowledge of tasks and skills in robotics. Since MBSE is a general methodology, there is a lot of flexibility on the way of proceeding with the analysis and design, and how to utilize models there. Using the concept of tasks and skills is helpful for better uderstanding of the development process. Our process is based on three main concepts: (1) stakeholders of User and Developer, (2) coordination between User and Developer using skills as communication interface, (3) extension development. Making the process explicit helps many stakeholders such as robot makers, system integrators, and engineers in various application domains to join the system development. It is also effective for accumulating experiences and work products of the development. In addition, we can expect that better understanding of the engineering process results in the improvement of the process performed by automation tools and humans cooperatively.     

A Representational Framework for Scenarios of System Use

Scenarios are becoming widely used in three areas of system development: software engineering, human–computer interaction (HCI), and organisational process design. There are many reasons to use scenarios during system design. The one usually advanced in support of the practice is to aid the processes of validating the developers’ understanding of the customers’ or users’ work practices, organisational goals and structures, and system requirements. All three areas identified above deal with these processes, and not surprisingly this has given rise to a profusion of scenario-based practices and representations. Yet there has been little analysis of why scenarios should be useful, let alone whether they are. Only by having such a framework for understanding what scenarios are, and what they are for, can we begin to evaluate different scenario approaches in specific development contexts. This paper is a contribution toward such a framework. We lay out a space of representational possibilities for scenarios and enumerate a set of values or criteria that are important for different uses of scenarios. We then summarise several salient representations drawn from the software engineering, HCI, and organisational process design communities to clarify how these representational choices contribute to or detract from the goals of the respective practices. Finally, we discuss how scenario representations from one area of design may be useful in others, and we discuss the relationship between these representations and other significant early-design and requirements engineering practices.     

Design of joint systems: a theoretical challenge for cognitive systems engineering

In this paper the “joint systems approach” will be discussed. The approach is considered as a new paradigm in cognitive systems engineering (CSE). Its central idea, that human and technology form a functional unity, is as such not new. Why, then, has this idea become so appealing right now? In the first part of the paper we seek answers to this question by analysing the current situation in product design. The conclusion is that the focus of design is shifting from single products to intelligent environments. This change in focus induces tensions into the design process that urge for solutions. One of them is re-conceptualising the relationship between human and technology, precisely what the joint system approach is about. Three different joint system approaches are considered: the joint cognitive systems approach (JCS), the Risö extended CSE approach, and the joint intelligent systems (JIS) approach proposed by us. Comparisons are made with regard to how these approaches understand the joint system to be organised. While the JCS approach focuses on the human–technology relationship, the other two consider this relationship as embedded in a context of object-oriented activity. As a consequence, environment becomes included in the joint system. In JIS approach we propose the use of the semiotic concept of habit to characterise the ways of functioning of the joint system. “Habit” is a tool for identifying generic patterns in the situation-specific behaviour of the system. Defining habits enables expressing the meaning or purpose of the system’s functioning. In the end of the paper, we propose a design process model for the development of JIS. This approach is aimed at designing systems in usage i.e. to design of practices.     

Scenario-based run-time adaptive MPSoC systems

The ever-increasing performance demand of modern embedded applications drives the development of multi-processor system-on-chip (MPSoC) systems in the embedded domain. Today’s MPSoC-based products increasingly have to deal with multiple application execution scenarios which may change dynamically at run time. To improve the system performance, a state-of-the-art solution is to dynamically adapting the allocation of system resources at run time for each execution scenario based on pre-determined resource schemes that have been optimized at design time. However, such approaches will not work well for MPSoC systems that have a large number of execution scenarios and/or frequent run-time variations in execution scenario behavior. In this work, we therefore propose a scalable run-time self-adaptive framework for MPSoC systems that addresses these problems, thereby considerably improving the system efficiency.     

Model-based engineering for change-tolerant systems

Developing and evolving today’s systems are often stymied by the sheer size and complexity of the capabilities being developed and integrated. At one end of the spectrum, we have sophisticated agent-based software with hundreds of thousands of collaborating nodes. These require modeling abstractions relevant to their complex workflow tasks as well as predictable transforms and mappings for the requisite elaborations and refinements that must be accomplished in composing these systems. At the other end of the spectrum, we have ever-increasing capabilities of reconfigurable hardware devices such as field-programmable gate arrays to support the emerging adaptability and flexibility needs of these systems. From a model-based engineering perspective, these challenges are very similar; both must move their abstraction and reuse levels up to meet growing productivity and quality objectives. Model-based engineering and software system variants such as the model-driven architecture (MDA) are increasingly being applied to systems development as the engineering community recognizes the benefits of managing complexity, separating key concerns, and automating transformations from high-level abstract requirements down through the implementation. However, there are challenges when it comes to establishing the correct boundaries for change-tolerant parts of the system. Capabilities engineering (CE) is a promising approach for defining long-lived components of a system to ensure some sense of change tolerance. For innovative initiatives such as the National Aeronautics and Space Administration (NASA)’s autonomous nanotechology swarms (ANTS), the development and subsequent evolution of such systems are of considerable importance as their missions involve complex, collaborative behaviors across distributed, reconfigurable satellites. In this paper, we investigate the intersection of these two technologies as they support the development of complex, change-tolerant systems. We present an effective approach for bounding computationally independent models so that, as they transition to the architecture, capabilities-based groupings of components are relevant to the change-tolerant properties that must convey in the design solution space. The model-based engineering approach is validated via a fully functional prototype and verified by generating nontrivial multiagent systems and reusing components in subsequent systems. We build off of this research completed on the collaborative agent architecture, discuss the CE approach for the transition to architecture, and then examine how this will be applied in the reconfigurable computing community with the new National Science Foundation Center for High-Performance Reconfigurable Computing. Based on this work and extrapolating from similar efforts, the model-based approach shows promise to reduce the complexities of software evolution and increase productivity—particularly as the model libraries are populated with canonical components.     

Socially distributed cognition in loosely coupled systems

Distributed cognition provides a theoretical framework for the analysis of data from socio-technical systems within a problem-solving framework. While the approach has been applied in tightly constrained activity domains, composed of well-structured problems and highly organised infrastructures, little is known about its use in other forms of activity systems. In this paper, we explore how distributed cognition could be applied in less well-constrained settings, with ill-structured problems and loosely organised resource sets, critically reflecting on this using data from a field study. The findings suggest that the use of distributed cognition in an augmented form can be useful in the analysis of a wide range of activity systems in loosely coupled settings.     

A survey of motivation frameworks for intelligent systems

The ability to achieve one?s goals is a defining characteristic of intelligent behaviour. A great many existing theories, systems and research programmes address the problems associated with generating behaviour to achieve a goal; much fewer address the related problems of how and why goals should be generated in an intelligent artifact, and how a subset of all possible goals are selected as the focus of behaviour. It is research into these problems of motivation, which this article aims to stimulate. Building from the analysis of a scenario involving a futuristic household robot, we extend an existing account of motivation in intelligent systems to provide a framework for surveying relevant literature in AI and robotics. This framework guides us to look at the problems of encoding drives (how the needs of the system are represented), goal generation (how particular instances of goals are generated from the drives with reference to the current state), and goal selection (how the system determines which goal instances to act on). After surveying a variety of existing approaches in these terms, we build on the results of the survey to sketch a design for a new motive management framework which goes beyond the current state of the art.     

Patient safety, systems design and ergonomics

The complexity of the health care environments necessitates an holistic and systematic ergonomics approach to understand the potential for accidents and errors to occur. The health service is also a socio-technical system, and design needs must be met within this context. This paper aims to present the design challenges and emphasises the specialised needs of the health care sector, when dealing with patient safety. It also provides examples of approaches and methods that ergonomists can bring to help inform our knowledge of these systems and the potential towards improving their safety. Mapping workshops provide an example of such methods. Results from these are used to illustrate how the knowledge base required for better design requirements can be generated. The workshops were developed specifically to help improve the design of medication packaging and thereby reduce the probability of medication error. The issues raised are now the subject of further research, design requirements guidance and new design concepts. The paper illustrates the need to engage with the design community and, through the use of robust scientific methods, to generate appropriate design requirements.     

Cooperation,reliability of socio-technical systems and allocation of function

When (re)designing a work environment, tasks or functions are allocated more or less explicitly among humans and between humans and machines. After a brief review and discussion of issues related to task allocation, we argue that an important aspect to be addressed when (re)designing socio-technical systems is the systematic evaluation of the impact of allocation decisions on the overall reliability of such systems. It is contended that the cooperative dimension of such systems is one of the main elements that contribute to this reliability. This claim leads us to present a conceptual framework for modelling the human contribution to the overall reliability of complex cooperative work systems. The framework is characterized here as a set of notions, mainly regulation and shared context, used to discuss and reason about this role of humans in the error tolerance properties of such systems. These notions are demonstrated with different examples derived from empirical studies of work practices in two complex cooperative work settings (air traffic and nuclear reactor control). We then show how this conceptual framework can be used for the evaluation of allocation decisions and more generally to inform design.     

Event-based control and filtering of networked systems: A survey

In recent years, theoretical and practical research on event-based communication strategies has gained considerable research attention due primarily to their irreplaceable superiority in resource-constrained systems (especially networked systems). For networked systems, event-based transmission scheme is capable of improving the efficiency in resource utilization and prolonging the lifetime of the network components compared with the widely adopted periodic transmission scheme. As such, it would be interesting to 1) examining how the event-triggering mechanisms affect the control or filtering performance for networked systems, and 2) developing some suitable approaches for the controller and filter design problems. In this paper, a bibliographical review is presented on event-based control and filtering problems for various networked systems. First, the event-driven communication scheme is introduced in detail according to its engineering background, characteristic, and representative research frameworks. Then, different event-based control and filtering (or state estimation) problems are categorized and then discussed. Finally, we conclude the paper by outlining future research challenges for event-based networked systems.     

A modelling method for the development of groupware applications as socio-technical systems

Special methodological approaches are needed, particularly in the area of modelling, to develop socio-technical systems in the field of CSCW. Engineering tasks and the development of organizational structures have to be integrated including elements of participatory design. From a theoretical background, needs for new modelling concepts can be derived: representation of contingency, explicit incompleteness, multiplicity of perspectives and self-referential meta-relations. Based on these concepts we developed the semi-structured, socio-technical modelling method SeeMe and used it in five empirical cases to assess its practical relevance. We found that SeeMe can help people to become aware of the specific features and requirements of 'their' socio-technical system and therefore enables them to take part in processes of learning and improvement.     

Industrial Applications of Software Synthesis via Category Theory—Case Studies Using Specware

Over the last two years, we have demonstrated the feasibility of applying category-theoretic methods in specifying, synthesizing, and maintaining industrial strength software systems. We have been using a first-of-its-kind tool for this purpose, Kestrel's Specware^TM software development system. In this paper, we describe our experiences and give an industrial perspective on what is needed to make this technology have broader appeal to industry. Our overall impression is that the technology does work for industrial strength applications, but that it needs additional work to make it more usable. We believe this work marks a turning point in the use of mathematically rigorous approaches to industrial strength software development and maintenance.It is interesting to note that when this technology is applied to software systems whose outputs are designs for airplane parts, the design rationale that is captured is not only software engineering design rationale, but also design rationale from other engineering disciplines (e.g., mechanical, material, manufacturing, electrical, human factors, etc.). This suggests the technology provides an approach to general systems engineering that enables one to structure and reuse engineering knowledge broadly.     

An approach for the systematic development of domain‐specific languages

Building tailored software systems for a particular application domain is a complex task. For this reason, domain‐specific languages (DSLs) receive a constantly growing attention in recent years. So far the main focus of DSL research is on case studies and experience reports for the development of individual DSLs, design approaches and implementation techniques for DSLs, and the integration of DSLs with other software development approaches on a technical level. In this paper, we identify and describe the different activities that we conduct when engineering a DSL, and describe how these activities can be combined in order to define a tailored DSL engineering process. Our research results are based on the experiences we gained from multiple different DSL development projects and prototyping experiments. Copyright © 2009 John Wiley & Sons, Ltd.     

Dynamic data sharing and security in a collaborative product definition management system

Product definition management (PDM) is a system that supports management of both engineering data and the product development process during the total product life cycle. The formation of a virtual enterprise is becoming a growing trend, and vendors of PDM systems have recently developed a new generation of PDM systems called collaborative product definition management (cPDM). This paper presents the concept of a virtual engineering community (VEC) to support concurrent product development within geographically distributed partners. A previous case study has shown that collaborative engineering design may be modelled from a parameter perspective [1]. Effective implementation of the parameter approach raises the following problems: how to support data sharing and secure that span the partner borders. This paper describes the system architecture, deployed security mechanisms, the prototype developed within cPDM, and the system demonstration using a real test. The implementation of this architecture extends a common commercial PDM system (Axalan™) and utilizes standard software to create a security framework for the involved resources. Collaboration infrastructure, shared team spaces and shared resources are essential to enable virtual teams to work together. Various organizational and technical challenges are implied. The outlined architecture features a federated data approach. These issues are discussed and potential perspectives in the area of collaboration engineering are identified.     

Dogmas in the assessment of usability evaluation methods

Usability evaluation methods (UEMs) are widely recognised as an essential part of systems development. Assessments of the performance of UEMs, however, have been criticised for low validity and limited reliability. The present study extends this critique by describing seven dogmas in recent work on UEMs. The dogmas include using inadequate procedures and measures for assessment, focusing on win–lose outcomes, holding simplistic models of how usability evaluators work, concentrating on evaluation rather than on design and working from the assumption that usability problems are real. We discuss research approaches that may help move beyond the dogmas. In particular, we emphasise detailed studies of evaluation processes, assessments of the impact of UEMs on design carried out in real-world systems development and analyses of how UEMs may be combined.     

Domain-oriented design environments

The field of knowledge-based software engineering has been undergoing a shift in emphasis from automatic programming to human augmentation and empowerment. In our research work, we support this shift with an approach that embedshuman-computer cooperative problem-solving tools intodomain-oriented, knowledge-based design environments. Domain orientation reduces the large conceptual distance between problem-domain semantics and software artifacts. Integrated environments support the coevolution of specification and construction while allowing designers to access relevant knowledge at each stage within the software development process.This paper argues thatdomain-oriented design environments (DODEs) are complementary to the approaches pursued withknowledge-based software assistant systems (KBSAs). The DODE extends the KBSA framework by emphasizing a human-centered and domain-oriented approach facilitating communication about evolving systems among all stakeholders. The paper discusses the major challenges for software systems, develops a conceptual framework to address these problems, illustrates DODE with two examples, and assesses the contributions of the KBSA and DODE approaches toward solving these problems.     

Modeling Big data‐based systems through ontological trading

One of the great challenges the information society faces is dealing with the huge amount of information generated and handled daily on the Internet. Today, progress in Big data proposals attempt to solve this problem, but there are certain limitations to information search and retrieval due basically to the large volumes handled, the heterogeneity of the information, and its dispersion among a multitude of sources. In this article, a formal framework is defined to facilitate the design and development of an environmental management information system, which works with a heterogeneous and large amount of data. Nevertheless, this framework can be applied to other information systems that work with Big data, because it does not depend on the type of data and can be utilized in other domains. The framework is based on an ontological web‐trading model (OntoTrader), which follows model‐driven engineering and ontology‐driven engineering guidelines to separate the system architecture from its implementation. The proposal is accompanied by a case study, SOLERES‐KRS, an environmental knowledge representation system designed and developed using software agents and multi‐agent systems. Copyright © 2017 John Wiley & Sons, Ltd.