MobileNurse: hand-held information system for point of nursing care

Healthcare information travels with patients and clinicians and therefore the need for information to be ubiquitously available is key to reliable patient care and reliable medical systems. We have implemented MobileNurse, a prototype point-of-care system using PDA. MobileNurse has four modules each of which performs: (1) patient information management; (2) medical order check; (3) nursing recording; and (4) nursing care plan. MobileNurse provides easy input interface and various outputs for nursing records. The system consists of PDAs and a mobile support system (MSS) which supports clinical data exchange between PDAs and hospital information system. Two synchronization modules have been developed to keep the patient data consistent between PDAs and MSS. Clinical trials were performed with six volunteered nurses. They tried MobileNurse for 1-day caring-simulated patients. According to the survey after the trials, most of volunteers agreed that MobileNurse is more helpful and convenient than other non-mobile care systems to check medical orders and retrieve the results of recent clinical tests at the bedside. Through the involvement, we found out that ease-to-use interface is the most critical successful factor for mobile patient care systems.     

Mobile architecture for the Wishard Memorial Hospital Ambulatory Service

The introduction of portable computers into existing database systems requires traditional transaction mechanisms to be adapted to meet the needs of the mobile user. These mobile users utilise hand-held and laptop computers called mobile hosts, that can access the database from any location at any time. In this paper, we describe a mobile architecture that places mobile on the ambulances of the Wishard Memorial Hospital Ambulatory Service. This mobile architecture is designed to support transactions created by a mobile host that may be disconnected from the network during the execution of these transactions. The environment for this mobile architecture is the Indianapolis Network for Patient Care and Research (INPCR) of Marion County, Indianapolis, Indiana. The INPCR, which was developed to provide patient care by presenting clinical data in a clear and efficient manner, utilises the distributed medical patient database of the Regenstrief Medical Record System (RMRS) as a data repository. The medical personnel of the ambulatory service are able to submit transactions to the patient database of the RMRS to access vital patient information. The transactions in this environment use the forward recovery of sagas to minimise the amount of recovery needed during transaction failure. By allowing the medical personnel (hospital emergency room staff and the emergency medical technicians of the Wishard Memorial Hospital Ambulance Service) the ability to store, retrieve, create, and evaluate the medical records of their patients, they can provide high-quality care in a time and cost effective manner.     

Ontology-based mobile augmented reality in cultural heritage sites: information modeling and user study

Augmented reality (AR) has received much attention in the cultural heritage domain as an interactive medium for requesting and accessing information regarding heritage sites. In this study, we developed a mobile AR system based on Semantic Web technology to provide contextual information about cultural heritage sites. Most location-based AR systems are designed to present simple information about a point of interest (POI), but the proposed system offers information related to various aspects of cultural heritage, both tangible and intangible, linked to the POI. This is achieved via an information modeling framework where a cultural heritage ontology is used to aggregate heterogeneous data and semantically connect them with each other. We extracted cultural heritage data from five web databases and modeled contextual information for a target heritage site (Injeongjeon Hall and its vicinity in Changdeokgung Palace in South Korea) using the selected ontology. We then implemented a mobile AR application and conducted a user study to assess the learning and engagement impacts of the proposed system. We found that the application provides an agreeable user experience in terms of its affective, cognitive, and operative features. The results of our analysis showed that specific usage patterns were significant with regard to learning outcomes. Finally, we explored how the study’s key findings can provide practical design guidance for system designers to enhance mobile AR information systems for heritage sites, and to show system designers how to support particular usage patterns in order to accommodate specific user experiences better.

     

Usability Principles for Augmented Reality Applications in a Smartphone Environment

Through the rapid spread of smartphones, users have access to many types of applications similar to those on desktop computer systems. Smartphone applications using augmented reality (AR) technology make use of users' location information. As AR applications will require new evaluation methods, improved usability and user convenience should be developed. The purpose of the current study is to develop usability principles for the development and evaluation of smartphone applications using AR technology. We develop usability principles for smartphone AR applications by analyzing existing research about heuristic evaluation methods, design principles for AR systems, guidelines for handheld mobile device interfaces, and usability principles for the tangible user interface. We conducted a heuristic evaluation for three popularly used smartphone AR applications to identify usability problems. We suggested new design guidelines to solve the identified problems. Then, we developed an improved AR application prototype of an Android-based smartphone, which later was conducted a usability testing to validate the effects of usability principles.     

A usability study of multimodal input in an augmented reality environment

In this paper, we describe a user study evaluating the usability of an augmented reality (AR) multimodal interface (MMI). We have developed an AR MMI that combines free-hand gesture and speech input in a natural way using a multimodal fusion architecture. We describe the system architecture and present a study exploring the usability of the AR MMI compared with speech-only and 3D-hand-gesture-only interaction conditions. The interface was used in an AR application for selecting 3D virtual objects and changing their shape and color. For each interface condition, we measured task completion time, the number of user and system errors, and user satisfactions. We found that the MMI was more usable than the gesture-only interface conditions, and users felt that the MMI was more satisfying to use than the speech-only interface conditions; however, it was neither more effective nor more efficient than the speech-only interface. We discuss the implications of this research for designing AR MMI and outline directions for future work. The findings could also be used to help develop MMIs for a wider range of AR applications, for example, in AR navigation tasks, mobile AR interfaces, or AR game applications.     

Pattern-Based Interactive Diagnosis of Multiple Disorders: The MEDAS System

A knowledge-based interactive sequential diagnostic system is introduced which provides for diagnosis of multiple disorders in several body systems. The knowledge base consists of disorder patterns in a hierarchical structure that constitute the background medical information required for diagnosis in the domain under consideration (emergency and critical care medicine, in our case). Utilizing this knowledge base, the diagnostic process is driven by a multimembership classification algorithm for diagnostic assessment as well as for information acquisition [1]. A key characteristic of the system is congenial man-machine interface which comes to expression in, for instance, the flexibility it offers to the user in controlling its operation. At any stage of the diagnostic process the user may decide on an operation strategy that varies from full user control, through mixed initiative to full system control. Likewise, the system is capable of explaining to the user the reasoning process for its decisions. The model is independent of the knowledge base, thereby permitting continuous update of the knowledge base, as well as expansions to include disorders from other disciplines. The information structure lends itself to compact storage and provides for efflcient computation. Presently, the system contains 53 high-level disorders which are diagnosed by means of 587 medical findings.     

Natural language interface for information management on mobile devices

A natural language interface (NLI) enables the ease-of-use of information systems in performing sophisticated human - computer interaction. To address the challenges of mobile devices to user interaction in information management, we propose an NLI as a promising solution. In this paper, we review state-of-the-art NLI technologies and analyse user requirements for managing notable information on mobile devices. To minimize any technical difficulties arising from developing and improving the usability of NLI systems we develop general principles for NLI design, which fills in a gap in the literature. In order to satisfy user requirements for information management on mobile devices, we innovatively design NLI-enabled information management architecture. It is shown from two usage scenarios that the architecture could lead to reduced effort in user navigation and improved efficiency and effectiveness of managing information on mobile devices. We conclude the article with the implications of this study and suggestions for future direction.     

Web-enabled smart card for ubiquitous access of patient's medical record

The combined benefits of smart card to support mobility in a pocket coupled with the ubiquitous access of Web technology, present a new paradigm for medical information access systems. The paper describes the framework of Java Card Web Servlet (JCWS) that is being developed to provide seamless access interface between a Web browser and a Java-enabled smart card. Importantly, the smart card is viewed as a mobile repository of Web objects comprised of HTML pages, medical data objects, and record browsing and updating applet. As the patient moves between hospitals, clinics and countries, the mobility of the smart-card database dynamically binds to the JCWS framework to facilitate a truly ubiquitous access and updating of medical information via a standard Web-browser interface.     

User preferences for adaptive user interfaces in health information systems

An adaptive user interface requires identification of user requirements. Interface designers and engineers must understand end-user interaction with the system to improve user interface design. A combination of interviews and observations is applied for user requirement analysis in health information systems (HIS). Then, user preferences are categorized in this paper as either data entry, language and vocabulary, information presentation, or help, warning and feedback. The user preferences in these categories were evaluated using the focus group method. Focus group sessions with different types of HIS users comprising medical staff (with and without computer skills) and system administrators identified each user group’s preference for the initial adaptation of the HIS user interface. User needs and requirements must be identified to adapt the interface to users during data entry into the system. System designers must understand user interactions with the system to identify their needs and preferences. Without this, interface design cannot be adapted to users and users will not be comfortable using the system and eventually abandon its use.     

Augmented reality agents for user interface adaptation

Most augmented reality (AR) applications are primarily concerned with letting a user browse a 3D virtual world registered with the real world. More advanced AR interfaces let the user interact with the mixed environment, but the virtual part is typically rather finite and deterministic. In contrast, autonomous behavior is often desirable in ubiquitous computing (Ubicomp), which requires the computers embedded into the environment to adapt to context and situation without explicit user intervention. We present an AR framework that is enhanced by typical Ubicomp features by dynamically and proactively exploiting previously unknown applications and hardware devices, and adapting the appearance of the user interface to persistently stored and accumulated user preferences. Our framework explores proactive computing, multi‐user interface adaptation, and user interface migration. We employ mobile and autonomous agents embodied by real and virtual objects as an interface and interaction metaphor, where agent bodies are able to opportunistically migrate between multiple AR applications and computing platforms to best match the needs of the current application context. We present two pilot applications to illustrate design concepts. Copyright © 2007 John Wiley & Sons, Ltd.     

Browsing large online data tables using generalized query previews

Companies, government agencies, and other organizations are making their data available to the world over the Internet. They often use large online relational tables for this purpose. Users query such tables with front-ends that typically use menus or form fillin interfaces, but these interfaces rarely give users information about the contents and distribution of the data. Such a situation leads users to waste time and network/server resources posing queries that have zero- or mega-hit results. Generalized query previews enable efficient browsing of large online data tables by supplying data distribution information to users. The data distribution information provides continuous feedback about the size of the result set as the query is being formed. Our paper presents a new user interface architecture and discusses three controlled experiments (with 12, 16, and 48 participants). Our prototype systems provide flexible user interfaces for research and testing of the ideas. The user studies show that for exploratory querying tasks, generalized query previews can speed user performance for certain user domains and can reduce network/server load.     

手机三维菜单绩效预测模型

菜单是手机界面的重要交互部件,手机功能增加所带来的菜单规模增大与手机小屏幕形成日益突出的矛盾.三维界面技术可以提高界面信息容量,因此,在手机上引入三维菜单概念是一个值得研究的课题.相应地,需要解决手机三维菜单用户操作绩效的评估问题.模型预测评估法能使研究人员和设计人员快速和以较低成本对用户界面进行评估.因此,基于Fitts定律和Hick-Hyman定律,建立了预测评估手机三维菜单操作绩效的定量模型.通过实验检验了模型的有效性,并将其与相关模型进行了对比分析.实验结果表明,实验得到的数据与该模型预测数据吻合较好,并且与相关模型相比该模型预测数据与实验数据的相关度最高.     

A new AR authoring tool using depth maps for industrial procedures

Several augmented reality systems have been proposed for different target fields such as medical, cultural heritage and military. However, most of the current AR authoring tools are actually programming interfaces that are exclusively suitable for programmers. In this paper, we propose an AR authoring tool which provides advanced visual effect, such as occlusion or media contents. This tool allows non-programming users to develop low-cost AR applications, specially oriented to on-site assembly and maintenance/repair tasks. A new 3D edition interface is proposed, using photos and Kinect depth information to improve 3D scenes composition. In order to validate our AR authoring tool, two evaluations have been performed, to test the authoring process and the task execution using AR. The evaluation results show that overlaying 3D instructions on the actual work pieces reduces the error rate for an assembly task by more than a 75%, particularly diminishing cumulative errors common in sequential procedures. Also, the results show how the new edition interface proposed, improves the 3D authoring process making possible create more accurate AR scenarios and 70% faster.     

Information at a glance [augmented reality user interfaces]

What if we could visualize and interact with information directly in the context of our surroundings? Our research group is exploring how augmented reality (AR) could someday make this possible. AR integrates a complementary virtual world with the physical world-for example, by using head-tracked see-through head-worn displays to overlay graphics on what we see. Instead of looking back and forth between the real world and a PDA, we look directly at the real world and the virtual information overlaid on it. At the heart of this approach is context-aware computing, computing systems that are sensitive to the context in which they operate, ranging from human relationships to physical location. For example, information might be tied to specific locations within a global, Earth-centered, coordinate system. How can we design effective mobile AR user interfaces? We've been trying to answer this question in part by developing experimental AR research prototypes. In AR, as in work on information visualization using desktop technologies, the amount of information available can far exceed what a system can legibly display at a given time, necessitating information filtering. Julier et al. (2000) have developed information filtering techniques for AR that depend on the user's goals, object importance, and proximity. We assume that a system can accomplish information filtering of this sort and that our system is displaying everything it should.     

Toward AI-enabled augmented reality to enhance the safety of highway work zones: Feasibility,requirements, and challenges

Highway work zones are considered among the most hazardous working environments. In 2018 alone, 124 workers lost their lives to fatal accidents. The lack of predictive safety systems that notify workers of upcoming dangers in advance is a major reason to blame in the highway maintenance and operation community. This article presents an integrative design framework for bringing recent advances in Augmented Reality (AR) and Artificial Intelligence (AI) to enhance the safety of highway workers through real-time multimodal notifications on-spot. To this end, this article conceptualizes and co-designs three major pillars: (1) AR user interface design for multimodal notification, (2) real-time AI at the edge for vehicle detection/classification from distance, and (3) real-time wireless communication in work zone setting to enable latency-aware operation between AI and AR components. Our early results demonstrate that we can achieve 24.83 FPS end-to-end execution latency on the Xavier AGX Jetson board with 48.7% mAP on BDD100K dataset, and a real-time communication covering 120 meters with an average latency of 5.1 ms at the farthest distance. Our mixed-method user research also reveals an acceptable level of excitement and engagement from the body of highway workers toward both the proposed technology and the designed user interface. Overall, this article provides a proof-of-concept toward AI-enabled AR safety systems in highway work zones.     

Information tools for the shop-floor

New production and work practices set new proficiency requirements for employees in operative organisations. Most of these requirements involve information and communication. We can train and educate people to meet with these challenges and design their physical and information environments to support and facilitate them in their work. New information tools play a central role in this development. But what kind of tools? How to apply these tools to enhance employees’ learning by reflection. In this article we will discuss the emergency of shop-floor information environment from the sociotechnical point of view. In addition, the possibilities and limitations of electronic performance support as a solution are evaluated. Shop-floor information artifacts are tools used to produce, organise, store and distribute work related information in a shop-floor environment. They may be very simple like handwritten notes or user interfaces to extremely complex data management systems.     

General-purpose modular hardware and software framework for mobile outdoor augmented reality applications in engineering

This paper presents a reusable, general-purpose, mobile augmented reality (AR) framework developed to address the critical and repetitive challenges specific to visualization in outdoor AR. In all engineering applications of AR developed thus far, basic functionality that supports accurate user registration, maximizes the range of user motion, and enables data input and output has had to be repeatedly re-implemented. This is primarily due to the fact that designed methods have been traditionally custom created for their respective applications and are not generic enough to be readily shared and reused by others. The objective of this research was to remedy this situation by designing and implementing a reusable and pluggable hardware and software framework that can be used in any AR application without the need to re-implement low-level communication interfaces with selected hardware. The underlying methods of hardware communication as well as the object-oriented design (OOD) of the reusable interface are presented. Details on the validation of framework reusability and pluggability are also described.     

User interface design for military AR applications

Designing a user interface for military situation awareness presents challenges for managing information in a useful and usable manner. We present an integrated set of functions for the presentation of and interaction with information for a mobile augmented reality application for military applications. Our research has concentrated on four areas. We filter information based on relevance to the user (in turn based on location), evaluate methods for presenting information that represents entities occluded from the user??s view, enable interaction through a top-down map view metaphor akin to current techniques used in the military, and facilitate collaboration with other mobile users and/or a command center. In addition, we refined the user interface architecture to conform to requirements from subject matter experts. We discuss the lessons learned in our work and directions for future research.     

基于移动终端日志数据的人群特征可视化

随着我国移动互联网的迅猛发展,如何从海量移动终端日志数据中提取出有效信息,并进行合理、清晰的可视化分析,为工业界等提供有价值的统计分析功能显得尤为重要.目前,对于移动终端日志数据的研究和分析多是基于对单一属性的统计结果分析,如应用下载排行、用户留存率等.为了进一步挖掘移动终端日志数据背后深层次的隐含信息,更加准确地概括出移动终端用户的特征,提出了一种基于移动应用程序日志数据的人群特征分析与画像计算方法,构造了基于移动应用程序数据的主题模型,并将移动设备用户按照与不同应用主题的相关度进行聚类,得到了具有不同特征的人群,从而提出了基于层次气泡图和Voronoi Treemap的可视化展现与分析方案.进一步将人群特征与时间信息、地理位置信息相结合,从多角度可视化展现人群特征.最后,根据该研究内容,实现了B/S架构的日志数据可视化分析原型系统,并通过案例分析验证了该方法的有效性.