Influence mechanism of icon semantics on visual search performance: Evidence from an eye-tracking study

Explicit icon semantics can reduce the difficulty of understanding complex visual information. Optimizing the icon semantics and text semantics of icons can effectively improve the cognitive performance of digital interfaces. This paper adopts visual search tasks to study the effects of different combinations of icon semantic familiarity and the presence or absence of text on icon search performance under horizontal and vertical layouts. The behavioral experiment results show that under two layouts: 1. The main effect of icon semantics is significant, and the search performance increases with the increase of semantic familiarity. 2. The main effect of text is significant, and the search performance is negatively correlated with the addition of text. The eye movement experiment found that the semantic familiarity of icons had a significant impact on average fixation time. Furthermore, the number of fixation points changed significantly after the text variable was added. Therefore, there was no significant difference in the number of fixation points in the horizontal layout, and icon semantics was the main influencing factor in visual search. In the vertical layout, there was no significant difference in average fixation time, and text was the main influencing factor of visual search. The results show that the semantic familiarity of icons and different combinations with or without text significantly affect visual search performance in horizontal and vertical layouts. This paper provides a theoretical reference for the combination of icons and text in interface design.     

The effects of icon internal characteristics on complex cognition

The interaction between humans and digital interfaces is a complex cognitive process which has a high demand on working memory resources. However, most previous research on icon perception seems to focus only on the influences of an icon's internal characteristics (e.g. visual complexity, concreteness and familiarity) on simple tasks such as visual search and recall tasks. The study reported here applied a math task to simulate the complicated process of human-computer interaction and investigated the effects of icon internal characteristics on the performance of complex cognition. First, we created and filtered 80 novel icons of 4 different types (simple & concrete, simple & abstract, complex & concrete, complex & abstract). Subsequently, participants were required to perform an icon-meaning recall task, during which icon familiarity was manipulated by different exposure frequencies. Finally, after viewing random icon-digit associations, participants' abilities to solve algebraic equations involving icons were tested. The results indicated that participants performed significantly better when the icons were simple, concrete or familiar and these beneficial effects increased as the task became more complex. Additionally, strong evidence was found to show that it was easier to encode and maintain familiar icons and associated information in working memory. The findings of this study provide useful information for the icon design and interface design of human-computer interaction.     

Icon identification in context: the changing role of icon characteristics with user experience

OBJECTIVE: This research examined the relative importance of icon characteristics in determining the speed and accuracy of icon identification. BACKGROUND: Studies to date have focused on the role of one or two icon characteristics when users first experience an icon set. This means that little is known about the relative importance of icon characteristics or how the role of icon characteristics might change as users gain experience with icons. METHODS: Thirty participants carried out an icon identification task over a long series of trials to simulate learning through experience. Icon characteristics investigated included semantic distance, concreteness, familiarity, and visual complexity. RESULTS: Icon characteristics were major determinants of performance, accounting for up to 69% of the variance observed in performance. However, the importance of icon characteristics changed with experience: Semantic distance is crucial initially while icon-function relationships are learned, but familiarity is important later because it has lasting effects on access to long-term memory representations. CONCLUSION: These findings suggest that icon concreteness may not be of primary importance when identifying icons and that semantic distance and familiarity may be more important. APPLICATION: Designers need to take into account icon characteristics other than concreteness when creating icons, particularly semantic distance and familiarity. The precise importance of the latter characteristics will vary depending on whether icons are rarely encountered or frequently used.     

Effects of perceptual complexity on older and younger adults’ target acquisition performance

The present study evaluated the effects of the three perceptual complexity factors: number of elements, colour variety, and graphical background clutter level, on older and younger adults’ target acquisition time within a computer display. Experiment 1 manipulated the total number of icons, Experiment 2 manipulated the variety of icon colours, and Experiment 3 manipulated the clutter level of the graphical background on the display. In each experiment, 12 older and 12 younger adults were asked to move a cursor to a target icon on the display as quickly and accurately as possible. Target size and distance to the cursor were also manipulated to yield different difficulties of targets. Target acquisition time and Fitts’ law slope were analysed. Results showed that target acquisition time increased for more difficult targets under all the three complexity factors. The amount of increase was more evident for the factors of colour variety and graphical background clutter than for the number of icons. Older participants performed more slowly than younger participants did, particularly for more difficult targets. However, the impact of the three complexity factors on acquisition time appeared to be comparable for both age groups. The results suggest that implementations of colour varieties and graphical backgrounds on interfaces should be restricted, especially when icon acquisition is a common activity involved in interacting with an interface.     

Age-related differences in the initial usability of mobile device icons

Mobile devices offer much potential to support older adults (age 65+). However, older adults have been relatively slow to adopt mobile devices. Although much ongoing HCI research has examined usability problems to address this issue, little work has looked at whether existing graphical icons are harder to use for this population compared with younger adults. We conducted a qualitative exploratory study and a follow-up experimental study to determine which icon characteristics help initial icon usability for older adults. We found that our older participants did have more problems using existing mobile device icons, but that particular icon characteristics – semantically close meaning (i.e. natural, close link between depicted objects and associated function), familiar, labelled and concrete (i.e. those depicting real-world objects) – improved icon usability for them. We discuss how these findings can help icon designers to create mobile device icons that are more suited to the abilities and technology experience of older adults.     

EvIcon: Designing High-Usability Icon with Human-in-the-loop Exploration and IconCLIP

Interface icons are prevalent in various digital applications. Due to limited time and budgets, many designers rely on informal evaluation, which often results in poor usability icons. In this paper, we propose a unique human-in-the-loop framework that allows our target users, that is novice and professional user interface (UI) designers, to improve the usability of interface icons efficiently. We formulate several usability criteria into a perceptual usability function and enable users to iteratively revise an icon set with an interactive design tool, EvIcon. We take a large-scale pre-trained joint image-text embedding (CLIP) and fine-tune it to embed icon visuals with icon tags in the same embedding space (IconCLIP). During the revision process, our design tool provides two types of instant perceptual usability feedback. First, we provide perceptual usability feedback modelled by deep learning models trained on IconCLIP embeddings and crowdsourced perceptual ratings. Second, we use the embedding space of IconCLIP to assist users in improving icons' visual distinguishability among icons within the user-prepared icon set. To provide the perceptual prediction, we compiled IconCEPT10K, the first large-scale dataset of perceptual usability ratings over 10,000 interface icons, by conducting a crowdsourcing study. We demonstrated that our framework could benefit UI designers' interface icon revision process with a wide range of professional experience. Moreover, the interface icons designed using our framework achieved better semantic distance and familiarity, verified by an additional online user study.     

Learning and retention of associations between auditory icons and denotative referents: implications for the design of auditory warnings

OBJECTIVE: This study examined the way in which the type and preexisting strength of association between an auditory icon and a warning event affects the ease with which the icon/event pairing can be learned and retained. BACKGROUND: To be effective, an auditory warning must be audible, identifiable, interpretable, and heeded. Warnings consisting of familiar environmental sounds, or auditory icons, have potential to facilitate identification and interpretation. The ease with which pairings between auditory icons and warning events can be learned and retained is likely to depend on the type and strength of the preexisting icon/event association. METHOD: Sixty-three participants each learned eight auditory-icon/denotative-referent pairings and attempted to recall them 4 weeks later. Three icon/denotative-referent association types (direct, related, and unrelated) were employed. Participants rated the strength of the association for each pairing on a 7-point scale. RESULTS: The number of errors made while learning pairings was greater for unrelated than for either related or direct associations, whereas the number of errors made while attempting to recall pairings 4 weeks later was greater for unrelated than for related associations and for related than for direct associations. Irrespective of association type, both learning and retention performance remained at very high levels, provided the strength of the association was rated greater than 5. CONCLUSION: This suggests that strong preexisting associations are used to facilitate learning and retention of icon/denotative-referent pairings. APPLICATION: The practical implication of this study is that auditory icons having either direct or strong, indirect associations with warning events should be preferred.     

Is visual content in textual search interfaces beneficial to dyslexic users?

Dyslexia is a learning disability characterised by problems with accurate or fluent word recognition, poor decoding, and poor spelling abilities. Although several studies have addressed dyslexia and Web accessibility, less is known about how dyslexia affects information search. This study investigated whether the inclusion of icons in search user interfaces enhances performance among dyslexics. A total of 21 dyslexics and 21 controls completed 52 search tasks in 4 conditions: icons only, words only, and both icons and words in a grid layout and a list layout, while eye movements were recorded. Dyslexics took significantly longer than controls to locate targets in tasks containing text, but not in the icon-only condition. Dyslexics had longer fixation durations than controls in both icon and text based search arrays, suggesting higher mental load associated with search tasks generally. The addition of words to icon arrays led to faster search times within controls, but not dyslexics. Dyslexics also exhibited more fixations on dual-modality tasks, and longer scanpaths than controls in list layout. Both groups were fastest searching the list layout, with icons and words listed in columns. Results are discussed in terms of the design of accessible search interfaces for dyslexic users, taking into account mental load of dual-modality information display, and the arrangement of search items. Empirical data is provided for the design of accessible search results interfaces for dyslexics.     

基于粗糙集理论的文本分类算法研究

文本分类是中文信息处理的重要研究领域。给文本分配一个或多个不同的类别,可提高文本检索和存储的处理效率。粗糙集是一种不需要任何先验信息的分类方法,通过对文本分词、过滤掉停用词之后把剩余的词语作为特征项,然后把文本用向量空间模型表示出来,将文本集转化成不带决策属性的信息系统,用粗糙集理论中核心内容属性约简实现对文本的分类。实验表明,该方法的查准率和查全率都有所提高。     

Iterative process of design and evaluation of icons for interactive TV menu

This paper shows an iterative process of design and evaluation of icons for future interactive TV services. In doing the RNRT (French National Network of Research in Telecommunications) iTV project, we tried to generate icons easy to identify, associate and memorise for 32 categories and services of our iTV system.

Through an iterative process, the Multiple Index Approach was applied until an acceptable icon set was achieved. In addition to existing evaluation criteria such as the intuitiveness, associativeness, preference and suitability with subjective certainty of users, we emphasised the importance of the learnability measured by recall tests.

As a conclusion, we propose a methodology of icon design and evaluation for information appliances that integrate unfamiliar features with common users.     

Iterative process of design and evaluation of icons for interactive TV menu

This paper shows an iterative process of design and evaluation of icons for future interactive TV services. In doing the RNRT (French National Network of Research in Telecommunications) iTV project, we tried to generate icons easy to identify, associate and memorise for 32 categories and services of our iTV system.

Through an iterative process, the Multiple Index Approach was applied until an acceptable icon set was achieved. In addition to existing evaluation criteria such as the intuitiveness, associativeness, preference and suitability with subjective certainty of users, we emphasised the importance of the learnability measured by recall tests.

As a conclusion, we propose a methodology of icon design and evaluation for information appliances that integrate unfamiliar features with common users.     

Development and validation of icons varying in their abstractness

Icons are used widely in human-computer interfaces. The level of abstractness-concreteness of an icon and its effect upon performance is of widespread interest. The authors have devised a quantitative measure for abstractness based on the complexity of the icon. They test their metric against subjective judgments of abstractness as identified by two different groups of subjects. After ranking two sets of ‘abstract’ and ‘concrete’ icons, the authors examined how well the icons were matched to the Pascal constructs that they represented. Further experiments were conducted using different groups of subjects to check whether correct matching of the icons with constructs was influenced by context. In summary the authors found that their metric was a good match for subjective measures of abstractness-concreteness. They also found that there is a better identification of concrete icons than abstract icons. Finally, it was shown that context does affect the correct identification of icons.     

Training adult novices to use computers: Effects of different types of illustrations

Two experiments examined the acquisition of word-processing skills (Experiment 1) and internet usage skills (Experiment 2) by novice adults using three types of illustration, specifically, full-screen illustrations with the text superimposed, icons embedded in the text, and a control, text-only condition. Training with the full-screen or embedded icons yielded significant improvements in performance relative to the text-only condition only in tests that simply required the participants to repeat the step-by-step directions. When tests required that the participants themselves retrieve and reproduce the sequence of steps, the training with illustrations – especially the full-screen illustrations – repeatedly had a negative effect on performance. The results suggest that the illustrations, by making it easy to execute each step of a procedure during training, encouraged superficial processing, which in turn yielded a set of stimulus-response associations rather than a well-linked sequence of actions in long-term memory.     

Layout attributes and recall

The spatial arrangement of elements such as icons in a computer interface may influence learning the interface. However, the effects of layout organization on users' information processing is relatively little studied so far. The three experiments of this paper examined two attributes of layouts: spatial grouping by proximity and semantic coherence. Learning was assessed by tasks in which 30 participants recalled icon-like items' labels, locations, or both as a series of study-recall trials. The results show that layout organization interacts with task demands. Semantic organization improves recall of labels, and spatial grouping supports recall of locations. When both labels and locations are learned concurrently, the best recall performance is associated with a simultaneously grouped and semantically coherent layout. However, semantic and spatial organization may interact unexpectedly on learning. The findings are discussed from the viewpoint of information chunking in memory processes and interface design.     

Older observers' tolerance of optical blur: age differences in the identification of defocused text signs

To determine if visual aging affects the ability to identify blurred text signs in daytime or nighttime viewing conditions, the photopic and mesopic legibility thresholds of young and old adult observers were compared at three levels of optically induced acuity (Experiment 1). For both age groups, legibility was reduced by nighttime luminance and degraded acuity. Surprisingly, older observers were better than younger ones in identifying defocused (optically blurred) text in both daytime and nighttime conditions. In Experiment 2, older observers were also superior to younger ones in identifying defocused standard and spatially matched novel text signs. These findings indicate that older observers' tolerance of optical blur is a generic ability, rather than one explained by familiarity with the low-pass optical profile of specific signs. Consistent with the notion that factors beyond acuity contribute more importantly to text legibility for older than for younger observers, acuity was a stronger predictor of legibility thresholds in the younger group. Actual or potential applications of this research include the need to consider functional acuity in visual screening protocols and the design of text displays, as well as the utilization of compensatory letter-recognition training for older observers or others with diminished acuity.     

External tools for collaborative medication scheduling

Medication adherence—taking medication as prescribed—is critical for successful self-care, especially among older adults. Adherence depends on developing and implementing plans for taking medications. Age-related cognitive declines that affect adherence may be mitigated using external tools that support patient-provider collaboration needed to develop these adherence plans. We tested the effectiveness of structured collaborative medication tools to support better medication planning and adherence practices. Evidence for benefits of collaborative tools has been mixed in terms of their usefulness for medication-scheduling tasks, perhaps because the tools have not been explicitly designed to support patient-provider collaboration. A total of 144 community-dwelling older adults participated in pairs and performed the role of a patient or provider in a simulated patient-provider medication-scheduling task. Each pair worked with a structured (MedTable^? and e-MedTable) or unstructured (Medcard) scheduling tool and completed four problems (2 simple and 2 complex). Performance was measured using the following: problem-solving (medication schedule) accuracy, problem-solving time, solution (schedule) optimality, tool usability, collaborative effectiveness, and subjective workload involved in creating the medication schedules. Participants using structured tools produced more accurate and optimal schedules. They also rated subjective workload as lower and thought that the structured tools were easier to use, reduced subjective workload associated with creating the schedules. There was also suggestive evidence that participants using the structured tools rated more highly the quality of their collaboration. Structured medication-scheduling tools have the potential to improve medication adherence among older adults because they support collaborative planning and reduce the cognitive load involved in creating these adherence plans.     

Layout attributes and recall

The spatial arrangement of elements such as icons in a computer interface may influence learning the interface. However, the effects of layout organization on users' information processing is relatively little studied so far. The three experiments of this paper examined two attributes of layouts: spatial grouping by proximity and semantic coherence. Learning was assessed by tasks in which 30 participants recalled icon-like items' labels, locations, or both as a series of study-recall trials. The results show that layout organization interacts with task demands. Semantic organization improves recall of labels, and spatial grouping supports recall of locations. When both labels and locations are learned concurrently, the best recall performance is associated with a simultaneously grouped and semantically coherent layout. However, semantic and spatial organization may interact unexpectedly on learning. The findings are discussed from the viewpoint of information chunking in memory processes and interface design.     

Testing the effectiveness of icons for supporting distributed team decision making under time pressure

There has been minimal experimentation testing the effectiveness of icons (or interface features in general) on distributed team decision making. To overcome this deficiency, an experiment tested the effectiveness of a "send" icon to remind team members to send information to their teammates, and a "receive" icon to tell them when they had received information, for a simulated, military task. As predicted, the "send" icon was effective in maintaining information flow, particularly when time pressure was high and simulated teammates sent less information, because it reduced memory burden and supported proactive behavior. The "receive" icon was only effective in supporting decision accuracy when time pressure was low. As time pressure increased, participants' with the "receive" icon increasingly used a strategy of making decisions before reading the most important information, completely counter to expectations. These results illustrate the subtle, sometimes surprising way task characteristics (e.g., time pressure) can affect participants' strategies and, thereby, ify the positive effect of displays on performance. The experiment also examined other task characteristics and working memory capacity, and showed how the lens model equation (LME) helped explain all effects on decision accuracy.     

Age differences in trust and reliance of a medication management system

The present study examined age differences in trust and reliance of an automated decision aid. In Experiment 1, older and younger participants performed a simple mathematical task concurrent with a simulated medication management task. The decision aid was designed to facilitate medication management, but with varying reliability. Trust, self-confidence and usage of the aid were measured. The results indicated that older adults had greater trust in the aid and were less confident in their performance, but they did not calibrate trust differently than younger adults. In Experiment 2, a variant of the same task was used to investigate whether older adults are subject to over-reliance on the automation. Differences in omission and commission errors were examined. The results indicated that older adults were more reliant on the decision aid and committed more automation-related errors. A signal detection analyses indicated that older adults were less sensitive to automation failures. Results are discussed with respect to the perceptual and cognitive factors that influence age differences in the use of fallible automation.