An empirical investigation of the novice experience with soft keyboards

An experiment with 12 participants tested text entry rates on two sizes of soft keyboards with either a Qwerty layout or a layout presenting a randomized letter arrangement after each tap. The randomized layout simulated the novice experience by requiring users to visually scan the layout for each tap to find the intended letter. Rates for the Qwerty layouts were about 20 wpm with no significant difference between the large and small size. Rates for both sizes of the randomized layouts were very low, about 5.4 wpm. This is the expected walk-up text entry rate with a soft keyboard bearing an unfamiliar layout. This empirical result allows us to reject a previous model of novice interaction that used Fitts' law for stylus movement and the Hick-Hyman law for visual scan time.     

Text entry using soft keyboards

Text entry rates are explored for several variations of soft keyboards. We present a model to predict novice and expert entry rates and present an empirical test with 24 subjects. Six keyboards were examined: the Qwerty, ABC, Dvorak, Fitaly, JustType, and telephone. At 8-10 wpm, novice predictions are low for all layouts because the dominant factor is the visual scan time, rather than the movement time. Expert predictions are in the range of 22-56 wpm, although these were not tested empirically. In a quick, novice test with a representative phrase of text, subjects achieved rates of 20.2 wpm (Qwerty), 10.7 wpm (ABC), 8.5 wpm (Dvorak), 8.0 wpm (Fitaly), 7.0 wpm (JustType), and 8.0 wpm (telephone). The Qwerty rate of 20.2 wpm is consistent with observations in other studies. The relatively high rate for Qwerty suggests that there is skill transfer from users' familiarity with desktop computers to the stylus tapping task.     

标准键盘布局比较与优化设计

键盘作为基本的人机交互设备之一,在多个领域有广泛的应用。对于标准键盘,现有的Qwerty键盘布局并不是为提高工作效率而设计的。而键盘输入作为一个复杂的人体工程学问题,依赖于操作者的习惯、不同手指的易用性差异以及不同的文本信息等诸多因素。在键盘输入过程分解、手指参数假定和语言文本分析之上,通过定义距离矩阵和文本矩阵,为标准键盘布局比较和优化设计建立较为客观的评价模型,并对现有键盘布局进行比较。作为应用,给出了针对汉语拼音输入的新键盘布局,并利用实际文本进行了测试与比较。     

Theoretical upper and lower bounds on typing speed using a stylus and a soft keyboard

A theoretical model is presented to predict upper-and lower-bound text-entry rates using a stylus to tap on a soft QWERTY keyboard. The model is based on the Hick-Hyman law for choice reaction time, Fitts law for rapid aimed movements, and linguistic tables for the relative frequencies of letter-pairs, or digrams, in common English. The model's importance lies not only in the predictions provided, but in its characterization of text-entry tasks using keyboards. Whereas previous studies only use frequency probabilities of the 26 × 26 digrams in the Roman alphabet, our model accommodates the space har—the most common character in typing tasks. Using a very large linguistic table that decomposes digrams by position-within-words, we established start-of-word (space-letter) and end-of-word (letter-space) probabilities and worked from a 27 × 27 digram table. The model predicts a typing rate of 8.9wpm for novices unfamiliar with the QWERTY keyboard, and 30.1wpm for experts. Comparisons are drawn with empirical studies using a stylus and other forms of text entry.     

Theoretical upper and lower bounds on typing speed using a stylus and a soft keyboard

Abstract

A theoretical model is presented to predict upper-and lower-bound text-entry rates using a stylus to tap on a soft QWERTY keyboard. The model is based on the Hick-Hyman law for choice reaction time, Fitts law for rapid aimed movements, and linguistic tables for the relative frequencies of letter-pairs, or digrams, in common English. The model's importance lies not only in the predictions provided, but in its characterization of text-entry tasks using keyboards. Whereas previous studies only use frequency probabilities of the 26 × 26 digrams in the Roman alphabet, our model accommodates the space har—the most common character in typing tasks. Using a very large linguistic table that decomposes digrams by position-within-words, we established start-of-word (space-letter) and end-of-word (letter-space) probabilities and worked from a 27 × 27 digram table. The model predicts a typing rate of 8.9wpm for novices unfamiliar with the QWERTY keyboard, and 30.1wpm for experts. Comparisons are drawn with empirical studies using a stylus and other forms of text entry.     

Experimental Evaluations of the Twiddler One-Handed Chording Mobile Keyboard

The HandyKey Twiddler? is a one-handed chording mobile keyboard that employs a 3 × 4 button design, similar to that of a standard mobile telephone. We present a longitudinal study of novice users' learning rates on the Twiddler. Ten participants typed for 20 sessions using 2 different text entry methods. Each session was composed of 20 min of typing with multitap and 20 min of one-handed chording on the Twiddler. We found that users initially had a faster average typing rate with multitap; however, after 4 sessions the difference became negligible, and by the 8th session participants typed faster with chording on the Twiddler. Five participants continued our study and achieved an average rate of 47 words per minute (wpm) after approximately 25 hr of practice in varying conditions. One participant achieved an average rate of 67 wpm, equivalent to the typing rate of the 2nd author, who has been a Twiddler user for 10 years.We analyze the effects of learning on various aspects of chording, provide evidence that lack of visual feedback does not hinder expert typing speed, and examine the potential use of multicharacter chords (MCCs) to increase text entry speed. Finally, we explore improving novice user's experience with the Twiddler through the use of a chording tutorial.     

Space, colour and typography on visual display terminals

Some guidelines are given to meet the observed need for rules about layout, the use of colour and typography on display screens so as to create texts with optimal legibility. Examples of videotex pages are used to illustrate right and wrong layouts, applications of colour and of letter type. The guidelines can be generalized to other types of display such as those used in personal computers and, to a more limited extent, to the use of graphics instead of text. Finally, figures are given on the general public's subjective appreciation of some alternative display layouts.     

Improvised layout of keypad entry system for mobile phones

In this paper, we present a critical review of the current layout of alphabets on a mobile keypad and seek to improvise it for optimized text entry to facilitate user interaction with or without the use of predictive text input techniques. Currently, most mobile phones use multi-press as the preferred method of text entry though they also offer word-based disambiguation schemes. However, keypad layouts are not optimized for text entry using word based disambiguation schemes as several matches exist for the same numeric combination, some of which are frequently used words. This scheme effectively slows down text entry speeds, requiring more tapping for disambiguating matches. The proposed model seeks to reduce the number of matches for any possible numeric combinations, by optimization of keypad layout by repositioning alphabets on the keypad. For users not using dictionary, our model requires lesser tapping for commonly used alphabets and groups commonly used key combinations together such as in a computer keyboard.The proposed model was derived by simulation of the mobile phone keypad on a computer system and it uses cognitive agents to derive the most optimum keypad layout. Our model uses frequently used English words from a dictionary and attempts to minimize the number of matches for any given numeric key combinations, though the same could easily be duplicated for other languages. The model is expected to cause a significant rise in the text input speeds of mobile phones and other embedded devices with limited text entry capabilities, leading to better usability and customer satisfaction.     

Productivity prediction by extrapolation: Using workload memory as a predictor of target performance

In order to assess if productivity based on extrapolated data is a good predictor of longer texts, an experimental study was conducted. Two full-sized text input devices for touch typing and two miniaturized for tapping were used, all featuing QWERTY layout, in a repeated measurement design. Twenty subjects were exposed to both a task within the limit of working memory (nine words) and four running memory tasks (approx. 275 words). For miniaturized tapping keyboards, extrapolated data significantly underestimated both entry speed (uncorrected wpm, up to 17%) and character error rate (up to 61%) whereas it significantly overestimated ratio of correct words (up to 62%) of running memory tasks. Further, error-corrected entry speed was significantly overestimated up to a factor of 2.7. Results based on extrapolated productivity metrics must therefore be interpreted with caution. Running memory tasks with text length of more than 32 words is needed to assess productivity of text input devices if tapping is used.     

Productivity prediction by extrapolation: using workload memory as a predictor of target performance

In order to assess if productivity based on extrapolated data is a good predictor of longer texts, an experimental study was conducted. Two full-sized text input devices for touch typing and two miniaturized for tapping were used, all featuing QWERTY layout, in a repeated measurement design. Twenty subjects were exposed to both a task within the limit of working memory (nine words) and four running memory tasks (approx. 275 words). For miniaturized tapping keyboards, extrapolated data significantly underestimated both entry speed (uncorrected wpm, up to 17%) and character error rate (up to 61%) whereas it significantly overestimated ratio of correct words (up to 62%) of running memory tasks. Further, error-corrected entry speed was significantly overestimated up to a factor of 2.7. Results based on extrapolated productivity metrics must therefore be interpreted with caution. Running memory tasks with text length of more than 32 words is needed to assess productivity of text input devices if tapping is used.     

Multilingual Touchscreen Keyboard Design and Optimization

A keyboard design, once adopted, tends to have a longlasting and worldwide impact on daily user experience. There is a substantial body of research on touch-screen stylus keyboard optimization. Most of it has focused on English only. Applying rigorous mathematical optimization methods and addressing diacritic character design issues, this article expands this body of work to French, Spanish, German, and Chinese. More important and counter to the intuition that optimization by nature is necessarily specific to each language, this article demonstrates that it is possible to find common layouts that are highly optimized across multiple languages for stylus (or single finger) typing. We first obtained a layout that is highly optimized for both English and French input. We then obtained a layout that is optimized for English, French, Spanish, German, and Chinese pinyin simultaneously, reducing its stylus travel distance to about half of QWERTY's for all of the five languages. In comparison to QWERTY's 3.31, 3.51, 3.7, 3.26, and 3.85 keys of movement for English, French, Spanish, German, and Chinese, respectively, the optimized multilingual layout has an average travel distance of 1.88, 1.86, 1.91, 1.77, and 1.68 keys, correspondingly. Applying Fitts's law with parameters validated by a word tapping experiment, we show that these multilingual keyboards also significantly reduce text input time for multiple languages over the standard QWERTY for experienced users. In comparison to layouts individually optimized for each language, which are also obtained in this article, simultaneously optimizing for multiple languages caused only a minor performance degradation for each language. This surprising result could help to reduce the burden of multilingual users having to switch and learn new layouts for different languages. In addition, we also present and analyze multiple ways of incorporating diacritic characters on multilingual keyboards. Taken together, the present work provides a quantitative foundation for the understanding and designing of multilingual touch-screen keyboards.     

Banner layout retargeting with hierarchical reinforcement learning and variational autoencoder

In many advertising areas, banners are often generated with different display sizes, so designers have to make huge efforts to retarget their designs to each size. Automating such retargeting process can greatly save time for designers and let them put creativity on new ads. This paper proposes a hierarchical reinforcement learning-based (HRL-based) method and a variational autoencoder-based (VAE-based) method by treating the automated banner retargeting problem as a layout retargeting task. The HRL and VAE models are trained separately to learn the scaling and positioning policy of the design elements from an original (base) layout. Hence, the proposed method can generate appropriate layouts for different target banner sizes. Meanwhile, evaluation metrics are proposed to assess the quality of generated layouts and are also reward conditions during the training process. To evaluate performances of the two models, SOTA methods such as Non-linear Inverse Optimization (NIO), Triangle Interpolation (TI), and Layout GAN (LGAN) are implemented and compared. Experimental results show that both HRL- and VAE-based methods retarget design layouts effectively, and the VAE model achieves better performance than the HRL model.

     

BlinkWrite: efficient text entry using eye blinks

In this paper, a new text entry system is proposed, implemented, and evaluated. BlinkWrite provides a communication gateway for cognitively able motor-impaired individuals who cannot use a traditional eye-tracking system. In contrast to most hands-free systems, BlinkWrite allows text to be entered and corrected using a single input modality: blinks. The system was implemented using a scanning ambiguous keyboard, a new form of scanning keyboard that allows English text to be entered in less than two scanning intervals per character. In a user study, 12 participants entered text using the system with three settings for scanning interval: 1,000, 850, and 700 ms. An average text entry rate of 4.8 wpm was observed with accuracy >97%. The highest average text entry rate was achieved with the scanning interval of 850 ms.     

Visual perception of multi-column-layout text: insight from repeated and non-repeated reading

Information processing speed affects reading performance and interaction with text. Understanding how column type in an online context influences reading effectiveness can help us to identify less effective layouts. This study explored the visual perception of 23 participants while they read text arranged in a multi-column layout. Two tasks (repeated reading and non-repeated reading) were designed and assessed to have the same level of difficulty. Information was organised according to three types of column layout (one, two, or three columns). Eye movement analysis showed that participants performed best in a three-column layout for repeated reading, and with one column for normal reading. We also found that the repeated reading technique reduced readers’ distraction and therefore increased their visual performance, which in turn increased information processing, regardless of column layout. These findings with regard to single- and multi-column layouts can help suggest effective reading configurations for online readers and provide insights for human–computer interaction theories on human interaction with different typographic elements.     

Performance comparison between the preferred right and preferred left hands in text entry using Qwerty touch keyboard smartphones

In one-handed text entry with a smartphone, we investigate whether the use of the preferred hand presents better performance than when the non-preferred hand is used. In particular, we study whether the performance of a skilled left hand is worse than that of a skilled right hand among right-handers. A total of 30 young, male, right-handed undergraduate students participated in the experiment. Half of the participants preferred to use their right hands for the smartphone text entry when only one hand was available, whereas the other half preferred to use their left hands. The participants were instructed to type a text message using a Qwerty touch keyboard with both hands, with right hand only, and left hand only through three different experiments. The text completion time and number of errors were measured, and words per minute and keystrokes per minute were calculated to compare the performance among the text entry tasks. In entering a text message with only one hand, approximately 70% of the 15 participants who preferred to use their right hands and approximately 30% of the other 15 participants who preferred to use their left hands have shown to perform better when using their preferred hand than when using their non-preferred hand. The performance of the participants with a skilled left hand was not worse than that of those with a skilled right hand in smartphone text entry.     

Assessing two new wearable input paradigms: The Finger-Joint-Gesture palm-keypad glove and the invisible phone clock

Text and digit entry speed of two wearable one-handed input paradigms, the Finger-Joint-Gesture palm-keypad glove and the Invisible Phone Clock, were benchmarked against traditional one-handed cellular phone keypad input in a repeated-measurement design employing 18 subjects using a 9-word sentence. No significant difference in error-corrected text entry speed (5.3 ec-wpm) was found. Digit entry speed was significantly faster (8.3 ec-wpm) and differed significantly between input paradigms. Furthermore, digit entry was fastest for the traditional cellular phone keypad and slowest for the Invisible Clock keypad. A prediction model based on Fitts' law slightly overestimated text entry speed for novice users. Another prediction model, where each movement time between successive keys was corrected for key repeat time for each specific input paradigm, predicted the experimental results more accurately. Thirteen of the subjects ranked the Invisible Phone Clock as 1^st choice. The subjects' mapping of the Ericsson cellular phone functions (YES, CLR, NO, <,> and Address Book) was not according to the designer's model, partly due to functional fixedness. The input paradigms could be suitable candidates for new fragmentised interfaces where wearability is the key issue.     

Text input for motor-impaired people

This paper provides an overview of 150 publications regarding text input for motor-impaired people and describes current state of the art. It focuses on common techniques of text entry including selection of keys, approaches to character layouts, use of language models, and interaction modalities. These aspects of text entry methods are further analyzed, and examples are given. The paper also focuses on an overview of reported evaluations by describing experiments, which can be conducted, to assess the performance of a text entry method. Following this overview, a summary of 61 text entry methods for motor-impaired people found in the related literature is presented, classifying those methods according to the aforementioned aspects and reported evaluation. This overview was assembled with the aim to provide a starting point to the new researchers in the field of accessible text entry. The text entry methods are also categorized according to the suitability for various conditions of the users.     

Influence mechanism of icon semantics on visual search performance: Evidence from an eye-tracking study

Explicit icon semantics can reduce the difficulty of understanding complex visual information. Optimizing the icon semantics and text semantics of icons can effectively improve the cognitive performance of digital interfaces. This paper adopts visual search tasks to study the effects of different combinations of icon semantic familiarity and the presence or absence of text on icon search performance under horizontal and vertical layouts. The behavioral experiment results show that under two layouts: 1. The main effect of icon semantics is significant, and the search performance increases with the increase of semantic familiarity. 2. The main effect of text is significant, and the search performance is negatively correlated with the addition of text. The eye movement experiment found that the semantic familiarity of icons had a significant impact on average fixation time. Furthermore, the number of fixation points changed significantly after the text variable was added. Therefore, there was no significant difference in the number of fixation points in the horizontal layout, and icon semantics was the main influencing factor in visual search. In the vertical layout, there was no significant difference in average fixation time, and text was the main influencing factor of visual search. The results show that the semantic familiarity of icons and different combinations with or without text significantly affect visual search performance in horizontal and vertical layouts. This paper provides a theoretical reference for the combination of icons and text in interface design.     

Decoding Substitution Ciphers by Means of Word Matching with Application to OCR

A substitution cipher consists of a block of natural language text where each letter of the alphabet has been replaced by a distinct symbol. As a problem in cryptography, the substitution cipher is of limited interest, but it has an important application in optical character recognition. Recent advances render it quite feasible to scan documents with a fairly complex layout and to classify (cluster) the printed characters into distinct groups according to their shape. However, given the immense variety of type styles and forms in current use, it is not possible to assign alphabetical identities to characters of arbitrary size and typeface. This gap can be bridged by solving the equivalent of a substitution cipher problem, thereby opening up the possibility of automatic translation of a scanned document into a standard character code, such as ASCII. Earlier methods relying on letter n-gram frequencies require a substantial amount of ciphertext for accurate n-gram estimates. A dictionary-based approach solves the problem using relatively small ciphertext samples and a dictionary of fewer than 500 words. Our heuristic backtrack algorithm typically visits only a few hundred among the 26! possible nodes on sample texts ranging from 100 to 600 words.     

Bengali text input interface design for mobile devices

Text entry has become one of the most frequent activities performed using mobile devices such as PDAs. Virtual keyboards (VK), which allow text to be entered by tapping keys displayed on the screen, are among the predominant mode of text input for such devices. It is important to design VK layouts in a way such that the users can achieve high text entry speed with high accuracy. Several layouts, primarily in English and also in some other languages, have been proposed to achieve the twin objectives. However, no such work has been reported for Bengali, the second and fifth most popular language of India and the world, respectively. The existing methods cannot be applied directly to Bengali VK design due to the problem of accommodating the large Bengali alphabet (more than 60 characters) on a small display area. In order to resolve the resulting usability-performance trade-off, this paper proposes a two-level design. Five two-level VKs representing three design paradigms (alphabetic, frequency-based and adaptive) have been designed and compared in an empirical study. The study results show that for mobile devices, the two-level adaptive design is expected to give best performance in terms of text entry rate and accuracy. The layouts as well as the procedure and results of the study are discussed in this paper.