Visual attention by consumers to check-all-that-apply questions: Insights to support methodological development

The use by consumers of simplifying response strategies when answering check-all-that-apply (CATA) questions for sensory characterization cannot be eliminated. However, ways to motivate consumers to cognitively engage more in the processing of CATA questions and provide their full attention to the task may be welcome. In this context, the aim of the present work was to evaluate patterns of visual attention to CATA questions by consumers and use the insights gained to advance knowledge about CATA question design. A consumer study was carried out in which participants were asked to evaluate two product sets with five samples each (potato chips or plain crackers) and to answer a check-all-that-apply question composed of 20 sensory terms, which was presented on a computer screen. Half of the consumers received the CATA question with the terms in a fixed order, whereas for the other half of participants the order of terms was different for each sample (i.e., simulating within participants balancing of term presentation order). While consumers completed the CATA tasks their eye movements were recorded using a remote eye-tracker. Results showed that the first time that consumers read the CATA question (i.e., for sample 1), they processed the terms from left to right and from top to bottom. In general, they considered the whole list of terms and then selected those that applied to the sample. As the task progressed consumers changed how they answered the CATA question, making fewer and shorter eye fixations to complete the task. Presenting the terms in different order for each sample significantly increased the total number and duration of fixations, which indicates higher cognitive effort. Implications for the design of CATA questions and suggestions for further research are discussed.     

Check-all-that-apply (CATA) questions for sensory product characterization by consumers: Investigations into the number of terms used in CATA questions

In recent years methodological research into application of CATA questions has gained momentum. Yet, key questions for this approach remain unaddressed – how to generate the sensory terms that populate CATA questions and how many terms should be used. The second of these questions was addressed in seven consumer studies, involving a total of 735 consumers and five product categories (crackers, cheese, fruit-flavored drinks, chocolate, milk desserts). Sensory product characterizations elicited with “short” and “long” CATA questions (10–17 terms vs. 20–28 terms) were compared on a number of criteria such as frequency of CATA term use, product differences, spatial configurations (samples and terms) and task perceptions. Two strategies for generating “long” lists of CATA terms were examined: adding synonym terms to those already featuring on the “short” list (e.g., ‘hard’ and ‘firm’), and adding antonym terms to those already featuring on the “short” list (e.g., ‘hard’ and ‘not hard’ or ‘natural’ and ‘artificial’). Between-subjects experimental designs were used to compare product characterizations from “short” and “long” CATA questions. Results revealed that “short” and “long” lists of CATA terms generated largely similar results. In general, sample configurations were very similar, as were task perceptions. However, there were, at times, differences in frequency of CATA term use and term configurations, as well as instances where conclusions about sample differences depended on whether “short” or “long” CATA lists were used. Additionally, here was some evidence that CATA questions with “long” lists of synonym or antonym terms may cause a “dilution” effect of the responses. This fits expectations of idiosyncrasy in consumer perception/expression of sensory stimuli, but may be associated with reduced discriminatory ability of the CATA question. How to best balance these opposing considerations is deserving of further investigation.     

Comparison of Correspondence Analysis based on Hellinger and chi-square distances to obtain sensory spaces from check-all-that-apply (CATA) questions

Sample configurations from check-all-that-apply (CATA) questions are obtained using Correspondence Analysis (CA). Classical CA is based on chi-square distance, which has been reported to be strongly affected by infrequently selected terms. The Hellinger distance has been proposed as an alternative distance metric, and the aim of the present work was to compare product spaces from CATA questions obtained using CA based on chi-square and Hellinger distances. Data sets from 71 studies (5121 consumers), differing by product category, number of consumers, number of samples and number of terms included in the CATA question, as well as frequency of infrequently used terms, were analyzed. For each of the studies, frequency tables were input to CA based on chi-square and Hellinger distances. Sample and term configurations in the first two dimensions were compared using the RV coefficient. Furthermore, the stability of sample and term configurations for each type of distance was evaluated by simulating repeated experiments using a bootstraping resampling approach. Sample and term configurations obtained using Hellinger and chi-square distances were similar (average RV coefficients for sample configurations = 0.99; average RV coefficients between term configurations = 0.89). The stability of sample and term configurations were not largely affected by the type of distance used to analyze frequency tables. Results from the present work suggest that CA based on chi-square and Hellinger distances provide similar results. Contributing to guidelines for practitioners, this research therefore supports classical CA analysis as an acceptable approach to the analysis of sensory-specific CATA data.     

Recommendations for use of balanced presentation order of terms in CATA questions

Balancing presentation order of the terms of a CATA question between and within participants has been recommended to minimize the influence of primacy bias on consumer responses and maintain their attention throughout the task. Compared to balancing presentation order only between participants, this experimental factor may make the task more difficult/tedious for participants and shift their attention away from the sensory characterization task with possible detrimental effects for sample discrimination. The aim of the present work was to compare sensory product characterizations obtained using CATA questions in which presentation order of the terms was balanced between participants with those obtained using CATA questions in which presentation order of the terms was balanced both between and within participants. Nine studies with 1028 consumers involving different product categories were conducted. Between-subjects experimental designs were used in all studies. No major differences were found in the sensory product characterizations obtained using presentation orders balanced between participants (BB) and both between and within participants (BW).     

Further investigations into the reproducibility of check-all-that-apply (CATA) questions for sensory product characterization elicited by consumers

The purpose of the present research was further investigate the reproducibility of check-all-that-apply (CATA) questions for sensory product characterization. Evaluations obtained when such questions are used by consumers are rarely replicated and therefore reproducibility of the data may be at risk. Results from the present work, which included five studies, each with 100–200 consumers across a range of product categories, revealed that sensory product characterizations obtained using CATA questions with consumers are highly reproducible. Hence, the research confirms previous research by Jaeger, Chheang, et al. (2013) and extends it to the use of CATA terms in randomised presentation order as has been recommended to avoid satisficing response behavior. In the future, if CATA studies are conducted without replication and researches seek to examine the reliability of CATA data, the use of a posterior bootstrapping re-sampling approach is suggested.     

List length has little impact on consumers’ visual attention to CATA questions

Check-all-that-apply (CATA) questions continue to gain popularity and attract interest in developing guidelines regarding their implementation and analysis. The current research continues on this path and considers visual attention by consumers to CATA questions. This is a pre-requisite for elicitation of valid data. Extending previous research, we consider whether CATA question list length influences consumers’ visual attention to the task when it moderately increases from 12 to 20 terms. In a study with six wheat crackers consumers (n ∼ 120) used a CATA questions with either a “short” list (12 terms) or a “long” list (20 terms). The main difference in visual processing was less attention to individual CATA terms when using “long” lists, but greater sustained visual attention to the task. The sensory characterisations for wheat crackers elicited by “long” and “short” lists differed only in minor ways, pointing to little practical impact of the observed differences in visual processing. Upon replication of the current results, consideration of how these findings generalise to test situations with longer CATA questions (30+ terms), tests with fewer/more samples and/or samples with more complex sensory characteristics is warranted.     

Using the emotion circumplex to uncover sensory drivers of emotional associations to products: six case studies

While emotional associations to products are increasingly used alongside sensory product characterisation in consumer research, studies that link these product responses remain less common. Six studies (104–270 people per study) were conducted using diverse products (salted snacks, potato chips, yoghurt, cheese, snack bars and fruit; 5–10 samples per study). To indicate their emotional response during product tasting, consumers used the emotion circumplex which is a single-response questionnaire that spans the dimensions of valence (pleasure to displeasure) and arousal (activation to deactivation). Sensory characterisation was obtained using check-all-that-apply (CATA) questions. Through regression analysis, sensory drivers of emotional associations were established, with different linkages to valence and arousal being identified. Many study-specific linkages were found, but the main result pertained to generalised patterns in some linkages including positive associations between ‘sweet’ and “pleasure”, often combined with “pleasant activation”. For ‘sour’, positive linkages were established to “activation”, typically with, or without, “negative pleasure”. Low flavour intensity was positively linked to the emotion circumplex quadrant that connected “deactivation” to “displeasure.” In general, linkages had straightforward and meaningful interpretations, confirming suitability of the emotion circumplex for use in product-focused emotion research.     

Lack of evidence that concurrent sensory product characterisation using CATA questions bias hedonic scores

In response to the growing use of consumers for sensory product characterisation, methodological research contributing to development of best practise guidelines is ongoing. We focus here on concurrent elicitation of hedonic and sensory product characterisation by check-all-that-apply (CATA) questions. Jaeger et al. (2013b) reported that CATA questions only caused weak and transient bias of co-elicited hedonic scores. In the current research six studies were conducted, in which more than 700 consumers took part. Five product categories were tested (rice crackers, lite bread, cheese, kiwifruit, black currant drinks) with 4–7 samples per study. In none of these studies was evidence obtained suggesting bias of hedonic scores and it is now possible to conclude with greater certainty that co-elicitation of hedonic scores and product attribute information using CATA questions is unlikely to bias hedonic scores. A second result of the current research was that the use of designs that rotate presentation order of CATA terms was not associated with hedonic bias, and neither was the use of the forced Yes–No CATA question format. In future research, in light of a strong dominance of positive CATA terms used in these studies, we recommend studying more thoroughly the influence of positive/negative/neutral words in CATA lists as a possible source of hedonic bias. An exploratory component to this research suggested that consumers perceive the concurrent elicitation of hedonic and CATA responses as easy, but that too many samples may make the task tedious.     

Check-all-that-apply (CATA) questions: Sensory term citation frequency reflects rated term intensity and applicability

CATA questions (check-all-that-apply) are popular for sensory product characterisation tasks with consumers. Lingering uncertainty over the meaning of term citation frequency was addressed in the present research, which directly compared CATA questions with intensity and applicability ratings obtained on 10 cm unstructured line scales. In three studies (n ~ 210 consumers per study), evidence fitting with expectations based on the extant literature was obtained to confirm that citation frequency reflects intensity. The two response types were strongly linearly related, meaning that it is possible to infer significant differences between samples for a given CATA term as representing differences in perceived intensity. This finding emphasises how, despite the simplicity of the CATA task and the fact that individual responses are not measures of intensity, the average citation frequencies do reflect perceived intensity. Therefore, across a group of consumers, there is no information loss relative to rating scales. A systematic difference between citation frequencies and intensity ratings was attenuation of scale use in the latter, which was between 15 and 80 on a 0 to 100 scale. Attenuation was less pronounced in CATA responses which spanned approx. 5 to 85%. In the second part of the research, which included six studies (n ~ 135 to 205 consumers per study) CATA questions were compared with applicability ratings obtained on 10 cm unstructured line scales. The results were highly similar to the comparison between CATA questions and intensity scales and further confirmed that average CATA term citation frequency can be interpreted as representing (but not directly measuring) consumers’ intensity perceptions. The attenuation in scale range was again observed. Collectively, the results contribute further support for the popularity of CATA questions for sensory characterization by consumers in product testing.     

A theoretical description and experimental exploration of tri-reference point theory with respect to food choice

Recent findings within behavioural decision-making suggest that individuals make use of a tri-reference point set when making choices. This implies that choices and preference formation among competing products that are considered acceptable, but differ in desirability, are formed differently along the continuum from bottom line to target level. This study examined whether personal goals, as multiple reference points in relation to food product choice, inherit the properties of a value function. It was posited that goals as cognitive constructs are translated through the target object (the product) and through judgement and context into a representation of identified product preferences. The types of preferences that characterise the different goal levels were then analysed using data collected in an in-store, non-hypothetical consumer experiment with a random sample of 236 consumers. The existence of tri-reference point dependence was strongly supported, with the data indicating that product choices and preferences were moderated by transitions across reference states. Moreover, during transitions notable relative changes in evaluation of the product were identified. These results have normative implications for food product marketing in terms of targeting consumer needs. More importantly, they have strong methodological implications for studies on consumer preferences.     

The use of an ethological approach to evaluate consumers’ appreciation of luxury facial skincare and discriminate between products: A preliminary study

The hedonic dimension of products is a central challenge for cosmetic companies. It is widely measured by liking ratings of the products by consumers, but although this method does help to understand consumer choices, it also carries several experimental biases and provides no information on consumers’ feelings. Here we tested the relevance of combining consumers’ liking ratings with a detailed behavioral approach involving analysis of activities, postures, gazes, and head and body movements of consumers during skincare application to discriminate two products. The sensory characteristics of the two products were also described by a panel of experts.Results showed that behavioral analysis was able to discriminate the two products in terms of satisfaction whereas liking ratings assessment failed to distinguish them. Behavioral analysis also provided relevant complementary information on consumers’ feelings during and after product application, such as interest, detachment or physical comfort. When linked to the sensorial evaluation, the study of behavioral reactions helped understand how the two products’ respective sensory characteristics were perceived by consumers. This preliminary study brings new insight with promising perspectives for research on cosmetics and consumer products in general.     

The influence of taste-congruent soundtracks on visual attention and food choice: A cross-cultural eye-tracking study in Chinese and Danish consumers

Sound can have a profound impact on our eating experience and behavior. The term “sonic seasoning”, arising from crossmodal correspondences, denotes the tendency for soundtracks with congruent taste/flavor attributes to alter people’s food perception. However, the implicit behavior effects of such sound-taste correspondence have not yet been tested. Employing eye-tracking technology, the current study explored the influence of custom-composed taste-congruent soundtracks on visual attention to food, and how this audio-visual relationship differs across cultures. Seventy-two participants (37 Chinese; 35 Danish) were each exposed to three sound conditions (“sweet music”, “salty music”, no music) while observing different food items in a choice paradigm. Across both cultures, participants spent more time fixating on sweet food while listening to “sweet music” and salty food when listening to “salty music”, while no differences were observed in the no music condition. Danish participants had, regardless of sound condition, longer fixation times on the food images compared to their Chinese counterparts. Participants’ choices in each sound condition were consistent with fixation time spent, implying a clear congruency effect between music and choice behavior. Our findings provide evidence of how specifically tailored music can guide consumers’ visual attention to specific food items, suggesting that the brain indeed integrates multiple streams of sensory information during decision-making. The cross-cultural aspect of our study can ultimately be valuable for understanding auditory nudging in different market segments.     

From sensory evaluation to sensory and consumer research of food: An autobiographical perspective

During the past decades, sensory evaluation of food quality has evolved and grown into a discipline that covers sensory and consumer research of foods and beverages. The present review deals with those aspects of the development in which I have been personally involved and have considered inspiring and important subject matters in the field. They are consumer responses to (1) salt, (2) fat, and (3) unfamiliar foods; (4) food choice and socio-cognitive segmentation, (5) responses to food in the elderly and young; and (6) genetic origins of food preferences. Perspectives of the field and of these specific areas are discussed, and some “words of wisdom” are offered for the younger generation of sensory-consumer researchers.     

Design factors influence consumers’ gazing behaviour and decision time in an eye-tracking test: A study on food images

The eye-tracking method has been increasingly used for studying consumer behaviour over the last few years. Understanding factors influencing consumers’ gazing behaviour in an eye-tracking test will contribute to a better organisation and a more valid application of the method. The aim of this work is to study how test design influences gazing behaviour and decision time of food consumers in an eye-tracking test. Three factors of the test design were investigated: (1) Number of images in one testing picture (two, three, four, five, and six images/picture); (2) content of question (tastiness, healthiness, price, convenience, and familiarity); and (3) type of evaluation (maximum choice, minimum choice, ranking, rating, and grouping). Two experiments were conducted. In the first experiment, performed with 100 participants, the influence of individual factors was studied. In the second experiment, performed with 64 participants, the joint effects (interactions) of the tested factors were investigated. The results showed that gazing behaviour and decision time are strongly influenced by the type of evaluation and the number of images, but not by the content of question. No joint effect of influencing factors (number of images and type of evaluation) was found. Findings are discussed in considering the relationship between eye-movements, cognitive goals, and tasks. This study highlights the importance of understanding factors influencing gazing behaviour and decision time in an eye-tracking test.     

Dynamics of food preferences: a case study with chewing gums

The dynamics of food perception is critical in sensory science. However, although much research deals with time–intensity measurement as a tool for analytical sensory evaluation, it is striking that almost no published research investigates the dynamics of hedonic responses. Here we studied how consumers scored mint flavored sugar-free gums when the test duration varied. Six coated dragée-like gums from the French market were tested by each consumer according to three test conditions: 1 min of chewing, 5 min and half-an-hour. One and five minute tests took place in the sensory lab evaluation booths whereas consumers were free to move about for the 30 min modality. All consumers thus participated to a total of 18 sessions. Our main finding is that the average liking varies with the test duration but that this variation differs for the six products in a way that one of the initially least liked products becomes the most liked when it is chewed during 30 min. Also, the individual liking patterns and the resulting consumer segmentation varied widely across the three test conditions. Additionally, a Flash profile was performed with experienced subjects in an attempt to relate the observed differences in liking to the gum sensory characteristics. Trigeminal-related sensations seem to play an important role in these changes. Clearly such aspects are to be taken into account for new product development and market analysis. This kind of concerns may be even more challenging for hedonic evaluation of many non-food products.     

Development of the engagement questionnaire (EQ): A tool to measure panelist engagement during sensory and consumer evaluations

Sensory and consumer testing are primary strategies companies use to collect insights about consumer products. However, frequently, unreliable results from these tests can lead to failed product launches which can have enormous financial and/or brand equity consequences. One potential reason is sensory tasks are often deliberately devoid of additional stimulation which reasonably has the possibility to reduce a panelist’s engagement with the task. Engagement may drive involvement and focus during testing and help generate reliable data. The overall objective of this study was to develop (Study 1), refine (Study 2), and validate (Studies 3–5) an engagement questionnaire (EQ), a tool to empirically assess varying engagement levels with a task. Prior literature and an exploratory qualitative assessment through an online platform (Study 1, N = 87) were used to operationalize the dimensionality of engagement and develop a preliminary questionnaire consisting of 54 items covering 7 themes (ability, attention, passivity, involvement, interest, motivation, and relevance). To assess the effectiveness of the initial item set, participants recalled being engaged with a task and answered the preliminary EQ (Study 2, N = 186). Results collected underwent an exploratory factor analysis (EFA) in order to explore the dimensionality and refine the scale by strategically eliminating items. To validate the tool, the EQ was distributed following numerous consumer sensory tests that spanned a variety of food and beverage products, test lengths, number, and types of questions (Studies 3–5). EFA was conducted in Study 3 (N = 774), which resulted in a three-factor, 10-item model with good reliability (α = 0.856). The new model was subsequently validated in two additional consumer sensory studies (Study 4A, N = 416 and Study 4B, N = 446). Both validation studies produced acceptable to good model fit indices with an RMSEA below 0.07, SRMR below 0.04, both CFI and TLI above 0.95 and good reliability (α > 0.850). When tested in a different sensory testing facility, the EQ produced acceptable model fit indices (Study 5, N = 247) and tracked as expected with participants’ perceived workload collected using the NASA Task Load Index; confirming the content and convergent and divergent validity of the instrument. Such an instrument allows for deeper understanding of panelist engagement and its impact on data quality, enabling the design of sensory tests capable of generating more reliable data both within the field of food science and beyond.