Exploring the Effects of Gesture-Based Interaction on Virtual Product Experience

RESEARCH BACKGROUND

Deprivation of Sensory Input in Online Shopping

In real world context, consumers evaluate a product based on

various sensory inputs from sight, touch, smell, hear and taste.

But in online setting, most of the sensory inputs are missing.

The lack of sensory information about products make it difficult

for consumers to obtain a comprehensive evaluation of their

desired products, which may further lead to unpleasant shopping

experience. Among different types of sensory inputs, touch has

been recognized as a critical way to get information.

​

Virtual Product Experience Design to Facilitate Product Evaluation
In the past, VPE research mainly focused on visual product presentation and interactivity (mouse-based). Nowadays, retailers are exploring innovative ways to create a realistic and immersive online shopping experience. Gesture-based interaction is widely adopted by retailers because of its potential ability to generate haptic experience (sense of touch).

 

 

 

 

 

 

 

 

RESEARCH QUESTIONS:

1. What are the mechanism of different interaction mode (mid-air vs. touchscreen gestures) in generating haptic experience (sense of touch)?

2. How does Gesture-Based Interaction influence consumers’ online shopping experience by eliciting haptic experience?

3. What’s the boundary condition of such effects?

​

THEORETICAL FOUNDATION:

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

METHODOLOGY:

Laboratory Experiment & Survey
Procedure

Design: 2 (interaction method: touchscreen interaction vs. mid-air interaction) × 2 (importance of product haptic information: high vs. low) between-subjects
Participants: 120 participants from a public university
Pre-experiment training in all conditions: participants were instructed on how to interact with the product using a different stimulus (i.e., a handbag)
Task: Complete a shopping task
Post-experimental questionnaire: 

Sense of Touch, Product Uncertainty, Product Attachment, Shopping Experience Satisfaction, Demographic Information, Need for Touch, Importance of Product Haptic Information, Perceived Novelty of Interaction Method
 

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

 

 

 

 

KEY FINDINGS:

1. Different interaction mode generates different levels of haptic experience.

2. Compared with mid-air interaction, touchscreen interaction can elicit a higher sense of touch when haptic information is highly instrumental to product evaluation.

3. Sense of touch enhances online shopping satisfaction by affecting both cognitive and affective dimensions of virtual product experience.

​

IMPACTS:

Theoretical Impacts:

1. Sense of touch enhances online shopping satisfaction by affecting both cognitive and affective dimensions of virtual product experience.

2. Underscore the interplay between consumers’ bodily interaction with the devices in the physical world and their product perception in the virtual environment.

3. Provide theoretical framework to evaluate the effects of bodily interaction on users’ attitude and behavior.

Practical Impacts:

1. Demonstrates the advantage of touchscreen interaction over mid-air interaction in eliciting online consumers’ sense of touch when the haptic information of the products is of high importance.

2. Point out the need to take consumers’ bodily sensations into consideration and enrich consumers’ sensory experience when designing for virtual product experience.

​

LIMITATIONS:

1. We only compared simple common gestures in our study, but did not consider the specific gestures supported by different gesture-based interaction.

2. This study focused on the impacts of gesture-based interaction on people’s haptic sensation. (Visual?)

3. Only assessed two types of gesture-based interaction.

​

FUTURE RESEARCH DIRECTIONS:

1. Investigate the impacts of more advanced mid-air interaction technologies that support complex mid-air gestures.

2. Explores how the interplay of different sensation (e.g. visual sensation) affects consumers’ online shopping experience.

3. Compare other novel interaction modes (e.g., eye-gaze, voice, and head movements)