ThingMoji: User-Captured Cut-Outs For In-Stream VisualCommunication

Abstract

Live streaming has become increasingly popular, driven by the desire for direct and real-time interactionsbetween streamers and viewers. However, current text-based interactions and pre-defined emojis limit expres-siveness, especially when referring to specific stream moments. We propose ThingMoji, a type of user-capturedcut-outs to enhance user expression and foster more effective communication between streamers and theiraudience in the comment section. ThingMojis are unique digital icons created by users by capturing snapshotsand annotating specific areas at any point during the stream. We developed StreamThing, a live-streamingplatform integrated with ThingMojis, to explore their use during object-focused live streaming contexts.In a user study with three in-the-wild deployments reveals the expressive use of ThingMojis in diverselive-streaming scenarios with rich visual contents. Our findings show that ThingMojis enable viewers toreference specific objects, express emotions, and create shared visual narratives. Streamers found ThingMojisvaluable for facilitating on-the-fly communication around visual content and fostering playful interactions. Thestudy also uncovered challenges in ThingMoji comprehension, issues for long-term uses of ThingMojis, andpotential concerns regarding misuse. Based on these insights, we discussed new opportunities for supportingobject-focused communication during live streaming environments.