Mellow: A Tangible Music Interface Exploring Interaction Elements for Engagement in Music Editing

Abstract

Music listening has increasingly become a passive experience, shaped by algorithm-driven recommendation systems and one-shot AI generation tools that deliver completed outcomes with limited user intervention. Although recent AI-based music systems have reduced technical barriers, they often rely on linguistic prompts, offer limited real-time controllability, and undermine users’ sense of authorship. As a result, many music listeners remain disengaged from the process of actively editing or shaping music. This study explores how engagement in music editing can be shaped at the interaction level through tangible user interface (TUI) elements, focusing on embodied participation rather than algorithmic personalization. I present Mellow, a tangible music interface designed to investigate how a set of interaction elements—including tactile input, material properties, and light-based visual feedback— work together to support active participation in music editing while listening. By integrating pressure- based interaction with deformable materials and ambient light feedback into a real-time music editing workflow, Mellow enables users to explore musical aspects such as rhythm, spatiality, and texture without relying on formal musical training or complex graphical interfaces. To inform the design, I conducted a preliminary qualitative study with everyday music listeners to identify limitations in existing AI-based music systems and to understand factors influencing engagement during music interaction. Based on these insights, I iteratively prototyped Mellow and conducted a controlled user study evaluating four surface materials with distinct physical properties. The findings indicate that interaction elements embedded within the tangible interface play a central role in shaping engagement by mediating users’ sense of intervention and control during music editing. In particular, the combination of low-viscosity, moderately elastic materials and immediate visual feedback significantly improved perceived suitability, comfort, enjoyment, and consistency across users. Excessive physical resistance disrupted continuous engagement, whereas appropriately tuned tactile and visual responses supported transparent and sustained music editing experiences. This study contributes (1) an exploration of tactile and light-based interaction elements that support engagement in music editing, (2) a tangible music interface case study demonstrating how tactile and light-based feedback function as complementary TUI elements, and (3) empirical design guidelines for integrating materiality and visual feedback in deformable music interfaces.