Harnessing the Time and Channel Dynamics of Motion Time Series Classification

Abstract

Time series data consists of two axes: time and channel. The time axis records information about tem- poral progression and events, while different channels (i.e. sensors) encode unique information based on their measurement properties and positioning. These time and channel characteristics vary significantly depending on the type of time series, and effectively harnessing these characteristics is essential for de- veloping deep learning methodologies suitable for the specific time series in task. Our thesis focuses on motion time series, a time series that captures different human motion activities with different sensors. We specifically examine how we can effectively utilize time and channel axes to develop methodolo- gies that address practical challenges each of the domain. We focus on motion time series for two key reasons: the repetitive temporal periodicity and strong cross-sensor correlation present unique method- ological challenges in the time series domain, while its applications in healthcare, sports science, and human-computer interaction offer practical real-world usage. This work addresses three fundamental research questions at the intersection of deep learning and motion time series classification: (1) How to effectively harness time and channel characteristics to enable representation learning from unlabeled time series data, reducing dependence on scarce labeled datasets; (2) How to identify and validate the contribution of individual channels to classification performance, enabling sensor redundancy detection and enhancing model interpretability; and (3) How to develop transfer learning approaches that adapt models across users with different temporal patterns and channel dynamics, accounting for time and channel-wise variability. To address these challenges, this dissertation is structured as follows. Chapter 2 introduces PPT (Patch- order aware Pretext Task), a novel self-supervised learning strategy that explicitly supervises the order- ing of patches across both temporal and channel dimensions, enabling effective representation learning from unlabeled motion time series. Chapter 3 presents CAFO (Channel Attention and Feature Or- thogonalization), an explainable framework that identifies and validates channel importance through channel attention, enabling the identification of important or redundant sensors. Chapter 4 develops TransPL (Transitional Pseudo-Labeling), a domain adaptation methodology that generates high-quality pseudo-labels by explicitly modeling temporal and channel dynamics between source and target domains. TransPL outperforms traditional pseudo-labeling approaches by incorporating domain-specific temporal patterns and channel-wise shifts, enabling effective knowledge transfer across different users. Through this thesis, we highlight the need to thoroughly understand and leverage the unique time and channel dynamics of motion time series to develop novel and suitable deep learning methodologies. Our approaches demonstrably reduce reliance on labeled data, improve model interpretability, and enhance transferability across different domains, ultimately providing practical deep learning methodologies for real-world motion time series challenges.