Shock-Biased Attention: Enhancing Transformer Hawkes Processes with Amplitude-Driven Temporal Kernels

Abstract

Temporal point processes (TPPs) are fundamental for modeling event sequences in the fields of finance, where each event is often accompanied by a continuous mark (e.g., transaction volume) that modulates future event dynamics. Among these, Hawkes processes have emerged as the most widely used class of TPPs in the financial domain due to their ability to capture the self-exciting nature of events such as trades, order placements, and cancelations. However, existing Hawkes Process models do not take these marks into account in their prediction model, and marked temporal point process models use post-attention methods, in which the mark information is embedded into the intensity function and not the attention module itself. In this work, we propose a novel shock-biased attention Transformer Hawkes Process that explicitly incorporates mark-modulated temporal dependencies through a shock-biased attention mechanism. Our approach augments the attention matrix of the transformer with a domain-informed bias term, whose amplitude is a function of both the event mark and the elapsed time with exponential temporal decay. Experiments on real-world datasets from financial markets demonstrate that our model outperforms existing methods in event prediction tasks, while providing interpretable parameters that align with domain knowledge. Our framework offers a unified and interpretable approach to mark-aware temporal modeling, enabling data-efficient event sequence analysis across diverse domains.