NWPU researchers pinpoint emotional peaks in video

A video may spend most of its time in a neutral or transitional state, then turn emotionally charged for only a few seconds. Traditional systems often label the whole clip anyway. At Northwestern Polytechnical University, Professor Xie Songyun and researchers at the School of Artificial Intelligence have built a dataset designed to find that peak instead, and their tests lifted eight emotion-recognition algorithms from 34.7% to 38.5% average accuracy.

The team calls the multimodal dataset FIRMED. Participants watched experimental videos, immediately replayed them and marked one or more obvious emotional moments, reporting both category and intensity. That immediate recall was intended to avoid interrupting natural emotion and physiological-signal collection while reducing the memory bias that can come with delayed reporting. The researchers then centered a four-second emotion event window on each mark: two seconds before it and two seconds after it.

Ablation comparisons found that four seconds balanced completeness and precision; a five-second window diluted features associated with surprise and disgust, while weakening fear-related alpha suppression and happiness-related gamma enhancement. In a check of 150 randomly selected records, two independent reviewers reached a Cohen’s kappa of 0.96 for emotion-category agreement, and 97.7% of time markers fell within one second.

The practical shift is from asking which emotion fills a video to asking when the signal becomes meaningful. That could help a human-computer interface respond at the most emotionally charged instant, let mental-health screening focus on clinically meaningful segments, or support real-time warnings in wearable devices. Those uses are proposed applications, not deployments described in the study.

So what changes now? For researchers building emotion models, the result offers a more precise labeling method and a measurable gain across eight representative algorithms, with deeper spatiotemporal models benefiting most. The reported evidence comes from the NWPU team’s evaluation.