Understanding multimodal dynamics of behavior

I study how physiological systems respond together during cognitive and affective tasks. The timing of a neural, autonomic, or behavioral response can be as informative as its magnitude, especially when these processes unfold at different timescales.

Combining modalities can improve prediction without explaining their relationships. My work brings synchronized experiments together with probabilistic modeling to understand stimulus-driven responses, spontaneous variation, and the effects of individual and experimental context. The distinction matters when several signals change together but may be responding independently to the same event.

In PRECOG, I contributed to experimental design, modality alignment, and computational analysis in a study of 160 participants. Its synchronized EEG, ECG, electrodermal activity, respiration, eye tracking, and facial recordings connect physiological responses to controlled cognitive and affective tasks. A shared timeline lets me relate changes in each signal to the same experimental events. [1]

Within this experimental setting, I modeled skin conductance responses during an emotional Stroop task as a point process. Six interpretable parameters describe how response timing depends on stimuli and behavior, providing an alternative to reducing a session to aggregate signal features. [2]

In collaboration with Toyota, I examined these timing questions through cardiovascular, electrodermal, respiratory, and behavioral recordings collected during simulated driving. A central challenge is separating responses to individual stressors from slower physiological changes throughout a recording. [3]

Across these settings, I want to model how slower autonomic and respiratory processes coordinate with faster neural and behavioral activity. Comparing individual systems, shared responses to external events, and explicit models of cross-system coupling will help establish when multimodal relationships explain more than the separate signals do.

References

1. PRECOG: Comprehensive Experimental Design and Stimuli Protocol for Investigating Depression and Suicidal Ideation

Myzelle Hughes, Colin McDaniel, Takfarinas Medani, Woojae Jeong, Idan Blank, Dani Byrd, Thomas McGee, Kleanthis Avramidis, Sudarsana Kadiri, Chinmay Chinara, Kristina Lerman, Elsi Kaiser, Richard Leahy, Shrikanth Narayanan, Baruch Rael Cahn

Open Science Framework · 2025

Paper

2. A Point Process Model of Skin Conductance Responses in a Stroop Task for Predicting Depression and Suicidal Ideation

Kleanthis Avramidis, Myzelle Hughes, Idan A Blank, Dani Byrd, Assal Habibi, Takfarinas Medani, Richard M Leahy, Shrikanth Narayanan

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · 2026

Paper PDF

3. Estimating Markers of Driving Stress Through Multimodal Physiological Monitoring

Kleanthis Avramidis, Emily Zhou, Tiantian Feng, Hossein Hamidi Shishavan, Frederico Marcolino Quintao Severgnini, Danny J Lohan, Paul Schmalenberg, Ercan M Dede, Shrikanth Narayanan

arXiv preprint · 2025

Paper PDF Code