Towards Automated Motor Impulsivity Monitoring in Real-world Scenarios: A Multiple Object Tracking Approach

Fahmy Dalimarta; Pulung Nurtantio Andono; Moch. Arief Soeleman; Zainal Arifin  Hasibuan

doi:10.32734/jocai.v9.i1-16686

Authors

Fahmy Dalimarta Universitas Dian Nuswantoro
Pulung Nurtantio Andono Universitas Dian Nuswantoro https://orcid.org/0000-0001-7408-0558
Moch. Arief Soeleman Universitas Dian Nuswantoro https://orcid.org/0000-0001-6099-7023
Zainal Arifin Hasibuan Universitas Dian Nuswantoro https://orcid.org/0000-0003-3307-915X

DOI:

https://doi.org/10.32734/jocai.v9.i1-16686

Keywords:

Multiple Object Tracking, Motor Impulsivity, Quantification

Abstract

Assessment of motor impulsivity often faces several challenges. Conventional assessments that rely on controlled settings often fail to capture impulsive behaviors in real-world contexts. This study proposes an automated approach using Multiple Object Tracking (MOT) technology to assess motor impulsivity. The aim was to develop a system for detecting and quantifying motor impulsivity in naturalistic, multi-person environments. By employing cutting-edge MOT algorithms, the solution tracks multiple individuals concurrently, enabling movement and interaction analyses. This methodology integrates MOT with behavioral models to identify motor impulsivity patterns such as abrupt trajectory changes or impulsive gesturing. Trained on real-world annotated datasets, the system ensures adaptability across settings. Our approach successfully distinguished impulsive movements from typical behavioral patterns, with an accuracy of 95.43%. This approach could revolutionize assessments by providing objective and quantitative measurements and facilitating enhanced diagnostics and personalized interventions. Extensive evaluations are required to assess real-time capabilities, robustness in occluded environments, and accurate impulsive pattern identification. These findings could enable broader clinical, research, and behavioral monitoring applications, advancing our understanding of the implications of motor impulsivity.

Downloads

Download data is not yet available.

References

E. K. Farran et al., “Is the Motor Impairment in Attention Deficit Hyperactivity Disorder (ADHD) a Co-Occurring Deficit or a Phenotypic Characteristic?,” Adv Neurodev Disord, vol. 4, no. 3, pp. 253–270, Sep. 2020, doi: 10.1007/s41252-020-00159-6.

O. Grimm et al., “Impulsivity and Venturesomeness in an Adult ADHD Sample: Relation to Personality, Comorbidity, and Polygenic Risk,” Front Psychiatry, vol. 11, Dec. 2020, doi: 10.3389/fpsyt.2020.557160.

E. K. Edmiston et al., “Assessing Relationships Among Impulsive Sensation Seeking, Reward Circuitry Activity, and Risk for Psychopathology: A Functional Magnetic Resonance Imaging Replication and Extension Study,” Biol Psychiatry Cogn Neurosci Neuroimaging, vol. 5, no. 7, pp. 660–668, Jul. 2020, doi: 10.1016/j.bpsc.2019.10.012.

X. Chen and S. Li, “Serial mediation of the relationship between impulsivity and suicidal ideation by depression and hopelessness in depressed patients,” BMC Public Health, vol. 23, no. 1, p. 1457, Jul. 2023, doi: 10.1186/s12889-023-16378-0.

J. Salles et al., “Indirect effect of impulsivity on suicide risk through self-esteem and depressive symptoms in a population with treatment-resistant depression: A FACE-DR study,” J Affect Disord, vol. 347, pp. 306–313, Feb. 2024, doi: 10.1016/j.jad.2023.11.063.

T. Veliki, Z. Uzarevic, and S. Dubovicki, “Self-Evaluated ADHD Symptoms as Risk Adaptation Factors in Elementary School Children,” Drustvena istrazivanja, vol. 28, no. 3, pp. 503–522, Oct. 2019, doi: 10.5559/di.28.3.07.

A. Bandyopadhyay et al., “Behavioural difficulties in early childhood and risk of adolescent injury,” Arch Dis Child, vol. 105, no. 3, pp. 282–287, Mar. 2020, doi: 10.1136/archdischild-2019-317271.

C. K. Conners, J. Pitkanen, and S. R. Rzepa, “Conners 3rd Edition (Conners 3; Conners 2008),” in Encyclopedia of Clinical Neuropsychology, New York, NY: Springer New York, 2011, pp. 675–678. doi: 10.1007/978-0-387-79948-3_1534.

G. J. DuPaul, T. J. Power, A. D. Anastopoulos, and R. C. Reid, “Adhd Rating Scale-IV: Checklists, Norms, and Clinical Interpretation,” 1998. [Online]. Available: https://api.semanticscholar.org/CorpusID:141673166.

G. J. DuPaul and G. Stoner, ADHD in the schools: Assessment and intervention strategies, 3rd ed. New York, NY, US: The Guilford Press, 2014.

G. Mirabella, C. Mancini, S. Pasifici, D. Guerrini, A. Terrinoni, "Enhanced reactive inhibition in adolescents with non-suicidal self-injury disorder," Dev Med Child Neurol, vol. 66, no.5, pp. 645-666, May 2024, doi: 10.1111/dmcn.15794.

J. Bertram et al., “Accuracy and repeatability of the Microsoft Azure Kinect for clinical measurement of motor function,” PLoS One, vol. 18, no. 1, p. e0279697, Jan. 2023, doi: 10.1371/journal.pone.0279697.

A. Jalal, M. A. K. Quaid, S. B. ud din Tahir, and K. Kim, “A Study of Accelerometer and Gyroscope Measurements in Physical Life-Log Activities Detection Systems,” Sensors, vol. 20, no. 22, p. 6670, Nov. 2020, doi: 10.3390/s20226670.

D. Kobsar et al., “Wearable Inertial Sensors for Gait Analysis in Adults with Osteoarthritis—A Scoping Review,” Sensors, vol. 20, no. 24, p. 7143, Dec. 2020, doi: 10.3390/s20247143.

Y. Zhang et al., “ByteTrack: Multi-object Tracking by Associating Every Detection Box,” 2022, pp. 1–21. doi: 10.1007/978-3-031-20047-2_1.

D. Zhao, G. Su, G. Cheng, P. Wang, W. Chen, and Y. Yang, “Research on real-time perception method of key targets in the comprehensive excavation working face of coal mine,” Meas Sci Technol, vol. 35, no. 1, p. 015410, Jan. 2024, doi: 10.1088/1361-6501/ad060e.

A. Graves and J. Schmidhuber, “Framewise phoneme classification with bidirectional LSTM and other neural network architectures,” Neural Networks, vol. 18, no. 5–6, pp. 602–610, Jul. 2005, doi: 10.1016/j.neunet.2005.06.042.

A. Dey, S. Biswas, and D.-N. Le, “Recognition of Wh-Question Sign Gestures in Video Streams using an Attention Driven C3D-BiLSTM Network,” Procedia Comput Sci, vol. 235, pp. 2920–2931, 2024, doi: 10.1016/j.procs.2024.04.276.

K. Nguyen-Trong, H. N. Vu, N. N. Trung, and C. Pham, “Gesture Recognition Using Wearable Sensors With Bi-Long Short-Term Memory Convolutional Neural Networks,” IEEE Sens J, vol. 21, no. 13, pp. 15065–15079, Jul. 2021, doi: 10.1109/JSEN.2021.3074642.

R. P. Singh and L. D. Singh, “Dyhand: dynamic hand gesture recognition using BiLSTM and soft attention methods,” Vis Comput, Mar. 2024, doi: 10.1007/s00371-024-03307-4.

J. Wu, P. Ren, B. Song, R. Zhang, C. Zhao, and X. Zhang, “Data glove-based gesture recognition using CNN-BiLSTM model with attention mechanism,” PLoS One, vol. 18, no. 11, p. e0294174, Nov. 2023, doi: 10.1371/journal.pone.0294174.

X. Wu and Q. Ji, “TBRNet: Two-Stream BiLSTM Residual Network for Video Action Recognition,” Algorithms, vol. 13, no. 7, p. 169, Jul. 2020, doi: 10.3390/a13070169.

A.-A. Liu, Z. Shao, Y. Wong, J. Li, Y.-T. Su, and M. Kankanhalli, “LSTM-based multi-label video event detection,” Multimed Tools Appl, vol. 78, no. 1, pp. 677–695, Jan. 2019, doi: 10.1007/s11042-017-5532-x.

X. Tong, X. Tan, and X. Sun, “Abnormal behavior detection based on GCN-BiLSTM,” in Third International Conference on Machine Learning and Computer Application (ICMLCA 2022), F. Zhou and S. Ba, Eds., SPIE, May 2023, p. 58. doi: 10.1117/12.2675168.

A. Soeleman, C. Supriyanto, D. P. Prabowo, and P. N. Andono, “Video Violence Detection Using LSTM and Transformer Networks Through Grid Search-Based Hyperparameters Optimization,” International Journal of Safety and Security Engineering, vol. 12, no. 05, pp. 615–622, Nov. 2022, doi: 10.18280/ijsse.120510.

F. Carrara, P. Elias, J. Sedmidubsky, and P. Zezula, “LSTM-based real-time action detection and prediction in human motion streams,” Multimed Tools Appl, vol. 78, no. 19, pp. 27309–27331, Oct. 2019, doi: 10.1007/s11042-019-07827-3.

W. Ullah, A. Ullah, I. U. Haq, K. Muhammad, M. Sajjad, and S. W. Baik, “CNN features with bi-directional LSTM for real-time anomaly detection in surveillance networks,” Multimed Tools Appl, vol. 80, no. 11, pp. 16979–16995, May 2021, doi: 10.1007/s11042-020-09406-3.

F.-C. Lin, H.-H. Ngo, C.-R. Dow, K.-H. Lam, and H. L. Le, “Student Behavior Recognition System for the Classroom Environment Based on Skeleton Pose Estimation and Person Detection,” Sensors, vol. 21, no. 16, p. 5314, Aug. 2021, doi: 10.3390/s21165314.

F. F. Dalimarta, Z. A. Hasibuan, P. N. Andono, Pujiono, and M. A. Soeleman, “Lower Body Detection and Tracking with AlphaPose and Kalman Filters,” in Proceedings - 2021 International Seminar on Application for Technology of Information and Communication: IT Opportunities and Creativities for Digital Innovation and Communication within Global Pandemic, iSemantic 2021, 2021. doi: 10.1109/iSemantic52711.2021.9573221.

R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V. Koltun, “Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer,” IEEE Trans Pattern Anal Mach Intell, vol. 44, no. 3, pp. 1623–1637, Mar. 2022, doi: 10.1109/TPAMI.2020.3019967.

T. Li, Z. Li, Y. Mu, and J. Su, “Pedestrian multi-object tracking based on YOLOv7 and BoT-SORT,” in Third International Conference on Computer Vision and Pattern Analysis (ICCPA 2023), L. Shen and G. Zhong, Eds., SPIE, Aug. 2023, p. 68. doi: 10.1117/12.2684256.

S. Nindam, S.-H. Na, and H. J. Lee, “MultiFusedNet: A Multi-Feature Fused Network of Pretrained Vision Models via Keyframes for Student Behavior Classification,” Applied Sciences, vol. 14, no. 1, p. 230, Dec. 2023, doi: 10.3390/app14010230.