Skip to content

A curated list of awesome work on machine learning for mental health applications. Includes topics broadly captured by affective computing. Facial expressions, speech analysis, emotion prediction, depression, interactions, psychiatry etc. etc.

Notifications You must be signed in to change notification settings

matthewvowels1/Awesome_ML_for_mental_health

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 

Repository files navigation

Awesome_ML_for_mental_health

A curated list (currently 252 papers) of awesome work on machine learning for mental health applications, as collected for my PhD. Includes topics broadly captured by affective computing. Facial expressions, speech analysis, emotion prediction, depression, interactions, psychiatry etc. etc. Papers are listed in reverse chronological order. If I've missed your paper, or a paper you want to add to the list, feel free to contribute or email me : ]

2020

Emotion based facial expression detection using machine learning. Aslam, Qadri, Shehzad et al. http://www.lifesciencesite.com/lsj/lsj170820/06_36691lsj170820_35_43.pdf

Jointly fine-tuning 'bert-like' self supervised models to improve multimodal speech emotion recognition. Siriwardhana, Reis, Weerasekera, Nanayakkara https://arxiv.org/pdf/2008.06682.pdf

Detecting deepfakes using emotional irregularities. Murray https://search.proquest.com/openview/d6645ec6d4f7216a83e3ac1906727ee8/1?pq-origsite=gscholar&cbl=18750&diss=y

Internet of emotional people: towards continual affective computing cross cultures via audiovisual signals. Han, Zhang, Pantic, Schuller https://www.sciencedirect.com/science/article/pii/S0167739X20302594

Generating synthetic aging trajectories with a weighted network model using cross-sectional data. Farrell, Mitnitski, Rockwood, Rutenberg https://www.biorxiv.org/content/10.1101/2020.02.14.949560v2.full.pdf

Looking at the body: automatic analysis of body gestures and self-adaptors in psychological distress . Lin, Orton, Li, Pavarini, Mahmoud https://arxiv.org/pdf/2007.15815.pdf

Toward emotion recognition from physiological signals in the wild: approaching the methodological issues in real-life data collection. Larradet, Niewiadomski, Barresi, Caldwell, Mattos https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7374761/

Adversarial graph representation adaptation for cross-domain facial expression recognition. Xie, Chen, Pu, Wu, Lin https://arxiv.org/pdf/2008.00859.pdf

Deep learning-based classification of posttraumatic stress disorder and depression following trauma utilizing visual and auditory markers of arousal and mood. Schultebraucks, Yadav, Shalev, Bonanno https://www.cambridge.org/core/journals/psychological-medicine/article/deep-learningbased-classification-of-posttraumatic-stress-disorder-and-depression-following-trauma-utilizing-visual-and-auditory-markers-of-arousal-and-mood/733197598EA30BC8379D151173AEFF8F

Semi-supervised learning for facial expression-based emotion recognition in the continuous domain. Choi, Song https://link.springer.com/article/10.1007/s11042-020-09412-5

Deep learning-based emotion recognition from real-time videos. Zhou, Cheng, Lei, Benes, Adamo https://link.springer.com/chapter/10.1007/978-3-030-49062-1_22

2019

Fully supervised speaker diarization. Zhang, Wang, Zhu, Paisley, Wang https://arxiv.org/pdf/1810.04719.pdf

Deep learning for human affect recognition: insights and new development. Rouast, Adam, Chiong https://arxiv.org/pdf/1901.02884

Multi-modal analysis for the automatic evaluation of epilepsy. Aristizabal https://eprints.qut.edu.au/132537/1/David_Ahmedt%20Aristizabal_Thesis.pdf

Facial expression recognition: a survey. Huang, Chen, Wang http://scholar.google.co.uk/scholar_url?url=https://www.mdpi.com/2073-8994/11/10/1189/pdf&hl=en&sa=X&d=12696794651620539335&scisig=AAGBfm3y7ccZYuDWNtQJzacogJijOTjUug&nossl=1&oi=scholaralrt

Emotion recognition in low-resource settings: an evaluation of automatic feature selection methods. Haider https://arxiv.org/pdf/1908.10623.pdf

An interaction-aware attention network for speech emotion recognition in spoken dialogs. Yeh, Lin, Lee https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8683293

ANA at SemEval-2019 Task 3: contextual emotion detection in conversations through hierarchical LSTMs and BERT. Huang, Trabelsi, Zaiane https://arxiv.org/pdf/1904.00132.pdf

NELEC at SemEval-2109 Task 3: Think twice before going deep. Agrawal, Suri https://arxiv.org/pdf/1904.03223.pdf

Emotional expressions reconsidered: challenges to inferring emotion from human facial movements. Barrett, Adolphs, Marsella, Martinez, Pollak https://journals.sagepub.com/eprint/SAUES8UM69EN8TSMUGF9/full

Emotion schemas are embedded in the human visual system. Kragel, Reddan, LeBar, Wager https://advances.sciencemag.org/content/5/7/eaaw4358

The ambiguous world of emotion representation. Sethu, Provost, Epps, Busso, Cummins, Narayanan https://arxiv.org/pdf/1909.00360.pdf

DialogueGCN: A graph convolutional neural network for emotion recognition in conversation. Ghosal, Majumder, Poria, Chhaya, Gelbukh https://arxiv.org/pdf/1908.11540.pdf

Multi-grained spatio-temporal modeling for lip-reading. Wang, https://arxiv.org/pdf/1908.11618.pdf

Improving human pose estimation with self-attention generative adversarial networks. Wang, Cao, Wang, Liu, Zhu https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8808903

Multimodal deep learning for mental disorders prediction from audio speech samples. Naderi, Soleimani, Rempel, Matwin, Uher https://arxiv.org/pdf/1909.01067.pdf

TIM: a tool for gaining insights into psychotherapy. Cummins, Ewbank, Martin, Tablan, Catarino, Blackwell https://www.researchgate.net/publication/333071745_TIM_A_Tool_for_Gaining_Insights_into_Psychotherapy

Quantifying the association between psychotherapy content and clinical outcomes using deep learning. Ewbank, Cummins, Tablan https://jamanetwork.com/journals/jamapsychiatry/article-abstract/2748757

DialogueRNN: An attentive RNN for emotion detection in conversations. Majumder, Poria, Hazarika, Mihalcea, Gelbukh, Cambria https://arxiv.org/pdf/1811.00405.pdf

Towards multimodal sarcasm detection (an obviously perfect paper). Castro, Hazarika, Perez-Rosas, Zimmermann, Mihalcea, Poria https://arxiv.org/pdf/1906.01815.pdf

Recent trends in deep learning based personality detection. Mehta, Majumder, Gelbukh, Cambria https://arxiv.org/pdf/1908.03628.pdf

Multi-task learning for multi-modal emotion recognition and sentiment analysis. Akhtar, Chauhan, Ghosal, Poria, Ekbal, Bhattacharyya https://arxiv.org/pdf/1905.05812.pdf

All-in-one: emotion, sentiment and intensity prediction using a multi-task ensemble framework. Akhtar, Ghosal, Ekbal, Bhattacharyya, Kurohashi https://ieeexplore.ieee.org/abstract/document/8756111

Emotion recognition in conversation: research challenges, datasets, and recent advances. Poria, Majumder, Mihalcea, Hovy https://arxiv.org/pdf/1905.02947.pdf

Computational intelligence for affective computing and sentiment analysis. Cambria, Poria, Hussain, Liu https://ieeexplore.ieee.org/document/8686323

Variational fusion for multimodal sentiment analysis. Majumder, Poria, Krishnamurthy, Chhaya, Mihalcea, Gelbukh https://arxiv.org/pdf/1908.06008.pdf

AVEC 2019 Workshop and challenge: State-of-mind, detecting depression with AI, and cross-cultural affect recognition. Ringeval, Schuller, Valstar, Cummins, Cowie, Tavabi, Schmitt, Alisamir, Amiriparian, Messner, Song, Liu, Zhao, Mallol-Ragolta, Ren, Soleymani, Pantic https://arxiv.org/pdf/1907.11510.pdf

Virtual human questionnaire for analysis of depression, anxiety and personality. Jaiswai, Valstar, Kusumam, Greenhalgh https://dl.acm.org/citation.cfm?id=3329469

Modeling mental stress using a deep learning framework. Masood, Alghamdi, https://ieeexplore.ieee.org/document/8718667

ARBEE: Towards automated recognition of bodily expression of emotion in the wild. Luo, Ye, Adams, Li, Newman, Wang https://arxiv.org/pdf/1808.09568.pdf

Alone vesus in-a-group: a multi-modal framework for automatic affect recognition. Mou, Gunes, Patras https://www.repository.cam.ac.uk/handle/1810/290132

An investigation of deep learning systems for suicide risk assessment. Morales, Belitz, Chernova, Dey, Theisen https://www.aclweb.org/anthology/W19-3023

Deep neural networks in psychiatry. Durstewitz, Koppe, Meyer-Lindenberg https://www.nature.com/articles/s41380-019-0365-9.pdf

Machine learning in mental health: a systematic scoping review of methods and applications. Shatte, Hutchinson, Teague https://osf.io/hjrw8/download

Predicting personalized process-outcome associations in psychotherapy using machine learning approaches - a demonstration. Rubel, Zilcha-Mano, Giesemann, Prinz, Lutz https://www.tandfonline.com/doi/full/10.1080/10503307.2019.1597994?af=R

A review of the role of artificial intelligence (AI) in psychotherapy and its feasibility in real life situations. Aich, Chakraborty, Kim https://www.researchgate.net/publication/335490459_A_Review_on_the_Role_of_Artificial_Intelligence_AI_in_Psychotherapy_its_Feasibility_in_Real_Life_Situations

Regret induces rapid learning from experience-based decisions: a model-based facial expression analysis approach. Haines, Rass, Shin, Busemeyer, Brown, O'Donnell, Ahn https://www.biorxiv.org/content/biorxiv/early/2019/02/25/560011.full.pdf

Using computer-vision and machine learning to automate facial coding of positive and negative affect intensity. Haines, Southward, Cheavens, Beauchaine, Ahn https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0211735

Machine learning for the recognition of emotion in the speech of couples in psychotherapy using the Stanford Suppes Brain Lab psychotherapy dataset. Crangle, Wang, Perreau-Guimaraes, Nguyen, Nguyen, Suppes https://arxiv.org/pdf/1901.04110.pdf

Primed for psychiatry: The role of artificial intelligence and machine learning in the optimization of depression treatment. Tan, Rollins, Israel, Benrimoh http://www.utmj.org/index.php/UTMJ/article/view/1153/1179

Emotion recognition for self-aid in addiction treatment, psychotherapy, and nonviolent communication. Franzoni, Milani https://link.springer.com/chapter/10.1007/978-3-030-24296-1_32

Psychotherapy in the era of artificial intelligence: Therapist Panoptes. Alexios https://ejournals.epublishing.ekt.gr/index.php/homvir/article/view/20197/17673

Valence and arousal estimation in-the-wild with Tensor methods. Mitenkova, Kossaifi, Panagakis, Pantic https://ieeexplore.ieee.org/document/8756619

Personalized estimation of engagement from videos using active learning with deep reinforcement learning. Rudovic, Park, Busche, Schuller, Breazeal, Picard https://dam-prod.media.mit.edu/x/2019/04/16/AMFG_2019_CVPR_Personalized_RL.pdf

Exploring deep spectrum representations via attention-based recurrent and convolutional neural networks for speech emotion recognition. Zhao, Bao, Zhao, Zhang, Cummins, Ren, Schuller https://ieeexplore.ieee.org/document/8762126

EmoBed: strengthening monomodal emotion recognition via training with crossmodal emotion embeddings. Han, Zhang, Ren, Schuller https://ieeexplore.ieee.org/document/8762142

Inferring dynamic representations of facial actions from a still image. Song, Sanchez-Lozano, Shen, Johnston, Valstar https://arxiv.org/pdf/1904.02382.pdf

Speech emotion classification using attention-based LSTM. Xie, Liang, Liang, Huang, Zou, Schuller https://ieeexplore.ieee.org/document/8752054

Multi-modal active learning from human data: a deep reinforcement learning approach. Rudovic, Zhang, Schuller, Picard https://arxiv.org/pdf/1906.03098.pdf

I know how you feel now, and here's why! Demystifying time-continuous high resolution text-based affect predictions in the wild. Pandi, Schmitt, Cummins, Schuller https://ieeexplore.ieee.org/document/8787535

Deep affect prediction in-the-wild: Aff-Wild database and challenge, deep architectures, and beyond. Kollias, Tzirakis, Nicolaou, Papaioannou, Zhao, Schuller, Kotsia, Zafeiriou https://arxiv.org/pdf/1804.10938.pdf

Performance analysis of unimodal and multimodal models in valence-based empathy recognition. Mallal-Ragolta, Schmitt, Baird, Cummins, Schuller https://ieeexplore.ieee.org/abstract/document/8756517

Design feasibility of an automated, machine-learning based feedback system for motivational interviewing. Imel, Pace, Soma, Tanana, Hirsch, Gibson, Georgiou, Narayanan, Atkins https://www.ncbi.nlm.nih.gov/pubmed/30958018

Multi-level attention network using text, audio and video for depression prediction. Ray, Kumar, Reddy, Mukherrjee, Garg https://arxiv.org/pdf/1909.01417.pdf

Modeling both context- and speaker-sensitive dependence for emotion detection in multi-speaker conversations. Zhang, Wu, Sun, Li, Zhu, Zhou https://www.ijcai.org/proceedings/2019/0752.pdf

Personalized expression synthesis using a hybrid geometric-machine learning method. Zaied, Soladie, Richard https://link.springer.com/chapter/10.1007/978-3-030-30645-8_3

Responding to uncertainty in emotion recognition. Schuller https://www.emerald.com/insight/content/doi/10.1108/JICES-07-2019-0080/full/html

Segment based emotion recognition using combined reduced features. Mohanty, Palo https://link.springer.com/article/10.1007/s10772-019-09628-3

Modelling sample informativeness for deep affective computing. Rizos, Schuller https://ieeexplore.ieee.org/abstract/document/8683729

Context modelling using hierarchical attention networks for sentiment and self-assessed emotion detection in spoken narratives. Stappen, Cummins, Mebner, Baumeister, Dineley, Schuller https://ieeexplore.ieee.org/document/8683801

Attention-augmented end-to-end multi-task learning for emotion prediction from speech. Zhang, Wu, Schuller https://arxiv.org/pdf/1903.12424.pdf

SEWA DB: A rich database for audio-visual emotion and sentiment research in the wild. Kossaifi, Walecki, Panagakis, Shen, Schmitt, Ringeval, Han, Pandit, Schuller, Star, Hajiyev, Pantic https://arxiv.org/pdf/1901.02839.pdf

Emotion recognition from physiolocial signal analysis: A review. Maria, Matthias, Sten https://www.sciencedirect.com/science/article/pii/S157106611930009X

Improving the prediction of therapist behaviors in addiction counseling by exploiting class confusions. Chen, Singla, Gibson, Can, Imel, Atkins, Georgiou, Narayanan https://ieeexplore.ieee.org/document/8682885

Toward robust interpretable human movement pattern analysis in a workplace setting Booth, Feng, Jangalwa, Narayanan https://ieeexplore.ieee.org/document/8683730

Modeling interpersonal linguistic coordination in conversations using word mover's distance. Nasic, Chakravarthula, Baucom, Atkins, Georgiou, Narayanan https://arxiv.org/pdf/1904.06002.pdf

Total variability layer in deep neural network embeddings for speaker verification. Travadi, Narayanan https://ieeexplore.ieee.org/document/8686172

Multimodal representation learning using deep multiset canonical correlation analysis. Somandepalli, Kumar, Travadi, Narayanan https://arxiv.org/pdf/1904.01775.pdf

Predicting behavior in cancer-afflicted patient and spouse interactions using speech and language. Chakravarthula, Li, Tseng, Reblin, Georgiou https://arxiv.org/pdf/1908.00908.pdf

The impact of extraneous variables on the performance of recurrent neural network models in clinical tasks. Laksana, Aczon, Ho, Carlin, Ledbetter, Wetzel, https://arxiv.org/pdf/1904.01125.pdf

Psychotherapy and artificial intelligence: a proposal for alignment. de Mello, de Souza https://www.frontiersin.org/articles/10.3389/fpsyg.2019.00263/full

How to prepare prospective psychiatrists in the era of artificial intelligence. Kim, Jones, D'Angelo https://link.springer.com/article/10.1007/s40596-019-01025-x

Artificial intelligence and the future of psychiatry: insights from a global physicial survey. Doraiswamy, Blease, Bodner https://arxiv.org/pdf/1907.12386.pdf

Your robot therapist will see you now: ethical implications of embodied artificial intelligence in psychiatry, psychology and psychotherapy. Fiske, Henningsen, Buyx https://www.jmir.org/2019/5/e13216/

Computer-assisted psychological assessment and psychotherapy for collegians. Heesacker, Perez, Quinn, Benton https://onlinelibrary.wiley.com/doi/abs/10.1002/jclp.22854

Using artificial intelligence to assess clinicians' communication skills. Ryan, Luz, Albert, Vogel, Normand, Elwyn https://www.bmj.com/content/364/bmj.l161.long

Dynamic emotion modelling and anomaly detection in conversation based on emotional transition tensor. Sun, Zhang, Li https://www.sciencedirect.com/science/article/pii/S156625351730667X

Comparative studies for the human facial expressions recognition techniques. Gaur, Dixit, Hasan, Wani, Kazi, Rizvi https://www.researchgate.net/profile/Sayed_Hasan2/publication/335516989_Comparative_Studies_for_the_Human_Facial_Expressions_Recognition_Techniques_of_the_Creative_Commons_Attribution_License_CC_BY_40/links/5d6a0a69299bf1808d59ca58/Comparative-Studies-for-the-Human-Facial-Expressions-Recognition-Techniques-of-the-Creative-Commons-Attribution-License-CC-BY-40.pdf

A baseline approach for early detection of signs of anorexia and self-harm in Reddit posts. Naderi, Gobeil, Teodoro, Pasche, Ruch http://www.dei.unipd.it/~ferro/CLEF-WN-Drafts/CLEF2019/paper_111.pdf

Feature fusion of face and body for engagemenent intensity detection. Li, Hung https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8803488

Multimodal embeddings from language models. Tseng, Georgiou, Narayanan https://arxiv.org/pdf/1909.04302.pdf

Twofold-multimodal pain recognition with the X-ITE pain database. Werner, Al-Hamadi, Gruss, Walter https://www.researchgate.net/profile/Philipp_Werner/publication/335723494_Twofold-Multimodal_Pain_Recognition_with_the_X-ITE_Pain_Database/links/5d77a34d92851cacdb2e36b5/Twofold-Multimodal-Pain-Recognition-with-the-X-ITE-Pain-Database.pdf

Fixed points in a changing world. Robinson https://www.cl.cam.ac.uk/~pr10/publications/heai20.pdf

Ordinal triplet loss: investigating sleepiness detection from speech. Wu, Rallabandi, Black, Nyberg https://www.isca-speech.org/archive/Interspeech_2019/pdfs/2278.pdf

Continuous emotion recognition in speech - do we need recurrence? Schmitt, Cummins, Schuller https://www.isca-speech.org/archive/Interspeech_2019/pdfs/2710.pdf

Bag-of-acoustic-words for mental health assessment: a deep autoencoding approach. Du, Morency, Cohn, Black https://www.isca-speech.org/archive/Interspeech_2019/pdfs/3059.pdf

Automatic depression level detection via lp-norm pooling. Niu, Tao, Liu, Fan https://www.isca-speech.org/archive/Interspeech_2019/pdfs/1617.pdf

Optimizing speech-input length for speaker-independent depression classification. Rutowski, Harati, Lu, Shriberg https://www.isca-speech.org/archive/Interspeech_2019/pdfs/3095.pdf

A hierarchical attention network-based approach for depression detection from transcribed clinical interviews. Mallol-Ragolta, Zhao, Stappen, Cummins, Schuller https://www.isca-speech.org/archive/Interspeech_2019/pdfs/2036.pdf

Depression state assessment: application for detection of depression by speech. Kiss, Sztaho, Vicsi https://www.isca-speech.org/archive/Interspeech_2019/pdfs/8004.pdf

Using speech to predict sequentially measured cortisol levels during a trier social stress test. Baird, Amiriparian, Cummins, Sturmbauer, Janson, Messner, Baumeister, Rohleder, Schuller https://www.isca-speech.org/archive/Interspeech_2019/pdfs/1352.pdf

Robust speech emotion recognition under different encoding conditions. Oates, Triantafyllopoulos, Steiner, Schuller https://www.isca-speech.org/archive/Interspeech_2019/pdfs/1658.pdf

Into the wild: transitioning from recognizing mood in clinical interactions to personal conversations for individuals with bipolar disorder. Matton, McInnis, Provost https://www.isca-speech.org/archive/Interspeech_2019/pdfs/2698.pdf

MFCC-based recurrent neural network for automatic clinical depression recognition and assessment from speech. Rejaibi, Komaty, Meriaudeau, Agrebi, Othmani https://arxiv.org/pdf/1909.07208.pdf

Improved end-to-end speech emotion recognition using self attention mechanism and multitask learning. Li, Zhao, Kawahara https://www.isca-speech.org/archive/Interspeech_2019/pdfs/2594.pdf

Maintaining prediction robustness of the multimodal emotion recognition systems. Schuster, Tu https://www6.in.tum.de/fileadmin/w00bxu/www/Teaching/SS19/Presentation_AMER_H.pdf

Speech emotion recognition in dyadic dialogues with attentive interaction modeling. zhao, Chen, Liang, Jin https://www.isca-speech.org/archive/Interspeech_2019/pdfs/2103.pdf

Fusion techniques for utterance-level emotion recognition combining speech and transcripts. Sebastian, Pierucci https://www.isca-speech.org/archive/Interspeech_2019/pdfs/3201.pdf

Towards automatic face-to-face translation. Prajwal, Mukhodpadhyay, Philip, Jha, Namboodiri, Jawahar http://cdn.iiit.ac.in/cdn/cvit.iiit.ac.in/images/ConferencePapers/2019/ACM_Final_Face2Face.pdf

Heterogeneity in psychiatric diagnostic classification Allsopp, Read, Corcoran, Kinderman https://www.ncbi.nlm.nih.gov/pubmed/31279246

Failing grade: 89% of introduction to psychology textbooks that define or explain statistical significance do so incorrectly. Cassidy, Dimova, Giguere, Spence, Stanley https://journals.sagepub.com/doi/abs/10.1177/2515245919858072

Hidden invalidity among fifteen commonly used measures in social and personality psychology Hussey, Hughes https://psyarxiv.com/7rbfp/

The verbal and non verbal signals of depression - combining acoustics, text and visuals for estimating depression level. Qureshi, Hasanuzzaman, Saha, Dias https://arxiv.org/pdf/1904.07656.pdf

Uncertainty in emotion recognition. Landowska https://www.emerald.com/insight/content/doi/10.1108/JICES-03-2019-0034/full/pdf?title=uncertainty-in-emotion-recognition

No replication, no trust? How low replicability influences trust in psychology. Wingen, Berkessel https://osf.io/4ukq5/

Video emotion analysis via deep temporal convolution. Bao, Xu http://zhipengbao.cn/files/multimodal.pdf

Towards automatic detection of misinformation in online medical videos. Hou, Perez-Rosas, Loeb, Mihalcea https://arxiv.org/pdf/1909.01543.pdf

When to intervene: detecting abnormal mood using everyday smartphone conversations. Gideon, Matton, Anderau, McInnis, Provost https://arxiv.org/pdf/1909.11248.pdf

Detecting depressed users in online forums. Shrestha, Spezzano https://asonamdata.com/ASONAM2019_Proceedings/pdf/papers/161_0945_038.pdf

Visceral versus verbal: can we see depression? Zhu, Gedeon, Caldwell, Jones http://www.uni-obuda.hu/journal/Zhu_Gedeon_Caldwell_Jones_96.pdf

Occluded facial expression recognition enhanced through privileged information. Pan, Wang, Xia https://dl.acm.org/citation.cfm?id=3351049

Human-like decision making: document-level aspect sentiment classification via hierarchical reinforcement learning. Wang, Sun, Li, Wang, Si, Zhang, Liu, Zhou https://arxiv.org/pdf/1910.09260.pdf

Facial expression recognition via relation-based conditional generative adversarial network. Lee, Choi, Song https://dl.acm.org/citation.cfm?id=3353753

Multimodal learning for identifying opportunities for empathetic responses. Tavabi, Stefanov, Gilani, Traum, Soleymani https://dl.acm.org/citation.cfm?id=3353750

CorrFeat: correlation-based feature extraction algorithm using skin conductance and pupil diameter for emotion recognition. Zhang, Ali, Wang, Zhu, Cesar http://delivery.acm.org/10.1145/3360000/3353716/p404-zhang.html?ip=86.162.136.199&id=3353716&acc=OPEN&key=4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E6D218144511F3437&__acm__=1572891885_c5baeb0b7793b7ec45db7e823775e882

Continuous emotion recognition in videos by fusing facial expression, head pose and eye gaze. Wu, Du, Li, Huang, Wang https://dl.acm.org/citation.cfm?id=3353739

Dynamic facial models for video-based dimensional affect estimation. Song, Sanchez-Lozano, Tellamekala, Shen, Johnston, Valstar http://openaccess.thecvf.com/content_ICCVW_2019/papers/CVPM/Song_Dynamic_Facial_Models_for_Video-Based_Dimensional_Affect_Estimation_ICCVW_2019_paper.pdf

Suicidal ideation detection: a review of machine learning methods and applications. Ji, Pan, Li, Cambria, Long, Huang https://arxiv.org/pdf/1910.12611.pdf

Can a humanoid robot be part of the organizational workforce? A user study leveraging sentiment analysis. Mishra, Ramanathan, Satapathy, Cambria, Magnenat-Thalmann https://arxiv.org/abs/1905.08937

Automatic detection of students' affective states in classroom environment using hybrid convolutional neural networks. Ashwin, Guddeti https://link.springer.com/article/10.1007/s10639-019-10004-6

Dynamic facial expression recognition based on deep learning. Deng, Wang, Yuan https://ieeexplore.ieee.org/document/8845493

Multi-modal correlated network for emotion recognition in speech. Ren, Nie, Liu, Su https://www.sciencedirect.com/science/article/pii/S2468502X19300488

Expression, affect, action unit recognition: Aff-Wild2, multi-task learning and ArcFace. Kollias, Zafeiriou https://ibug.doc.ic.ac.uk/media/uploads/documents/0399-paper.pdf

Aff-wild database and affwildnet. Liu, Kollias https://arxiv.org/pdf/1910.05318.pdf

Acoustic differences between healthy and depressed people: a cross-situation study. Wang, Zhang, Liu, Pan, Hu, Zhu https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-019-2300-7

Emotion recognition in conversations with transfer learning from generative conversation modeling. Hazarika, Poria, Zimmermann, Mihalcea https://arxiv.org/pdf/1910.04980.pdf

Facial expression recognition using human to animated-character expression translation. Ali, Isler, Hughes https://arxiv.org/pdf/1910.05595.pdf

Linking emotions to behaviors through deep transfer learning. Li, Baucom, Georgiou https://arxiv.org/pdf/1910.03641.pdf

To react or not to react: end-to-end visual pose forecasting for personalized avatar during dyadic conversations. Ahuja, Ma, Morency, Sheikh https://arxiv.org/abs/1910.02181

Your body reveals your impressions about others: a study on multimodal impression detection. Wang, Pun, Chanel https://archive-ouverte.unige.ch/unige:123960

Detecting deception in political debates using acoustic and textual features. Kopev, Ali, Koychev, Nakov https://arxiv.org/pdf/1910.01990.pdf

Objective human affective vocal expression detection and automatic classification with stochastic models and learning systems .Vieira, Coelho, de Assis https://arxiv.org/pdf/1910.01967.pdf

Lexical features are more vulnerable, syntactic features have more predictive power. Novikova, Balagopalan, Shkaruta, Rudzicz https://arxiv.org/pdf/1910.00065.pdf

Facial expression recognition using disentangled adversarial learning. Ali, Hughes https://arxiv.org/pdf/1909.13135.pdf

Multimodal big data affective analytics: a comprehensive survey using text, audio, visual and physiological signals. Shoumy, Ang, Sng, Rahaman, Zia https://www.sciencedirect.com/science/article/pii/S1084804519303078

Spatio-temporal encoder-decoder fully convolutional network for video-based dimensional emotion recognition. Du, Wu, Huang, Li, Wang https://ieeexplore.ieee.org/document/8827932/

Exploiting multi-CNN features in CNN-RNN based dimensional emotion recognition on the OMG in-the-wild dataset. Kollias, Zafeiriou https://arxiv.org/pdf/1910.01417.pdf

2018

Iterpretable deep neural networks for facial expression and dimensional emotion recognition in-the-wild. Richer, Kollias https://arxiv.org/abs/1910.05877

Multimodal sentiment analysis: addressing key issues and setting up baselines. Poria, Majumder, Hazarika, Cambria, Hussain, Gelbukh https://arxiv.org/pdf/1803.07427.pdf

Deep facial expression recognition: a survey. Li, Deng https://arxiv.org/pdf/1804.08348.pdf

Survey on emotional body gesture recognition. Noroozi, Corneanu, Kaminska, Sapinski, Escalera, Anbarjafari https://arxiv.org/pdf/1801.07481.pdf

Emotion-preserving representation learning via generative adversarial network for multi-view facial expression recognition. Lai, Lai https://ieeexplore.ieee.org/document/8373839

Who is really talking? A visual-based speaker diarization strategy. Marin-Reyes, Lorenzo-Navarro, Santana, Nielsen https://www.researchgate.net/publication/322708912_Who_is_Really_Talking_A_Visual-Based_Speaker_Diarization_Strategy

ICON: Interactive conversational memory network for multimodal emotion detection. Hazarika, Poria, Mihalcea, Cambria, Zimmermann https://www.aclweb.org/anthology/D18-1280

Facial emotion recognition: a survey and real-world user experiences in mixed reality. Mehta, Siddiqui, Javaid https://www.ncbi.nlm.nih.gov/pubmed/29389845

MELD: a multimodal multi-party dataset for emotion recognition in conversations. Poria, Hazarika, Majumder, Maik, Cambria, Mihalcea https://arxiv.org/pdf/1810.02508.pdf

Guest editorial:The computational face. Escalera, Baro, Guyon, Escalante, Tzimiropoulos, Valstar, Panti, Cohn, Kanade https://ieeexplore.ieee.org/document/8479325

Multimodal assessment of depression from behavioral signals. Cohn, Cummins, Epps, Goecke, Joshi, Scherer http://www.jeffcohn.net/wp-content/uploads/2019/03/VII3-MultimodalAssessmentDepression_Final_V05.pdf

A linguistically-informed fusion approach for multimodal depression detection. Morales, Scherer, Levita https://pdfs.semanticscholar.org/bc7d/8ba1a3b7c328dc707defca90800258ff6eb1.pdf

Lexical and acoustic deep learning model for personality recognition. An, Levitan https://www.isca-speech.org/archive/Interspeech_2018/pdfs/2263.pdf

Deep personality recognition for deception detection. An, Levitan, Hirschberg, Levitan http://www.cs.columbia.edu/speech/PaperFiles/2018/an_is18.pdf

Multimodal depression detection: an investigation of features and fusion techniques for automated systems. Morales https://pdfs.semanticscholar.org/48b0/ac9dbbb8d45add15672dcf033f8e3b388090.pdf

Deep learning for language understanding of mental health concepts derived from cognitive behavioural therapy. Rojas-Barahona, Tseng, Dai, Mansfield, Ramadan, Ultes, Crawford, Gasic https://arxiv.org/pdf/1809.00640.pdf

Machine learning approaches for clinical psychology and psychiatry. Dwyer, Falkai, Koutsouleris https://www.annualreviews.org/doi/abs/10.1146/annurev-clinpsy-032816-045037

Online attention for interpretable conflict estimation in political debates. Vereecken, Petridis, Panagakis, Pantic https://ieeexplore.ieee.org/document/8373855

Multimodal framework for analyzing the affect of a group of people. Huang, Dhall, Goecke, Pietikainen, Zhao https://ieeexplore.ieee.org/document/8323249

Towards reading hidden emotions: a comparative study of spontaneous micro-expression spotting and recognition methods. Li, Hong, Moilanen, Huang, Pfister, Zhao, Pietikainen https://ieeexplore.ieee.org/document/7851001

Emotion detection from speech and text. de Velasco, Justo, Anton, Carrilero, Torres https://addi.ehu.es/bitstream/handle/10810/35128/IberS18_P1-11_de-Velasco.pdf?sequence=1

Adversarial training in affective computing and sentiment analysis: recent advances and perspectives. Han, Zhang, Cummins, Schuller https://arxiv.org/pdf/1809.08927.pdf

Emotional expression in psychiactric conditions: new technology for clinicians. Grabowski, Rynkiewicz, Lassalle, Baron-Cohen, Schuller, Cummins, Baird, Podgorska-Bednarz, Pieniazek, Lucka http://docs.autismresearchcentre.com/papers/2018_Grabowski_Emotional_expression_psychiatric_conditions.pdf

The age of artificial emotional intelligence. Schuller, Schuller https://ieeexplore.ieee.org/document/8481266

Deep multimodal fusion: a hybrid approach. Amer, Shields, Siddiquie, Tamrakar, Divakaran, Chai https://link.springer.com/content/pdf/10.1007%2Fs11263-017-0997-7.pdf

Emotion recognition in speech with latent discriminative representations learning. Han, Zhang, Keren, Schuller https://www.ingentaconnect.com/content/dav/aaua/2018/00000104/00000005/art00005;jsessionid=1chh3irnqo2lk.x-ic-live-02#

MEC 2017: Multimodal emotion recognition challenge. Li, Tao, Schuller, Shan, Jiang, Jia https://ieeexplore.ieee.org/document/8470342

Speech emotion recognition: two decades in a nutshell, benchmarks, and ongoing trends. Schuller https://dl.acm.org/citation.cfm?doid=3210350.3129340

Tracking authentic and in-the-wild emotions using speech. Pandit, Cummins, Schmitt, Hantke https://ieeexplore.ieee.org/document/8470340

Multi-label multi-task deep learning for behavioral coding. Gibson, Atkins, Creed, Imel, Georgiou, Naranayan https://arxiv.org/pdf/1810.12349.pdf

The language of interpersonal interaction: an interdisciplinary approach to assessing and processing vocal and speech data. Weusthoff, Gaut, Steyvers, Atkins, Hahlweg, Hogan, Zimmermann, Fischer, Baucom, Georgiou, Narayanan, Baucom https://ejcop.psychopen.eu/article/view/82/pdf

Normalization before shaking towrad learning symmetrically distributed representation without margin in speech emotion recognition. Huang, Narayanan https://arxiv.org/pdf/1808.00876.pdf

Towards and unsupervised entrainment distance in conversational speech using deep neural network. Nasir, Baucom, Narayanan, Georgiou https://arxiv.org/pdf/1804.08782.pdf

Modeling interpersonal influence of verbal behavior in couples therapy dyadic interactions. Chakravarthula, Baucom, Georgiou https://arxiv.org/pdf/1805.09436.pdf

A novel method for human bias correction of continuous-time annotions. Booth, Mundnich, Narayanan https://sail.usc.edu/publications/files/08461645.pdf

Using prosodic and lexical information for learning utterance-level behaviors in psychotherapy. Singla, Chen, Flemotomos, Gibson, Can, Atkins, Narayanan https://www.isca-speech.org/archive/Interspeech_2018/pdfs/2551.pdf

Everyday couples' communication research: overcoming methodological barriers with technology. Reblin, Heyman, Ellington, Baucom, Georgiou, Vadaparampil https://www.sciencedirect.com/science/article/pii/S0738399117305980

An interdisciplinary approach to assessing and processing vocal and speech data. Weusthoff, Gaut, Steyvers, Atkins, Hahlweg, Hogan, Zimmermann, Fischer, Baucom, Georgiou, Narayanan, Baucom https://ejcop.psychopen.eu/article/view/82/html

Characterizing types of convolution in deep convolutional recurrent neural networks for robust speech emotion recognition. Huang, Narayanan https://arxiv.org/pdf/1706.02901.pdf

Medicine and the rise of the robots: a qualitative review of recent advances of artificial intelligence in health. Loh https://bmjleader.bmj.com/content/2/2/59

challenges and applications in multimodal machine learning. Baltrusaitis, Ahuja, Morency https://dl.acm.org/citation.cfm?id=3107993

Conversation memory network for emotion recognition in dyadic dialogue videos. Hazarika, Poria, Zadeh, Cambria, Morency, Zimmermann https://www.aclweb.org/anthology/N18-1193

Towards visual behavior markers of suicidal ideation. Eigbe, Baltrusaitis, Morency, Pestian https://ieeexplore.ieee.org/abstract/document/8373878

Multi-attention recurrent network for human communication comprehension. Zadeh, Liang, Poria, Vij, Cambria, Morency https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/viewFile/17390/16123

From Eliza to XiaoIce: Challenges and opportunities with social chatbots. Shum, He, Li https://arxiv.org/pdf/1801.01957.pdf

Modeling multi-turn conversation with deep utterance aggregation. Zhang, Li, Zhu, Zhao, Liu https://arxiv.org/pdf/1806.09102.pdf . Prediction and localization of student engagement in the wild. Kaur, Mustafa, Mehta, Dhall https://arxiv.org/pdf/1804.00858.pdf

Emotion recognition in speech using cross-modal transfer in the wild. Albanie, Nagrani, Vedaldi, Zisserman https://arxiv.org/pdf/1808.05561.pdf

2017

From hard to soft: towards more human-like emotion recognition by modelling the perception uncertainty. Han, Zhang, Schmitt, Pantic, Schuller https://www.researchgate.net/publication/320542701_From_Hard_to_Soft_Towards_more_Human-like_Emotion_Recognition_by_Modelling_the_Perception_Uncertainty

Learning visual emotion distributions via multi-modal features fusion. Zhao, Ding, Gao, Han http://gaoyue.org/paper/Learning%20Visual%20Emotion%20Distributions%20via%20Multi-Modal%20Features%20Fusion,%20ACM%20Conference%20on%20Multimedia.pdf

Multimodal machine learning: a survey and taxonomy. Baltrusaitis, Ahuja, Morency. https://arxiv.org/pdf/1705.09406.pdf

Multi-level multiple attention for contextual multimodal sentiment analysis. Poria, Cambria, Hazarika, Mazumder, Zadeh, Morency https://ieeexplore.ieee.org/document/8215597/

FERA 2017 - Addressing head pose in the third facial expression recognition and analysis challenge. Valstar, Sanchez-Lozano, Cohn, Jeni, Girard, Zhang, Yin, Panti https://arxiv.org/pdf/1702.04174.pdf

An investigation into three visual characteristics of complex scenes that evoke human emotion. Lu, Adams, Li, Newman, Wang http://infolab.stanford.edu/~wangz/project/imsearch/Aesthetics/ACII2017/lu.pdf

A cross-modal review of indicators for depression detection systems. Morales, Scherer, Levita https://www.aclweb.org/anthology/W17-3101

End-to-end audiovisual fusion with LSTMs. Petridis, Wang, Li, Pantic https://arxiv.org/pdf/1709.04343.pdf

Deep structured learning for facial action unit intensity estimation. Walecki, Rudovic, Pavlovic, Schuller, Pantic https://arxiv.org/pdf/1704.04481.pdf

Behavior prediction in-the-wild. Georgakis, Panagakis, Pantic https://ieeexplore.ieee.org/document/8272617

Machine learning for precision psychiatry. Bzdok, Meyer-Lindenberg https://arxiv.org/pdf/1705.10553.pdf

Easml: easily build and evaluate machine learning models. Hendricks, Ahn https://www.biorxiv.org/content/biorxiv/early/2017/05/12/137240.full.pdf

Challenges and promises for translating computational tools into clinical practice. Ahn, Busemeyer https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4834893/

Developing technology to enhance learning interpersonal skills in counsellor education. Murphy, Slovak, Thieme, Jackson, Olivier, Fitzpatrick https://www.tandfonline.com/doi/full/10.1080/03069885.2017.1377337

learning emotion-discriminative and domain-invariant features for domain adaptation in speech emotion recognition. Mao, Xu, Xue, Gou, Zhan https://www.sciencedirect.com/science/article/pii/S0167639316301923

Signal processing and machine learning for mental health research and clinical applications. Bone, Lee, Chaspari, Gibson, Narayanan https://ieeexplore.ieee.org/abstract/document/8026204

Attention networks for modeling behaviors in addiction counseling. Gibson, Can, Georgiou, Atkins, Narayanan https://pdfs.semanticscholar.org/e7cb/1ffc76948344074630ce123a6ae3200ca8af.pdf

Complexity in speech and its relation to emotional bond in therapist-patient interactions during suicide risk assessment interviews Nasir, Baucom, Bryan, Narayanan, Georgiou https://www.isca-speech.org/archive/Interspeech_2017/pdfs/1641.PDF

Deep convolutional recurrent neural network with attention mechanism for robust speech emotion recognition. Huang, Narayanan https://ieeexplore.ieee.org/document/8019296

New frontiers in ambulatory assessment: big data methods for capturing couples' emotions, vocalizations, and physiology in daily life. Timmons, Baucom, Han, Perrone, Chaspari, Narayanan, Margolin https://journals.sagepub.com/doi/full/10.1177/1948550617709115

An affect prediction approach through depression severity parameter incorporation in neural networks. Gupta, Sahu, Espy-Wilson, Narayanan https://guptarah.github.io/myPapers/rahul_incorporation_IS17.pdf

Investigating facial behavior indicators of suicidal ideation. Laksana, Baltrusaitis, Morency, Pestian

Artificial intelligence-assisted online social therapy for youth mental health. D'Alfonso, Satesteban-Echarri, Rice, Wadley, Lederman, Miles, Gleeson, Alvarez-Jimenez https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5454064/

Automatic detection of a driver's complex mental states. Ma, Mahmoud, Robinson https://www.repository.cam.ac.uk/bitstream/handle/1810/282964/iccsa17.pdf?sequence=1

Choosing prediction over explanation in psychology: lessons from machine learning. Yarkoni, Westfall https://www.ncbi.nlm.nih.gov/pubmed/28841086

2016

Multi-velocity neural networks for gesture recognition in videos. Gupta, Raviv, Raskar. https://arxiv.org/pdf/1603.06829.pdf

SoundNet: Learning sound representations from unlabeled video. Aytar, Vondrick, Torralba https://arxiv.org/pdf/1610.09001.pdf

A comprehensive survey on pose-invariant face recognition. Ding, Tao https://arxiv.org/pdf/1502.04383.pdf

Machine-learning identifies substance-specific behavioral markers for opiate and stimulant dependence. Ahn, Vassileva https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4955649/

Predicting couple therapy outcomes based on speech acoustic features. Nasir, Baucom, Georgiou, Narayanan https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0185123&type=printable

Use of machine learning to improve autism screening and diagnostic instruments: effectiveness, efficiency, and multi-instrument fusion. Bone, Bishop, Black, Goodwin, Lord, Narayanan https://onlinelibrary.wiley.com/doi/epdf/10.1111/jcpp.12559

Speech and language processing for mental health research and care. Bone, Gibson, Chaspari, Can, Narayanan https://ieeexplore.ieee.org/document/7869164

A technology prototype system for rating therapist empathy from audio recordings in addiction counseling. Xiao, Huang, Imel, Atkins, Georgiou, Narayanan https://peerj.com/articles/cs-59.pdf

2015

Computer-based personality judgements are more accurate than those made by humans. Youyou, Kosinski, Stillwell http://www.richardbenjamintrust.co.uk/uploads/finalreports/2013/DStillwell.pdf

2014

Artificial intelligence in psychological practice: current and future applications and implications. Luxton https://psycnet.apa.org/record/2013-39417-001

Learning face representation from scratch. Yi, Lei, Liao, Li https://arxiv.org/pdf/1411.7923.pdf

2011

Erroneous analyses of interactions in neuroscience: a problem of significance. Nieuwenhuis, Forstmann, Wagenmakers https://www.nature.com/articles/nn.2886

2009

Modeling mutual influence of interlocutor emotion states in dyadic spoken interactions. Lee, Busso, Lee, Narayanan https://ecs.utdallas.edu/research/researchlabs/msp-lab/publications/Lee_2009_2.pdf

2008

Meeting mediator: enchancing group collaboration using sociometric feedback. Kim, Chang, Holland, Pentland http://alumni.media.mit.edu/~taemie/papers/200811_CSCW_TKim.pdf

About

A curated list of awesome work on machine learning for mental health applications. Includes topics broadly captured by affective computing. Facial expressions, speech analysis, emotion prediction, depression, interactions, psychiatry etc. etc.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published