Research

Social skills training by computer agents

People with autism have superior skills and stronger interests in technology. It is very effective to utilize technology to train complex human communication skills. Using conversational agents, we develop a system that automates the cognitive behavioral therapy such as social skills training (SST).

[Blog (in Japanese)] [Youtube] [iTunes (for iPad)]

Related papers:

  • Hiroki Tanaka, Sakriani Sakti, Graham Neubig, Tomoki Toda, Hideki Negoro, Hidemi Iwasaka, Satoshi Nakamura, Automated Social Skills Trainer, International Conference on Intelligent User Interfaces (IUI), Mar. 2015.
  • Hiroki Tanaka, Hideki Negoro, Hidemi Iwasaka, Satoshi Nakamura, Embodied Conversational Agents for Multimodal Automated Social Skills Training in People with Autism Spectrum Disorders, Plos One, Aug. 2017.
  • Hiroki Tanaka, Hideki Negoro, Hidemi Iwasaka, Satoshi Nakamura, Listening Skills Assessment through Computer Agents, ACM International Conference on Multimodal Interaction (ICMI), Oct. 2018.

Predicting mental states by machine learning

We try to predict and understand human mental states such as neurodegenerative disorders, emotion, and mismatched feeling by multimodal information processing and machine learning (e.g. neural networks, tensor decompositions).

Related papers:

  • Hiroki Tanaka, Nick Campbell, Classification of Social Laughter in Natural Conversational Speech, Computer Speech & Language, vol.28(1), pp.314-325, 2014.
  • Hiroki Tanaka, Hiroyoshi Adachi, Norimichi Ukita, Manabu Ikeda, Kazui Hiroaki, Takashi Kudo, Satoshi Nakamura, Automatic Detection of Very Early Stage of Dementia through Multimodal Interaction with Computer Avatars, ACM International Conference on Multimodal Interaction (ICMI), Nov. 2016.
  • Hiroki Tanaka, Hiroki Watanabe, Hayato Maki, Sakriani Sakti, Satoshi Nakamura, Single-trial Detection of Semantic Anomalies from EEG during Listening to Spoken Sentences,  International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 977-980, July 2018.

Training for recognizing verbal and non-verbal information

We proposed an application aiming at training to read changes in verbal and non-verbal information that matched situation.

Related papers:

  • Hiroki Tanaka, Sakriani Sakti, Graham Neubig, Tomoki Toda, Satoshi Nakamura, NOCOA: A Computer-Based Training Tool for Social and Communication Skills That Exploits Non-verbal Behaviors, The Journal of Information and Systems in Education, vol.12(1), pp.19-26, 2014.
  • Hiroki Tanaka, Sakriani Sakti, Graham Neubig, Tomoki Toda, Satoshi Nakamura, NOCOA+: Multimodal Computer-Based Training for Social and Communication Skills, IEICE transactions on Information and Systems, vol.E98-D(8), pp.1536-1544, 2015.