About

Projects

People

References

Events

Contact

Research

iMAP : Tracking Multimodal Communication in Humans and Agents

Max Louwerse (PI), Ellen Bard (Co-PI), Mark Steedman (Co-PI), Art Graesser (Co-PI) & Xiangen Hu (Co-PI) (2004-2008)

When we talk to somebody we are involved in a rich complex of communicative activity, facial expressions, hand gestures, direction of gaze, to name but the most obvious ones. The interpretation of "are you hungry" depends on the context (e.g. just before going to a restaurant, during dinner), depends on prosody (e.g. stress on ‘you’ or ‘hungry’), facial expressions (e.g. brows raised, brows and gaze both raised) and gestures (e.g. rubbing stomach, pointing at a restaurant). We know that hearers adapt to the speaker (e.g. maintaining the theme of the conversation, smiling etc.). Research into the interaction of these channels is however limited, often focusing on the interaction between a pair of channels.

The iMAP (Intelligent MapTask Agent) project investigates multimodal communication in humans and agents, focusing on two linguistic modalities - prosody and dialog structure, which reflect major communicative events, and two non-linguistic modalities - eye gaze and facial expressions. It aims to determine

1. which of the non-linguistic modalities align with events marked by prosody and dialogue structure, and with one another;

2. whether, and if so when, these modalities are observed by the interlocutor;

3. whether the correct use of these channels actually aids the interlocutor’s comprehension.

Answers to these questions should provide a better understanding of the use of communicative resources in discourse and can subsequently aid the development of more effective animated conversational agents. The research resulting from this project will benefit a large variety of fields, including cognitive science, computational linguistics, artificial intelligence, and computer science. In addition, the integration of the modalities into a working model will advance the development and use of intelligent conversational systems.

AutoTutor - emotions : Monitoring Emotions While Students Learn with AutoTutor

Art Graesser (PI) & Max Louwerse (Senior Researcher) (2003-2008)

iSTART : Interactive Strategy Training for Active Reading and Thinking

Danielle McNamara (PI) & Max Louwerse (Senior Researcher) (2004-2007)

iSTART (Interactive Strategy Trainer for Active Reading and Thinking) is an automated strategy trainer designed to help students become better readers via multi-media technologies.

Coh-Metrix : Automated Cohesion and Coherence Scores to Predict Text Readability and Facilitate Comprehension

Danielle McNamara (PI), Max Louwerse (Co-PI) & Art Graesser (Co-PI) (2002-2005)

Using advanced technologies, Coh-Metrix will allow readers, writers, educators, and researchers to instantly gauge the difficulty of written material, based on the target audience.

Why2000 : A Tutor that Teaches Mental Models Using Natural Language Dialogs

Art Graesser (PI) & Max Louwerse (Senior Researcher) (2000-2005)

AutoTutor - Literacy / Physics : Developing AutoTutor for Computer Literacy and Physics

Art Graesser (PI) & Max Louwerse (Co-PI) (2001-2004)

AutoTutor is a web-based computer tutor architecture that simulates the dialog moves of effective human tutors.

QUAID : Developing and Testing a Computer Tool that Critiques Survey Questions

Art Graesser (PI) & Max Louwerse (Senior Researcher) (2000-2003)

QUAID (Question Understanding Aid) is a software program designed to assist survey methodologists, social scientists, and designers of questionnaires in improving the wording, syntax, and semantics of questions.