1 - AAAC emotion

human
language
technologies
Preparing Future
multiSensorial
inTerAction Research
10.2002 – 9.2004
PF-STAR – the project.
HUMAINE WP5-WS,
Belfast, Dec. 2004
1
Introduction
Speech-to-Speech
Translation
Multilingual and Multisensorial
Communication (MMC)
Detection and expressions
of emotional states
Core speech technologies
for children
• Project Motivation: to contribute to advance
research and lay the foundations for future efforts on
the topic of MMC
2
• The project has capitalised on years of research
already conducted in several national and
international research projects (C-STAR, NESPOLE!,
Verbmobil, SmartKom).
Participants
Istituto Trentino di Cultura – Centro per la Ricerca
Scientifica e Tecnologica (ITC-irst)
Interactive Systems Laboratories at
Universitaet Karlsruhe (UKA)
Institute for Pattern Recognition of Friedrich-Alexander
Universitaet - Erlangen – Nurnberg (UERLN)
Department of Electronic, Electrical & Computing
Engineering of the University of Birmingham (UB)
Kungl Tekniska Hogskolan (KTH)
RWTH Computer Science Department
3
Istituto di Scienze e Tecnologie della
Cognizione, Sezione di Padova – “Fonetica e
Dialettologia”, CNR
Project objectives
•
to contribute to establish future activities
in the field of MMC on firmer bases by
providing:
– technological baselines
– comparative evaluations
– and assessment of prospects of core
technologies, which future research and
development efforts can build from.
•
4
to improve on, refine, establish, and
align current achievements to turn them
into true technological baselines along
with
careful
assessments
and
evaluations.
WorkPackages

(1)
WP2 - Technologies for speech translation
Comparative evaluation and integration of different
technological baselines for speech translation over a range of
application scenarios

WP3 - Emotions: Speech
Identification, extraction and assessment of prosodic and other
linguistic cues correlated with and indicating the expression of
emotional states in speech. Collection of database of
audiovisual emotional speech
5
Workpackages

(2)
WP4 - Emotions: faces
Definition and assessment of technological baselines for
believable virtual agents in the form of talking heads which
produce speech and communicate emotions using
audiovisual speech synthesis. Collection of databases of
audiovisual emotional speech.

WP5 - Speech technologies for
children
Development of speech recognition baselines for English,
German, Italian, and Swedish. They have been assessed
with the aim of advancing technologies for children and
aligning them to those for adults.
6
Management - organizational structure

Workpackages’ Leaders
–
–
–
–
–
–
7
WP1: Fabio Pianesi (ITC-irst)
WP2: Marcello Federico (ITC-irst)
WP3: Anton Batliner (UERLN)
WP4: Björn Granström (KTH)
WP5: Martin Russell (UB)
WP6: Fabio Pianesi (ITC-irst)
Structure of the talk



WP4: synthesis of faces
WP3: synthesis (analysis) of speech
WP3: analysis (recognition) of speech
note: no recognition of faces and thus no
fusion!
note: technological baselines, not just basic
research
8
More information:
http://pfstar.itc.it/
9