Table des matières

Machine reading for narrative generation of spatial experiences in service robots

Abstract

The goal of the project is to provide a service robot with the ability of reading written sentences found on the walls, the doors, or even the road signs and to produce a well-informed narrative of what he read during his journey. For instance, if the robot visit a lab office, he would be able to produce the following paragraph:

I was at Iván V. Meza Ruiz office. Iván V. Meza Ruiz is a research assistant at IIMAS, UNAM working at the Department of Computer Science on the Golem group. In his office I saw a Tonerre de brest poster. “Tonerre de Brest” is a nonsensical expression uttered by Captain Haddock, a fictional character in The Adventures of Tintin, the comics series by Belgian cartoonist Hergé. Then I went back to the corridor and I saw an Emergency exit sign. The “emergency exit” is a special exit for emergencies such as a fire or earthquake.

We suppose that this task can be performed by embedding the machine reading system FRED into the Golem robot. Golem is able to wander around the lab, while FRED is able to fetch keywords from DBPedia and find a suitable interpretation. The main discursive criteria for the generation of the narratives would be Golem's spatial experience: Golem (with FRED inside) tells what he read (and searched in Wikipedia) during his way.

References

Consortium

IIMAS, UNAM (Mexico)

LIPN, Paris North University

Partner on NLG

Budget

Concept Duration Amount
Aldo Gangemi's (LIPN, France) seminar and project meetings at IIMAS (Mexico) 5 days €2000
Luis Pineda's (IIMAS, Mexico) seminar and project meetings at LIPN (France) 5 days €2000
Expert on NLG (?) seminar and project meetings at IIMAS (Mexico) 5 days €2000
Young researcher 1 (YR1) working on FRED+GOLEM interface at IIMAS (Mexico) 720 hours over 6 months €3000
Young researcher 2 (YR2) working on coherent narrative generation from Golem's spatial experiences and FRED's definitions both at IIMAS at NLG lab (we could hire a Mexican working both at IIMAS and NLG lab or we could hire a local from the NLG lab to come and work one month at IIMAS and two months at his local lab) 360 hours over 3 months and two countries €3000
€12000

Schedule

September 2015: hiring of young researcher 1 (YR1)

Work Package 1: “in vitro” trajectory (one month)

Task 1: Textual prototype for Golem

Task 2: Description generation from keywords with FRED's web service

Task 3: First "in vitro" trajectory: Fred + flashcodes + very simple narrative

Work Package 2: optimization of the "in vitro" trajectory (4.5 months)

Task 1: Evaluation method for "in vitro" trajectories

Task 2: FRED's fork adapted and optimized for Golem

Task 3: Experiments for "in vitro" trajectory improvement

January 2016: hiring of young researcher 2 (YR2)

Work Package 3: First "in vivo" trajectory (two months)

Task 1a: Publication on AAAI of the "in vitro" results

Task 2a: Documenting

Task 1b: FRED agent embedded to Golem

Task 2b: NLG system implementation for rough narrative generation

WP 4: Coherent narrative generation from spatial experiences (two months)

Task 1: Disambiguation method for FRED keywords base on Golem's phenomenological perception

Task 2: Narrative quality evaluation method

Task 3: Narrative Optimization

WP5: Public demo (one month)

WP6: Results publication and documentation (two months)

1a: Publication on TALN of "in vivo" results

2a: Documenting

Project Calls