Machine reading for narrative generation of spatial experiences in service robots
Abstract
The goal of the project is to provide a service robot with the ability of reading written sentences found on the walls, the doors, or even the road signs and to produce a well-informed narrative of what he read during his journey. For instance, if the robot visit a lab office, he would be able to produce the following paragraph:
I was at Iván V. Meza Ruiz office. Iván V. Meza Ruiz is a research assistant at IIMAS, UNAM working at the Department of Computer Science on the Golem group. In his office I saw a Tonerre de brest poster. “Tonerre de Brest” is a nonsensical expression uttered by Captain Haddock, a fictional character in The Adventures of Tintin, the comics series by Belgian cartoonist Hergé. Then I went back to the corridor and I saw an Emergency exit sign. The “emergency exit” is a special exit for emergencies such as a fire or earthquake.
We suppose that this task can be performed by embedding the machine reading system FRED into the Golem robot. Golem is able to wander around the lab, while FRED is able to fetch keywords from DBPedia and find a suitable interpretation. The main discursive criteria for the generation of the narratives would be Golem's spatial experience: Golem (with FRED inside) tells what he read (and searched in Wikipedia) during his way.
References
- Aldo Gangemi, Valentina Presutti, Diego Reforgiato Recupero. Frame-based detection of opinion holders and topics: a model and a tool. IEEE Computational Intelligence, 9(1), 2014
- Luis A. Pineda, Arturo Rodríguez, Gibran Fuentes, Caleb Rascon and Ivan V. Meza. Concept and Functional Structure of a Service Robot. International Journal of Advanced Robotic Systems, pages 1-12, 2013.
Consortium
IIMAS, UNAM (Mexico)
- Luis A. Pineda C. (coordinator)
- Iván Vladimir Meza R.
LIPN, Paris North University
- Aldo Gangemi (coordinador)
- Jorge Garcia Flores
- Davide Buscaldi
Partner on NLG
- ?
Budget
| Concept | Duration | Amount |
|---|---|---|
| Aldo Gangemi's (LIPN, France) seminar and project meetings at IIMAS (Mexico) | 5 days | €2000 |
| Luis Pineda's (IIMAS, Mexico) seminar and project meetings at LIPN (France) | 5 days | €2000 |
| Expert on NLG (?) seminar and project meetings at IIMAS (Mexico) | 5 days | €2000 |
| Young researcher 1 (YR1) working on FRED+GOLEM interface at IIMAS (Mexico) | 720 hours over 6 months | €3000 |
| Young researcher 2 (YR2) working on coherent narrative generation from Golem's spatial experiences and FRED's definitions both at IIMAS at NLG lab (we could hire a Mexican working both at IIMAS and NLG lab or we could hire a local from the NLG lab to come and work one month at IIMAS and two months at his local lab) | 360 hours over 3 months and two countries | €3000 |
| €12000 |
Schedule
September 2015: hiring of young researcher 1 (YR1)
Work Package 1: “in vitro” trajectory (one month)
Task 1: Textual prototype for Golem
- Duration: 0.5m
- Assigned to: YR1, Jorge, Iván
Task 2: Description generation from keywords with FRED's web service
- Duration: 0.25m
- Assigned to: YR1, Jorge
Task 3: First "in vitro" trajectory: Fred + flashcodes + very simple narrative
- Duration: 0.25m
- Assigned tos: YR1, Jorge, Iván
Work Package 2: optimization of the "in vitro" trajectory (4.5 months)
Task 1: Evaluation method for "in vitro" trajectories
- Duration: 0.5m
- Assigned to: YR1, Jorge, Iván
Task 2: FRED's fork adapted and optimized for Golem
- Duration: 2m
- Assigned to: YR1, Jorge
Task 3: Experiments for "in vitro" trajectory improvement
- Duration: 1m
- Assigned to: YR1
January 2016: hiring of young researcher 2 (YR2)
Work Package 3: First "in vivo" trajectory (two months)
Task 1a: Publication on AAAI of the "in vitro" results
- Duration: 1m
- Assigned to: YR1, Iván, Jorge, NLG partner, Luis
Task 2a: Documenting
- Duration: 0.5m
- Assigned to: YR1
Task 1b: FRED agent embedded to Golem
- Duration: 1m
- Assigned to: YR2, YR1, Iván
Task 2b: NLG system implementation for rough narrative generation
- Duration: 1m
- Assigned to: YR2, NLG partner
WP 4: Coherent narrative generation from spatial experiences (two months)
Task 1: Disambiguation method for FRED keywords base on Golem's phenomenological perception
- Duration: 0.5m
- Assigned to: YR2, NLG partner
Task 2: Narrative quality evaluation method
- Duration: 0.5m
- Assigned to: YR2, NLG partner
Task 3: Narrative Optimization
- Duration: 1m
- Assigned to: YR2, NLG partner
WP5: Public demo (one month)
- Duration: 1m
- Assigned tos: Luis, Iván, Jorge, YR2
WP6: Results publication and documentation (two months)
1a: Publication on TALN of "in vivo" results
- Duration: 1m
- Assigned to: YR2, YR1, NLG partner, Jorge, Iván
2a: Documenting
- Duration: 0.5m
- Assigned to: YR2