Machine reading for narrative generation of spatial experiences in service robots

De wikiRcln
Aller à : navigation, rechercher

Abstract

The goal of the project is to provide a service robot with the ability of reading written sentences found on the walls, the doors, or even the road signs and to produce a well-informed narrative of what he read during his journey. For instance, if the robot visit a lab office, he would be able to produce the following paragraph:

I was at Iván V. Meza Ruiz office. Iván V. Meza Ruiz is a research assistant at IIMAS, UNAM working at the Department of Computer Science on the Golem group. In his office I saw a Tonerre de brest poster. "Tonerre de Brest" is a nonsensical expression uttered by Captain Haddock, a fictional character in The Adventures of Tintin, the comics series by Belgian cartoonist Hergé. Then I went back to the corridor and I saw an Emergency exit sign. The "emergency exit" is a special exit for emergencies such as a fire or earthquake.

We suppose that this task can be performed by embedding the machine reading system FRED into the Golem robot. Golem is able to wander around the lab, while FRED is able to fetch keywords from DBPedia and find a suitable interpretation. The main discursive criteria for the generation of the narratives would be Golem's spatial experience: Golem (with FRED inside) tells what he read (and searched in Wikipedia) during his way.

References

  • Aldo Gangemi, Valentina Presutti, Diego Reforgiato Recupero. Frame-based detection of opinion holders and topics: a model and a tool. IEEE Computational Intelligence, 9(1), 2014
  • Luis A. Pineda, Arturo Rodríguez, Gibran Fuentes, Caleb Rascon and Ivan V. Meza. Concept and Functional Structure of a Service Robot. International Journal of Advanced Robotic Systems, pages 1-12, 2013.


Consortium

IIMAS, UNAM (Mexico)

  • Luis A. Pineda C. (coordinator)
  • Iván Vladimir Meza R.

LIPN, Paris North University

  • Aldo Gangemi (coordinador)
  • Jorge Garcia Flores
  • Davide Buscaldi

Partner on NLG

  •  ?

Budget

Concept
Duration Amount
Aldo Gangemi's (LIPN, France) seminar and project meetings at IIMAS (Mexico) 5 days €2000
Luis Pineda's (IIMAS, Mexico) seminar and project meetings at LIPN (France) 5 days €2000
Expert on NLG (?) seminar and project meetings at IIMAS (Mexico) 5 days €2000
Young researcher 1 (YR1) working on FRED+GOLEM interface at IIMAS (Mexico) 720 hours over 6 months €3000
Young researcher 2 (YR2) working on coherent narrative generation from Golem's spatial experiences and FRED's definitions both at IIMAS at NLG lab (we could hire a Mexican working both at IIMAS and NLG lab or we could hire a local from the NLG lab to come and work one month at IIMAS and two months at his local lab) 360 hours over 3 months and two countries €3000
€12000

Schedule

September 2015: hiring of young researcher 1 (YR1)

Work Package 1: “in vitro” trajectory (one month)

Task 1: Textual prototype for Golem

  • Duration: 0.5m
  • Assigned to: YR1, Jorge, Iván

Task 2: Description generation from keywords with FRED's web service

  • Duration: 0.25m
  • Assigned to: YR1, Jorge

Task 3: First in vitro trajectory: Fred + flashcodes + very simple narrative

  • Duration: 0.25m
  • Assigned tos: YR1, Jorge, Iván

Work Package 2: optimization of the in vitro trajectory (4.5 months)

Task 1: Evaluation method for in vitro trajectories

  • Duration: 0.5m
  • Assigned to: YR1, Jorge, Iván

Task 2: FRED's fork adapted and optimized for Golem

  • Duration: 2m
  • Assigned to: YR1, Jorge

Task 3: Experiments for in vitro trajectory improvement

  • Duration: 1m
  • Assigned to: YR1

January 2016: hiring of young researcher 2 (YR2)

Work Package 3: First in vivo trajectory (two months)

Task 1a: Publication on AAAI of the in vitro results

  • Duration: 1m
  • Assigned to: YR1, Iván, Jorge, NLG partner, Luis

Task 2a: Documenting

  • Duration: 0.5m
  • Assigned to: YR1

Task 1b: FRED agent embedded to Golem

  • Duration: 1m
  • Assigned to: YR2, YR1, Iván

Task 2b: NLG system implementation for rough narrative generation

  • Duration: 1m
  • Assigned to: YR2, NLG partner

WP 4: Coherent narrative generation from spatial experiences (two months)

Task 1: Disambiguation method for FRED keywords base on Golem's phenomenological perception

  • Duration: 0.5m
  • Assigned to: YR2, NLG partner

Task 2: Narrative quality evaluation method

  • Duration: 0.5m
  • Assigned to: YR2, NLG partner

Task 3: Narrative Optimization

  • Duration: 1m
  • Assigned to: YR2, NLG partner

WP5: Public demo (one month)

  • Duration: 1m
  • Assigned tos: Luis, Iván, Jorge, YR2

WP6: Results publication and documentation (two months)

1a: Publication on TALN of in vivo results

  • Duration: 1m
  • Assigned to: YR2, YR1, NLG partner, Jorge, Iván

2a: Documenting

  • Duration: 0.5m
  • Assigned to: YR2

Project Calls