Golfred Phase 2: Knowledge extraction with FRED and Golem ''in vivo'' path

De wikiRcln
Aller à : navigation, rechercher

Golem tasks

Static path

  • Due date: between Friday, November 20 and Friday, November 27
  1. Robot navigation
  2. Photos of text with robot's camera (agent)
  3. Behaviour programming for cammera
  4. OCR with ROS.read_text library

Dynamic path

  1. Two alternatives:
    • wether the robot finds text inspired by Gibran's FIND.ME task
    • or the robot follows someone who points to the texts to read

Other tasks

  1. Integrate knowledge from Golem's internal map
  2. Make use of Golem's history

Fred tasks

Complete Phase 1

  • Due date: between Friday, November 20 and Friday, December 4
  1. Query FRED's italian web service
  2. Choose the best OCR string candidate
  3. Concaenate FRED/Tipalo outpout to produce a rudimentary text

FRED & Tipalo Instalation on TAL server

FRED & Tipalo installation on Golem

First in vivo iteration

  • Due date: between December 12 and January 15

The Golem is able to perform the Static Path up the rudimentary text generation goal.


  • Produce an evaluation method for in vitro and in vivo paths



  • Ivan Vladimir Meza, Luis Pineda, Ricardo Montalvo, Jorge Garcia Flores
  1. It is necessary to program a behaviour and an agent for taking the pictures
  2. OCR: maybe it's worth to explor Raul Rojas binnarization
  3. OCR: Gibran's openCV solution is not viable in the short term
  4. OCR: Right now it could be programmed not as a full agent, but like an external service based on ROS_read_text or any other service
  5. Dynamic Path (DP): has the robot a model of a text?
  6. DP: an intermediary solution would be to look for frame text, and a more ambicious solution would be to send the robot look for existent texts on the space.
  7. DP: the dynamic path would not be predefined
  8. Use Case (UC): the use case is not yet mature enough, the DP might help to better defined it in a clearer way: a museum guide or follower.
  9. UC: Another possible use case (for a further project) would be to make scene description
  10. FRED could be installed as an external service interacting with the dialog model
  11. RDF (TODO) find a python library capable of querying and extracting Tipalos's RDF documents for Phase 1
  12. A first iteration of phase 2 would be to perform the in vivo prototype path of the Static Path and to query the LIPN's Fred web service
  13. FRED (TODO) How heavy is FRED? Would the embedded FRED version for Golem would slow the overall Golem system?
  14. A momentary lapse of unreason: Could we ever take Golem to Villetaneuse?
  15. General Ledger: Luis travel has been already funded by a Conacyt project: what to do with the extra funding?
  16. HISTORY: we might need to use Golem's history in order to improve the selection of what to include on the final tale
  17. Dialog Models: Golem's dialog model is based on Kamp's DRT
  18. GENERATION: Extract logic propositions from Golem's history in order to provide Geni with logical propositions for text generation.
  19. Montague moment: Philosopical discussion about DRT and Montague
  20. If Golem provides Claire the logical representation for text generation, we could start discussing and arguing a theoretical contribution of the Golem model to the DRT community.
  21. The task by itself it would not be totally coherent, but if we find a coherence vanishing point, we could propose a "coherence line" for the generated text (coreference, references)
  22. The scientific idea here would be to explore the "mental" discursive representations of Golem with the logical input propositions required by Geni, Claire's system.


  1. We agree to continue the project with Ricardo after our bureaucratic nightmare
  2. Two main tasks_
    1. FRED's last version on the Cluster TAL
    2. OCR from Golem's in vivo path
  • Next (virtual) meeting: Thursday, February 18, 11am from Mexico, 19h from Paris