Golfred Phase 2: Knowledge extraction with FRED and Golem "in vivo" path

• Due date: between Friday, November 20 and Friday, November 27

1. Robot navigation
2. Photos of text with robot's camera (agent)
3. Behaviour programming for cammera
4. OCR with ROS.read_text library

1. Two alternatives:
   • wether the robot finds text inspired by Gibran's FIND.ME task
   • or the robot follows someone who points to the texts to read

1. Integrate knowledge from Golem's internal map
2. Make use of Golem's history

• Due date: between Friday, November 20 and Friday, December 4

1. Query FRED's italian web service
2. Choose the best OCR string candidate
3. Concaenate FRED/Tipalo outpout to produce a rudimentary text

• Due date: between December 12 and January 15

The Golem is able to perform the Static Path up the rudimentary text generation goal.

• Produce an evaluation method for in vitro and in vivo paths

• Ivan Vladimir Meza, Luis Pineda, Ricardo Montalvo, Jorge Garcia Flores

1. It is necessary to program a behaviour and an agent for taking the pictures
2. OCR: maybe it's worth to explor Raul Rojas binnarization
3. OCR: Gibran's openCV solution is not viable in the short term
4. OCR: Right now it could be programmed not as a full agent, but like an external service based on ROSreadtext or any other service
5. Dynamic Path (DP): has the robot a model of a text?
6. DP: an intermediary solution would be to look for frame text, and a more ambicious solution would be to send the robot look for existent texts on the space.
7. DP: the dynamic path would not be predefined
8. Use Case (UC): the use case is not yet mature enough, the DP might help to better defined it in a clearer way: a museum guide or follower.
9. UC: Another possible use case (for a further project) would be to make scene description
10. FRED could be installed as an external service interacting with the dialog model
11. RDF (TODO) find a python library capable of querying and extracting Tipalos's RDF documents for Phase 1
12. A first iteration of phase 2 would be to perform the in vivo prototype path of the Static Path and to query the LIPN's Fred web service
13. FRED (TODO) How heavy is FRED? Would the embedded FRED version for Golem would slow the overall Golem system?
14. A momentary lapse of unreason: Could we ever take Golem to Villetaneuse?
15. General Ledger: Luis travel has been already funded by a Conacyt project: what to do with the extra funding?
16. HISTORY: we might need to use Golem's history in order to improve the selection of what to include on the final tale
17. Dialog Models: Golem's dialog model is based on Kamp's DRT
18. GENERATION: Extract logic propositions from Golem's history in order to provide Geni with logical propositions for text generation.
19. Montague moment: Philosopical discussion about DRT and Montague
20. If Golem provides Claire the logical representation for text generation, we could start discussing and arguing a theoretical contribution of the Golem model to the DRT community.
21. The task by itself it would not be totally coherent, but if we find a coherence vanishing point, we could propose a “coherence line” for the generated text (coreference, references)
22. The scientific idea here would be to explore the “mental” discursive representations of Golem with the logical input propositions required by Geni, Claire's system.

1. We agree to continue the project with Ricardo after our bureaucratic nightmare
2. Two main tasks_
3. FRED's last version on the Cluster TAL
4. OCR from Golem's in vivo path

• Next (virtual) meeting: Thursday, February 18, 11am from Mexico, 19h from Paris