Golfred Phase 2: Knowledge extraction with FRED and Golem ''in vivo'' path
- Due date: between Friday, November 20 and Friday, November 27
- Robot navigation
- Photos of text with robot's camera (agent)
- Behaviour programming for cammera
- OCR with ROS.read_text library
- Two alternatives:
- wether the robot finds text inspired by Gibran's FIND.ME task
- or the robot follows someone who points to the texts to read
- Integrate knowledge from Golem's internal map
- Make use of Golem's history
Complete Phase 1
- Due date: between Friday, November 20 and Friday, December 4
- Query FRED's italian web service
- Choose the best OCR string candidate
- Concaenate FRED/Tipalo outpout to produce a rudimentary text
FRED & Tipalo Instalation on TAL server
FRED & Tipalo installation on Golem
First in vivo iteration
- Due date: between December 12 and January 15
The Golem is able to perform the Static Path up the rudimentary text generation goal.
- Produce an evaluation method for in vitro and in vivo paths
- Ivan Vladimir Meza, Luis Pineda, Ricardo Montalvo, Jorge Garcia Flores
- It is necessary to program a behaviour and an agent for taking the pictures
- OCR: maybe it's worth to explor Raul Rojas binnarization
- OCR: Gibran's openCV solution is not viable in the short term
- OCR: Right now it could be programmed not as a full agent, but like an external service based on ROS_read_text or any other service
- Dynamic Path (DP): has the robot a model of a text?
- DP: an intermediary solution would be to look for frame text, and a more ambicious solution would be to send the robot look for existent texts on the space.
- DP: the dynamic path would not be predefined
- Use Case (UC): the use case is not yet mature enough, the DP might help to better defined it in a clearer way: a museum guide or follower.
- UC: Another possible use case (for a further project) would be to make scene description
- FRED could be installed as an external service interacting with the dialog model
- RDF (TODO) find a python library capable of querying and extracting Tipalos's RDF documents for Phase 1
- A first iteration of phase 2 would be to perform the in vivo prototype path of the Static Path and to query the LIPN's Fred web service
- FRED (TODO) How heavy is FRED? Would the embedded FRED version for Golem would slow the overall Golem system?
- A momentary lapse of unreason: Could we ever take Golem to Villetaneuse?
- General Ledger: Luis travel has been already funded by a Conacyt project: what to do with the extra funding?
- HISTORY: we might need to use Golem's history in order to improve the selection of what to include on the final tale
- Dialog Models: Golem's dialog model is based on Kamp's DRT
- GENERATION: Extract logic propositions from Golem's history in order to provide Geni with logical propositions for text generation.
- Montague moment: Philosopical discussion about DRT and Montague
- If Golem provides Claire the logical representation for text generation, we could start discussing and arguing a theoretical contribution of the Golem model to the DRT community.
- The task by itself it would not be totally coherent, but if we find a coherence vanishing point, we could propose a "coherence line" for the generated text (coreference, references)
- The scientific idea here would be to explore the "mental" discursive representations of Golem with the logical input propositions required by Geni, Claire's system.
- We agree to continue the project with Ricardo after our bureaucratic nightmare
- Two main tasks_
- FRED's last version on the Cluster TAL
- OCR from Golem's in vivo path
- Next (virtual) meeting: Thursday, February 18, 11am from Mexico, 19h from Paris