Skip to content

FastReact

FastReact is a cognitive agent designed to optimize the reasoning and action process using the ReAct methodology (Reason + Act). Its primary goal is to accelerate task execution by combining reasoning and action in a single iteration, reducing latency while maintaining system efficiency.

Motivation and Approach

The motivation behind FastReact arises from the limitations of previous models like Xplore, which, while robust, were slow due to the separation between reasoning and action execution. FastReact overcomes this by ensuring that each iteration performs both processes in a single call to the LLM. This is achieved by training the LLM to always respond in a structured JSON format, where it is forced to reason and generate the Python code that represents the action to be executed.

FastReact Workflow

The workflow of FastReact follows these primary steps:

  1. Initialization: Cognitive memory is configured with key instructions, and initial messages are preserved to establish the task's starting state.
  2. Capturing the Current State: In each iteration, a screenshot of the system is taken and converted into a message that is added to the history, providing the LLM with a visual understanding of the current context.
  3. Generating the Response: The LLM receives the full interaction history and produces a structured JSON.
  4. Execution: The Python code is parsed using the Exelent language and executed through the system's interpreter.
  5. Memory Update: The LLM's response is saved into the cognitive memory to ensure continuity and progress in future iterations.

This process repeats until the LLM indicates that the task has been completed. Cognitive memory plays an essential role, preserving critical instructions and optimizing token usage by removing unnecessary images, thereby reducing computational costs.

This mechanism can be obtained by using the following prompt:

class JsonFastReactResponse(BaseModel):
    reasoning: str = Field(description="A reasoning for solving the task")
    function: str = Field(
        description="The function with pythonic notation. E.g: myfunc(2, 3, 'foo')"
    )
    all_tasks_completed: bool = Field(
        description="True if and only if all tasks from the user have been completed"
    )

FR_PROMPT = f"""
You are a ReAct agent. On each call you must observe the history and then generate the appropriate reasoning + action in json format.

The response should use the following json template:
{FastReactParser.get_format_instructions()}

Ensure the arguments use pythonic notation.
The system will call you with the observation from your action.
"""

Conclusion

FastReact represents an evolution in cognitive agent design respect to other agents such as Xpore or RePlan. By consolidating reasoning and action into a single call and automating execution through Exelent, it offers a fast, efficient, and adaptable solution. Its integration of cognitive memory and visual perception through screenshots makes it a robust and agile agent, capable of handling complex tasks in changing environments.