Skip to content

Introduction

The ItemRegistry is a class designed similar to a database. Its main purpose is to store what we call Actions, Tools and Items in order to enable modules such as the Exelent Parser or the Interpreters to retrieve complex objects by just using its name (an string). In the following sections, you will learn how to use, interact and register your own items inside the ItemRegistry and some deprecations that you can still found in some directories.

Initializing

The ItemRegistry acts as a database, thus it is able to be shared between different layers and threads even when you don't have access to the same instance. For obtaining this, you can use 2 different types of instantiation:

  1. If you call the ItemRegistry without arguments, you will receive the default ItemRegistry, all the threads will share the same default instance.
from ecm.tools.item_registry_v2 import ItemRegistry

obj_1 = object()
obj_2 = object()

registry_1 = ItemRegistry()
registry_2 = ItemRegistry()

print(obj_1 is obj_2) # -> False
print(registry_1 is registry_2) # -> True
  1. If you want to force the registry to maintain a separated database, you can add a name to you registry, this will return an empty registry that can be called from other threads only if they use the same name:
from ecm.tools.item_registry_v2 import ItemRegistry

default_registry = ItemRegistry()
my_registry = ItemRegistry(name="different")
my_registry_2 = ItemRegistry(name="different")

print(default_registry is my_registry) # -> False
print(my_registry is my_registry_2) # -> True

For obtaining information about the current instances and its content you can use 2 funcions:

ItemRegistry.summary()
# We add some items...

==================== Item Registry ====================
[default]
* (ACTION): Action(name='click', active=True, content=<function click at 0x7607e0f7fa30>, is_callable=True, description="Clicks with the mouse on the specified element. Example: click('Firefox Icon') or click('Navigation Bar'). Ensure to provide an especific description if needed.", labels=[])
* (TOOL): Tool(name='screenshot', active=True, content=<function screenshot at 0x7607e79c8af0>, is_callable=True, description='', labels=[])
[different]
* (TOOL): Tool(name='move_mouse_to', active=True, content=<function move_mouse_to at 0x760800cee0e0>, is_callable=True, description='', labels=[])
* (TOOL): Tool(name='send_click_event', active=True, content=<function send_click_event at 0x7607e0f7f9a0>, is_callable=True, description='', labels=[])

Adding Actions and Tools

When working with agents we will want the AI to retrieve the function use_my_mouse() with only requesting to the agent a string that says: "use_my_mouse". For this, since only some functions are targeted to be used by an AI, you must register your functions in the ItemRegistry.

Here it is important to differerentiate between two types of functions: - Actions: These are those functions targeted to the AI, they should contain a docstring, be fully typed and clear and be simple to use. This means, you should at most receive one or two arguments (if any) and the use should be self-explanatory. Use arguments as int or str for best practices. A good action could be the following:

@ItemRegistry.register(type="action")
def close_app(name_of_the_app: str) -> None:
  """Closes an app by its name. The app should be active to be closed. Usage: close("firefox")."""
  app = my_app_retriever.retrieve(name_of_the_app)
  app.close()
  ...
  • Tools: These are functions that are stored in the ItemRegistry are not targeted to the AI but the developer. This can be defined as any other function.
@ItemRegistry.register(type="tool")
def close_app(app_name: str, app_registry: AppRegistry, check_for_errors: bool = True) -> None:
  """
  Generates a summary of all the current apps
  """
  ...

As you can see, all the actions and tools are registered by just using the @ItemRegistry().register decorator. Note that this decorator will be executed as the same time as the file is imported, for this reason, you will see multiple imports of actions when the developer wants the actions to be available. These functions should be saved in the /action_space directory, usually in a file called as actions.py.

# We use the `noqa` label to avoid intellisense or formatters to flag this import as an error
import action_space.experimental.screenshot.actions # noqa

Loading Actions

Following the Principle of Least Atonishment in python the imports that are used to save the actions/tools, are not fully loaded into the registry, so imports don't alter the functioning of your program. After you have selected which actions you have imported, they must be loaded in the ItemRegistry instance you want to work with.

For example, you can use the load_all() method to enable all tools and actions retrieved in the imports into a selected instance:

from ecm.tools.item_registry_v2 import ItemRegistry
import action_space.my_lib.actions # noqa

# No action avaliable
ItemRegistry.summary()

# Note that we are calling the default instance
ItemRegistry().load_all()

# All actions are available
ItemRegistry.summary()

If you want to load only a specific action, you can use the load() method:

ItemRegistry().load("screenshot")

Sometimes you will want to add multiple actions that are related, for example, all the keyboard utils. For this, the version V2 of the ItemRegistry enables the actions to be grouped inside a package. Then you can easily load them all at the same time.

from ecm.tools.item_registry_v2 import ItemRegistry
@ItemRegistry.register(package="keyboard")
def press_A():
  """I press the Key A"""
  ...
@ItemRegistry.register(package="keyboard")
def press_B():
  """I press the Key A"""
  ...

ItemRegistry().load_package("keyboard")

Note: Some functions in this repository are still using the version V1 of the ItemRegistry to maintain backward-compatibility and thus, cannot be encapsulated into a package.

Extracting Actions and Tools

Now that you know how to integrate actions and tools, you can exit here if you only are going to use these registry for building new actions or tools to AI's (all the agents extract their capacities from this registry). However, if you are building your own agent, you will need to know how to manage the actions and tools saved in the ItemRegistry.

All the functions that are labeled as actions, will be saved in the following dataclass:

@dataclass
class Action(Item):
    name: str
    content: Callable
    description: str
    is_callable: bool = True

As you can see, the name of the function, callable itself and a description are mandatory, and will be automatically retrieved when using ItemRegistry.register from the name of the function and docstring.

In the other hand, the Tools have the following dataclass:

class Tool(Item):
    name: str
    content: Callable
    is_callable: bool = True

Similar to the actions the contain a callable, but they not enforce a docstring.

Note that if you want to add some metadata to the tools or actions you can use the .labelproperty inherited from the Item class. For example, if you want to mark the tool as dangerous you could add:

# .label is a list[str]

item = Item("fire", do_fire)
item.label.append("danger")

All the information about this classes can be found at the item_registry_v2.py file.

Now if you want to extract the actions or tools that have been loaded into a registry (for example using load_all()) you can access the .tools or .actions dictionaries, that will return the items by giving their name:

registry = ItemRegistry()

all_actions_loaded = registry.actions.values()     # -> list[Action]
all_tools_loaded = registry.tools.values()         # -> list[Tool]

screenshot = registry.actions["screenshot"]
screenshot()

press_A = registry.actions["keyboard.press_A"]
press_A

Note: The actions saved inside a package will be accessible by using their package name followed with a dot. This enables to have multiple actions with the same name, however, is generally not a good idea to use them at the same time, since the AI could missunderstand the difference between each one

Some Addons

When developing using the ItemRegistry, we enable to use some tools that will ease the debug of the actions:

  • Invalidation: By invalidating a registry, all the actions and tools saved will be replaced with a mock that won't execute the function itself but a logging method that will act as the function. For example you could use this to simulate the AI could call a function that shut downs the pc:
registry = ItemRegistry()
registry.invalidate()

mock = registry.actions["shutdown"]
mock("now", force=True)

# Output
-> mock([now], {force: True})

  • Remote Execution: The ItemRegistry can be configured to be used on a remote machine, obtain more infomation at the RemoteExecution Guide

  • Alias: You can add multiple names to the same function, this is useful for AI when it hallucinates different names for an specific function. Inside the ItemRegistry, they will be saved as different functions:


@ItemRegistry.alias([
  "type",
  "write_text",
])
@ItemRegistry.register(type="action")
def write(text: str):
  """Explanation..."""
  ...

Note: Alias only works for action not tools.

This could enlarge the size of your prompt (thera are more actions to be explained), add at most 1 or 2 aliases.

Conclusion

With this you have fully learned how to use the ItemRegistry, this class is used in multiple layers over the system such as interpreters, exelent parsers, agents...

You can try to add your own actions and register them to the default ItemRegistry, they should be automatically used in the agents integrated in this repo.

Note: The deprecated version ItemRegistryV1 can be still found at /ecm/tools/item_registry_v1.py. The file /ecm/tools/registry.py will automatilcally redirect to the v2 registry. This class has been intended to be backward-compatible but the previus versions will not be continuated. If you want to disable the enforcing of the version v2, you can change the flag PATCH_ITEM_REGISTRY_V1 at the constats.py file.