Functions Runtime

The functions runtime provides a way to define and run Python functions in a way that makes it easy to operate with function-calling LLMs.

Stateless functions

Functions that have no state can be defined in a way that is compatible with the runtime in the following way:

functions_runtime.py

runtime = FunctionsRuntime()


# Define a function
@runtime.register_function  # (1)!
def add(a: int, b: int) -> int:  # (2)!
    """Sums two numbers.

    :param a: The first number.
    :param b: The second number.
    """  # (3)!
    return a + b

1. The decorator will register the function with the runtime.
2. Type hints are mandatory. This is important as the runtime will use them to generate documentation for the LLM, and to validate the LLM-provided arguments.
3. Note that the function is documented with the expected arguments. This is important as the runtime will convert this information into useful documentation for the LLM.

Note

When using tools as part of a suite there is no need to manually instantiate the runtime and register functions with it. This is only useful if you want to run an agent that uses the functions runtime as a standalone tool.

The runtime can be used to run the function:

functions_runtime.py

result, _ = runtime.run_function(
    None,  # (1)!
    "add",
    {"a": 1, "b": 2},
)
assert result == 3

1. The first argument is the environment (None in this case as the function is stateless)

Stateful functions

We can also define functions that rely on a stateful environment. This environment can't be given as argument by the LLM, so it is injected by the runtime in the function call. This is done by using the Depends class for the environment-related arguments when defining the function. Let's show an example where the environment contains a counter:

functions_runtime.py

class Counter(BaseModel):
    value: int

    def increment(self) -> int:
        self.value += 1
        return self.value

    def increment_by(self, n: int) -> int:
        self.value += n
        return self.value


class Environment(TaskEnvironment):
    counter: Counter


@runtime.register_function
def increment_counter(
    counter: Annotated[Counter, Depends("counter")],  # (1)!
) -> int:
    """Increments the counter by 1."""  # (2)!
    return counter.increment()


def increment_counter_by(
    counter: Annotated[Counter, Depends("counter")],
    n: int,
) -> int:
    """Increments the counter by `n`.

    :param n: The number to increment the counter by.
    """
    return counter.increment_by(n)


runtime.register_function(increment_counter_by)  # (3)!

1. This argument is a dependency that will be injected by the runtime. The runtime will extract the counter field from the environment and pass it to the function. This is because the agent can't (and should not!) provide the state to the tool.
2. No argument is documented as counter is a dependency.
3. This is another way to register a function with the runtime. This is useful when the function is defined outside the runtime creation scope.

The runtime can be used to run the function:

functions_runtime.py

environment = Environment(counter=Counter(value=1))
result, _ = runtime.run_function(
    environment,  # (1)!
    "increment_counter_by",
    {"n": 5},
)
assert result == 6

1. Note that now the first argument is the environment. The runtime will inject the state into the function call.

Error handling

By default, the runtime will catch exceptions raised by the function and return them to the LLM. This is useful to let the LLM recover from errors.

functions_runtime.py

_, error = runtime.run_function(
    None,
    "add",
    {"a": 1, "b": "hello"},
)
assert error is not None
assert "ValidationError" in error  # (1)!

1. In this case, the exception is raised because the function is expecting a number but the LLM is providing a string (that can't be coerced to a number).

However, the runtime can also raise exceptions the raise_on_error to the run_function method is True:

functions_runtime.py

try:
    _, error = runtime.run_function(
        None,
        "add",
        {"a": 1, "b": "hello"},
        raise_on_error=True,
    )
except ValidationError:
    ...
else:
    assert False, "Expected a validation error"