Introducing Agentic Tool Use

Currently, everyone is talking about AI workflows, and there are several tools available to help us orchestrate AI workflows, such as Dify, n8n, and make.com. However, those who have used these orchestration tools may have experienced that not only do they have a significant learning curve, but the flexibility of the orchestrated workflows is also limited.

In real life, many of the tasks we perform daily are not pre-scripted. Highly repetitive processes are indeed suitable for orchestration, but for variable and non-repetitive processes, these fixed-pattern workflows clearly lack flexibility. Orchestrating everything according to fixed patterns can be quite challenging when dealing with complex and ever-changing real-life scenarios.

To address this issue, ConsoleX has introduced a more innovative approach called "Agentic tool invocation", based on the tool use mechanisms of large language models. This allows users to create a series of "Unit Tools", each designed to perform the most basic tasks. Essentially, this equips the AI with a toolbox and a user manual for each tool, enabling the AI to autonomously combine these tools in various ways to achieve the final task goal.

This dynamic workflow offers greater flexibility and adaptability compared to pre-orchestrated workflows.

First, we create two functions: one for fetching exchange rates and another for fetching weather information. Then, we ask a question: "I'm going to Bangkok tomorrow, please check the local weather and the exchange rate of THB to USD." Next, we attach these two tools in the AI conversation and pose the question to the AI. The AI will autonomously use these tools to provide an answer.

Different large models may handle autonomous tool invocation differently. We first ask the question to OpenAI's GPT-4o model. It simultaneously invokes both tools before providing a comprehensive answer. Then we try Claude-3.5-Sonnet. When we ask the same question, it chooses to invoke the tools sequentially, first calling one tool, then the other, and finally summarizing the response.

As we can see, both parallel and sequential tool invocations can be easily achieved on ConsoleX, theoretically unlocking unlimited possibilities for large models.

ConsoleX provides two different methods for attaching tools in chat conversations: manual mode and auto mode.

Manual tool calls means manually specifying the tools needed for each conversation. Auto tool calls allows setting some commonly used tools to be automatically invoked and attached in every initiated conversation.

When using manual mode, you can also enable the Force Tool Use switch. In this case, the model is required to invoke one of the currently attached tools.

Once tools are attached in a conversation, when a question is posed to the AI model, ConsoleX will use the AI model's own tool invocation mechanism to automate the tool use agentically, including parallel and chained tool invocation, to help users solve problems.

We offer several out-of-the-box tools as pre-configured options for users to utilize in conversations. These tools include image generation, bing web search, file parsing, academic paper search, prompt generation and optimization etc. Tools marked with a Pro label are available only to users on the Builder Plan. We will continuously enrich the variety of pre-configured tools to facilitate their use in conversations.

Using pre-configured tools might incur costs and consume Credits balance on ConsoleX, as they may require calling AI models or using third-party paid services (e.g., using Flux Pro for image generation). Specific pricing details can be found in the help section on pre-configured tool usage.

Creating custom tools on ConsoleX is straightforward. Simply open the Tools interface, Click "Add Tool," and enter the tool's definition. The tool definition schema format is in JSON format, fully compliant with OpenAI's tool definition standards. Next, choose if to enable the tool invocation. If not enabled, the AI will only return the arguments for tool invocation, not actually invoke the tool; if enabled, you need to set the specific tool invocation details for actually invoking the tool.

When enabling the tool invocation, you can choose to call the tool function using GET or POST requests, and set the source type and Webhook URL for the tool function.

1. You can implement and publish the tool functions yourself, then add them as tools via custom Webhook URL.

2. You can use some AI workflow orchestration tools like n8n, Dify, and Make.com. Create workflows on these platforms and add them as tools.

Finally, set the result synthesis method, with three choices:

- Direct Output: Directly return the tool's output and end the conversation.

- AI Model Generation: The tool's output is provided to the model for generating the final result.

- Output by Template: Further set the template, where you can insert key names using {...}. If the variable name matches the attribute name in the tool's returned JSON object, the variable name will be replaced with the attribute value.

For example, if the template content is:

and the tool returns:

the output content after replacement would be:

Starter Plan users can create up to three custom tools, while Build Plan users can create an unlimited number of tools.