Utility Tools 101

• Why Utility Tools
• How It Works
• Evidence
• Examples

The Problem: Modern LLMs struggle when presented with too many tools in their context window. Every tool definition consumes tokens, bloats prompts, and dilutes reasoning ability.Key Issues:

• Accuracy drops when choosing between hundreds of tools (MTU‑Bench, ToolACE, BFCL v3)
• Context limits hit quickly; “Lost in the Middle” shows long prompts degrade reasoning
• Provider caps: OpenAI/Vertex limit ~128 tools/functions (OpenAI community)

The Solution: More tools ≠ more capability. Without structure, they become noise. Utility Tools solve this.

Search → Execute PatternUtility Tools act as an intelligent router:

1. Search – tool_search semantically matches user requests to relevant tool(s)
   • Powered by BM25 and Orama-style indexing for fast search across thousands of tools
   • Example: “fetch employee ID 123” → returns employee.get tool
2. Execute – tool_execute calls chosen tool(s) with right inputs
   • Agent only sees relevant tools, not the entire catalog

This keeps agent prompts lean while expanding their capabilities.

Research Sources: Accuracy decreases as candidate tool sets grow and with multi-turn complexity.

• MTU‑Bench (ICLR’25): accuracy drops as turns/tools increase (Fig. 5)
• ToolACE (2024/25): error correlates with #candidate APIs (Fig. 2)
• Broader evals: BFCL v3, Gorilla, Databricks show tool-pool pitfalls

Real-World Use Cases:

• HR Agent: “Approve Alice’s time off request” → finds hibob_create_time_off → executes in HiBob
• Support Agent: “Search for Jira ticket ABC‑123” → finds ticketing_ticket_list → returns details
• Knowledge Agent: “Find training docs on GDPR” → finds documents_file_search → queries KB

When to Use: ✅ Agent spans many SaaS systems (dozens→thousands of tools) ✅ Need higher selection accuracy and scalability ❌ Very small, static toolset

​

Core Benefits

​

Architecture Overview

• tool_search: Searches for relevant tools based on natural language queries using BM25 semantic search
• tool_execute: Executes discovered tools with the provided parameters

​

How It Works

​

Benefits

• Dynamic Discovery: No need to hardcode tool names or maintain static lists
• Multi-Tenant Ready: Agents adapt to available integrations per customer
• Natural Language: Search tools using descriptive queries
• Intelligent Routing: BM25 algorithm finds the most relevant tools automatically

​

Python Implementation

• BM25S Library: Efficient BM25 implementation for semantic search
• NumPy Integration: Fast numerical operations for score calculation
• Pydantic Models: Type-safe data structures for search results and tool parameters

​

Next Steps

• Get started with utility tools - Practical implementation guide
• Advanced Utility Tools usage - Framework integration patterns
• LangChain integration - Use utility tools with LangChain
• CrewAI integration - Build multi-agent teams