Tool Search 101

Tool search enables AI agents to discover the right tools using natural language queries, without requiring exact action names.

Why Tool Search
How It Works
Benchmarks
Examples

The Problem: StackOne manages over 10,000 actions across connectors, with some containing 2,000+ actions. Loading all of them into an LLM’s context window is not practical:

Token bloat: every tool definition consumes tokens, leaving less room for reasoning
Accuracy drops: LLMs make worse tool selections as the candidate set grows
Provider caps: OpenAI caps function definitions at ~128 per request

The Solution: More tools does not mean more capability. Without structure, they become noise. Tool search finds only the relevant tools for each query, keeping agent prompts lean while expanding their capabilities.

Search Flow

Search: Your agent calls the search method with a natural language query
Discover: The SDK fetches available connectors and tool definitions from your linked accounts via MCP
Match: Each connector is searched in parallel via the semantic search API
Rank: Results are matched to MCP tool definitions, ranked by relevance, deduplicated
Return: A Tools collection is returned, ready for any framework

The agent only sees the tools it needs, not the entire catalog.Three Search Modes

Mode	Behavior	When to use
`auto` (default)	Tries the semantic API first, falls back to local search on failure	Production use: best accuracy with built-in resilience
`semantic`	Semantic API only. Throws an error if the API is unavailable.	When you need strict control and want to handle errors yourself
`local`	Hybrid BM25 + TF-IDF search, entirely in-process. No network call.	Offline environments or when you need minimal latency

Local search scores results with 0.2 * BM25 + 0.8 * TF-IDF and runs with no network calls.Semantic search uses enriched embeddings where action descriptions are expanded with related terms before embedding. For example, “Create Employee” is enriched with synonyms like “add new”, “insert”, “worker”, “staff”, “team member”, so natural language queries match even when wording differs.

Tested across 103 semantically-challenging queries against 9,340 actions:

Approach	Hit@5 (all connectors)	Hit@5 (per connector)
BM25 only	21%	65%
Enhanced Embeddings	84%	90%

BM25 keyword matching works well for exact terms but fails on natural language. For example, searching “onboard new hire” won’t match “Create Employee” with BM25 alone.For comparison, Anthropic’s built-in tool search uses BM25 matching. StackOne’s semantic mode uses enriched embeddings for higher accuracy on natural language queries.

Real-World Use Cases:

HR Agent: “Approve Alice’s time off request” finds hibob_create_time_off and executes in HiBob
Support Agent: “Search for Jira ticket ABC-123” finds ticketing_ticket_list and returns details
Knowledge Agent: “Find training docs on GDPR” finds documents_file_search and queries KB

When to Use:Use tool search when your agent spans many SaaS systems (dozens to thousands of tools) and you need higher selection accuracy and scalability.Skip tool search if you have a very small, static toolset where you can load all tools directly.

Key Features

Scales to Thousands of Tools

StackOne has 10,000+ actions. Search returns only the relevant ones for each query.

Improves Accuracy

Only relevant tools exposed per request, reducing misfires and hallucinations.

Account-Aware

Filters results to tools available for configured account IDs, respecting auth boundaries.

Framework-Ready

Returns a Tools collection with converters for OpenAI, LangChain, Vercel AI SDK, and more.

Quick Example

from stackone_ai import StackOneToolSet

toolset = StackOneToolSet()

# Get search & execute tools. The LLM discovers and runs tools on demand
tools = toolset.openai(mode="search_and_execute")

# Pass these to your OpenAI agent
# The agent gets tool_search + tool_execute, only 2 tools regardless of catalog size

Two Approaches

Search & Execute (recommended)
Pre-filtered

Give the LLM 2 tools — it discovers and runs actions on demand.Instead of pre-filtering, you give the LLM tool_search and tool_execute. The LLM searches for relevant tools, reads their parameter schemas, and executes them within its own reasoning loop.Best for:

Open-ended assistants (“help me with any HR task”)
Large tool catalogs where pre-filtering is impractical
When you don’t know which tools will be needed upfront
Minimal token overhead — always 2 tools regardless of catalog size

You search first, LLM gets a focused set of tools.Your code calls searchTools() or fetchTools() with filters, gets back a Tools collection, and passes those to the LLM. The LLM only sees relevant tools.Best for:

Known tasks where you can predict which tools are needed
Constrained agents with a focused scope
When you want full control over which tools the LLM can access

Aspect	Search & Execute	Pre-filtered
Tools sent to LLM	Always 2 (`tool_search` + `tool_execute`)	Filtered set (5-20)
Token cost	Constant	Scales with number of tools
Discovery	LLM discovers on-demand	You search upfront
Use case	Open-ended, exploratory	Constrained, known tasks

Architecture Overview

Flow:

User sends a natural language query to your AI agent
Agent calls Search Tools to find relevant actions
Search Tools discovers available connectors and tool schemas via MCP, then ranks actions via the Semantic Search API
Search results are matched to MCP tool definitions, sorted by relevance
Agent receives a ranked Tools collection
Agent calls execute() on the selected tool

Next Steps

Tool Search for full API reference and all three methods
Basic Usage for fetching and executing tools
Tool Filtering for glob pattern filtering

Overview

Getting Started

Frameworks

API Behavior

Key Features

Scales to Thousands of Tools

Improves Accuracy

Account-Aware

Framework-Ready

Quick Example

Two Approaches

Architecture Overview

Next Steps

​Key Features

Scales to Thousands of Tools

Improves Accuracy

Account-Aware

Framework-Ready

​Quick Example

​Two Approaches

​Architecture Overview

​Next Steps

Key Features

Quick Example

Two Approaches

Architecture Overview

Next Steps