Tool Search 101

Tool search enables AI agents to discover the right tools using natural language queries, without requiring exact action names.

Why Tool Search
How It Works
Benchmarks
Examples

The Problem: StackOne manages over 10,000 actions across connectors, with some containing 2,000+ actions. Loading all of them into an LLM’s context window is not practical:

Token bloat: every tool definition consumes tokens, leaving less room for reasoning
Accuracy drops: LLMs make worse tool selections as the candidate set grows
Provider caps: OpenAI caps function definitions at ~128 per request

The Solution: More tools does not mean more capability. Without structure, they become noise. Tool search finds only the relevant tools for each query, keeping agent prompts lean while expanding their capabilities.

Search Flow

Search: Your agent calls the search method with a natural language query
Fetch: The SDK fetches tool definitions from your linked accounts via MCP
Match: Each connector is searched in parallel via the semantic search API
Rank: Results are ranked by relevance and matched back to tool definitions
Return: A Tools collection is returned, ready for any framework

The agent only sees the tools it needs, not the entire catalog.Three Search Modes

Mode	Behavior	When to use
`auto` (default)	Tries the semantic API first, falls back to local search on failure	Production use: best accuracy with built-in resilience
`semantic`	Semantic API only. Throws an error if the API is unavailable.	When you need strict control and want to handle errors yourself
`local`	Hybrid BM25 + TF-IDF search, entirely in-process. No network call.	Offline environments or when you need minimal latency

Local search scores results with 0.2 * BM25 + 0.8 * TF-IDF and runs with no network calls.Semantic search uses enriched embeddings where action descriptions are expanded with related terms before embedding. For example, “Create Employee” is enriched with synonyms like “add new”, “insert”, “worker”, “staff”, “team member”, so natural language queries match even when wording differs.

Tested across 103 semantically-challenging queries against 9,340 actions:

Approach	Hit@5 (all connectors)	Hit@5 (per connector)
BM25 only	21%	65%
Enhanced Embeddings	84%	90%

BM25 keyword matching works well for exact terms but fails on natural language. For example, searching “onboard new hire” won’t match “Create Employee” with BM25 alone.For comparison, Anthropic’s built-in tool search uses BM25 matching. StackOne’s semantic mode uses enriched embeddings for higher accuracy on natural language queries.

Real-World Use Cases:

HR Agent: “Approve Alice’s time off request” finds hibob_create_time_off and executes in HiBob
Support Agent: “Search for Jira ticket ABC-123” finds ticketing_ticket_list and returns details
Knowledge Agent: “Find training docs on GDPR” finds documents_file_search and queries KB

When to Use:Use tool search when your agent spans many SaaS systems (dozens to thousands of tools) and you need higher selection accuracy and scalability.Skip tool search if you have a very small, static toolset where you can load all tools directly.

Key Features

Scales to Thousands of Tools

StackOne has 10,000+ actions. Search returns only the relevant ones for each query.

Improves Accuracy

Only relevant tools exposed per request, reducing misfires and hallucinations.

Account-Aware

Filters results to tools available for configured account IDs, respecting auth boundaries.

Framework-Ready

Returns a Tools collection with converters for OpenAI, LangChain, Vercel AI SDK, and more.

Architecture Overview

Flow:

User sends a natural language query to your AI agent
Agent calls Search Tools to find relevant actions
Search Tools fetches tool definitions from MCP and ranks them via the Semantic Search API
Agent receives a ranked Tools collection
Agent calls execute() on the selected tool

Quick Example

import { StackOneToolSet } from '@stackone/ai';

const toolset = new StackOneToolSet();

// Search for tools by intent
const tools = await toolset.searchTools('manage employee time off', {
  topK: 5,
  search: 'auto',
});

// Use with OpenAI, Vercel AI SDK, etc.
const openAITools = tools.toOpenAI();

Next Steps

Tool Search for full API reference and all three methods
Basic Usage for fetching and executing tools
Tool Filtering for glob pattern filtering

Overview

Getting Started

Frameworks

API Behavior

Key Features

Scales to Thousands of Tools

Improves Accuracy

Account-Aware

Framework-Ready

Architecture Overview

Quick Example

Next Steps

Overview

Getting Started

Frameworks

API Behavior

​Key Features

Scales to Thousands of Tools

Improves Accuracy

Account-Aware

Framework-Ready

​Architecture Overview

​Quick Example

​Next Steps

Key Features

Architecture Overview

Quick Example

Next Steps