Tool Search 101

Two Search Patterns

There are two ways to use tool search, depending on who drives the discovery:

Developer-Driven
Agent-Driven

You search, LLM gets pre-filtered tools.Your code calls searchTools() with a natural language query, gets back a filtered Tools collection, and passes those tools to the LLM. The LLM only sees relevant tools - not the entire catalog.Best for:

Known tasks where you can predict which tools are needed
Constrained agents with a focused scope
When you want full control over which tools the LLM can access

LLM discovers and executes tools on its own.Instead of pre-filtering, you give the LLM just two tools: tool_search and tool_execute. The LLM autonomously searches for relevant tools, reads their parameter schemas, and executes them - all within its own reasoning loop.Best for:

Open-ended assistants (“help me with any HR task”)
Large tool catalogs (thousands of tools) where pre-filtering is impractical
When you don’t know which tools will be needed upfront
Minimal token overhead - always 2 tools regardless of catalog size

Aspect	Developer-Driven	Agent-Driven
Tools sent to LLM	Filtered set (5-20)	Always 2 (`tool_search` + `tool_execute`)
Token cost	Scales with number of tools	Constant
Discovery	Developer searches upfront	LLM discovers on-demand
Execution	Developer or framework executes	LLM calls `tool_execute` or framework handles it
Use case	Constrained, known tasks	Open-ended, exploratory

Tool search enables AI agents to discover the right tools using natural language queries, without requiring exact action names.

Why Tool Search
How It Works
Benchmarks
Examples

The Problem: StackOne manages over 10,000 actions across connectors, with some containing 2,000+ actions. Loading all of them into an LLM’s context window is not practical:

Token bloat: every tool definition consumes tokens, leaving less room for reasoning
Accuracy drops: LLMs make worse tool selections as the candidate set grows
Provider caps: OpenAI caps function definitions at ~128 per request

The Solution: More tools does not mean more capability. Without structure, they become noise. Tool search finds only the relevant tools for each query, keeping agent prompts lean while expanding their capabilities.

Search Flow

Search: Your agent calls the search method with a natural language query
Discover: The SDK fetches available connectors and tool definitions from your linked accounts via MCP
Match: Each connector is searched in parallel via the semantic search API
Rank: Results are matched to MCP tool definitions, ranked by relevance, deduplicated
Return: A Tools collection is returned, ready for any framework

The agent only sees the tools it needs, not the entire catalog.Three Search Modes

Mode	Behavior	When to use
`auto` (default)	Tries the semantic API first, falls back to local search on failure	Production use: best accuracy with built-in resilience
`semantic`	Semantic API only. Throws an error if the API is unavailable.	When you need strict control and want to handle errors yourself
`local`	Hybrid BM25 + TF-IDF search, entirely in-process. No network call.	Offline environments or when you need minimal latency

Local search scores results with 0.2 * BM25 + 0.8 * TF-IDF and runs with no network calls.Semantic search uses enriched embeddings where action descriptions are expanded with related terms before embedding. For example, “Create Employee” is enriched with synonyms like “add new”, “insert”, “worker”, “staff”, “team member”, so natural language queries match even when wording differs.

Tested across 103 semantically-challenging queries against 9,340 actions:

Approach	Hit@5 (all connectors)	Hit@5 (per connector)
BM25 only	21%	65%
Enhanced Embeddings	84%	90%

BM25 keyword matching works well for exact terms but fails on natural language. For example, searching “onboard new hire” won’t match “Create Employee” with BM25 alone.For comparison, Anthropic’s built-in tool search uses BM25 matching. StackOne’s semantic mode uses enriched embeddings for higher accuracy on natural language queries.

Real-World Use Cases:

HR Agent: “Approve Alice’s time off request” finds hibob_create_time_off and executes in HiBob
Support Agent: “Search for Jira ticket ABC-123” finds ticketing_ticket_list and returns details
Knowledge Agent: “Find training docs on GDPR” finds documents_file_search and queries KB

When to Use:Use tool search when your agent spans many SaaS systems (dozens to thousands of tools) and you need higher selection accuracy and scalability.Skip tool search if you have a very small, static toolset where you can load all tools directly.

Key Features

Scales to Thousands of Tools

StackOne has 10,000+ actions. Search returns only the relevant ones for each query.

Improves Accuracy

Only relevant tools exposed per request, reducing misfires and hallucinations.

Account-Aware

Filters results to tools available for configured account IDs, respecting auth boundaries.

Framework-Ready

Returns a Tools collection with converters for OpenAI, LangChain, Vercel AI SDK, and more.

Architecture Overview

Flow:

User sends a natural language query to your AI agent
Agent calls Search Tools to find relevant actions
Search Tools discovers available connectors and tool schemas via MCP, then ranks actions via the Semantic Search API
Search results are matched to MCP tool definitions, sorted by relevance
Agent receives a ranked Tools collection
Agent calls execute() on the selected tool

Quick Example

import { StackOneToolSet } from '@stackone/ai';

const toolset = new StackOneToolSet();

// Search for tools by intent
const tools = await toolset.searchTools('manage employee time off', {
  topK: 5,
  search: 'auto',
});

// Use with OpenAI, Vercel AI SDK, etc.
const openAITools = tools.toOpenAI();

Next Steps

Tool Search for full API reference and all three methods
Basic Usage for fetching and executing tools
Tool Filtering for glob pattern filtering

Overview

Getting Started

Frameworks

API Behavior

Two Search Patterns

Key Features

Scales to Thousands of Tools

Improves Accuracy

Account-Aware

Framework-Ready

Architecture Overview

Quick Example

Next Steps

Overview

Getting Started

Frameworks

API Behavior

​Two Search Patterns

​Key Features

Scales to Thousands of Tools

Improves Accuracy

Account-Aware

Framework-Ready

​Architecture Overview

​Quick Example

​Next Steps

Two Search Patterns

Key Features

Architecture Overview

Quick Example

Next Steps