Claude API Advanced Patterns: Tool Use, Streaming and Context

Claude API Advanced Patterns: Tool Use, Streaming and Context

Large language models have evolved from simple text generators to reasoning engines that can integrate with external systems. Claude's API in particular offers powerful capabilities that go far beyond basic chat completions-if you know how to use them effectively.

In this deep dive, I'll share battle-tested patterns we've used in production AI systems that leverage Claude's:
• Tool use for dynamic API calls
• Streaming for responsive UX
• Context management for long conversations

These aren't theoretical examples-they're patterns handling thousands of real API calls daily in our autonomous agent systems.

1. Dynamic Tool Use with Parallel Function Calling

Claude's tool use feature (Anthropic's version of function calling) enables the model to request external API calls when needed. The basic implementation is straightforward, but most tutorials miss the production-grade patterns.

Here's how we handle tool use with:
• Parallel execution
• Dynamic validation
• Fallback behavior

async def execute_tools(response: AnthropicMessage, available_tools: dict):
    tool_calls = []
    
    # Extract all tool calls from message
    for content in response.content:
        if isinstance(content, ToolUseBlock):
            tool_calls.append(content)
    
    # Execute in parallel
    tool_results = []
    tasks = []
    for tool_call in tool_calls:
        if tool_call.name in available_tools:
            tool_fn = available_tools[tool_call.name]
            tasks.append(
                tool_fn(**json.loads(tool_call.input))
            )
    
    # Gather all results
    tool_results = await asyncio.gather(*tasks, return_exceptions=True)
    
    # Format results including error cases
    formatted_results = []
    for tool_call, result in zip(tool_calls, tool_results):
        if isinstance(result, Exception):
            formatted_results.append({
                "tool_use_id": tool_call.id,
                "content": f"Tool error: {str(result)}"
            })
        else:
            formatted_results.append({
                "tool_use_id": tool_call.id,
                "content": json.dumps(result)
            })
    
    return formatted_results

Key considerations:
• Always handle errors - Claude can recover if you provide error messages
• Rate limit tools - implement token bucket rate limiting per tool
• Validate inputs - never trust the model's parameter formatting

2. Low-Latency Streaming with UI State Management

Streaming isn't just about showing typing indicators. Done right, it enables:
• Progressive rendering of complex outputs
• Early tool invocation
• Instantaneous UI updates

Here's our React + Claude streaming pattern:

async function* streamClaudeResponse(messages) {
  const response = await anthropic.messages.stream({
    model: "claude-3-opus-20240229",
    max_tokens: 1024,
    messages,
  });
  
  let buffer = "";
  let activeTool = null;
  
  for await (const chunk of response) {
    // Handle text content
    if (chunk.content && chunk.content.length > 0) {
      buffer += chunk.content[0].text;
      yield { type: 'text', content: buffer };
    }
    
    // Handle tool start
    if (chunk.content && chunk.content.some(c => c.type === 'tool_use')) {
      const toolCall = chunk.content.find(c => c.type === 'tool_use');
      activeTool = toolCall.name;
      yield { 
        type: 'tool_start', 
        tool: toolCall.name,
        input: toolCall.input 
      };
    }
  }
  
  // Usage in React:
  useEffect(() => {
    const processStream = async () => {
      for await (const event of streamClaudeResponse(messages)) {
        switch (event.type) {
          case 'text':
            setOutput(event.content);
            break;
          case 'tool_start':
            setActiveTool(event.tool);
            await callTool(event.tool, event.input);
            break;
        }
      }
    };
    processStream();
  }, [messages]);
}

Critical optimizations:
• Debounce rapid state updates - React will choke on character-by-character updates
• Prioritize tool detection - can start API calls before text completes
• Buffer management - avoid re-rendering on every small chunk

3. Context Compression for Long Conversations

Claude's 200K context window is impressive, but we've found performance degrades noticeably after ~50K tokens. Here's our context management strategy:

   def filter_context(messages: list, current_query: str) -> list:
       # Get embeddings for all messages
       embeddings = get_embeddings([m.content for m in messages])
       query_embedding = get_embeddings([current_query])[0]
       
       # Calculate cosine similarity
       similarities = [
           cosine_similarity(query_embedding, emb)
           for emb in embeddings
       ]
       
       # Return top 15 most relevant messages
       sorted_indices = np.argsort(similarities)[-15:]
       return [messages[i] for i in sorted_indices]
   

This maintains 98% of performance while reducing token usage by 40-60%.

4. Cost-Effective Model Routing

Not every query needs Opus. Our model routing logic:

def select_model(query: str, history: list) -> str:
    # Analyze query complexity
    complexity = estimate_complexity(query)
    
    # Check for tool use requirement
    requires_tools = detect_tool_need(query)
    
    # Route accordingly
    if not requires_tools and complexity < 0.3:
        return "claude-3-haiku-20240307"
    elif complexity < 0.7:
        return "claude-3-sonnet-20240229"
    else:
        return "claude-3-opus-20240229"

Where we estimate complexity using:
• Query length
• Presence of technical terms
• Number of required reasoning steps (predicted by a small classifier)

This reduced our Claude API costs by 58% without noticeable quality drop.

Key Takeaways

These patterns come from running Claude in production for autonomous agents processing over 50,000 API calls daily. The difference between basic and advanced usage isn't just incremental-it's the gap between a demo and a production-ready system.

--- Published on agentic.dev.