Skip to content

Commit cab2454

Browse files
committed
imrpoved toolspec features
1 parent d9e8a02 commit cab2454

24 files changed

+3161
-36
lines changed

README.md

Lines changed: 196 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -44,12 +44,15 @@ Unlike full-fledged LLM frameworks (LangChain, LlamaIndex, etc.), CHUK Tool Proc
4444
Research code vs production code is about handling the edges:
4545

4646
- **Timeouts**: Every tool execution has proper timeout handling
47-
- **Retries**: Automatic retry with exponential backoff
47+
- **Retries**: Automatic retry with exponential backoff and deadline awareness
4848
- **Rate Limiting**: Global and per-tool rate limits with sliding windows
49-
- **Caching**: Intelligent result caching with TTL
50-
- **Error Handling**: Graceful degradation, never crashes your app
49+
- **Caching**: Intelligent result caching with TTL and idempotency key support
50+
- **Circuit Breakers**: Prevent cascading failures with automatic fault detection
51+
- **Error Handling**: Machine-readable error codes with structured details
5152
- **Observability**: Structured logging, metrics, request tracing
5253
- **Safety**: Subprocess isolation for untrusted code
54+
- **Type Safety**: Pydantic validation with LLM-friendly argument coercion
55+
- **Tool Discovery**: Formal schema export (OpenAI, Anthropic, MCP formats)
5356

5457
### It's About Stacks
5558

@@ -63,11 +66,13 @@ CHUK Tool Processor uses a **composable stack architecture**:
6366
│ tool calls
6467
6568
┌─────────────────────────────────┐
66-
│ Caching Wrapper │ ← Cache expensive results
69+
│ Caching Wrapper │ ← Cache expensive results (idempotency keys)
6770
├─────────────────────────────────┤
6871
│ Rate Limiting Wrapper │ ← Prevent API abuse
6972
├─────────────────────────────────┤
70-
│ Retry Wrapper │ ← Handle transient failures
73+
│ Retry Wrapper │ ← Handle transient failures (exponential backoff)
74+
├─────────────────────────────────┤
75+
│ Circuit Breaker Wrapper │ ← Prevent cascading failures (CLOSED/OPEN/HALF_OPEN)
7176
├─────────────────────────────────┤
7277
│ Execution Strategy │ ← How to run tools
7378
│ • InProcess (fast) │
@@ -611,6 +616,192 @@ processor = ToolProcessor(
611616
)
612617
```
613618

619+
### Advanced Production Features
620+
621+
Beyond basic configuration, CHUK Tool Processor includes several advanced features for production environments:
622+
623+
#### Circuit Breaker Pattern
624+
625+
Prevent cascading failures by automatically opening circuits for failing tools:
626+
627+
```python
628+
from chuk_tool_processor.core.processor import ToolProcessor
629+
630+
processor = ToolProcessor(
631+
enable_circuit_breaker=True,
632+
circuit_breaker_threshold=5, # Open after 5 failures
633+
circuit_breaker_timeout=60.0, # Try recovery after 60s
634+
)
635+
636+
# Circuit states: CLOSED → OPEN → HALF_OPEN → CLOSED
637+
# - CLOSED: Normal operation
638+
# - OPEN: Blocking requests (too many failures)
639+
# - HALF_OPEN: Testing recovery with limited requests
640+
```
641+
642+
**How it works:**
643+
1. Tool fails repeatedly (hits threshold)
644+
2. Circuit opens → requests blocked immediately
645+
3. After timeout, circuit enters HALF_OPEN
646+
4. If test requests succeed → circuit closes
647+
5. If test requests fail → back to OPEN
648+
649+
**Benefits:**
650+
- Prevents wasting resources on failing services
651+
- Fast-fail for better UX
652+
- Automatic recovery detection
653+
654+
#### Idempotency Keys
655+
656+
Automatically deduplicate LLM tool calls using SHA256-based keys:
657+
658+
```python
659+
from chuk_tool_processor.models.tool_call import ToolCall
660+
661+
# Idempotency keys are auto-generated
662+
call1 = ToolCall(tool="search", arguments={"query": "Python"})
663+
call2 = ToolCall(tool="search", arguments={"query": "Python"})
664+
665+
# Same arguments = same idempotency key
666+
assert call1.idempotency_key == call2.idempotency_key
667+
668+
# Used automatically by caching layer
669+
processor = ToolProcessor(enable_caching=True)
670+
results1 = await processor.execute([call1]) # Executes
671+
results2 = await processor.execute([call2]) # Cache hit!
672+
```
673+
674+
**Benefits:**
675+
- Prevents duplicate executions from LLM retries
676+
- Deterministic cache keys
677+
- No manual key management needed
678+
679+
#### Tool Schema Export
680+
681+
Export tool definitions to multiple formats for LLM prompting:
682+
683+
```python
684+
from chuk_tool_processor.models.tool_spec import ToolSpec, ToolCapability
685+
from chuk_tool_processor.models.validated_tool import ValidatedTool
686+
687+
@register_tool(name="weather")
688+
class WeatherTool(ValidatedTool):
689+
"""Get current weather for a location."""
690+
691+
class Arguments(BaseModel):
692+
location: str = Field(..., description="City name")
693+
694+
class Result(BaseModel):
695+
temperature: float
696+
conditions: str
697+
698+
# Generate tool spec
699+
spec = ToolSpec.from_validated_tool(WeatherTool)
700+
701+
# Export to different formats
702+
openai_format = spec.to_openai() # For OpenAI function calling
703+
anthropic_format = spec.to_anthropic() # For Claude tools
704+
mcp_format = spec.to_mcp() # For MCP servers
705+
706+
# Example OpenAI format:
707+
# {
708+
# "type": "function",
709+
# "function": {
710+
# "name": "weather",
711+
# "description": "Get current weather for a location.",
712+
# "parameters": {...} # JSON Schema
713+
# }
714+
# }
715+
```
716+
717+
**Use cases:**
718+
- Generate tool definitions for LLM system prompts
719+
- Documentation generation
720+
- API contract validation
721+
- Cross-platform tool sharing
722+
723+
#### Machine-Readable Error Codes
724+
725+
Structured error handling with error codes for programmatic responses:
726+
727+
```python
728+
from chuk_tool_processor.core.exceptions import (
729+
ErrorCode,
730+
ToolNotFoundError,
731+
ToolTimeoutError,
732+
ToolCircuitOpenError,
733+
)
734+
735+
try:
736+
results = await processor.process(llm_output)
737+
except ToolNotFoundError as e:
738+
if e.code == ErrorCode.TOOL_NOT_FOUND:
739+
# Suggest available tools to LLM
740+
available = e.details.get("available_tools", [])
741+
print(f"Try one of: {available}")
742+
except ToolTimeoutError as e:
743+
if e.code == ErrorCode.TOOL_TIMEOUT:
744+
# Inform LLM to use faster alternative
745+
timeout = e.details["timeout"]
746+
print(f"Tool timed out after {timeout}s")
747+
except ToolCircuitOpenError as e:
748+
if e.code == ErrorCode.TOOL_CIRCUIT_OPEN:
749+
# Tell LLM this service is temporarily down
750+
reset_time = e.details.get("reset_timeout")
751+
print(f"Service unavailable, retry in {reset_time}s")
752+
753+
# All errors include .to_dict() for logging
754+
error_dict = e.to_dict()
755+
# {
756+
# "error": "ToolCircuitOpenError",
757+
# "code": "TOOL_CIRCUIT_OPEN",
758+
# "message": "Tool 'api_tool' circuit breaker is open...",
759+
# "details": {"tool_name": "api_tool", "failure_count": 5, ...}
760+
# }
761+
```
762+
763+
**Available error codes:**
764+
- `TOOL_NOT_FOUND` - Tool doesn't exist in registry
765+
- `TOOL_EXECUTION_FAILED` - Tool execution error
766+
- `TOOL_TIMEOUT` - Tool exceeded timeout
767+
- `TOOL_CIRCUIT_OPEN` - Circuit breaker is open
768+
- `TOOL_RATE_LIMITED` - Rate limit exceeded
769+
- `TOOL_VALIDATION_ERROR` - Argument validation failed
770+
- `MCP_CONNECTION_FAILED` - MCP server unreachable
771+
- Plus 11 more for comprehensive error handling
772+
773+
#### LLM-Friendly Argument Coercion
774+
775+
Automatically coerce LLM outputs to correct types:
776+
777+
```python
778+
from chuk_tool_processor.models.validated_tool import ValidatedTool
779+
780+
class SearchTool(ValidatedTool):
781+
class Arguments(BaseModel):
782+
query: str
783+
limit: int = 10
784+
category: str = "all"
785+
786+
# Pydantic config for LLM outputs:
787+
# - str_strip_whitespace=True → Remove accidental whitespace
788+
# - extra="ignore" → Ignore unknown fields
789+
# - use_enum_values=True → Convert enums to values
790+
# - coerce_numbers_to_str=False → Keep type strictness
791+
792+
# LLM outputs often have quirks:
793+
llm_output = {
794+
"query": " Python tutorials ", # Extra whitespace
795+
"limit": "5", # String instead of int
796+
"unknown_field": "ignored" # Extra field
797+
}
798+
799+
# ValidatedTool automatically coerces and validates
800+
tool = SearchTool()
801+
result = await tool.execute(**llm_output)
802+
# ✅ Works! Whitespace stripped, "5" → 5, extra field ignored
803+
```
804+
614805
## Advanced Topics
615806

616807
### Using Subprocess Strategy

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
44

55
[project]
66
name = "chuk-tool-processor"
7-
version = "0.7.0"
7+
version = "0.8"
88
description = "Async-native framework for registering, discovering, and executing tools referenced in LLM responses"
99
readme = "README.md"
1010
requires-python = ">=3.11"
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,32 @@
11
# chuk_tool_processor/core/__init__.py
2+
"""Core functionality for the tool processor."""
3+
4+
from chuk_tool_processor.core.exceptions import (
5+
ErrorCode,
6+
MCPConnectionError,
7+
MCPError,
8+
MCPTimeoutError,
9+
ParserError,
10+
ToolCircuitOpenError,
11+
ToolExecutionError,
12+
ToolNotFoundError,
13+
ToolProcessorError,
14+
ToolRateLimitedError,
15+
ToolTimeoutError,
16+
ToolValidationError,
17+
)
18+
19+
__all__ = [
20+
"ErrorCode",
21+
"ToolProcessorError",
22+
"ToolNotFoundError",
23+
"ToolExecutionError",
24+
"ToolTimeoutError",
25+
"ToolValidationError",
26+
"ParserError",
27+
"ToolRateLimitedError",
28+
"ToolCircuitOpenError",
29+
"MCPError",
30+
"MCPConnectionError",
31+
"MCPTimeoutError",
32+
]

0 commit comments

Comments
 (0)