Super Report

An agentic application for analyzing dmesg and opensm logs in AI data centers, generating comprehensive human-readable reports using pydantic-ai.

Features

• Intelligent Log Analysis: Uses AI to understand and categorize log entries
• Automated Pattern Recognition: Generates whitelist rules from example logs
• Multi-format Output: HTML, JSON, and Markdown reports
• Rich Visualizations: Charts and graphs for better insights
• Command-line Interface: Easy-to-use CLI with rich output
• Extensible Architecture: Modular design for easy customization

Workflow

1. Pattern Generation: Scan example logs to create whitelist regex patterns
2. Log Filtering: Apply patterns to filter input logs, keeping only essential messages
3. Analysis: Analyze filtered logs to identify issues, patterns, and trends
4. Visualization: Generate charts and graphs for visual insights
5. Report Generation: Create comprehensive reports in multiple formats

Installation

# Clone the repository
git clone <repository-url>
cd super_report

# Install dependencies
pip install -e .

# Or install with development dependencies
pip install -e ".[dev]"

Configuration

Set your OpenAI API key and optionally a custom endpoint:

# Required: API key
export OPENAI_API_KEY="your-api-key-here"

# Optional: Custom OpenAI-compatible endpoint
export OPENAI_BASE_URL="http://localhost:8000/v1"

OpenAI-Compatible Endpoints

Super Report supports any OpenAI-compatible API:

• OpenAI: https://api.openai.com/v1 (default)
• Azure OpenAI: https://your-resource.openai.azure.com/
• Local models: http://localhost:8000/v1 (vLLM, Ollama, etc.)
• Other providers: Any service with OpenAI-compatible API

Logfire Integration

For enhanced logging and tracing, install with logfire support:

pip install "super-report[logfire]"

Enable logfire in your commands:

super-report analyze /var/log/dmesg --enable-logfire

Usage

Basic Analysis

# Analyze log files
super-report analyze /var/log/dmesg /var/log/opensm.log

# With example files for better pattern generation
super-report analyze /var/log/dmesg -e examples/sample_dmesg.log

# Generate JSON report
super-report analyze /var/log/dmesg --format json --output report.json

# Use different AI model
super-report analyze /var/log/dmesg --model gpt-3.5-turbo

# Use custom OpenAI-compatible endpoint
super-report analyze /var/log/dmesg --base-url http://localhost:8000/v1

# Enable logfire logging
super-report analyze /var/log/dmesg --enable-logfire

Preview Log Files

# Preview log format
super-report preview /var/log/dmesg --lines 20

Get Help

# Show examples and usage
super-report examples

# Show help
super-report --help
super-report analyze --help

Project Structure

super_report/
├── super_report/
│   ├── __init__.py          # Package initialization
│   ├── models.py            # Pydantic data models
│   ├── agent.py             # Main agent implementation
│   ├── tools.py             # Chart generation and rendering tools
│   └── cli.py               # Command-line interface
├── examples/
│   ├── sample_dmesg.log     # Example dmesg log
│   └── sample_opensm.log    # Example opensm log
├── pyproject.toml           # Project configuration
└── README.md               # This file

Data Models

The application uses Pydantic models for type safety and validation:

• LogType: Enum for supported log types (dmesg, opensm)
• LogSeverity: Severity levels (critical, error, warning, info, debug)
• WhitelistRule: Regex patterns for log filtering
• FilteredLogEntry: Processed log entries with metadata
• LogAnalysisResult: Analysis results and insights
• ReportOutput: Final report structure

Agent Workflow

The main agent orchestrates the analysis through these steps:

1. generate_whitelist_rules(): Create filtering patterns from examples
2. filter_logs(): Apply patterns to input logs
3. analyze_filtered_logs(): Extract insights and identify issues
4. generate_charts(): Create visualizations
5. render_report(): Generate final report

Supported Log Formats

dmesg Logs

• Format: [timestamp] component: message
• Focuses on kernel messages, hardware errors, driver issues

opensm Logs

• Format: timestamp opensm[pid]: message
• Focuses on InfiniBand subnet management, connectivity issues

Chart Types

• Pie Charts: Severity distribution
• Bar Charts: Issues by component
• Timeline Charts: Events over time
• Heatmaps: Pattern analysis

Report Formats

HTML Report

• Interactive charts with Plotly
• Rich styling and formatting
• Executive summary and recommendations
• Detailed analysis sections

JSON Report

• Machine-readable format
• Complete data export
• API integration friendly

Markdown Report

• Human-readable text format
• Version control friendly
• Documentation integration

Development

Setup Development Environment

# Install with development dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Format code
black super_report/
isort super_report/

# Type checking
mypy super_report/

Adding New Log Types

1. Add new enum value to LogType in models.py
2. Implement detection logic in agent.py
3. Add parsing rules in helper methods
4. Update CLI help and examples

Extending Analysis

1. Add new fields to LogAnalysisResult model
2. Implement analysis logic in analyze_filtered_logs()
3. Update report templates in tools.py
4. Add corresponding visualizations

Examples

See the examples/ directory for sample log files that demonstrate the supported formats and typical issues found in AI data center environments.

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

License

MIT

Support

For issues and questions:

1. Check the examples with super-report examples
2. Preview your log format with super-report preview
3. Review the generated reports for insights
4. Open an issue on GitHub for bugs or feature requests