Skip to content

Commit f3aa3ad

Browse files
committed
Update LLM Agent Alerts section in README
- Refactor LLM Alerts section to focus on anomaly-agent approach - Update description to highlight new agent-based anomaly detection - Remove references to OpenAI and Anthropic direct integrations - Add note about future Ollama support - Adjust formatting and terminology
1 parent 92d9782 commit f3aa3ad

File tree

1 file changed

+7
-10
lines changed

1 file changed

+7
-10
lines changed

README.md

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ Painless open source anomaly detection for your metrics! 📈📉🚀
3636
- [Visualization](#visualization)
3737
- [Concepts](#concepts)
3838
- [Alerts](#alerts)
39-
- [LLM Alerts](#llm-alerts)
39+
- [LLM Agent Alerts](#llm-agent-alerts)
4040
- [Contributing](#contributing)
4141

4242
Supported sources and databases for your metrics to live in and be queried from:
@@ -172,7 +172,7 @@ Here is a list of features of Anomstack (emoji alert warning!)
172172
5. 🛠️ - Ability to define your own custom python preprocess function instead of the default at [`/metrics/defaults/python/preprocess.py`](./metrics/defaults/python/preprocess.py).
173173
6. 📧 - Email [alerting](#alerts) with fancy(ish) ascii art plots of your metrics and anomaly scores.
174174
7. 💬 - Slack alerts too (want to make these nicer).
175-
8. 🤖 - LLM based alerts ([OpenAI](./anomstack/llm/openai.py) & [Anthropic](./anomstack/llm/anthropic.py)) - see [LLM Alerts](#llm-alerts). p.s. they don't work great yet - experimental :)
175+
8. 🤖 - aGeNtIc LLM based alerts - use an [anomaly-agent](https://github.com/andrewm4894/anomaly-agent) to do anomaly detection and alerting - see [LLM Agent Alerts](#llm-agent-alerts).
176176
9. 🕒 - Ability to ingest at whatever frequency you want and then agg to a different level for training/scoring, see [`freq`](/metrics/examples/freq/README.md) example.
177177
10. 📊 - Plot jobs so you can just eyeball your metrics in Dagster job logs, see [#dagster-ui-plots](#dagster-ui-plots).
178178
11. 🏗️ - Minimal infrastructure requirements, Anomstack just reads from and writes to whatever database you use.
@@ -557,21 +557,18 @@ Below is an example of an alert via email. Attached is a png plot with more deta
557557

558558
![plot](./docs/img/random_1.png)
559559

560-
## LLM Alerts
560+
## LLM Agent Alerts
561561

562562
[back to top](#anomstack)
563563

564-
Yes! I have managed to find a way to ram a large language model (LLM) into this project. But you know what, it might just work...
564+
Yes! I have managed to find a way to ram a large language (LLM) ~~model~~ 🚀AGENT🚀 into this project.
565565

566-
~~**Update**: It works horribly, but it works! 🤣. Still need to do a lot more prompt engineering to get this to work well, but it's a start.~~
567-
568-
**Update Update**: I know how to make this work much better and more reliable + latest models are better - going to refactor this soon (done [here](https://github.com/andrewm4894/anomstack/pull/127)).
569-
570-
Idea here is to just send the metric data and prompt to a LLM (ChatGPT) and ask it if it thinks the metric looks anomalous (and provide back an explanation). If it does, we alert.
566+
Idea here is to just send the metric data and prompt to an LLM Agent (built with the [anomaly-agent](https://github.com/andrewm4894/anomaly-agent)) and ask it if it thinks the metric looks anomalous (and run a verification chain to check it and also provide back an explanation for each anomaly). If it does, we alert.
571567

572568
Notes:
573569
- If you don't want to send your metric data to OpenAI then just set `disable_llmalert` to `True` in your metric batch config.
574-
- Support for Anthropic models added [here](https://github.com/andrewm4894/anomstack/pull/128)
570+
- Going to add and validate [Ollama](https://ollama.com/) support soon for local LLM Agent.
571+
- Will be developing this more in its own library (for use in both Anomstack and any other projects) over at [anomaly-agent](https://github.com/andrewm4894/anomaly-agent) dig in there for more details.
575572

576573
<details>
577574
<summary>Click to see some LLM Alert screenshots</summary>

0 commit comments

Comments
 (0)