Skip to content

test(W-18772941): test against LLM Gateway #60

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 35 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
cc4eba1
feat: add telemetry
mdonnalley Jun 10, 2025
1e8e9a2
Merge branch 'main' into mdonnalley/telemetry
mdonnalley Jun 10, 2025
e15d1e8
fix: ensure all events have same props
mdonnalley Jun 10, 2025
5ba0e32
chore: clean up
mdonnalley Jun 10, 2025
4df91ce
feat: add runtimeMs
mdonnalley Jun 11, 2025
e5c062a
fix: handle failed connection to appinsights
mdonnalley Jun 11, 2025
8e393fa
feat: logging
mdonnalley Jun 12, 2025
fe2d52b
chore: extract method signatures
cristiand391 Jun 12, 2025
5e85f75
chore: code review
mdonnalley Jun 12, 2025
35aad85
Merge branch 'mdonnalley/telemetry' into mdonnalley/logging
mdonnalley Jun 12, 2025
c6d7f79
feat: add client info to telemetry events
mdonnalley Jun 12, 2025
7eda5d1
fix: init telemetry in catch
mdonnalley Jun 12, 2025
05d188b
fix: consolidate TOOL_CALLED and TOOL_ERROR
mdonnalley Jun 12, 2025
6a2c895
Merge branch 'mdonnalley/telemetry' into mdonnalley/logging
mdonnalley Jun 12, 2025
024fa77
feat: count tokens of each tool
mdonnalley Jun 12, 2025
a909d69
chore: clean up token count logging
mdonnalley Jun 13, 2025
14ce216
Merge branch 'main' into mdonnalley/logging
mdonnalley Jun 13, 2025
b7dc943
Merge remote-tracking branch 'origin/main' into mdonnalley/logging
cristiand391 Jun 16, 2025
c5bb910
chore: bump core
cristiand391 Jun 16, 2025
e93b915
fix: remove faulty token counting
mdonnalley Jun 16, 2025
fd6d746
Merge branch 'mdonnalley/logging' of github.com:salesforcecli/mcp int…
mdonnalley Jun 16, 2025
91523e2
fix: set SF_LOG_LEVEL when using --debug
mdonnalley Jun 16, 2025
03b7a3d
test: add testing against LLM Gateway
mdonnalley Jun 16, 2025
394cb50
Merge branch 'main' into mdonnalley/llmg
mdonnalley Jun 16, 2025
5cefe3d
chore: clean up
mdonnalley Jun 16, 2025
15aa440
test: list token counts
mdonnalley Jun 16, 2025
d18028f
test: make more developer friendly
mdonnalley Jun 16, 2025
bb3754a
chore: clean up
mdonnalley Jun 17, 2025
0b534d1
chore: clean up
mdonnalley Jun 17, 2025
4856d5b
chore: clean up
mdonnalley Jun 17, 2025
3ce1fc8
chore: make tables prettier
mdonnalley Jun 17, 2025
f66b793
test: allow longer chats
mdonnalley Jun 18, 2025
a6b3705
test: clean up implementation
mdonnalley Jun 18, 2025
6f86505
test: add --entry-point flag
mdonnalley Jun 18, 2025
1446d75
Merge branch 'main' into mdonnalley/llmg
mdonnalley Jun 26, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,4 @@ node_modules

.cursor
src/tools/test.ts
llmg-test.yml
8 changes: 6 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@
"prepare": "sf-install",
"start": "yarn build && npm link && mcp-inspector sf-mcp-server",
"test": "wireit",
"test:only": "wireit"
"test:only": "wireit",
"test:llmg": "node --no-warnings --loader ts-node/esm test/llmg.ts"
},
"repository": "salesforcecli/mcp",
"bugs": {
Expand Down Expand Up @@ -54,15 +55,18 @@
},
"devDependencies": {
"@modelcontextprotocol/inspector": "^0.14.3",
"@oclif/table": "^0.4.8",
"@salesforce/cli-plugins-testkit": "^5.3.39",
"@salesforce/dev-scripts": "11.0.2",
"@types/node": "^22.15.32",
"eslint-config-salesforce-license": "^1.0.1",
"eslint-plugin-sf-plugin": "^1.20.25",
"gpt-tokenizer": "^3.0.1",
"oclif": "^4.18.0",
"ts-node": "^10.9.2",
"ts-patch": "^3.3.0",
"typescript": "^5.8.3"
"typescript": "^5.8.3",
"yaml": "^2.8.0"
},
"publishConfig": {
"access": "public"
Expand Down
Loading
Loading