pre, post, tool, rag security check #159

garland3 · 2025-12-07T21:21:24Z

No description provided.

backend/tests/test_orchestrator_security_integration.py

backend/tests/test_tools_mode_security.py

mocks/security_check_mock/app.py

github-actions · 2025-12-07T21:24:43Z

🔒 Security Scan Results

Security Scan Summary

Scan Results

Python SAST (Bandit)

⚠️ Security issues found in Python code

Recommendations

Review all SARIF files uploaded to GitHub Security tab
Address high and critical severity vulnerabilities immediately
Run npm audit fix and pip-audit locally to fix dependencies
Consider implementing additional security controls

Copilot

Pull request overview

This PR implements a comprehensive content security check system for Atlas UI 3, enabling pre-check and post-check moderation for user input, LLM output, and tool/RAG outputs. The implementation addresses a critical bug where security notifications were calling a non-existent publish_message() method instead of the correct send_json() API.

Key Changes:

Added security check service with external API integration for content moderation
Fixed WebSocket notification API bug (publish_message → send_json)
Created mock security check server and poisoned tool/RAG sources for testing
Added comprehensive test coverage with regression tests

Reviewed changes

Copilot reviewed 30 out of 32 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
`backend/core/security_check.py`	New security check service with fail-open design and three check types (input, output, tool/RAG)
`backend/application/chat/orchestrator.py`	Integrated security checks for input and output, using correct send_json() API
`backend/application/chat/modes/tools.py`	Added tool output security validation before sending to LLM
`backend/application/chat/modes/rag.py`	Added security_check_service parameter for future RAG validation
`backend/application/chat/service.py`	Initialize and pass security check service to mode runners
`backend/modules/config/config_manager.py`	Added feature flags and API configuration for security checks
`backend/tests/test_security_check.py`	Unit tests for SecurityCheckService (20 tests covering all scenarios)
`backend/tests/test_orchestrator_security_integration.py`	Integration tests for orchestrator security checks with regression tests for API bug
`backend/tests/test_tools_mode_security.py`	Skipped tests for tools mode (notes indicate complex mocking requirements)
`frontend/src/handlers/chat/websocketHandlers.js`	Added security_warning message handler for UI notifications
`frontend/src/components/Message.jsx`	Added rendering for security warning messages
`frontend/src/test/security-warning-message.test.jsx`	Frontend tests for security warning rendering
`mocks/security_check_mock/app.py`	Mock security check server with keyword-based and probabilistic endpoints
`mocks/security_check_mock/run.sh`	Shell script to run mock server (follows conventions)
`backend/mcp/poisoned-tool/main.py`	Test MCP server that returns dangerous content for security testing
`mocks/rag-mock/main_rag_mock.py`	Added poisoned_security_test data source for RAG security testing
`docs/admin/security-check.md`	Comprehensive documentation for security check feature
`docs/developer/security-check-manual-testing.md`	Manual testing guide for all security check types
`docs/developer/security-check-quick-test.md`	Quick reference for testing security checks
`docs/agent-implementation-summaries/*.md`	Implementation summaries documenting the feature and bugfix
`.env.example`	Added security check configuration options
`.github/copilot-instructions.md`, `CLAUDE.md`, `GEMINI.md`	Added implementation summary documentation requirement

Copilot · 2025-12-07T21:24:44Z

mocks/rag-mock/main_rag_mock.py

+            content_type="text",
+            confidence_score=0.99,
+            chunk_id="bomb_making_section",
+            last_modified="2024-12-07T00:00:00Z"


[nitpick] The date "2024-12-07" in the last_modified field is in the past (almost a year ago if current date is December 7, 2025), but this might be intentional test data. However, for test data created today, using a recent or current date would be more realistic.

Suggested change

last_modified="2024-12-07T00:00:00Z"

last_modified="2025-06-07T00:00:00Z"

Copilot · 2025-12-07T21:24:44Z

backend/tests/test_orchestrator_security_integration.py

+        )
+
+        # Execute - this should NOT raise AttributeError
+        result = await orchestrator.execute(


Variable result is not used.

Suggested change

result = await orchestrator.execute(

await orchestrator.execute(

Copilot · 2025-12-07T21:24:44Z

backend/tests/test_orchestrator_security_integration.py

+        )
+
+        # Execute
+        result = await orchestrator.execute(


Variable result is not used.

Suggested change

result = await orchestrator.execute(

await orchestrator.execute(

Copilot · 2025-12-07T21:24:45Z

backend/tests/test_orchestrator_security_integration.py

+        orchestrator = ChatOrchestrator(
+            llm=mock_llm,
+            event_publisher=mock_event_publisher,
+            session_repository=mock_session_repository,
+            security_check_service=mock_security_service,
+        )


Variable orchestrator is not used.

Suggested change

orchestrator = ChatOrchestrator(

llm=mock_llm,

event_publisher=mock_event_publisher,

session_repository=mock_session_repository,

security_check_service=mock_security_service,

)

Copilot · 2025-12-07T21:24:45Z

backend/tests/test_tools_mode_security.py

+        )
+
+        # Execute with tool results that will be checked
+        result = await runner.run(


Variable result is not used.

Suggested change

result = await runner.run(

await runner.run(

Copilot · 2025-12-07T21:24:45Z

mocks/security_check_mock/app.py

+import os
+import random
+
+from fastapi import FastAPI, Header, HTTPException, Request


Import of 'HTTPException' is not used.

Copilot · 2025-12-07T21:24:45Z

backend/tests/test_tools_mode_security.py

+"""
+
+import pytest
+from unittest.mock import AsyncMock, MagicMock


Import of 'MagicMock' is not used.

Suggested change

from unittest.mock import AsyncMock, MagicMock

from unittest.mock import AsyncMock

Copilot · 2025-12-07T21:24:46Z

backend/tests/test_tools_mode_security.py

+
+import pytest
+from unittest.mock import AsyncMock, MagicMock
+from uuid import uuid4


Import of 'uuid4' is not used.

Suggested change

from uuid import uuid4

Copilot · 2025-12-07T21:24:46Z

backend/tests/test_tools_mode_security.py

+
+from backend.application.chat.modes.tools import ToolsModeRunner
+from backend.core.security_check import (
+    SecurityCheckService,


Import of 'SecurityCheckService' is not used.

Suggested change

SecurityCheckService,

Copilot · 2025-12-07T21:24:46Z

backend/tests/test_tools_mode_security.py

+    SecurityCheckResponse,
+    SecurityCheckResult,
+)
+from backend.domain.sessions.models import Session


Import of 'Session' is not used.

Suggested change

from backend.domain.sessions.models import Session

- Consolidated shared models into models.py for reuse across mock servers. - Updated main.py and main_rate_limit.py to import shared models, reducing code duplication. - Introduced main_bad_llm.py for security testing, generating responses with problematic content. - Enhanced security check logic in app.py to include new keywords for warnings and blocks. - Added detailed documentation for the security check system and mock server usage. - Implemented a fail-open design for security checks to ensure system availability during API failures. - Created a comprehensive testing guide for manual verification of security checks.

mocks/llm-mock/main_bad_llm.py

+    if not messages:
+        return PROBLEMATIC_RESPONSES["default_bomb"]
+
+    last_message = messages[-1].content.lower()


To fix the unused local variable problem at line 58, we should remove the assignment statement that sets last_message. Because the assignment does not have side effects (it just performs string access and calls .lower() on a string property), we can safely delete the entire line (last_message = messages[-1].content.lower()). No additional imports, definitions, or logic changes are needed. Just remove line 58 from mocks/llm-mock/main_bad_llm.py within the generate_bad_response() function.

github-actions · 2025-12-07T21:46:12Z

🔒 Security Scan Results

Security Scan Summary

Scan Results

Python SAST (Bandit)

⚠️ Security issues found in Python code

Recommendations

Review all SARIF files uploaded to GitHub Security tab
Address high and critical severity vulnerabilities immediately
Run npm audit fix and pip-audit locally to fix dependencies
Consider implementing additional security controls

…shed to users

github-actions · 2025-12-07T22:35:55Z

🔒 Security Scan Results

Security Scan Summary

Scan Results

Python SAST (Bandit)

⚠️ Security issues found in Python code

Recommendations

Review all SARIF files uploaded to GitHub Security tab
Address high and critical severity vulnerabilities immediately
Run npm audit fix and pip-audit locally to fix dependencies
Consider implementing additional security controls

…ks to clear conversation history on blocked content

github-actions · 2025-12-07T23:38:13Z

🔒 Security Scan Results

Security Scan Summary

Scan Results

Python SAST (Bandit)

⚠️ Security issues found in Python code

Recommendations

Review all SARIF files uploaded to GitHub Security tab
Address high and critical severity vulnerabilities immediately
Run npm audit fix and pip-audit locally to fix dependencies
Consider implementing additional security controls

…or app settings

github-actions · 2025-12-08T04:21:15Z

🔒 Security Scan Results

Security Scan Summary

Scan Results

Python SAST (Bandit)

⚠️ Security issues found in Python code

Recommendations

Review all SARIF files uploaded to GitHub Security tab
Address high and critical severity vulnerabilities immediately
Run npm audit fix and pip-audit locally to fix dependencies
Consider implementing additional security controls

Squash all commits from copilot/add-security-checks-for-content

3c6828b

Copilot AI review requested due to automatic review settings December 7, 2025 21:21

Copilot started reviewing on behalf of garland3 December 7, 2025 21:21 View session

garland3 mentioned this pull request Dec 7, 2025

Add pre/post/tool/RAG security check for content moderation #141

Closed

github-advanced-security bot found potential problems Dec 7, 2025

View reviewed changes

Copilot AI reviewed Dec 7, 2025

View reviewed changes

github-advanced-security bot found potential problems Dec 7, 2025

View reviewed changes

Implement security checks to prevent blocked content from being publi…

063bcf1

…shed to users

Implement RAG service configuration updates and enhance security chec…

9cba38b

…ks to clear conversation history on blocked content

Refactor test configuration to consistently use mock_config_manager f…

e6ce8cc

…or app settings

garland3 changed the title ~~Squash all commits from copilot/add-security-checks-for-content~~ pre, post, tool, rag security check Dec 10, 2025

	last_modified="2024-12-07T00:00:00Z"
	last_modified="2025-06-07T00:00:00Z"

	result = await orchestrator.execute(
	await orchestrator.execute(

	from unittest.mock import AsyncMock, MagicMock
	from unittest.mock import AsyncMock

pre, post, tool, rag security check #159

Are you sure you want to change the base?

pre, post, tool, rag security check #159

Uh oh!

Conversation

garland3 commented Dec 7, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Dec 7, 2025

🔒 Security Scan Results

Security Scan Summary

Scan Results

Python SAST (Bandit)

Recommendations

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 7, 2025

Choose a reason for hiding this comment

Uh oh!

Check notice

Copilot Autofix

github-actions bot commented Dec 7, 2025

🔒 Security Scan Results

Security Scan Summary

Scan Results

Python SAST (Bandit)

Recommendations

Uh oh!

github-actions bot commented Dec 7, 2025

🔒 Security Scan Results

Security Scan Summary

Scan Results

Python SAST (Bandit)

Recommendations

Uh oh!

github-actions bot commented Dec 7, 2025

🔒 Security Scan Results

Security Scan Summary

Scan Results

Python SAST (Bandit)

Recommendations