Add Integration Tests

[Sneagan]

• Kafka token issuance
• Kafka token redemption
• V1 HTTP Issuer GET endpoint
• V1 HTTP Token Redemption POST endpoint
• V3 HTTP Token Redemption POST endpoint

This checks the entire token redemption flow for all known production
use cases as well as checks that duplicate redemtion fails. There are
some cases with time limited issuers where a token is issued for a
future issuer whose redemption fails. These tokens are tested and
verified to fail as expected.

[socket-security]

All alerts resolved. Learn more about Socket for GitHub.

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

View full report

[Copilot]

Pull Request Overview

This PR adds comprehensive integration tests for the challenge-bypass system, including complete end-to-end flows for token issuance and redemption via Kafka and HTTP endpoints. The tests verify the entire system working together with all dependencies (PostgreSQL, Kafka, DynamoDB).

• Added full integration test suite covering Kafka token flows and HTTP endpoints
• Enhanced logging and debugging capabilities for development and testing
• Updated build configuration and CI pipeline to support integration testing

Reviewed Changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
integration-tests/intergration_test.go	New comprehensive integration test file with end-to-end token flows
docker-compose.integration.yml	Test environment configuration with all required services
Dockerfile.test	Test-specific container configuration for running integration tests
Makefile	Added integration test commands and cleanup utilities
.github/workflows/integration-tests.yml	CI workflow for automated integration testing
README.md	Updated documentation with integration test instructions
utils/test/dynamodb.go	Adjusted timeout for table activation in test environment
server/.go, kafka/.go	Enhanced logging and debugging output for integration tests
go.mod	Updated testify dependency to latest version

[kdenhartog]

I can't see any issues with the postgres warnings and I see v3 was pinned

This LGTM from security perspective

[clD11]

Approved with comments

[github-actions]

[puLL-Merge] - brave-intl/challenge-bypass-server@816

Description

This PR adds comprehensive integration tests to the challenge-bypass-server project. The changes introduce a new GitHub Actions workflow, Docker-based integration test setup, and extensive test coverage for the token issuance and redemption flows. The integration tests verify end-to-end functionality including Kafka messaging, database operations, and HTTP endpoints using containerized services (PostgreSQL, Kafka, Zookeeper, LocalStack for DynamoDB).

Possible Issues

• Build dependencies: The Rust builder stage in Dockerfile.test uses a hardcoded version (1.0.1) of the challenge-bypass-ristretto-ffi library, which could become outdated
• Test reliability: The integration tests rely on timing-based waits (10-second loop) instead of proper health checks for service readiness
• Resource consumption: The tests create and tear down multiple containers for each test run, which could be resource-intensive in CI environments
• Test isolation: Multiple test functions share the same Kafka topics and database, potentially causing interference between tests
• Error handling: Some error cases use require.NoError where more specific error assertions might be more appropriate

Security Hotspots

• Test credentials: Hardcoded AWS credentials (test/test) in docker-compose.integration.yml - while acceptable for testing, ensure these are only used in test environments
• Network exposure: The integration test setup exposes multiple ports (2416, 9090, 5432, 9092, etc.) that could potentially be accessed by other processes during testing
• DynamoDB endpoint: The test uses LocalStack with hardcoded endpoint configuration - ensure this doesn't leak into production configurations

Changes

Changes

• .github/workflows/integration-tests.yml: New GitHub Actions workflow that runs integration tests on PRs to master/production branches
• Dockerfile.test: New multi-stage Docker image for running integration tests with Rust FFI library and Go test dependencies
• Makefile: Added comprehensive integration test commands with proper cleanup and logging capabilities
• README.md: Extensive documentation for running and understanding integration tests
• docker-compose.integration.yml: Complete test environment setup with PostgreSQL, Kafka, Zookeeper, LocalStack, and the application
• go.mod/go.sum: Updated testify dependency to v1.10.0
• integration-tests/integration_test.go: Comprehensive test suite covering token issuance via Kafka and redemption via both Kafka and HTTP endpoints
• kafka/main.go: Modified Prometheus registry handling for test environments to avoid conflicts
• kafka/signed_blinded_token_issuer_handler.go: Enhanced error logging for invalid token count scenarios
• kafka/signed_token_redeem_handler.go: Improved error handling and logging for redemption failures
• server/issuers.go: Removed redundant error logging for issuer not found cases
• server/tokens.go: Added debug logging for token verification failures
• utils/test/dynamodb.go: Improved DynamoDB table creation timing for test reliability

sequenceDiagram
    participant GH as GitHub Actions
    participant Docker as Docker Compose
    participant Kafka as Kafka
    participant DB as PostgreSQL
    participant DDB as LocalStack/DynamoDB
    participant CBP as Challenge Bypass Server
    participant Tests as Integration Tests

    GH->>Docker: make integration-test
    Docker->>DB: Start PostgreSQL
    Docker->>Kafka: Start Kafka + Zookeeper
    Docker->>DDB: Start LocalStack
    Docker->>CBP: Start application
    
    Note over Docker: Wait for services to be ready
    
    Docker->>Tests: Build and run test container
    
    Tests->>CBP: POST /v3/issuer/ (create issuer)
    CBP->>DB: Store issuer configuration
    
    Tests->>Kafka: Send token signing request
    CBP->>Kafka: Consume signing request
    CBP->>CBP: Generate signed tokens
    CBP->>Kafka: Publish signing results
    Tests->>Kafka: Consume signing results
    
    Tests->>CBP: POST /v1/blindedToken/{issuer}/redemption/
    CBP->>DB: Check token redemption
    CBP->>DDB: Store redemption record
    CBP-->>Tests: Return success response
    
    Tests->>Kafka: Send redemption request
    CBP->>Kafka: Consume redemption request
    CBP->>DB: Verify token and issuer
    CBP->>DDB: Check for duplicate redemption
    CBP->>Kafka: Publish redemption result
    Tests->>Kafka: Consume redemption result
    
    Tests->>CBP: POST same token (duplicate test)
    CBP-->>Tests: Return 409 Conflict
    
    Tests-->>Docker: Test completion
    Docker->>Docker: Clean up containers and volumes

Loading