DuckDB Age Extension

A DuckDB extension providing modern encryption functions using the age encryption specification. This extension combines native secret management with public/private encryption functions, implemented using a hybrid C++/Rust architecture using the age rust library. It makes it possible to insert data using only the public key that can't be read until the private key is used so any access to the database or even host machine doesn't allow the decryption of data.

Status: experimental, mostly generated by AI. Needs more testing and validation. Do not use for now.

Features

Core Encryption Functions

• Key Generation: Generate X25519 key pairs (age_keygen())
• Single Recipient Encryption: Encrypt data for one recipient (age_encrypt)
• Multi-Recipient Encryption: Encrypt data for multiple recipients (age_encrypt_multi)
• Decryption: Decrypt age-encrypted data (age_decrypt)
• Hybrid Architecture: High-performance Rust cryptography with DuckDB C++ integration

Secret Management Integration

• Native Secret Management: Integrates with DuckDB's built-in secret management system
• Age Key Validation: Validates age public keys (age1...) and private keys (AGE-SECRET-KEY-1...)
• File-based Keys: Support for reading keys from external files (recommended for security)
• Inline Keys: Support for inline key specification
• Key Redaction: Private keys are automatically redacted in logs and error messages
• Flexible Configuration: Mix and match inline keys with file-based keys

Security & Performance

• Modern Cryptography: X25519 (key exchange) + ChaCha20-Poly1305 (encryption)
• Age Format Compatibility: Full compatibility with standard age tools and libraries
• Comprehensive Error Handling: Detailed error messages for invalid keys and operations
• Memory Safe: Rust implementation with proper FFI memory management

Libraries and Dependencies

This extension leverages high-quality cryptographic libraries to provide secure, efficient age encryption:

Core Cryptographic Libraries

• rage (v0.11) - Pure Rust implementation of the age encryption specification

  • Provides the core age::Encryptor and age::Decryptor functionality
  • Implements X25519 key exchange and ChaCha20-Poly1305 encryption
  • Full compatibility with the age specification

• secrecy (v0.10) - Memory protection for sensitive data

  • Ensures private keys are properly zeroed from memory when dropped
  • Prevents accidental leakage of secrets in debug output or logs

• x25519-dalek (v2.0) - High-performance X25519 elliptic curve implementation

  • Used for X25519 key generation and exchange operations
  • Constant-time implementation resistant to timing attacks

Supporting Libraries

• rand (v0.8) - Cryptographically secure random number generation

  • Powers secure key pair generation in age_keygen()
  • Provides entropy for all cryptographic operations

• sha2 (v0.10) - SHA-2 family of hash functions

  • Used for deterministic key derivation in age_keygen_from_seed() (planned)
  • Provides cryptographic hashing for seed-based operations

• rand_chacha (v0.3) - ChaCha-based pseudorandom number generator

  • Used for deterministic random number generation from seeds
  • Ensures reproducible key generation when implemented

C++ Integration

• DuckDB Core - Native integration with DuckDB's secret management system
• FFI Layer - Safe memory management across C++/Rust boundary
• CMake Build System - Seamless integration with DuckDB's build process

Why These Libraries?

1. Security: All libraries are widely audited and used in production systems
2. Performance: Pure Rust implementations with optimized cryptographic primitives
3. Compatibility: Full adherence to age specification and interoperability with standard tools
4. Memory Safety: Rust's ownership system prevents common security vulnerabilities
5. Cross-Platform: Works consistently across different operating systems and architectures

Related Tools

This extension is compatible with standard age ecosystem tools:

• age - Original Go implementation by Filippo Valsorda
• rage - Rust implementation (same library we use)
• age-plugin-* - Various age plugins for hardware keys
• SSH integration - Can encrypt for SSH public keys (planned feature)

Installation

Building from Source

1. Clone the repository with submodules:

git clone --recurse-submodules https://github.com/your-repo/duckdb-age.git
cd duckdb-age

2. Build the extension:

make

The build produces:

• ./build/release/duckdb - DuckDB binary with age extension pre-loaded
• ./build/release/extension/age/age.duckdb_extension - Loadable extension binary

Usage

Loading the Extension

LOAD 'age';

Encryption Functions

For detailed documentation of all age encryption functions, see FUNCTIONS.md.

Quick Examples

Generate Keys

-- Generate a new key pair
SELECT age_keygen() AS keys;

Encrypt Data

-- Encrypt with raw public key
SELECT age_encrypt('secret data'::BLOB, 'age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p');

-- Encrypt with secret name
CREATE SECRET my_key (TYPE age, PUBLIC_KEY 'age1...', PRIVATE_KEY 'AGE-SECRET-KEY-1...');
SELECT age_encrypt('secret data'::BLOB, 'my_key');

Decrypt Data

-- Decrypt with raw private key
SELECT age_decrypt(encrypted_data, 'AGE-SECRET-KEY-1...');

-- Decrypt with secret name
SELECT age_decrypt(encrypted_data, 'my_key');

Multi-Recipient Encryption

-- Encrypt for multiple recipients
SELECT age_encrypt_multi('data'::BLOB, ['key1', 'key2', 'key3']);

Secret Management

Creating Age Secrets

Using Inline Keys

CREATE SECRET my_age_key (
    TYPE 'age',
    public_key 'age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p',
    private_key 'AGE-SECRET-KEY-1QTAYQ69LA4P3QQN0VQPSJMG2WHVSQPQ3SG2F55M0XWDE9VQN0SZQCGUGJ8',
    key_id 'personal_key'
);

Using File-based Keys (Recommended)

CREATE SECRET my_age_key (
    TYPE 'age',
    public_key_file '/path/to/public_key.txt',
    private_key_file '/path/to/private_key.txt',
    key_id 'file_key'
);

Mixed Approach

CREATE SECRET mixed_key (
    TYPE 'age',
    public_key 'age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p',
    private_key_file '/path/to/private_key.txt'
);

Managing Secrets

List Age Secrets

SELECT name, type, provider FROM duckdb_secrets() WHERE type = 'age';

Drop Secrets

DROP SECRET my_age_key;

Parameters

Parameter	Type	Required	Description
public_key	VARCHAR	No*	Age public key (starts with age1)
private_key	VARCHAR	No*	Age private key (starts with AGE-SECRET-KEY-1)
public_key_file	VARCHAR	No*	Path to file containing public key
private_key_file	VARCHAR	No*	Path to file containing private key
key_id	VARCHAR	No	Optional identifier for the key pair

*At least one key (public or private) must be specified. Cannot specify both inline and file versions of the same key type.

Validation Rules

• Public keys must start with age1
• Private keys must start with AGE-SECRET-KEY-1
• Cannot specify both public_key and public_key_file
• Cannot specify both private_key and private_key_file
• Key files must exist and be readable
• Key files have a 1MB size limit
• Key files are automatically trimmed of whitespace

Testing

Running Tests

Quick Test Script

Run the comprehensive test script:

./test_extension.sh

Manual Testing via Make

Run the complete test suite:

make test

Individual Test Files

Run specific test files using the unittest runner:

# Note: .test files use DuckDB's test format and require the unittest runner
# Manual SQL testing is recommended for development

Manual Testing

1. Start DuckDB with the extension:

./build/release/duckdb

2. Test basic functionality:

-- Load extension
LOAD 'age';

-- Verify extension loads
SELECT age_version();

-- Test key generation
SELECT age_keygen() AS keys;

-- Test encryption functions
WITH keys AS (SELECT age_keygen() AS kp)
SELECT 
    age_encrypt('hello world'::BLOB, (kp).public_key) AS encrypted_data,
    (kp).private_key AS private_key
FROM keys;

-- Test decryption 
WITH keys AS (SELECT age_keygen() AS kp),
     encrypted AS (
         SELECT age_encrypt('test message'::BLOB, (kp).public_key) AS data,
                (kp).private_key AS priv
         FROM keys
     )
SELECT age_decrypt(data, priv) = 'test message'::BLOB AS round_trip_success
FROM encrypted;

-- Test secret creation and usage
CREATE SECRET test_key (
    TYPE 'age',
    PUBLIC_KEY 'age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p',
    PRIVATE_KEY 'AGE-SECRET-KEY-1QTAYQ69LA4P3QQN0VQPSJMG2WHVSQPQ3SG2F55M0XWDE9VQN0SZQCGUGJ8'
);

-- Test encryption with secret names
SELECT age_encrypt('secret data'::BLOB, 'test_key') IS NOT NULL AS encrypted;

-- Test multi-recipient encryption
SELECT age_encrypt_multi('multi test'::BLOB, ['test_key', 'age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p']) IS NOT NULL AS multi_encrypted;

-- Clean up
DROP SECRET test_key;

Test Coverage

The test suite covers:

• Encryption Functions: Key generation, encryption, decryption, multi-recipient encryption
• Secret Integration: Using secret names in encryption functions
• Error Handling: Invalid keys, malformed data, empty parameters
• Round-trip Testing: Encrypt/decrypt validation
• Secret Management: Creation with inline keys, file-based key loading
• Validation: Key format validation, mutual exclusivity
• Edge Cases: Empty recipients, wrong keys, missing secrets

Integration with Other Extensions

This extension provides both standalone encryption functions and secret management integration:

-- Standalone usage
LOAD 'age';

-- Generate keys and encrypt data directly
WITH keys AS (SELECT age_keygen() AS kp)
SELECT age_encrypt('sensitive data'::BLOB, (kp).public_key) AS encrypted;

-- Create secrets for reusable keys
CREATE SECRET company_key (
    TYPE 'age',
    PUBLIC_KEY 'age1...',
    PRIVATE_KEY 'AGE-SECRET-KEY-1...'
);

-- Use secrets in encryption functions
SELECT age_encrypt('confidential'::BLOB, 'company_key');
SELECT age_encrypt_multi('team data'::BLOB, ['company_key', 'backup_key']);

Security Considerations

• File-based keys are recommended over inline keys for production use
• Private keys are automatically marked for redaction in logs
• Use appropriate file permissions (e.g., 600) on key files
• Store key files outside the database directory
• Consider using dedicated key management systems for production deployments

Development

Project Structure

duckdb-age/
├── src/
│   ├── age_extension.cpp      # Main C++ extension implementation
│   └── include/
│       └── age_extension.hpp  # Extension header
├── rust/
│   ├── src/lib.rs            # Rust FFI implementation
│   └── Cargo.toml           # Rust dependencies
├── test/sql/
│   ├── age.test              # Encryption function tests
│   └── age_secret.test       # Secret functionality tests
├── FUNCTIONS.md              # Detailed function documentation
├── extension_config.cmake    # Extension configuration
└── README.md                # This file

Building for Development

1. Make changes to source files
2. Rebuild: make
3. Test: make test
4. Run specific tests: ./build/release/duckdb < test/sql/age_secret.test

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Ensure all tests pass
6. Submit a pull request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Related Projects

• age - Modern encryption tool
• DuckDB - In-process SQL OLAP database
• DuckDB Extensions - DuckDB extension ecosystem