Pain

A high-performance distributed storage system designed for scalability and reliability.

System Requirements

Prerequisites

The following tools and dependencies are required for building and running Pain:

• bazel(>=8.3) - Build system
• Python 3 - Script execution and automation
• Ansible - Deployment automation
• Docker - Container runtime

Installation Instructions

# Install bazel
# Please refer to https://bazel.build/install

# Install Ansible automation tool
sudo apt install ansible

# Install Prometheus monitoring roles
ansible-galaxy collection install prometheus.prometheus

# Install development tools and system packages
sudo apt install -y clang-format-16
sudo apt install -y libaio-dev
sudo apt install -y uuid-dev
sudo apt install -y libibverbs-dev
sudo apt install -y librdmacm-dev
sudo apt install -y libelf-dev
sudo apt install -y libssl-dev
sudo apt install -y liblz4-dev
sudo apt install -y libnuma-dev
sudo apt install -y python3
sudo apt install -y python3-pip
sudo apt install -y patchelf
sudo apt install -y python3-pyelftools
sudo apt install -y meson
sudo apt install -y nasm
sudo apt install -y ccache
sudo apt install -y autoconf automake libtool pkg-config
sudo apt install -y openssh-server 

Build and Development

Environment Variable

export CC=$(which gcc)
export CXX=$(which g++)

Core Build Commands

# set build cache dir
# /mnt/bazel_cache is default path
export PAIN_BAZEL_CACHE_DIR=/mnt/bazel_cache

# Build the project
./z.py build    # or ./z.py b

# Run test suite
./z.py test     # or ./z.py t

# Install the system
./z.py install  # or ./z.py i

# Deploy Pain to ~/deployment directory
./z.py deploy -a start

Additional Development Commands

# Display available build targets
./z.py show     # or ./z.py s

# Build specific target (e.g., sad component)
./z.py b sad

# Execute specific test suite
./z.py test test_manusya    # or ./z.py t test_manusya

# Run complete test suite
./z.py t

# Format source code (recommended before submitting pull requests)
./z.py format   # or ./z.py f

# Perform code quality analysis
./z.py lint     # or ./z.py l

For comprehensive command information, execute ./z.py -h.

Deployment

Pain provides an automated Ansible-based deployment solution for establishing minimal cluster configurations.

Deployment Prerequisites

# Configure SSH key authentication
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

# Install Prometheus monitoring collection
ansible-galaxy collection install prometheus.prometheus

Cluster Deployment Commands

# Deploy core components (deva/manusya)
ansible-playbook -i ./deploy/hosts ./deploy/deploy.yml -t start

# Deploy Jaeger distributed tracing
ansible-playbook -i ./deploy/hosts ./deploy/deploy.yml -t start-jaeger

# Deploy Prometheus monitoring
ansible-playbook -i ./deploy/hosts ./deploy/deploy.yml -t start-prometheus

Observability and Tracing

Pain integrates with OpenTelemetry for comprehensive distributed tracing capabilities. Jaeger can be deployed using the following Docker command:

docker run --rm \
  -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  -p 9411:9411 \
  jaegertracing/all-in-one:latest

Software Development Kit (SDK)

C++ Integration Example

The following example demonstrates basic file operations using the Pain SDK:

#include <gflags/gflags.h>
#include <pain/base/plog.h>
#include <pain/base/scope_exit.h>
#include <pain/base/tracer.h>
#include <pain/pain.h>
#include <memory>
#include <spdlog/spdlog.h>

DEFINE_string(filesystem, "list://192.168.10.1:8001,192.168.10.2:8001,192.168.10.3:8001", "filesystem");

int main(int argc, char* argv[]) {
    gflags::ParseCommandLineFlags(&argc, &argv, true);
    spdlog::set_level(spdlog::level::debug);
    pain::init_tracer("pain_demo");
    SCOPE_EXIT {
        PLOG_WARN(("desc", "pain_demo exit"));
        pain::cleanup_tracer();
    };

    pain::FileSystem* fs = nullptr;
    auto status = pain::FileSystem::create(FLAGS_filesystem.c_str(), &fs);
    if (!status.ok()) {
        std::cerr << "create filesystem failed: " << status.error_str() << std::endl;
        return 1;
    }
    std::unique_ptr<pain::FileSystem> fs_guard(fs);
    pain::FileStream* file = nullptr;
    status = fs->open("/hello.txt", O_CREAT | O_WRONLY, &file);
    if (!status.ok()) {
        std::cerr << "open file failed: " << status.error_str() << std::endl;
        return 1;
    }
    std::unique_ptr<pain::FileStream> file_guard(file);
    pain::proto::FileService::Stub stub(file);
    pain::Controller cntl;
    pain::proto::AppendRequest request;
    pain::proto::AppendResponse response;
    cntl.request_attachment().append("hello world");
    stub.Append(&cntl, &request, &response, nullptr);
    if (cntl.Failed()) {
        file->close();
        std::cerr << "append failed: " << cntl.ErrorText() << std::endl;
        return 1;
    }
    std::cout << "append success, offset: " << response.offset() << std::endl;
    file->close();
    return 0;
}

Contributing

We welcome contributions from the community. Please refer to our Contributing Guidelines for detailed information on how to participate in the project development.

License

This project is licensed under the terms specified in the LICENSE file.

Support

For technical support and community discussions, please refer to the project's GitHub Issues and Discussions sections.