Skip to content

Support Separate Process for Reward Computation via Inter-Process Communication #146

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: trunk
Choose a base branch
from

Conversation

SeonghwanSeo
Copy link

@SeonghwanSeo SeonghwanSeo commented Feb 19, 2025

Implement Inter-Process Communication (IPC) Module Between GFlowNet and Reward Function

This PR introduces an inter-process communication (IPC) module that enables reward computation in a separate process from GFlowNet.

Implementation Details

The implementation includes three key classes (gflownet.utils.communication):

  • IPCModule – A core module for communication between GFlowNet and the reward function.
  • IPCTask – A GFlowNet task with IPCModule.
  • RewardModule – A reward function with IPCModule.

Supported IPC Methods

The current implementation supports three IPC methods:

  1. NetworkIPC – Uses a TCP/IP protocol (without file system). Recommended.
  2. FileSystemIPC – Uses the file system with serialized data. Recommended.
  3. FileSystemIPC_CSV – Uses the file system (csv format). It may be useful for non-Python reward functions or human feedback integration.

User can setup IPC at config.communication.

There are two example scripts: src/gflownet/tasks/seh_frag_ipc.py, src/gflownet/tasks/seh_frag_moo_ipc.py

Impact on Existing Code

This PR introduces minimal changes to the existing codebase. The key differences compared to the latest trunk are:

  1. Added a numpy version constraint (<2.0.0) in pyproject.toml.
  2. Add a field communication to config.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant