Support Separate Process for Reward Computation via Inter-Process Communication #146
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement Inter-Process Communication (IPC) Module Between GFlowNet and Reward Function
This PR introduces an inter-process communication (IPC) module that enables reward computation in a separate process from GFlowNet.
Implementation Details
The implementation includes three key classes (
gflownet.utils.communication
):IPCModule
– A core module for communication between GFlowNet and the reward function.IPCTask
– A GFlowNet task withIPCModule
.RewardModule
– A reward function withIPCModule
.Supported IPC Methods
The current implementation supports three IPC methods:
NetworkIPC
– Uses a TCP/IP protocol (without file system). Recommended.FileSystemIPC
– Uses the file system with serialized data. Recommended.FileSystemIPC_CSV
– Uses the file system (csv format). It may be useful for non-Python reward functions or human feedback integration.User can setup IPC at
config.communication
.There are two example scripts:
src/gflownet/tasks/seh_frag_ipc.py
,src/gflownet/tasks/seh_frag_moo_ipc.py
Impact on Existing Code
This PR introduces minimal changes to the existing codebase. The key differences compared to the latest trunk are:
<2.0.0
) inpyproject.toml
.communication
toconfig
.