Skip to content

Commit 7307035

Browse files
Add process sharing guide.
1 parent 5dca347 commit 7307035

File tree

2 files changed

+273
-0
lines changed

2 files changed

+273
-0
lines changed

guides/links.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,4 @@
11
getting-started:
22
order: 1
3+
process-sharing:
4+
order: 2

guides/process-sharing/readme.md

Lines changed: 271 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,271 @@
1+
# Process Memory Sharing
2+
3+
This guide demonstrates how to share memory between processes using `io-memory` for high-performance inter-process communication (IPC).
4+
5+
## Overview
6+
7+
`IO::Memory` enables true zero-copy memory sharing between processes by:
8+
9+
- Creating memory-mapped regions that persist beyond the creating process
10+
- Passing file descriptors through Unix domain sockets
11+
- Allowing multiple processes to map the same physical memory
12+
- Providing automatic cleanup when all references are closed
13+
14+
This is significantly faster than traditional IPC methods like pipes or message queues since no data copying occurs.
15+
16+
## Basic Process Sharing
17+
18+
Here's a simple example of sharing memory between a parent and child process:
19+
20+
```ruby
21+
require 'io/memory'
22+
23+
# Create shared memory buffer
24+
handle = IO::Memory.new(1024)
25+
buffer = handle.map
26+
27+
# Write data in parent process
28+
message = "Hello from parent!"
29+
buffer.set_string(message, 0)
30+
31+
pid = fork do
32+
# Child process automatically inherits the file descriptor
33+
child_buffer = handle.map
34+
35+
# Read parent's data
36+
received = child_buffer.get_string(0, message.length)
37+
puts "Child received: #{received}"
38+
39+
# Write response
40+
response = "Hello from child!"
41+
child_buffer.set_string(response, 100)
42+
end
43+
44+
Process.wait(pid)
45+
46+
# Parent can see child's response
47+
child_response = buffer.get_string(100, 17)
48+
puts "Parent received: #{child_response}"
49+
50+
handle.close
51+
```
52+
53+
## Unix Domain Socket File Descriptor Passing
54+
55+
For more flexible process communication, you can pass file descriptors through Unix sockets:
56+
57+
```ruby
58+
require 'io/memory'
59+
require 'socket'
60+
61+
def parent_process
62+
# Create shared memory
63+
handle = IO::Memory.new(2048)
64+
buffer = handle.map
65+
66+
# Write initial data
67+
buffer.set_string("Shared data from parent", 0)
68+
69+
# Create socket pair for communication
70+
parent_socket, child_socket = UNIXSocket.socketpair
71+
72+
pid = fork do
73+
parent_socket.close
74+
child_process(child_socket)
75+
end
76+
77+
child_socket.close
78+
79+
# Send file descriptor to child
80+
parent_socket.send_io(handle.io)
81+
82+
# Wait for child to process
83+
response = parent_socket.read(100)
84+
puts "Parent received: #{response}"
85+
86+
# Check if child modified the buffer
87+
child_data = buffer.get_string(1000, 20)
88+
puts "Child wrote: #{child_data}"
89+
90+
Process.wait(pid)
91+
parent_socket.close
92+
handle.close
93+
end
94+
95+
def child_process(socket)
96+
# Receive file descriptor from parent
97+
received_io = socket.recv_io
98+
99+
# Map the shared memory
100+
buffer = ::IO::Buffer.map(received_io, 2048)
101+
102+
# Read parent's data
103+
parent_data = buffer.get_string(0, 23)
104+
puts "Child read: #{parent_data}"
105+
106+
# Write data back
107+
buffer.set_string("Data from child proc", 1000)
108+
109+
# Send confirmation
110+
socket.write("Child processing complete")
111+
112+
received_io.close
113+
socket.close
114+
end
115+
116+
parent_process
117+
```
118+
119+
## Multi-Process Worker Pool
120+
121+
Here's a more advanced example showing a worker pool sharing memory:
122+
123+
```ruby
124+
require 'io/memory'
125+
require 'socket'
126+
127+
class SharedMemoryWorkerPool
128+
def initialize(worker_count: 4, buffer_size: 1024 * 1024)
129+
@worker_count = worker_count
130+
@buffer_size = buffer_size
131+
@workers = []
132+
@sockets = []
133+
134+
setup_shared_memory
135+
spawn_workers
136+
end
137+
138+
def submit_work(data, offset = 0)
139+
# Write work data to shared memory
140+
@buffer.set_string(data, offset)
141+
142+
# Signal a worker
143+
worker_socket = @sockets.sample
144+
worker_socket.write("WORK:#{offset}:#{data.length}\n")
145+
146+
# Read result
147+
response = worker_socket.readline.chomp
148+
response
149+
end
150+
151+
def shutdown
152+
@sockets.each { |socket| socket.write("QUIT\n") }
153+
@workers.each { |pid| Process.wait(pid) }
154+
@sockets.each(&:close)
155+
@handle.close
156+
end
157+
158+
private
159+
160+
def setup_shared_memory
161+
@handle = IO::Memory.new(@buffer_size)
162+
@buffer = @handle.map
163+
end
164+
165+
def spawn_workers
166+
@worker_count.times do |i|
167+
parent_socket, child_socket = UNIXSocket.socketpair
168+
169+
pid = fork do
170+
parent_socket.close
171+
worker_process(child_socket, i)
172+
end
173+
174+
child_socket.close
175+
@workers << pid
176+
@sockets << parent_socket
177+
178+
# Send shared memory to worker
179+
parent_socket.send_io(@handle.io)
180+
end
181+
end
182+
183+
def worker_process(socket, worker_id)
184+
# Receive shared memory
185+
shared_io = socket.recv_io
186+
buffer = ::IO::Buffer.map(shared_io, @buffer_size)
187+
188+
puts "Worker #{worker_id} started"
189+
190+
loop do
191+
command = socket.readline.chomp
192+
break if command == "QUIT"
193+
194+
if command.start_with?("WORK:")
195+
_, offset, length = command.split(":")
196+
offset = offset.to_i
197+
length = length.to_i
198+
199+
# Read work data from shared memory
200+
data = buffer.get_string(offset, length)
201+
202+
# Process data (example: reverse it)
203+
result = data.reverse
204+
205+
# Write result back to shared memory at a different offset
206+
result_offset = offset + 10000
207+
buffer.set_string(result, result_offset)
208+
209+
socket.write("DONE:#{result_offset}:#{result.length}\n")
210+
end
211+
end
212+
213+
shared_io.close
214+
socket.close
215+
puts "Worker #{worker_id} finished"
216+
end
217+
end
218+
219+
# Usage example
220+
pool = SharedMemoryWorkerPool.new(worker_count: 3)
221+
222+
# Submit work
223+
result1 = pool.submit_work("Hello World", 0)
224+
result2 = pool.submit_work("Shared Memory", 100)
225+
result3 = pool.submit_work("Zero Copy IPC", 200)
226+
227+
puts "Results: #{result1}, #{result2}, #{result3}"
228+
229+
pool.shutdown
230+
```
231+
232+
## Performance Considerations
233+
234+
### Memory Alignment
235+
236+
For best performance, align your data to cache line boundaries:
237+
238+
```ruby
239+
# Align data to 64-byte boundaries (typical cache line size)
240+
CACHE_LINE_SIZE = 64
241+
242+
def aligned_offset(offset)
243+
(offset + CACHE_LINE_SIZE - 1) & ~(CACHE_LINE_SIZE - 1)
244+
end
245+
246+
handle = IO::Memory.new(4096)
247+
buffer = handle.map
248+
249+
# Write data at aligned offsets
250+
buffer.set_string("Data 1", aligned_offset(0)) # offset 0
251+
buffer.set_string("Data 2", aligned_offset(100)) # offset 128
252+
buffer.set_string("Data 3", aligned_offset(200)) # offset 256
253+
```
254+
255+
### Batch Operations
256+
257+
Group operations to minimize system call overhead:
258+
259+
```ruby
260+
handle = IO::Memory.new(1024)
261+
buffer = handle.map
262+
263+
# Instead of multiple small writes:
264+
# buffer.set_string("A", 0)
265+
# buffer.set_string("B", 1)
266+
# buffer.set_string("C", 2)
267+
268+
# Use a single larger write:
269+
data = "ABC"
270+
buffer.set_string(data, 0)
271+
```

0 commit comments

Comments
 (0)