Sender plugin: Nats

[muhammad-asghar-ali]

feat(plugin): add prototype NATS.io plugin for sender subsystem

• Add experimental NATS plugin with full NATS.io integration
• Support NATS URL parsing in sender subsystem (nats://host:port/subject)
• Implement physical receiver support for JSON format
• Add configurable workers, timeouts, and retry logic
• Include comprehensive test suite with mock clients
• Support various NATS URL formats (auth, multiple servers, wildcards)
• Integrate with existing router subsystem for tag-based matching
• Add metrics integration and error handling
• Maintain consistent snake_case naming conventions

Key new changes include:

• New cmd/plugin_nats.go to register the NATS plugin.
• Updates to go.mod to include NATS dependencies.
• Creation of new modules for the NATS plugin (internal/app/plugins/nats)
  and the NATS API subsystem (internal/app/subsystems/api/nats).
• Implementation of NATS request/response handling for various API
  operations (promises, schedules, locks, tasks).
• Updates to the sender subsystem to support the nats:// scheme.

[codecov]

Codecov Report

❌ Patch coverage is 77.77778% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 52.52%. Comparing base (1659cdd) to head (2c65de2).

Files with missing lines	Patch %	Lines
internal/app/subsystems/aio/sender/sender.go	77.77%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #846      +/-   ##
==========================================
+ Coverage   52.51%   52.52%   +0.01%     
==========================================
  Files         133      133              
  Lines       13281    13290       +9     
==========================================
+ Hits         6974     6981       +7     
- Misses       5762     5763       +1     
- Partials      545      546       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[dfarr]

Thank you so much @muhammad-asghar-ali you are on fire!

[dfarr]

Sorry for the delay in getting back to you @muhammad-asghar-ali. I finally got around to testing this out (and learning more about nats).

I noticed that we can specify the nats server host in the address, but we don't use it because the connection is created once at server startup and therefore we need to know the url of the nats server at this time.

I think this is a fine, even a preferable, setup. But it does make the host part of the uri misleading. I wonder if for the time being we only specify uris like nats:///my-subject? This seems to be a valid uri (I've seen it used for file:///path before) and it removes the misleading part, what do you think?

[muhammad-asghar-ali]

I noticed that we can specify the nats server host in the address, but we don't use it because the connection is created once at server startup and therefore we need to know the url of the nats server at this time.

I think this is a fine, even a preferable, setup. But it does make the host part of the uri misleading. I wonder if for the time being we only specify uris like nats:///my-subject? This seems to be a valid uri (I've seen it used for file:///path before) and it removes the misleading part, what do you think?

Thanks for reviewing, I'll look into your suggestion and share the solution with you

[muhammad-asghar-ali]

Sorry for the delay in getting back to you @muhammad-asghar-ali. I finally got around to testing this out (and learning more about nats).

I noticed that we can specify the nats server host in the address, but we don't use it because the connection is created once at server startup and therefore we need to know the url of the nats server at this time.

I think this is a fine, even a preferable, setup. But it does make the host part of the uri misleading. I wonder if for the time being we only specify uris like nats:///my-subject? This seems to be a valid uri (I've seen it used for file:///path before) and it removes the misleading part, what do you think?

You're right, nats:///my-subject is cleaner and removes the misleading host part.
Good news is that, strings.TrimPrefix(u.Path, "/") already works for both formats, so this is a zero-breaking-change improvement.
I'll update the test cases to use the new format and we're done.

[dfarr]

Hey @muhammad-asghar-ali, we have decided to hold on off on additional plugins until we have a plugin architecture in place (see #886). Let's leave this PR as is for now and we will let you know when the new architecture is ready and we can then port this PR to the new design.

[muhammad-asghar-ali]

Hey @dfarr need some guidance on github workflow failing, the error says

Error: internal/app/plugins/nats/nats.go:112:59: cannot use proc (variable of type *processor) as base.Processor value in argument to base.NewPlugin: *processor does not implement base.Processor (wrong type for method Process)
		have Process([]byte, []byte) (bool, error)
		want Process([]byte, map[string]string, []byte) (bool, error)
Error: Process completed with exit code 1.

Basically, Go is complaining that my processor type doesn’t match the base.Processor interface. The method signature for Process is different: my implementation currently takes ([]byte, []byte), but the interface expects ([]byte, map[string]string, []byte).

So the solution is to update the Process method on processor to match the interface. Something like this mock solution could work:

// Old method
func (p *processor) Process(key []byte, value []byte) (bool, error) {
    // existing logic
    return true, nil
}

// Updated method to satisfy base.Processor
func (p *processor) Process(key []byte, headers map[string]string, value []byte) (bool, error) {
    // handle headers if needed
    return true, nil
}

Once the method signature matches, processor should correctly implement base.Processor, and the workflow error should disappear and also change the other plugin according to new signature. Need your thought on this before implementing.

[dfarr]

Hey @muhammad-asghar-ali I think you could rebase this PR on the current head of main and then you should see the compilation errors locally. If you can fix those and push we should be good to go!

[muhammad-asghar-ali]

compilation

I merge the main with this feature branch nats-plugin and the build the run the resonate project locally, it working fine for... here is screen-short of steps I followed. Please guide me if I'm missing something!! @dfarr