v0.2.4

In this release, I've cleaned up the code as best as I can. This release also contains a new hook, OnHook, and the introduction of custom hook names to be mapped with OnHook. This way, any hook name can be used to call the OnHook method of the plugins and then return the results based on the hook name in the method's args.

What's Changed

• Minor fixes by @mostafa in #104
• Fix time.Duration parsing by @mostafa in #105
• Clean-ups for CLI flag parsing by @mostafa in #106
• Hooks cleanup and merger by @mostafa in #109
• Make hooks more generic with generic gRPC methods by @mostafa in #110

Full Changelog: v0.2.3...v0.2.4

v0.2.3

This release is considered a stable release, because it contains fixes to the connection handling. So far, GatewayD had the issue of connections becoming stale over time, which was is partially due to how GatewayD works and mainly due to the authentication timeout of TCP connections made to PostgreSQL server. PostgreSQL server has a timeout for TCP connections that connect, but do not authenticate. After the timeout, the connection is alive, and sending data seems to work, however, receiving data from the database server causes io.EOF and connection timeouts. This issue was really hard to detect and it happened often. I tried so many different ways to overcome this, but to no avail. In PR #101 I have introduced an in-process task scheduler to run a cleanup, aka. recycle or retry, job every 60 seconds (default, configurable). The job will iterate through all the available connections in the proxy's connection pool and runs this sequence:

1. Close the connection.
2. Removes the connection from the pool.
3. Create a new connection.
4. Add the new connection to the pool.

And manual testing shows that the issue is fixed. On top of the 60s interval between running the above loop (job), the scheduled job starts with a delay of 60s. That is, new connections won't be recycled; only stale ones. The rest of the changes are cleanups, fixes, slight refactoring and disabling of the test plugin in favor of the cache plugin.

What's Changed

• Implement onTrafficToClient by @mostafa in #98
• Recycle connection on timeout by @mostafa in #101

Full Changelog: v0.2.2...v0.2.3

v0.2.2

There are two major changes to in this release:

1. GatewayD now supports configuration parameters from environment variables. The list of environment variables are here and the order of precedence for configs are as follows:

   This will be the order of precedence for loading and applying config:
   
   1. Plugin default values.
   2. Plugin config file.
   3. Environment variables for plugins.
   4. Global default values.
   5. Environment variables for global config.
   6. Global config updated by plugins via `OnConfigLoaded` hooks.
   
   Plugins override the global configs and has the highest precedence overall.
   

2. The introduction of new hooks for traffic control. Previously there were 3 traffic control hooks, now they are 5:

   hook type	description
   OnTraffic	Notification of incoming traffic from the client.
   OnTrafficFromClient	Traffic is received from the client. This can terminate the connection.1️⃣
   OnTrafficToServer	Traffic is sent to the database server.
   OnTrafficFromServer	Traffic is received from the database server.
   OnTrafficToClient	Traffic is sent to the client.

   1️⃣ As explained here, it is now possible to terminate the connection before the request even reaches the database server. To achieve this, the plugins needs to have the OnTrafficFromClient RPC endpoint and also expose the hook in their config. Then they can achieve connection termination by setting two variables in the return value of the RPC endpoint:

   return structpb.NewStruct(map[string]interface{}{
       "terminate": true,
       "response":  base64.StdEncoding.EncodeToString([]byte{'X', 0, 0, 0, 4}),
   })

   The terminate variable instructs the proxy to stop sending the received data (from client) to the server. The response variable will be sent back to the client, instead. If there is no response in the return value, the connection from the client will close abruptly.

The use case of the second change mentioned above is to let plugins stop requests from reaching the server, for example in a cache plugin scenario, the plugin should decide whether to return the cached value (of a query, which is the previous server's response) or let the message reach the server (and cache it afterwards). This is now possible using connection termination.

The rest are refactorings, cleanups and fixes.

What's Changed

• Load env-vars for configs by @mostafa in #86
• Cleanup interfaces by @mostafa in #94
• Slight refactoring of the hook system plus adding termination logic by @mostafa in #96

Full Changelog: v0.2.1...v0.2.2

v0.2.1

This release contains a fix to the flaky TestRunServer test, which used to fail the CI workflow every often. The other major change is the introduction of a new package, config, that contains almost all the constants from different packages, new types for unmarshaling koanf config into structs and getters for certain fields that has complex types or multiple values from different internal and external packages. Running run command loads the default values for plugins and global configs first before loading the corresponding config files.

A breaking change is the introduction of the default config object for loggers, clients, pool and proxy config. Later I'll add support for loading multiple config objects for running multiple objects of those types for dealing with different database servers (#83).

What's Changed

• Fix flaky TestRunServer test by @mostafa in #82
• New config parser by @mostafa in #84

Full Changelog: v0.2.0...v0.2.1

v0.2.0

This release contains enhacements to the plugin subsystem.

What's Changed

• Add cmdline args to plugin cmd by @mostafa in #75
• Pass environment variabless to plugins on load by @mostafa in #77
• Use koanf getter functions by @mostafa in #80
• Implement requires field for checking if required plugins are loaded or not by @mostafa in #81

Full Changelog: v0.1.4...v0.2.0

v0.1.4

Changes in this milestone (v0.1.x)

This is the latest release of the v0.1.x milestone that significantly contributed to the overall stability of GatewayD. A lot of changes has gone into this milestone including the following:

1. Fixed concurrent connection handling and management.
2. Changed parameter type of HookConfig.Run to receive map[string]interface{} and convert it to structpb.Struct internally to remove lots of boilerplate code.
3. Improved error handling by wrapping/unwrapping package-level errors into GatewayDError with lots of custom errors for different parts of the system.
4. Added more tests.
5. Improved client code by introducing chunk reading of the buffer to prevent hanging and introduction of deadlines (timeouts).
6. Added mechanism to retry the connection after close by the server.
7. Used structured logging everywhere.
8. Added comments to code and docstrings to all functions.
9. Improved traffic hooks by removing a lot of duplicate code and cleanups.
10. Added TCP Keep-Alive to client connections.
11. Partially fixed stale connection issues until the issue I investigated is resolved, as quoted below:

    PostgreSQL has an authentication timeout of 60s that waits till the client authenticates. The current design of GatewayD makes the TCP connection and waits for the connecting clients of GatewayD to authenticate via the proxied connection to PostgreSQL. This causes connections to become stale over time due to timeout. I need to start developing PostgreSQL-specific features for authentication and other related components, either with the plugins or the core.

Changes in the current release

In this release, I've tried to improve the code as best as I can. The connection stability is greatly improved, however the connections will somehow time out and become stale, although new connections will help re-instantiate the client object, which fixes the problem, but this is not enough. My fixes in this release help detect the issue earlier and close the ingress connection abruptly, which is not what I intend to do. The other change is improvements to hook system by addressing a few edge cases. I also added a TCP keepalive to the client connections.

What's Changed

• Traffic hooks improvements by @mostafa in #64
• Add TCP keepalive by @mostafa in #68
• Fix stale connection issue by @mostafa in #69

Full Changelog: v0.1.3...v0.1.4

v0.1.3

This release contains various improvements and fixes to different parts of the code. The retry mechanism is improved, but still needs more work, which I am gonna invest in the next few releases. I used structured logging and also added as much docstring and comments as I can to clarify everything.

What's Changed

• Retry connection after close by server by @mostafa in #58
• Use structured logging everywhere by @mostafa in #63
• Add docstring to all functions by @mostafa in #60

Full Changelog: v0.1.2...v0.1.3

v0.1.2

This release contains an important fix that significantly improves stability of the server (9a090cf) 🎉. I also added more tests and improved the overall client code by introducing configurable parameters for timeouts with defaults.

What's Changed

• Add tests by @mostafa in #53
• Improve client code by @mostafa in #54

Full Changelog: v0.1.1...v0.1.2

v0.1.1

This release contains two changes:

1. Change args parameter type in HookConfig.Run to accept map[string]interface{} instead of *structpb.Struct. This effectively removed a lot of boilerplate needed to deal with structpb.NewStruct.
2. Introduction of GatewayDError error type to wrap package errors and deal with special error cases.

What's Changed

• Change parameter type of HookConfig.Run by @mostafa in #51
• Improve error handling by @mostafa in #52

Full Changelog: v0.1.0...v0.1.1

v0.1.0

This release is the first in a series from the v0.1.x milestone. I tried to fix the concurrent connection handling and sudden freezes when testing GatwayD using psql command. This is unstable as is, and it needs more time to fix the underlying problem. There original issue presented two major problems:

1. Small buffer sizes caused the truncation of the result from the server, hence psql would consider it as chunked response and wait forever, hence freezing.
2. The previous event flow was erroneous. Connection handling and closing is the second major problem. Aborting the connection abruptly causes the clients to wait forever, hence freezing or abnormal behavior.

I will revisit this throughout the current milestone, and will try to fix it and make it more stable.

What's Changed

• Fix concurrent connection handling by @mostafa in #49

Full Changelog: v0.0.9...v0.1.0