Newly added/edited routes are not applied after write

[incogni996]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

Kong 3.7.0 Community Edition

Current Behavior

After I added or edited a Route through the Kong Manager Admin UI, it is not applied directly afterwards. I have to manually enter the VM and do a kong reload for it to be applied.

Expected Behavior

The upserted Route should be reflected as database write finishes, even if it's an eventual guarantee.

Steps To Reproduce

1. Upsert a Route through the Admin UI and/or Admin API, add a host in the route.
2. Assert that the new Route is there
3. Check the database for the new Route existence.
4. See if you can access the newly added host.

Anything else?

A trimmed down version of the config, with keys I assume to be related.

{
  "tagline": "Welcome to kong",
  "edition": "community",
  "version": "3.7.0",
  "configuration": {
    "db_resurrect_ttl": 30,
    "nginx_daemon": "off",
    "anonymous_reports": true,
    "nginx_main_daemon": "off",
    "nginx_worker_processes": "auto",
    "worker_state_update_frequency": 5,
    "pg_port": 5432,
    "pg_ssl": false,
    "pg_ssl_verify": false,
    "pg_max_concurrent_queries": 0,
    "pg_semaphore_timeout": 60000,
    "_debug_pg_ttl_cleanup_interval": 300,
    "worker_consistency": "eventual",
    "log_level": "notice",
    "nginx_main_worker_rlimit_nofile": "auto",
    "db_update_frequency": 5,
    "nginx_events_worker_connections": "auto",
    "db_update_propagation": 0,
    "nginx_events_multi_accept": "on",
    "mem_cache_size": "128m",
    "dns_no_sync": false,
    "pg_timeout": 5000,
    "role": "traditional",
    "db_cache_warmup_entities": [
      "services"
    ],
    "db_cache_ttl": 0,
    "database": "postgres",
    "client_ssl": false,
    "pg_database": "kong",
    "pg_user": "kong"
  },
  "lua_version": "LuaJIT 2.1.0-20231117"
}

[chronolaw]

Could you upgrade to 3.9.1 and try again? thanks.

[Subhadip-decode]

I am currently using kong community edition v3.8.0 on EC2 instances. I have 5 Kong servers with a common postgres database. I am facing the same issue, where I am updating a route or editing a plugin configuration using the Admin APIs or using the Kong manager. Whenever I perform any of these actions. I see the request gets matched to the updated the values and the existing values both in the same server where I made the change and also for other servers. I am using all the default db level configs. Can you please share the root cause before I upgrade Kong to 3.9.1 , since I did not observe this behaviour in v2.8.1.

[chronolaw]

We always use admin api to update kong's configuration and didn't see this issue before, could you share more details? like environment/re-produce steps.

[Subhadip-decode]

The reproduction steps are same as mentioned in the thread. I am using graviton based EC2 instance for the deploymnet of Kong and I am using Postgres as the database. I am using either Konga dashboard or Kong manager or directly using the Admin APIs to onboard or edit APIs but the changes are reflecting partially on the same server.

To be more precise about the situation. I have a route A with / as the route path and that serves any wildcard route which comes for host A. So, prior to making a separate route entry for say route B it was getting matched to route A. Now, on creating a new route with path as route B, the new route is being picked from the next time. But, now if I update the route to something say route path C then on sending request with route C it is getting mapped via route A (the wildcard entry), and on sending request of route path B it is getting routed via route C again (when the route path does not consist of route path B anymore).

[chronolaw]

Does it happen in the latest kong gateway (3.9.1)?

[incogni996]

Hi, is there any other troubleshooting switch that we can turn on to help you investigate that does not involve updating the Kong version? Maybe some other configuration settings, or a log somewhere? Thanks.

For additional context: we have the database living in a separate VM than the gateway itself.

[chronolaw]

The latest kong gateway has many improvements and bug fixes, the old versions may not get the fixes.