CORTX-32806: Query Deployment api

[lakshita-jain]

Signed-off-by: Lakshita Jain [email protected]

Problem Statement

• Query Deployment API code for CSM to utilize for query deployment api which returns the topology as per the requirement of the API using Gconf Data

Design

Approach followed for generating query deployment topology:

1. Load input url and copy it to temporary yaml url
2. Remove num_ and unwanted keys from temporary url
3. convert temporary url to dict
4. process dict data to according to given topology format in HLD.
5. return dict topology.

Coding

• Coding conventions are followed and code is consistent [Y/N]:
• Confirm All CODACY errors are resolved [Y/N]:

Testing

• Are test cases updated along with code changes due to Enhancements/Bugs [Y/N]:
• Confirm that new test cases are added to regression and sanity plan files and relevant feature plan files [Y/N]:
• Confirm that Test Cases are added for new features added [Y/N]:
• Confirm Test Cases cover Happy Path, Non-Happy Path and Scalability [Y/N]:
• Confirm Testing was performed with installed RPM/K8s deployment [Y/N]:

Review Checklist

Before posting the PR please ensure:

• PR is self reviewed
• Is there a change in filename/package/module or signature [Y/N]:
• If yes for above point, Is a notification sent to all other cortx components [Y/N]
• New package/s added to setup.py?
• Jira is updated
• Check if the description is clear and explained.
• Check Acceptance Criterion is defined.
• All the tests performed should be mentioned before Resolving a JIRA.
• Verification needs to be done before marked as Closed/Verified.

KVstore/Confstore feature (changes/fixes) checklist

Confirm changes are made for:

• ConfStore
• KVStore
• ConfCli
• Test cases added for all above 3
• changes done in all the KVpayloads (ConsulKvPayload, IniKvPayload, KvPayload)

Documentation

• Changes done to WIKI / Confluence page

[welcome]

Thanks for your contribution in opening this pull request! Now you can be rewarded with a CORTX sticker by requesting cortx sticker
In the meantime, please check out the contributing guidelines and explore other ways you can get involved.

[cortx-admin]

Can one of the admins verify this patch?

[tanujashinde0405]

LGTM

[tanujashinde0405]

please make sure , After deployment API is working and returning correct topology.
Paste the results in PR description.

[lakshita-jain]

@tanujashinde0405 : here's the output
`>>> from cortx.utils.query_deployment import QueryDeployment

QueryDeployment.get_cortx_topology('consul://cortx-consul-server:8500/conf')
{'cortx': {'common': {'release': {'name': 'CORTX', 'version': '2.0.0-7423'}}}, 'cluster': ['{"security": {"device_certificate": "/etc/cortx/solution/ssl/stx.pem", "domain_certificate": "/etc/cortx/solution/ssl/stx.pem", "ssl_certificate": "/etc/cortx/solution/ssl/s3.seagate.com.pem"}, "storage_set": [{"durability": {"dix": {"data": "1", "parity": "0", "spare": "0"}, "sns": {"data": "1", "parity": "0", "spare": "0"}}, "name": "storage-set-1"}]}'], 'nodes': ['{"machine_id": "3b404b96fc01319b5edb2e84e323595a", "cluster_id": "8ba28deb-4e10-4f07-a32d-ab0d2aed7ffe", "components": [{"name": "utils", "version": "2.0.0-7423"}, {"name": "hare", "version": "2.0.0-7423"}, {"name": "rgw", "services": ["rgw_s3"], "version": "2.0.0-7423"}], "hostname": "cortx-server-1.cortx-server-headless.cortx.svc.cluster.local", "name": "cortx-server-1", "node_id": "cortx-server-1.cortx-server-headless.cortx.svc.cluster.local", "version": "2.0.0-7423", "storage_set": "storage-set-1", "type": "server_node"}', '{"machine_id": "6ac2b08cfe659acf910fed4d11b7df53", "cluster_id": "8ba28deb-4e10-4f07-a32d-ab0d2aed7ffe", "components": [{"name": "utils", "version": "2.0.0-7423"}, {"name": "motr", "services": ["io"], "version": "2.0.0-7423"}, {"name": "hare", "version": "2.0.0-7423"}], "cvg": [{"devices": {"data": ["/dev/sdd", "/dev/sde"], "metadata": ["/dev/sdc"]}, "name": "cvg-01", "type": "ios"}], "hostname": "cortx-data-g0-2.cortx-data-headless.cortx.svc.cluster.local", "name": "cortx-data-g0-2", "node_group": "ssc-vm-g2-rhev4-3385.colo.seagate.com", "node_id": "cortx-data-g0-2.cortx-data-headless.cortx.svc.cluster.local", "version": "2.0.0-7423", "storage_set": "storage-set-1", "type": "data_node/0"}', '{"machine_id": "7ba8990a85d38508ca9bac4ff516d3ea", "cluster_id": "8ba28deb-4e10-4f07-a32d-ab0d2aed7ffe", "components": [{"name": "utils", "version": "2.0.0-7423"}, {"name": "motr", "services": ["io"], "version": "2.0.0-7423"}, {"name": "hare", "version": "2.0.0-7423"}], "cvg": [{"devices": {"data": ["/dev/sdd", "/dev/sde"], "metadata": ["/dev/sdc"]}, "name": "cvg-01", "type": "ios"}], "hostname": "cortx-data-g0-0.cortx-data-headless.cortx.svc.cluster.local", "name": "cortx-data-g0-0", "node_group": "ssc-vm-g2-rhev4-3387.colo.seagate.com", "node_id": "cortx-data-g0-0.cortx-data-headless.cortx.svc.cluster.local", "version": "2.0.0-7423", "storage_set": "storage-set-1", "type": "data_node/0"}', '{"machine_id": "7d9ee991ae6ce628a0e10294bdebaa6b", "cluster_id": "8ba28deb-4e10-4f07-a32d-ab0d2aed7ffe", "components": [{"name": "utils", "version": "2.0.0-7423"}, {"name": "ha", "version": "2.0.0-7423"}], "hostname": "cortx-ha-headless", "name": "cortx-ha", "node_id": "cortx-ha-headless", "version": "2.0.0-7423", "storage_set": "storage-set-1", "type": "ha_node"}', '{"machine_id": "aa378e768a2ba5291d786b3dc48d5650", "cluster_id": "8ba28deb-4e10-4f07-a32d-ab0d2aed7ffe", "components": [{"name": "utils", "version": "2.0.0-7423"}, {"name": "motr", "services": ["io"], "version": "2.0.0-7423"}, {"name": "hare", "version": "2.0.0-7423"}], "cvg": [{"devices": {"data": ["/dev/sdg", "/dev/sdh"], "metadata": ["/dev/sdf"]}, "name": "cvg-02", "type": "ios"}], "hostname": "cortx-data-g1-0.cortx-data-headless.cortx.svc.cluster.local", "name": "cortx-data-g1-0", "node_group": "ssc-vm-g2-rhev4-3387.colo.seagate.com", "node_id": "cortx-data-g1-0.cortx-data-headless.cortx.svc.cluster.local", "version": "2.0.0-7423", "storage_set": "storage-set-1", "type": "data_node/1"}', '{"machine_id": "ac2f9656e38a92990522025e1f3ccaef", "cluster_id": "8ba28deb-4e10-4f07-a32d-ab0d2aed7ffe", "components": [{"name": "utils", "version": "2.0.0-7423"}, {"name": "hare", "version": "2.0.0-7423"}, {"name": "rgw", "services": ["rgw_s3"], "version": "2.0.0-7423"}], "hostname": "cortx-server-0.cortx-server-headless.cortx.svc.cluster.local", "name": "cortx-server-0", "node_id": "cortx-server-0.cortx-server-headless.cortx.svc.cluster.local", "version": "2.0.0-7423", "storage_set": "storage-set-1", "type": "server_node"}', '{"machine_id": "cd1e4c61d297a784245dd88106262cd3", "cluster_id": "8ba28deb-4e10-4f07-a32d-ab0d2aed7ffe", "components": [{"name": "utils", "version": "2.0.0-7423"}, {"name": "motr", "services": ["io"], "version": "2.0.0-7423"}, {"name": "hare", "version": "2.0.0-7423"}], "cvg": [{"devices": {"data": ["/dev/sdd", "/dev/sde"], "metadata": ["/dev/sdc"]}, "name": "cvg-01", "type": "ios"}], "hostname": "cortx-data-g0-1.cortx-data-headless.cortx.svc.cluster.local", "name": "cortx-data-g0-1", "node_group": "ssc-vm-g2-rhev4-3386.colo.seagate.com", "node_id": "cortx-data-g0-1.cortx-data-headless.cortx.svc.cluster.local", "version": "2.0.0-7423", "storage_set": "storage-set-1", "type": "data_node/0"}', '{"machine_id": "d62dc0d2c3ee12aee9b946379519133a", "cluster_id": "8ba28deb-4e10-4f07-a32d-ab0d2aed7ffe", "components": [{"name": "utils", "version": "2.0.0-7423"}, {"name": "hare", "version": "2.0.0-7423"}, {"name": "rgw", "services": ["rgw_s3"], "version": "2.0.0-7423"}], "hostname": "cortx-server-2.cortx-server-headless.cortx.svc.cluster.local", "name": "cortx-server-2", "node_id": "cortx-server-2.cortx-server-headless.cortx.svc.cluster.local", "version": "2.0.0-7423", "storage_set": "storage-set-1", "type": "server_node"}', '{"machine_id": "ecabccb4bced858e282f568d6bf04558", "cluster_id": "8ba28deb-4e10-4f07-a32d-ab0d2aed7ffe", "components": [{"name": "utils", "version": "2.0.0-7423"}, {"name": "csm", "services": ["agent"], "version": "2.0.0-7423"}], "hostname": "cortx-control", "name": "cortx-control", "node_id": "cortx-control", "version": "2.0.0-7423", "storage_set": "storage-set-1", "type": "control_node"}', '{"machine_id": "f488442b875fe8c4328e148f8aca7aa7", "cluster_id": "8ba28deb-4e10-4f07-a32d-ab0d2aed7ffe", "components": [{"name": "utils", "version": "2.0.0-7423"}, {"name": "motr", "services": ["io"], "version": "2.0.0-7423"}, {"name": "hare", "version": "2.0.0-7423"}], "cvg": [{"devices": {"data": ["/dev/sdg", "/dev/sdh"], "metadata": ["/dev/sdf"]}, "name": "cvg-02", "type": "ios"}], "hostname": "cortx-data-g1-2.cortx-data-headless.cortx.svc.cluster.local", "name": "cortx-data-g1-2", "node_group": "ssc-vm-g2-rhev4-3385.colo.seagate.com", "node_id": "cortx-data-g1-2.cortx-data-headless.cortx.svc.cluster.local", "version": "2.0.0-7423", "storage_set": "storage-set-1", "type": "data_node/1"}', '{"machine_id": "fae544140e673b0f2a0d26cb2011516a", "cluster_id": "8ba28deb-4e10-4f07-a32d-ab0d2aed7ffe", "components": [{"name": "utils", "version": "2.0.0-7423"}, {"name": "motr", "services": ["io"], "version": "2.0.0-7423"}, {"name": "hare", "version": "2.0.0-7423"}], "cvg": [{"devices": {"data": ["/dev/sdg", "/dev/sdh"], "metadata": ["/dev/sdf"]}, "name": "cvg-02", "type": "ios"}], "hostname": "cortx-data-g1-1.cortx-data-headless.cortx.svc.cluster.local", "name": "cortx-data-g1-1", "node_group": "ssc-vm-g2-rhev4-3386.colo.seagate.com", "node_id": "cortx-data-g1-1.cortx-data-headless.cortx.svc.cluster.local", "version": "2.0.0-7423", "storage_set": "storage-set-1", "type": "data_node/1"}']}
`

[nitin-seagate]

LGTM

[sachinpunadikar]

+1

[lakshita-jain]

Deployement successful; https://eos-jenkins.colo.seagate.com/job/Cortx-Automation/job/RGW/job/setup-cortx-rgw-cluster/10281/

[welcome]

Thanks for your contribution to CORTX! 🎉