Backport 1.x PR #119 #129 #126 #135 #132 (#139)

peterzhuamazon · heiderich · mpsOxygen · web-flow · commit 99f8eb471dc6 · 2023-08-10T12:04:36.000-04:00
* Fixes #124: Replace curl by uri (#129) * replace curl by ansible.builtin.uri Signed-off-by: Florian Heiderich <florian@heiderich.org> * use os_status.json instead of os_status.stdout Signed-off-by: Florian Heiderich <florian@heiderich.org> * use ansible.builtin.uri instead of curl to get roles Signed-off-by: Florian Heiderich <florian@heiderich.org> --------- Signed-off-by: Florian Heiderich <florian@heiderich.org> * Fix a couple of ansible-lint errors (#126) * capitalize task names to pass ansible-lint Signed-off-by: Ravzan Valceanu <razvan.valceanu@metaminds.com> * Truthy value should be one of \[false, true]ansible-lint Signed-off-by: Ravzan Valceanu <razvan.valceanu@metaminds.com> * All tasks should be named.ansible-lint Signed-off-by: Ravzan Valceanu <razvan.valceanu@metaminds.com> * Use FQCN for builtin module actions - ansible-lint Use FQCN for module actions, such `ansible.posix.sysctl`. - ansible-lint Use FQCN for module actions, such `ansible.posix.selinux`. - ansible-lint Deprecated module. include - ansible-lint You can improve the task key order to: name, when, block - ansible-lint Signed-off-by: Ravzan Valceanu <razvan.valceanu@metaminds.com> * resolve conflict with main use wait_for module instead of pause Signed-off-by: Ravzan Valceanu <razvan.valceanu@metaminds.com> * add "" to variables in task Wait for opensearch to startup Signed-off-by: Ravzan Valceanu <razvan.valceanu@metaminds.com> * fix ansible lint inline vs yaml Signed-off-by: Ravzan Valceanu <razvan.valceanu@metaminds.com> --------- Signed-off-by: Ravzan Valceanu <razvan.valceanu@metaminds.com> Co-authored-by: Ravzan Valceanu <razvan.valceanu@metaminds.com> * Add missing logstash environment on documentation (#135) * doc: add missing logstash environment on documentation Update documentation to add missing logstash environment password. Signed-off-by: Charles Santos <thyarles@gmail.com> * fix: remove auto format from vscode Signed-off-by: Charles Santos <thyarles@gmail.com> --------- Signed-off-by: Charles Santos <thyarles@gmail.com> * Duplicate of previous task (#132) Duplicate of previous task Signed-off-by: maxtimofeev <75588026+maxtimofeev@users.noreply.github.com> * Updating the CODEOWNERS and MAINTAINERS file (#119) Signed-off-by: bbarani <bbarani@amazon.com> --------- Signed-off-by: Florian Heiderich <florian@heiderich.org> Signed-off-by: Ravzan Valceanu <razvan.valceanu@metaminds.com> Signed-off-by: Charles Santos <thyarles@gmail.com> Signed-off-by: maxtimofeev <75588026+maxtimofeev@users.noreply.github.com> Signed-off-by: bbarani <bbarani@amazon.com> Co-authored-by: Florian Heiderich <florian@heiderich.org> Co-authored-by: mpsOxygen <undercover_black_hat@yahoo.com> Co-authored-by: Ravzan Valceanu <razvan.valceanu@metaminds.com> Co-authored-by: Charles Santos <thyarles@gmail.com> Co-authored-by: maxtimofeev <75588026+maxtimofeev@users.noreply.github.com> Co-authored-by: Barani <70038446+bbarani@users.noreply.github.com>
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -1,2 +1,2 @@
 # This should match the owning team set up in https://github.com/orgs/opensearch-project/teams
-*   @opensearch-project/engineering-effectiveness @saravanan30erd
+*   @bbarani @saravanan30erd @peterzhuamazon @prudhvigodithi @gaiksaya @TheAlgo
diff --git a/MAINTAINERS.md b/MAINTAINERS.md
@@ -1,11 +1,15 @@
-## Maintainers
+## Overview
 
-| Maintainer | GitHub ID | Affiliation |
-| --------------- | --------- | ----------- |
-| Barani Bikshandi | [bbarani](https://github.com/bbarani) | Amazon |
-| Peter Zhu | [peterzhuamazon](https://github.com/peterzhuamazon) | Amazon |
-| Sayali Gaikawad | [gaiksaya](https://github.com/gaiksaya) | Amazon |
-| Prudhvi Godithi | [prudhvigodithi](https://github.com/prudhvigodithi) | Amazon |
-| Saravanan Palanisamy | [saravanan30erd](https://github.com/saravanan30erd) | Community |
+This document contains a list of maintainers in this repo. See [opensearch-project/.github/RESPONSIBILITIES.md](https://github.com/opensearch-project/.github/blob/main/RESPONSIBILITIES.md#maintainer-responsibilities) that explains what the role of maintainer means, what maintainers do in this and other repos, and how they should be doing it. If you're interested in contributing, and becoming a maintainer, see [CONTRIBUTING](CONTRIBUTING.md).
+
+## Current Maintainers
+
+| Maintainer          | GitHub ID                                           | Affiliation |
+| ------------------- | --------------------------------------------------- | ----------- |
+| Barani Bikshandi    | [bbarani](https://github.com/bbarani)               | Amazon      |
+| Peter Zhu           | [peterzhuamazon](https://github.com/peterzhuamazon) | Amazon      |
+| Sayali Gaikawad     | [gaiksaya](https://github.com/gaiksaya)             | Amazon      |
+| Prudhvi Godithi     | [prudhvigodithi](https://github.com/prudhvigodithi) | Amazon      |
+|Dhiraj Kumar Jain | [TheAlgo](https://github.com/TheAlgo) | Amazon   |
+| Saravanan Palanisamy | [saravanan30erd](https://github.com/saravanan30erd) | Community   |
 
-[This document](https://github.com/opensearch-project/.github/blob/main/MAINTAINERS.md) explains what maintainers do in this repo, and how they should be doing it. If you're interested in contributing, see [CONTRIBUTING](CONTRIBUTING.md).
diff --git a/README.md b/README.md
@@ -95,17 +95,17 @@ cluster_type: single-node
 
 
     # Deploy with ansible playbook - run the playbook as root
-    ansible-playbook -i inventories/opensearch/hosts opensearch.yml --extra-vars "admin_password=Test@123 kibanaserver_password=Test@6789"
+    ansible-playbook -i inventories/opensearch/hosts opensearch.yml --extra-vars "admin_password=Test@123 kibanaserver_password=Test@6789 logstash_password=Test@456"
 
-You should set the reserved users(`admin` and `kibanaserver`) password using `admin_password` and `kibanaserver_password` variables.
+You should set the reserved users(`admin`, `kibanaserver`, and `logstash`) password using `admin_password`, `kibanaserver_password`, and `logstash_password` variables.
 
-If you define your own internal users (in addition to the reserved `admin` and `kibanaserver`) in custom configuration 
+If you define your own internal users (in addition to the reserved `admin`, `kibanaserver`, and `logstash`) in custom configuration
 files, then passwords to them should be set via variables on the principle of `<username>_password`
 
 It will install and configure the opensearch. Once the deployment completed, you can access the opensearch Dashboards with user `admin` and password which you provided for variable `admin_password`.
 
     # Deploy with ansible playbook - run the playbook as non-root user which have sudo privileges,
-    ansible-playbook -i inventories/opensearch/hosts opensearch.yml --extra-vars "admin_password=Test@123 kibanaserver_password=Test@6789" --become
+    ansible-playbook -i inventories/opensearch/hosts opensearch.yml --extra-vars "admin_password=Test@123 kibanaserver_password=Test@6789 logstash_password=Test@456" --become
 
 **Note**: Change the user details in `ansible_user` parameter in `inventories/opensearch/hosts`  inventory file.
 
diff --git a/opensearch.yml b/opensearch.yml
@@ -1,12 +1,12 @@
 ---
 
-- name: opensearch installation & configuration
+- name: Opensearch installation & configuration
   hosts: os-cluster
   gather_facts: true
   roles:
     - { role: linux/opensearch }
 
-- name: opensearch dashboards installation & configuration
+- name: Opensearch dashboards installation & configuration
   hosts: dashboards
   gather_facts: true
   roles:
diff --git a/roles/linux/dashboards/handlers/main.yml b/roles/linux/dashboards/handlers/main.yml
@@ -1,3 +1,3 @@
 ---
 - name: restart dashboards
-  systemd: name=dashboards state=restarted enabled=yes
+  ansible.builtin.systemd: name=dashboards state=restarted enabled=true
diff --git a/roles/linux/dashboards/tasks/dashboards.yml b/roles/linux/dashboards/tasks/dashboards.yml
@@ -1,13 +1,13 @@
 ---
 
 - name: Dashboards Install | Download opensearch dashbaord {{ os_dashboards_version }}
-  get_url:
+  ansible.builtin.get_url:
     url: "{{ os_download_url }}-dashboards/{{ os_dashboards_version }}/opensearch-dashboards-{{ os_dashboards_version }}-linux-x64.tar.gz"
     dest: "/tmp/opensearch-dashboards.tar.gz"
   register: download
 
 - name: Dashboards Install | Create opensearch dashboard user
-  user:
+  ansible.builtin.user:
     name: "{{ os_dashboards_user }}"
     state: present
     shell: /bin/false
@@ -16,41 +16,41 @@
   when: download.changed or iac_enable
 
 - name: Dashboards Install | Create home directory
-  file:
+  ansible.builtin.file:
     path: "{{ os_dashboards_home }}"
     state: directory
     owner: "{{ os_dashboards_user }}"
     group: "{{ os_dashboards_user }}"
   when: download.changed or iac_enable
 
 - name: Dashboards Install | Extract the tar file
-  command: chdir=/tmp/ tar -xvzf opensearch-dashboards.tar.gz -C "{{ os_dashboards_home }}" --strip-components=1
+  ansible.builtin.command: chdir=/tmp/ tar -xvzf opensearch-dashboards.tar.gz -C "{{ os_dashboards_home }}" --strip-components=1
   when: download.changed or iac_enable
 
 - name: Dashboards Install | Copy Configuration File
-  template:
+  ansible.builtin.template:
     src: opensearch_dashboards.yml
-    dest: "{{os_conf_dir}}/opensearch_dashboards.yml"
+    dest: "{{ os_conf_dir }}/opensearch_dashboards.yml"
     owner: "{{ os_dashboards_user }}"
     group: "{{ os_dashboards_user }}"
     mode: 0644
-    backup: yes
+    backup: true
 
 - name: Dashboards Install | Set the file ownerships
-  file:
+  ansible.builtin.file:
     dest: "{{ os_dashboards_home }}"
     owner: "{{ os_dashboards_user }}"
     group: "{{ os_dashboards_user }}"
-    recurse: yes
+    recurse: true
 
 - name: Dashboards Install | Set the folder permission
-  file:
+  ansible.builtin.file:
     dest: "{{ os_conf_dir }}"
     owner: "{{ os_dashboards_user }}"
     group: "{{ os_dashboards_user }}"
     mode: 0700
 
 - name: Dashboards Install | create systemd service
-  template:
+  ansible.builtin.template:
     src: dashboards.service
     dest: "{{ systemctl_path }}/dashboards.service"
diff --git a/roles/linux/dashboards/tasks/etchosts.yml b/roles/linux/dashboards/tasks/etchosts.yml
@@ -1,13 +1,13 @@
 ---
 - name: Hosts | populate inventory into hosts file
-  blockinfile:
+  ansible.builtin.blockinfile:
     dest: /etc/hosts
     block: |-
       {% for item in groups['dashboards'] %}
       {{ hostvars[item]['ip'] }} {{ item }}.{{ domain_name }} {{ item }}
       {% endfor %}
     state: present
-    create: yes
-    backup: yes
+    create: true
+    backup: true
     marker: "# Ansible inventory hosts {mark}"
   when: populate_inventory_to_hosts_file
diff --git a/roles/linux/dashboards/tasks/main.yml b/roles/linux/dashboards/tasks/main.yml
@@ -1,33 +1,34 @@
 ---
 
-- hostname:
+- name: Set hostname
+  ansible.builtin.hostname:
     name: "{{ inventory_hostname }}"
 
 # Disabling for Amazon Linux 2 as selinux is disabled by default.
 - name: Disable the selinux
-  selinux:
+  ansible.posix.selinux:
     state: disabled
   when: (ansible_distribution != "Ubuntu") and (ansible_distribution != "Amazon")
 
 - name: Populate the nodes to /etc/hosts
-  import_tasks: etchosts.yml
+  ansible.builtin.import_tasks: etchosts.yml
 
 - name: Tune the system settings
-  import_tasks: tune.yml
+  ansible.builtin.import_tasks: tune.yml
 
-- name: include dashboards installation
-  include: dashboards.yml
+- name: Include dashboards installation
+  ansible.builtin.import_tasks: dashboards.yml
 
 - name: Make sure opensearch dashboards is started
-  service:
+  ansible.builtin.service:
     name: dashboards
     state: started
-    enabled: yes
+    enabled: true
 
 - name: Get all the installed dashboards plugins
-  command: "sudo -u {{ os_dashboards_user }} {{ os_plugin_bin_path }} list"
+  ansible.builtin.command: "sudo -u {{ os_dashboards_user }} {{ os_plugin_bin_path }} list"
   register: list_plugins
 
 - name: Show all the installed dashboards plugins
-  debug:
+  ansible.builtin.debug:
     msg: "{{ list_plugins.stdout }}"
diff --git a/roles/linux/dashboards/tasks/tune.yml b/roles/linux/dashboards/tasks/tune.yml
@@ -1,6 +1,6 @@
 ---
 - name: Set open files limit in sysctl.conf
-  sysctl:
+  ansible.posix.sysctl:
     name: fs.file-max
     value: 65536
     state: present
diff --git a/roles/linux/opensearch/handlers/main.yml b/roles/linux/opensearch/handlers/main.yml
@@ -1,10 +1,10 @@
 ---
 # handlers file for opensearch
 - name: reload systemd configuration
-  become: yes
-  command: systemctl daemon-reload
+  become: true
+  ansible.builtin.command: systemctl daemon-reload
 
 # Restart service and ensure it is enabled
 
 - name: restart opensearch
-  systemd: name=opensearch state=restarted enabled=yes
+  ansible.builtin.systemd: name=opensearch state=restarted enabled=yes
diff --git a/roles/linux/opensearch/tasks/etchosts.yml b/roles/linux/opensearch/tasks/etchosts.yml
@@ -1,13 +1,13 @@
 ---
 - name: Hosts | populate inventory into hosts file
-  blockinfile:
+  ansible.builtin.blockinfile:
     dest: /etc/hosts
     block: |-
       {% for item in groups['os-cluster'] %}
       {{ hostvars[item]['ip'] }} {{ item }}.{{ domain_name }} {{ item }}
       {% endfor %}
     state: present
-    create: yes
-    backup: yes
+    create: true
+    backup: true
     marker: "# Ansible inventory hosts {mark}"
   when: populate_inventory_to_hosts_file
diff --git a/roles/linux/opensearch/tasks/main.yml b/roles/linux/opensearch/tasks/main.yml
@@ -1,63 +1,77 @@
 ---
 
-- hostname:
+- name: Set hostname
+  ansible.builtin.hostname:
     name: "{{ inventory_hostname }}"
 
 # Disabling for Amazon Linux 2 as selinux is disabled by default.
 - name: Disable the selinux
-  selinux:
+  ansible.posix.selinux:
     state: disabled
   when: (ansible_distribution != "Ubuntu") and (ansible_distribution != "Amazon")
 
 - name: Populate the nodes to /etc/hosts
-  import_tasks: etchosts.yml
+  ansible.builtin.import_tasks: etchosts.yml
 
 - name: Tune the system settings
-  import_tasks: tune.yml
+  ansible.builtin.import_tasks: tune.yml
 
-- name: include opensearch installation
-  include: opensearch.yml
+- name: Include opensearch installation
+  ansible.builtin.import_tasks: opensearch.yml
 
-- name: include security plugin for opensearch
-  include: security.yml
+- name: Include security plugin for opensearch
+  ansible.builtin.import_tasks: security.yml
 
 # After the cluster forms successfully for the first time,
 # remove the cluster.initial_master_nodes setting from each nodes' configuration.
 - name: Remove `cluster.initial_master_nodes` setting from configuration
-  command: sed -i '/cluster.initial_master_nodes/d' "{{os_conf_dir}}/opensearch.yml"
+  ansible.builtin.command: sed -i '/cluster.initial_master_nodes/d' "{{ os_conf_dir }}/opensearch.yml"
 
 - name: Make sure opensearch is started
-  service:
+  ansible.builtin.service:
     name: opensearch
     state: started
-    enabled: yes
+    enabled: true
 
 - name: Get all the installed ES plugins
-  command: "{{ os_plugin_bin_path }} list"
+  ansible.builtin.command: "{{ os_plugin_bin_path }} list"
   register: list_plugins
 
 - name: Show all the installed ES plugins
-  debug:
+  ansible.builtin.debug:
     msg: "{{ list_plugins.stdout }}"
 
 - name: Wait for opensearch to startup
-  wait_for: host={{ hostvars[inventory_hostname]['ip'] }} port={{os_api_port}} delay=5 connect_timeout=1
+  ansible.builtin.wait_for:
+    host: "{{ hostvars[inventory_hostname]['ip'] }}"
+    port: "{{ os_api_port }}"
+    delay: 5
+    connect_timeout: 1
+    timeout: 120
 
 - name: Check the opensearch status
-  command: curl https://{{ inventory_hostname }}:9200/_cluster/health?pretty -u 'admin:{{ admin_password }}' -k
+  ansible.builtin.uri:
+    url: "https://{{ inventory_hostname }}:9200/_cluster/health?pretty"
+    user: admin
+    password: "{{ admin_password }}"
+    validate_certs: false
   register: os_status
 
 - name: Show the opensearch status
   debug:
-    msg: "{{ os_status.stdout }}"
-  failed_when: "'number_of_nodes' not in os_status.stdout"
+    msg: "{{ os_status.json }}"
+  failed_when: "'number_of_nodes' not in os_status.json"
 
 - name: Verify the roles of opensearch cluster nodes
-  command: curl https://{{ inventory_hostname }}:9200/_cat/nodes?v -u 'admin:{{ admin_password }}' -k
+  ansible.builtin.uri:
+    url: "https://{{ inventory_hostname }}:9200/_cat/nodes?v"
+    user: admin
+    password: "{{ admin_password }}"
+    validate_certs: false
   register: os_roles
   run_once: true
 
 - name: Show the roles of opensearch cluster nodes
   debug:
-    msg: "{{ os_roles.stdout }}"
+    msg: "{{ os_roles }}"
   run_once: true
diff --git a/roles/linux/opensearch/tasks/opensearch.yml b/roles/linux/opensearch/tasks/opensearch.yml
diff --git a/roles/linux/opensearch/tasks/security.yml b/roles/linux/opensearch/tasks/security.yml
diff --git a/roles/linux/opensearch/tasks/tune.yml b/roles/linux/opensearch/tasks/tune.yml

Original file line number	Diff line number	Diff line change
`@@ -1,2 +1,2 @@`
`1`	`1`	`# This should match the owning team set up in https://github.com/orgs/opensearch-project/teams`
`2`		`-* @opensearch-project/engineering-effectiveness @saravanan30erd`
	`2`	`+* @bbarani @saravanan30erd @peterzhuamazon @prudhvigodithi @gaiksaya @TheAlgo`