Updated Knative installation and tutorials (#635)

* RBAC profile

* Initial changes to platformsetup and quickstart

* Updated platform setup and quickstart

* Namespace creation as part of platform setup

* Improved helm tutorial

* Tutorial documentation changes

* Updated e2e tests and fortio

* fortio updates

* Replace 6000s in Fortio with 120s

* Fixed e2e tests

* Updated e2e tests

* source  pptx

* curl.yaml also fixed to 6000s

* Fixed metric analysis tabs and moved up understanding

Author: sriumcp

.github/workflows/knative-e2e-tests.yaml

@@ -38,13 +38,17 @@ jobs:
     - name: create Fortio job and create experiment
       run: |
         export ITER8=$(pwd)
+        sed -i "s/6000s/120s/g" $ITER8/samples/knative/quickstart/fortio.yaml
         URL_VALUE=$(kubectl get ksvc sample-app -o json | jq .status.address.url)
         sed "s+URL_VALUE+${URL_VALUE}+g" $ITER8/samples/knative/quickstart/fortio.yaml | kubectl apply -f -
         
         # Wait for Fortio to be ready
         pod_name=$(kubectl get pods --selector=job-name=fortio -o jsonpath='{.items[*].metadata.name}')
         kubectl wait --for=condition=Ready pods/"$pod_name" --timeout=240s
 
+        # Define metrics
+        kubectl apply -f $ITER8/samples/knative/quickstart/metrics.yaml
+
         # Starting the experiment
         kubectl apply -f $ITER8/samples/knative/quickstart/experiment.yaml
 
@@ -85,16 +89,26 @@ jobs:
         helm install --repo https://raw.githubusercontent.com/iter8-tools/iter8/master/samples/knative/canaryprogressive/helm-repo sample-app sample-app --namespace=default
         kubectl wait ksvc/sample-app --for condition=Ready --timeout=240s
         helm upgrade --install --repo https://raw.githubusercontent.com/iter8-tools/iter8/master/samples/knative/canaryprogressive/helm-repo sample-app sample-app --values=https://raw.githubusercontent.com/iter8-tools/iter8/master/samples/knative/canaryprogressive/experimental-values.yaml --namespace=default
+        kubectl wait --for=condition=Ready ksvc/sample-app --timeout=240s
 
     - name: Generate requests
       run: |
         export ITER8=$(pwd)
-        kubectl wait --for=condition=Ready ksvc/sample-app --timeout=240s
+        sed -i "s/6000s/120s/g" $ITER8/samples/knative/canaryprogressive/fortio.yaml        
         URL_VALUE=$(kubectl get ksvc sample-app -o json | jq .status.address.url)
         sed "s+URL_VALUE+${URL_VALUE}+g" $ITER8/samples/knative/canaryprogressive/fortio.yaml | kubectl apply -f -
 
+        # Wait for Fortio to be ready
+        pod_name=$(kubectl get pods --selector=job-name=fortio -o jsonpath='{.items[*].metadata.name}')
+        kubectl wait --for=condition=Ready pods/"$pod_name" --timeout=240s
+
     - name: Create Iter8 experiment
       run: |
+        export ITER8=$(pwd)
+
+        # Define metrics
+        kubectl apply -f $ITER8/samples/knative/quickstart/metrics.yaml
+
         export ITER8=$(pwd)
         kubectl apply -f $ITER8/samples/knative/canaryprogressive/experiment.yaml
 
@@ -133,12 +147,21 @@ jobs:
       run: |
         export ITER8=$(pwd)
         kubectl wait --for=condition=Ready ksvc/sample-app --timeout=240s
+        sed -i "s/6000s/120s/g" $ITER8/samples/knative/canaryfixedsplit/fortio.yaml        
         URL_VALUE=$(kubectl get ksvc sample-app -o json | jq .status.address.url)
-        sed "s+URL_VALUE+${URL_VALUE}+g" $ITER8/samples/knative/canaryfixedsplit/fortio.yaml | kubectl apply -f -        
-        
+        sed "s+URL_VALUE+${URL_VALUE}+g" $ITER8/samples/knative/canaryfixedsplit/fortio.yaml | kubectl apply -f -
+
+        # Wait for Fortio to be ready
+        pod_name=$(kubectl get pods --selector=job-name=fortio -o jsonpath='{.items[*].metadata.name}')
+        kubectl wait --for=condition=Ready pods/"$pod_name" --timeout=240s
+
     - name: Create Iter8 experiment
       run: |
         export ITER8=$(pwd)
+
+        # Define metrics
+        kubectl apply -f $ITER8/samples/knative/quickstart/metrics.yaml
+
         kubectl apply -f $ITER8/samples/knative/canaryfixedsplit/experiment.yaml
 
     - name: Sleep until end of experiment
@@ -169,17 +192,26 @@ jobs:
       run: |
         export ITER8=$(pwd)
         kubectl apply -f $ITER8/samples/knative/conformance/baseline.yaml        
+        kubectl wait --for=condition=Ready ksvc/sample-app --timeout=240s
 
     - name: Generate requests
       run: |
         export ITER8=$(pwd)
-        kubectl wait --for=condition=Ready ksvc/sample-app --timeout=240s
+        sed -i "s/6000s/120s/g" $ITER8/samples/knative/conformance/fortio.yaml
         URL_VALUE=$(kubectl get ksvc sample-app -o json | jq .status.address.url)
-        sed "s+URL_VALUE+${URL_VALUE}+g" $ITER8/samples/knative/conformance/fortio.yaml | kubectl apply -f -        
-        
+        sed "s+URL_VALUE+${URL_VALUE}+g" $ITER8/samples/knative/conformance/fortio.yaml | kubectl apply -f -
+
+        # Wait for Fortio to be ready
+        pod_name=$(kubectl get pods --selector=job-name=fortio -o jsonpath='{.items[*].metadata.name}')
+        kubectl wait --for=condition=Ready pods/"$pod_name" --timeout=240s
+
     - name: Create Iter8 experiment
       run: |
         export ITER8=$(pwd)
+
+        # Define metrics
+        kubectl apply -f $ITER8/samples/knative/quickstart/metrics.yaml
+
         kubectl apply -f $ITER8/samples/knative/conformance/experiment.yaml
 
     - name: Sleep until end of experiment

.gitignore

@@ -118,4 +118,4 @@ material
 site
 
 # backup pptx
-~$iter8++images.pptx
+~$*.pptx

mkdocs/docs/getting-started/install.md

@@ -1,22 +1,30 @@
 ---
 template: main.html
 title: Installation
+hide:
+- toc
 ---
 
 # Installation
 
-## Iter8
-
-Install Iter8 in your Kubernetes cluster as follows. This installation requires [`kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/).
+Install Iter8 in your Kubernetes cluster using [Kustomize v3+](https://kubectl.docs.kubernetes.io/installation/kustomize/) as follows.
 
 ```shell
-export TAG=v0.4.3
-curl -s https://raw.githubusercontent.com/iter8-tools/iter8-install/main/install.sh | bash
+export TAG=v0.4.5
+kustomize build github.com/iter8-tools/iter8-install/core?ref=$TAG | kubectl apply -f -
 ```
 
-## iter8ctl (optional)
-The `iter8ctl` client facilitates real-time observability of Iter8 experiments. Install `iter8ctl` on your local machine as follows. This installation requires Go 1.13+.
+The above command installs Iter8's controller and analytics services in the `iter8-system` namespace, the Experiment and Metric CRDs, and the following RBAC permissions.
 
-```shell
-GO111MODULE=on GOBIN=/usr/local/bin go get github.com/iter8-tools/[email protected]
-```
+??? info "Default RBAC Permissions"
+    | Resource | Permissions | Scope |
+    | ----- | ---- | ----------- |
+    | experiments.iter8.tools | get, list, patch, update, watch | Cluster-wide |
+    | experiments.iter8.tools/status | get, patch, update | Cluster-wide |
+    | metrics.iter8.tools | get, list | Cluster-wide |
+    | jobs.batch | create, delete, get, list, watch | Cluster-wide |
+    | leases.coordination.k8s.io | get, list, watch, create, update, patch, delete | `iter8-system` namespace |
+    | events | create | `iter8-system` namespace |
+    | services.serving.knative.dev | get, list, patch, update | Cluster-wide |
+    | virtualservices.networking.istio.io | get, list, patch, update, create, delete | Cluster-wide |
+    | destinationrules.networking.istio.io | get, list, patch, update, create, delete | Cluster-wide |

mkdocs/docs/getting-started/quick-start/with-knative.md

@@ -19,7 +19,7 @@ template: main.html
 ???+ warning "Before you begin, you will need... "
     1. **Kubernetes cluster.** You can also use [Minikube](https://minikube.sigs.k8s.io/docs/) or [Kind](https://kind.sigs.k8s.io/).
     2. The `kubectl` CLI. Install `kubectl` [here](https://kubernetes.io/docs/tasks/tools/install-kubectl/).
-    3. **Go 1.13+** (recommended; required for using `iter8ctl` in [Step 7](/getting-started/quick-start/with-knative/#7-observe-experiment)). Install Go [here](https://golang.org/doc/install).
+    3. **Go 1.13+** (recommended; required for using `iter8ctl` in [Step 8](/getting-started/quick-start/with-knative/#8-observe-experiment)). Install Go [here](https://golang.org/doc/install).
 
 ## 1. Create Kubernetes cluster
 
@@ -43,21 +43,16 @@ Create a local cluster using Minikube or Kind as follows, or use a managed Kuber
 
         ![Resources](../../images/ddresourcepreferences.png)
 
-
-    
-
 ## 2. Clone Iter8 repo
 ```shell
 git clone https://github.com/iter8-tools/iter8.git
 cd iter8
 export ITER8=$(pwd)
 ```
 
-## 3. Install Knative and Iter8
+## 3. Install Knative, Iter8, and Prometheus add-on
 Knative can work with multiple networking layers. So can Iter8's Knative extension. Choose a networking layer for Knative.
 
-This also installs sample metrics for Knative and Iter8's Prometheus add-on. These optional components are used by the sample experiment below.
-
 === "Contour"
 
     ```shell
@@ -156,28 +151,133 @@ sed "s+URL_VALUE+${URL_VALUE}+g" $ITER8/samples/knative/quickstart/fortio.yaml |
         spec:
           volumes:
           - name: shared
-            emptyDir: {}    
+            emptyDir: {}
           containers:
           - name: fortio
             image: fortio/fortio
-            command: ["fortio", "load", "-t", "100s", "-qps", "16","-json", "/shared/fortiooutput.json", $(URL)]
+            command: ["fortio", "load", "-t", "6000s", "-qps", "16", "-json", "/shared/fortiooutput.json", $(URL)]
             env:
             - name: URL
               value: URL_VALUE
             volumeMounts:
             - name: shared
-              mountPath: /shared
-          # useful for extracting fortiooutput.json out of here
-          - name: busybox 
+              mountPath: /shared         
+          - name: busybox
             image: busybox:1.28
-            command: ['sh', '-c', 'echo busybox is running! && sleep 600']          
+            command: ['sh', '-c', 'echo busybox is running! && sleep 600']
             volumeMounts:
             - name: shared
               mountPath: /shared       
           restartPolicy: Never
     ```
 
-## 6. Launch Iter8 experiment
+## 6. Define metrics
+Define the Iter8 metrics used in this experiment.
+
+```shell
+kubectl apply -f $ITER8/samples/knative/quickstart/metrics.yaml
+```
+
+??? info "Look inside metrics.yaml"
+    ```yaml linenums="1"
+    apiVersion: iter8.tools/v2alpha2
+    kind: Metric
+    metadata:
+      labels:
+        creator: iter8
+      name: 95th-percentile-tail-latency
+      namespace: iter8-knative
+    spec:
+      description: 95th percentile tail latency
+      jqExpression: .data.result[0].value[1] | tonumber
+      params:
+      - name: query
+        value: |
+          histogram_quantile(0.95, sum(rate(revision_app_request_latencies_bucket{revision_name='$revision'}[${elapsedTime}s])) by (le))
+      provider: prometheus
+      sampleSize: request-count
+      type: Gauge
+      units: milliseconds
+      urlTemplate: http://prometheus-operated.iter8-system:9090/api/v1/query
+    ---
+    apiVersion: iter8.tools/v2alpha2
+    kind: Metric
+    metadata:
+      labels:
+        creator: iter8
+      name: error-count
+      namespace: iter8-knative
+    spec:
+      description: Number of error responses
+      jqExpression: .data.result[0].value[1] | tonumber
+      params:
+      - name: query
+        value: |
+          sum(increase(revision_app_request_latencies_count{response_code_class!='2xx',revision_name='$revision'}[${elapsedTime}s])) or on() vector(0)
+      provider: prometheus
+      type: Counter
+      urlTemplate: http://prometheus-operated.iter8-system:9090/api/v1/query
+    ---
+    apiVersion: iter8.tools/v2alpha2
+    kind: Metric
+    metadata:
+      labels:
+        creator: iter8
+      name: error-rate
+      namespace: iter8-knative
+    spec:
+      description: Fraction of requests with error responses
+      jqExpression: .data.result[0].value[1] | tonumber
+      params:
+      - name: query
+        value: |
+          (sum(increase(revision_app_request_latencies_count{response_code_class!='2xx',revision_name='$revision'}[${elapsedTime}s])) or on() vector(0)) / (sum(increase(revision_app_request_latencies_count{revision_name='$revision'}[${elapsedTime}s])) or on() vector(0))
+      provider: prometheus
+      sampleSize: request-count
+      type: Gauge
+      urlTemplate: http://prometheus-operated.iter8-system:9090/api/v1/query
+    ---
+    apiVersion: iter8.tools/v2alpha2
+    kind: Metric
+    metadata:
+      labels:
+        creator: iter8
+      name: mean-latency
+      namespace: iter8-knative
+    spec:
+      description: Mean latency
+      jqExpression: .data.result[0].value[1] | tonumber
+      params:
+      - name: query
+        value: |
+          (sum(increase(revision_app_request_latencies_sum{revision_name='$revision'}[${elapsedTime}s])) or on() vector(0)) / (sum(increase(revision_app_request_latencies_count{revision_name='$revision'}[${elapsedTime}s])) or on() vector(0))
+      provider: prometheus
+      sampleSize: request-count
+      type: Gauge
+      units: milliseconds
+      urlTemplate: http://prometheus-operated.iter8-system:9090/api/v1/query
+    ---
+    apiVersion: iter8.tools/v2alpha2
+    kind: Metric
+    metadata:
+      labels:
+        creator: iter8
+      name: request-count
+      namespace: iter8-knative
+    spec:
+      description: Number of requests
+      jqExpression: .data.result[0].value[1] | tonumber
+      params:
+      - name: query
+        value: |
+          sum(increase(revision_app_request_latencies_count{revision_name='$revision'}[${elapsedTime}s])) or on() vector(0)
+      provider: prometheus
+      type: Counter
+      urlTemplate: http://prometheus-operated.iter8-system:9090/api/v1/query
+    ```
+The `urlTemplate` field in these metrics point to the Prometheus instance that was created in Step 3 above. If you wish to use these metrics in your production/staging/dev/test K8s cluster, change the `urlTemplate` values to match the URL of your Prometheus instance.
+
+## 7. Launch experiment
 Launch the Iter8 experiment. Iter8 will orchestrate the canary release of the new version with SLO validation and progressive deployment as specified in the experiment.
 
 ```shell
@@ -245,10 +345,10 @@ The process automated by Iter8 during this experiment is depicted below.
 
 ![Iter8 automation](../../images/canary-progressive-kubectl-iter8.png)
 
-## 7. Observe experiment
+## 8. Observe experiment
 Observe the experiment in realtime. Paste commands from the tabs below in separate terminals.
 
-=== "iter8ctl"
+=== "Metrics-based analysis"
     Install `iter8ctl`. You can change the directory where `iter8ctl` binary is installed by changing `GOBIN` below.
     ```shell
     GO111MODULE=on GOBIN=/usr/local/bin go get github.com/iter8-tools/[email protected]
@@ -319,7 +419,7 @@ Observe the experiment in realtime. Paste commands from the tabs below in separa
 
     As the experiment progresses, you should eventually see that all of the objectives reported as being satisfied by both versions. The candidate is identified as the winner and is recommended for promotion. When the experiment completes (in ~2 mins), you will see the experiment stage change from `Running` to `Completed`.
 
-=== "kubectl get experiment"
+=== "Experiment progress"
 
     ```shell
     kubectl get experiment quickstart-exp --watch
@@ -342,7 +442,7 @@ Observe the experiment in realtime. Paste commands from the tabs below in separa
 
     When the experiment completes (in ~ 2 mins), you will see the experiment stage change from `Running` to `Completed`.    
 
-=== "kubectl get ksvc"
+=== "Traffic split"
 
     ```shell
     kubectl get ksvc sample-app -o json --watch | jq .status.traffic
@@ -370,15 +470,15 @@ Observe the experiment in realtime. Paste commands from the tabs below in separa
         ```
     As the experiment progresses, you should see traffic progressively shift from `sample-app-v1` to `sample-app-v2`. When the experiment completes, all of the traffic will be sent to the winner, `sample-app-v2`.
 
-## 8. Cleanup
-```shell
-kubectl delete -f $ITER8/samples/knative/quickstart/fortio.yaml
-kubectl delete -f $ITER8/samples/knative/quickstart/experiment.yaml
-kubectl delete -f $ITER8/samples/knative/quickstart/experimentalservice.yaml
-```
-
 ???+ info "Understanding what happened"
     1. You created a Knative service with two revisions, `sample-app-v1` (baseline) and `sample-app-v2` (candidate).
     2. You generated requests for the Knative service using a Fortio job. At the start of the experiment, 100% of the requests are sent to the baseline and 0% to the candidate.
     3. You created an Iter8 experiment with canary testing and progressive deployment patterns. In each iteration, Iter8 observed the mean latency, 95th percentile tail-latency, and error-rate metrics collected by Prometheus, verified that the candidate satisfied all objectives, identified the candidate as the winner, progressively shifted traffic from the baseline to the candidate, and eventually promoted the candidate using the `kubectl apply` command embedded within its finish action.
     4. Had the candidate failed to satisfy objectives, then the baseline would have been promoted.
+
+## 9. Cleanup
+```shell
+kubectl delete -f $ITER8/samples/knative/quickstart/fortio.yaml
+kubectl delete -f $ITER8/samples/knative/quickstart/experiment.yaml
+kubectl delete -f $ITER8/samples/knative/quickstart/experimentalservice.yaml
+```