Skip to content

Commit 4506c0f

Browse files
kalantarkalantar
authored
Istio canary tutoral (#673)
* initial istio tutorial * sample files * fix broken link
1 parent 05f0469 commit 4506c0f

File tree

6 files changed

+265
-2
lines changed

6 files changed

+265
-2
lines changed

‎mkdocs/docs/tutorials/testing-patterns/ab.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,4 @@ template: main.html
44

55
# A/B Testing
66

7-
A/B testing pattern is documented as part of [quick start](../../../../getting-started/quick-start/).
7+
A/B testing pattern is documented as part of [quick start](../../../getting-started/quick-start/).

‎mkdocs/docs/tutorials/testing-patterns/canary.md

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,72 @@ Please follow steps 1 through 6 of the [quick start tutorial](../../../getting-s
2727
## 7. Launch experiment
2828
Launch the Iter8 experiment that orchestrates canary testing for the app in this tutorial.
2929

30+
=== "Istio"
31+
32+
```shell
33+
kubectl apply -f $ITER8/samples/istio/canary/experiment.yaml
34+
```
35+
36+
??? info "Look inside experiment.yaml"
37+
```yaml linenums="1"
38+
apiVersion: iter8.tools/v2alpha2
39+
kind: Experiment
40+
metadata:
41+
name: canary-exp
42+
spec:
43+
# target identifies the service under experimentation using its fully qualified name
44+
target: bookinfo-iter8/productpage
45+
strategy:
46+
# this experiment will perform a Canary test
47+
testingPattern: Canary
48+
# this experiment will progressively shift traffic to the winning version
49+
deploymentPattern: Progressive
50+
actions:
51+
# when the experiment completes, promote the winning version using kubectl apply
52+
finish:
53+
- task: common/exec
54+
with:
55+
cmd: /bin/bash
56+
args: [ "-c", "kubectl -n bookinfo-iter8 apply -f {{ .promote }}" ]
57+
criteria:
58+
objectives: # metrics used to validate versions
59+
- metric: iter8-istio/mean-latency
60+
upperLimit: 100
61+
- metric: iter8-istio/error-rate
62+
upperLimit: "0.01"
63+
requestCount: iter8-istio/request-count
64+
duration: # product of fields determines length of the experiment
65+
intervalSeconds: 10
66+
iterationsPerLoop: 10
67+
versionInfo:
68+
# information about the app versions used in this experiment
69+
baseline:
70+
name: productpage-v1
71+
variables:
72+
- name: namespace # used in metric queries
73+
value: bookinfo-iter8
74+
- name: promote # used by final action if this version is the winner
75+
value: https://raw.githubusercontent.com/iter8-tools/iter8/master/samples/istio/quickstart/vs-for-v1.yaml
76+
weightObjRef:
77+
apiVersion: networking.istio.io/v1beta1
78+
kind: VirtualService
79+
namespace: bookinfo-iter8
80+
name: bookinfo
81+
fieldPath: .spec.http[0].route[0].weight
82+
candidates:
83+
- name: productpage-v2
84+
variables:
85+
- name: namespace # used in metric queries
86+
value: bookinfo-iter8
87+
- name: promote # used by final action if this version is the winner
88+
value: https://raw.githubusercontent.com/iter8-tools/iter8/master/samples/istio/quickstart/vs-for-v2.yaml
89+
weightObjRef:
90+
apiVersion: networking.istio.io/v1beta1
91+
kind: VirtualService
92+
namespace: bookinfo-iter8
93+
name: bookinfo
94+
fieldPath: .spec.http[0].route[1].weight
95+
```
3096
=== "Knative"
3197

3298
```shell
@@ -104,6 +170,12 @@ Follow [step 8 of quick start tutorial](../../../getting-started/quick-start/#8-
104170
3. You created an Iter8 experiment with canary testing pattern and progressive deployment pattern. In each iteration, Iter8 observed the latency and error-rate metrics collected by Prometheus; Iter8 verified that the candidate satisfied all the SLOs, identified candidate as the winner, progressively shifted traffic from the baseline to the candidate, and promoted the candidate.
105171

106172
## 9. Cleanup
173+
=== "Istio"
174+
```shell
175+
kubectl delete -f $ITER8/samples/istio/quickstart/fortio.yaml
176+
kubectl delete -f $ITER8/samples/istio/canary/experiment.yaml
177+
kubectl delete namespace bookinfo-iter8
178+
```
107179
=== "Knative"
108180
```shell
109181
kubectl delete -f $ITER8/samples/knative/quickstart/fortio.yaml

‎samples/istio/canary/check.sh

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
#!/bin/bash
2+
3+
set -e
4+
5+
# Ensure ITER8 environment variable is set
6+
if [[ -z ${ITER8} ]]; then
7+
echo "ITER8 environment variable needs to be set to the root folder of Iter8"
8+
exit 1
9+
else
10+
echo "ITER8 is set to " $ITER8
11+
fi
12+
13+
# Check if experiment has completed
14+
completed="Completed"
15+
stage=$(kubectl get experiment $EXPERIMENT -o json | jq -r .status.stage)
16+
if [[ $stage = $completed ]]; then
17+
echo "Experiment has Completed"
18+
else
19+
echo "Experiment must be $completed;"
20+
echo "Experiment is $stage;"
21+
exit 1
22+
fi
23+
24+
# Check if no packets have been lost by Fortio
25+
pod_name=$(kubectl get pods --selector=job-name=fortio -o jsonpath='{.items[*].metadata.name}')
26+
kubectl cp default/"$pod_name":shared/fortiooutput.json /tmp/fortiooutput.json -c busybox
27+
28+
REQUESTSTOTAL=$(jq -r .DurationHistogram.Count /tmp/fortiooutput.json)
29+
REQUESTS200=$(jq -r '.RetCodes."200"' /tmp/fortiooutput.json)
30+
if [[ $REQUESTSTOTAL -eq $REQUESTS200 ]]; then
31+
echo "Packets were not lost"
32+
else
33+
echo "Packets were lost"
34+
echo "total requests:" $REQUESTSTOTAL
35+
echo "200 requests:" $REQUESTS200
36+
exit 1
37+
fi
38+
39+
# Check if versionRecommendedForPromotion is B
40+
expectedVrfp="productpage-v2"
41+
vrfp=$(kubectl get experiment $EXPERIMENT -o json | jq -r .status.versionRecommendedForPromotion)
42+
if [[ $vrfp = $expectedVrfp ]]; then
43+
echo "versionRecommendedForPromotion is $vrfp"
44+
else
45+
echo "versionRecommendedForPromotion must be $expectedVrfp; is" $vrfp
46+
exit 1
47+
fi
48+
49+
# This experiment does not include a finish action; these are commented out.
50+
51+
# # Check if traffic percent to productpage-v2 is > 50
52+
# expectedPercent=100
53+
# actualPercent=$(kubectl -n bookinfo-iter8 get vs bookinfo -o json | jq -r '.spec.http[0].route[1].weight')
54+
# if (( $actualPercent >= $expectedPercent )); then
55+
# echo "percent is $actualPercent"
56+
# else
57+
# echo "percent must be greater than or equal to $expectedPercent; is" $actualPercent
58+
# exit 1
59+
# fi
60+
61+
#check if traffic percent is 100
62+
expectedPercent=100
63+
actualPercent=$(kubectl -n bookinfo-iter8 get vs bookinfo -o json | jq -r '.spec.http[0].route[0].weight')
64+
if [[ $actualPercent -eq $expectedPercent ]]; then
65+
echo "percent is 100"
66+
else
67+
echo "percent must be $expectedPercent; is" $actualPercent
68+
exit 1
69+
fi
70+
71+
set +e

‎samples/istio/canary/e2etest.sh

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
#!/bin/bash
2+
3+
set -e -x
4+
5+
export EXPERIMENT=canary-exp
6+
7+
# method to report failure
8+
reportFailure () {
9+
status=$?
10+
if (( $status != 0 )); then
11+
echo -e "\033[0;31mFAILED:\033[0m $0"
12+
fi
13+
exit $status
14+
}
15+
trap "reportFailure" EXIT
16+
17+
# create cluster
18+
kind create cluster
19+
kubectl cluster-info --context kind-kind
20+
21+
# platform setup
22+
echo "Setting up platform"
23+
$ITER8/samples/istio/quickstart/platformsetup.sh
24+
25+
echo "Create bookinfo app with two productpage versions"
26+
kubectl apply -n bookinfo-iter8 -f $ITER8/samples/istio/quickstart/bookinfo-app.yaml
27+
kubectl apply -n bookinfo-iter8 -f $ITER8/samples/istio/quickstart/productpage-v2.yaml
28+
kubectl -n bookinfo-iter8 wait --for=condition=Ready pods --all --timeout=600s
29+
30+
echo "Generate requests"
31+
URL_VALUE="http://$(kubectl -n istio-system get svc istio-ingressgateway -o jsonpath='{.spec.clusterIP}'):80/productpage"
32+
sed "s+URL_VALUE+${URL_VALUE}+g" $ITER8/samples/istio/quickstart/fortio.yaml | sed "s/6000s/120s/g" | kubectl apply -f -
33+
pod_name=$(kubectl get pods --selector=job-name=fortio -o jsonpath='{.items[*].metadata.name}')
34+
kubectl wait --for=condition=Ready pods/"$pod_name" --timeout=240s
35+
36+
echo "Defining metrics"
37+
kubectl apply -f $ITER8/samples/istio/quickstart/metrics.yaml
38+
39+
echo "Creating Canary experiment"
40+
kubectl apply -f $ITER8/samples/istio/canary/experiment.yaml
41+
42+
# Wait
43+
kubectl wait experiment $EXPERIMENT --for=condition=Completed --timeout=300s
44+
45+
# Log final experiment
46+
kubectl get experiment $EXPERIMENT -o yaml
47+
48+
# Check
49+
source $ITER8/samples/istio/canary/check.sh
50+
51+
# Cleanup .. not needed since cluster is getting deleted
52+
# included to test manual instructions
53+
# continue even if errors
54+
kubectl delete -f $ITER8/samples/istio/quickstart/fortio.yaml
55+
kubectl delete -f $ITER8/samples/istio/canary/experiment.yaml
56+
kubectl delete namespace bookinfo-iter8
57+
58+
# delete cluster
59+
kind delete cluster
60+
61+
set +e
62+
63+
echo -e "\033[0;32mSUCCESS:\033[0m $0"

‎samples/istio/canary/experiment.yaml

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
apiVersion: iter8.tools/v2alpha2
2+
kind: Experiment
3+
metadata:
4+
name: canary-exp
5+
spec:
6+
# target identifies the service under experimentation using its fully qualified name
7+
target: bookinfo-iter8/productpage
8+
strategy:
9+
# this experiment will perform a Canary test
10+
testingPattern: Canary
11+
# this experiment will progressively shift traffic to the winning version
12+
deploymentPattern: Progressive
13+
actions:
14+
# when the experiment completes, promote the winning version using kubectl apply
15+
finish:
16+
- task: common/exec
17+
with:
18+
cmd: /bin/bash
19+
args: [ "-c", "kubectl -n bookinfo-iter8 apply -f {{ .promote }}" ]
20+
criteria:
21+
objectives: # metrics used to validate versions
22+
- metric: iter8-istio/mean-latency
23+
upperLimit: 100
24+
- metric: iter8-istio/error-rate
25+
upperLimit: "0.01"
26+
requestCount: iter8-istio/request-count
27+
duration: # product of fields determines length of the experiment
28+
intervalSeconds: 10
29+
iterationsPerLoop: 10
30+
versionInfo:
31+
# information about the app versions used in this experiment
32+
baseline:
33+
name: productpage-v1
34+
variables:
35+
- name: namespace # used in metric queries
36+
value: bookinfo-iter8
37+
- name: promote # used by final action if this version is the winner
38+
value: https://raw.githubusercontent.com/iter8-tools/iter8/master/samples/istio/quickstart/vs-for-v1.yaml
39+
weightObjRef:
40+
apiVersion: networking.istio.io/v1beta1
41+
kind: VirtualService
42+
namespace: bookinfo-iter8
43+
name: bookinfo
44+
fieldPath: .spec.http[0].route[0].weight
45+
candidates:
46+
- name: productpage-v2
47+
variables:
48+
- name: namespace # used in metric queries
49+
value: bookinfo-iter8
50+
- name: promote # used by final action if this version is the winner
51+
value: https://raw.githubusercontent.com/iter8-tools/iter8/master/samples/istio/quickstart/vs-for-v2.yaml
52+
weightObjRef:
53+
apiVersion: networking.istio.io/v1beta1
54+
kind: VirtualService
55+
namespace: bookinfo-iter8
56+
name: bookinfo
57+
fieldPath: .spec.http[0].route[1].weight

‎samples/istio/quickstart/platformsetup.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ fi
2323

2424
# Step 1: Export correct tags for install artifacts
2525
export TAG="${TAG:-v0.5.1}"
26-
export ISTIO_VERSION="${ISTIO_VERSION:-1.9.3}"
26+
export ISTIO_VERSION="${ISTIO_VERSION:-1.9.4}"
2727
echo "TAG = $TAG"
2828
echo "ISTIO_TAG = $ISTIO_VERSION"
2929

0 commit comments

Comments
 (0)