Skip to content

Commit 8158b95

Browse files
sriumcpsriumcp
authored
Update task descriptions; include readiness task (#833)
* updated task descriptions Signed-off-by: Srinivasan Parthasarathy <[email protected]> * Updated versions Signed-off-by: Srinivasan Parthasarathy <[email protected]> * removed unwanted comments Signed-off-by: Srinivasan Parthasarathy <[email protected]>
1 parent c602676 commit 8158b95

File tree

12 files changed

+195
-179
lines changed

12 files changed

+195
-179
lines changed

‎install/core/iter8-controller/kustomization.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
resources:
22
- etc3.yaml
33
# to change Iter8 CRDs version, edit the next line
4-
- github.com/iter8-tools/etc3/config/crd/?ref=v0.1.27
4+
- github.com/iter8-tools/etc3/config/crd/?ref=v0.1.28
55

66
images:
77
- name: iter8/etc3:latest
88
# to change etc3 version, edit the next line
9-
newTag: 0.1.27
9+
newTag: 0.1.28
1010

1111
# even if you do not change the CRD,
1212
# keep the two version numbers in sync to keep them the same

‎install/core/iter8-handler/handler.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ spec:
99
containers:
1010
- name: iter8-handler
1111
# to change handler version, edit the line below
12-
image: iter8/handler:0.1.17
12+
image: iter8/handler:0.1.20
1313
command: ["handler"]
1414
args: ["run", "-a", "$(ACTION)"]
1515
env:

‎mkdocs/docs/metrics/builtin.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ The following are the set of builtin Iter8 metrics.
2525
| iter8-system | latency-99th-percentile | Gauge | 99th percentile response latency |
2626

2727
## Collecting builtin metrics
28-
Use the [`metrics/collect` task](../reference/tasks/metrics.md) in an experiment to collect builtin metrics for your app/ML model versions.
28+
Use the [`metrics/collect` task](../reference/tasks/metrics-collect.md) in an experiment to collect builtin metrics for your app/ML model versions.
2929

3030
## Example
3131
For an example of an experiment that uses builtin metrics, look inside the Knative experiment in [this tutorial](../../tutorials/knative/testing-strategies/conformance/#5-launch-experiment).

‎mkdocs/docs/reference/experiment.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -209,7 +209,7 @@ Standard Kubernetes [meta.v1/ObjectMeta](https://kubernetes.io/docs/reference/ge
209209

210210
| Field name | Field type | Description | Required |
211211
| ----- | ------------ | ----------- | -------- |
212-
| aggregatedBuiltinHists | [AggregatedBuiltinHists](#aggregatedbuiltinhists) | This field is used to store intermediate results from the [`metrics/collect` task](tasks/metrics.md#metrics-tasks) that enables [builtin metrics](../metrics/builtin.md). Reserved for Iter8 internal use. | No |
212+
| aggregatedBuiltinHists | [AggregatedBuiltinHists](#aggregatedbuiltinhists) | This field is used to store intermediate results from the [`metrics/collect` task](tasks/metrics-collect.md#metrics-tasks) that enables [builtin metrics](../metrics/builtin.md). Reserved for Iter8 internal use. | No |
213213
| aggregatedMetrics | [AggregatedMetricsAnalysis](#aggregatedmetricsanalysis) | Most recently observed metric values for all metrics referenced in the experiment criteria. | No |
214214
| winnerAssessment | [WinnerAssessmentAnalysis](#winnerassessmentanalysis) | Information about the `winner` of the experiment. | No |
215215
| versionAssessments | [VersionAssessmentAnalysis](#versionassessmentanalysis) | For each version, a summary analysis identifying whether or not the version is satisfying the experiment criteria. | No |

‎mkdocs/docs/reference/tasks.md

Lines changed: 0 additions & 42 deletions
This file was deleted.

‎mkdocs/docs/reference/tasks/common-bash.md

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
---
2+
template: main.html
3+
---
4+
5+
# `common/bash`
6+
The `common/bash` task executes a bash script. The script can be written to use placeholders that are [dynamically substituted at runtime](#dynamic-variable-substitution). For example, the `common/bash` task can be used as part of a finish action to promote the winning version at the end of an experiment.
7+
8+
## Example
9+
10+
The following (partially-specified) experiment executes the one line script `kubectl apply` using a YAML manifest at the end of the experiment. The URL for the manifest contains a placeholder `{{ .promotionManifest }}`, which is dynamically substituted at the end of the experiment.
11+
12+
```yaml
13+
kind: Experiment
14+
...
15+
spec:
16+
...
17+
strategy:
18+
...
19+
actions:
20+
finish: # run the following sequence of tasks at the end of the experiment
21+
- task: common/bash # promote the winning version
22+
with:
23+
script: |
24+
kubectl apply -f {{ .promotionManifest }}
25+
...
26+
versionInfo:
27+
# information about app versions used in this experiment
28+
baseline:
29+
name: sample-app-v1
30+
variables:
31+
- name: promotionManifest
32+
value: https://raw.githubusercontent.com/iter8-tools/iter8/master/samples/knative/quickstart/baseline.yaml
33+
candidates:
34+
- name: sample-app-v2
35+
variables:
36+
- name: promotionManifest
37+
value: https://raw.githubusercontent.com/iter8-tools/iter8/master/samples/knative/quickstart/candidate.yaml
38+
```
39+
40+
## Inputs
41+
42+
| Field name | Field type | Description | Required |
43+
| ----- | ---- | ----------- | -------- |
44+
| script | string | The bash script that will be executed | Yes |
45+
46+
## Result
47+
48+
The script will be executed.
49+
50+
In the example above, a YAML file corresponding to the baseline or candidate version will be applied to the cluster.
51+
52+
If this task exits with a non-zero error code, the experiment to which it belongs will fail.
53+
54+
## Dynamic Variable Substitution
55+
The script input to the `common/bash` task may contain placeholders, or template variables, which will be dynamically substituted when the task is executed by Iter8. For example, in the task:
56+
57+
```bash
58+
- task: common/bash # promote the winning version
59+
with:
60+
script: |
61+
kubectl apply -f {{ .promoteManifest }}
62+
```
63+
64+
`{{ .promotionManifest}}` is a placeholder.
65+
66+
Placeholders are specified using the Go language specification for data-driven [templates](https://golang.org/pkg/html/template/). In particular, placeholders are specified between double curly braces.
67+
68+
The `common/bash` task supports placeholders for:
69+
70+
- Values of variables of the version recommended for promotion. To specify such placeholders, use the name of the variable as defined in the [`versionInfo` section](../experiment.md#versioninfo) of the experiment definition. For example, in the above example, `{{ .promotionManifest }}` is a placeholder for the value of the variable with the name `promotionManifest` of the version Iter8 recommends for promotion (see [`.status.versionRecommendedForPromotion`](../experiment.md#status)).
71+
72+
- Values defined in the experiment itself. To specify such placeholders, use the prefix `.this`. For example, `{{ .this.metadata.name }}` is a placeholder for the name of the experiment.
73+
74+
If Iter8 cannot evaluate a placeholder expression, a blank value ("") will be substituted.

‎mkdocs/docs/reference/tasks/common.md renamed to ‎mkdocs/docs/reference/tasks/common-exec.md

Lines changed: 6 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -2,69 +2,12 @@
22
template: main.html
33
---
44

5-
# Common Tasks
6-
7-
## `common/bash`
8-
9-
### Overview
10-
11-
The `common/bash` task executes a bash script. The script can be written to use placeholders that are [dynamically substituted at runtime](../tasks.md#dynamic-variable-substitution). For example, the `common/bash` task can be used as part of a finish action to promote the winning version at the end of an experiment.
12-
13-
### Example
14-
15-
The following (partially-specified) experiment executes the one line script `kubectl apply` using a YAML manifest at the end of the experiment. The URL for the manifest contains a placeholder `{{ .promotionManifest }}`, which is dynamically substituted at the end of the experiment.
16-
17-
```yaml
18-
kind: Experiment
19-
...
20-
spec:
21-
...
22-
strategy:
23-
...
24-
actions:
25-
finish: # run the following sequence of tasks at the end of the experiment
26-
- task: common/bash # promote the winning version
27-
with:
28-
script: |
29-
kubectl apply -f {{ .promotionManifest }}
30-
...
31-
versionInfo:
32-
# information about app versions used in this experiment
33-
baseline:
34-
name: sample-app-v1
35-
variables:
36-
- name: promotionManifest
37-
value: https://raw.githubusercontent.com/iter8-tools/iter8/master/samples/knative/quickstart/baseline.yaml
38-
candidates:
39-
- name: sample-app-v2
40-
variables:
41-
- name: promotionManifest
42-
value: https://raw.githubusercontent.com/iter8-tools/iter8/master/samples/knative/quickstart/candidate.yaml
43-
```
44-
45-
### Inputs
46-
47-
| Field name | Field type | Description | Required |
48-
| ----- | ---- | ----------- | -------- |
49-
| script | string | The bash script that will be executed | Yes |
50-
51-
### Result
52-
53-
The script will be executed.
54-
55-
In the example above, a YAML file corresponding to the baseline or candidate version will be applied to the cluster.
56-
57-
If this task exits with a non-zero error code, the experiment to which it belongs will fail.
58-
59-
## `common/exec` (Deprecated)
60-
61-
### Overview
62-
5+
# `common/exec` (deprecated)
636
The `common/exec` task executes a shell command with arguments. Arguments may be specified using placeholders that are dynamically substituted at runtime. The `common/exec` task can be used as part of a finish action to promote the winning version at the end of an experiment.
647

658
`common/exec` is deprecated; use `common/bash` instead.
669

67-
### Example
10+
## Example
6811

6912
The following (partially-specified) experiment executes `kubectl apply` using a YAML manifest at the end of the experiment. The URL for the manifest contains a placeholder `{{ .promote }}`, which is dynamically substituted at the end of the experiment.
7013

@@ -99,24 +42,24 @@ spec:
9942
value: candidate
10043
```
10144
102-
### Inputs
45+
## Inputs
10346
10447
| Field name | Field type | Description | Required |
10548
| ----- | ---- | ----------- | -------- |
10649
| cmd | string | The command that should be executed | Yes |
10750
| args | []string | A list of command line arguments that should be passed to `cmd`. | No |
10851
| disableInterpolation | bool | Flag indicating whether or not to disable placeholder subsitution. For details, see [below](#disabling-placeholder-substitution). Default is `false`. | No |
10952

110-
### Result
53+
## Result
11154

11255
The command with the supplied arguments will be executed.
11356

11457
In the example above, a YAML file corresponding to the baseline or candidate version will be applied to the cluster.
11558

11659
If this task exits with a non-zero error code, the experiment to which it belongs will fail.
11760

118-
### Dynamic Variable Substitution
61+
## Dynamic Variable Substitution
11962

120-
The `common/exec` task only supports [dynamic variable subsitution](../tasks.md#dynamic-variable-substitution) for variables of the version recommended for promotion.
63+
The `common/exec` task supports [dynamic variable subsitution](common-bash.md#dynamic-variable-substitution) only using the variables of the version recommended for promotion.
12164

12265
Instead of defaulting to a blank value when Iter8 has not determined a version to recommend for promotion (that is, in start tasks), this task supports the `disableInterpolation` option to prevent dynamic variable substitution.

‎mkdocs/docs/reference/tasks/common-readiness.md

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
---
2+
template: main.html
3+
---
4+
5+
# `common/readiness`
6+
The `common/readiness` task can be used to verify that Kubernetes resources (for example, deployments, Knative services, or Istio virtual services) that are required for the experiment are available and ready. This task is intended to be included in the start action of an experiment.
7+
8+
## Example
9+
The following is an experiment snippet with a `common/readiness` task.
10+
11+
```yaml
12+
...
13+
spec:
14+
strategy:
15+
actions:
16+
start:
17+
- task: common/readiness
18+
with:
19+
# verify that the following deployments exist
20+
objRefs:
21+
- kind: Deployment
22+
name: hello
23+
namespace: default
24+
# verify that the deployment is available
25+
waitFor: condition=available
26+
- kind: Deployment
27+
name: hello-candidate
28+
namespace: default
29+
# verify that the deployment is available
30+
waitFor: condition=available
31+
...
32+
# `common/readiness` task will also inspect the versionInfo section.
33+
# If resources are specified as part of weightObjRef fields within versionInfo,
34+
# the task will verify the existence of these resources as well.
35+
versionInfo:
36+
baseline:
37+
name: stable
38+
weightObjRef:
39+
apiVersion: networking.istio.io/v1beta1
40+
kind: VirtualService
41+
namespace: default
42+
name: hello-routing
43+
fieldPath: .spec.http[0].route[0].weight
44+
candidates:
45+
- name: candidate
46+
weightObjRef:
47+
apiVersion: networking.istio.io/v1beta1
48+
kind: VirtualService
49+
namespace: default
50+
name: hello-routing
51+
fieldPath: .spec.http[0].route[1].weight
52+
```
53+
54+
## Inputs
55+
| Field name | Field type | Description | Required |
56+
| ----- | ---- | ----------- | -------- |
57+
| initialDelaySeconds | int | Verification will be attempted only after this initial delay. Default value is `5`. | No |
58+
| numRetries | int | If the task cannot verify the existence/conditions of the resources after the first attempt, it will retry with further attempts. Total number of attempts = 1 + numRetries. Default value for `numRetries` is `12`. | No |
59+
| intervalSeconds | int | Time interval between each attempt. Default value is `5`. | No |
60+
| objRefs | [][ObjRef](#objref) | A list of Kubernetes object references along with any associated conditions which need to be verified. | No |
61+
62+
### ObjRef
63+
| Field name | Field type | Description | Required |
64+
| ----- | ---- | ----------- | -------- |
65+
| kind | string | Kind of the Kubernetes resource. Specified in the TYPE[.VERSION][.GROUP] format used by the [`kubectl get` command](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#get) | Yes |
66+
| namespace | string | Namespace of the Kubernetes resource. Default value is the namespace of the experiment resource. | No |
67+
| name | string | Name of the Kubernetes resource. | Yes |
68+
| waitFor | string | Any value that is accepted by the `--for` flag of the [`kubectl wait` command](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#wait). | No |
69+
70+
71+
## Result
72+
The task will succeed if all the specified resources are verified to exist (along with any associated conditions) within the specified number of attempts. Otherwise, the task will fail, resulting in the failure of the experiment.

0 commit comments

Comments
 (0)