You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**Is your feature request related to a problem? Please describe.**
11
-
A clear and concise description of the problem that will be solved by this feature.
10
+
**Is your feature request related to a **problem**? Please describe the **problem**.**
11
+
A clear and concise description of the **problem**.
12
12
13
-
**Why is the feature useful for Iter8 users?**
14
-
How will this feature add value to Iter8 users and improve Iter8 experiments.
15
-
16
-
**Describe the solution you'd like**
13
+
**Describe the feature/solution you'd like**
17
14
A clear and concise description of what you want to happen.
18
15
19
-
**Describe alternatives you've considered**
20
-
A clear and concise description of any alternative solutions or features you've considered.
16
+
**Will this feature/solution bring new benefits for Iter8 users? What are they?**
17
+
How will this feature/solution add value to Iter8 users.
18
+
19
+
**Will this feature/solution bring new benefits for Iter8 developers? What are they?**
20
+
How will this feature/solution add value to Iter8 developers.
21
21
22
22
**Does this issue require a design doc/discussion? If there is a link to the design document/discussions, please provide it below.**
23
-
You can use this issue thread, or a discussion thread in this repo, for design discussions. If you do so, please provide a link to the specific issue/discussion comment(s) that describes the design.
23
+
You can use this issue thread, or a discussion thread in this repo, for design discussions. Please provide the appropriate links. You can use the issue/discussion to also discuss and describe alternatives.
24
24
25
25
**How will this feature be tested?**
26
-
[] A checklist of behaviors that will be tested as part of this feature implementation.
27
-
28
-
Mark this as TBD if testing requirements are unclear at this time.
26
+
-[ ] A checklist of behaviors that will be tested as part of this feature implementation.
29
27
30
28
**How will this feature be documented?**
31
-
[] A checklist which maybe a combination of user and developer documentation. This includes new (or modifications to existing) code-samples, references, or other forms of user documentation that will be added to https://iter8.tools as part of this feature implementation. Also includes developer documentation, that must always accompany code in the `iter8-analytics`, `etc3`, `handler`, and `iter8ctl` repos.
32
-
33
-
Mark this as TBD if documentation requirements are unclear at this time.
29
+
-[ ] Most new features require end-user documentation at https://iter8.tools. Describe the specific documentation that will be added.
34
30
35
31
**Additional context**
36
32
Add any other context or screenshots about the feature request here.
Copy file name to clipboardExpand all lines: mkdocs/docs/concepts/buildingblocks.md
+8-17Lines changed: 8 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ template: main.html
4
4
5
5
# Building Blocks
6
6
7
-
> Iter8 defines a Kubernetes resource called **Experiment** that automates A/B, A/B/n, Canary, and Conformance experiments. During an experiment, Iter8 can compare multiple versions, find, and safely promote the **winning version (winner)** based on business metrics and SLOs.
7
+
> Iter8 defines a Kubernetes resource called **Experiment** that automates SLO validation, A/B, and A/B/n testing experiments. During an experiment, Iter8 can compare multiple versions, find, and safely promote the **winning version (winner)** based on business metrics and performance metrics like latency and error-rate.
8
8
9
9
We now introduce the building blocks of an Iter8 experiment.
10
10
@@ -75,10 +75,15 @@ A version of your app/ML model is considered **validated**, if it satisfies the
75
75
76
76
## Traffic engineering
77
77
78
-
**Traffic engineering** refers to features such as **traffic mirroring/shadowing** and **user segmentation** that provide fine-grained controls over how traffic is routed to and from app versions.
78
+
**Traffic engineering** refers to features such as **dark launch, traffic mirroring/shadowing, user segmentation** and **session affinity** that provide fine-grained controls over how traffic is routed to and from app versions.
79
79
80
80
Iter8 enables you to take total advantage of all the traffic engineering features available in the service mesh, ingress technology, or networking layer present in your Kubernetes cluster.
81
81
82
+
=== "Dark launch"
83
+
**Dark launch** enables you to deploy and experiment with a new version of your application/ML model in such a way that it is hidden from all (or most) of your end-users.
84
+
85
+
86
+
82
87
=== "Traffic mirroring/shadowing"
83
88
**Traffic mirroring** or **shadowing** enables experimenting with a *dark* launched version with zero-impact on end-users. Mirrored traffic is a replica of the real user requests[^1] that is routed to the dark version. Metrics are collected and evaluated for the dark version, but responses from the dark version are ignored.
84
89
@@ -104,22 +109,8 @@ Iter8 enables you to take total advantage of all the traffic engineering feature
104
109
105
110
When two or more versions participate in an experiment, Iter8 **recommends a version for promotion**; if the experiment yielded a winner, then the version recommended for promotion is the winner; otherwise, the version recommended for promotion is the **baseline** version of your app/ML model.
106
111
107
-
Iter8 can optionally **promote the recommended version** at the end of an experiment.
112
+
Iter8 can **promote the recommended version** at the end of an experiment.
108
113
109
114
110
115
111
-
## Resource config tools
112
-
113
-
Iter8 can be easily integrated with Helm and Kustomize. This integration is especially useful if you use these tools for configuring Kubernetes resources needed for releases of your app/ML model.
114
-
115
-
=== "Helm charts"
116
-
An experiment that uses `helm` charts for version promotion is illustrated below.
117
-
118
-
119
-
120
-
=== "Kustomize resources"
121
-
An experiment that uses `kustomize` resources for version promotion is illustrated below.
122
-
123
-
124
-
125
116
[^1]: It is possible to mirror only a certain percentage of the requests instead of all requests.
Copy file name to clipboardExpand all lines: mkdocs/docs/concepts/whatisiter8.md
+5-3Lines changed: 5 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,12 +4,14 @@ template: main.html
4
4
5
5
# What is Iter8?
6
6
7
-
**Iter8** is an AI-powered platform for cloud native release automation, validation, and experimentation. Iter8 makes it easy to unlock business value and guarantee SLOs by identifying the best performing app/ML model version and promoting it safely.
7
+
**Iter8** is an AI-powered platform for cloud native release automationand experimentation platform that enables SLO validation, A/B testing and progressive delivery.
8
8
9
-
Iter8 is designed for **developers, SREs, service operators, data scientists, and ML engineers** who wish to maximize release velocity and business value with their apps/ML models while protecting end-user experience.
9
+
Iter8 makes it easy to **unlock business value and guarantee SLOs** by identifying the best performing version of your app/ML model and promoting it safely.
10
+
11
+
Iter8 is designed for **DevOps and MLOps teams** interested in maximizing release velocity and business value with their apps/ML models while protecting end-user experience.
10
12
11
13
## What is an Iter8 experiment?
12
-
Iter8 defines a Kubernetes resource called **Experiment** that automates A/B, A/B/n, Canary, and Conformance experiments. During an experiment, Iter8 can compare multiple versions, find, and safely promote the winning version based on business metrics and SLOs.
14
+
Iter8 defines a Kubernetes resource called **Experiment** that automates SLO validation, A/B, and A/B/n testing experiments. During an experiment, Iter8 can compare multiple versions, find, and safely promote the **winning version (winner)**based on business metrics and performance metrics like latency and error-rate.
13
15
14
16
Copy file name to clipboardExpand all lines: mkdocs/docs/reference/tasks/common.md
+48-24Lines changed: 48 additions & 24 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,49 +8,73 @@ template: main.html
8
8
9
9
### Overview
10
10
11
-
The `common/exec` task executes a shell commands with any specified arguments. Arguments are specified using placeholders, or templated variables, that are dynamically instantiated at runtime using values defined when the task is. The `common/exec` task is used in most tutorials as part of a finish action to promote the winning version at the end of an experiment.
11
+
The `common/exec` task executes a shell command with arguments. Arguments are specified using placeholdersthat are dynamically substituted at runtime. The `common/exec` task can be used as part of a finish action to promote the winning version at the end of an experiment.
12
12
13
-
### Examples
13
+
### Example
14
14
15
-
The following example executes `kubectl apply` using the file defined by the placeholder `.promte` when the experiment has completed.
15
+
The following (partially-specified) experiment executes `kubectl apply` using a YAML manifest at the end of the experiment. The URL for the manifest contains a placeholder `.promote`, which is dynamically substituted at the end of the experiment.
16
16
17
17
```yaml
18
-
- finish:
19
-
task: common/exec
20
-
with:
21
-
- cmd: /bin/bash
22
-
- args:
23
-
- -c
24
-
- |
25
-
kubectl apply -f {{ .promote }}
18
+
kind: Experiment
19
+
...
20
+
spec:
21
+
...
22
+
strategy:
23
+
...
24
+
actions:
25
+
finish: # run the following sequence of tasks at the end of the experiment
# information about app versions used in this experiment
36
+
baseline:
37
+
name: sample-app-v1
38
+
variables:
39
+
- name: promote
40
+
value: baseline
41
+
candidates:
42
+
- name: sample-app-v2
43
+
variables:
44
+
- name: promote
45
+
value: candidate
26
46
```
27
47
28
-
### Result
29
-
30
-
The command will be executed; in this case, the yaml file will be applied to the cluster. If a task exits with a non-zero error code, the experiment to which it belongs will also fail.
31
-
32
-
### Arguments
48
+
### Inputs
33
49
34
50
| Field name | Field type | Description | Required |
35
51
| ----- | ---- | ----------- | -------- |
36
52
| cmd | string | The command that should be executed | Yes |
37
53
| args | []string | A list of command line arguments that should be passed to `cmd`. | No |
38
54
| disableInterpolation | bool | Flag indicating whether or not to disable placeholder subsitution. For details, see [below](#disabling-placeholder-substitution). Default is `false`. | No |
39
55
40
-
### Details
56
+
### Result
41
57
42
-
#### Dynamic Placeholder Substitution
58
+
The command with the supplied arguments will be executed.
43
59
44
-
Inputs to tasks can contain placeholders, or template variables, which will be dynamically substituted when the task is executed by Iter8. For example, in the sample experiment above, one input is:
60
+
In the [example above](#example), a YAML file corresponding to the baseline or candidate version will be applied to the cluster.
45
61
46
-
kubectl apply -f {{ .promote }}
62
+
If this task exits with a non-zero error code, the experiment to which it belongs will fail.
63
+
64
+
### Dynamic placeholder substitution
65
+
66
+
Inputs to tasks can contain placeholders, or template variables, which will be dynamically substituted when the task is executed by Iter8. In the [example above](#example), one input is:
In this case, the placeholder is `{{ .promote }}`.
47
71
48
-
In this case, the placeholder is `{{ .promote }}`. Placeholder substitution in task inputs works as follows.
72
+
Placeholder substitution in task inputs works as follows.
49
73
50
-
Iter8 will find the version recommended for promotion. This information is set by Iter8 as the experiment exectuted and in stored in the `status.versionRecommendedForPromotion` field of the experiment. The version recommended for promotion is the winner, if a winner has been found in the experiment. Otherwise, it is the baseline version supplied in the `spec.versionInfo` field of the experiment.
74
+
Iter8 will find the version recommended for promotion which is determined by Iter8 during the course of the experiment, and stored in the `status.versionRecommendedForPromotion` field of the experiment resource. The version recommended for promotion is the winner, if a winner has been found in the experiment. Otherwise, it is the baseline version supplied in the `spec.versionInfo` field of the experiment.
51
75
52
-
If the placeholder is `{{ .name }}`, Iter8 will substitute it with the name of the version recommended for promotion. If it is any other variable, Iter8 will substitute it with the value of the corresponding variable for the version recommended for promotion. Variable values are specified in the variables field of the version detail. Note that variable values could have been supplied by the creator of the experiment, or by other tasks that may already have been executed by Iter8 as part of the experiment.
76
+
If the placeholder is `{{ .name }}`, Iter8 will substitute it with the name of the version recommended for promotion. If it is any other variable, Iter8 will substitute it with the value of the corresponding variable for the version recommended for promotion. Variable values are specified in the variables field of the version detail when the experiment is created.
53
77
54
-
#### Disabling Placeholder Substitution
78
+
### Disabling placeholder substitution
55
79
56
80
By default, the `common/exec` task will attempt to find the version recommended for promotion, and use the values defined for it (in the spec.versionInfo portion of the experiment). However, this behavior will lead to task failure when the version recommended for promotion is not available. This is usually the case when a start task is executed. To use the `common/exec` task as part of an experiment start action, set `disableInterpolation` to true.
| Field name | Field type | Description | Required |
33
27
| ----- | ---- | ----------- | -------- |
34
28
| channel | string | Name of the slack channel to which messages should be posted. | Yes |
35
29
| secret | string | Identifies a secret containing a `token` to be used for authentication. Expressed as `namespace/name`. If `namespace` is not specified, the namespace of the experiment is used. | Yes |
36
30
31
+
### Result
32
+
33
+
A slack message describing the experiment will be posted to the specified channel. Below is a sample slack notification from this task.
An API token allowing posting messages to the desired slack channel is needed. To obtain a suitable token, see [Sending messages using Incoming Webhooks](https://api.slack.com/messaging/webhooks). Once you have the token, store it in a Kubernetes secret. For example, to create the secret _slack-secret_ in the default namespace:
# This role binding enables Iter8 handler to read secrets in the default namespace.
70
-
# To change the namespace apply to the target namespace
71
-
apiVersion: rbac.authorization.k8s.io/v1
72
-
kind: RoleBinding
73
-
metadata:
74
-
name: iter8-secret-reader-handler
75
-
roleRef:
76
-
apiGroup: rbac.authorization.k8s.io
77
-
kind: ClusterRole
78
-
name: iter8-secret-reader
79
-
subjects:
80
-
- kind: ServiceAccount
81
-
name: iter8-handlers
82
-
namespace: iter8-system
83
-
57
+
# This role enables reading of secrets
58
+
apiVersion: rbac.authorization.k8s.io/v1
59
+
kind: ClusterRole
60
+
metadata:
61
+
name: iter8-secret-reader
62
+
rules:
63
+
- apiGroups:
64
+
- ""
65
+
resources:
66
+
- secrets
67
+
verbs: ["get", "list"]
68
+
---
69
+
# This role binding enables Iter8 handler to read secrets in the default namespace.
70
+
# To change the namespace apply to the target namespace
71
+
apiVersion: rbac.authorization.k8s.io/v1
72
+
kind: RoleBinding
73
+
metadata:
74
+
name: iter8-secret-reader-handler
75
+
roleRef:
76
+
apiGroup: rbac.authorization.k8s.io
77
+
kind: ClusterRole
78
+
name: iter8-secret-reader
79
+
subjects:
80
+
- kind: ServiceAccount
81
+
name: iter8-handlers
82
+
namespace: iter8-system
84
83
```
85
84
86
-
#### Target Slack Channel ID
85
+
#### Slack channel ID
87
86
88
87
A slack channel is identified by an id. To find the id, open the slack channel in a web browser. The channel id is the portion of the URL of the form: `CXXXXXXXX`
0 commit comments