Skip to content

Commit 7f8adf4

Browse files
mikebzmikebz
authored
effective customizations chapter (#2659)
merging this as an MVP and will iterate based on additional requests and customer feedback.
1 parent 7ee57d6 commit 7f8adf4

File tree

7 files changed

+305
-1
lines changed

7 files changed

+305
-1
lines changed

‎site/book/06-deploying-packages/00.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
In the last chapter of this book, we are going to cover how you deploy a kpt
1+
In this chapter of this book, we are going to cover how you deploy a kpt
22
package to a Kubernetes cluster and how the cluster state is managed as the
33
package evolves over time.
44

‎site/book/07-effective-customizations/00.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
Kubernetes configuration packages and customizations go hand in hand, all the
2+
packaging tools enable package customization, since every package needs to be adapted to each specific use. In this chapter we cover effective
3+
customizations techniques that kpt rendering and packaging enables. We show how
4+
providing customization through parameters has some [pitfalls] and recommend
5+
alternatives where the contents of the package are not hidden behind a facade.
6+
Some of these alternatives are only possible because kpt has made an investment
7+
into bulk editing with [KRM functions] and upstream merging.
8+
9+
### Prerequisites
10+
11+
Before reading this chapter you should familiarize yourself with [chapter 4]
12+
which talks about using functions as well as [updating a package page] in
13+
[chapter 3].
14+
15+
[chapter 4]: /book/04-using-functions/
16+
[chapter 3]: /book/03-packages/
17+
[pitfalls]: https://github.com/kubernetes/design-proposals-archive/blob/main/architecture/declarative-application-management.md#parameterization-pitfalls
18+
[KRM functions]: https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md
19+
[updating a package page]: /book/03-packages/05-updating-a-package.md

‎site/book/07-effective-customizations/01-single-value-replacement.md

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
## Scenario
2+
3+
I have a single value replacement in my package. I don’t want package consumers
4+
to look through all the yaml files to find the value I want them to set. It
5+
seems easier to just create a parameter for this value and have the user look
6+
at Kptfile for inputs.
7+
8+
Example storage bucket:
9+
10+
```yaml
11+
apiVersion: storage.cnrm.cloud.google.com/v1beta1
12+
kind: StorageBucket
13+
metadata:
14+
name: my-bucket # kpt-set: ${project-id}-${name}
15+
namespace: ns-test # kpt-set: ${namespace}
16+
annotations:
17+
cnrm.cloud.google.com/force-destroy: "false"
18+
cnrm.cloud.google.com/project-id: my-project # kpt-set: ${project-id}
19+
spec:
20+
storageClass: standard # kpt-set: ${storage-class}
21+
uniformBucketLevelAccess: true
22+
versioning:
23+
enabled: false
24+
```
25+
26+
The corresponding Kptfile:
27+
28+
```yaml
29+
apiVersion: kpt.dev/v1
30+
kind: Kptfile
31+
metadata:
32+
name: bucket
33+
info:
34+
description: A Google Cloud Storage bucket
35+
pipeline:
36+
mutators:
37+
- image: gcr.io/kpt-fn/apply-setters:v0.2
38+
configMap:
39+
name: todo-bucket-name
40+
namespace: todo-namespace
41+
project-id: todo-project-id
42+
storage-class: standard
43+
```
44+
45+
46+
## Problems
47+
48+
1. With package popularity the single values inevitably expand to provide a
49+
facade to a large portion of the data. That defeats the purpose of minimizing
50+
the cognitive load. With this small example almost half of the StorageBucket configuration is now covered with parameters.
51+
1. Some values like resource names are used as references so setting them in
52+
one place needs to trigger updates in all the places where they are referenced.
53+
1. If additional resources that have similar values are added to the package
54+
new string replacements need to be added. In this case everything will need
55+
to also be marked up with project ID and namespace.
56+
1. If a package is used as a sub-package the string replacement parameters need
57+
to be surfaced to the parent package and if the parent package already expects
58+
some values to be set and the parameters do not exist, the sub-package needs to
59+
be updated.
60+
61+
## Solutions:
62+
63+
1. kpt allows the user to edit a particular value directly in the configuration
64+
data and will handle upstream merge. When [editing the yaml] directly the
65+
consumers are not confined to the parameters that the package author has
66+
provided. [kpt pkg update] merges the local edits made by consumer with the
67+
changes in the upstream package made by publisher. In this case `storageClass`
68+
can be set directly by the user.
69+
1. Attributes like resource names which are often updated by consumers to add
70+
prefix or suffix (e.g. *-dev, *-stage, *-prod, na1-*, eu1-*) are best handled
71+
by the [ensure-name-substring] function that will handle dependency updates as
72+
well as capture all the resources in the package.
73+
1. Instead of setting a particular value on a resource a bulk operation can be
74+
applied to all the resources that fit a particular interface. This can be done
75+
by a custom function or by [set-namespace], [search-and-replace] , [set-labels]
76+
and [set-annotations] functions.
77+
78+
New bucket configuration:
79+
80+
```yaml
81+
apiVersion: storage.cnrm.cloud.google.com/v1beta1
82+
kind: StorageBucket
83+
metadata:
84+
name: bucket
85+
annotations:
86+
cnrm.cloud.google.com/force-destroy: "false"
87+
spec:
88+
storageClass: standard
89+
uniformBucketLevelAccess: true
90+
versioning:
91+
enabled: false
92+
```
93+
94+
The suggested customizations are now in the Kptfile:
95+
96+
```yaml
97+
apiVersion: kpt.dev/v1
98+
kind: Kptfile
99+
metadata:
100+
name: bucket
101+
info:
102+
description: A Google Cloud Storage bucket
103+
pipeline:
104+
mutators:
105+
- image: gcr.io/kpt-fn/set-namespace:v0.2.0
106+
configMap:
107+
namespace: example-ns
108+
- image: gcr.io/kpt-fn/ensure-name-substring:v0.1.1
109+
configMap:
110+
prepend: project111-
111+
- image: gcr.io/kpt-fn/set-annotations:v0.1.4
112+
configMap:
113+
cnrm.cloud.google.com/project-id: project111
114+
```
115+
116+
The resource configuration YAML doesn't need to be marked up with where the
117+
namespace value needs to go. The [set-namespace] function is smart enough to
118+
find all the appropriate resources that need the namespace.
119+
120+
We have put in the starter name `bucket` and have an [ensure-name-substring]
121+
that shows the package consumer that the project ID prefix is what we suggest.
122+
However if they have a different naming convention they can alter the name
123+
prefix or suffix on all the resources in the pacakge.
124+
125+
Since we are trying to set the annotation to the project ID we can use the
126+
[set-annotations] function one time and the annotation are going to be set on
127+
all the resources in the package. If we add additional resources or whole
128+
sub packages we will get the consistent annotations across all resources
129+
without having to find all the places where annotations can go.
130+
131+
[editing the yaml]: /book/03-packages/03-editing-a-package
132+
[kpt pkg update]: /book/03-packages/05-updating-a-package
133+
[ensure-name-substring]: https://catalog.kpt.dev/ensure-name-substring/v0.1/
134+
[search-and-replace]: https://catalog.kpt.dev/search-replace/v0.2/
135+
[set-labels]: https://catalog.kpt.dev/set-labels/v0.1/
136+
[set-annotations]: https://catalog.kpt.dev/set-annotations/v0.1/
137+
[set-namespace]: https://catalog.kpt.dev/set-namespace/v0.2/

‎site/book/07-effective-customizations/02-limiting-package-changes.md

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
## Scenario:
2+
3+
I’d like to limit what my package consumers can do with my package and it feels
4+
safer to just provide a string replacement in one place so they know not to
5+
alter the configuration outside of the few places that I designated as OK
6+
places to change.
7+
8+
Example deployment:
9+
```yaml
10+
apiVersion: apps/v1
11+
kind: Deployment
12+
metadata: # kpt-merge: /nginx-deploy
13+
name: nginx-deploy
14+
spec:
15+
selector:
16+
matchLabels:
17+
app: nginx
18+
template:
19+
metadata:
20+
labels:
21+
app: nginx
22+
spec:
23+
containers:
24+
- name: backend
25+
image: nginx:1.16.1 # kpt-set: nginx:${tag}
26+
```
27+
28+
kpt configuration that uses a setter:
29+
```yaml
30+
apiVersion: kpt.dev/v1
31+
kind: Kptfile
32+
metadata:
33+
name: dont-change-much
34+
pipeline:
35+
mutators:
36+
- image: gcr.io/kpt-fn/apply-setters:v0.2.0
37+
configMap:
38+
tag: 1.21
39+
```
40+
41+
## Problems:
42+
43+
1. The limitation by parameters does not guarantee that consumers are in fact
44+
going to limit their changes to the parameters. A popular pattern is using
45+
kustomize to change output of other tools no matter what parameters had. In
46+
this particular case I am able to fork or patch this package and add:
47+
48+
```yaml
49+
securityContext:
50+
runAsNonRoot: false
51+
```
52+
53+
2. String replacements rarely describe the intent of the package author.
54+
When additional resources are added I need additional places where parameters
55+
need to be applied. I can easily add other containers to this deployment and
56+
the package author's rules are not clear and not easily validated.
57+
58+
## Solutions:
59+
60+
1. General ways to describe policy already exist. kpt has a [gatekeeper]
61+
function that allows the author to describe intended limitations for a class
62+
of resources or the entire package giving the consumer the freedom to customize
63+
and get an error or a warning when the policy is violated.
64+
65+
In the sample provided by the function we see how to provide a policy that will
66+
clearly describe the intent using rego:
67+
68+
```yaml
69+
apiVersion: templates.gatekeeper.sh/v1
70+
kind: ConstraintTemplate
71+
metadata: # kpt-merge: /disallowroot
72+
name: disallowroot
73+
spec:
74+
crd:
75+
spec:
76+
names:
77+
kind: DisallowRoot
78+
targets:
79+
- target: admission.k8s.gatekeeper.sh
80+
rego: |-
81+
package disallowroot
82+
violation[{"msg": msg}] {
83+
not input.review.object.spec.template.spec.securityContext.runAsNonRoot
84+
msg := "Containers must not run as root"
85+
}
86+
---
87+
apiVersion: constraints.gatekeeper.sh/v1beta1
88+
kind: DisallowRoot
89+
metadata: # kpt-merge: /disallowroot
90+
name: disallowroot
91+
spec:
92+
match:
93+
kinds:
94+
- apiGroups:
95+
- 'apps'
96+
kinds:
97+
- Deployment
98+
```
99+
100+
The Kptfile can enforce that resources comply with this policy every time
101+
`kpt fn render` is used:
102+
103+
```yaml
104+
apiVersion: kpt.dev/v1
105+
kind: Kptfile
106+
metadata:
107+
name: gatekeeper-disallow-root-user
108+
pipeline:
109+
validators:
110+
- image: gcr.io/kpt-fn/gatekeeper:v0.2
111+
```
112+
113+
[gatekeeper]: https://catalog.kpt.dev/gatekeeper/v0.2/

‎site/book/07-effective-customizations/03-generation.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
## Scenario:
2+
3+
When using template languages I am able to provide conditional statements based
4+
on parameter values. This allows me to ask the user for a little bit of
5+
information and generate a lot of boilerplate configuration. Some template
6+
languages like [Jinja] are very robust and feature rich.
7+
8+
## Problems:
9+
10+
1. Increased usage and additional edge cases make a template a piece of code
11+
that required tets and debugging.
12+
1. The interplay between different conditionals and loops is interleaved in the template making it hard to understand what exactly is configuration and what is
13+
the logic that alters the configuration. The consumer is left with one choice
14+
supply different parameter values, execute the template rendering code and see
15+
what happens.
16+
1. Templates are generally monolithic, when a change is introduced the package consumers need to either pay the cost of updating or the new consumers pay the
17+
cost of having to decipher more optional parameters.
18+
19+
## Solutions:
20+
21+
1. When the generated configuration is simple consider just using a sub-package
22+
and running customizations using [single value replacement] techniques.
23+
1. When a complex configuration needs to be generated the package author can
24+
create a generator function using turing complete languages and debugging tools. Example of such a function is [folder generation]. The output of the function
25+
is plain old KRM.
26+
27+
[folder generation]: https://catalog.kpt.dev/generate-folders/v0.1/
28+
[Jinja]: https://palletsprojects.com/p/jinja/
29+
[single value replacement]: /book/07-effective-customizations/01-single-value-replacement.md

‎site/book/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ This book is organized as follows:
1313
- [Chapter 4] covers how to use kpt functions to automate configuration changes.
1414
- [Chapter 5] guides you through developing custom functions.
1515
- [Chapter 6] covers how to deploy a package to a Kubernetes cluster.
16+
- [Chapter 7] covers effective customizations techniques.
1617

1718
Let's get started!
1819

@@ -22,3 +23,4 @@ Let's get started!
2223
[chapter 4]: /book/04-using-functions/
2324
[chapter 5]: /book/05-developing-functions/
2425
[chapter 6]: /book/06-deploying-packages/
26+
[chapter 7]: /book/07-effective-customizations/

‎site/sidebar.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,10 @@
3030
- [6.1 Initializing a Package for Apply](book/06-deploying-packages/01-initializing-a-package-for-apply.md)
3131
- [6.2 Applying a Package](book/06-deploying-packages/02-applying-a-package.md)
3232
- [6.3 Handling Dependencies](book/06-deploying-packages/03-handling-dependencies.md)
33+
- [7 Effective Customizations](book/07-effective-customizations/)
34+
- [7.1 Single Value Replacement](book/07-effective-customizations/01-single-value-replacement.md)
35+
- [7.2 Limiting Package Changes](book/07-effective-customizations/02-limiting-package-changes.md)
36+
- [7.3 Generation](book/07-effective-customizations/03-generation.md)
3337
- [Reference](reference/)
3438
- [Annotations](reference/annotations/)
3539
- [apply-time mutation](reference/annotations/apply-time-mutation/)

0 commit comments

Comments
 (0)