Skip to content

Commit 62e73f3

Browse files
a7ia7i
committed
feat: manage PodDisruptionBudget for SparkApplication driver and executor
Signed-off-by: Amir Alavi <[email protected]>
1 parent 22e4fb8 commit 62e73f3

12 files changed

+750
-69
lines changed

‎api/v1beta2/sparkapplication_types.go

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ package v1beta2
1919
import (
2020
corev1 "k8s.io/api/core/v1"
2121
networkingv1 "k8s.io/api/networking/v1"
22+
policyv1 "k8s.io/api/policy/v1"
2223
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
2324
)
2425

@@ -549,6 +550,9 @@ type DriverSpec struct {
549550
// PriorityClassName is the name of the PriorityClass for the driver pod.
550551
// +optional
551552
PriorityClassName *string `json:"priorityClassName,omitempty"`
553+
// PodDisruptionBudgetSpec is the PodDisruptionBudget specification for the Spark Driver.
554+
// +optional
555+
PodDisruptionBudgetSpec *policyv1.PodDisruptionBudgetSpec `json:"podDisruptionBudgetSpec,omitempty"`
552556
}
553557

554558
// ExecutorSpec is specification of the executor.
@@ -579,6 +583,9 @@ type ExecutorSpec struct {
579583
// PriorityClassName is the name of the PriorityClass for the executor pod.
580584
// +optional
581585
PriorityClassName *string `json:"priorityClassName,omitempty"`
586+
// PodDisruptionBudgetSpec is the PodDisruptionBudget specification for the Spark Executors.
587+
// +optional
588+
PodDisruptionBudgetSpec *policyv1.PodDisruptionBudgetSpec `json:"podDisruptionBudgetSpec,omitempty"`
582589
}
583590

584591
// NamePath is a pair of a name and a path to which the named objects should be mounted to.

‎api/v1beta2/zz_generated.deepcopy.go

Lines changed: 11 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎config/crd/bases/sparkoperator.k8s.io_scheduledsparkapplications.yaml

Lines changed: 212 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2981,6 +2981,112 @@ spec:
29812981
NodeSelector is the Kubernetes node selector to be added to the driver and executor pods.
29822982
This field is mutually exclusive with nodeSelector at SparkApplication level (which will be deprecated).
29832983
type: object
2984+
podDisruptionBudgetSpec:
2985+
description: PodDisruptionBudgetSpec is the PodDisruptionBudget
2986+
specification for the Spark Driver.
2987+
properties:
2988+
maxUnavailable:
2989+
anyOf:
2990+
- type: integer
2991+
- type: string
2992+
description: |-
2993+
An eviction is allowed if at most "maxUnavailable" pods selected by
2994+
"selector" are unavailable after the eviction, i.e. even in absence of
2995+
the evicted pod. For example, one can prevent all voluntary evictions
2996+
by specifying 0. This is a mutually exclusive setting with "minAvailable".
2997+
x-kubernetes-int-or-string: true
2998+
minAvailable:
2999+
anyOf:
3000+
- type: integer
3001+
- type: string
3002+
description: |-
3003+
An eviction is allowed if at least "minAvailable" pods selected by
3004+
"selector" will still be available after the eviction, i.e. even in the
3005+
absence of the evicted pod. So for example you can prevent all voluntary
3006+
evictions by specifying "100%".
3007+
x-kubernetes-int-or-string: true
3008+
selector:
3009+
description: |-
3010+
Label query over pods whose evictions are managed by the disruption
3011+
budget.
3012+
A null selector will match no pods, while an empty ({}) selector will select
3013+
all pods within the namespace.
3014+
properties:
3015+
matchExpressions:
3016+
description: matchExpressions is a list of label selector
3017+
requirements. The requirements are ANDed.
3018+
items:
3019+
description: |-
3020+
A label selector requirement is a selector that contains values, a key, and an operator that
3021+
relates the key and values.
3022+
properties:
3023+
key:
3024+
description: key is the label key that the selector
3025+
applies to.
3026+
type: string
3027+
operator:
3028+
description: |-
3029+
operator represents a key's relationship to a set of values.
3030+
Valid operators are In, NotIn, Exists and DoesNotExist.
3031+
type: string
3032+
values:
3033+
description: |-
3034+
values is an array of string values. If the operator is In or NotIn,
3035+
the values array must be non-empty. If the operator is Exists or DoesNotExist,
3036+
the values array must be empty. This array is replaced during a strategic
3037+
merge patch.
3038+
items:
3039+
type: string
3040+
type: array
3041+
required:
3042+
- key
3043+
- operator
3044+
type: object
3045+
type: array
3046+
matchLabels:
3047+
additionalProperties:
3048+
type: string
3049+
description: |-
3050+
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
3051+
map is equivalent to an element of matchExpressions, whose key field is "key", the
3052+
operator is "In", and the values array contains only "value". The requirements are ANDed.
3053+
type: object
3054+
type: object
3055+
x-kubernetes-map-type: atomic
3056+
unhealthyPodEvictionPolicy:
3057+
description: |-
3058+
UnhealthyPodEvictionPolicy defines the criteria for when unhealthy pods
3059+
should be considered for eviction. Current implementation considers healthy pods,
3060+
as pods that have status.conditions item with type="Ready",status="True".
3061+
3062+
3063+
Valid policies are IfHealthyBudget and AlwaysAllow.
3064+
If no policy is specified, the default behavior will be used,
3065+
which corresponds to the IfHealthyBudget policy.
3066+
3067+
3068+
IfHealthyBudget policy means that running pods (status.phase="Running"),
3069+
but not yet healthy can be evicted only if the guarded application is not
3070+
disrupted (status.currentHealthy is at least equal to status.desiredHealthy).
3071+
Healthy pods will be subject to the PDB for eviction.
3072+
3073+
3074+
AlwaysAllow policy means that all running pods (status.phase="Running"),
3075+
but not yet healthy are considered disrupted and can be evicted regardless
3076+
of whether the criteria in a PDB is met. This means perspective running
3077+
pods of a disrupted application might not get a chance to become healthy.
3078+
Healthy pods will be subject to the PDB for eviction.
3079+
3080+
3081+
Additional policies may be added in the future.
3082+
Clients making eviction decisions should disallow eviction of unhealthy pods
3083+
if they encounter an unrecognized policy in this field.
3084+
3085+
3086+
This field is beta-level. The eviction API uses this field when
3087+
the feature gate PDBUnhealthyPodEvictionPolicy is enabled (enabled by default).
3088+
type: string
3089+
type: object
29843090
podName:
29853091
description: |-
29863092
PodName is the name of the driver pod that the user creates. This is used for the
@@ -7767,6 +7873,112 @@ spec:
77677873
NodeSelector is the Kubernetes node selector to be added to the driver and executor pods.
77687874
This field is mutually exclusive with nodeSelector at SparkApplication level (which will be deprecated).
77697875
type: object
7876+
podDisruptionBudgetSpec:
7877+
description: PodDisruptionBudgetSpec is the PodDisruptionBudget
7878+
specification for the Spark Executors.
7879+
properties:
7880+
maxUnavailable:
7881+
anyOf:
7882+
- type: integer
7883+
- type: string
7884+
description: |-
7885+
An eviction is allowed if at most "maxUnavailable" pods selected by
7886+
"selector" are unavailable after the eviction, i.e. even in absence of
7887+
the evicted pod. For example, one can prevent all voluntary evictions
7888+
by specifying 0. This is a mutually exclusive setting with "minAvailable".
7889+
x-kubernetes-int-or-string: true
7890+
minAvailable:
7891+
anyOf:
7892+
- type: integer
7893+
- type: string
7894+
description: |-
7895+
An eviction is allowed if at least "minAvailable" pods selected by
7896+
"selector" will still be available after the eviction, i.e. even in the
7897+
absence of the evicted pod. So for example you can prevent all voluntary
7898+
evictions by specifying "100%".
7899+
x-kubernetes-int-or-string: true
7900+
selector:
7901+
description: |-
7902+
Label query over pods whose evictions are managed by the disruption
7903+
budget.
7904+
A null selector will match no pods, while an empty ({}) selector will select
7905+
all pods within the namespace.
7906+
properties:
7907+
matchExpressions:
7908+
description: matchExpressions is a list of label selector
7909+
requirements. The requirements are ANDed.
7910+
items:
7911+
description: |-
7912+
A label selector requirement is a selector that contains values, a key, and an operator that
7913+
relates the key and values.
7914+
properties:
7915+
key:
7916+
description: key is the label key that the selector
7917+
applies to.
7918+
type: string
7919+
operator:
7920+
description: |-
7921+
operator represents a key's relationship to a set of values.
7922+
Valid operators are In, NotIn, Exists and DoesNotExist.
7923+
type: string
7924+
values:
7925+
description: |-
7926+
values is an array of string values. If the operator is In or NotIn,
7927+
the values array must be non-empty. If the operator is Exists or DoesNotExist,
7928+
the values array must be empty. This array is replaced during a strategic
7929+
merge patch.
7930+
items:
7931+
type: string
7932+
type: array
7933+
required:
7934+
- key
7935+
- operator
7936+
type: object
7937+
type: array
7938+
matchLabels:
7939+
additionalProperties:
7940+
type: string
7941+
description: |-
7942+
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
7943+
map is equivalent to an element of matchExpressions, whose key field is "key", the
7944+
operator is "In", and the values array contains only "value". The requirements are ANDed.
7945+
type: object
7946+
type: object
7947+
x-kubernetes-map-type: atomic
7948+
unhealthyPodEvictionPolicy:
7949+
description: |-
7950+
UnhealthyPodEvictionPolicy defines the criteria for when unhealthy pods
7951+
should be considered for eviction. Current implementation considers healthy pods,
7952+
as pods that have status.conditions item with type="Ready",status="True".
7953+
7954+
7955+
Valid policies are IfHealthyBudget and AlwaysAllow.
7956+
If no policy is specified, the default behavior will be used,
7957+
which corresponds to the IfHealthyBudget policy.
7958+
7959+
7960+
IfHealthyBudget policy means that running pods (status.phase="Running"),
7961+
but not yet healthy can be evicted only if the guarded application is not
7962+
disrupted (status.currentHealthy is at least equal to status.desiredHealthy).
7963+
Healthy pods will be subject to the PDB for eviction.
7964+
7965+
7966+
AlwaysAllow policy means that all running pods (status.phase="Running"),
7967+
but not yet healthy are considered disrupted and can be evicted regardless
7968+
of whether the criteria in a PDB is met. This means perspective running
7969+
pods of a disrupted application might not get a chance to become healthy.
7970+
Healthy pods will be subject to the PDB for eviction.
7971+
7972+
7973+
Additional policies may be added in the future.
7974+
Clients making eviction decisions should disallow eviction of unhealthy pods
7975+
if they encounter an unrecognized policy in this field.
7976+
7977+
7978+
This field is beta-level. The eviction API uses this field when
7979+
the feature gate PDBUnhealthyPodEvictionPolicy is enabled (enabled by default).
7980+
type: string
7981+
type: object
77707982
podSecurityContext:
77717983
description: PodSecurityContext specifies the PodSecurityContext
77727984
to apply.

0 commit comments

Comments
 (0)