Key Default Type Description
kubernetes.operator.job.autoscaler.catch-up.duration
10 min Duration The target duration for fully processing any backlog after a scaling operation. Set to 0 to disable backlog based scaling.
kubernetes.operator.job.autoscaler.enabled
false Boolean Enable job autoscaler module.
kubernetes.operator.job.autoscaler.history.max.age
86400000 ms Duration Maximum age for past scaling decisions to retain.
kubernetes.operator.job.autoscaler.history.max.count
1 Integer Maximum number of past scaling decisions to retain per vertex.
kubernetes.operator.job.autoscaler.metrics.busy-time.aggregator
MAX

Enum

Metric aggregator to use for busyTime metrics. This affects how true processing/output rate will be computed. Using max allows us to handle jobs with data skew more robustly, while avg may provide better stability when we know that the load distribution is even.

Possible values:
  • "AVG"
  • "MAX"
  • "MIN"
kubernetes.operator.job.autoscaler.metrics.window
5 min Duration Scaling metrics aggregation window size.
kubernetes.operator.job.autoscaler.restart.time
5 min Duration Expected restart time to be used until the operator can determine it reliably from history.
kubernetes.operator.job.autoscaler.scale-down.max-factor
0.6 Double Max scale down factor. 1 means no limit on scale down, 0.6 means job can only be scaled down with 60% of the original parallelism.
kubernetes.operator.job.autoscaler.scale-up.grace-period
10 min Duration Period in which no scale down is allowed after a scale up
kubernetes.operator.job.autoscaler.scaling.effectiveness.detection.enabled
true Boolean Whether to enable detection of ineffective scaling operations and allowing the autoscaler to block further scale ups.
kubernetes.operator.job.autoscaler.scaling.effectiveness.threshold
0.1 Double Processing rate increase threshold for detecting ineffective scaling threshold. 0.1 means if we do not accomplish at least 10% of the desired capacity increase with scaling, the action is marked ineffective.
kubernetes.operator.job.autoscaler.scaling.enabled
true Boolean Enable vertex scaling execution by the autoscaler. If disabled, the autoscaler will only collect metrics and evaluate the suggested parallelism for each vertex but will not upgrade the jobs.
kubernetes.operator.job.autoscaler.scaling.sources.enabled
true Boolean Whether to enable scaling source vertices. Source vertices set the baseline ingestion rate for the processing based on the backlog size. If disabled, only regular job vertices will be scaled and source vertices will be unchanged.
kubernetes.operator.job.autoscaler.stabilization.interval
5 min Duration Stabilization period in which no new scaling will be executed
kubernetes.operator.job.autoscaler.target.utilization
0.7 Double Target vertex utilization
kubernetes.operator.job.autoscaler.target.utilization.boundary
0.1 Double Target vertex utilization boundary. Scaling won't be performed if utilization is within (target - boundary, target + boundary)
kubernetes.operator.job.autoscaler.vertex.max-parallelism
2147483647 Integer The maximum parallelism the autoscaler can use. Note that this limit will be ignored if it is higher than the max parallelism configured in the Flink config or directly on each operator.
kubernetes.operator.job.autoscaler.vertex.min-parallelism
1 Integer The minimum parallelism the autoscaler can use.