Edit This Page

v1.15 Release Notes

v1.15.0

Documentation

Downloads for v1.15.0

Client Binaries

Server Binaries

Node Binaries

Kubernetes v1.15 Release Notes

1.15 What’s New

A complete changelog for the release notes is now hosted in a customizable format at https://relnotes.k8s.io/. Check it out and please give us your feedback!

Kubernetes 1.15 consists of 25 enhancements: 2 moving to stable, 13 in beta, and 10 in alpha. The main themes of this release are:

Continuous Improvement

• Project sustainability is not just about features. Many SIGs have been working on improving test coverage, ensuring the basics stay reliable, and stability of the core feature set and working on maturing existing features and cleaning up the backlog.

Extensibility

• The community has been asking for continuing support of extensibility, so this cycle features more work around CRDs and API Machinery. Most of the enhancements in this cycle were from SIG API Machinery and related areas.

Extensibility around core Kubernetes APIs

CustomResourceDefinitions Pruning

To enforce both data consistency and security, Kubernetes performs pruning, or the automatic removal of unknown fields in objects sent to a Kubernetes API. An “unknown” field is one that is not specified in the OpenAPI validation schema. This behavior is already in place for native resources and ensures only data structures specified by the CRD developer are persisted to etcd. It will be available as a beta feature in Kubernetes 1.15.

Pruning is activated by setting spec.preserveUnknownFields: false in the CustomResourceDefinition. A future apiextensions.k8s.io/v1 variant of CRDs will enforce pruning.

Pruning requires that CRD developer provides complete, structural validation schemas, either at the top-level or for all versions of the CRD.

CustomResourceDefinition Defaulting

CustomResourceDefinitions also have new support for defaulting, with defaults specified using the default keyword in the OpenAPI validation schema. Defaults are set for unspecified fields in an object sent to the API, and when reading from etcd.

Defaulting will be available as alpha in Kubernetes 1.15 and requires structural schemas.

CustomResourceDefinition OpenAPI Publishing

OpenAPI specs for native types have long been served at /openapi/v2, and they are consumed by a number of components, notably kubectl client-side validation, kubectl explain and OpenAPI based client generators.

With Kubernetes 1.15 as beta, OpenAPI schemas are also published for CRDs, as long as their schemas are structural.

These changes are reflected in the following Kubernetes enhancements: (#383), (#575 ), (#492 ), (#598 ), (#692 ), (#95 ), (#995 ), (#956 )

Cluster Lifecycle Stability and Usability Improvements

Work on making Kubernetes installation, upgrade and configuration even more robust has been a major focus for this cycle for SIG Cluster Lifecycle (see the May 6, 2019 Community Update). Bug fixes across bare metal tooling and production-ready user stories, such as the high availability use cases have been given priority for 1.15.

kubeadm, the cluster lifecycle building block, continues to receive features and stability work required for bootstrapping production clusters efficiently. kubeadm has promoted high availability (HA) capability to beta, allowing users to use the familiar kubeadm init and kubeadm join commands to configure and deploy an HA control plane. An entire new test suite has been created specifically for ensuring these features will stay stable over time.

Certificate management has become more robust in 1.15, with kubeadm now seamlessly rotating all your certificates (on upgrades) before they expire. Check the kubeadm documentation for information on how to manage your certificates.

The kubeadm configuration file API is moving from v1beta1 to v1beta2 in 1.15.

These changes are reflected in the following Kubernetes enhancements: (#357 ), (#970 )

Continued improvement of CSI

In Kubernetes v1.15, SIG Storage continued work to enable migration of in-tree volume plugins to the Container Storage Interface (CSI). SIG Storage worked on bringing CSI to feature parity with in-tree functionality, including functionality like resizing, inline volumes, and more. SIG Storage introduces new alpha functionality in CSI that doesn’t exist in the Kubernetes Storage subsystem yet, like volume cloning.

Volume cloning enables users to specify another PVC as a “DataSource” when provisioning a new volume. If the underlying storage system supports this functionality and implements the “CLONE_VOLUME” capability in its CSI driver, then the new volume becomes a clone of the source volume.

These changes are reflected in the following Kubernetes enhancements: (#625)

Additional Notable Feature Updates

• Support for go modules in Kubernetes Core.
• Continued preparation for cloud provider extraction and code organization. The cloud provider code has been moved to kubernetes/legacy-cloud-providers for easier removal later and external consumption.
• Kubectl get and describe now works with extensions
• Nodes now support third party monitoring plugins.
• A new Scheduling Framework for schedule plugins is now Alpha.
• ExecutionHook API designed to trigger hook commands in containers is now Alpha.
• Continued deprecation of extensions/v1beta1, apps/v1beta1, and apps/v1beta2 APIs; these extensions will be retired in 1.16!

Check the release notes website for the complete changelog of notable features and fixes.

Known Issues

• Concurrently joining control-plane nodes does not work as expected in kubeadm 1.15.0. The feature was planned for release in 1.15.0, but a fix may come in a follow up patch release.

• Using --log-file is known to be problematic in 1.15. This presents as things being logged multiple times to the same file. The behaviour and details of this issue, as well as some preliminary attempts at fixing it are documented here

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

API Machinery

• k8s.io/kubernetes and published components (such as k8s.io/client-go and k8s.io/api) now contain go module files including dependency version information. See go-modules for details on consuming k8s.io/client-go using go modules. (#74877, @liggitt)

Apps

• Hyperkube short aliases have been removed from source code, because hyperkube docker image currently creates these aliases. (#76953, @Rand01ph)

Auth

• The Rancher credential provider has now been removed. This only affects you if you are using the downstream Rancher distro. (#77099, @dims)

AWS

• The system:aws-cloud-provider cluster role, deprecated in v1.13, is no longer auto-created. Deployments using the AWS cloud provider should grant required permissions to the aws-cloud-provider service account in the kube-system namespace as part of deployment. (#66635, @wgliang)

Azure

• Kubelet can now run without identity on Azure. A sample cloud provider configuration is: {"vmType": "vmss", "useInstanceMetadata": true, "subscriptionId": "<subscriptionId>"} (#77906, @feiskyer)
• Multiple Kubernetes clusters can now share the same resource group
  • When upgrading from previous releases, issues will arise with public IPs if multiple clusters share the same resource group. To solve these problems, make the following changes to the cluster: Recreate the relevant LoadBalancer services, or add a new tag ‘kubernetes-cluster-name: ’ manually for existing public IPs. Configure each cluster with a different cluster name using kube-controller-manager --cluster-name=<cluster-name> (#77630, @feiskyer)
• The cloud config for Azure cloud provider can now be initialized from Kubernetes secret azure-cloud-provider in kube-system namespace
  • the secret is a serialized version of azure.json file with key cloud-config. And the secret name is azure-cloud-provider.
  • A new option cloudConfigType has been added to the cloud-config file. Supported values are: file, secret and merge (merge is the default value).
  • To allow Azure cloud provider to read secrets, the RBAC rules should be configured.

CLI

• kubectl scale job, deprecated since 1.10, has been removed. (#78445, @soltysh)
• The deprecated --pod/-p flag for kubectl exec has been removed. The flag has been marked as deprecated since k8s version v1.12. (#76713, @prksu)

Lifecycle

• Support for deprecated old kubeadm v1alpha3 config has been totally removed. (#75179, @rosti)
• kube-up.sh no longer supports “centos” and “local” providers. (#76711, @dims)

Network

• The deprecated flag --conntrack-max has been removed from kube-proxy. Users of this flag should switch to --conntrack-min and --conntrack-max-per-core instead. (#78399, @rikatz)
• The deprecated kube-proxy flag --cleanup-iptables has been removed. (#78344, @aramase)

Node

• The deprecated kubelet security controls AllowPrivileged, HostNetworkSources, HostPIDSources, and HostIPCSources have been removed. Enforcement of these restrictions should be done through admission control (such as PodSecurityPolicy) instead. (#77820, @dims)
• The deprecated Kubelet flag --allow-privileged has been removed. Remove any use of the flag from your kubelet scripts or manifests. (#77820, @dims)
• The kubelet now only collects cgroups metrics for the node, container runtime, kubelet, pods, and containers. (#72787, @dashpole)

Storage

• The Node.Status.Volumes.Attached.DevicePath field is now unset for CSI volumes. You must update any external controllers that depend on this field. (#75799, @msau42)
• CSI alpha CRDs have been removed (#75747, @msau42)
• The StorageObjectInUseProtection admission plugin is enabled by default, so the default enabled admission plugins are now NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,StorageObjectInUseProtection. Please note that if you previously had not set the --admission-control flag, your cluster behavior may change (to be more standard). (#74610, @oomichi)

Deprecations and Removals

• kubectl

  • kubectl convert, deprecated since v1.14, will be removed in v1.17.
  • The --export flag for the kubectl get command, deprecated since v1.14, will be removed in v1.18.
  • The --pod/-p flag for kubectl exec, deprecated since 1.12, has been removed.
  • kubectl scale job, deprecated since 1.10, has been removed. (#78445, @soltysh)

• kubelet

  • The beta.kubernetes.io/os and beta.kubernetes.io/arch labels, deprecated since v1.14, are targeted for removal in v1.18.
  • The --containerized flag, deprecated since v1.14, will be removed in a future release.
  • cAdvisor json endpoints have been deprecated. (#78504, @dashpole)

• kube-apiserver

  • The --enable-logs-handler flag and log-serving functionality is deprecated, and scheduled to be removed in v1.19. (#77611, @rohitsardesai83)

• kube-proxy

  • The deprecated --cleanup-iptables has been removed,. (#78344, @aramase)

• API

  • Ingress resources will no longer be served from extensions/v1beta1 in v1.19. Migrate use to the networking.k8s.io/v1beta1 API, available since v1.14. Existing persisted data can be retrieved via the networking.k8s.io/v1beta1 API.
  • NetworkPolicy resources will no longer be served from extensions/v1beta1 in v1.16. Migrate use to the networking.k8s.io/v1 API, available since v1.8. Existing persisted data can be retrieved via the networking.k8s.io/v1 API.
  • PodSecurityPolicy resources will no longer be served from extensions/v1beta1 in v1.16. Migrate to the policy/v1beta1 API, available since v1.10. Existing persisted data can be retrieved via the policy/v1beta1 API.
  • DaemonSet, Deployment, and ReplicaSet resources will no longer be served from extensions/v1beta1, apps/v1beta1, or apps/v1beta2 in v1.16. Migrate to the apps/v1 API, available since v1.9. Existing persisted data can be retrieved via the apps/v1 API.
  • PriorityClass resources will no longer be served from scheduling.k8s.io/v1beta1 and scheduling.k8s.io/v1alpha1 in v1.17. Migrate use to the scheduling.k8s.io/v1 API, available since v1.14. Existing persisted data can be retrieved via the scheduling.k8s.io/v1 API.
  • The export query parameter for list API calls, deprecated since v1.14, will be removed in v1.18.
  • The series.state field in the events.k8s.io/v1beta1 Event API is deprecated and will be removed in v1.18 (#75987, @yastij)

• kubeadm

  • The kubeadm upgrade node config and kubeadm upgrade node experimental-control-plane commands are deprecated in favor of kubeadm upgrade node, and will be removed in a future release. (#78408, @fabriziopandini)
  • The flag --experimental-control-plane is now deprecated in favor of --control-plane. The flag --experimental-upload-certs is now deprecated in favor of --upload-certs (#78452, @fabriziopandini)
  • kubeadm config upload has been deprecated, as its replacement is now graduated. Please use kubeadm init phase upload-config instead. (#77946, @Klaven)

• The following features are now GA, and the associated feature gates are deprecated and will be removed in v1.17:

  • GCERegionalPersistentDisk

Metrics Changes

Added metrics

• The metric kube_proxy_sync_proxy_rules_last_timestamp_seconds is now available, indicating the last time that kube-proxy successfully applied proxying rules. (#74027, @squeed)
• process_start_time_seconds has been added to kubelet’s ‘/metrics/probes’ endpoint (#77975, @logicalhan)
• Scheduler: added metrics to record the number of pending pods in different queues (#75501, @Huang-Wei)
• Exposed CSI volume stats via kubelet volume metrics (#76188, @humblec)
• Added a new storage_operation_status_count metric for kube-controller-manager and kubelet to count success and error statues. (#75750, @msau42)

Deprecated/changed metrics

• kubelet probe metrics are now of the counter type rather than the gauge type, and the prober_probe_result has been replaced by prober_probe_total. (#76074, @danielqsj)
• The transformer_failures_total metric is deprecated in favor of transformation_operation_total. The old metric will continue to be populated but will be removed in a future release. (#70715, @immutableT)

• Introducing new semantic for metric volume_operation_total_seconds to be the end to end latency of volume provisioning/deletion. Existing metric “storage_operation_duration_seconds” will remain untouched, however it is exposed to the following potential issues:

  1. For volumes provisioned/deleted via external provisioner/deleter, storage_operation_duration_seconds will NOT wait for the external operation to be done before reporting latency metric (effectively close to 0). This will be fixed by using volume_operation_total_seconds instead
  2. if there’s a transient error happened during “provisioning/deletion”, i.e., a volume is still in-use while a deleteVolume has been called, original storage_operation_duration_seconds will NOT wait until a volume has been finally deleted before reporting an inaccurate latency metric. The newly implemented metric volume_operation_total_seconds, however, waits until a provisioning/deletion operation has been fully executed.

  Potential impacts: If an SLO/alert has been defined based on volume_operation_total_seconds, it might get violated because of the more accurate metric might be significantly larger than previously reported. The metric is defined to be a histogram and the new semantic could change the distribution. (#78061, @yuxiangqian)

• Implement the scheduling framework with Reserve, Prebind, Permit, Post-bind, Queue sort and Unreserve extension points. (#77567, @wgliang) (#77559, @ahg-g) (#77529, @draveness) (#77598, @danielqsj) (#77501, @JieJhih) (#77457, @danielqsj)

• Replaced *_admission_latencies_milliseconds_summary and *_admission_latencies_milliseconds metrics because they were reporting seconds rather than milliseconds. They were also subject to multiple naming guideline violations (units should be in base units and “duration” is the best practice labelling to measure the time a request takes). Please convert to use *_admission_duration_seconds and *_admission_duration_seconds_summary, as these now report the unit as described, and follow the instrumentation best practices. (#75279, @danielqsj)

• Fixed admission metrics histogram bucket sizes to cover 25ms to ~2.5 seconds. (#78608, @jpbetz)

• Fixed incorrect prometheus azure metrics. (#77722, @andyzhangx)

• kubectl scale job, deprecated since 1.10, has been removed. (#78445, @soltysh)

Notable Features

Stable

• You can now create a non-preempting Pod priority. If set on a class, the pod will continue to be prioritized above queued pods of a lesser class, but will not preempt running pods. (#74614, @denkensk)

• Third party device monitoring is now enabled by default (KubeletPodResources). (#77274, @RenaudWasTaken)

• The kube-apiserver’s watch can now be enabled for events using the --watch-cache-sizes flag. (#74321, @yastij)

Beta

• Admission webhooks can now register for a single version of a resource (for example, apps/v1 deployments) and be called when any other version of that resource is modified (for example extensions/v1beta1 deployments). This allows new versions of a resource to be handled by admission webhooks without needing to update every webhook to understand the new version. See the API documentation for the matchPolicy: Equivalent option in MutatingWebhookConfiguration and ValidatingWebhookConfiguration types. (#78135, @liggitt)
• The CustomResourcePublishOpenAPI feature is now beta and enabled by default. CustomResourceDefinitions with structural schemas now publish schemas in the OpenAPI document served at /openapi/v2. CustomResourceDefinitions with non-structural schemas have a NonStructuralSchema condition added with details about what needs to be corrected in the validation schema. (#77825, @roycaihw)
• Online volume expansion (ExpandInUsePersistentVolumes) is now a beta feature. As such, it is enabled by default. (#77755, @gnufied)
• The SupportNodePidsLimit feature is now beta, and enabled by default. It is no longer necessary to set the feature gate SupportNodePidsLimit=true. (#76221, @RobertKrawitz)
• kubeadm now includes the ability to specify certificate encryption and decryption keys for the upload and download certificate phases as part of the new v1beta2 kubeadm config format. (#77012, @rosti)
• You can now use kubeadm’s InitConfiguration and JoinConfiguration to define which preflight errors will be ignored. (#75499, @marccarre)
• CustomResourcesDefinition conversion via Web Hooks is promoted to beta. Note that you must set spec.preserveUnknownFields to false. (#78426, @sttts)
• Group Managed Service Account support has moved to a new API for beta. Special annotations for Windows GMSA support have been deprecated. (#75459, @wk8)
• The storageVersionHash feature is now beta. StorageVersionHash is a field in the discovery document of each resource. It enables clients to detect whether the storage version of that resource has changed. Its value must be treated as opaque by clients. Only equality comparison on the value is valid. (#78325, @caesarxuchao)
• Ingress objects are now persisted in etcd using the networking.k8s.io/v1beta1 version (#77139, @cmluciano)
• NodeLocal DNSCache graduating to beta. (#77887, @prameshj)

Alpha

• kubelet now allows the use of XFS quotas (on XFS and suitably configured ext4fs filesystems) to monitor storage consumption for ephemeral storage. This method of monitoring consumption, which is currently available only for emptyDir volumes, is faster and more accurate than the old method of walking the filesystem tree. Note that it does not enforce limits, it only monitors consumption. To utilize this functionality, set the feature gate LocalStorageCapacityIsolationFSQuotaMonitoring=true. For ext4fs filesystems, create the filesystem with mkfs.ext4 -O project <block_device> and run tune2fs -Q prjquotablock device; XFS filesystems need no additional preparation. The filesystem must be mounted with optionprojectin/etc/fstab. If the primary partition is the root filesystem, addrootflags=pquota` to the GRUB config file. (#66928, @RobertKrawitz)
• Finalizer Protection for Service LoadBalancers (ServiceLoadBalancerFinalizer) has been added as an Alpha feature, which is disabled by default. This feature ensures the Service resource is not fully deleted until the correlating load balancer resources are deleted. (#78262, @MrHohn)
• Inline CSI ephemeral volumes can now be controlled with PodSecurityPolicy when the CSIInlineVolume alpha feature is enabled. (#76915, @vladimirvivien)
• Kubernetes now includes an alpha field, AllowWatchBookmarks, in ListOptions for requesting the watching of bookmarks from apiserver. The implementation in apiserver is hidden behind the feature gate WatchBookmark. (#74074, @wojtek-t)

Staging Repositories

• The CRI API is now available in the k8s.io/cri-api staging repository. (#75531, @dims)
• Support for the Azure File plugin has been added to csi-translation-lib (CSIMigrationAzureFile). (#78356, @andyzhangx)
• Added support for Azure Disk plugin to csi-translation-lib (CSIMigrationAzureDisk) (#78330, @andyzhangx)

CLI Improvements

• Added kubeadm upgrade node. This command can be used to upgrade both secondary control-plane nodes and worker nodes. The kubeadm upgrade node config and kubeadm upgrade node experimental-control-plane commands are now deprecated. (#78408, @fabriziopandini)
• The kubectl top command now includes a --sort-by option to sort by memory or cpu. (#75920, @artmello)
• kubectl rollout restart now works for DaemonSets and StatefulSets. (#77423, @apelisse)
• kubectl get --watch=true now prints custom resource definitions with custom print columns. (#76161, @liggitt)
• Added kubeadm alpha certs certificate-key command to generate secure random key to use on kubeadm init --experimental-upload-certs (#77848, @yagonobre)
• Kubernetes now supports printing the volumeMode using kubectl get pv/pvc -o wide (#76646, @cwdsuzhou)
• Created a new kubectl rollout restart command that does a rolling restart of a deployment. (#76062, @apelisse)
• kubectl exec now allows using the resource name to select a matching pod and --pod-running-timeout flag to wait till at least one pod is running. (#73664, @prksu)
• kubeadm alpha certs renew and kubeadm upgrade now supports renewal of certificates embedded in KubeConfig files managed by kubeadm; this does not apply to certificates signed by external CAs. (#77180, @fabriziopandini)
• Kubeadm: a new command kubeadm alpha certs check-expiration was created in order to help users in managing expiration for local PKI certificates (#77863, @fabriziopandini)

Misc

• Service account controller clients to now use the TokenRequest API, and tokens are periodically rotated. (#72179, @WanLinghao)
• Added ListPager.EachListItem utility function to client-go to enable incremental processing of chunked list responses (#75849, @jpbetz)
• Object count quota is now supported for namespaced custom resources using the count/<resource>.<group> syntax. (#72384, @zhouhaibing089)
• Added completed job status in Cron Job event. (#75712, @danielqsj)
• Pod disruption budgets can now be updated and patched. (#69867, @davidmccormick)
• Add CRD spec.preserveUnknownFields boolean, defaulting to true in v1beta1 and to false in v1 CRDs. If false, fields not specified in the validation schema will be removed when sent to the API server or when read from etcd. (#77333, @sttts)
• Added RuntimeClass restrictions and defaulting to PodSecurityPolicy. (#73795, @tallclair)
• Kubelet plugin registration now has retry and exponential backoff logic for when registration of plugins (such as CSI or device plugin) fail. (#73891, @taragu)
• proxy/transport now supports Content-Encoding: deflate (#76551, @JieJhih)
• Admission webhooks are now properly called for scale and deployments/rollback subresources. (#76849, @liggitt)

API Changes

• CRDs get support for x-kubernetes-int-or-string to allow faithful representation of IntOrString types in CustomResources.(#78815, @sttts)
• Introduced the v1beta2 config format to kubeadm. (#76710, @rosti)
• Resource list requests for PartialObjectMetadata now correctly return list metadata like the resourceVersion and the continue token. (#75971, @smarterclayton)
• Added a condition NonStructuralSchema to CustomResourceDefinition listing Structural Schema violations as defined in the KEP. CRD authors should update their validation schemas to be structural in order to participate in future CRD features. (#77207, @sttts)
• Promoted meta.k8s.io/v1beta1 Table and PartialObjectMetadata to v1. (#77136, @smarterclayton)
• Introduced the flag --ipvs-strict-arp to configure stricter ARP sysctls, defaulting to false to preserve existing behaviors. This was enabled by default in 1.13.0, which impacted a few CNI plugins. (#75295, @lbernail)
• CRD validation schemas should not specify metadata fields other than name and generateName. A schema will not be considered structural (and therefore ready for future features) if metadata is specified in any other way. (#77653, @sttts)

Other notable changes

API Machinery

• Added port configuration to Admission webhook configuration service reference.
• Added port configuration to AuditSink webhook configuration service reference.
• Added port configuration to CRD Conversion webhook configuration service reference.
• Added port configuration to kube-aggregator service reference. (#74855, @mbohlool)
• Implemented deduplication logic for v1beta1.Event API (#65782, @yastij)
• Added objectSelector to admission webhook configurations. objectSelector is evaluated the oldObject and newObject that would be sent to the webhook, and is considered to match if either object matches the selector. A null object (oldObject in the case of create, or newObject in the case of delete) or an object that cannot have labels (like a DeploymentRollback or a PodProxyOptions object) is not considered to match. Use the object selector only if the webhook is opt-in, because end users may skip the admission webhook by setting the labels. (#78505, @caesarxuchao)
• Watch will now support converting response objects into Table or PartialObjectMetadata forms. (#71548, @smarterclayton)
• In CRD webhook conversion, Kubernetes will now ignore changes to metadata other than for labels and annotations. (#77743, @sttts)
• Added ListMeta.RemainingItemCount. When responding to a LIST request, if the server has more data available, and if the request does not contain label selectors or field selectors, the server sets the ListOptions.RemainingItemCount to the number of remaining objects. (#75993, @caesarxuchao)

• Clients may now request that API objects are converted to the v1.Table and v1.PartialObjectMetadata forms for generic access to objects. (#77448, @smarterclayton)

• Fixed a spurious error where update requests to the status subresource of multi-version custom resources would complain about an incorrect API version. (#78713, @liggitt)

• Fixed a bug in apiserver storage that could cause just-added finalizers to be ignored immediately following a delete request, leading to premature deletion. (#77619, @caesarxuchao)

• API requests rejected by admission webhooks which specify an http status code < 400 are now assigned a 400 status code. (#77022, @liggitt)

• Fixed a transient error API requests for custom resources could encounter while changes to the CustomResourceDefinition were being applied. (#77816, @liggitt) @smarterclayton)

• Added name validation for dynamic client methods in client-go (#75072, @lblackstone)

• CustomResourceDefinition with invalid regular expression in the pattern field of OpenAPI v3 validation schemas are no longer considered structural. (#78453, @sttts)

• API paging is now enabled by default in k8s.io/apiserver recommended options, and in k8s.io/sample-apiserver (#77278, @liggitt)

• Increased verbose level for local openapi aggregation logs to avoid flooding the log during normal operation (#75781, @roycaihw)

• k8s.io/client-go/dynamic/dynamicinformer.NewFilteredDynamicSharedInformerFactory now honours the namespace argument. (#77945, @michaelfig)

• client-go and kubectl no longer write cached discovery files with world-accessible file permissions. (#77874, @yuchengwu)

• Fixed an error with stuck informers when an etcd watch receives update or delete events with missing data. (#76675, @ryanmcnamara)

• DelayingQueue.ShutDown() can now be invoked multiple times without causing a closed channel panic. (#77170, @smarterclayton)

• When specifying an invalid value for a label, it was not always clear which label the value was specified for. Starting with this release, the label’s key is included in such error messages, which makes debugging easier. (#77144, @kenegozi)

• Fixed a regression error when proxying responses from aggregated API servers, which could cause watch requests to hang until the first event was received. (#75887, @liggitt)

• Fixed a bug where dry-run is not honored for pod/eviction sub-resource. (#76969, @apelisse)

• DeleteOptions parameters for deletecollection endpoints are now published in the OpenAPI spec. (#77843, @roycaihw)

• Active watches of custom resources now terminate properly if the CRD is modified. (#78029, @liggitt)

• Fixed a potential deadlock in the resource quota controller. Enabled recording partial usage info for quota objects specifying multiple resources, when only some of the resources’ usage can be determined. (#74747, @liggitt)

• Updates that remove remaining metadata.finalizers from an object that is pending deletion (non-nil metadata.deletionTimestamp) and has no graceful deletion pending (nil or 0 metadata.deletionGracePeriodSeconds) now results in immediate deletion of the object. (#77952, @liggitt)

• client-go: The rest.AnonymousClientConfig(*rest.Config) *rest.Config helper method no longer copies custom Transport and WrapTransport fields, because those can be used to inject user credentials. (#75771, @liggitt)

• Validating admission webhooks are now properly called for CREATE operations on the following resources: pods/binding, pods/eviction, bindings (#76910, @liggitt)

• Removed the function Parallelize, please convert to use the function ParallelizeUntil. (#76595, @danielqsj)

Apps

• Users can now specify a DataSource/Kind of type PersistentVolumeClaim in their PVC spec. This can then be detected by the external csi-provisioner and plugins if capable. (#76913, @j-griffith)
• Fixed bug in DaemonSetController causing it to stop processing some DaemonSets for 5 minutes after node removal. (#76060, @krzysztof-jastrzebski)
• StatefulSet controllers no longer force a resync every 30 seconds when nothing has changed. (#75622, @jonsabo)
• Enhanced the daemonset sync logic to avoid a problem where pods are thought to be unavailable when the controller’s clock is slower than the node’s clock. (#77208, @DaiHao)
• Fixed a bug that caused a DaemonSet rolling update to hang when its pod gets stuck at terminating. (#77773, @DaiHao)
• Route controller now respects rate limiting to the cloud provider on deletion; previously it was only for create. (#78581, @andrewsykim)
• Removed extra pod creation expectations when daemonset fails to create pods in batches. (#74856, @draveness)
• Resolved spurious rollouts of workload controllers when upgrading the API server, due to incorrect defaulting of an alpha procMount field in pods. (#78885, @liggitt)

Auth

• Fixed OpenID Connect (OIDC) token refresh when the client secret contains a special character. (#76914, @tsuna)
• Improved kubectl auth can-i command by warning users when they try to access a resource out of scope. (#76014, @WanLinghao)
• Validating admission webhooks are now properly called for CREATE operations on the following resources: tokenreviews, subjectaccessreviews, localsubjectaccessreviews, selfsubjectaccessreviews, selfsubjectrulesreviews (#76959, @sbezverk)

Autoscaling

• Horizontal Pod Autoscaling can now scale targets up even when one or more metrics are invalid/unavailable, as long as one metric indicates a scale up should occur. (#78503, @gjtempleton)

AWS

• Kubernetes will now use the zone from the node for topology aware aws-ebs volume creation to reduce unnecessary cloud provider calls. (#78276, @zhan849)
• Kubernetes now supports configure accessLogs for AWS NLB. (#78497, @M00nF1sh)
• Kubernetes now supports update LoadBalancerSourceRanges for AWS NLB(#74692, @M00nF1sh)
• Kubernetes now supports configure TLS termination for AWS NLB(#74910, @M00nF1sh)
• Kubernetes will now consume the AWS region list from the AWS SDK instead of a hard-coded list in the cloud provider. (#75990, @mcrute)
• Limit use of tags when calling EC2 API to prevent API throttling for very large clusters. (#76749, @mcrute)
• The AWS credential provider can now obtain ECR credentials even without the AWS cloud provider or being on an EC2 instance. Additionally, AWS credential provider caching has been improved to honor the ECR credential timeout. (#75587, @tiffanyfay)

Azure

• Kubernetes now supports specifying the Resource Group of the Route Table when updating the Pod network route on Azure. (#75580, @suker200)
• Kubernetes now uses instance-level update APIs for Azure VMSS loadbalancer operations. (#76656, @feiskyer)
• Users can now specify azure file share name in the azure file plugin, making it possible to use existing shares or specify a new share name. (#76988, @andyzhangx)
• You can now run kubelet with no Azure identity. A sample cloud provider configuration is: {"vmType": "vmss", "useInstanceMetadata": true, "subscriptionId": "<subscriptionId>"} (#77906, @feiskyer)
• Fixed some service tags not supported issues for Azure LoadBalancer service. (#77719, @feiskyer)
• Fixed an issue where pull image fails from a cross-subscription Azure Container Registry when using MSI to authenticate. (#77245, @norshtein)
• Azure cloud provider can now be configured by Kubernetes secrets and a new option cloudConfigType has been introduced. Candidate values are file, secret or merge (default is merge). Note that the secret is a serialized version of azure.json file with key cloud-config. And the secret name is azure-cloud-provider in kube-system namespace. (#78242, @feiskyer)

CLI

• Fixed kubectl exec usage string to correctly reflect flag placement. (#77589, @soltysh)
• Fixed kubectl describe cronjobs error of Successful Job History Limit. (#77347, @danielqsj)
• In the kubectl describe output, the fields with names containing special characters are now displayed as-is without any pretty formatting, avoiding awkward outputs. (#75483, @gsadhani)
• Fixed incorrect handling by kubectl of custom resources whose Kind is “Status”. (#77368, @liggitt)
• Report cp errors consistently, providing full message whether copying to or from a pod. (#77010, @soltysh)
• Preserved existing namespace information in manifests when running set ... --local commands. (#77267, @liggitt)
• Support for parsing more v1.Taint forms has been added. For example, key:effect, key=:effect- are now accepted. (#74159, @dlipovetsky)

Cloud Provider

• The GCE-only flag cloud-provider-gce-lb-src-cidrs is now optional for external cloud providers. (#76627, @timoreimann)
• Fixed a bug where cloud-controller-manager initializes nodes multiple times. (#75405, @tghartland)

Cluster Lifecycle

• kubeadm upgrade now renews all the certificates used by a component before upgrading the component itself, with the exception of certificates signed by external CAs. User can eventually opt-out of certificate renewal during upgrades by setting the new flag --certificate-renewal to false. (#76862, @fabriziopandini)
• kubeadm still generates RSA keys when deploying a node, but also accepts ECDSA keys if they already exist in the directory specified in the --cert-dir option. (#76390, @rojkov)
• kubeadm now implements CRI detection for Windows worker nodes (#78053, @ksubrmnn)

• Added --image-repository flag to kubeadm config images. (#75866, @jmkeyes)

• kubeadm: The kubeadm reset command has now been exposed as phases. (#77847, @yagonobre)

• kubeadm: Improved resiliency when it comes to updating the kubeadm-config configmap upon new control plane joins or resets. This allows for safe multiple control plane joins and/or resets. (#76821, @ereslibre)

• kubeadm: Bumped the minimum supported Docker version to 1.13.1 (#77051, @chenzhiwei)

• Reverted the CoreDNS version to 1.3.1 for kubeadm (#78545, @neolit123)

• kubeadm: Fixed the machine readability of kubeadm token create --print-join-command (#75487, @displague)

• kubeadm alpha certs renew --csr-only now reads the current certificates as the authoritative source for certificates attributes (same as kubeadm alpha certs renew). (#77780, @fabriziopandini)

• kubeadm: You can now delete multiple bootstrap tokens at once. (#75646, @bart0sh)

• util/initsystem: Added support for the OpenRC init system (#73101, @oz123)

• Default TTL for DNS records in kubernetes zone has been changed from 5s to 30s to keep consistent with old dnsmasq based kube-dns. The TTL can be customized with command kubectl edit -n kube-system configmap/coredns. (#76238, @Dieken)

• Communication between the etcd server and kube-apiserver on master is now overridden to use HTTPS instead of HTTP when mTLS is enabled in GCE. (#74690, @wenjiaswe)

GCP

• [stackdriver addon] Bumped prometheus-to-sd to v0.5.0 to pick up security fixes. [fluentd-gcp addon] Bumped fluentd-gcp-scaler to v0.5.1 to pick up security fixes. [fluentd-gcp addon] Bumped event-exporter to v0.2.4 to pick up security fixes. [fluentd-gcp addon] Bumped prometheus-to-sd to v0.5.0 to pick up security fixes. [metatada-proxy addon] Bumped prometheus-to-sd v0.5.0 to pick up security fixes. (#75362, @serathius)
• [fluentd-gcp addon] Bump fluentd-gcp-scaler to v0.5.2 to pick up security fixes. (#76762, @serathius)
• The GCERegionalPersistentDisk feature gate (GA in 1.13) can no longer be disabled. The feature gate will be removed in v1.17. (#77412, @liggitt)
• GCE/Windows: When the service cannot be stopped Stackdriver logging processes are now force killed (#77378, @yujuhong)
• Reduced GCE log rotation check from 1 hour to every 5 minutes. Rotation policy is unchanged (new day starts, log file size > 100MB). (#76352, @jpbetz)
• GCE/Windows: disabled stackdriver logging agent to prevent node startup failures (#76099, @yujuhong)
• API servers using the default Google Compute Engine bootstrapping scripts will have their insecure port (:8080) disabled by default. To enable the insecure port, set ENABLE_APISERVER_INSECURE_PORT=true in kube-env or as an environment variable. (#77447, @dekkagaijin)
• Fixed a NPD bug on GCI, so that it disables glog writing to files for log-counter. (#76211, @wangzhen127)
• Windows nodes on GCE now have the Windows firewall enabled by default. (#78507, @pjh)
• Added CNI_VERSION and CNI_SHA1 environment variables in kube-up.sh to configure CNI versions on GCE. (#76353, @Random-Liu)
• GCE clusters will include some IP ranges that are not used on the public Internet in the list of non-masq IPs. Bumped ip-masq-agent version to v2.3.0 with flag nomasq-all-reserved-ranges turned on. (#77458, @grayluck)
• GCE/Windows: added support for the stackdriver logging agent (#76850, @yujuhong)
• GCE Windows nodes will rely solely on kubernetes and kube-proxy (and not the GCE agent) for network address management. (#75855, @pjh)
• Ensured that the node-role.kubernetes.io/master taint is applied to the master with NoSchedule on GCE. (#78183, @cheftako)
• Windows nodes on GCE now use a known-working 1809 image rather than the latest 1809 image. (#76722, @pjh)
• kube-up.sh scripts now disable the KubeletPodResources feature for Windows nodes, due to issue #78628. (#78668, @mtaufen)

Instrumentation

• [metrics-server addon] Restored the ability to connect to nodes via IP addresses. (#76819, @serathius)
• If a pod has a running instance, the stats of its previously terminated instances will not show up in the kubelet summary stats any more for CRI runtimes such as containerd and cri-o. This keeps the behavior consistent with Docker integration, and fixes an issue that some container Prometheus metrics don’t work when there are summary stats for multiple instances of the same pod. (#77426, @Random-Liu)

Network

• Ingress objects are now persisted in etcd using the networking.k8s.io/v1beta1 version (#77139, @cmluciano)
• Transparent kube-proxy restarts when using IPVS are now allowed. (#75283, @lbernail)
• Packets considered INVALID by conntrack are now dropped. In particular, this fixes a problem where spurious retransmits in a long-running TCP connection to a service IP could result in the connection being closed with the error “Connection reset by peer” (#74840, @anfernee)
• kube-proxy no longer automatically cleans up network rules created by running kube-proxy in other modes. If you are switching the kube-proxy mode (EG: iptables to IPVS), you will need to run kube-proxy --cleanup, or restart the worker node (recommended) before restarting kube-proxy. If you are not switching kube-proxy between different modes, this change should not require any action. (#76109, @vllry)
• kube-proxy: HealthzBindAddress and MetricsBindAddress now support ipv6 addresses. (#76320, @JieJhih)

• The userspace proxy now respects the IPTables proxy’s minSyncInterval parameter. (#71735, @dcbw)

• iptables proxier: now routes local traffic to LB IPs to service chain (#77523, @andrewsykim)

• IPVS: Disabled graceful termination for UDP traffic to solve issues with high number of UDP connections (DNS / syslog in particular) (#77802, @lbernail)

• Fixed a bug where kube-proxy returns error due to existing ipset rules using a different hash type. (#77371, @andrewsykim)

• Fixed spurious error messages about failing to clean up iptables rules when using iptables 1.8. (#77303, @danwinship)

• Increased log level to 2 for IPVS graceful termination (#78395, @andrewsykim)

• kube-proxy: os exit when CleanupAndExit is set to true (#76732, @JieJhih)

• Kubernetes will now allow trailing dots in the externalName of Services of type ExternalName. (#78385, @thz)

Node

• The dockershim container runtime now accepts the docker runtime handler from a RuntimeClass. (#78323, @tallclair)
• The init container can now get its own field value as environment variable values using downwardAPI support. (#75109, @yuchengwu)
• UpdateContainerResources is no longer recorded as a container_status operation. It now uses the label update_container. (#75278, @Nessex)
• kubelet: fix fail to close kubelet->API connections on heartbeat failure when bootstrapping or client certificate rotation is disabled (#78016, @gaorong)
• Set selinux label at plugin socket directory (#73241, @vikaschoudhary16)
• Fixed detection of non-root image user ID.(#78261, @tallclair)
• Signal handling is now initialized within hyperkube commands that require it, such as apiserver and kubelet. (#76659, @S-Chan)
• The Kubelet now properly requests protobuf objects where they are supported from the apiserver, reducing load in large clusters. (#75602, @smarterclayton)

OpenStack

• You can now define a kubeconfig file for the OpenStack cloud provider. (#77415, @Fedosin)
• OpenStack user credentials can now be read from a secret instead of a local config file. (#75062, @Fedosin)

Release

• Removed hyperkube short aliases from source code because hyperkube docker image currently create these aliases. (#76953, @Rand01ph)

Scheduling

• Tolerations with the same key and effect will be merged into one that has the value of the latest toleration for best effort pods. (#75985, @ravisantoshgudimetla)
• Achieved 2X performance improvement on both required and preferred PodAffinity. (#76243, @Huang-Wei)
• Fixed a scheduler racing issue to ensure low priority pods are unschedulable on the node(s) where high priority pods have NominatedNodeName set to the node(s). (#77990, @Huang-Wei)

Storage

• Fixed issue with kubelet waiting on invalid devicepath on AWS (#78595, @gnufied)
• StorageOS volumes now show correct mount information (node and mount time) in the StorageOS administration CLI and UI. (#78522, @croomes)
• Fixed issue in Portworx volume driver causing controller manager to crash. (#76341, @harsh-px)
• For an empty regular file, stat --printf %F will now display regular empty file instead of regular file. (#62159, @dixudx)
• You can now have different operation names for different storage operations. This still prevents two operations on same volume from happening concurrently but if the operation changes, it resets the exponential backoff. (#75213, @gnufied)
• Reduced event spam for AttachVolume storage operation. (#75986, @mucahitkurt)
• Until this release, the iscsi plugin was waiting 10 seconds for a path to appear in the device list. However this timeout is not enough, or is less than the default device discovery timeout in most systems, which prevents certain devices from being discovered. This timeout has been raised to 30 seconds, which should help to avoid mount issues due to device discovery. (#78475, @humblec)
• Added a field to store CSI volume expansion secrets (#77516, @gnufied)
• Fixed a bug in block volume expansion. (#77317, @gnufied)
• Count PVCs that are unbound towards attach limit. (#73863, @gnufied)

VMware

• SAML token delegation (required for Zones support in vSphere) is now supported (#78876, @dougm)
• vSphere SAML token auth is now supported when using Zones (#75515, @dougm)

Windows

• Kubectl port-forward for Windows containers was added in v1.15. To use it, you’ll need to build a new pause image including WinCAT. (#75479, @benmoss)
• We’re working to simplify the Windows node join experience with better scripts and kubeadm. Scripts and doc updates are still in the works, but some of the needed improvements are included in 1.15. These include:
  • Windows kube-proxy will wait for HNS network creation on start (#78612, @ksubrmnn)
  • kubeadm: implemented CRI detection for Windows worker nodes (#78053, @ksubrmnn)
• Worked toward support for Windows Server version 1903, including adding Windows support for preserving the destination IP as the VIP when loadbalancing with DSR. (#74825, @ksubrmnn)
• Bug fix: Windows Kubelet nodes will now correctly search the default location for Docker credentials (%USERPROFILE%\.docker\config.json) when pulling images from a private registry. (https://kubernetes.io/docs/concepts/containers/images/#configuring-nodes-to-authenticate-to-a-private-registry) (#78528, @bclau)

Dependencies

Changed

• The default Go version was updated to 1.12.5. (#78528)
• cri-tools has been updated to v1.14.0. (#75658)
• Cluster Autoscaler has been updated to v1.15.0. (#78866)
• Kibana has been upgraded to v6.6.1. (#71251)
• CAdvisor has been updated to v0.33.2. (#76291)
• Fluentd-gcp-scaler has been upgraded to v0.5.2. (#76762)
• Fluentd in fluentd-elasticsearch has been upgraded to v1.4.2. (#76854)
• fluentd-elasticsearch has been updated to v2.5.2. (#76854)
• event-exporter has been updated to v0.2.5. (#77815)
• es-image has been updated to Elasticsearch 6.7.2. (#77765)
• metrics-server has been updated to v0.3.3. (#77950)
• ip-masq-agent has been updated to v2.4.1. (#77844)
• addon-manager has been updated to v9.0.1 (#77282)
• go-autorest has been updated to v11.1.2 (#77070)
• klog has been updated to 0.3.0 (#76474)
• k8s-dns-node-cache image has been updated to v1.15.1 (#76640, @george-angel)

Unchanged

• Default etcd server version remains unchanged at v3.3.10. The etcd client version was updated to v3.3.10. (#71615, #70168, #76917)
• The list of validated docker versions remains unchanged.
  • The current list is 1.13.1, 17.03, 17.06, 17.09, 18.06, 18.09. (#72823, #72831)
• CNI remains unchanged at v0.7.5. (#75455)
• CSI remains unchanged at to v1.1.0. (#75391)
• The dashboard add-on remains unchanged at v1.10.1. (#72495)
• kube-dns is unchanged at v1.14.13 as of Kubernetes 1.12. (#68900)
• Influxdb is unchanged at v1.3.3 as of Kubernetes 1.10. (#53319)
• Grafana is unchanged at v4.4.3 as of Kubernetes 1.10. (#53319)
• The fluent-plugin-kubernetes_metadata_filter plugin in fluentd-elasticsearch is unchanged at v2.1.6. (#71180)
• fluentd-gcp is unchanged at v3.2.0 as of Kubernetes 1.13. (#70954)
• OIDC authentication is unchanged at coreos/go-oidc v2 as of Kubernetes 1.10. (#58544)
• Calico is unchanged at v3.3.1 as of Kubernetes 1.13. (#70932)
• crictl on GCE was updated to v1.14.0. (#75658)
• CoreDNS is unchanged at v1.3.1 as of Kubernetes 1.14. (#78691)
• GLBC remains unchanged at v1.2.3 as of Kubernetes 1.12. (#66793)
• Ingress-gce remains unchanged at v1.2.3 as of Kubernetes 1.12. (#66793)
• v1.15.0-rc.1
• v1.15.0-beta.2
• v1.15.0-beta.1
• v1.15.0-alpha.3
• v1.15.0-alpha.2
• v1.15.0-alpha.1

v1.15.0-rc.1

Documentation

Downloads for v1.15.0-rc.1

Client Binaries

Server Binaries

Node Binaries

Changelog since v1.15.0-beta.2

Other notable changes

• Resolves spurious rollouts of workload controllers when upgrading the API server, due to incorrect defaulting of an alpha procMount field in pods (#78885, @liggitt)
• vSphere: allow SAML token delegation (required for Zones support) (#78876, @dougm)
• Update Cluster Autoscaler to 1.15.0; changelog: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.15.0 (#78866, @losipiuk)
• Revert the CoreDNS version to 1.3.1 (#78691, @rajansandeep)
• CRDs get support for x-kuberntes-int-or-string to allow faithful representation of IntOrString types in CustomResources. (#78815, @sttts)
• fix: retry detach azure disk issue (#78700, @andyzhangx)
  • try to only update vm if detach a non-existing disk when got <200, error> after detach disk operation
• Fix issue with kubelet waiting on invalid devicepath on AWS (#78595, @gnufied)
• Fixed a spurious error where update requests to the status subresource of multi-version custom resources would complain about an incorrect API version. (#78713, @liggitt)
• Fix admission metrics histogram bucket sizes to cover 25ms to ~2.5 seconds. (#78608, @jpbetz)
• Revert Promotion of resource quota scope selector to GA (#78696, @ravisantoshgudimetla)

v1.15.0-beta.2

Documentation

Downloads for v1.15.0-beta.2

Client Binaries

Server Binaries

Node Binaries

Changelog since v1.15.0-beta.1

Action Required

• ACTION REQUIRED The deprecated flag –conntrack-max has been removed from kube-proxy. Users of this flag should switch to –conntrack-min and –conntrack-max-per-core instead. (#78399, @rikatz)
• ACTION REQUIRED: kubeadm: the mixture of “–config” and “–certificate-key” is no longer allowed. The InitConfiguration and JoinConfiguration objects now support the “certificateKey” field and this field should be used instead of the command line argument in case a configuration file is already passed. (#78542, @neolit123)
• Azure cloud provider could now be configured by Kubernetes secrets and a new option cloudConfigType is introduced, whose candicate values are file, secret and merge (default is merge). (#78242, @feiskyer)
  • action required:
  • Since Azure cloud provider would read Kubernetes secrets, the following RBAC should be configured:
  • —
  • apiVersion: rbac.authorization.k8s.io/v1beta1
  • kind: ClusterRole
  • metadata:
  • labels:
  • kubernetes.io/cluster-service: “true”
  • name: system:azure-cloud-provider-secret-getter
  • rules:
  • - apiGroups: [“”]
  • resources: [“secrets”]
  • verbs:
  • - get
  • —
  • apiVersion: rbac.authorization.k8s.io/v1beta1
  • kind: ClusterRoleBinding
  • metadata:
  • labels:
  • kubernetes.io/cluster-service: “true”
  • name: system:azure-cloud-provider-secret-getter
  • roleRef:
  • apiGroup: rbac.authorization.k8s.io
  • kind: ClusterRole
  • name: system:azure-cloud-provider-secret-getter
  • subjects:
  • - kind: ServiceAccount
  • name: azure-cloud-provider
  • namespace: kube-system

Other notable changes

• kube-up.sh scripts now disable the KubeletPodResources feature for Windows nodes, due to issue #78628. (#78668, @mtaufen)
• StorageOS volumes now show correct mount information (node and mount time) in the StorageOS administration CLI and UI. (#78522, @croomes)
• Horizontal Pod Autoscaling can now scale targets up even when one or more metrics are invalid/unavailable as long as one metric indicates a scale up should occur. (#78503, @gjtempleton)
• kubeadm: revert the CoreDNS version to 1.3.1 (#78545, @neolit123)
• Move online volume expansion to beta (#77755, @gnufied)
• Fixes a memory leak in Kubelet on Windows caused by not not closing containers when fetching container metrics (#78594, @benmoss)
• Windows kube-proxy will wait for HNS network creation on start (#78612, @ksubrmnn)
• Fix error handling for loading initCfg in kubeadm upgrade and apply (#78611, @odinuge)
• Route controller now respects rate limiting to the cloud provider on deletion, previously it was only for create. (#78581, @andrewsykim)
• Windows Kubelet nodes will now correctly search the default location for Docker credentials (`%USERPROFILE%.docker* Windows nodes on GCE now have the Windows firewall enabled by default. (#78507, @pjh)
• Added objectSelector to admission webhook configurations. objectSelector is evaluated the oldObject and newObject that would be sent to the webhook, and is considered to match if either object matches the selector. A null object (oldObject in the case of create, or newObject in the case of delete) or an object that cannot have labels (like a DeploymentRollback or a PodProxyOptions object) is not considered to match. Use the object selector only if the webhook is opt-in, because end users may skip the admission webhook by setting the labels. (#78505, @caesarxuchao)
• Deprecate kubelet cAdvisor json endpoints (#78504, @dashpole)
• Supports configure accessLogs for AWS NLB (#78497, @M00nF1sh)
• Till this release, iscsi plugin was waiting 10 seconds for a path to appear in the device list. However this timeout is not enough or less than default device discovery timeout in most of the systems which cause certain device to be not accounted for the volume. This timeout has been lifted to 30seconds from this release and it should help to avoid mount issues due to device discovery. (#78475, @humblec)
• Remove deprecated –pod/-p flag from kubectl exec. The flag has been marked as deprecated since k8s version v1.12 (#76713, @prksu)
• CustomResourceDefinition with invalid regular expression in the pattern field of OpenAPI v3 validation schemas are not considere structural. (#78453, @sttts)
• Fixed panic in kube-proxy when parsing iptables-save output (#78428, @luksa)
• Remove deprecated flag –cleanup-iptables from kube-proxy (#78344, @aramase)
• The storageVersionHash feature is beta now. “StorageVersionHash” is a field in the discovery document of each resource. It allows clients to detect if the storage version of that resource has changed. Its value must be treated as opaque by clients. Only equality comparison on the value is valid. (#78325, @caesarxuchao)
• Use zone from node for topology aware aws-ebs volume creation to reduce unnecessary cloud provider calls (#78276, @zhan849)
• Finalizer Protection for Service LoadBalancers is now added as Alpha (disabled by default). This feature ensures the Service resource is not fully deleted until the correlating load balancer resources are deleted. (#78262, @MrHohn)
• Introducing new semantic for metric “volume_operation_total_seconds” to be the end to end latency of volume provisioning/deletion. Existing metric “storage_operation_duration_seconds” will remain untouched however exposed to the following potential issues: (#78061, @yuxiangqian)
  • 1. for volume’s provisioned/deleted via external provisioner/deleter, “storage_operation_duration_seconds” will NOT wait for the external operation to be done before reporting latency metric (effectively close to 0). This will be fixed by using “volume_operation_total_seconds” instead
  • 2. if there’s a transient error happened during “provisioning/deletion”, i.e., a volume is still in-use while a deleteVolume has been called, original “storage_operation_duration_seconds” will NOT wait until a volume has been finally deleted before reporting a not accurate latency metric. The newly implemented metric “volume_operation_total_seconds”, however, wait util a provisioning/deletion operation has been fully executed.
  • Potential impacts:
  • If an SLO/alert has been defined based on “volume_operation_total_seconds”, it might get violated because of the more accurate metric might be significantly larger than previously reported. The metric is defined to be a histogram and the new semantic could change the distribution.
• metrics added to kubelet endpoint ‘metrics/probes’: (#77975, @logicalhan)
  • process_start_time_seconds
• NodeLocal DNSCache graduating to beta. (#77887, @prameshj)
• Kubelet will attempt to use wincat.exe in the pause container for port forwarding when running on Windows (#75479, @benmoss)
• iptables proxier: route local traffic to LB IPs to service chain (#77523, @andrewsykim)
• When the number of jobs exceeds 500, cronjob should schedule without error. (#77475, @liucimin)
• Enable 3rd party device monitoring by default (#77274, @RenaudWasTaken)
• This change enables a user to specify a DataSource/Kind of type “PersistentVolumeClaim” in their PVC spec. This can then be detected by the external csi-provisioner and plugins if capable. (#76913, @j-griffith)
• proxy/transport: Support Content-Encoding: deflate (#76551, @JieJhih)
• Add –sort-by option to kubectl top command (#75920, @artmello)
• Introduce Topolgy into the runtimeClass API (#75744, @yastij)
• Kubelet plugin registration now has retry and exponential backoff logic for when registration of plugins (like CSI or device plugin) fail. (#73891, @taragu)
• Windows support for preserving the destination IP as the VIP when loadbalancing with DSR. (#74825, @ksubrmnn)
• Add NonPrempting field to the PriorityClass. (#74614, @denkensk)
• The kubelet only collects metrics for the node, container runtime, kubelet, pods, and containers. (#72787, @dashpole)
• Improved README for k8s.io/sample-apiserver (#73447, @MikeSpreitzer)
• kubeadm: flag “–experimental-control-plane” is now deprecated. use “–control-plane” instead (#78452, @fabriziopandini)
  • kubeadm: flag “–experimental-upload-certs” is now deprecated. use “–upload-certs” instead
• Promote resource quota scope selector to GA (#78448, @ravisantoshgudimetla)
• kubectl scale job, deprecated since 1.10, has been removed (#78445, @soltysh)
• CustomResourcesDefinition conversion via webhooks is promoted to beta. It requires that spec.preserveUnknownFields is set to false. (#78426, @sttts)
• kubeadm: a new command kubeadm upgrade node is introduced for upgrading nodes (both secondary control-plane nodes and worker nodes) (#78408, @fabriziopandini)
  • The command kubeadm upgrade node config is now deprecated; use kubeadm upgrade node instead.
  • The command kubeadm upgrade node experimental-control-plane is now deprecated; use kubeadm upgrade node instead.
• Increase log level to 2 for IPVS graceful termination (#78395, @andrewsykim)
• Add support for Azure File plugin to csi-translation-lib (#78356, @andyzhangx)
• refactor AWS NLB securityGroup handling (#74692, @M00nF1sh)
• Handle resize operation for volume plugins migrated to CSI (#77994, @gnufied)
• Inline CSI ephemeral volumes can now be controlled with PodSecurityPolicy when the CSIInlineVolume alpha feature is enabled (#76915, @vladimirvivien)
• Add support for Azure Disk plugin to csi-translation-lib (#78330, @andyzhangx)
• Ensures that the node-role.kubernetes.io/master taint is applied to the master with NoSchedule on GCE. (#78183, @cheftako)
• Add Post-bind extension point to the scheduling framework (#77567, @wgliang)
• Add CRD support for default values in OpenAPI v3 validation schemas. default values are set for object fields which are undefined in request payload and in data read from etcd. Defaulting is alpha and disabled by default, if the feature gate CustomResourceDefaulting is not enabled. (#77558, @sttts)
• kubeadm: v1beta2 InitConfiguration no longer embeds ClusterConfiguration it it. (#77739, @rosti)
• kube-apiserver: the --enable-logs-handler flag and log-serving functionality is deprecated, and scheduled to be removed in v1.19. (#77611, @rohitsardesai83)
• Fix vSphere SAML token auth when using Zones (#78137, @dougm)
• Admission webhooks can now register for a single version of a resource (for example, apps/v1 deployments) and be called when any other version of that resource is modified (for example extensions/v1beta1 deployments). This allows new versions of a resource to be handled by admission webhooks without needing to update every webhook to understand the new version. See the API documentation for the matchPolicy: Equivalent option in MutatingWebhookConfiguration and ValidatingWebhookConfiguration types. (#78135, @liggitt)
• Add kubeadm alpha certs certificate-key command to generate secure random key to use on kubeadm init --experimental-upload-certs (#77848, @yagonobre)
• IPVS: Disable graceful termination for UDP traffic to solve issues with high number of UDP connections (DNS / syslog in particular) (#77802, @lbernail)
• In CRD webhook conversion ignore changes to metadata other than for labels and annotations. (#77743, @sttts)
• Allow trailing dots in the externalName of Services of type ExternalName. (#78385, @thz)
• Fix a bug where kube-proxy returns error due to existing ipset rules using a different hash type. (#77371, @andrewsykim)
• kubeadm: implement CRI detection for Windows worker nodes (#78053, @ksubrmnn)

v1.15.0-beta.1

Documentation

Downloads for v1.15.0-beta.1

Client Binaries

Server Binaries

Node Binaries

Changelog since v1.15.0-alpha.3

Action Required

• ACTION REQUIRED: Deprecated Kubelet security controls AllowPrivileged, HostNetworkSources, HostPIDSources, HostIPCSources have been removed. Enforcement of these restrictions should be done through admission control instead (e.g. PodSecurityPolicy). (#77820, @dims)
  • ACTION REQUIRED: The deprecated Kubelet flag --allow-privileged has been removed. Remove any use of --allow-privileged from your kubelet scripts or manifests.
• Fix public IPs issues when multiple clusters are sharing the same resource group. (#77630, @feiskyer)
  • action required:
    • If the cluster is upgraded from old releases and the same resource group would be shared by multiple clusters, please recreate those LoadBalancer services or add a new tag ‘kubernetes-cluster-name: ’ manually for existing public IPs.
    • For multiple clusters sharing the same resource group, they should be configured with different cluster name by kube-controller-manager --cluster-name=<cluster-name>

Other notable changes

• fix azure retry issue when return 2XX with error (#78298, @andyzhangx)
• The dockershim container runtime now accepts the docker runtime handler from a RuntimeClass. (#78323, @tallclair)
• GCE: Disable the Windows defender to work around a bug that could cause nodes to crash and reboot (#78272, @yujuhong)
• The CustomResourcePublishOpenAPI feature is now beta and enabled by default. CustomResourceDefinitions with structural schemas now publish schemas in the OpenAPI document served at /openapi/v2. CustomResourceDefinitions with non-structural schemas have a NonStructuralSchema condition added with details about what needs to be corrected in the validation schema. (#77825, @roycaihw)
• kubeadm’s ignored pre-flight errors can now be configured via InitConfiguration and JoinConfiguration. (#75499, @marccarre)
• Fix broken detection of non-root image user ID (#78261, @tallclair)
• kubelet: fix fail to close kubelet->API connections on heartbeat failure when bootstrapping or client certificate rotation is disabled (#78016, @gaorong)
• remove vmsizelist call in azure disk GetVolumeLimits which happens in kubelet finally (#77851, @andyzhangx)
• reverts an aws-ebs volume provisioner optimization as we need to further discuss a viable optimization (#78200, @zhan849)
• API changes and deprecating the use of special annotations for Windows GMSA support (version beta) (#75459, @wk8)
• apiextensions: publish (only) structural OpenAPI schemas (#77554, @sttts)
• Set selinux label at plugin socket directory (#73241, @vikaschoudhary16)
• Fix a bug that causes DaemonSet rolling update to hang when its pod gets stuck at terminating. (#77773, @DaiHao)
• Kubeadm: a new command kubeadm alpha certs check-expiration was created in order to help users in managing expiration for local PKI certificates (#77863, @fabriziopandini)
• kubeadm: fix a bug related to volume unmount if the kubelet run directory is a symbolic link (#77507, @cuericlee)
• n/a (#78059, @figo)
• Add configuration options for the scheduling framework and its plugins. (#77501, @JieJhih)
• Publish DeleteOptions parameters for deletecollection endpoints in OpenAPI spec (#77843, @roycaihw)
• CoreDNS is now version 1.5.0 (#78030, @rajansandeep)
  • - A ready plugin has been included to report pod readiness
  • - The proxy plugin has been deprecated. The forward plugin is to be used instead.
  • - CoreDNS fixes the logging now that kubernetes’ client lib switched to klog from glog.
• Upgrade Azure network API version to 2018-07-01, so that EnableTcpReset could be enabled on Azure standard loadbalancer (SLB). (#78012, @feiskyer)
• Fixed a scheduler racing issue to ensure low priority pods to be unschedulable on the node(s) where high priority pods have NominatedNodeName set to the node(s). (#77990, @Huang-Wei)
• Support starting Kubernetes on GCE using containerd in COS and Ubuntu with KUBE_CONTAINER_RUNTIME=containerd. (#77889, @Random-Liu)
• DelayingQueue.ShutDown() is now able to be invoked multiple times without causing a closed channel panic. (#77170, @smarterclayton)
• For admission webhooks registered for DELETE operations on k8s built APIs or CRDs, the apiserver now sends the existing object as admissionRequest.Request.OldObject to the webhook. (#76346, @caesarxuchao)
  • For custom apiservers they uses the generic registry in the apiserver library, they get this behavior automatically.
• Expose CSI volume stats via kubelet volume metrics (#76188, @humblec)
• Active watches of custom resources now terminate properly if the CRD is modified. (#78029, @liggitt)
• Add CRD spec.preserveUnknownFields boolean, defaulting to true in v1beta1 and to false in v1 CRDs. If false, fields not specified in the validation schema will be removed when sent to the API server or when read from etcd. (#77333, @sttts)
• Updates that remove remaining metadata.finalizers from an object that is pending deletion (non-nil metadata.deletionTimestamp) and has no graceful deletion pending (nil or 0 metadata.deletionGracePeriodSeconds) now results in immediate deletion of the object. (#77952, @liggitt)
• Deprecates the kubeadm config upload command as it’s replacement is now graduated. Please see kubeadm init phase upload-config (#77946, @Klaven)
• k8s.io/client-go/dynamic/dynamicinformer.NewFilteredDynamicSharedInformerFactory now honours namespace argument (#77945, @michaelfig)
• kubectl rollout restart now works for daemonsets and statefulsets. (#77423, @apelisse)
• Fix incorrect azuredisk lun error (#77912, @andyzhangx)
• Kubelet could be run with no Azure identity now. A sample cloud provider configure is: {"vmType": "vmss", "useInstanceMetadata": true} (#77906, @feiskyer)
• client-go and kubectl no longer write cached discovery files with world-accessible file permissions (#77874, @yuchengwu)
• kubeadm: expose the kubeadm reset command as phases (#77847, @yagonobre)
• kubeadm: kubeadm alpha certs renew –csr-only now reads the current certificates as the authoritative source for certificates attributes (same as kubeadm alpha certs renew) (#77780, @fabriziopandini)
• Support “queue-sort” extension point for scheduling framework (#77529, @draveness)
• Allow init container to get its own field value as environment variable values(downwardAPI spport) (#75109, @yuchengwu)
• The metric kube_proxy_sync_proxy_rules_last_timestamp_seconds is now available, indicating the last time that kube-proxy successfully applied proxying rules. (#74027, @squeed)
• Fix panic logspam when running kubelet in standalone mode. (#77888, @tallclair)
• consume the AWS region list from the AWS SDK instead of a hard-coded list in the cloud provider (#75990, @mcrute)
• Add Option field to the admission webhook AdmissionReview API that provides the operation options (e.g. DeleteOption or CreateOption) for the operation being performed. (#77563, @jpbetz)
• Fix bug where cloud-controller-manager initializes nodes multiple times (#75405, @tghartland)
• Fixed a transient error API requests for custom resources could encounter while changes to the CustomResourceDefinition were being applied. (#77816, @liggitt)
• Fix kubectl exec usage string (#77589, @soltysh)
• CRD validation schemas should not specify metadata fields other than name and generateName. A schema will not be considered structural (and therefore ready for future features) if metadata is specified in any other way. (#77653, @sttts)
• Implement Permit extension point of the scheduling framework. (#77559, @ahg-g)
• Fixed a bug in the apiserver storage that could cause just-added finalizers to be ignored on an immediately following delete request, leading to premature deletion. (#77619, @caesarxuchao)
• add operation name for vm/vmss update operations in prometheus metrics (#77491, @andyzhangx)
• fix incorrect prometheus azure metrics (#77722, @andyzhangx)
• Clients may now request that API objects are converted to the v1.Table and v1.PartialObjectMetadata forms for generic access to objects. (#77448, @smarterclayton)
• ingress: Update in-tree Ingress controllers, examples, and clients to target networking.k8s.io/v1beta1 (#77617, @cmluciano)
• util/initsystem: add support for the OpenRC init system (#73101, @oz123)
• Signal handling is initialized within hyperkube commands that require it (apiserver, kubelet) (#76659, @S-Chan)
• Fix some service tags not supported issues for Azure LoadBalancer service (#77719, @feiskyer)
• Add Un-reserve extension point for the scheduling framework. (#77598, @danielqsj)
• Once merged, legacy cloud providers unit tests will run as part of ci, just as they were before they move from ./pkg/cloudproviders/providers (#77704, @khenidak)
• Check if container memory stats are available before accessing it (#77656, @yastij)
• Add a field to store CSI volume expansion secrets (#77516, @gnufied)
• Add a condition NonStructuralSchema to CustomResourceDefinition listing Structural Schema violations as defined in KEP https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/20190425-structural-openapi.md. CRD authors should update their validation schemas to be structural in order to participate in future CRD features. (#77207, @sttts)
• NONE (#74314, @oomichi)
• Update to use go 1.12.5 (#77528, @cblecker)
• Fix race conditions for Azure loadbalancer and route updates. (#77490, @feiskyer)
• remove VM API call dep in azure disk WaitForAttach (#77483, @andyzhangx)
• N/A (#77425, @figo)
• Fix TestEventChannelFull random fail (#76603, @changyaowei)
• aws-cloud-provider service account in the kube-system namespace need to be granted with list node permission with this optimization (#76976, @zhan849)
• Remove hyperkube short aliases from source code, Because hyperkube docker image currently create these aliases. (#76953, @Rand01ph)
• Allow to define kubeconfig file for OpenStack cloud provider. (#77415, @Fedosin)
• API servers using the default Google Compute Engine bootstrapping scripts will have their insecure port (:8080) disabled by default. To enable the insecure port, set ENABLE_APISERVER_INSECURE_PORT=true in kube-env or as an environment variable. (#77447, @dekkagaijin)
• GCE clusters will include some IP ranges that are not in used on the public Internet to the list of non-masq IPs. (#77458, @grayluck)
  • Bump ip-masq-agent version to v2.3.0 with flag nomasq-all-reserved-ranges turned on.
• Implement un-reserve extension point for the scheduling framework. (#77457, @danielqsj)
• If a pod has a running instance, the stats of its previously terminated instances will not show up in the kubelet summary stats any more for CRI runtimes like containerd and cri-o. (#77426, @Random-Liu)
  • This keeps the behavior consistent with Docker integration, and fixes an issue that some container Prometheus metrics don’t work when there are summary stats for multiple instances of the same pod.
• Limit use of tags when calling EC2 API to prevent API throttling for very large clusters (#76749, @mcrute)
• When specifying an invalid value for a label, it was not always (#77144, @kenegozi)
  • clear which label the value was specified for. Starting with this release, the
  • label’s key is included in such error messages, which makes debugging easier.

v1.15.0-alpha.3

Documentation

Downloads for v1.15.0-alpha.3

Client Binaries

Server Binaries

Node Binaries

Changelog since v1.15.0-alpha.2

Other notable changes

• Adding ListMeta.RemainingItemCount. When responding a LIST request, if the server has more data available, and if the request does not contain label selectors or field selectors, the server sets the ListOptions.RemainingItemCount to the number of remaining objects. (#75993, @caesarxuchao)
• This PR removes unused soak test cauldron (#77335, @loqutus)
• N/A (#76966, @figo)
• kubeadm: kubeadm alpha certs renew and kubeadm upgrade now supports renews of certificates embedded in KubeConfig files managed by kubeadm; this does not apply to certificates signed by external CAs. (#77180, @fabriziopandini)
• As of Kubernetes 1.15, the SupportNodePidsLimit feature introduced as alpha in Kubernetes 1.14 is now beta, and the ability to utilize it is enabled by default. It is no longer necessary to set the feature gate SupportNodePidsLimit=true. In all other respects, this functionality behaves as it did in Kubernetes 1.14. (#76221, @RobertKrawitz)
• Bump addon-manager to v9.0.1 (#77282, @MrHohn)
  • - Rebase image on debian-base:v1.0.0
• Fix kubectl describe CronJobs error of Successful Job History Limit. (#77347, @danielqsj)
• Remove extra pod creation expections when daemonset fails to create pods in batches. (#74856, @draveness)
• enhance the daemonset sync logic in clock-skew scenario (#77208, @DaiHao)
• GCE-only flag cloud-provider-gce-lb-src-cidrs becomes optional for external cloud providers. (#76627, @timoreimann)
• The GCERegionalPersistentDisk feature gate (GA in 1.13) can no longer be disabled. The feature gate will be removed in v1.17. (#77412, @liggitt)
• API requests rejected by admission webhooks which specify an http status code < 400 are now assigned a 400 status code. (#77022, @liggitt)
• kubeadm: Add ability to specify certificate encryption and decryption key for the upload/download certificates phases as part of the new v1beta2 kubeadm config format. (#77012, @rosti)
• Fixes incorrect handling by kubectl of custom resources whose Kind is “Status” (#77368, @liggitt)
• kubeadm: disable the kube-proxy DaemonSet on non-Linux nodes. This step is required to support Windows worker nodes. (#76327, @neolit123)
• Add etag for NSG updates so as to fix nsg race condition (#77210, @feiskyer)
• The series.state field in the events.k8s.io/v1beta1 Event API is deprecated and will be removed in v1.18 (#75987, @yastij)
• API paging is now enabled by default in k8s.io/apiserver recommended options, and in k8s.io/sample-apiserver (#77278, @liggitt)
• GCE/Windows: force kill Stackdriver logging processes when the service cannot be stopped (#77378, @yujuhong)
• ingress objects are now persisted in etcd using the networking.k8s.io/v1beta1 version (#77139, @cmluciano)
• [fluentd-gcp addon] Bump fluentd-gcp-scaler to v0.5.2 to pick up security fixes. (#76762, @serathius)
• Add RuntimeClass restrictions & defaulting to PodSecurityPolicy. (#73795, @tallclair)
• Promote meta.k8s.io/v1beta1 Table and PartialObjectMetadata to v1. (#77136, @smarterclayton)
• Fix bug with block volume expansion (#77317, @gnufied)
• Fixes spurious error messages about failing to clean up iptables rules when using iptables 1.8. (#77303, @danwinship)
• Add TLS termination support for NLB (#74910, @M00nF1sh)
• Preserves existing namespace information in manifests when running kubectl set ... --local commands (#77267, @liggitt)
• fix issue that pull image failed from a cross-subscription Azure Container Registry when using MSI to authenticate (#77245, @norshtein)
• Clean links handling in cp’s tar code (#76788, @soltysh)
• Implement and update interfaces and skeleton for the scheduling framework. (#75848, @bsalamat)
• Fixes segmentation fault issue with Protobuf library when log entries are deeply nested. (#77224, @qingling128)
• kubeadm: support sub-domain wildcards in certificate SANs (#76920, @sempr)
• Fixes an error with stuck informers when an etcd watch receives update or delete events with missing data (#76675, @ryanmcnamara)

v1.15.0-alpha.2

Documentation

Downloads for v1.15.0-alpha.2

Client Binaries

Server Binaries

Node Binaries

Changelog since v1.15.0-alpha.1

Other notable changes

• Kubemark scripts have been fixed for IKS clusters. (#76909, @Huang-Wei)
• fix azure disk list corruption issue (#77187, @andyzhangx)
• kubeadm: kubeadm upgrade now renews all the certificates used by one component before upgrading the component itself, with the exception of certificates signed by external CAs. User can eventually opt-out from certificate renewal during upgrades by setting the new flag –certificate-renewal to false. (#76862, @fabriziopandini)
• kube-proxy: os exit when CleanupAndExit is set to true (#76732, @JieJhih)
• kubectl exec now allows using resource name (e.g., deployment/mydeployment) to select a matching pod. (#73664, @prksu)
  • kubectl exec now allows using –pod-running-timeout flag to wait till at least one pod is running.
• kubeadm: add optional ECDSA support. (#76390, @rojkov)
  • kubeadm still generates RSA keys when deploying a node, but also accepts ECDSA
  • keys if they exist already in the directory specified in –cert-dir option.
• kube-proxy: HealthzBindAddress and MetricsBindAddress support ipv6 address. (#76320, @JieJhih)
• Packets considered INVALID by conntrack are now dropped. In particular, this fixes (#74840, @anfernee)
  • a problem where spurious retransmits in a long-running TCP connection to a service
  • IP could result in the connection being closed with the error “Connection reset by
  • peer”
• Introduce the v1beta2 config format to kubeadm. (#76710, @rosti)
• kubeadm: bump the minimum supported Docker version to 1.13.1 (#77051, @chenzhiwei)
• Rancher credential provider has now been removed (#77099, @dims)
• Support print volumeMode using kubectl get pv/pvc -o wide (#76646, @cwdsuzhou)
• Upgrade go-autorest to v11.1.2 (#77070, @feiskyer)
• Fixes a bug where dry-run is not honored for pod/eviction sub-resource. (#76969, @apelisse)
• Reduce event spam for AttachVolume storage operation (#75986, @mucahitkurt)
• Report cp errors consistently (#77010, @soltysh)
• specify azure file share name in azure file plugin (#76988, @andyzhangx)
• Migrate oom watcher not relying on cAdviosr’s API any more (#74942, @WanLinghao)
• Validating admission webhooks are now properly called for CREATE operations on the following resources: tokenreviews, subjectaccessreviews, localsubjectaccessreviews, selfsubjectaccessreviews, selfsubjectrulesreviews (#76959, @sbezverk)
• Fix OpenID Connect (OIDC) token refresh when the client secret contains a special character. (#76914, @tsuna)
• kubeadm: Improve resiliency when it comes to updating the kubeadm-config config map upon new control plane joins or resets. This allows for safe multiple control plane joins and/or resets. (#76821, @ereslibre)
• Validating admission webhooks are now properly called for CREATE operations on the following resources: pods/binding, pods/eviction, bindings (#76910, @liggitt)
• Default TTL for DNS records in kubernetes zone is changed from 5s to 30s to keep consistent with old dnsmasq based kube-dns. The TTL can be customized with command kubectl edit -n kube-system configmap/coredns. (#76238, @Dieken)
• Fixed a kubemark panic when hollow-node is morphed as proxy. (#76848, @Huang-Wei)
• k8s-dns-node-cache image version v1.15.1 (#76640, @george-angel)
• GCE/Windows: add support for stackdriver logging agent (#76850, @yujuhong)
• Admission webhooks are now properly called for scale and deployments/rollback subresources (#76849, @liggitt)
• Switch to instance-level update APIs for Azure VMSS loadbalancer operations (#76656, @feiskyer)
• kubeadm: kubeadm alpha cert renew now ignores certificates signed by external CAs (#76865, @fabriziopandini)
• Update to use go 1.12.4 (#76576, @cblecker)
• [metrics-server addon] Restore connecting to nodes via IP addresses (#76819, @serathius)
• fix detach azure disk back off issue which has too big lock in failure retry condition (#76573, @andyzhangx)
• Updated klog to 0.3.0 (#76474, @vincepri)
• kube-up.sh no longer supports “centos” and “local” providers (#76711, @dims)
• Ensure the backend pools are set correctly for Azure SLB with multiple backend pools (e.g. outbound rules) (#76691, @feiskyer)
• Windows nodes on GCE use a known-working 1809 image rather than the latest 1809 image. (#76722, @pjh)
• The userspace proxy now respects the IPTables proxy’s minSyncInterval parameter. (#71735, @dcbw)
• Kubeadm will now include the missing certificate key if it is unable to find an expected key during kubeadm join when used with the --experimental-control-plane flow (#76636, @mdaniel)

v1.15.0-alpha.1

Documentation

Downloads for v1.15.0-alpha.1

Client Binaries

Server Binaries

Node Binaries

Changelog since v1.14.0

Action Required

• client-go: The rest.AnonymousClientConfig(*rest.Config) *rest.Config helper method no longer copies custom Transport and WrapTransport fields, because those can be used to inject user credentials. (#75771, @liggitt)
• ACTION REQUIRED: The Node.Status.Volumes.Attached.DevicePath field is now unset for CSI volumes. Update any external controllers that depend on this field. (#75799, @msau42)

Other notable changes

• Remove the function Parallelize, please convert to use the function ParallelizeUntil. (#76595, @danielqsj)
• StorageObjectInUseProtection admission plugin is additionally enabled by default. (#74610, @oomichi)
  • So default enabled admission plugins are now NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,StorageObjectInUseProtection. Please note that if you previously had not set the --admission-control flag, your cluster behavior may change (to be more standard).
• Juju provider source moved to the Charmed Kubernetes org (#76628, @kwmonroe)
• improve kubectl auth can-i command by warning users when they try access resource out of scope (#76014, @WanLinghao)
• Introduce API for watch bookmark events. (#74074, @wojtek-t)
  • Introduce Alpha field AllowWatchBookmarks in ListOptions for requesting watch bookmarks from apiserver. The implementation in apiserver is hidden behind feature gate WatchBookmark (currently in Alpha stage).
• Override protocol between etcd server and kube-apiserver on master with HTTPS instead HTTP when mTLS is enabled in GCE (#74690, @wenjiaswe)
• Fix issue in Portworx volume driver causing controller manager to crash (#76341, @harsh-px)
• kubeadm: Fix a bug where if couple of CRIs are installed a user override of the CRI during join (via kubeadm join –cri-socket …) is ignored and kubeadm bails out with an error (#76505, @rosti)
• UpdateContainerResources is no longer recorded as a container_status operation. It now uses the label update_container (#75278, @Nessex)
• Bump metrics-server to v0.3.2 (#76437, @brett-elliott)
• The kubelet’s /spec endpoint no longer provides cloud provider information (cloud_provider, instance_type, instance_id). (#76291, @dims)
• Change kubelet probe metrics to counter type. (#76074, @danielqsj)
  • The metrics prober_probe_result is replaced by prober_probe_total.
• Reduce GCE log rotation check from 1 hour to every 5 minutes. Rotation policy is unchanged (new day starts, log file size > 100MB). (#76352, @jpbetz)
• Add ListPager.EachListItem utility function to client-go to enable incremental processing of chunked list responses (#75849, @jpbetz)
• Added CNI_VERSION and CNI_SHA1 environment variables in kube-up.sh to configure CNI versions on GCE. (#76353, @Random-Liu)
• Update cri-tools to v1.14.0 (#75658, @feiskyer)
• 2X performance improvement on both required and preferred PodAffinity. (#76243, @Huang-Wei)
• scheduler: add metrics to record number of pending pods in different queues (#75501, @Huang-Wei)
• Create a new kubectl rollout restart command that does a rolling restart of a deployment. (#76062, @apelisse)
• - Added port configuration to Admission webhook configuration service reference. (#74855, @mbohlool)
  • - Added port configuration to AuditSink webhook configuration service reference.
  • - Added port configuration to CRD Conversion webhook configuration service reference.
  • - Added port configuration to kube-aggregator service reference.
• kubectl get -w now prints custom resource definitions with custom print columns (#76161, @liggitt)
• Fixes bug in DaemonSetController causing it to stop processing some DaemonSets for 5 minutes after node removal. (#76060, @krzysztof-jastrzebski)
• no (#75820, @YoubingLi)
• Use stdlib to log stack trace when a panic occurs (#75853, @roycaihw)
• Fixes a NPD bug on GCI, so that it disables glog writing to files for log-counter (#76211, @wangzhen127)
• Tolerations with the same key and effect will be merged into one which has the value of the latest toleration for best effort pods. (#75985, @ravisantoshgudimetla)
• Fix empty array expansion error in cluster/gce/util.sh (#76111, @kewu1992)
• kube-proxy no longer automatically cleans up network rules created by running kube-proxy in other modes. If you are switching the mode that kube-proxy is in running in (EG: iptables to IPVS), you will need to run kube-proxy --cleanup, or restart the worker node (recommended) before restarting kube-proxy. (#76109, @vllry)
  • If you are not switching kube-proxy between different modes, this change should not require any action.
• Adds a new “storage_operation_status_count” metric for kube-controller-manager and kubelet to count success and error statues. (#75750, @msau42)
• GCE/Windows: disable stackdriver logging agent to prevent node startup failures (#76099, @yujuhong)
• StatefulSet controllers no longer force a resync every 30 seconds when nothing has changed. (#75622, @jonsabo)
• Ensures the conformance test image saves results before exiting when ginkgo returns non-zero value. (#76039, @johnSchnake)
• Add –image-repository flag to “kubeadm config images”. (#75866, @jmkeyes)
• Paginate requests from the kube-apiserver watch cache to etcd in chunks. (#75389, @jpbetz)
  • Paginate reflector init and resync List calls that are not served by watch cache.
• k8s.io/kubernetes and published components (like k8s.io/client-go and k8s.io/api) now publish go module files containing dependency version information. See http://git.k8s.io/client-go/INSTALL.md#go-modules for details on consuming k8s.io/client-go using go modules. (#74877, @liggitt)
• give users the option to suppress detailed output in integration test (#76063, @Huang-Wei)
• CSI alpha CRDs have been removed (#75747, @msau42)
• Fixes a regression proxying responses from aggregated API servers which could cause watch requests to hang until the first event was received (#75887, @liggitt)
• Support specify the Resource Group of Route Table when update Pod network route (Azure) (#75580, @suker200)
• Support parsing more v1.Taint forms. key:effect, key=:effect- are now accepted. (#74159, @dlipovetsky)
• Resource list requests for PartialObjectMetadata now correctly return list metadata like the resourceVersion and the continue token. (#75971, @smarterclayton)
• StubDomains and Upstreamnameserver which contains a service name will be omitted while translating to the equivalent CoreDNS config. (#75969, @rajansandeep)
• Count PVCs that are unbound towards attach limit (#73863, @gnufied)
• Increased verbose level for local openapi aggregation logs to avoid flooding the log during normal operation (#75781, @roycaihw)
• In the ‘kubectl describe’ output, the fields with names containing special characters are displayed as-is without any pretty formatting. (#75483, @gsadhani)
• Support both JSON and YAML for scheduler configuration. (#75857, @danielqsj)
• kubeadm: fix “upgrade plan” not defaulting to a “stable” version if no version argument is passed (#75900, @neolit123)
• clean up func podTimestamp in queue (#75754, @denkensk)
• The AWS credential provider can now obtain ECR credentials even without the AWS cloud provider or being on an EC2 instance. Additionally, AWS credential provider caching has been improved to honor the ECR credential timeout. (#75587, @tiffanyfay)
• Add completed job status in Cronjob event. (#75712, @danielqsj)
• kubeadm: implement deletion of multiple bootstrap tokens at once (#75646, @bart0sh)
• GCE Windows nodes will rely solely on kubernetes and kube-proxy (and not the GCE agent) for network address management. (#75855, @pjh)
• kubeadm: preflight checks on external etcd certificates are now skipped when joining a control-plane node with automatic copy of cluster certificates (–certificate-key) (#75847, @fabriziopandini)
• [stackdriver addon] Bump prometheus-to-sd to v0.5.0 to pick up security fixes. (#75362, @serathius)
  • [fluentd-gcp addon] Bump fluentd-gcp-scaler to v0.5.1 to pick up security fixes.
  • [fluentd-gcp addon] Bump event-exporter to v0.2.4 to pick up security fixes.
  • [fluentd-gcp addon] Bump prometheus-to-sd to v0.5.0 to pick up security fixes.
  • [metatada-proxy addon] Bump prometheus-to-sd v0.5.0 to pick up security fixes.
• Support describe pod with inline csi volumes (#75513, @cwdsuzhou)
• Object count quota is now supported for namespaced custom resources using the count/. syntax. (#72384, @zhouhaibing089)
• In case kubeadm can’t access the current Kubernetes version remotely and fails to parse (#72454, @rojkov)
  • the git-based version it falls back to a static predefined value of
  • k8s.io/kubernetes/cmd/kubeadm/app/constants.CurrentKubernetesVersion.
• Fixed a potential deadlock in resource quota controller (#74747, @liggitt) * Enabled recording partial usage info for quota objects specifying multiple resources, when only some of the resources’ usage can be determined.
• CRI API will now be available in the kubernetes/cri-api repository (#75531, @dims)
• Support vSphere SAML token auth when using Zones (#75515, @dougm)
• Transition service account controller clients to TokenRequest API (#72179, @WanLinghao)
• kubeadm: reimplemented IPVS Proxy check that produced confusing warning message. (#75036, @bart0sh)
• Allow to read OpenStack user credentials from a secret instead of a local config file. (#75062, @Fedosin)
• watch can now be enabled for events using the flag –watch-cache-sizes on kube-apiserver (#74321, @yastij)
• kubeadm: Support for deprecated old kubeadm v1alpha3 config is totally removed. (#75179, @rosti)
• The Kubelet now properly requests protobuf objects where they are (#75602, @smarterclayton)
  • supported from the apiserver, reducing load in large clusters.
• Add name validation for dynamic client methods in client-go (#75072, @lblackstone)
• Users may now execute get-kube-binaries.sh to request a client for an OS/Arch unlike the one of the host on which the script is invoked. (#74889, @akutz)
• Move config local to controllers in kube-controller-manager (#72800, @stewart-yu)
• Fix some potential deadlocks and file descriptor leaking for inotify watches. (#75376, @cpuguy83)
• [IPVS] Introduces flag ipvs-strict-arp to configure stricter ARP sysctls, defaulting to false to preserve existing behaviors. This was enabled by default in 1.13.0, which impacted a few CNI plugins. (#75295, @lbernail)
• [IPVS] Allow for transparent kube-proxy restarts (#75283, @lbernail)
• Replace *_admission_latencies_milliseconds_summary and *_admission_latencies_milliseconds metrics due to reporting wrong unit (was labelled milliseconds, but reported seconds), and multiple naming guideline violations (units should be in base units and “duration” is the best practice labelling to measure the time a request takes). Please convert to use *_admission_duration_seconds and *_admission_duration_seconds_summary, these now report the unit as described, and follow the instrumentation best practices. (#75279, @danielqsj)
• Reset exponential backoff when storage operation changes (#75213, @gnufied)
• Watch will now support converting response objects into Table or PartialObjectMetadata forms. (#71548, @smarterclayton)
• N/A (#74974, @goodluckbot)
• kubeadm: fix the machine readability of “kubeadm token create –print-join-command” (#75487, @displague)
• Update Cluster Autoscaler to 1.14.0; changelog: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.14.0 (#75480, @losipiuk)

フィードバック

このページは役に立ちましたか？

Thanks for the feedback. If you have a specific, answerable question about how to use Kubernetes, ask it on Stack Overflow. Open an issue in the GitHub repo if you want to 問題を報告する or 改善を提案.