[Bug 1867398] Re: [Regression] unsupported protocol scheme

[Dariusz Gadomski]

** Tags removed: sts-needs-sponsor
** Tags added: sts-sponsor-dgadomski

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1867398

Title:
  [Regression] unsupported protocol scheme

Status in containerd package in Ubuntu:
  Fix Released
Status in containerd source package in Bionic:
  New
Status in containerd source package in Eoan:
  Fix Released
Status in containerd source package in Focal:
  Fix Released

Bug description:
  [Environment]

  Kubernetes 1.16.17
  Containerd 1.3.3
  Ubuntu Bionic

  [Affected Releases]

   containerd | 1.3.3-0ubuntu1~18.04.1 | bionic-updates/universe  | source, amd64, arm64, armhf, i386, ppc64el, s390x
   containerd | 1.3.3-0ubuntu1~19.10.1 | eoan-updates/universe    | source, amd64, arm64, armhf, i386, ppc64el, s390x
   containerd | 1.3.3-0ubuntu1         | focal                    | source, amd64, arm64, armhf, ppc64el, s390x

  
  [Description]

  Reported upstream:
  https://github.com/containerd/containerd/issues/4108

  The bump of to version 1.3.3 through [0]
  https://bugs.launchpad.net/ubuntu/+source/containerd/+bug/1854841

  Caused a regression.

  The following endpoint description works with containerd 1.2.X without defining
  a protocol scheme. (/etc/containerd/config.toml).

      [plugins."io.containerd.grpc.v1.cri".registry.mirrors."niedbalski-bastion.cloud.sts:5000"]
        endpoint = ["niedbalski-bastion.cloud.sts:5000"]
  This stopped working on 1.3.X , scheduling pods with k8s 1.16-1.17 doesn't
  works using the same registry mirror definition.

  The pod definition is:

  apiVersion: v1
  kind: Pod
  metadata:
    name: busybox
    namespace: default
  spec:
    containers:
      - name: busybox
        image: niedbalski-bastion.cloud.sts:5000/busybox:latest
        command:
          - sleep
          - "3600"
    imagePullSecrets:
      - name: regcred
    restartPolicy: Always
  New pods fail with the following error:

  " failed to do request: Head niedbalski-
  bastion.cloud.sts:///v2/busybox/manifests/latest: unsupported protocol
  scheme "niedbalski-bastion.cloud.sts"

  Normal Scheduled default-scheduler Successfully assigned default/busybox to juju-3a79d2-00268738-4
  Normal Pulling 8m39s (x4 over 10m) kubelet, juju-3a79d2-00268738-4 Pulling image "niedbalski-bastion.cloud.sts:5000/busybox:latest"
  Warning Failed 8m39s (x4 over 10m) kubelet, juju-3a79d2-00268738-4 Failed to pull image "niedbalski-bastion.cloud.sts:5000/busybox:latest": rpc error: code = Unknown desc = failed to pull and unpack image "niedbalski-bastion.cloud.sts:5000/busybox:latest": failed to resolve reference "niedbalski-bastion.cloud.sts:5000/busybox:latest": failed to do request: Head niedbalski-bastion.cloud.sts:///v2/busybox/manifests/latest: unsupported protocol scheme "niedbalski-bastion.cloud.sts"
  Warning Failed 8m39s (x4 over 10m) kubelet, juju-3a79d2-00268738-4 Error: ErrImagePull
  Warning Failed 8m27s (x6 over 10m) kubelet, juju-3a79d2-00268738-4 Error: ImagePullBackOff
  Normal BackOff 4m56s (x21 over 10m) kubelet, juju-3a79d2-00268738-4 Back-off pulling image "niedbalski-bastion.cloud.sts:5000/busybox:latest"

  [Steps to reproduce]

  Configure a private docker repository repository

  Modify the containerd registry mirror config as follows:
  ** http://paste.ubuntu.com/p/yP63WMkVT6/

  Execute the following pod (http://paste.ubuntu.com/p/BVYQFMfCmk/)

  Status of the scheduled pod should be ImagePullBackOff
  and the before mentioned error should be raised.

  [Possible workaround and solution]

  As a workaround change the endpoint to support the scheme (https://)
  Provide a fallback mechanism for URL parsing validation to fallback to http or https.
  I suspect that this change introduced on 1.3.3 through
  0b29c9c) may be the offending commit.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/containerd/+bug/1867398/+subscriptions