[Bug 1915345] Re: [SRU] eic_harvest_hostkeys fails in local zones

[Robie Basak]

Hello Robert, or anyone else affected,

Accepted ec2-instance-connect into xenial-proposed. The package will
build now and be available at https://launchpad.net/ubuntu/+source/ec2
-instance-connect/1.1.12+dfsg1-0ubuntu3~16.04.2 in a few hours, and then
in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.  Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, what testing has been
performed on the package and change the tag from verification-needed-
xenial to verification-done-xenial. If it does not fix the bug for you,
please add a comment stating that, and change the tag to verification-
failed-xenial. In either case, without details of your testing we will
not be able to proceed.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance for helping!

N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.

** Changed in: ec2-instance-connect (Ubuntu Xenial)
       Status: New => Fix Committed

** Tags added: verification-needed-xenial

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ec2-instance-connect in Ubuntu.
https://bugs.launchpad.net/bugs/1915345

Title:
  [SRU] eic_harvest_hostkeys fails in local zones

Status in Ec2 Instance Connect:
  New
Status in ec2-instance-connect package in Ubuntu:
  Fix Released
Status in ec2-instance-connect source package in Xenial:
  Fix Committed
Status in ec2-instance-connect source package in Bionic:
  Fix Committed
Status in ec2-instance-connect source package in Focal:
  Fix Committed
Status in ec2-instance-connect source package in Groovy:
  Fix Committed

Bug description:
  [Impact]

  * ec2-instance-connect breaks during host key harvesting for instances
  launched in local zones [1] making the system boot to degraded mode
  only.

  [Test Plan]

  * Start a system with the the fixed ec2-instance-connect package in a
  local zone [1] or break the the /usr/share/ec2-instance-
  connect/eic_harvest_hostkeys script to exit with failure.

  [Where problems could occur]

  * The fix is ignoring the eic_harvest_hostkeys script's exit code
  which may hide actual problems in the script or in the infrastructure
  preventing connecting to the instance using Instance Connect. This is
  a decision by upstream. There are no other expected issues.

  [Original Bug Text]

  ec2-instance-connect breaks during host key harvesting for instances
  launched in local zones[1].  Here are is the relevant debug data:

  $ systemctl is-system-running
  degraded

  $ systemctl list-units --failed
    UNIT                         LOAD   ACTIVE SUB    DESCRIPTION
  ● ec2-instance-connect.service loaded failed failed EC2 Instance Connect Host Key Harvesting

  $ journalctl --unit ec2-instance-connect
  -- Logs begin at Wed 2021-02-10 22:47:47 UTC, end at Wed 2021-02-10 22:55:46 UTC. --
  Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: Starting EC2 Instance Connect Host Key Harvesting...
  Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: ec2-instance-connect.service: Main process exited, code=exited, status=255/EXCEPTION
  Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: ec2-instance-connect.service: Failed with result 'exit-code'.
  Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: Failed to start EC2 Instance Connect Host Key Harvesting.

  $ dpkg-query -l ec2-instance-connect
  ii  ec2-instance-connect 1.1.13-0ubuntu1 all          Configures ssh daemon to accept EC2 Instance Connect ssh keys

  $ lsb_release -c
  Codename:	hirsute

  $ cat /etc/cloud/build.info
  build_name: server
  serial: 20210208

  $ ec2metadata --availability-zone --ami-id
  us-west-2-lax-1a
  ami-098f71a7a25a0f1f2

  $ bash -x /usr/share/ec2-instance-connect/eic_harvest_hostkeys
  ...
  ++ /usr/bin/curl -s -f -m 1 -H 'X-aws-ec2-metadata-token: AQAEAEvStI0Ugwz1C3GQh7oubFTah7bXQllCmFU6BtMI6b6l5zMkVQ==' http://169.254.169.254/latest/meta-data/placement/availability-zone/
  + zone=us-west-2-lax-1a
  + zone_exit=0
  + '[' 0 -ne 0 ']'
  + /bin/echo us-west-2-lax-1a
  + /bin/grep -Eq '^([a-z]+-){2,3}[0-9][a-z]$'
  + /usr/bin/head -n 1
  + exit 255

  [1] https://aws.amazon.com/about-aws/global-infrastructure/localzones/

To manage notifications about this bug go to:
https://bugs.launchpad.net/ec2-instance-connect/+bug/1915345/+subscriptions