Re: Multi install with existing MAAS starts all services except for “IP Pending” on Glance Simplestreams Image Sync
Jeff McLamb
mclamb at gmail.com
Wed Jul 29 17:32:38 UTC 2015
Hey Mike -
I am running MAAS Version 1.8.0+bzr4001-0ubuntu2 (vivid1) according to
the version strip at the bottom of the MAAS Web GUI.
Sure, will gladly file the bug report.
At this point, in hopes of getting something up and running to a
horizon dashboard I can log into, I am going to try to do the
install-only mode of openstack-install, then manually select services
but not glance sync. I presume I can get a working setup without sync,
which I don't really need in the first place, right?
Thanks,
Jeff
On Wed, Jul 29, 2015 at 1:26 PM, Mike McCracken
<mike.mccracken at canonical.com> wrote:
> Hi Jeff, your pastebin is showing that MAAS is complaining about the distro
> series, and the installer is just forwarding that error along.
> What version of MAAS are you running?
>
> As for the error with running a vivid container, that seems like a bug in
> juju, and if you have the time, I'd appreciate it if you could file it as a
> bug against lp:juju-core, including the info in your earlier email, in
> particular the stuff around "Failed to mount cgroup at
> /sys/fs/cgroup/systemd: Permission denied".
>
> It seems like there's a systemd-changeover related issue here that the juju
> folks will want to know about.
>
> Thanks!
> -mike
>
> On Wed, Jul 29, 2015 at 10:21 AM, Jeff McLamb <mclamb at gmail.com> wrote:
>>
>> This bug report seems to be related, if not the same thing:
>>
>> https://bugs.launchpad.net/landscape/+bug/1451385
>>
>> On Wed, Jul 29, 2015 at 1:10 PM, Jeff McLamb <mclamb at gmail.com> wrote:
>> > Here is the output from trying with --series vivid:
>> >
>> > http://paste.ubuntu.com/11960777/
>>
>> >
>> > I also did a re-install with --series trusty (really no different from
>> > the first time) and everything turned out the same, except the order
>> > in which containers came up and what got assigned to which container.
>> >
>> > However, logging into the physical node (machine 3) that hosts the
>> > containers, I found the console log of the actual vivid template lxc
>> > when it is in the RUNNING state and the glance sync container is the
>> > only one left to provision.
>> >
>> > The errors below seem to make sense with respect to the complaint in
>> > `juju status` that says it could not clone properly because the
>> > template container did not terminate:
>> >
>> > ubuntu at cooperative-crowd:~$ sudo cat
>> > /var/lib/juju/containers/juju-vivid-lxc-template/console.log
>> > Failed to mount cgroup at /sys/fs/cgroup/systemd: Permission denied
>> > systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA
>> > +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT -GNUTLS +ACL
>> > +XZ -LZ4 -SECCOMP +BLKID -ELFUTILS +KMOD -IDN)
>> > Detected virtualization lxc.
>> > Detected architecture x86-64.
>> >
>> > Welcome to Ubuntu 15.04!
>> >
>> > Set hostname to <juju-vivid-lxc-template>.
>> > Initializing machine ID from random generator.
>> > Failed to install release agent, ignoring: No such file or directory
>> > Failed to create root cgroup hierarchy: No such file or directory
>> > Failed to allocate manager object: No such file or directory
>> > [!!!!!!] Failed to allocate manager object, freezing.
>> >
>> > On Wed, Jul 29, 2015 at 12:35 PM, Mike McCracken
>> > <mike.mccracken at canonical.com> wrote:
>> >> Can you share what goes wrong when you tried --series vivid ?
>> >> It's always supported specifying whatever you like there, so it should
>> >> work,
>> >> and if something complained, that's a bug.
>> >>
>> >> Thanks,
>> >> -mike
>> >>
>> >> On Wed, Jul 29, 2015 at 9:20 AM, Jeff McLamb <mclamb at gmail.com> wrote:
>> >>>
>> >>> It seems as though vivid is not a valid --series name in my version of
>> >>> openstack-installer 0.99.19
>> >>>
>> >>> I can run it with trusty just fine, but I suspect that that will lead
>> >>> to the same issue where glance is installed as vivid and the rest are
>> >>> installed as trusty?
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Jeff
>> >>>
>> >>>
>> >>> On Wed, Jul 29, 2015 at 11:48 AM, Mike McCracken
>> >>> <mike.mccracken at canonical.com> wrote:
>> >>> > If you're able, I think the easiest way is to tear it down and
>> >>> > re-deploy.
>> >>> > Anything else would be a lot of manual steps. Juju has some upgrade
>> >>> > capabilities, but the openstack installer pre-dates them, and so
>> >>> > you'd
>> >>> > have
>> >>> > to hand-edit some config files, etc.
>> >>> >
>> >>> > -mike
>> >>> >
>> >>> > On Wed, Jul 29, 2015 at 8:44 AM, Jeff McLamb <mclamb at gmail.com>
>> >>> > wrote:
>> >>> >>
>> >>> >> Cool, thanks for the tip, Mike.
>> >>> >>
>> >>> >> Yes the host is vivid running juju 1.24.3-vivid-amd64
>> >>> >>
>> >>> >> Given my current state, should I kill the environment and start
>> >>> >> again,
>> >>> >> or is there a way I can deploy or update what's currently there
>> >>> >> without having to start over?
>> >>> >>
>> >>> >> Thanks,
>> >>> >>
>> >>> >> Jeff
>> >>> >>
>> >>> >>
>> >>> >> On Wed, Jul 29, 2015 at 11:39 AM, Mike McCracken
>> >>> >> <mike.mccracken at canonical.com> wrote:
>> >>> >> > Hi Jeff, so it looks like you're running vivid on the host, is
>> >>> >> > that
>> >>> >> > right?
>> >>> >> > Your point that the sync charm is the only one deployed as vivid
>> >>> >> > is a
>> >>> >> > good
>> >>> >> > catch - while this should still work, it might be causing
>> >>> >> > problems
>> >>> >> > for
>> >>> >> > juju's local provider.
>> >>> >> > I will fix the deploy code for the sync charm to use the same
>> >>> >> > series
>> >>> >> > as
>> >>> >> > the
>> >>> >> > other charms, but one thing you can try in the meantime is use
>> >>> >> > --series
>> >>> >> > vivid to just deploy the whole thing as vivid.
>> >>> >> > This has been tested and should work.
>> >>> >> >
>> >>> >> > -mike
>> >>> >> >
>> >>> >> > On Wed, Jul 29, 2015 at 8:25 AM, Jeff McLamb <mclamb at gmail.com>
>>
>> >>> >> > wrote:
>> >>> >> >>
>> >>> >> >> Hey Adam, Mike -
>> >>> >> >>
>> >>> >> >> The environment is up, but the container that should contain
>> >>> >> >> glance-simplestreams is not up at all. Here is the lxc-ls output
>> >>> >> >> from
>> >>> >> >> the node that contains all of the infrastructure containers:
>> >>> >> >>
>> >>> >> >> ubuntu at cooperative-crowd:~$ sudo lxc-ls --fancy
>> >>> >> >> NAME STATE IPV4 IPV6 AUTOSTART
>> >>> >> >> ----------------------------------------------------------------
>> >>> >> >> juju-machine-3-lxc-0 RUNNING 192.168.1.17 - YES
>> >>> >> >> juju-machine-3-lxc-1 RUNNING 192.168.1.18 - YES
>> >>> >> >> juju-machine-3-lxc-2 RUNNING 192.168.1.19 - YES
>> >>> >> >> juju-machine-3-lxc-3 RUNNING 192.168.1.20 - YES
>> >>> >> >> juju-machine-3-lxc-4 RUNNING 192.168.1.21 - YES
>> >>> >> >> juju-machine-3-lxc-6 RUNNING 192.168.1.22 - YES
>> >>> >> >> juju-machine-3-lxc-7 RUNNING 192.168.1.23 - YES
>> >>> >> >> juju-trusty-lxc-template STOPPED - - NO
>> >>> >> >> juju-vivid-lxc-template STOPPED - - NO
>> >>> >> >> ubuntu at cooperative-crowd:~$
>> >>> >> >>
>> >>> >> >> As you can see, the lxc-5 machine that should be running is not.
>> >>> >> >>
>> >>> >> >> When I look at openstack-status all of the services have a green
>> >>> >> >> checkmark and are [started] except for Glance Simplestreams
>> >>> >> >> which is
>> >>> >> >> in the [pending] stage.
>> >>> >> >>
>> >>> >> >> In the place of an IP address it says IP Pending and "waiting
>> >>> >> >> for
>> >>> >> >> initial status".
>> >>> >> >>
>> >>> >> >> I was wondering if I was running out of IP addresses in that
>> >>> >> >> range
>> >>> >> >> on
>> >>> >> >> the MAAS DHCP server, but there are only 8 containers and there
>> >>> >> >> are
>> >>> >> >> plenty of addresses left in that range.
>> >>> >> >>
>> >>> >> >> Here is the output of juju status where it is revealed that:
>> >>> >> >>
>> >>> >> >> 3/lxc/5:
>> >>> >> >> agent-state-info: 'failed to retrieve the template to
>> >>> >> >> clone:
>> >>> >> >> template container
>> >>> >> >> "juju-vivid-lxc-template" did not stop'
>> >>> >> >> instance-id: pending
>> >>> >> >> series: vivid
>> >>> >> >>
>> >>> >> >> For some reason this is the only container that it tries to
>> >>> >> >> install
>> >>> >> >> using the vivid series. All other containers use trusty.
>> >>> >> >>
>> >>> >> >> Here is the full juju status output:
>> >>> >> >>
>> >>> >> >> http://paste.ubuntu.com/11960193/
>> >>> >> >>
>> >>> >> >> Thanks!
>> >>> >> >>
>> >>> >> >> Jeff
>> >>> >> >>
>> >>> >> >> On Wed, Jul 29, 2015 at 11:15 AM, Adam Stokes
>> >>> >> >> <adam.stokes at canonical.com>
>> >>> >> >> wrote:
>> >>> >> >> > Jeff,
>> >>> >> >> >
>> >>> >> >> > Is the environment still up? Can you `juju ssh
>> >>> >> >> > glance-simplestreams-sync/0`
>> >>> >> >> > and grab the unit logs from `/var/log/juju/`.
>> >>> >> >> >
>> >>> >> >> > CCing Mike for further investigation into that service
>> >>> >> >> >
>> >>> >> >> > On Wed, Jul 29, 2015 at 10:25 AM, Jeff McLamb
>> >>> >> >> > <mclamb at gmail.com>
>> >>> >> >> > wrote:
>> >>> >> >> >>
>> >>> >> >> >> Hey Adam -
>> >>> >> >> >>
>> >>> >> >> >> Sure, here it is:
>> >>> >> >> >>
>> >>> >> >> >> http://paste.ubuntu.com/11959888/
>> >>> >> >> >>
>> >>> >> >> >> Thanks!
>> >>> >> >> >>
>> >>> >> >> >> Jeff
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >> On Wed, Jul 29, 2015 at 10:07 AM, Adam Stokes
>> >>> >> >> >> <adam.stokes at canonical.com>
>> >>> >> >> >> wrote:
>> >>> >> >> >> > Hey Jeff,
>> >>> >> >> >> >
>> >>> >> >> >> > Can you paste.ubuntu.com your
>> >>> >> >> >> > ~/.cloud-install/commands.log?
>> >>> >> >>
>> >>> >> >> >>
>> >>> >> >> >> >
>> >>> >> >> >> > On Wed, Jul 29, 2015 at 1:38 AM, Jeff McLamb
>> >>> >> >> >> > <mclamb at gmail.com>
>>
>> >>>
>> >>> >>
>> >>> >> >> >> > wrote:
>> >>> >> >> >> >>
>> >>> >> >> >> >> Hello -
>> >>> >> >> >> >>
>> >>> >> >> >> >> Due to Adam’s suggestion in a previous thread about
>> >>> >> >> >> >> Autopilot
>> >>> >> >> >> >> not
>> >>> >> >> >> >> working properly with a proxy running (e.g. masa-proxy) on
>> >>> >> >> >> >> the
>> >>> >> >> >> >> MAAS
>> >>> >> >> >> >> server, I killed the environment and started over with the
>> >>> >> >> >> >> Multi
>> >>> >> >> >> >> option.
>> >>> >> >> >> >>
>> >>> >> >> >> >> Interesting aside: you must openstack-install -k and
>> >>> >> >> >> >> openstack-install
>> >>> >> >> >> >> -u or otherwise remove ~/.cloud-install entirely because
>> >>> >> >> >> >> if
>> >>> >> >> >> >> you
>> >>> >> >> >> >> just
>> >>> >> >> >> >> do a -k and do not remove the landscape related files, it
>> >>> >> >> >> >> will
>> >>> >> >> >> >> still
>> >>> >> >> >> >> try to install an Autopilot-based setup even if you choose
>> >>> >> >> >> >> Multi.
>> >>> >> >> >> >>
>> >>> >> >> >> >> Anyway, the Multi install progressed well, bootstrapping
>> >>> >> >> >> >> juju
>> >>> >> >> >> >> with
>> >>> >> >> >> >> the
>> >>> >> >> >> >> VM node, followed by deploying services across 3 other
>> >>> >> >> >> >> physical
>> >>> >> >> >> >> nodes
>> >>> >> >> >> >> that are all big RAM and big core.
>> >>> >> >> >> >>
>> >>> >> >> >> >> One physical node was chosen as dedicated compute, one as
>> >>> >> >> >> >> dedicated
>> >>> >> >> >> >> Neutron, and the third was chosen for infrastructure,
>> >>> >> >> >> >> deploying
>> >>> >> >> >> >> what
>> >>> >> >> >> >> should be 8 containers.
>> >>> >> >> >> >>
>> >>> >> >> >> >> In the openstack-status view I see that all services have
>> >>> >> >> >> >> “started”
>> >>> >> >> >> >> except for the Glance - Simplestreams Image Sync service
>> >>> >> >> >> >> which
>> >>> >> >> >> >> is
>> >>> >> >> >> >> to
>> >>> >> >> >> >> be deployed as Container 5 on the infrastructure machine
>> >>> >> >> >> >> (Machine
>> >>> >> >> >> >> 3
>> >>> >> >> >> >> in
>> >>> >> >> >> >> my case). It is stuck in “IP Pending” state.
>> >>> >> >> >> >>
>> >>> >> >> >> >> I have rebooted the infrastructure node and all lxc
>> >>> >> >> >> >> containers
>> >>> >> >> >> >> came
>> >>> >> >> >> >> back up, except a lxc-ls —fancy does not show the
>> >>> >> >> >> >> existence of
>> >>> >> >> >> >> juju-machine-3-lxc-5. It does have STOPPED
>> >>> >> >> >> >> juju-trusty-lxc-template
>> >>> >> >> >> >> and juju-vivid-lxc-template. The vivid one was actually
>> >>> >> >> >> >> RUNNING
>> >>> >> >> >> >> before
>> >>> >> >> >> >> I rebooted the system, but now it is STOPPED.
>> >>> >> >> >> >>
>> >>> >> >> >> >> I can get to the horizon dashboard GUI but I cannot login
>> >>> >> >> >> >> with
>> >>> >> >> >> >> a
>> >>> >> >> >> >> “An
>> >>> >> >> >> >> error occurred authenticating. Please try again later.”
>> >>> >> >> >> >>
>> >>> >> >> >> >> Any ideas how to get that last service running so I can
>> >>> >> >> >> >> hopefully
>> >>> >> >> >> >> log
>> >>> >> >> >> >> into the horizon GUI for the first time?
>> >>> >> >> >> >>
>> >>> >> >> >> >> Thanks,
>> >>> >> >> >> >>
>> >>> >> >> >> >> Jeff
>> >>> >> >> >> >>
>> >>> >> >> >> >> --
>> >>> >> >> >> >> ubuntu-openstack-installer mailing list
>> >>> >> >> >> >> ubuntu-openstack-installer at lists.ubuntu.com
>> >>> >> >> >> >> Modify settings or unsubscribe at:
>> >>> >> >> >> >>
>> >>> >> >> >> >>
>> >>> >> >> >> >>
>> >>> >> >> >> >> https://lists.ubuntu.com/mailman/listinfo/ubuntu-openstack-installer
>> >>> >> >> >> >
>> >>> >> >> >> >
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >
>> >>> >> >
>> >>> >
>> >>> >
>> >>
>> >>
>
>
More information about the ubuntu-openstack-installer
mailing list