Proposal: making apt-get upgrade optional
Antonio Rosales
antonio.rosales at canonical.com
Sat Jul 5 08:05:03 UTC 2014
On Thu, Jul 3, 2014 at 7:41 PM, Andrew Wilkins
<andrew.wilkins at canonical.com> wrote:
> On Fri, Jul 4, 2014 at 5:23 AM, Tim Penhey <tim.penhey at canonical.com> wrote:
>>
>> I do just want to make the point that we are not just an ubuntu only
>> system any more, nor even linux only.
>>
>> I'd prefer if we kept away from terms like "apt-get" as it doesn't make
>> sense for windows nor centos. While we could certainly treat those
>> values differently on the other platforms, it definitely gives the
>> feeling that we are *mainly* ubuntu and (hand wavey) some others.
cloud-init generally supporst other Linux distros and I think
Cloudbase has cloudbase-init for Windows.
>>
>> Any ideas for better names?
>
>
> "upgrade-packages"? Still kinda Linux, so alternatively "upgrade-system".
>
> In cloud-init, it's "package_upgrade", with "apt_upgrade" as an alias.
+1 suggest to keep close to the cloud-init packaging as that is what
Juju will be leveraging. Is there a straightforward mechanism to allow
users and/or charm authors access to the cloud-init configuration?
-thanks,
Antonio
>
>>
>> Tim
>>
>>
>> On 04/07/14 02:56, Matt Bruzek wrote:
>> > +1 to making these options configurable and having sane defaults.
>> >
>> > Thanks!
>> >
>> > - Matt Bruzek <matthew.bruzek at canonical.com
>> > <mailto:matthew.bruzek at canonical.com>>
>> >
>> >
>> > On Thu, Jul 3, 2014 at 9:50 AM, Antonio Rosales
>> > <antonio.rosales at canonical.com <mailto:antonio.rosales at canonical.com>>
>> > wrote:
>> >
>> > On Tue, Jul 1, 2014 at 7:19 PM, Andrew Wilkins
>> > <andrew.wilkins at canonical.com <mailto:andrew.wilkins at canonical.com>>
>> > wrote:
>> > > On Wed, Jul 2, 2014 at 3:38 AM, Antonio Rosales
>> > > <antonio.rosales at canonical.com
>> > <mailto:antonio.rosales at canonical.com>> wrote:
>> > >>
>> > >> Suggest we make an environments.yaml key value of say
>> > "apt-get-update"
>> > >> set to a boolean with the default being "true". Existing charms
>> > are
>> > >> timing out[0] when apt-get update is turned off due to stale
>> > apt-get
>> > >> metadata. Users then can them make the choice, and we can make
>> > >> suggestions in the docs as to what this key value means and how
>> > it can
>> > >> improve performance especially in the developer scenario when the
>> > care
>> > >> more about fast iterative deploys.
>> > >>
>> > >> Thoughts?
>> > >
>> > >
>> > > I'm not suggesting we turn off update, just upgrade. We add repos
>> > > (cloud-tools, ppa), so we need to update for juju's dependencies
>> > anyway. I
>> > > don't think my proposal will affect charms.
>> >
>> > Ah yes, sorry. However, I would still suggest upgrade and update be
>> > config parameter with the default being past behavior. On that note
>> > it
>> > would also be nice to have a utility for Juju to pass on additional
>> > user defined cloud-init config options.
>> >
>> > -thanks,
>> > Antonio
>> >
>> > >
>> > >>
>> > >> [0] https://bugs.launchpad.net/juju-core/+bug/1336353
>> > >>
>> > >> -thanks,
>> > >> Antonio
>> > >>
>> > >> On Tue, Jul 1, 2014 at 4:43 AM, Andrew Wilkins
>> > >> <andrew.wilkins at canonical.com
>> > <mailto:andrew.wilkins at canonical.com>> wrote:
>> > >> > On Tue, Jul 1, 2014 at 5:45 PM, John Meinel
>> > <john at arbash-meinel.com <mailto:john at arbash-meinel.com>>
>> > >> > wrote:
>> > >> >>
>> > >> >> I would just caution that we'd really prefer behavior to be
>> > consistent
>> > >> >> across platforms and clouds, and if we can work with Microsoft
>> > to make
>> > >> >> 'apt-get update' faster in their cloud everyone wins who uses
>> > Ubuntu
>> > >> >> there,
>> > >> >> not just us.
>> > >> >
>> > >> >
>> > >> > I was meaning to disable it across all providers. It would be
>> > ideal to
>> > >> > improve upgrades for all Ubuntu users, but from what I can tell
>> > it's a
>> > >> > case
>> > >> > of Azure's OS disks being a tad slow. If you start going up the
>> > >> > instance-type scale, then you do get more IOPS. I haven't
>> > measured how
>> > >> > much
>> > >> > of a difference it makes.
>> > >> >
>> > >> >>
>> > >> >> Have we looked into why Upgrade is taking 3m+? Is it the time
>> > to
>> > >> >> download
>> > >> >> things, is it the time to install things? I've certainly heard
>> > things
>> > >> >> like
>> > >> >> "disk ops is a bit poor" on Azure (vs CPU is actually better
>> > than
>> > >> >> average).
>> > >> >> Given the variance of 6m+ to 3m20s with Eat my data, it would
>> > seem disk
>> > >> >> sync
>> > >> >> performance is at least a factor here.
>> > >> >
>> > >> >
>> > >> > I just looked, and it is mostly not network related (I assume
>> > mostly I/O
>> > >> > bound). On ec2 an upgrade fetches all the bits in 0s; on Azure
>> > it's
>> > >> > taking
>> > >> > 5s.
>> > >> >
>> > >> >> Given I believe apt-get update is also disabled for local (it
>> > is run on
>> > >> >> the initial template, and then not run for the other instances
>> > copied
>> > >> >> from
>> > >> >> that), there is certainly precedence. I think a big concern is
>> > that we
>> > >> >> would
>> > >> >> probably still want to do apt-get update for security related
>> > updates.
>> > >> >> Though perhaps that is all of the updates we are applying
>> > anyway...
>> > >> >>
>> > >> >> If I read the "aws.json" file correctly, I see only 8 releases
>> > of the
>> > >> >> 'precise' image. 6 of 'trusty' and 32 total dates of released
>> > items.
>> > >> >> And
>> > >> >> some of the trusty releases are 2014-01-22.1 which means it is
>> > likely
>> > >> >> to be
>> > >> >> beta releases.
>> > >> >>
>> > >> >> Anyway, that means that they are actually averaging an update
>> > only
>> > >> >> 1/month, which is a fairly big window of updates to apply by
>> > the end of
>> > >> >> month (I would imagine). And while that does mean it takes
>> > longer to
>> > >> >> boot,
>> > >> >> it also means you would be open to more security holes without
>> > it.
>> > >> >
>> > >> >
>> > >> > My contention is that if we don't *keep* it updated, we may as
>> > well just
>> > >> > leave it to the user. When you create an instance in ec2 or
>> > Azure or
>> > >> > whatever, it doesn't come fully up-to-date. You get the
>> > released image,
>> > >> > and
>> > >> > then you can update it if you want to.
>> > >> >
>> > >> >> John
>> > >> >> =:->
>> > >> >>
>> > >> >>
>> > >> >>
>> > >> >>
>> > >> >> On Mon, Jun 30, 2014 at 5:05 PM, Andrew Wilkins
>> > >> >> <andrew.wilkins at canonical.com
>> > <mailto:andrew.wilkins at canonical.com>> wrote:
>> > >> >>>
>> > >> >>> Hi folks,
>> > >> >>>
>> > >> >>> I've been debugging a bootstrap bug [0] that was caused by
>> > ssh timing
>> > >> >>> out
>> > >> >>> (and the client not noticing), which was caused by "apt-get
>> > upgrade"
>> > >> >>> taking
>> > >> >>> an awfully long time (6 minutes on Azure).
>> > >> >>> [0] https://bugs.launchpad.net/juju-core/+bug/1316185
>> > >> >>>
>> > >> >>> I just filed
>> > https://bugs.launchpad.net/juju-core/+bug/1335822, and
>> > >> >>> did a
>> > >> >>> quick and dirty hack that brought the upgrade down to 3
>> > minutes on
>> > >> >>> Azure. I
>> > >> >>> don't know the variance, so I can't be sure that it's all due
>> > to
>> > >> >>> eatmydata,
>> > >> >>> but smoser's results are similar.
>> > >> >>>
>> > >> >>> Even with eatmydata, a full bootstrap on Azure just took me
>> > 10
>> > >> >>> minutes.
>> > >> >>> That's roughly broken down into:
>> > >> >>> - apt-get update: 20s
>> > >> >>> - apt-get upgrade: 3m20s
>> > >> >>> - apt-get install <various>: 10s
>> > >> >>> - Download tools (from shared Azure storage account): 5s
>> > >> >>> - jujud bootstrap: 1m50s
>> > >> >>>
>> > >> >>> We could bring the 10m down to 6m40s. Still not brilliant,
>> > but
>> > >> >>> considerably better IMO.
>> > >> >>>
>> > >> >>> I propose that we remove the "apt-get upgrade" altogether.
>> > Cloud
>> > >> >>> images
>> > >> >>> are regularly updated and tested, and I think we should be
>> > able to
>> > >> >>> rely on
>> > >> >>> that alone. If users want something more up-to-date, they can
>> > use the
>> > >> >>> daily
>> > >> >>> images which are not tested as a whole, but are composed of
>> > SRUs,
>> > >> >>> which is
>> > >> >>> effectively what users get today.
>> > >> >>>
>> > >> >>> Cheers,
>> > >> >>> Andrew
>> > >> >>>
>> > >> >>> --
>> > >> >>> Juju-dev mailing list
>> > >> >>> Juju-dev at lists.ubuntu.com <mailto:Juju-dev at lists.ubuntu.com>
>> > >> >>> Modify settings or unsubscribe at:
>> > >> >>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> > >> >>>
>> > >> >>
>> > >> >
>> > >> >
>> > >> > --
>> > >> > Juju-dev mailing list
>> > >> > Juju-dev at lists.ubuntu.com <mailto:Juju-dev at lists.ubuntu.com>
>> > >> > Modify settings or unsubscribe at:
>> > >> > https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> > >> >
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> Antonio Rosales
>> > >> Juju Ecosystem
>> > >> Canonical
>> > >
>> > >
>> >
>> >
>> >
>> > --
>> > Antonio Rosales
>> > Juju Ecosystem
>> > Canonical
>> >
>> > --
>> > Juju-dev mailing list
>> > Juju-dev at lists.ubuntu.com <mailto:Juju-dev at lists.ubuntu.com>
>> > Modify settings or unsubscribe at:
>> > https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >
>> >
>> >
>> >
>>
>>
>> --
>> Juju-dev mailing list
>> Juju-dev at lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>
--
Antonio Rosales
Juju Ecosystem
Canonical
More information about the Juju-dev
mailing list