Proposal: making apt-get upgrade optional

[Andrew Wilkins]

On Wed, Jul 2, 2014 at 3:38 AM, Antonio Rosales <
antonio.rosales at canonical.com> wrote:

> Suggest we make an environments.yaml key value of say "apt-get-update"
> set to a boolean with the default being "true". Existing charms are
> timing out[0] when apt-get update is turned off due to stale apt-get
> metadata. Users then can them make the choice, and we can make
> suggestions in the docs as to what this key value means and how it can
> improve performance especially in the developer scenario when the care
> more about fast iterative deploys.
>
> Thoughts?
>

I'm not suggesting we turn off update, just upgrade. We add repos
(cloud-tools, ppa), so we need to update for juju's dependencies anyway. I
don't think my proposal will affect charms.


> [0] https://bugs.launchpad.net/juju-core/+bug/1336353
>
> -thanks,
> Antonio
>
> On Tue, Jul 1, 2014 at 4:43 AM, Andrew Wilkins
> <andrew.wilkins at canonical.com> wrote:
> > On Tue, Jul 1, 2014 at 5:45 PM, John Meinel <john at arbash-meinel.com>
> wrote:
> >>
> >> I would just caution that we'd really prefer behavior to be consistent
> >> across platforms and clouds, and if we can work with Microsoft to make
> >> 'apt-get update' faster in their cloud everyone wins who uses Ubuntu
> there,
> >> not just us.
> >
> >
> > I was meaning to disable it across all providers. It would be ideal to
> > improve upgrades for all Ubuntu users, but from what I can tell it's a
> case
> > of Azure's OS disks being a tad slow. If you start going up the
> > instance-type scale, then you do get more IOPS. I haven't measured how
> much
> > of a difference it makes.
> >
> >>
> >> Have we looked into why Upgrade is taking 3m+? Is it the time to
> download
> >> things, is it the time to install things? I've certainly heard things
> like
> >> "disk ops is a bit poor" on Azure (vs CPU is actually better than
> average).
> >> Given the variance of 6m+ to 3m20s with Eat my data, it would seem disk
> sync
> >> performance is at least a factor here.
> >
> >
> > I just looked, and it is mostly not network related (I assume mostly I/O
> > bound). On ec2 an upgrade fetches all the bits in 0s; on Azure it's
> taking
> > 5s.
> >
> >> Given I believe apt-get update is also disabled for local (it is run on
> >> the initial template, and then not run for the other instances copied
> from
> >> that), there is certainly precedence. I think a big concern is that we
> would
> >> probably still want to do apt-get update for security related updates.
> >> Though perhaps that is all of the updates we are applying anyway...
> >>
> >> If I read the "aws.json" file correctly, I see only 8 releases of the
> >> 'precise' image. 6 of 'trusty' and 32 total dates of released items. And
> >> some of the trusty releases are 2014-01-22.1 which means it is likely
> to be
> >> beta releases.
> >>
> >> Anyway, that means that they are actually averaging an update only
> >> 1/month, which is a fairly big window of updates to apply by the end of
> >> month (I would imagine). And while that does mean it takes longer to
> boot,
> >> it also means you would be open to more security holes without it.
> >
> >
> > My contention is that if we don't *keep* it updated, we may as well just
> > leave it to the user. When you create an instance in ec2 or Azure or
> > whatever, it doesn't come fully up-to-date. You get the released image,
> and
> > then you can update it if you want to.
> >
> >> John
> >> =:->
> >>
> >>
> >>
> >>
> >> On Mon, Jun 30, 2014 at 5:05 PM, Andrew Wilkins
> >> <andrew.wilkins at canonical.com> wrote:
> >>>
> >>> Hi folks,
> >>>
> >>> I've been debugging a bootstrap bug [0] that was caused by ssh timing
> out
> >>> (and the client not noticing), which was caused by "apt-get upgrade"
> taking
> >>> an awfully long time (6 minutes on Azure).
> >>>     [0] https://bugs.launchpad.net/juju-core/+bug/1316185
> >>>
> >>> I just filed https://bugs.launchpad.net/juju-core/+bug/1335822, and
> did a
> >>> quick and dirty hack that brought the upgrade down to 3 minutes on
> Azure. I
> >>> don't know the variance, so I can't be sure that it's all due to
> eatmydata,
> >>> but smoser's results are similar.
> >>>
> >>> Even with eatmydata, a full bootstrap on Azure just took me 10 minutes.
> >>> That's roughly broken down into:
> >>>  - apt-get update: 20s
> >>>  - apt-get upgrade: 3m20s
> >>>  - apt-get install <various>: 10s
> >>>  - Download tools (from shared Azure storage account): 5s
> >>>  - jujud bootstrap: 1m50s
> >>>
> >>> We could bring the 10m down to 6m40s. Still not brilliant, but
> >>> considerably better IMO.
> >>>
> >>> I propose that we remove the "apt-get upgrade" altogether. Cloud images
> >>> are regularly updated and tested, and I think we should be able to
> rely on
> >>> that alone. If users want something more up-to-date, they can use the
> daily
> >>> images which are not tested as a whole, but are composed of SRUs,
> which is
> >>> effectively what users get today.
> >>>
> >>> Cheers,
> >>> Andrew
> >>>
> >>> --
> >>> Juju-dev mailing list
> >>> Juju-dev at lists.ubuntu.com
> >>> Modify settings or unsubscribe at:
> >>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
> >>>
> >>
> >
> >
> > --
> > Juju-dev mailing list
> > Juju-dev at lists.ubuntu.com
> > Modify settings or unsubscribe at:
> > https://lists.ubuntu.com/mailman/listinfo/juju-dev
> >
>
>
>
> --
> Antonio Rosales
> Juju Ecosystem
> Canonical
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20140702/9632d975/attachment-0001.html>