Faster LXD bootstraps and provisioning

[Reed O'Brien]

Yes thanks for doing the work to share this menn0. It is much appreciated.

I also needed to change the following in squid-deb-proxy (even though I
added a bunch of domains to
/etc/squid-deb-proxy/mirror-dstdomain.acl.d/10-default):
--- a/squid-deb-proxy/squid-deb-proxy.conf
+++ b/squid-deb-proxy/squid-deb-proxy.conf
@@ -80,12 +80,12 @@ http_access deny blockedpkgs
 # allow access only to official archive mirrors
 # uncomment the third and fouth line to permit any unlisted domain
 #http_access deny !to_archive_mirrors
-http_access allow !to_archive_mirrors
+http_access allow all #!to_archive_mirrors

 # don't cache domains not listed in the mirrors file
 # uncomment the third and fourth line to cache any unlisted domains
 #cache deny !to_archive_mirrors
-cache allow !to_archive_mirrors
+cache allow all

 # allow access from our network and localhost
 http_access allow allowed_networks

And update some firewall rules aside from the ones shipped with squid (tcp
only is prolly fine here):
diff --git a/ufw/user.rules b/ufw/user.rules
index 121b5b5..4cca3b0 100644
--- a/ufw/user.rules
+++ b/ufw/user.rules
@@ -47,6 +47,10 @@
 ### tuple ### allow tcp 2048,3128,3130,3401,4827 0.0.0.0/0 any 0.0.0.0/0
Squid - in
 -A ufw-user-input -p tcp -m multiport --dports 2048,3128,3130,3401,4827 -j
ACCEPT -m comment --comment 'dapp_Squid'

+### tuple ### allow any 8000 0.0.0.0/0 any 0.0.0.0/0 in
+-A ufw-user-input -p tcp --dport 8000 -j ACCEPT
+-A ufw-user-input -p udp --dport 8000 -j ACCEPT
+
 ### END RULES ###

 ### LOGGING ###
diff --git a/ufw/user6.rules b/ufw/user6.rules
index 2c1aac4..d48de60 100644
--- a/ufw/user6.rules
+++ b/ufw/user6.rules
@@ -47,6 +47,10 @@
 ### tuple ### allow tcp 2048,3128,3130,3401,4827 ::/0 any ::/0 Squid - in
 -A ufw6-user-input -p tcp -m multiport --dports 2048,3128,3130,3401,4827
-j ACCEPT -m comment --comment 'dapp_Squid'

+### tuple ### allow any 8000 ::/0 any ::/0 in
+-A ufw6-user-input -p tcp --dport 8000 -j ACCEPT
+-A ufw6-user-input -p udp --dport 8000 -j ACCEPT

I've also added the config items to clouds.yaml and they work there. I'll
update the wiki later this or next week.


On Mon, Aug 15, 2016 at 2:27 PM Casey Marshall <casey.marshall at canonical.com>
wrote:

> Menno,
> This is great and thanks for sharing!
>
> In case anyone else runs into this.. charms that install from PPAs will
> fail with this squid-deb-proxy setup. You'll need to allow archive mirrors
> for this to work. See
> https://1337.tips/ubuntu-cache-packages-using-squid-deb-proxy/ for an
> example.
>
> On Mon, Aug 15, 2016 at 9:31 AM, Rafael Gonzalez <
> rafael.gonzalez at canonical.com> wrote:
>
>> Hi Menno,
>>
>> Thanks for putting this together, great tips.  I recently ran into an
>> issue which others could see as well.
>>
>> One may need to adjust the following for large bundle deployments on
>> LXD.  A bundle deployment fails with errors about "Too many files open."
>>  This will increase number of max open files:
>>
>> echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf &&
>> sudo sysctl -p
>>
>>
>> Regards,
>>
>> Rafael O. Gonzalez
>> Canonical, Solutions Architect
>> rgonza at canonical.com
>> 1-646-481-7232
>>
>>
>>
>> On Sun, Aug 14, 2016 at 8:07 PM, Menno Smits <menno.smits at canonical.com>
>> wrote:
>>
>>> I've put together a few tips on the wiki for speeding up bootstrap and
>>> provisioning times when using the Juju lxd provider. I find these
>>> techniques helpful when checking my work or investigating bugs - situations
>>> where you end up bootstrapping and deploying many times.
>>>
>>> https://github.com/juju/juju/wiki/Faster-LXD
>>>
>>> If you have your own techniques, or improvements to what I'm doing,
>>> please update the article.
>>>
>>> - Menno
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Juju-dev mailing list
>>> Juju-dev at lists.ubuntu.com
>>> Modify settings or unsubscribe at:
>>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>>
>>>
>>
>> --
>> Juju-dev mailing list
>> Juju-dev at lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>
>>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20160815/f2fc3b75/attachment-0001.html>