network spaces - aws support

Wed Aug 3 13:37:13 UTC 2016

FYI, I've filed a bug for the issue below:
https://bugs.launchpad.net/juju-core/+bug/1609343

And there's a live tested fix up for review:
https://github.com/juju/juju/pull/5922

James, I'd appreciate if you can give the fix a try (let me know by mail
or in #juju-dev on FreeNode IRC, if you need some help how to do that)!

Cheers,
Dimiter
On 08/03/2016 01:17 PM, Dimiter Naydenov wrote:
> Hey James,
> 
> Thanks for the detailed information! I think you did find a bug, let me
> try to explain.
> 
> First, it looks like your vpc-ff069a98 meets the minimum connectivity
> requirements of Juju (i.e. it's deemed suitable to host a controller),
> so passing --config vpc-id-force=true at bootstrap shouldn't be
> necessary (if it is, I'd like to know more about how your VPC's subnets
> and route tables look like, if possible!).
> 
> The bug I think you've discovered is related to how Juju does automatic
> distribution of instances across AZ for redundancy. Since EC2 subnets
> can only be part of one AZ, and when using a non-default VPC the AWS API
> requires you to pass SubnetID, not just AvailZone to RunInstances used
> to start an instance. Security groups are VPC-specific as well.
> 
> The essence of the issue I think is demonstrated here:
> 
> 2016-08-02 16:57:13 INFO juju.provider.ec2 environ.go:516 selected
> random subnet "subnet-f30d3fd9" from all matching in zone "us-east-1a":
> [subnet-9b013ab1 subnet-f30d3fd9 subnet-fb380ad1]
> 
> I strongly suspect those other 2 subnets: subnet-9b013ab1,
> subnet-fb380ad1 in AZ us-east-1a are part of a different VPC, but are
> still picked up "randomly" during StartInstance in the EC2 provider.
> 
> In other words Juju is not filtering subnets in a given AZ by the
> user-specified vpc-id when deciding which one to pick.
> 
> I looked at the code and it looks easy to fix this, fortunately.
> If I give you a patch to apply on top of the tip of juju/juju master
> branch, would you be willing to give it a try to see if it works?
> 
> Thanks for your patience! I'll file a bug about what you've discovered
> and start working on a fix for beta14!
> 
> Cheers,
> Dimiter
> 
> On 08/02/2016 08:33 PM, James Beedy wrote:
>> Ok, here we go .... hopefully this will provide a better overhead view.
>>
>> `juju bootstrap creativedrive aws --credential creativedrive --config
>> vpc-id=vpc-ff069a98 --config vpc-id-force='true' --upload-tools --debug
>> --config logging-config='<root>=TRACE'` <- http://paste.ubuntu.com/21914113/
>>
>> cat machine-0.log <- http://paste.ubuntu.com/21914847/
>>
>>
>> ### debug add-model
>>
>> `juju add-model consul --credential creativedrive --config
>> vpc-id=vpc-ff069a98 --config vpc-id-force='true' --debug` <-
>> http://paste.ubuntu.com/21915182/
>>
>> ### juju status on new model
>>
>> juju status --format yaml <- http://paste.ubuntu.com/21915512/
>>
>> ### debug add space and subnet
>>
>> `juju add-space common-infrastructure --debug` <-
>> http://paste.ubuntu.com/21915888/
>>
>> #### add a subnet in my vpc to the newly created space
>> `juju add-subnet subnet-9b013ab1 common-infrastructure us-east-1a
>> --debug` <- http://paste.ubuntu.com/21916055/
>>
>> ### list spaces and subnets
>>
>> `juju subnets && juju spaces` <- http://paste.ubuntu.com/21916258/
>>
>>
>> ### deploy something to the model
>> `juju deploy ubuntu --debug` <- http://paste.ubuntu.com/21916540/
>>
>>
>> ### deploy something to a network space
>>
>> `juju deploy ubuntu ubuntu-spaces --constraints
>> spaces=common-infrastructure --debug` <- http://paste.ubuntu.com/21916804/
>>
>>
>> ### juju status now shows a success for the machine deployed to the
>> model w/out a space constraint, and and error for the instance deployed
>> to the space.
>>
>> `juju status --format yaml` <- http://paste.ubuntu.com/21917006/
>>
>>
>> No matter what, I can't seem to get anything deployed to a "space"....
>>
>> So strange ... possibly I have stumbled upon a bug?
>>
>>
>> Thanks again for your insight here.
>>
>>
>>
>> On Tue, Aug 2, 2016 at 9:51 AM, James Beedy <jamesbeedy at gmail.com
>> <mailto:jamesbeedy at gmail.com>> wrote:
>>
>>     To my utter dismay, setting the correct config 'vpc-id-force' gave
>>     me the same result....
>>
>>     Let me scrub and collect my machine-0.log for you.
>>
>>
>>      
>>
>>     On Tue, Aug 2, 2016 at 9:36 AM, James Beedy <jamesbeedy at gmail.com
>>     <mailto:jamesbeedy at gmail.com>> wrote:
>>
>>         Dimiter,
>>
>>         Thanks for the insight.
>>         /
>>         /
>>         /Can you please also paste the full logs (scrubbed of secrets)
>>         of `juju
>>         bootstrap ... --debug` (with the vpc-id etc., but please also
>>         include
>>         `--config logging-config='<root>=TRACE'`), and machine-0.log from
>>         /var/log/juju on the bootstrap node, once completed? That will help
>>         figuring out the issue.
>>         /
>>         `juju bootstrap creativedrive aws --credential creativedrive
>>         --config vpc-id=vpc-ff069a98 --config force-vpc-id='true'
>>         --config loggin-config='<root>=TRACE' --upload-tools --debug` <-
>>         http://paste.ubuntu.com/21908548/
>>
>>         machine-0.log shows  "2016-08-02 16:16:16 TRACE juju.apiserver
>>         request_notifier.go:127 -> [2] machine-0
>>         {"request-id":53,"response":{"config":{"access-key":"","agent-version":"2.0-beta13","authorized-keys":"juju-client-key\nssh-rsa
>>         ssh-rsa
>>         juju-system-key\n","automatically-retry-hooks":true,"default-series":"","development":false,"disable-network-management":false,"firewall-mode":"instance","force-vpc-id":true,"ignore-machine-addresses":false,"logging-config":"\u003croot\u003e=TRACE;unit=DEBUG","name":"controller","proxy-ssh":false,"region":"us-east-1","secret-key":"/E","ssl-hostname-verification":true,"storage-default-block-source":"ebs","test-mode":false,"type":"ec2","uuid":"259be235-a255-416d-8bbf-75e128d05794","vpc-id":"vpc-ff069a98","vpc-id-force":false}}}"
>>
>>
>>         Just realizing now, I have been specifying 'vpc-force-id', not
>>         'vpc-id-force' (grrrr).
>>
>>         I would expect to see this resolved when I apply the correct
>>         config. I'll report back shortly.
>>
>>         Thanks for your time!
>>
>>         /From what I can understand, you're trying to bootstrap on a
>>         non-default,
>>         possibly private VPC (accessed via its internal address over a VPN
>>         connection maybe?), and then add a model with the same VPC and
>>         credentials.
>>
>>         /
>>         ^ Exactly/
>>         /
>>         /
>>          If OTOH, the VPC used for add-model is different, the
>>         machines there won't be able to talk to the controller's VPC
>>         unless it
>>         has a public address (cross VPC communication currently relies
>>         on having
>>         that, fancier setups with VPN gateways is not yet supported)./
>>
>>         ^
>>
>>         The error in status implies 2 separate VPCs are used (or a VPC and
>>         EC2-Classic - i.e. no VPC) for the controller and hosted model.
>>
>>         Cheers,
>>         Dimiter
>>
>>
>>
>>
>>
> 
> 
> 
> 

-- 
Dimiter Naydenov <dimiter.naydenov at canonical.com>
Juju Core Sapphire team <http://juju.ubuntu.com>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20160803/d866551c/attachment.pgp>