Landing bot setup steps

[Martin Packman]

Here's what I did today to get the bot working again. In summary,
everything should be fine, but it's not been fully tested, and there
may be differences and issues that still need ironing out. Please poke
me if you notice anything or have issues trying to land your branches.

Feel free not to read the following notes unless curious. :)


Source the creds for the juju-tools-upload user (sent in gpg email
from John a while ago).

$ source ~/.gobotcreds
$ export OS_REGION_NAME=lcy02

Get the various bits and pieces from the old setup:

$ swift download tarmac
$ bzr branch lp:~jameinel/charms/precise/tarmac/add-install-dependencies
charms/precise/tarmac

Edit and add gobot config to environments.yaml:

  gobot:
    type: openstack
    admin-secret: <same as in old environments.yaml file from tarmac container>
    control-bucket: juju-gobot-container

$ export JUJU_ENV=gobot

$ juju bootstrap
$ juju status

Due to some fun networking issues on canonistack, I needed various
hackery to get status to work, see the end of the message if curious.


Add top level 'tarmac:' key to goose2.yaml from tarmac container. Also
needed to come back and change package "golang-stable" to "golang"
here as well, and there were some other missing packages like
distro-info that could do with being added.

$ juju deploy --config goose2.yaml --constraints=cpu-cores=2
local:precise/tarmac

Without the golang package name change, that gave:

  agent-state-info: 'hook failed: "config-changed"'

So, sshed in, see in /var/log/juju/unit-tarmac-0.log

2013/07/29 12:50:25 INFO worker/uniter: HOOK E: Unable to locate
package golang-stable
2013/07/29 12:50:25 ERROR worker/uniter: hook failed: exit status 100

$ juju destroy-service tarmac

Gets stuck in dying.

$ juju resolved tarmac/0
$ juju terminate-machine 1

Okay, deploy again, works. We now have:

4829b364-72ad-4ee7-a21c-3ba640f28854 | juju-gobot-machine-0 |
10.55.32.55 (m1.tiny)
97a7c226-a195-4014-9df5-c998bba3a491 | juju-gobot-machine-3 |
10.55.32.52 (m1.medium)

Hopefully mongo doesn't kill that tiny too fast.


Connect to the tarmac machine:

$ ssh 10.55.32.52

Let some other people access the bot:

$ ssh-import-id dimitern rogpeppe wallyworld fwereade jameinel thumper

Missing some people here, yell if you want adding.


Use go 1.1 from our ppa:

$ sudo apt-add-repository ppa:juju/golang
$ sudo apt-get update
$ sudo apt-get upgrade


Use mongo 2.2 (which we need to get from old tools tarball for now):

$ sudo apt-get remove mongodb-server
$ wget http://juju-dist.s3.amazonaws.com/tools/mongo-2.2.0-precise-amd64.tgz
$ tar -xzf mongo-2.2.0-precise-amd64.tgz -C /tmp
$ sudo cp /tmp/mongo/bin/* /usr/local/bin/


Install distro-info (missed this initially):

$ sudo apt-get install distro-info


Get a modern bzr which handles reconnecting after connection timeouts as the
test suite takes too long to run, and tarmac holds onto the branch object:

$ sudo apt-get install python-dev cython

Though cython shouldn't be needed, the tarballs have prebuilt .c files.

$ wget https://launchpad.net/bzr/2.6/2.6.0/+download/bzr-2.6.0.tar.gz

Then need to do the further steps as the tarmac user:

$ sudo su - tarmac

$ tar -xzf ~ubuntu/bzr-2.6.0.tar.gz
$ (cd bzr-2.6.0/&&python setup.py install --user)
$ export $PATH=~/.local/bin:$PATH

PATH shouldn't be needed, but the script in /usr/bin was taken for some reason.

Get the juju-core dependencies:

$ export GOPATH=~/trees
$ go get launchpad.net/juju-core/...

Update some paths in the crontab:

$ crontab -l > ~tarmac/crontab.EDITED

Currently need to set:

PATH=/home/tarmac/.local/bin:/usr/local/bin:/usr/bin:/bin
GOPATH=/home/tarmac/trees

Copy in those to the real crontab:

$ crontab -e


Missed this initially:

$ bzr whoami Tarmac

Warns that it doesn't contain and email address, that's fine.

$ bzr launchpad-login go-bot

Just in case it's needed.

Done!

$ logout


Issues:
* Can't find 'go' binary
  = Need to set all of $PATH in crontab, can't use $PATH
* Missing code.google.com/p/go.net/websocket
  = Used wrong GOPATH when running `go get` initially
* Test failures in bzr_test.go "Unable to determine your name."
  = Set `bzr whoami` for the tarmac user.. really test suite should isolate.
* Test failure in simplestreams_test.go "Unable to lookup series"
  = Needed to instal distro-info as the message states.
* Test failure in cmd/jujud/machine_test.go due to isolation issue with api
  = Filed lp:1206195 and skipped out the test on trunk for now.
* Tarmac logs different/lacking verbosity
  ?


The network hackery:

At the moment, lcy02 and lcy01 can no longer talk to each other
directly. As I want to control the juju environment from the bot from
a machine I already have on lcy01, but use the more modern lcy02, this
means routing fun. The answer is adding forwarding back through
chinstrap again, then routing all traffic to lcy02 (most notably the
juju state and api connections) over an ssh tunnel using sshuttle in a
background process.

Martin