<div dir="ltr">Hi Martin,<div><br></div><div><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Aug 19, 2014 at 12:29 PM, Martin Pitt <span dir="ltr"><<a href="mailto:martin.pitt@ubuntu.com" target="_blank">martin.pitt@ubuntu.com</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello Andres,<br> <br> thanks for the updates. (Also, I was on holiday, sorry for late reply)<br> <br> Andres Rodriguez [2014-08-13 19:58 -0400]:<br> <div>> > > testing.<br> > > > We will provide an upgrade path that "just works".<br> > ><br> > > Can you please expand that? I. e. how do you make sure that existing<br> > > MAAS deployments that were done with earlier versions continue to work<br> > > with proposed new versions? How much do protocol changes affect<br> > > particular hardware, i. e. up to which degree can this be tested<br> > > automatically with e. g. a bunch of commodity hardware or even QEMU,<br> > > and which parts would need the actual supported set of hardware,<br> > > things like ppc64el or ARM servers, or other hardware which is hard to<br> > > get into a CI environment?<br> > ><br> ><br> > We currently maintain backwards compatibility with all of the components,<br> > so every upgrade will yield a working MAAS that continues to work as<br> > expected. We currently do manual testing for upgrades, but we are in the<br> > process of adding automated upgrade testing to our CI.<br> <br> </div>How do you do the manual upgrade testing, and what's the rough<br> structure of testing this automatically then? The "what" is quite<br> clear now, I'm still interested in the "How" for regression-testing<br> newer versions against existing deployments.<br></blockquote><div><br></div><div>The manual upgrade testing is done as follows:</div><div><br></div><div>1. Install, configure MAAS</div><div>2. We register machines</div><div> 3. Commission machines</div><div>4. Perform deployments</div><div><br></div><div>This allow us to test the current released version.</div><div><br></div><div>5. Upgrade</div><div>6. Perform deployments</div><div>7. Register machines</div> <div>8. Perform deployments.</div><div><br></div><div>This allow us to test the new version, and check for any regressions.</div><div><br></div><div>In one of hour labs we have 100+ systems, and we test trunk version of MAAS:</div> <div><br></div><div>1. Stop machine deployments.</div><div>2. Perform upgrade</div><div>3. Resume deployments</div><div>4. Register / Re-Commission some machines to check for regressions</div><div>5. Perform new deployments.</div> <div><br></div><div>Automated Testing</div><div><br></div><div>The automated testing we are in the process of adding will allow us to do two things:</div><div><br></div><div>1. Fresh install of trunk</div><div>2. Register hardware</div> <div>3. Commission hardware</div><div><br></div><div>At this point, we are able to identify any regressions.</div><div><br></div><div>4. Perform deployments</div><div><br></div><div>At this point, we are also able to identify regressions.</div> <div><br></div><div>What's currently being added to the CI:</div><div><br></div><div>1. Install released MAAS version.</div><div>2. Register hardware</div><div>3. Commission hardware</div><div>4. Deploy hardware.</div> <div>5. Upgrade hardware</div><div><br></div><div>At this point we are able to identify regressions.</div><div><br></div><div>6. Perform deployments</div><div><br></div><div>At this point we will be able to identify regressions.</div> <div><br></div><div>Thanks</div><div><br></div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div><div><br> Thanks!<br> <br> Martin<br> <br> --<br> Martin Pitt | <a href="http://www.piware.de" target="_blank">http://www.piware.de</a><br> Ubuntu Developer (<a href="http://www.ubuntu.com" target="_blank">www.ubuntu.com</a>) | Debian Developer (<a href="http://www.debian.org" target="_blank">www.debian.org</a>)<br> </div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>Andres Rodriguez (RoAkSoAx)<br>Ubuntu Server Developer<br>Systems Engineer </div></div>