[Bug 1212481] Re: [FFE] CouchDB 1.4.0 needed to work with Erlang 16.b.1
Jason Gerard DeRose
jderose at novacut.com
Mon Sep 2 06:22:30 UTC 2013
** Description changed:
- The current CouchDB 1.2.0 package in Saucy is broken and can't be
- installed as it was built against Erlang 15.b.1, and can't be rebuilt
- with the version now in Saucy, Erlang 16.b.1.
+ CouchDB 1.2.0 doesn't work at all with Erlang 16.b.1, so the current
+ CouchDB package in Saucy is completely broken and should be removed if
+ it can't be upgraded.
- However, the soon to be released CouchDB 1.4.0 does work with Erlang
- 16.b.1. So the options are to remove CouchDB 1.2.0 from the archive
- before Saucy is released, or to upgrade to CouchDB 1.4.0.
+ To work with Erlang 16.b.1, we'll have to upgrade to the recently
+ released CouchDB 1.4.0. The needed compatibility fixes are extensive and
+ include upgrading CouchDB's internal mochiweb server, so backporting the
+ needed changes to CouchDB 1.2.0 isn't practical.
- My vote is to upgrade to CouchDB 1.4.0, which should be released before
- the Saucy feature freeze on August 29th (based on what was said at the
- latest CouchDB developers IRC meeting).
+ Following is an overview of the testing I've done, results, and known
+ issues:
- In preparation, I have a 1.4.0 pre-release git snapshot in the Novacut daily PPA ready for testing:
- https://launchpad.net/~novacut/+archive/daily?field.series_filter=saucy
+ == Unit Tests ==
- The packaging for which is here:
- https://code.launchpad.net/~jderose/+junk/couchdb
+ CouchDB 1.4.0 now has quite extensive unit tests, which I'm running
+ during the build. These tests are (generally) passing fine when building
+ with pbuilder, and when building in a PPA (i386 and amd64).
- I know at least several things are still incorrect in the packaging, but
- I wanted to start putting it through its paces as soon as possible.
+ Note I've not tried a build on armhf yet.
- ** Feedback Wanted **
+ There are a few unit tests that occasionally fail when building on
+ Raring, but so far I haven't encountered this when building on Saucy. It
+ is however possible to get successful builds on Raring, you just might
+ have to retry a build or two.
- I'm also hoping to get some feedback on my plan for CouchDB in Ubuntu
- going forward. I have some history with the CouchDB package in Ubuntu
- [1][2], and for better or worse, the current CouchDB 1.2.0 package is
- largely my doing.
+ == Reverse Dependencies ==
- Previously my goal was to stay as close as possible to the CouchDB
- package in Debian, changing just enough to split it into the `couchdb`
- and `couchdb-bin` binary packages. And my hope was to get the Debian
- maintainer to eventually warm up to this split so Ubuntu could just ship
- a zero-change sync from Debian.
+ I've built several CouchDB consumers against my proposed CouchDB package
+ in a PPA, on both Saucy and Raring. These reverse dependencies have
+ extensive CouchDB using unit tests and are therefor a good measure of
+ the correctness of my proposed CouchDB 1.4.0 package, and the quality of
+ the upstream CouchDB 1.4.0 release.
- (FYI, this split is critical for Novacut because it allows us to start
- our per-user CouchDB instances via `couchdb-bin`, but without having the
- needless system-wide `couchdb` instance running all the time.)
+ UserCouch: https://launchpad.net/usercouch
+ * Good coverage of key configuration permutations (bind address, port, basic auth, oauth, file_compression)
+ * Good coverage of SSL options, both for httpd and the replicator
+ * Provides good confidence that CouchDB actually starts, and starts in a timely manner, across all config permutations
+ * IPv4 and IPv6 tests
- But I think it's time to change my strategy, time for me to just take
- ownership for the CouchDB package in Ubuntu.
+ Microfiber: https://launchpad.net/microfiber
+ * Good coverage of core DB, doc, and attachment REST APIs
+ * Deep tests for _bulk_docs API, testing both "non-atomic" and "all-or-nothing" update semantics
+ * Basic view tests
+ * Basic replication tests
+ * IPv4 and IPv6 tests
- As CouchDB 1.2.0 is still the newest in Debian (even in unstable and
- experimental), I think the current Debian maintainer is probably a bit
- too busy with other things to spend much time on CouchDB packaging (hey,
- life happens). Plus, I've never heard back from him about my idea of
- bringing the couchdb/couchdb-bin split into Debian.
+ Dmedia: https://launchpad.net/dmedia
+ * Complex, demanding application with lots of "real-world" test scenarios
+ * Extensive view tests with complex map functions (but _count, _sum, and _stats are the only reduce functions Dmedia uses)
+ * Full-stack replication tests with SSL (and client-side SSL certs)
+ * Tests for complex conflict scenarios
- Whereas I *must* have a working CouchDB package for Saucy, even if it
- means me just maintaining it in a PPA, because otherwise Novacut doesn't
- work on Saucy, Novacut can't move forward during Saucy. If I'm doing the
- work either way, I'd rather it be on the actual package shipped in
- Saucy.
+ == Manual Testing ==
- So I'm not going to worry about staying close to the Debian package
- anymore. Instead, I'm just going to package CouchDB in the way I feel is
- best, in the way I'm most comfortable with, in the way the seems best
- for Novacut and Ubuntu. I've more or less started with a blank slate for
- my CouchDB 1.4.0 package, and I'd say there are three important changes:
+ I've done extensive manual testing with Dmedia. I peered 3 different
+ computers and imported around 20k files into Dmedia, made sure all
+ metadata got replicated between all peers. I ran through my standard
+ battery of abuse using things like PurgeAll and DowngradeAll, and in all
+ scenarios the library successfully converged to its equilibrium state in
+ a reasonable amount of time.
- 1) Simplicity: the old debian/rules was quite complicated, and as I
- didn't write it, I'm not well equipped to maintain it, especially with
- precious little free time; no doubt there was a reason for these hacks
- when they were written, but these days CouchDB is much better behaved
- when it comes to standard `./configure && make && make install`, so I
- was aiming for a near-empty debian/rules
+ I've also done extensive manual testing with the `couchdb` binary package and its new upstart job. I confirmed that:
+ 1) The daemon is started when the package is installed
+ 2) The daemon starts during the boot sequence after a reboot and a cold boot
+ 3) All daemon processes are killed with `sudo stop couchdb`
+ 4) You can start the daemon with `sudo start couchdb`
+ 5) `sudo restart couchdb` works as expected
+ 6) Upstart respawn works as expected if I manually `kill $PID` on the daemon process
+ 7) All daemon processes are killed with `sudo apt-get remove couchdb`
+ 8) `sudo apt-get purge couchdb` removes /var/lib/couchdb and /var/log/couchdb
- 2) debhelper: this is largely because I'm more comfortable with
- debhelper than I am with CDBS, but I also feel debhelper is the current
- best practice; and so far so good, my debian/rules is only 8 lines [3]
+ == Upgrade Testing ==
- 3) Upstart: I'll admit right now there were many and constant problems
- with the init.d script in CouchDB 1.2.0 that I never managed to solve;
- as I don't have time to babysit the ill behaved upstream init.d script,
- I switched to Upstart... which promptly solved *all* the problems,
- yeehaw =)
+ I tested the upgrade from 1.2.0 when only `couchdb-bin` is installed,
+ and when `couchdb` and `couchdb-bin` are installed. Both work as
+ expected, although these tests were done on Raring. I haven't yet done
+ an upgrade test via a Raring=>Saucy update, but that's difficult to do
+ when 1.4.0 is still only in a PPA (update process can't be properly
+ simulated as the PPA will be disabled during the update).
- I'd love help with this CouchDB package maintenance, if anyone is
- interested. Plus, I *need* at least a little help getting this into
- Saucy, as I'm not a MOTU :P
+ A remaining issue is that sometimes the `couchdb` 1.2.0 daemon is not
+ successfully terminated during the package upgrade. (Well, it sort of
+ is, and then Erlang respawns more processes that don't ever get killed.)
+ So you'll end up with the new, correct 1.4.0 processes running, plus
+ some lingering 1.2.0 garbage processes running. This is due to
+ fundamental problems in the upstream init.d script, and this is a big
+ part of why I switched to Upstart.
- Thoughts?
+ A work-around is to stop couchdb before you upgrade:
+ sudo service couchdb stop
- [1] https://bugs.launchpad.net/ubuntu/+source/couchdb/+bug/1022515
- [2] https://bugs.launchpad.net/ubuntu/+source/couchdb/+bug/903098
- [3] http://bazaar.launchpad.net/~jderose/+junk/couchdb/view/head:/debian/rules
+ Or simply reboot after you upgrade (which most will do anyway).
+
+ This is probably a difficult issue to solve as the problem is that the
+ previous package version isn't correctly shutting down its CouchDB
+ daemon. On the upside, no such problems exist with the Upstart job in my
+ 1.4.0 package.
--
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1212481
Title:
[FFE] CouchDB 1.4.0 needed to work with Erlang 16.b.1
Status in “couchdb” package in Ubuntu:
New
Bug description:
CouchDB 1.2.0 doesn't work at all with Erlang 16.b.1, so the current
CouchDB package in Saucy is completely broken and should be removed if
it can't be upgraded.
To work with Erlang 16.b.1, we'll have to upgrade to the recently
released CouchDB 1.4.0. The needed compatibility fixes are extensive
and include upgrading CouchDB's internal mochiweb server, so
backporting the needed changes to CouchDB 1.2.0 isn't practical.
Following is an overview of the testing I've done, results, and known
issues:
== Unit Tests ==
CouchDB 1.4.0 now has quite extensive unit tests, which I'm running
during the build. These tests are (generally) passing fine when
building with pbuilder, and when building in a PPA (i386 and amd64).
Note I've not tried a build on armhf yet.
There are a few unit tests that occasionally fail when building on
Raring, but so far I haven't encountered this when building on Saucy.
It is however possible to get successful builds on Raring, you just
might have to retry a build or two.
== Reverse Dependencies ==
I've built several CouchDB consumers against my proposed CouchDB
package in a PPA, on both Saucy and Raring. These reverse dependencies
have extensive CouchDB using unit tests and are therefor a good
measure of the correctness of my proposed CouchDB 1.4.0 package, and
the quality of the upstream CouchDB 1.4.0 release.
UserCouch: https://launchpad.net/usercouch
* Good coverage of key configuration permutations (bind address, port, basic auth, oauth, file_compression)
* Good coverage of SSL options, both for httpd and the replicator
* Provides good confidence that CouchDB actually starts, and starts in a timely manner, across all config permutations
* IPv4 and IPv6 tests
Microfiber: https://launchpad.net/microfiber
* Good coverage of core DB, doc, and attachment REST APIs
* Deep tests for _bulk_docs API, testing both "non-atomic" and "all-or-nothing" update semantics
* Basic view tests
* Basic replication tests
* IPv4 and IPv6 tests
Dmedia: https://launchpad.net/dmedia
* Complex, demanding application with lots of "real-world" test scenarios
* Extensive view tests with complex map functions (but _count, _sum, and _stats are the only reduce functions Dmedia uses)
* Full-stack replication tests with SSL (and client-side SSL certs)
* Tests for complex conflict scenarios
== Manual Testing ==
I've done extensive manual testing with Dmedia. I peered 3 different
computers and imported around 20k files into Dmedia, made sure all
metadata got replicated between all peers. I ran through my standard
battery of abuse using things like PurgeAll and DowngradeAll, and in
all scenarios the library successfully converged to its equilibrium
state in a reasonable amount of time.
I've also done extensive manual testing with the `couchdb` binary package and its new upstart job. I confirmed that:
1) The daemon is started when the package is installed
2) The daemon starts during the boot sequence after a reboot and a cold boot
3) All daemon processes are killed with `sudo stop couchdb`
4) You can start the daemon with `sudo start couchdb`
5) `sudo restart couchdb` works as expected
6) Upstart respawn works as expected if I manually `kill $PID` on the daemon process
7) All daemon processes are killed with `sudo apt-get remove couchdb`
8) `sudo apt-get purge couchdb` removes /var/lib/couchdb and /var/log/couchdb
== Upgrade Testing ==
I tested the upgrade from 1.2.0 when only `couchdb-bin` is installed,
and when `couchdb` and `couchdb-bin` are installed. Both work as
expected, although these tests were done on Raring. I haven't yet done
an upgrade test via a Raring=>Saucy update, but that's difficult to do
when 1.4.0 is still only in a PPA (update process can't be properly
simulated as the PPA will be disabled during the update).
A remaining issue is that sometimes the `couchdb` 1.2.0 daemon is not
successfully terminated during the package upgrade. (Well, it sort of
is, and then Erlang respawns more processes that don't ever get
killed.) So you'll end up with the new, correct 1.4.0 processes
running, plus some lingering 1.2.0 garbage processes running. This is
due to fundamental problems in the upstream init.d script, and this is
a big part of why I switched to Upstart.
A work-around is to stop couchdb before you upgrade:
sudo service couchdb stop
Or simply reboot after you upgrade (which most will do anyway).
This is probably a difficult issue to solve as the problem is that the
previous package version isn't correctly shutting down its CouchDB
daemon. On the upside, no such problems exist with the Upstart job in
my 1.4.0 package.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/couchdb/+bug/1212481/+subscriptions
More information about the Ubuntu-sponsors
mailing list