Usability issues with status-history

Thu Mar 17 05:30:04 UTC 2016

On Thu, Mar 17, 2016 at 8:41 AM, Ian Booth <ian.booth at canonical.com> wrote:

>
> Machines, services and units all now support recording status history. Two
> issues have come up:
>
> 1. https://bugs.launchpad.net/juju-core/+bug/1530840
>
> For units, especially in steady state, status history is spammed with
> update-status hook invocations which can obscure the hooks we really care
> about
>
> 2. https://bugs.launchpad.net/juju-core/+bug/1557918
>
> We now have the concept of recording a machine provisioning status. This is
> great because it gives observability to what is happening as a node is
> being
> allocated in the cloud. With LXD, this feature has been used to give
> visibility
> to progress of the image downloads (finally, yay). But what happens is
> that the
> machine status history gets filled with lots of "Downloading x%" type
> messages.
>
> We have a pruner which caps the history to 100 entries per entity. But we
> need a
> way to deal with the spam, and what is displayed when the user asks for
> juju
> status-history.
>
> Options to solve bug 1
>
> A.
> Filter out duplicate status entries when presenting to the user. eg say
> "update-status (x43)". This still allows the circular buffer for that
> entity to
> fill with "spam" though. We could make the circular buffer size much
> larger. But
> there's still the issue of UX where a user ask for the X most recent
> entries.
> What do we give them? The X most recent de-duped entries?
>
> B.
> If the we go to record history and the current previous entry is the same
> as
> what we are about to record, just update the timestamp. For update status,
> my
> view is we don't really care how many times the hook was run, but rather
> when
> was the last time it ran.
>

The problem is that it isn't the same as the "last" message. Going to the
original paste:

TIME                    TYPE    STATUS          MESSAGE
26 Dec 2015 13:51:59Z   agent   idle
26 Dec 2015 13:56:57Z   agent   executing       running update-status hook
26 Dec 2015 13:56:59Z   agent   idle
26 Dec 2015 14:01:57Z   agent   executing       running update-status hook
26 Dec 2015 14:01:59Z   agent   idle

Which means there is an "running update-status" *and* a "idle" message. So
we can't just say "is the last message == this message". It would have to
look deeper in history, and how deep should we be looking? what happens if
a given charm does one more "status-set" during its update-status hook to
set the status of the unit to "still happy". Then we would have 3. (agent
executing, unit happy, agent idle)

> Options to solve bug 2
>
> A.
> Allow a flag when setting status to say "this status value is transient"
> and so
> it is recorded in status but not logged in history.
>
> B.
> Do not record machine provisioning status in history. It could be argued
> this
> info is more or less transient and once the machine comes up, we don't
> care so
> much about it anymore. It was introduced to give observability to machine
> allocation.
>

Isn't this the same as (A)? We need a way to say that *this* message should
be showed but not saved forever. Or are you saying that until a machine
comes up as "running" we shouldn't save any of the messages? I don't think
we want that, because when provisioning fails you want to know what steps
were achieved.

>
> Any other options?
> Opinions on preferred solutions?
>
> I really want to get this fixed before Juju 2.0
>

We could do a "log level" rather than just "transient or not", and that
would decide what would get displayed by default. (so you can ask for
'update-status' messages but they wouldn't be shown by default). The
problem is that we want to keep status messages pruned at a sane level and
with 2 updates for every 'update-status' call history of 100 is only
100/2*5/60 ~ 4hours of history. If something interesting happened
yesterday, you're SOL.

What if we added a "interesting lifetime" to status messages. So the
status-set could indicate how long the message would be preserved?
"update-status" and "idle" could be flagged as preserved for only 1 hour,
and "dowloading %" could be flagged at say 5 minutes. Too complicated? It
certainly complicates the pruner (not terribly, when we record them we just
record an expire time that is indexed and the pruner just removes
everything that is over its expiry time.)

Alternatively we could have some sort of UUID for messages to indicate that
"this message is actually similar to other messages with this UUID" and we
prune them based on that. (UUIDs get flagged with a different number of
messages to keep than the global 100 for otherwise untagged messages.)

"Transient" is the easiest to understand, but doesn't really solve bug #1.

If we think of the "UUID" version as something like a named "status pocket"
maybe its actually tasteful. You'd have the "global" pocket that has our
default 100 most-recent-messages, and then you can create any new pocket
that has a default of say 10 messages. So you would be doing:
 status-set --pocket hook-execution update-status
 status-set --pocket download Downloading X% done

That also lets charms do nice things at hook execution time when they're
downloading large resources, without spamming the status-history log.

It does complicate the model....

John
=:->
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20160317/f76331f4/attachment-0001.html>