Maximum AllWatcher frame size?

[roger peppe]

On 14 June 2017 at 17:14, Cory Johns <cory.johns at canonical.com> wrote:
> So, I'm not sure if this resolves the issue.  The python websockets library
> supports fragmented messages but I think that is separate from the maximum
> message size that it's enforcing.  I think that maximum is for the final
> payload, which would be the entire AllWatcher JSON response.  If the
> controller doesn't break that in to multiple AllWatcher updates, then I
> think we'll still hit the maximum message size.
>
> I can increase the max size pretty easily, or even remove it entirely, as
> mentioned in the issue, but I'm a little bit leery of having no upper bound
> at all.  Do we know if there's a reasonable upper bound that even large
> models would fall within for the initial AllWatcher response?

Unfortunately there is no reasonable upper bound because models can
be arbitrarily large. I'd suggest setting it to a value that would be a
serious memory problem if it got that big.

Looking at the multiwatcher code, I don't think it would be too
difficult to make
it return only a limited number of delta elements in any single response.
That would not necessarily be a hard limit (theoretically an individual
delta could be very large - I don't think anyone imposes limits on
name sizes, for example) but would probably be good enough
in practice.

  cheers,
    rog.

>
> On Wed, Jun 14, 2017 at 12:08 PM, Tim Penhey <tim.penhey at canonical.com>
> wrote:
>>
>> It is configured in juju
>>
>>
>> // Use a 64k frame size for the websockets while we need to deal
>> // with x/net/websocket connections that don't deal with recieving
>> // fragmented messages.
>> const websocketFrameSize = 65536
>>
>>
>> On 14/06/17 12:00, Cory Johns wrote:
>> > Do we know what size the gorilla/websocket library uses for automatic
>> > chunking?
>> >
>> > On Wed, Jun 14, 2017 at 11:35 AM, John Meinel <john at arbash-meinel.com
>> > <mailto:john at arbash-meinel.com>> wrote:
>> >
>> >     In 2.1 we did not chunk, in 2.2 we switch to gorilla/websocket which
>> >     does support chunking into frames. I don't think we do any internal
>> >     "well that would be too much information so we wont send it all".
>> >
>> >     John
>> >     =:->
>> >
>> >
>> >     On Wed, Jun 14, 2017 at 7:11 PM, Cory Johns
>> >     <cory.johns at canonical.com <mailto:cory.johns at canonical.com>> wrote:
>> >
>> >         https://github.com/juju/python-libjuju/issues/136
>> >         <https://github.com/juju/python-libjuju/issues/136> raises the
>> >         issue that, for large models, the initial AllWatcher response
>> >         frame can be large enough that it overruns the default maximum
>> >         frame size of the websocket library (1MB).  We can increase this
>> >         limit fairly easily, but I wanted to find out whether there is
>> >         any maximum size from the Juju size and if large enough frames
>> >         could get chunked into multiple frames?
>> >
>> >         --
>> >         Juju-dev mailing list
>> >         Juju-dev at lists.ubuntu.com <mailto:Juju-dev at lists.ubuntu.com>
>> >         Modify settings or unsubscribe at:
>> >         https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >         <https://lists.ubuntu.com/mailman/listinfo/juju-dev>
>> >
>> >
>> >
>> >
>> >
>
>
>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>