Action return value too long for commandline

[Mark Shuttleworth]

We should definitely have a mechanism for large-file output from actions
and other hooks. I think resources are "inward flowing", they come from
the charmer or they are provided by the operator. You're looking for
"outward flowing" blobs.

As I recall, the intention was that actions could drop results into the
controller in a way that was friendly to large BLOB results. That's why
we have an async action output and result mechanism. I reckon it would
be fine to use resources for input to actions (there would just be a
specific user-provided resource name and there would be only one version
of that at any given time), but we want to make sure that output can
also be efficiently handled regardless of size, so if we need to improve
that, let's crack on.

Mark

On 04/04/16 16:56, Merlijn Sebrechts wrote:
> Hi Stuart
>
>
> Thanks for this explanation. 16MB is definitely a step in the right
> direction. However, in the future I'll need to transfer even bigger files.
> Is there a possibility for using resources for this in the future? It would
> be great if an action could upload a file to the resources server so the
> admin can download it from there.
>
>
>
> Kind regards
> Merlijn Sebrechts
>
> 2016-04-03 19:34 GMT-07:00 Stuart Bishop <stuart.bishop at canonical.com>:
>
>> On 4 April 2016 at 02:00, Merlijn Sebrechts <merlijn.sebrechts at gmail.com>
>> wrote:
>>> Hi all
>>>
>>>
>>> The apache-kafka charm has an action "read-topic" that can return a lot
>> of
>>> data. Sometimes, this data is too long to be passed to `action-set` by
>>> commandline. You get the following error:
>>>
>>> Traceback (most recent call last):
>>>   File "actions/read-topic", line 36, in <module>
>>>     hookenv.action_set({'output': output})
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/charmhelpers/core/hookenv.py",
>> line
>>> 615, in action_set
>>>     subprocess.check_call(cmd)
>>>   File "/usr/lib/python2.7/subprocess.py", line 535, in check_call
>>>     retcode = call(*popenargs, **kwargs)
>>>   File "/usr/lib/python2.7/subprocess.py", line 522, in call
>>>     return Popen(*popenargs, **kwargs).wait()
>>>   File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
>>>     errread, errwrite)
>>>   File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
>>>     raise child_exception
>>> OSError: [Errno 7] Argument list too long
>>>
>>>
>>> Is there any way to pass a file to action-set?
>> This bug affects many of the Juju tools causing charms to fail at
>> scale. https://bugs.launchpad.net/juju-core/+bug/1437366
>> (relation-set) is the only one I know of that has been fixed.
>> https://bugs.launchpad.net/juju-core/+bug/1274460 (juju-log) is still
>> open. leader-set also fails now I think of it, but I haven't tripped
>> over that one (it limits scalability the way I'm using leadershp
>> settings, but I should be able to squeeze out several hundred units).
>> Maybe we can use the opportunity to fix quoting and encoding issues.
>>
>> I suspect if you can get past command line length limitations, I
>> suspect the next glass ceiling is a 16MB document size limit in
>> MongoDB (which is large enough to not need fixing?)
>>
>> --
>> Stuart Bishop <stuart.bishop at canonical.com>
>>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20160405/f68c94bb/attachment.html>