LD_PRELOAD work in progress

[Jamie Strandboge]

On 02/24/2015 05:06 PM, Stéphane Graber wrote:

Hi Stephane, thanks for the reply :)

> On Tue, Feb 24, 2015 at 04:40:21PM -0600, Jamie Strandboge wrote:
>> On 02/24/2015 03:29 PM, Michael Terry wrote:
>>> Hello!  So I worked on a proof-of-concept for faking an overlayfs scenario by
>>> doing path-redirection in an LD_PRELOAD library.
>>>
>>> I've got working code here: lp:~mterry/+junk/snappy-preload
>>>
>> This is a very intriguing idea.
>>
...
>> I'm very curious what others have to say.
> 
> So the main problem with the LD_PRELOAD approach and why I discarded it
> back in CPT is that it will not work for file system access coming from
> a static binary.
> 
> So if I write a Go piece of software which opens some file from
> /usr/share provided by the overlay either in the app itself or one of
> its frameworks, it won't be able to access the file because it won't be
> using the local libc.
> 
So, thinking about what overlayfs is trying to solve more-- there are really
three things that could be in the overlay area:
 * libraries
 * executables
 * data files

There are essentially two things that we've thus far been thinking about that
may use overlays:
 * apps that reuse existing debs
 * frameworks

Apps that reuse existing debs and are dynamically compiled could potentially use
something like the LD_PRELOAD trick to get access to libraries, executables and
data files. Apps that are statically compiled would of course not use the
libraries from the existing debs and simply use the executables (which would use
the libraries) and data files there. However, reusing executables from an
existing deb is almost certainly going to break often because those executables
are running under the app's confinement that was not designed for them.
Furthermore for statically compiled apps, LD_PRELOAD would be set on statically
compiled app launch, but the app wouldn't propagate the LD_PRELOAD for a
fork/exec of something within the app area that is dynamically linked.
Therefore, reusing executables is not really feasible. The question then
becomes-- could we not just copy the data files into the app area?  However, do
we reasonably expect statically compiled apps to ship reused debs (that use
dynamic linking) in this manner? This seems like perhaps we are over
engineering-- we could just say, if you are statically compiling, you can ship
reused debs' data and you can keep track of where it is.

Frameworks are currently defined as existing primarily to provide mediation of
shared resources (eg, ports, device files, sensors, etc, etc). As such, a
framework will provide a service for apps to talk to and then the app and the
framework service will work together to give the app access to the resource.
This service is going to be an out of process helper and the app needs to
understand how to work with it, whether that app is statically compiled or not.
Because the service is out of process, there is no need for using executables in
the framework (indeed, there are rather complex exec transitions that must be
considered with security policy that make this hard regardless of overlayfs,
LD_PRELOAD or something else). Frameworks shouldn't be providing libs because
they won't work with statically compiled apps anyway. Which leaves us (again)
with data files. The LD_PRELOAD trick should work for data files in frameworks
and, like with reusing existing debs, statically compiled apps can keep track of
where the data is.

So, unless I am missing something, reusing debs seems really to cater to
dynamically linked apps to reuse libraries and data files. Statically compiled
apps could reuse deb data files. With the LD_PRELOAD trick, we don't make the
reusable deb data story any better for statically compiled, but why can't the
statically compiled app just reference the data files it ships in its install
directory rather than going through levels of overlayfs?

Perhaps there are some requirements I am missing?

> 
> My alternative to overlayfs at the time was a bind-mount farm in a
> separate mount namespace, that is, giving the exact same fs view as
> you'd get using read-only overlayfs but using read-only bind-mounts and
> tmpfs mounts instead (possibly hundreds/thousands of them).
> 
While I don't doubt we could make this work, this doesn't feel snappy to me and
is the least desirable of the options IMHO. If we don't like the LD_PRELOAD
idea, overlayfs can't be made to work for us and there isn't another
alternative, we could resurrect (gasp) aufs. Ubuntu has a history with it (eg,
we probably have patches for all kinds of old kernels laying about), AIUI it is
relatively self-contained so porting shouldn't be too bad and it could be made
to work with apparmor considerably more easily than the current implementation
of overlayfs. If we don't like LD_PRELOAD and would prefer overlays, perhaps we
would use aufs as a bridge until overlayfs works the way we need it to.

Thanks!

-- 
Jamie Strandboge                 http://www.ubuntu.com/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <https://lists.ubuntu.com/archives/snappy-devel/attachments/20150225/3b188b2e/attachment.pgp>