RFC on Cloud Images: Make /tmp a tmpfs

[Dimitri John Ledkov]

On 13 January 2016 at 12:26, Ben Howard <ben.howard at canonical.com> wrote:
> All,
>
> On the Ubuntu Cloud Images, we have a request to make /tmp a tmpfs. The
> rationale, from the bug:
>  * Performance - much faster read/write access to data in /tmp
>  * Security - sensitive data would be cleared from memory on boot,
>    rather than written (leaked) to disk -- important for encryption
>    scenarios
>
> Since the Ubuntu Cloud Images are used by a wide number of users, I
> wanted to gather feedback and gather consensus on whether or not we
> should make this change.


IMHO this is specifically a bad idea on cloud-images.

Sure, it may improve the performance of an individual instance
(however, i do doubt that).
But it will increases committed, non-shareable memory per instance.
The net effect would be increased memory usage on the compute node,
higher over-committed memory.

And hence lower container (lxd) / VM density per node. I don't think
neither public or private cloud users would be happy, with lower
instance count per node.

I would be ok, if overall steady state committed memory usage would be
reduced in 16.04.

Looking at the hardware trends, we are ever increasingly on the verge
of breakthrough into persistent memory. NVMe, 3D XPoint, HPE
Persistent memory. Which is certainly targeted at the cloud. It
implies that "hard-drive" access speeds will be catching up to those
of memory, and remain persistent across power interruption. When
clouds will be powered by that, instead of using RAM, one would be
increasingly expected to store everything on disk.

And it's not that far off. DAX (previously XIP) support has landed in
the kernel, and in qemu-kvm. Above mentioned technologies certainly
will become available during 16.04 support time-frame, and would be
used by clouds pioneering those technologies.

Or going further, memory map the disk direct into code memory and just
use it direct - no need to deal with filesystems, serialisation,
deserialisation as the binary in-memory representation of the data is
in-fact persistent on disk. But i'm skeptical / fuzzy about it.

/tmp on tmpfs on cloud-images is not a practical choice today, and
it's not forward-looking, given the impeding hardware advances.

-- 
Regards,

Dimitri.