RFC on Cloud Images: Make /tmp a tmpfs

[Dustin Kirkland]

On Thu, Jan 21, 2016 at 4:14 AM, Martin Pitt <martin.pitt at ubuntu.com> wrote:
> Robie Basak [2016-01-21  9:56 +0000]:
>> I think we need a single canonical (small c) answer. Let's say that I'm
>> an upstream maintainer of a project that currently uses large amounts of
>> space in /tmp (say debdiff on kernel sources, for example). A user files
>> a bug that his machine now explodes when he uses my program. How should
>> I fix the bug?
>
> How is that any different than what we have now? systems with more
> memory than space on the root partition do exist [1], systems with
> /tmp on tmpfs do exist. We are *not* going from "/tmp has indefinite
> space" to "/tmp has little space", we are just changing the limit (not
> necessarily downwards even!) to a differently fuzzy definition.

Martin is absolutely correct!

Ubuntu's AWS images which are EBS backed -- the VAST majority of
Ubuntu currently running in the world -- all have a root volume of 7.8
GB.  Here's a fresh instance:

ubuntu at ip-172-30-0-30:~⟫ df -h /
Filesystem                                              Size  Used
Avail Use% Mounted on
/dev/disk/by-uuid/0a76513a-37fc-43df-9833-34f8f9598ada  7.8G  790M  6.6G  11% /

With Ubuntu 14.04 LTS, /tmp is still on that root volume, giving me
~6.6 GB of usable space there:

ubuntu at ip-172-30-0-30:~⟫ df -h /tmp/
Filesystem                                              Size  Used
Avail Use% Mounted on
/dev/disk/by-uuid/0a76513a-37fc-43df-9833-34f8f9598ada  7.8G  790M  6.6G  11% /

However, the instance that I just launched has 16 GB of memory:

ubuntu at ip-172-30-0-30:~⟫ free
             total       used       free     shared    buffers     cached
Mem:      16433132     380392   16052740        428       8468     209584
-/+ buffers/cache:     162340   16270792
Swap:            0          0          0

Let's remount /tmp on tmpfs:

ubuntu at ip-172-30-0-30:~⟫ df -h /tmp/
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           7.9G     0  7.9G   0% /tmp

How about that...  My /tmp space actually went up!

In fact, 27 of the 39 current AWS instance types have >15GB of memory.

Oh, and EBS writes -- are SLOW!

ubuntu at ip-172-30-0-30:~⟫ dd if=/dev/zero of=/tmp/zero bs=2G count=1
2147479552 bytes (2.1 GB) copied, 1.35841 s, 1.6 GB/s

ubuntu at ip-172-30-0-30:~⟫ dd if=/dev/zero of=/home/ubuntu/zero bs=2G
count=1

                                                      2147479552 bytes
(2.1 GB) copied, 2.3478 s, 915 MB/s

ubuntu at ip-172-30-0-30:~⟫ dd if=/dev/zero of=/tmp/zero bs=2G count=1
oflag=dsync

                                                    2147479552 bytes
(2.1 GB) copied, 1.34014 s, 1.6 GB/s

ubuntu at ip-172-30-0-30:~⟫ dd if=/dev/zero of=/home/ubuntu/zero bs=2G
count=1 oflag=dsync

                                              2147479552 bytes (2.1
GB) copied, 23.1934 s, 92.6 MB/s

So on these instance types, /tmp on tmpfs provides both *more*
storage, and *faster* storage!

tmpfsffs,
Dustin

> If you program dumps vast amounts of data into /tmp, that is a problem
> somewhere no matter what you do. /var/tmp/ has traditionally had a
> better chance of carrying large files (but of course also not
> guaranteed).
>
> Martin
>
>
> [1] We've even had bug reports where people filled the root partition
> and then weren't able to log in any more because /tmp/ could not be
> written into. We detected this on boot and mounted a small tmpfs over
> /tmp/ to mitigate this.
>
> --
> Martin Pitt                        | http://www.piware.de
> Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)
>
> --
> ubuntu-devel mailing list
> ubuntu-devel at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
>