[Bug 1702777] Re: ceph package is not built with jemalloc support

[Herbert Alexander Faleiros]

On Fri, Jul 07, 2017 at 08:05:14AM -0000, James Page wrote:
> Marking this as a feature for now; we've used tcmalloc for a long time,
> and I'm not sure whether we can build the packages to support both
> versions.

and what about install libjemalloc1 and uncomment:

## use jemalloc instead of tcmalloc
#
# jemalloc is generally faster for small IO workloads and when
# ceph-osd is backed by SSDs.  However, memory usage is usually
# higher by 200-300mb.
#
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1

...in /etc/default/ceph?


> ** Changed in: ceph (Ubuntu)
>        Status: New => Triaged
> 
> ** Changed in: ceph (Ubuntu)
>    Importance: Undecided => Wishlist
> 
> -- 
> You received this bug notification because you are a member of Ubuntu
> OpenStack, which is subscribed to ceph in Ubuntu.
> https://bugs.launchpad.net/bugs/1702777
> 
> Title:
>   ceph package is not built with jemalloc support
> 
> Status in ceph package in Ubuntu:
>   Triaged
> 
> Bug description:
>   For all-flash scenarios it makes sense to use jemalloc instead of
>   tcmalloc, see:
> 
>   http://tracker.ceph.com/projects/ceph/wiki/Tuning_for_All_Flash_Deployments#Memory-Tuning
>   https://www.sebastien-han.fr/blog/2015/09/07/the-ceph-and-tcmalloc-performance-story/
> 
>   https://software.intel.com/en-us/articles/using-intel-optane-and-
>   intel-3d-nand-technology-with-ceph-to-build-high-performance-cloud
> 
>   I have seen other people asking about this for trusty and xenial:
>   https://www.spinics.net/lists/ceph-users/msg28010.html
> 
>   https://www.spinics.net/lists/ceph-users/msg35221.html
> 
>   
>   However, our package is compiled only with tcmalloc:
> 
>   https://git.launchpad.net/~ubuntu-server-
>   dev/ubuntu/+source/ceph/tree/debian/rules?h=ubuntu/xenial#n39
> 
>   
>   On a live system (no trace of jemalloc):
>   readelf -d `which ceph-osd` | grep malloc
>    0x0000000000000001 (NEEDED)             Shared library: [libtcmalloc.so.4]
> 
>   dpkg -l '*malloc*'
>   Desired=Unknown/Install/Remove/Purge/Hold
>   | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
>   |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
>   ||/ Name                              Version               Architecture          Description
>   +++-=================================-=====================-=====================-========================================================================
>   ii  libtcmalloc-minimal4              2.4-0ubuntu5.16.04.1  amd64                 efficient thread-caching malloc
> 
>   strings `which ceph-osd` | grep jemalloc ; echo $?
>   1
> 
>   People may get confused with the fact that LD_PRELOAD will not result in jemalloc usage due to the fact that a package simply does not have that support compiled-in:
>   https://github.com/ceph/ceph/blob/jewel/etc/default/ceph
> 
>   
>   ---
> 
>   Given the fact that NVMe (filestore journal or bluestore WAL) -> SSD
>   (data) or Optane (filestore journal or bluestore WAL) -> SSD (data)
>   setups are becoming more common, --with-jemalloc option should be
>   added to the list of compile flags.
> 
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/1702777/+subscriptions
> 
> -- 
> Ubuntu-openstack-bugs mailing list
> Ubuntu-openstack-bugs at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-openstack-bugs

-- 
Herbert