ACK: [Vivid][SRU][PATCH] perf trace: Fix race condition at the end of started workloads
Brad Figg
brad.figg at canonical.com
Fri Jun 26 16:39:22 UTC 2015
On Fri, Jun 26, 2015 at 10:06:21AM -0500, Chris J Arges wrote:
> From: Sukadev Bhattiprolu <sukadev at linux.vnet.ibm.com>
>
> BugLink: http://bugs.launchpad.net/bugs/1410673
>
> I get following crash on multiple systems and across several releases
> (at least since v3.18).
>
> Core was generated by `/tmp/perf trace sleep 0.2 '.
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0 perf_mmap__read_head (mm=0x3fff9bf30070) at util/evlist.h:195
> 195 u64 head = ACCESS_ONCE(pc->data_head);
> (gdb) bt
> #0 perf_mmap__read_head (mm=0x3fff9bf30070) at util/evlist.h:195
> #1 perf_evlist__mmap_read (evlist=0x10027f11910, idx=<optimized out>)
> at util/evlist.c:637
> #2 0x000000001003ce4c in trace__run (argv=<optimized out>,
> argc=<optimized out>, trace=0x3fffd7b28288) at builtin-trace.c:2259
> #3 cmd_trace (argc=<optimized out>, argv=<optimized out>,
> prefix=<optimized out>) at builtin-trace.c:2799
> #4 0x00000000100657b8 in run_builtin (p=0x10176798 <commands+480>, argc=3,
> argv=0x3fffd7b2b550) at perf.c:370
> #5 0x00000000100063e8 in handle_internal_command (argv=0x3fffd7b2b550, argc=3)
> at perf.c:429
> #6 run_argv (argv=0x3fffd7b2af70, argcp=0x3fffd7b2af7c) at perf.c:473
> #7 main (argc=3, argv=0x3fffd7b2b550) at perf.c:588
>
> The problem seems to be a race condition, when the application has just
> exited. Some/all fds associated with the perf-events (tracepoints) go
> into a POLLHUP/ POLLERR state and the mmap region associated with those
> events are unmapped (in perf_evlist__filter_pollfd()).
>
> But we go back and do a perf_evlist__mmap_read() which assumes that the
> mmaps are still valid and we hit the crash.
>
> If the mapping for an event is released, its refcnt is 0 (and ->base
> is NULL), so ensure we have non-zero refcount before accessing the map.
>
> Note that perf-record has a similar logic but unlike perf-trace, the
> record__mmap_read_all() checks the evlist->mmap[i].base before accessing
> the map.
>
> Signed-off-by: Sukadev Bhattiprolu <sukadev at linux.vnet.ibm.com>
> Cc: Jiri Olsa <jolsa at redhat.com>
> Cc: Li Zhang <zhlcindy at linux.vnet.ibm.com>
> Link: http://lkml.kernel.org/r/20150612060003.GA19913@us.ibm.com
> [ Fixed it up to use atomic_read() ]
> Signed-off-by: Arnaldo Carvalho de Melo <acme at redhat.com>
>
> (backported from commit 7951722da2963cc1f1a7831a37aa2311ac927056)
> Signed-off-by: Chris J Arges <chris.j.arges at canonical.com>
>
> Conflicts:
> tools/perf/util/evlist.c
> ---
> tools/perf/util/evlist.c | 9 ++++++++-
> 1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/util/evlist.c b/tools/perf/util/evlist.c
> index 2e507b5..5460e65 100644
> --- a/tools/perf/util/evlist.c
> +++ b/tools/perf/util/evlist.c
> @@ -635,11 +635,18 @@ static struct perf_evsel *perf_evlist__event2evsel(struct perf_evlist *evlist,
> union perf_event *perf_evlist__mmap_read(struct perf_evlist *evlist, int idx)
> {
> struct perf_mmap *md = &evlist->mmap[idx];
> - unsigned int head = perf_mmap__read_head(md);
> + unsigned int head;
> unsigned int old = md->prev;
> unsigned char *data = md->base + page_size;
> union perf_event *event = NULL;
>
> + /*
> + * Check if event was unmapped due to a POLLHUP/POLLERR.
> + */
> + if (!atomic_read(&md->refcnt))
> + return NULL;
> +
> + head = perf_mmap__read_head(md);
> if (evlist->overwrite) {
> /*
> * If we're further behind than half the buffer, there's a chance
> --
> 1.9.1
>
>
> --
> kernel-team mailing list
> kernel-team at lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/kernel-team
Looks good.
--
Brad Figg brad.figg at canonical.com http://www.canonical.com
More information about the kernel-team
mailing list