[3.13.y-ckt stable] Patch "[media] saa7134: Fix bytesperline not being set correctly for planar formats" has been added to the 3.13.y-ckt tree

[Kamal Mostafa]

This is a note to let you know that I have just added a patch titled

    [media] saa7134: Fix bytesperline not being set correctly for planar formats

to the linux-3.13.y-queue branch of the 3.13.y-ckt extended stable tree 
which can be found at:

    http://kernel.ubuntu.com/git/ubuntu/linux.git/log/?h=linux-3.13.y-queue

This patch is scheduled to be released in version 3.13.11-ckt38.

If you, or anyone else, feels it should not be added to this tree, please 
reply to this email.

For more information about the 3.13.y-ckt tree, see
https://wiki.ubuntu.com/Kernel/Dev/ExtendedStable

Thanks.
-Kamal

---8<------------------------------------------------------------

>From 7b81e2f3caa8eee8ea2e140261a1e7252b7c675d Mon Sep 17 00:00:00 2001
From: Hans de Goede <hdegoede at redhat.com>
Date: Sun, 14 Feb 2016 17:51:37 -0200
Subject: [media] saa7134: Fix bytesperline not being set correctly for planar
 formats

commit 3e71da19f9dc22e39a755d6ae9678661abb66adc upstream.

bytesperline should be the bytesperline for the first plane for planar
formats, not that of all planes combined.

This fixes a crash in xawtv caused by the wrong bpl.

BugLink: https://bugzilla.redhat.com/show_bug.cgi?id=1305389
Reported-and-tested-by: Stas Sergeev <stsp at list.ru>

Signed-off-by: Hans de Goede <hdegoede at redhat.com>
Signed-off-by: Hans Verkuil <hans.verkuil at cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab at osg.samsung.com>
Signed-off-by: Kamal Mostafa <kamal at canonical.com>
---
 drivers/media/pci/saa7134/saa7134-video.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/drivers/media/pci/saa7134/saa7134-video.c b/drivers/media/pci/saa7134/saa7134-video.c
index fb60da8..d11fdfb 100644
--- a/drivers/media/pci/saa7134/saa7134-video.c
+++ b/drivers/media/pci/saa7134/saa7134-video.c
@@ -1562,10 +1562,13 @@ static int saa7134_g_fmt_vid_cap(struct file *file, void *priv,
 	f->fmt.pix.height       = dev->height;
 	f->fmt.pix.field        = fh->cap.field;
 	f->fmt.pix.pixelformat  = dev->fmt->fourcc;
-	f->fmt.pix.bytesperline =
-		(f->fmt.pix.width * dev->fmt->depth) >> 3;
+	if (dev->fmt->planar)
+		f->fmt.pix.bytesperline = f->fmt.pix.width;
+	else
+		f->fmt.pix.bytesperline =
+			(f->fmt.pix.width * dev->fmt->depth) / 8;
 	f->fmt.pix.sizeimage =
-		f->fmt.pix.height * f->fmt.pix.bytesperline;
+		(f->fmt.pix.height * f->fmt.pix.width * dev->fmt->depth) / 8;
 	f->fmt.pix.colorspace   = V4L2_COLORSPACE_SMPTE170M;
 	f->fmt.pix.priv = 0;
 	return 0;
@@ -1646,10 +1649,13 @@ static int saa7134_try_fmt_vid_cap(struct file *file, void *priv,
 	if (f->fmt.pix.height > maxh)
 		f->fmt.pix.height = maxh;
 	f->fmt.pix.width &= ~0x03;
-	f->fmt.pix.bytesperline =
-		(f->fmt.pix.width * fmt->depth) >> 3;
+	if (fmt->planar)
+		f->fmt.pix.bytesperline = f->fmt.pix.width;
+	else
+		f->fmt.pix.bytesperline =
+			(f->fmt.pix.width * fmt->depth) / 8;
 	f->fmt.pix.sizeimage =
-		f->fmt.pix.height * f->fmt.pix.bytesperline;
+		(f->fmt.pix.height * f->fmt.pix.width * fmt->depth) / 8;
 	f->fmt.pix.colorspace   = V4L2_COLORSPACE_SMPTE170M;
 	f->fmt.pix.priv = 0;

--
2.7.4