[Bug 1921474] Re: Support client usage notification via V4l2 Event API

[Shih-Yuan Lee]

** Tags added: originate-from-1949435

** Changed in: oem-priority
       Status: New => In Progress

** Changed in: oem-priority
   Importance: Undecided => High

** Changed in: oem-priority
     Assignee: (unassigned) => Shih-Yuan Lee (fourdollars)

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1921474

Title:
  Support client usage notification via V4l2 Event API

Status in HWE Next:
  In Progress
Status in OEM Priority Project:
  In Progress
Status in v4l2loopback package in Ubuntu:
  Fix Released
Status in v4l2loopback source package in Focal:
  In Progress
Status in v4l2loopback source package in Hirsute:
  Fix Released

Bug description:
  [SRU Justification]

  [Impact]

  On a MIPI camera through Intel IPU6 platform that its raw V4L2
  loopback interface is preserved for Intel Camera HAL libraries, a
  relay daemon + v4l2loopback is used to allow the usage of legacy V4L2
  based apps.

  By design, the relayd will open v4l2sink to v4l2loopback OUTPUT
  deivce, and it will only open libcamhal based GStreamer source
  element, and therefore underlying camera hardware, until received new
  client notifications via V4L2 Event API that is introduced in this
  SRU.

  Besides, frame sizes/intervals enumeration is also fixed to meet
  better compliance with user apps.

  [Test Plan]

  v4l2loopback doesn't support V4L2 Event API until recently, so
  requests for the usage will always fail:

    struct v4l2_event_subscription sub;
    int fd;

    memset (&sub, 0, sizeof (sub));
    sub.type = ...;
    if (ioctl (fd, VIDIOC_SUBSCRIBE_EVENT, &sub) == 0) // fail

  With this fix, it shall support Event API operations, e.g.
  VIDIOC_SUBSCRIBE_EVENT, VIDIOC_UNSUBSCRIBE_EVENT, VIDIOC_DQEVENT.

  [Where problems could occur]

  While a custom type of V4L2 event is now registered for OEM projects,
  software expected same ID (numerically equivalent to
  V4L2_EVENT_PRIVATE_START) may get confused. While this falls in the
  private usage section, it's not supposed to be used in general, and
  when it does, it should be under some presumptions, e.g. selected
  hardware, so it's unlikely to happen on OEM projects.

  For generic Ubuntu, programs may begin to take advantage of this new
  capability to update its UI, or to take other actions after receiving
  desired events. There might be behavior changes, but should be under
  the original design if was done carefully.

  [Other Info]

  For Focal backports, 0.12.5-1 is equivalent to 0.12.3-1ubuntu0.3 plus
  micro version updates, so there should be little risk backport a new
  release, and we can drop additional patches carried. This implies bug
  1905613.

  This focal backport includes changes for bug 1921474, bug 1930208, and
  bug 1936250, so it will then become synced with Impish and newer
  again. The 0007-compliance-stop-
  declaring-V4L2_CAP_VIDEO_M2M-capabil.patch carried in bug 1930208 is
  deliberately skipped to avoid bug 1946660.

  ========= original bug report ==========

  In addition to kernel/firmware proposed in bug 1921345, several user
  space daemons/libraries, as well as fixes/features in v4l2loopback-
  dkms, are also required to have seamless support for legacy/existing
  Linux kernel V4L2 API based applications to adopt libcamera or Intel
  libcamhal provided camera interfaces.

  The idea is to add a v4l2 streaming relay daemon (currently developed
  in [1] and packaged in [2]) that helps redirecting V4L2 buffer streams
  into v4l2loopback output device, and then legacy apps open
  v4l2loopback capture device in the ordinary way.

    hw -> libcamera/libcamhal -> v4l2-relayd -> v4l2loopback ->
  v4l2-based apps

  To achieve this, we'd like v4l2-relayd to only open GStreamer
  icamerasrc pipeline (provided by [3],[4]) when there is actual usage
  request from v4l2-based apps for privacy and power consumption's
  concerns. However, current (0.12.5 and therefore Ubuntu/Debian
  0.12.5-1) doesn't have any available mechanism for this, therefore a
  V4L2 Event API based proposal has been sent and accepted by
  upstream[5][6].

  This is needed for Ubuntu OEM projects with MIPI cameras through Intel
  IPU6 (Imaging Processing Unit version 6).

  [1]: https://gitlab.com/vicamo/v4l2-relayd
  [2]: https://code.launchpad.net/~oem-solutions-engineers/v4l2-relayd/+git/packaging
  [3]: https://github.com/intel/ipu6-camera-hal
  [4]: https://github.com/intel/icamerasrc
  [5]: https://github.com/umlaeute/v4l2loopback/pull/345
  [6]: https://github.com/umlaeute/v4l2loopback/pull/352

To manage notifications about this bug go to:
https://bugs.launchpad.net/hwe-next/+bug/1921474/+subscriptions