[PULL v2][Noble] backport arm64 THP improvements from 6.9

[dann frazier]

BugLink: https://bugs.launchpad.net/bugs/2059316

These are all clean cherry-picks from upstream, save one that required
minor backporting due to an API change. This does touch generic code.
I've regression tested as much as I've had time to given I just
learned about this patchset last week. But with the performance
improvements I've measured, and the GPU workload improvements reported
upstream, I think this is something we need to seriously consider
for our next LTS.

I plan to monitor upstream for Fixes commits during the 6.9
development cycle, in case regressions are found/fixed.

Changes from v1:
 * Relabeled as SAUCE

The following changes since commit 4427c45609f6faf3b5b15e9e9246caa87894c36f:

  UBUNTU: Ubuntu-6.8.0-20.20 (2024-03-18 11:08:14 +0100)

are available in the Git repository at:

  git://git.launchpad.net/~dannf/ubuntu/+source/linux/+git/linux noble-mthp-sauce

for you to fetch changes up to 9ddb0a4d82ff0cf89be70d20c26ebfab83b9d3b0:

  UBUNTU: SAUCE: arm64/mm: improve comment in contpte_ptep_get_lockless() (2024-04-03 06:55:48 -0600)

----------------------------------------------------------------
David Hildenbrand (14):
      UBUNTU: SAUCE: arm/pgtable: define PFN_PTE_SHIFT
      UBUNTU: SAUCE: nios2/pgtable: define PFN_PTE_SHIFT
      UBUNTU: SAUCE: powerpc/pgtable: define PFN_PTE_SHIFT
      UBUNTU: SAUCE: riscv/pgtable: define PFN_PTE_SHIFT
      UBUNTU: SAUCE: s390/pgtable: define PFN_PTE_SHIFT
      UBUNTU: SAUCE: sparc/pgtable: define PFN_PTE_SHIFT
      UBUNTU: SAUCE: mm/pgtable: make pte_next_pfn() independent of set_ptes()
      UBUNTU: SAUCE: arm/mm: use pte_next_pfn() in set_ptes()
      UBUNTU: SAUCE: powerpc/mm: use pte_next_pfn() in set_ptes()
      UBUNTU: SAUCE: mm/memory: factor out copying the actual PTE in copy_present_pte()
      UBUNTU: SAUCE: mm/memory: pass PTE to copy_present_pte()
      UBUNTU: SAUCE: mm/memory: optimize fork() with PTE-mapped THP
      UBUNTU: SAUCE: mm/memory: ignore dirty/accessed/soft-dirty bits in folio_pte_batch()
      UBUNTU: SAUCE: mm/memory: ignore writable bit in folio_pte_batch()

Ryan Roberts (21):
      UBUNTU: SAUCE: arm64/mm: make set_ptes() robust when OAs cross 48-bit boundary
      UBUNTU: SAUCE: mm: clarify the spec for set_ptes()
      UBUNTU: SAUCE: mm: thp: batch-collapse PMD with set_ptes()
      UBUNTU: SAUCE: mm: introduce pte_advance_pfn() and use for pte_next_pfn()
      UBUNTU: SAUCE: arm64/mm: convert pte_next_pfn() to pte_advance_pfn()
      UBUNTU: SAUCE: x86/mm: convert pte_next_pfn() to pte_advance_pfn()
      UBUNTU: SAUCE: mm: tidy up pte_next_pfn() definition
      UBUNTU: SAUCE: arm64/mm: convert READ_ONCE(*ptep) to ptep_get(ptep)
      UBUNTU: SAUCE: arm64/mm: convert set_pte_at() to set_ptes(..., 1)
      UBUNTU: SAUCE: arm64/mm: convert ptep_clear() to ptep_get_and_clear()
      UBUNTU: SAUCE: arm64/mm: new ptep layer to manage contig bit
      UBUNTU: SAUCE: arm64/mm: dplit __flush_tlb_range() to elide trailing DSB
      UBUNTU: SAUCE: arm64/mm: wire up PTE_CONT for user mappings
      UBUNTU: SAUCE: arm64/mm: implement new wrprotect_ptes() batch API
      UBUNTU: SAUCE: arm64/mm: implement new [get_and_]clear_full_ptes() batch APIs
      UBUNTU: SAUCE: mm: add pte_batch_hint() to reduce scanning in folio_pte_batch()
      UBUNTU: SAUCE: arm64/mm: implement pte_batch_hint()
      UBUNTU: SAUCE: arm64/mm: __always_inline to improve fork() perf
      UBUNTU: SAUCE: arm64/mm: automatically fold contpte mappings
      UBUNTU: SAUCE: arm64/mm: export contpte symbols only to GPL users
      UBUNTU: SAUCE: arm64/mm: improve comment in contpte_ptep_get_lockless()

dann frazier (1):
      UBUNTU: [Config] arm64: ARM64_CONTPTE=y

 arch/arm/include/asm/pgtable.h      |   2 +
 arch/arm/mm/mmu.c                   |   2 +-
 arch/arm64/Kconfig                  |   9 +
 arch/arm64/include/asm/pgtable.h    | 431 +++++++++++++++++++++++++++++++-----
 arch/arm64/include/asm/tlbflush.h   |  13 +-
 arch/arm64/kernel/efi.c             |   4 +-
 arch/arm64/kernel/mte.c             |   2 +-
 arch/arm64/kvm/guest.c              |   2 +-
 arch/arm64/mm/Makefile              |   1 +
 arch/arm64/mm/contpte.c             | 408 ++++++++++++++++++++++++++++++++++
 arch/arm64/mm/fault.c               |  12 +-
 arch/arm64/mm/fixmap.c              |   4 +-
 arch/arm64/mm/hugetlbpage.c         |  40 ++--
 arch/arm64/mm/kasan_init.c          |   6 +-
 arch/arm64/mm/mmu.c                 |  16 +-
 arch/arm64/mm/pageattr.c            |   6 +-
 arch/arm64/mm/trans_pgd.c           |   6 +-
 arch/nios2/include/asm/pgtable.h    |   2 +
 arch/powerpc/include/asm/pgtable.h  |   2 +
 arch/powerpc/mm/pgtable.c           |   5 +-
 arch/riscv/include/asm/pgtable.h    |   2 +
 arch/s390/include/asm/pgtable.h     |   2 +
 arch/sparc/include/asm/pgtable_64.h |   2 +
 arch/x86/include/asm/pgtable.h      |   8 +-
 debian.master/config/annotations    |   3 +
 include/linux/efi.h                 |   5 +
 include/linux/pgtable.h             |  65 +++++-
 mm/huge_memory.c                    |  58 ++---
 mm/memory.c                         | 219 ++++++++++++++----
 29 files changed, 1151 insertions(+), 186 deletions(-)
 create mode 100644 arch/arm64/mm/contpte.c