What vOSTD adds on top

This chapter is about what vOSTD must do for the kernel proper that Linux does not, now that the last chapter has put the tenant’s page table in Linux’s hands: read the table, copy to and from the tenant’s memory, keep track of which frame each entry holds, wait for copies in progress before a frame is freed, lock a range for the kernel proper’s cursor, hold the one kind of page Linux cannot represent, dispose of an address space at a moment when sleeping is not allowed, and say when a service may sleep under a guard. Each of these is a few dozen lines. The walk, the copy, the frame count, the grace period and the withheld set are what keep the design sound; the rest keep the kernel proper correct or the host responsive.

Reading the table

vOSTD answers a cursor’s questions by reading Linux’s page table itself, through the direct map, from the root whose physical address the space’s record holds. The kernel proper asks a cursor what is at an address (query), where the next mapped page is (find_next), and how many pages an unmap removed; on a machine OSTD reads its own page table for these. It is the walk a processor does, in software, four dependent loads:

Simplified from prototype/vostd/src/mm/page_table.rs: read_entry, child, leaf_of, walk.

fn read_entry(node: Paddr, idx: usize) -> u64 {
    unsafe { read_volatile((paddr_to_vaddr(node) as *const u64).add(idx)) }   // one 64-bit load
}

fn child(e: u64) -> Option<Paddr> {                       // the next table down, if the entry points at one
    if e & PRESENT == 0 || e & HUGE != 0 { None } else { Some((e & ADDR_MASK) as Paddr) }
}

fn leaf_of(e: u64) -> Option<Leaf> {                      // a present, user-accessible page
    if e & PRESENT == 0 || e & USER == 0 { return None; }
    let mut flags = PageFlags::R;
    if e & WRITE != 0      { flags |= PageFlags::W; }
    if e & NO_EXECUTE == 0 { flags |= PageFlags::X; }
    Some(Leaf { paddr: (e & ADDR_MASK) as Paddr, flags })
}

pub(crate) fn walk(root: Paddr, va: Vaddr) -> Option<Leaf> {
    let mut node = root;
    for level in (2..=4).rev() {                          // four levels; vOSTD refuses a five-level host at start
        node = child(read_entry(node, index(va, level)))?;
    }
    leaf_of(read_entry(node, index(va, 1)))
}

The walk takes no lock, and two facts about Linux make that safe. First, a page the walk reads is never freed under it. Second, Linux writes an entry with a single 64-bit store and clears it with an atomic exchange, so a load never sees half an entry.

Linux frees an area’s page-table pages in four cases, and none applies here. (1) When an area is unmapped or moved, which only a system call to Linux does; the tenant’s system calls go to the kernel proper, not to Linux. (2) At exec, which never runs in this address space. (3) When khugepaged, Linux’s background thread that replaces a last-level table of 4 KiB entries with one 2 MiB huge entry, collapses a range, which it refuses for raw-frame-number areas. (4) In exit_mmap(), which the endovisor’s reference on the address space (the last chapter’s second reference) holds off. zap_vma_ptes() clears entries but leaves the tables.

The walk need not handle huge entries. The area holds only 4 KiB entries, since vm_map inserts one page at a time, so child treats a huge entry as absent.

Permissions are taken from the leaf only, never combined down the levels. Linux makes every upper-level user entry readable, writable and user-accessible, so only the leaf restricts anything, with one exception: the no-execute bit on the kernel-mode table’s top-level user entries, which vOSTD reads. Combined down the levels, that bit would make every tenant page non-executable. The prototype’s Linux is built without the mitigation, so this case is argued, not tested.

Copying to and from the tenant

Inside Linux, vOSTD may not dereference a tenant address as OSTD does on a machine. On a machine OSTD dereferences the address and lets the fault handler catch a bad one; for example, the hundred-line kernel’s write handler reads the program’s buffer through vm_space.reader(addr, len). vOSTD could open an SMAP window and could list an assembly copy in an exception table, but two deeper obstacles remain. First, the address space the processor is in is not necessarily the one being copied, because a carrier adopts its virtual CPU’s space only on the way to user mode. Second, a kernel-mode fault in the area never reaches the kernel proper: the endovisor’s fault handler fails it (the last chapter’s -EFAULT), and Linux then resumes at the instruction’s exception-table fixup or, for an instruction with none, oopses the host, so the copy would fail where on a machine the kernel proper’s handler would have mapped the page.

Instead, a copy walks the page table from the tenant address to the frame, and copies through the frame’s direct-map alias, an ordinary kernel address.

On a bad address, vOSTD returns the page-fault error at once, where OSTD would first call the kernel proper’s own page-fault handler and retry. An unmapped or read-only address is a result of the walk, not a fault.

Returning the error is enough for kernels that map eagerly, not for one that maps on demand. Asterinas’s full kernel proper maps on demand, so before it can run as a kernelet vOSTD must call its handler outside the read section described below and walk again. This prototype does not do that.

Simplified from prototype/vostd/src/mm/io.rs: Region, resolve, copy_regions, copy_opaque.

enum Region {
    Kernel { cursor: usize, end: usize },                  // a kernel buffer
    User { root: Paddr, cursor: Vaddr, end: Vaddr },       // tenant memory, to be walked through `root`
}

impl Region {
    fn resolve(&self, write: bool) -> Option<(*mut u8, usize)> {   // the head of the region, as a kernel pointer
        match *self {
            Region::Kernel { cursor, end } => Some((cursor as *mut u8, end - cursor)),
            Region::User { root, cursor, end } => {
                let leaf = page_table::walk(root, cursor & !(PAGE_SIZE - 1))?;    // None: a page fault
                if write && !leaf.flags.contains(PageFlags::W) { return None; }
                let offset = cursor % PAGE_SIZE;
                let len = (PAGE_SIZE - offset).min(end - cursor);               // up to the end of this page
                Some(((paddr_to_vaddr(leaf.paddr) + offset) as *mut u8, len))
            }
        }
    }
}

fn copy_regions(dst: &mut Region, src: &mut Region) -> (usize, bool) {
    while copied < total {
        let done = rcu::read(|| {                                      // run inside a read section (below)
            let (dptr, dlen) = dst.resolve(true)?;
            let (sptr, slen) = src.resolve(false)?;
            let n = dlen.min(slen).min(total - copied);
            unsafe { copy_opaque(dptr, sptr, n) };
            Some(n)
        });
        let Some(n) = done else { return (copied, false) };            // a fault: how far we got
        dst.advance(n); src.advance(n); copied += n;
    }
    (copied, true)
}

unsafe fn copy_opaque(dst: *mut u8, src: *const u8, n: usize) {
    asm!("rep movsb", inout("rcx") n => _, inout("rdi") dst => _, inout("rsi") src => _);
}

The copy is one rep movsb in assembly rather than a Rust copy, so that the compiler cannot assume tenant memory holds still. Tenant memory may change under the copy, since another virtual CPU’s thread may be writing it; a plain copy of memory that changes is a data race in Rust’s terms, and the optimizer is then free to assume the source did not change. It could, for instance, compile “copy a length out, check it, use it” into a check of the copy and a re-read of the source for the use, and a tenant that changed the length in between would have passed the check with a value it never had. An assembly copy is opaque to the compiler, so the code after the copy can read only the destination, never the source again.

Who holds the frame

Inside Linux, vOSTD counts each Linux entry as holding the reference that OSTD’s page table holds on a machine; Linux knows nothing of the count. A full kernel proper such as Asterinas’s relies on it: its copy-on-write path reuses a frame in place when the count is one.

The reference changes hands with the entry. map turns the handle into a raw address after the service succeeds, so the reference passes to the entry only once the entry exists. unmap reads the entries back before the service clears them. It turns each address into a handle again and drops it after the wait described below.

Simplified from prototype/vostd/src/mm/vm_space.rs and prototype/vostd/src/mm/frame.rs: CursorMut::map, CursorMut::unmap, Frame::into_raw, Frame::from_raw.

pub fn map(&mut self, frame: UFrame, prop: PageProperty) {
    let va = self.core.va;
    /* ... a page without read permission goes to the withheld set, below ... */
    space.backend().map(va, frame.paddr(), prot_bits(&prop));   // the vm_map service; panics the kernelet on failure
    let _ = frame.into_raw();                                    // the entry owns the reference now
    self.core.va += PAGE_SIZE;
}

pub fn unmap(&mut self, len: usize) -> usize {
    let mut frames: Vec<Paddr> = Vec::new();
    page_table::for_each_present(space.root(), start, end, &mut |_, leaf| frames.push(leaf.paddr));  // read first
    if !frames.is_empty() {
        space.backend().unmap(start, len);                       // the vm_unmap service: clears and flushes
    }
    /* ... plus the withheld set's pages in the range ... */
    rcu::synchronize();                                          // every copy in progress is done
    let n = frames.len();
    for p in frames { drop(UFrame::from_raw(p)); }               // now the frames may go back
    self.flusher.dispatch_tlb_flush();                           // issue flushes the kernel proper queued earlier on this cursor; this range was flushed by the zap
    n
}

pub(crate) fn into_raw(self) -> Paddr { let p = self.paddr; core::mem::forget(self); p }

pub(crate) fn from_raw(paddr: Paddr) -> Self {
    assert!(allocator::refcount(paddr).unwrap() > 0);          // the frame is the grant's and in use
    Self::from_raw_unchecked(paddr)
}

This is sound only if Linux never clears an entry of the area on its own initiative, and for this area it does not. The entries are marked special, so Linux finds no page of its own behind them. Every scanner that walks address spaces skips the area by its flags: the multi-generational LRU walk, khugepaged, same-page merging, NUMA balancing, the out-of-memory reaper. Reclaim finds pages to evict through its LRU lists and the entries that map a page through its reverse mapping; the grant’s frames are ordinary RAM on no LRU list, and no reverse mapping leads from a frame to this area, so swap-out never sees them. Compaction, which moves pages to make free memory contiguous, does pass over them and skips them as neither on an LRU list nor movable.

Memory pressure leaves the area’s entries alone: make mem-pressure tries to drive the Linux under test into swap with a host process while a tenant has 512 pages mapped, and afterwards every entry is present with the same frame and the same contents. The log reports how much was swapped; the check does not require it.

Freeing a frame a copy may be reading

unmap waits for every copy in progress before it frees a frame. A copy takes no cursor, since that would serialize every copy against every cursor, so it may still be reading the frame whose entry the unmap just cleared. The wait is a minimal RCU (read-copy-update: readers take no lock, and a writer waits until every reader in progress has finished before it frees anything): the rcu::read around each page of a copy, and the rcu::synchronize() in unmap before the frames are freed, which waits a grace period, until every reader in progress has finished.

synchronize waits for every virtual CPU that was inside a read section when it was called. Each virtual CPU has a sequence word, odd while it is inside one. synchronize skips its own virtual CPU: its caller is not in a read section, and since a read section holds a preemption guard, no other task on this virtual CPU can have been switched out partway through one.

The wait is short, one page’s walk and copy, unless Linux has descheduled the reader’s carrier; then the waiter spins, telling the host through vcpu_on_spin, until that carrier runs again.

The grace period holds in the test: make mem-conc runs copies without pause on one virtual CPU against map-and-unmap cycles on the other and finds no torn page, that is, no copy still reading a frame when the test rewrote it after its unmap returned (the rewrite stands in for reuse). A second phase frees the frames for real while copies write into the space, reallocates them, and finds none overwritten; in this run no grace period of that phase found a copy to wait for.

The range lock

vOSTD keeps a range lock of its own per space, because no Linux lock can stand in for a cursor’s locks. On a machine, a cursor holds the page table’s node locks for its whole life, and a full kernel proper such as Asterinas’s relies on that in two places: its page-fault handler queries, allocates and maps under one cursor, so two threads faulting on one page never both map it; and its page cache commits a page and maps it atomically under one cursor while its eviction and truncation unmap through another. A Linux lock cannot stay held while the carrier is back in kernelet code, where a tenant-controlled kernel could hold it forever against Linux’s own users of the address space.

The lock is sixteen 64-bit words in the space’s shared record. Each 2 MiB region, the span of one last-level table, is guarded by word (region number mod 16), so sixteen words stand for every region: a striped lock. Each word holds a count of readers in its low bits (READERS is their mask) and three flags: WRITER, set by an exclusive holder; WRITER_WAITING, which holds new readers off while a writer waits; and WAITERS, which says that someone is asleep in vm_wait on this word.

A cursor takes every word its range touches, exclusively for a cursor_mut and shared for a cursor. It takes them in ascending order, a total order, so two cursors cannot deadlock. It releases them when it is dropped. Uncontended, taking a word is one compare-and-swap on the shared record and no crossing.

Simplified from prototype/vostd/src/mm/vm_space.rs: lock_word, unlock_word.

fn lock_word(&self, idx: usize, exclusive: bool) {
    let w = &self.backend().words()[idx];
    loop {
        let cur = w.load(Ordering::Acquire);
        let free = if exclusive { cur & (WRITER | READERS) == 0 } else { cur & (WRITER | WRITER_WAITING) == 0 };
        if free {
            let new = if exclusive { (cur | WRITER) & !WRITER_WAITING } else { cur + 1 };
            if w.compare_exchange(cur, new, Ordering::Acquire, Ordering::Relaxed).is_ok() { return; }
            continue;
        }
        // Contended: say so in the word, then sleep in the host until it changes. Nothing spins.
        let want = cur | WAITERS | if exclusive { WRITER_WAITING } else { 0 };
        if want != cur && w.compare_exchange(cur, want, Ordering::Relaxed, Ordering::Relaxed).is_err() { continue; }
        self.backend().wait(idx, want);                       // the vm_wait service
    }
}

fn unlock_word(&self, idx: usize, exclusive: bool) {
    let (old, last) = if exclusive { (w.fetch_and(!(WRITER | WAITERS), Ordering::Release), true) }
                      else { /* decrement the readers, clearing WAITERS with the last; last: was this the last reader? */ };
    if old & WAITERS != 0 && last { self.backend().wake(idx); }   // the vm_wake service
}

A contended cursor sleeps rather than spins, because the lock’s holder may itself be asleep in Linux inside a vm_map. The waiter’s whole virtual CPU sleeps with it, which costs no more than spinning on a machine’s page-table lock would.

The slow path is a futex, Linux’s “sleep only if this word still holds this value” primitive. It is rebuilt for carriers because Linux’s futex keys on a user address in the caller’s address space and this word lives in kernel memory. A waiter sets the WAITERS bit and calls vm_wait(space, word, expected); the endovisor takes the space’s wait-queue lock, reads the word, returns at once if it no longer equals expected, and otherwise puts the carrier to sleep, killably. A releasing cursor clears the bit with one atomic read-modify-write and, if it was set and no reader remains, calls vm_wake, which wakes every waiter under the same lock.

No wake-up can be lost, by the argument chapter 9 made for vcpu_idle: one side marks, then checks; the other changes, then checks the mark; so one of the two always sees the other. Here the mark is the WAITERS bit, the check is the read under the lock, and the change is the single read-modify-write on release.

A cursor that waits behind a sleeping lock holder sleeps too, and takes the lock after the release without a single kernelet task switch: make mem-conc has one virtual CPU hold a cursor while its vm_map sleeps (a test hook makes it), and the other virtual CPU’s cursor on the same region sleeps in vm_wait and takes the lock after the release.

A page without read permission

A page without read permission has no Linux entry; vOSTD keeps it in a per-space withheld set instead. Such pages are of two kinds: PROT_NONE pages, with every permission cleared, and write-only pages, which x86 cannot express because a present entry is always readable. To the kernel proper both are mapped; on a machine OSTD encodes them as valid but not-present entries.

Linux has no usable encoding for them in this area. On a Linux built with NUMA balancing, Linux periodically makes entries no-access, so that the next touch faults and tells it which processor uses the page. Its fault handler therefore treats every no-access entry in an accessible area as such a marker, even in an area the balancer never scans, and rewrites it with the area’s page protection: read and execute here, because the area’s pfn_mkwrite makes Linux hold write back, plus write if the entry is dirty. A page encoded that way would become readable on its first touch. Without NUMA balancing, as in the prototype’s Linux, the entry is no better: Linux takes it for present and a read fault on it for spurious, so the access faults forever without reaching the area’s fault handler.

The set stands in for the table wherever a page has no entry. It is an ordered map from address to frame and property, in the kernelet’s heap (Linux’s kmalloc, charged to the sandbox). It holds the frame’s reference exactly as an entry would. It is under the range lock, which orders cursors on one region, and a spin lock of its own, because cursors on different regions share one map; the lock is taken per statement, never across a service. query, find_next, protect_next and unmap consult the set beside the table:

Simplified from prototype/vostd/src/mm/vm_space.rs: Slot, slot, protect_next.

enum Slot { Table(page_table::Leaf), Withheld(Paddr, PageProperty) }

fn slot(&self, va: Vaddr) -> Option<Slot> {
    if let Some(leaf) = page_table::walk(self.space.root(), va) { return Some(Slot::Table(leaf)); }
    self.space.withheld.lock().get(&va).map(|&(p, prop)| Slot::Withheld(p, prop))
}

pub fn protect_next(&mut self, len: usize, mut op: impl FnMut(&mut PageFlags, &mut CachePolicy)) -> Option<Range<Vaddr>> {
    let va = self.core.next_mapped(self.core.va, end)?;        // the table merged with the set
    let (paddr, old) = /* from the slot */;
    let mut new = old; op(&mut new.flags, &mut new.cache);
    match (in_table, new.flags.contains(PageFlags::R)) {
        (true, true)   => if rwx(new.flags) != rwx(old.flags) { backend.protect(va, prot_bits(&new)) },  // vm_protect
        (true, false)  => { backend.unmap(va, PAGE_SIZE); withheld.lock().insert(va, (paddr, new)); }     // into the set
        (false, true)  => { withheld.lock().remove(&va); backend.map(va, paddr, prot_bits(&new)); }       // out of the set
        (false, false) => { withheld.lock().insert(va, (paddr, new)); }
    }
    Some(va..va + PAGE_SIZE)
}

A tenant access to a withheld page faults as an absent page does, and the kernel proper’s own permission check turns that into a SIGSEGV for the program, as on a machine. make mem-func moves 512 pages into the set with protect_next clearing all three permissions, as a kernel proper’s mprotect(PROT_NONE) would, checks that a read of one is refused, moves them back, and reads them all intact with no fault.

A space that is dropped at the wrong moment

The last reference to a VmSpace can be dropped with virtual interrupts off, where Drop may do nothing that may sleep, yet vm_destroy sleeps. The drop can come from activate, which keeps one reference per virtual CPU to the space active there and drops the previous one’s when a new space is activated. A full kernel proper, such as Asterinas’s, activates with virtual interrupts off, from the handler it runs right after a task switch, so that drop can be the last; the test kernels of this book activate from main, but vOSTD is not written for them alone.

A helper task to do the work later is not available. A task spawned before the kernel proper calls inject_scheduler would make OSTD install its default scheduler, and the injection would then fail. A task spawned afterwards is never run by a kernel proper like Asterinas’s, whose scheduler enqueues only its own threads; a kernel proper that did accept it would still schedule it by its own policy, at no moment vOSTD can choose, so a busy higher-priority task could hold the frames and the Linux address space indefinitely.

Drop pushes the space onto a per-kernelet retire list, lock-free and without allocating, since an allocation is mem_alloc, which may sleep (the list node was allocated at new).

The list is drained at the next point where sleeping is allowed: every sleeping memory service’s wrapper (vm_create, vm_map, vm_unmap, vm_protect, vm_wait, vm_destroy) and vcpu_idle, whenever the caller holds no guard and no spin lock, has virtual interrupts on and is not in an upcall. A guard is excluded although a memory service may sleep under one (rule 1, below), because the drain may take long and the caller did not ask for it.

The drain is not prompt, but it is bounded. A sandbox that never idles keeps its retired spaces, but vm_create is a drain point, so a new space never waits behind retired ones, and the backlog is at most the 64 spaces a kernelet may hold, all charged to the sandbox’s own grant and to page tables Linux already counts.

The drain unmaps the whole space, which flushes and releases every frame, the withheld set’s included, and then calls vm_destroy.

Simplified from prototype/vostd/src/mm/vm_space.rs: Drop, retire::drain_if_safe, retire::destroy.

impl Drop for VmSpace {
    fn drop(&mut self) {                                   // may run with interrupts off: push, nothing else
        node.backend = self.backend.take();
        node.withheld = core::mem::take(&mut *self.withheld.lock());
        retire::push(node);
    }
}

pub(crate) fn drain_if_safe() {                            // called from sleeping stubs and vcpu_idle
    if HEAD.load(Ordering::Relaxed).is_null() || !safe() { return; }   // safe: no guard, no spin lock, interrupts on, not in an upcall
    let mut p = HEAD.swap(core::ptr::null_mut(), Ordering::Acquire);
    while !p.is_null() { let node = unsafe { Box::from_raw(p) }; p = node.next; destroy(*node); }
}

fn destroy(mut node: Node) {
    let backend = node.backend.take().unwrap();
    page_table::for_each_present(backend.root(), 0, MAX_USERSPACE_VADDR, &mut |_, leaf| frames.push(leaf.paddr));
    backend.unmap(0, MAX_USERSPACE_VADDR);
    frames.extend(node.withheld.values().map(|&(p, _)| p));
    rcu::synchronize();
    for p in frames { drop(UFrame::from_raw(p)); }
    backend.destroy();                                     // vm_destroy
}

The rule about sleeping

Inside Linux, a memory service may sleep even when the kernel proper calls it under a preemption guard, which OSTD forbids on a machine. The kernel proper calls map holding a preemption guard, because every cursor takes one. Every memory service except vm_flush, vm_activate and vm_wake may sleep in Linux: vm_map, vm_unmap and vm_protect wait for the address-space lock; vm_map may wait for a page-table page; vm_unmap’s zap may be preempted between page-table pages; and vm_wait sleeps by design. The design states the rule in four parts:

1. A service may sleep whenever its work needs it. Its prologue lifts the mirror of chapter 11, so Linux sees nothing wrong, and its epilogue restores it.
2. A service that may sleep must not be called while the caller holds a kernelet spin lock or has virtual interrupts off; under a plain preemption guard it may. The reason for the spin lock is lock-holder preemption: a spin lock held across a Linux sleep makes every other virtual CPU that wants it spin for the sleep’s length. The reason for the interrupts is a promise: with interrupts off the kernel proper is promised that no handler runs before it unmasks, so a kick sent during the sleep would be delayed past the carrier’s return until the unmask, where rule 3’s delay ends with the return.
3. A sleep delays this virtual CPU’s pending virtual interrupts until the carrier returns, exactly as a Linux preemption does; the design accepts that.
4. The mirror remains a latency optimization and is load-bearing for correctness nowhere.

Rule 1 is sound because a kernelet’s critical section promises only that no kernelet task switch happens on this virtual CPU until the guard drops, as Problem 2 of chapter 11 argued. A sleep in Linux is the same event as a Linux preemption, which the kernelet already survives at any instruction. On a machine a guard has a second job: OSTD frees page-table pages only after every processor has switched tasks, so a guard keeps them alive. Here no table is freed, and vOSTD’s grace period watches each virtual CPU’s sequence word rather than task switches, so a guard held across a Linux sleep changes nothing.

Rule 2 is the one place where vOSTD asks more of the kernel proper than OSTD does: a kernel proper that maps under a spin lock or with virtual interrupts off runs on a machine and is stopped here.

vOSTD asserts the rule before every sleeping memory service, with a count of spin locks held that is separate from the guard count, and stops the kernelet rather than counting violations, so a kernel proper under the tenant’s control cannot break the rule through the memory services; only the sandbox’s own virtual CPUs would spin, within its own share. Whether Asterinas’s full kernel proper keeps the rule everywhere has not been checked.

The heap services break rule 2, a known gap that costs latency, not soundness. They may sleep inside kmalloc, carry no assertion, and are called under spin locks, the withheld set’s insert among them. By rule 4 that costs latency only: a virtual CPU that wants the lock spins, and hands its turn on through vcpu_on_spin, for as long as kmalloc sleeps.

Simplified from prototype/vostd/src/mm/backend.rs and prototype/vostd/src/sync/spin.rs: sleeping, Backend::map, assert_may_sleep.

fn sleeping(name: &str) {                                  // every sleeping memory service's stub begins here
    r2::assert_may_sleep(name);                            // rule 2
    retire::drain_if_safe();                               // and a drain point for the retire list
}

pub(crate) fn map(&self, va: Vaddr, paddr: Paddr, prot: u32) {
    sleeping("vm_map");
    let r = unsafe { (svc().vm_map)(self.id, va as u64, paddr as u64, prot) };
    if r < 0 { map_failed(va, paddr, r); }                 // as on a machine: a page-table node it cannot get is a panic
}

pub(crate) fn assert_may_sleep(service: &str) {
    if held() != 0 || irq_is_off() {
        panic!("R2 violated: the sleeping service {} was called holding {} kernelet spin lock(s)", service, held());   // R2 is the code's name for rule 2
    }
}

Two tests exercise the rule. make mem-func runs on a Linux built with CONFIG_DEBUG_ATOMIC_SLEEP, which complains whenever Linux code might sleep with Linux’s preemption count raised. Inside a preemption guard it maps a page into a fresh 2 MiB region, which makes Linux allocate a page-table page, and Linux reports nothing. make mem-r2 does the same under a spin lock, and vOSTD stops the kernelet before the call crosses.