DRAFT: Guidance on CHERI-enabled allocator API guarantees and implementations - #51
DRAFT: Guidance on CHERI-enabled allocator API guarantees and implementations#51rwatson wants to merge 41 commits into
Conversation
gvnn3
left a comment
gvnn3
left a comment
There was a problem hiding this comment.
Well I at least think I understand it. I'd like to get this in soon so I can share it with an external consumer.
ae198d4 to
9f0552a
Compare
|
|
||
| The allocator must not return a capability that: | ||
|
|
||
| * Has the same integer address as the passed argument but different bounds |
There was a problem hiding this comment.
More generally, may not overlap the passed-in object. (It's equally bad to move the base down as it is to move the limit up). This probably deserves some expansion: we don't even allow returning the same object in the case of a realloc-to-be-smaller because we want to treat each realloc result as having its own lifetime, as if we'd moved the object, regardless of whether actually we moved the object or not.
But I think we actually don't want this constraint in a CHERI+MTE world: I think we can avoid moving the object so long as reallocation is within the original bounds (that is, we can make it smaller then take it back to its original size) (ncolors-1)/ncolors times by repainting, which is what we'd do if we were to move the object anyway.
There was a problem hiding this comment.
Part of the reason for adding this limitation was also about the fact that we define pointer comparison as considering only integer addresses, so if you return a pointer with different bounds, the caller cannot use an equality check to differentiate the two, and in common use will likely use the original pointer and not the one with modified bounds. Clearly we need to add some rationale notes, but .. does that change your view on the CHERI+MTE case?
|
|
||
| The allocator should: | ||
|
|
||
| * Fill reachable memory within bounds with zeroes before returning a pointer |
There was a problem hiding this comment.
I feel like this is really a non-CHERI-specific recommendation? I guess maybe more relevant to ensure you don't have stale capabilities?
There was a problem hiding this comment.
I wonder if we need to spell out more specifically that we are recommending allocator behaviours to also address uninitialised data aspects of temporal safety.
There was a problem hiding this comment.
I have added notes on this in new rationale text.
There was a problem hiding this comment.
@nwf, @qwattash: Do we want to draw out a specific benefit relating to capabilities flowing through uninitialised data from the perspective of use-after-free, etc? I think right now that's not a key part of the model as we revoke memory in quarantine not just out of quarantine, but that doesn't make it a generally good thing to do.
|
I have added new rationale text to the allocator guidance section to address some of the points here. This is a substantial addition to the pull request, and it would be appreciated if reviewers could read this new text as well! |
brooksdavis
left a comment
brooksdavis
left a comment
There was a problem hiding this comment.
I'm finding much of the discussion of padding to be redundant. Is there a particular misconception we're trying to address?
| | Is unsealed | Pointer must be dereferenceable for memory access on return | | ||
| | Has bounds that permit access to the full requested range of the allocation | Full allocation must be accessible for memory access on return | | ||
| | Has bounds that do not permit access to any other current allocation, nor to allocator metadata, implementing non-aliasing spatial safety | Allow access only to the memory allocation itself, and not either allocator-internal state (enabling a long history of heap corruption attacks or the leakage of privileged capabilities) or other allocations (violating non-aliasing requirements for spatial safety) | | ||
| | Has permissions that allow data load, data store, capability load, and capability store | C/C++ expect that memory will not only be accessible for read and write, but also that pointers can be held in arbitrary heap memory. While it is possible (and reasonable) to imagine heap allocators that return pointers without the ability to load or store capabilities, doing so with the general-purpose C/C++ allocators will break almost all extant software. | |
There was a problem hiding this comment.
We might want to relax this in a type-aware allocator world.
| | Has bounds that permit access to the full requested range of the allocation | Full allocation must be accessible for memory access on return | | ||
| | Has bounds that do not permit access to any other current allocation, nor to allocator metadata, implementing non-aliasing spatial safety | Allow access only to the memory allocation itself, and not either allocator-internal state (enabling a long history of heap corruption attacks or the leakage of privileged capabilities) or other allocations (violating non-aliasing requirements for spatial safety) | | ||
| | Has permissions that allow data load, data store, capability load, and capability store | C/C++ expect that memory will not only be accessible for read and write, but also that pointers can be held in arbitrary heap memory. While it is possible (and reasonable) to imagine heap allocators that return pointers without the ability to load or store capabilities, doing so with the general-purpose C/C++ allocators will break almost all extant software. | | ||
| | Be sufficiently aligned to allow capability loads and stores at relative offset 0 from the returned pointer | As with capability load and store positions, being unable to use pointers stored at pointer alignment within an alllocation will break almost all extant software. | |
There was a problem hiding this comment.
As an incidental feature of revocation we don't allow < 16-byte allocations. If we did, this requirement wouldn't hold. I'm not sure if we want to point that out.
There was a problem hiding this comment.
I don't think we need to require this for allocations < cap_size. You could have an allocator that has a bucket just for integers allocations. Maybe qualify the requirement with "for allocations >= cap_size"?
Also I believe there is a typo here? positions -> permissions?
There was a problem hiding this comment.
@arichardson: Typo corrected, thanks; will respond to the remainder of this thread separately.
There was a problem hiding this comment.
I've added a note about capability alignment being required only for allocations of at least pointer size.
There was a problem hiding this comment.
Probably this should in fact be capability size.
There was a problem hiding this comment.
I wonder if there's an interesting implication for realloc() here: Should realloc() always ensure at least pointer alignment if reallocation is requested to go from below to at or above capability size, meaning that it might need to allocate / copy memory? I think this may be implied currently, but perhaps should be stated explicitly... if we think it's useful / important.
There was a problem hiding this comment.
You could restate the requirement on malloc that all allocations that could hold a capability be capability aligned. It's required that memory allocated by realloc be interchangeable with memory allocated by malloc so it's IMO a pretty obvious requirement.
| | Requirement | Rationale | | ||
| |-------------|-----------| | ||
| | Implement pointer revocation by clearing the tag of capabilities pointing to freed memory | Clearing the tag ensures non-dereferenceability when a pointer is revoked, althugh this can also be achieved by other means such as changing page-table permissions on the page a pointer points to, on supporting architectures. | | ||
| | On virtual-memory-enabled systems, unmap reachable memory within the bounds of the allocation after it has been freed | This may be useful both for rapidly detecting use-after-free errors and also to allow the allocator to reuse physical memory while deferring revocation of capabilities to specific virtual addresses. | |
There was a problem hiding this comment.
I wonder if we want to include page permissions here. I worry that unmapped doesn't quite describe the state we want (backing store freed with fault on access). I think one could achieve the same effect with MADV_FREE and mprotect(PROT_NONE).
|
|
||
| | Requirement | Rationale | | ||
| |-------------|-----------| | ||
| | Return a new pointer from `realloc()` that has an identical address to the passed argument but differs in its bounds or other metadata | This requirement arises from the observation that callers to `realloc()` will immediately compare the passed pointer to the returned one to establish whether the memory was reallocated or not. If it was, it is frequently the case that they will use the passed pointer rather than the returned one, since they are assumed to be interchangeable. With CHERI C/C++, pointer equality testing with the `==` operator compares only the address and not bounds or other metadata, and as a result a pointer with the prior bounds, rather than the new bounds, may frequently be used. Rather than allowing this situation to occur, we require reallocation. This tradeoff point appears to maximise compatibility at the cost of performance and memory overhead. It may be that this requirement is weakened to "should not" in the future. | |
There was a problem hiding this comment.
We should probably make it explicit that this is catering to de facto C/C++ and that such program are relying on UB.
There was a problem hiding this comment.
Please feel free to suggest wording.
| The allocator must: | ||
|
|
||
| * Pad below and above allocations such that, when precise bounds are not | ||
| utilized, no other allocation is accessible within returned bounds. |
There was a problem hiding this comment.
In this context this seems redundant to the bounds above. Even dlmalloc-style or bump-the-pointer allocators pad so it's not that interesting.
| The allocator must select one of the following two choices for each memory allocation; different memory allocations may use different strategies (e.g., enabling precise bounds for smaller allocations, or imprecise ones for larger allocations, as well as other strategies): | ||
|
|
||
| | Requirement | Rationale | | ||
| |-------------|-----------| | ||
| | Provide precise bounds -- i.e., in which the lower bound is the lowest address of the returned allocation, and the upper bound is the highest address of the returned allocation plus one | Implement precise bounds, where achievable whtin the constraints of bounds imprecision, avoiding the use of padding while still allowing access to the full allocation. | |
There was a problem hiding this comment.
This seems like it is just repeating the section above? Maybe better have this as a separate "ensuring non-overlapping bounds" section rather than two tables?
…ardson and @brooksdavis. Co-authored-by: Alexander Richardson <Alexander.Richardson@cl.cam.ac.uk>
rwatson
changed the title
5 participants
This is a very early draft of requirements / guidance for memory-allocator APIs, for discussion / feedback / improvement.
It might be that we want to broaden it a little bit to also mention stack allocation using
alloca(), and to add some specific words that better enable garbage collection / allow GC-enabled heap allocators to conform. And, more generally, to consider allocators that are not the C standard allocator.