Add "trailing elements" module with facilities for tail-allocated storage #513

DougGregor · 2025-09-05T23:27:12Z

Description

Introduce new types TrailingArray and TrailingPadding that consist of a fixed-sized header followed by variable-sized storage. The design of these types is inspired by the existing ManagedBuffer type in the standard library, but has been adjusted both to make them non-copyable and to eliminate the requirement for heap allocation.

Detailed Design

It's several new types. Here's a sketch of the API for the primary type, IntrusiveManagedBuffer:

public struct TrailingArray<Header: TrailingElements>: ~Copyable
    where Header: ~Copyable
{
    /// Allocate an intrusive managed buffer with the given header and calling
    /// the initializer to fill in the trailing elements.
    public init<E>(
        header: consuming Header,
        initializingTrailingElementsWith initializer: (inout OutputSpan<Element>) throws(E) -> Void
    ) throws(E)

    /// Allocate an intrusive managed buffer with the given header and
    /// initializing each trailing element with the given `element`.
    public init(header: consuming Header, repeating element: Element)

    /// Take ownership over a pointer to memory containing the header followed
    /// by the trailing elements.
    ///
    /// Once this managed buffer instance is no longer used, this memory will
    /// be freed.
    public init(consuming pointer: UnsafeMutablePointer<Header>)

    /// Return the pointer to the underlying memory, including ownership over
    /// that memory. The underlying storage will not be freed by this buffer;
    /// it is the responsibility of the caller.
    public consuming func takePointer() -> UnsafeMutablePointer<Header>

    /// Access the header portion of the buffer.
    public var header: Header { get set }

    /// The number of trailing elements in the value.
    public var count: Int { get }

    /// Starting index for accessing the trailing elements. Always 0
    public var startIndex: Int { get }

    /// Ending index for accessing the trailing elements. Always `count`.
    public var endIndex: Int { get }

    /// Indices covering all of the trailing elements. Always `0..<count`.
    public var indices: Range<Int> { get }

    /// Access the trailing element at the given index.
    public subscript(index: Int) -> Element { get set }

    /// Accesses the trailing elements following the header.
    public var elements: Span<Element> { get }

    /// Accesses the trailing elements following the header, allowing mutation
    /// of those elements.
    public var mutableElements: MutableSpan<Element> { mutating get }

    /// Create a temporary intrusive managed buffer for the given header, whose
    /// trailing elements are initialized to copies of `element`. That instance
    /// is provided to the given `body` to operate on for the duration of the
    /// call. The temporary is allocated on the stack, unless it is very
    /// large according to `withUnsafeTemporaryAllocation`.
    public static func withTemporaryValue<R: ~Copyable, E>(
        header: consuming Header,
        repeating element: Element,
        body: (inout IntrusiveManagedBuffer<Header>) throws(E) -> R
    ) throws(E) -> R

    /// Create a temporary intrusive managed buffer for the given header, whose
    /// trailing elements are initialized with the given `initializer` function.
    /// That instance is provided to the given `body` to operate on for the
    /// duration of the call. The temporary is allocated on the stack, unless
    /// it is very large according to `withUnsafeTemporaryAllocation`.
    public static func withTemporaryValue<R: ~Copyable, E>(
        header: consuming Header,
        initializingTrailingElementsWith initializer: (inout OutputSpan<Element>) throws(E) -> Void,
        body: (inout IntrusiveManagedBuffer<Header>) throws(E) -> R
    ) throws(E) -> R
}

Documentation

The new types and protocol have full documentation in the source. The module itself has tutorial-like overview to guide users to the functionality they need.

Testing

New tests of the major functionality of the feature.

Performance

I did not. Everything is @frozen/@_alwaysEmitIntoClient and uses noncopyable types so there should be no extraneous runtime overhead.

Source Impact

No source impact. This is a new module with new API.

Checklist

I've read the Contribution Guidelines
My contributions are licensed under the Swift license.
I've followed the coding style of the rest of the project.
I've added tests covering all new code paths my change adds to the project (to the extent possible).
I've added benchmarks covering new functionality (if appropriate).
I've verified that my change does not break any existing tests or introduce unexpected benchmark regressions.
I've updated the documentation (if appropriate).

…rage Introduce new types `IntrusiveManagedBuffer` and `PaddedStorage` that consist of a fixed-sized header followed by variable-sized storage. The design of these types is inspired by the existing `ManagedBuffer` type in the standard library, but has been adjusted both to make them non-copyable and to eliminate the requirement for heap allocation.

lorentey

Nice!

This may end up sharing the same module as Box and other ownership utility constructs, but we'll have plenty of opportunities to set that up after this lands.

lorentey · 2025-09-08T17:53:15Z