Repository navigation
Document the intermediate data storage utilities in the SYCL backend - #2847
Conversation
| On construction it first attempts to allocate host USM, which allows the result | ||
| to be read back to the host without an explicit copy. If host USM is not | ||
| available, it falls back to the device USM or `sycl::buffer` strategy of | ||
| `__device_storage`. |
There was a problem hiding this comment.
Host USM is used only on Level Zero GPUs, by policy rather than by availability: __allocate_usm<_T, sycl::usm::alloc::host> returns nullptr on every other device, including OpenCL CPU and GPU devices that support host USM (
oneDPL/include/oneapi/dpl/pstl/hetero/dpcpp/utils_storage_sycl.h
Lines 112 to 136 in a86d55d
| On construction it first attempts to allocate host USM, which allows the result | |
| to be read back to the host without an explicit copy. If host USM is not | |
| available, it falls back to the device USM or `sycl::buffer` strategy of | |
| `__device_storage`. | |
| On construction it first attempts to allocate host USM, which allows the result | |
| to be read back to the host without an explicit copy. Host USM is only used on | |
| Level Zero GPUs; on other devices, including those that support host USM, it | |
| falls back to the device USM or `sycl::buffer` strategy of `__device_storage`. |
There was a problem hiding this comment.
Added a clarification.
| `__extract()` consumes the holder and converts its content into a tuple of | ||
| `__copyable_storage_state` objects with shared ownership of the underlying | ||
| allocations. The `__future` holds these states for its own lifetime, ensuring | ||
| the memory remains valid until the asynchronous result is consumed. The holder | ||
| itself is left in a state where its destructor has nothing to free. |
There was a problem hiding this comment.
Two details could be added here.
__futurehas no destructor that waits ( ), so the memory stays valid as long as the__futureor a copy of it exists, not until the result is consumed. If it is destroyed before the event completes, the USM is freed while the kernels may still run.- The third mode is not described: an algorithm called in
__deferrable_modeuses__checked_deferrable_wait(), which waits whenever the future carries a payload, so that the temporary storage is not destroyed before the kernels finish (). A short "Deferrable Algorithm" subsection would cover it.oneDPL/include/oneapi/dpl/pstl/hetero/dpcpp/parallel_backend_sycl_utils.h
Lines 671 to 683 in a86d55d
There was a problem hiding this comment.
1 - rephrased to avoid misinterpretation.
2 - not yet sure of importance.
| - `__get_accessor(access_mode_tag, storage, cgh, property_list)` — | ||
| acquires an accessor to the data storage region, or to the scratch-only | ||
| region for `__combined_storage`. |
There was a problem hiding this comment.
For a __combined_storage this is not a scratch-only region: __get_accessor binds to the __device_storage base (
oneDPL/include/oneapi/dpl/pstl/hetero/dpcpp/utils_storage_sycl.h
Lines 325 to 328 in a86d55d
sycl::buffer the accessor range, and with it no_init and the dependency tracking, includes the result. Maybe: "acquires an accessor to the whole data storage; for a __combined_storage that shares one allocation, this includes the result region".
There was a problem hiding this comment.
Accessing the result area is neither guaranteed nor intended. The intended usage of __get_accessor for __combined_storage is only for the scratch allocation.
| - `_NScratch` — the number of scratch storage slots. This must equal the | ||
| maximum number of `__device_storage` and `__combined_storage` objects | ||
| simultaneously used by any code path through the backend pattern. |
There was a problem hiding this comment.
_NScratch only needs to be at least that number. It may also help to say that exceeding it is caught only by an assert in __store_scratch and __store (
| - `_NScratch` — the number of scratch storage slots. This must equal the | |
| maximum number of `__device_storage` and `__combined_storage` objects | |
| simultaneously used by any code path through the backend pattern. | |
| - `_NScratch` — the number of scratch storage slots. This must be at least the | |
| maximum number of `__device_storage` and `__combined_storage` objects | |
| simultaneously used by any code path through the backend pattern; | |
| exceeding it is only caught by an assertion. |
| Care must be taken when a kernel is in flight and holds a reference to storage | ||
| memory: destroying a storage object or the holder before the kernel completes | ||
| will cause premature deallocation. The backend pattern is responsible for | ||
| ensuring that all in-flight kernels have completed or that ownership has been | ||
| transferred to the holder before any storage object goes out of scope. |
There was a problem hiding this comment.
It may be worth stating three details here:
- the destructor of
__storage_holder, like those of the storages, does not wait for kernels; - a storage backed by
sycl::bufferdoes not free early: the buffer destructor blocks until the kernels using it finish, so destroying it early silently synchronizes instead; - if an exception is thrown after a kernel has been submitted but before the wait (for example, by a later
submitin the same pattern), the USM is freed while that kernel may still be running. That is a known limitation rather than something a pattern can always prevent.
There was a problem hiding this comment.
Rephrased to cover these points.
| __parallel_pattern_impl(sycl::queue& __q, | ||
| __parallel_pattern_holder<ValueType>& __holder, ...) | ||
| { | ||
| __combined_storage<<ValueType> __scratch_and_result(__q, __scratch_n, 1); |
There was a problem hiding this comment.
Typo:
| __combined_storage<<ValueType> __scratch_and_result(__q, __scratch_n, 1); | |
| __combined_storage<ValueType> __scratch_and_result(__q, __scratch_n, 1); |
| template <bool _Bounded, typename _ValueType, typename _StopPosType> | ||
| using __transform_scan_storage_holder = | ||
| std::conditional_t<_Bounded, | ||
| __storage_holder<2, _ValueType, _StopPosType>, | ||
| __storage_holder<2, _ValueType>>; |
There was a problem hiding this comment.
In the long run, 2 is correct.
| ```mermaid | ||
| flowchart TB | ||
| subgraph CallerStart["Algorithm"] | ||
| S1["Construct holder\n__parallel_pattern_holder<N, Ts...> __holder(__q)"] |
There was a problem hiding this comment.
Nit: the alias hides the number of scratch slots (L184-L186), so the caller writes __parallel_pattern_holder<Ts...>, as in the pseudocode below (__parallel_pattern_holder<ValueType>):
| S1["Construct holder\n__parallel_pattern_holder<N, Ts...> __holder(__q)"] | |
| S1["Construct holder\n__parallel_pattern_holder<Ts...> __holder(__q)"] |
| sycl::event __event = __parallel_pattern_async(__q, __holder, ...); | ||
| __event.wait_and_throw(); | ||
|
|
||
| ValueType value; |
There was a problem hiding this comment.
Nit: value-initialized, as at L88 and as __get_results does for its destination:
| ValueType value; | |
| ValueType value{}; |
| } | ||
| ``` | ||
|
|
||
| Note that `_NScratch` is set to 1; the small-size path stores no scratch, while |
There was a problem hiding this comment.
Trailing whitespace.
| Note that `_NScratch` is set to 1; the small-size path stores no scratch, while | |
| Note that `_NScratch` is set to 1; the small-size path stores no scratch, while |
|
Added to the next milestone and moved to |
danhoeflinger
left a comment
There was a problem hiding this comment.
Just a couple minor things which I think would help, not required.
Otherwise LGTM
This RFC describes how to use the intermediate data storage in the SYCL backend patterns and their callers. It covers both the existing storage types (
__{device|result|combined}_storage) and the new__storage_holderclass template and related utilities introduced in #2819.