Repository navigation
Refactoring of internal storage types (continued) - #2819
Conversation
cf23f8d to
7ed97ca
Compare
5d61050 to
69d0f26
Compare
fd4584f to
7be2138
Compare
8ef73cf to
641a89a
Compare
641a89a to
f67f551
Compare
Co-authored-by: Sergey Kopienko <sergey.kopienko@intel.com>
e263656 to
e124db1
Compare
| auto __temp_acc = __get_accessor(sycl::write_only, __scratch_storage, __cgh, __dpl_sycl::__no_init{}); | ||
| auto __stop_pos_acc = | ||
| __internal::__get_stop_pos_accessor_opt<_Bounded>(sycl::write_only, __cgh, __stop_pos_storage); | ||
| auto __stop_pos_acc = __get_accessor(sycl::write_only, __stop_pos_storage, __cgh, __dpl_sycl::__no_init{}); |
There was a problem hiding this comment.
This is not a new (benign) bug, just a propagated one, but we are touching it so lets fix it.
__no_init is wrong here, when talking about any block past block 0. However, instead of trying to have it dependent on block, lets just remove it. When subsequent blocks get an accessor that is write with no_init, the originally block 0 written value can be overwritten by uninitialized data on the way out.
When USM is available this is USM data anyway so it isn't used in most cases.
If we wanted to get fancy, we don't even need this accessor after block zero in this kernel, but again with USM it is not costly so lets just remove no_init.
| auto __stop_pos_acc = __get_accessor(sycl::write_only, __stop_pos_storage, __cgh, __dpl_sycl::__no_init{}); | |
| auto __stop_pos_acc = __get_accessor(sycl::write_only, __stop_pos_storage, __cgh); |
There was a problem hiding this comment.
Aha, so my initial variant was right :) I added no_init in response to Sergey's feedback; but I should have looked thoroughly.
There was a problem hiding this comment.
If we wanted to get fancy, we don't even need this accessor after block zero in this kernel, but again with USM it is not costly so lets just remove no_init.
Actually, I have got an idea that if a constructor of __result_storage takes an initial value, the reduce submitter would not need __stop_pos_storage and __stop_pos_initial_state at all. And the initialization from host should not add much extra overhead, except maybe for device USM. Maybe it is worth exploring.
There was a problem hiding this comment.
Hmm, I suppose if we expect the norm to be host_usm for __stop_pos_storage, then yes we could do this for a small optimization. However, it would be at a detriment for the other two options, device USM and sycl::buffer.
It would save a little bit of time in the most optimized case, and simplify the code, but risk I think a lot more in the other cases (extra kernel launch for a fill or extra host->device transfer). I don't have strong opinions here against this, but I'm not sure its worth a lot of effort either.
There was a problem hiding this comment.
Will it be really a detriment for sycl::buffer, once we removed no_init?
Also do we have any hardware and SYCL platform in mind with device USM but no host USM?
There was a problem hiding this comment.
I believe so, because creating a sycl::buffer from a range without initial value likely allows the runtime to avoid any initial transfer from host to device. If it is initialized with a value or backed with data on the host, then it would need to have a host device transfer on the first kernel launch. I think the way it is created now with a range may skip that first host device transfer (even after removing no_init), or at least enables that option for the runtime (whether or not they take it is up to them I suppose).
There was a problem hiding this comment.
I think the cases which support USM but not host USM are rare and not something we really care much about. No modern GPUs are in this category as far as I understand.
danhoeflinger
left a comment
There was a problem hiding this comment.
LGTM. I believe this is in good shape now. (needs a rebase with main)
SergeyKopienko
left a comment
There was a problem hiding this comment.
LGTM
Left some optional comments
5f3a7d4
* move storage types and related utilities to a separate header * generalize optional result storage from reduce-then-scan * add __storage_holder to manage storage lifetime for asynchronous patterns * use the holder with merge sort * add unit test for __storage_holder --------- Co-authored-by: Sergey Kopienko <sergey.kopienko@intel.com>
This PR continues improving the internal SYCL backend infrastructure to deal with temporary buffers and device-computed results. It complements the storage types (
__device/result/combined/_storage) with a "storage holder" to keep the allocated memory alive and pass it from a backend pattern to its caller.Changes made:
__result_and_scratch_storageand__futureare not moved and will mostly remain intact until fully replaced.__storage_holderclass tempate to specify a set of storage buffers that a backend pattern may deal with.__result_and_scratch_storagein merge sort (__parallel_sort_impl)How it is supposed to work:
__storage_holderto specialize its return types and limit the number of scratch buffers.sycl::eventthat signals kernel completion remains (a part of) the returned value.__copy_resultmethod.__future.