Handle non-trivially-copyable types in Loop/Scan output concatenation
Merged
YU
4mo ago+343+343+012345678901234567890123456789-13-13-01234567890123456789
This pull request improves the handling of string tensors in the CPU implementation of the Loop operator, ensuring correct memory management and copy semantics for non-trivially-copyable types like std::string. It also adds comprehensive unit tests to verify these behaviors, especially for cases involving string scan outputs and loop-carried variables.
Enhancements for string tensor support:
- Updated
ConcatenateCpuOutputinloop.ccto properly detect string tensors and usestd::copyfor concatenation, ensuring correct handling of heap-allocated string payloads and avoiding unsafe byte-wise copying. [1] [2] - Modified
OutputIterator::ZeroOutCurrentinscan_utils.hto skip zeroing for string tensors, as their elements are already default-constructed and cannot be safely set withmemset.
New and extended unit tests for string tensor scenarios:
- Added tests to
loop_test.cccovering:- String scan outputs, including long strings that exceed the small-string-optimization threshold and multi-element outputs.
- Loop-carried string variables to ensure they are copied correctly.
- Zero-iteration cases to confirm empty string scan outputs are handled without errors.