[yuxi-liu-wired]

After an unpaired high surrogate followed by a BMP code unit, the UTF-16 decoder reports Malformed(2, 2) and keeps the BMP unit pending in lead_surrogate with pending_bmp set. additional_from_state() counts the pending unit only when lead_surrogate != 0, so a pending U+0000 is not counted. A buffer of exactly max_utf8_buffer_length(), max_utf8_buffer_length_without_replacement() or max_utf16_buffer_length() can then get OutputFull, which the docs say won't happen:

let mut d = UTF_16LE.new_decoder_without_bom_handling();
let mut out = [0u8; 16];
d.decode_to_utf8_without_replacement(&[0x00, 0xD8], &mut out, false);          // InputEmpty
d.decode_to_utf8_without_replacement(&[0x00, 0x00, 0x00, 0x4E], &mut out, false); // Malformed(2, 2), U+0000 pending
let rest = [0x00, 0x4E, 0x00, 0x4E];
let max = d.max_utf8_buffer_length_without_replacement(rest.len()).unwrap();    // 7
let mut out = vec![0u8; max];
d.decode_to_utf8_without_replacement(&rest, &mut out, true);                    // OutputFull

With U+0041 in the same spot, max is 10 and the call succeeds. The fix counts the pending unit whenever pending_bmp is set.

The new test test_utf_16le_pending_nul_max_length fails without the change (OutputFull). cargo test passes on stable, 1.88, --no-default-features, std, simd-accel (nightly), and the fast-legacy-encode / less-slow-* features. cargo fmt --check passes too.

I found this by fuzzing the streaming decoders with random chunking and output buffers just at the documented bounds. The test is dedicated to the public domain like the rest of the test module.

[hsivonen]

Thank you!