Why didn’t you just use UTF8? It’s more compact than UTF16.
This was considered. However, this would have forced far more conversions throughout the VCL code as it talks to the Windows API, and it would have introduced a lot of very subtle breakages in much of user code. While a lot of code out there already handles DBCS (Double-byte character sets), that same code does not correctly handle characters that consist of > 2 bytes.
In UTF8 a single character can be represented by as many as 6 bytes.[Correction: This is not the case in true UTF8. 5 and 6 byte sequences are illegal in UTF8 (thanks Aleksander)] In UTF8 a single character can be represented by as many as 4 bytes. Finally, UTF16 is the native format used internally by Windows itself. By calling directly to the "W" APIs, the "A" translation layer that Windows has is bypassed and should, in theory, increase performance in some cases.
Delphi Parallel Library (DPL)