ethers.js/docs.wrm/api/utils/strings.wrm

_section: Strings  @<strings>

A **String** is a representation of a human-readable input of output,
which are often taken for granted.

When dealing with blockchains, properly handling human-readable and
human-provided data is important to prevent loss of funds, assets,
incorrect permissions, etc.

_subsection: Bytes32String @<Bytes32String>

A string in Solidity is length prefixed with its 256-bit (32 byte)
length, which means that even short strings require 2 words (64 bytes)
of storage.

In many cases, we deal with short strings, so instead of prefixing
the string with its length, we can null-terminate it and fit it in a
single word (32 bytes). Since we need only a single byte for the
null termination, we can store strings up to 31 bytes long in a
word.

_note: Note
Strings that are 31 __//bytes//__ long may contain fewer than 31 __//characters//__,
since UTF-8 requires multiple bytes to encode international characters.

_property: ethers.utils.parseBytes32String(aBytesLike) => string  @<utils-parseBytes32> @SRC<strings>
Returns the decoded string represented by the ``Bytes32`` encoded data.

_property: ethers.utils.formatBytes32String(text) => string<[[DataHexString]]<32>>  @<utils-formatBytes32> @SRC<strings>
Returns a ``bytes32`` string representation of //text//. If the
length of //text// exceeds 31 bytes, it will throw an error.


_subsection: UTF-8 Strings @<strings-utf8>

_property: ethers.utils.toUtf8Bytes(text [ , form = current ] ) => Uint8Array  @<utils-toUtf8Bytes> @SRC<strings>
Returns the UTF-8 bytes of //text//, optionally normalizing it using the
[[strings--unicode-normalization-form]] //form//.

_property: ethers.utils.toUtf8CodePoints(text [ , form = current ] ) => Array<number>  @<utils-toUtf8CodePoints> @SRC<strings>
Returns the Array of codepoints of //text//, optionally normalized using the
[[strings--unicode-normalization-form]] //form//.

_note: Note
This function correctly splits each **user-perceived character** into
its codepoint, accounting for surrogate pairs. This should not be confused with
``string.split("")``, which destroys surrogate pairs, splitting between each UTF-16
codeunit instead.

_property: ethers.utils.toUtf8String(aBytesLike [ , onError = error ] ) => string  @<utils-toUtf8String> @SRC<strings>
Returns the string represented by the UTF-8 bytes of //aBytesLike//.

The //onError// is a [Custom UTF-8 Error function](strings--error-handling) and if not specified
it defaults to the [error](strings--Utf8Error) function, which throws an error
on **any** UTF-8 error.

_subsection: UnicodeNormalizationForm @<strings--unicode-normalization-form> @SRC<strings/utf8:enum.UnicodeNormalizationForm>

There are several [commonly used forms](link-wiki-unicode-equivalence)
when normalizing UTF-8 data, which allow strings to be compared or hashed in a stable
way.

_property: ethers.utils.UnicodeNormalizationForm.current
Maintain the current normalization form.

_property: ethers.utils.UnicodeNormalizationForm.NFC
The Composed Normalization Form. This form uses single codepoints
which represent the fully composed character.

For example, the **&eacute;** is a single codepoint, ``0x00e9``.

_property: ethers.utils.UnicodeNormalizationForm.NFD
The Decomposed Normalization Form. This form uses multiple codepoints
(when necessary) to compose a character.

For example, the **&eacute;**
is made up of two codepoints, ``"0x0065"`` (which is the letter ``"e"``)
and ``"0x0301"`` which is a special diacritic UTF-8 codepoint which
indicates the previous character should have an acute accent.

_property: ethers.utils.UnicodeNormalizationForm.NFKC
The Composed Normalization Form with Canonical Equivalence. The Canonical
representation folds characters which have the same syntactic representation
but different semantic meaning.

For example, the Roman Numeral **I**, which has a UTF-8
codepoint ``"0x2160"``, is folded into the capital letter I, ``"0x0049"``.

_property: ethers.utils.UnicodeNormalizationForm.NFKD
The Decomposed Normalization Form with Canonical Equivalence.
See NFKC for more an example.

_note: Note
Only certain specified characters are folded in Canonical Equivalence, and thus
it should **not** be considered a method to achieve //any// level of security from
[homoglyph attacks](link-wiki-homoglyph).


_subsection: Custom UTF-8 Error Handling @<strings--error-handling>

When converting a string to its codepoints, there is the possibility
of invalid byte sequences. Since certain situations may need specific
ways to handle UTF-8 errors, a custom error handling function can be used,
which has the signature:

_property: errorFunction(reason, offset, bytes, output [ , badCodepoint ]) => number
The //reason// is one of the [UTF-8 Error Reasons](strings--error-reasons), //offset// is the index
into //bytes// where the error was first encountered, output is the list
of codepoints already processed (and may be modified) and in certain Error
Reasons, the //badCodepoint// indicates the currently computed codepoint,
but which would be rejected because its value is invalid.

This function should return the number of bytes to skip past keeping in
mind the value at //offset// will already be consumed.


_heading: UTF-8 Error Reasons @<strings--error-reasons> @SRC<strings/utf8:Utf8ErrorReason>

_property: ethers.utils.Utf8ErrorReason.BAD_PREFIX
A byte was encountered which is invalid to begin a UTF-8 byte
sequence with.

_property: ethers.utils.Utf8ErrorReason.MISSING_CONTINUE
A UTF-8 sequence was begun, but did not have enough continuation
bytes for the sequence. For this error the //ofset// is the index
at which a continuation byte was expected.

_property: ethers.utils.Utf8ErrorReason.OUT_OF_RANGE
The computed codepoint is outside the range for valid UTF-8
codepoints (i.e. the codepoint is greater than 0x10ffff).
This reason will pass the computed //badCountpoint// into
the custom error function.

_property: ethers.utils.Utf8ErrorReason.OVERLONG
Due to the way UTF-8 allows variable length byte sequences
to be used, it is possible to have multiple representations
of the same character, which means
[overlong sequences](link-wiki-utf8-overlong)
allow for a non-distinguished string to be formed, which can
impact security as multiple strings that are otherwise
equal can have different hashes.

Generally, overlong sequences are an attempt to circumvent
some part of security, but in rare cases may be produced by
lazy libraries or used to encode the null terminating
character in a way that is safe to include in a ``char*``.

This reason will pass the computed //badCountpoint// into the
custom error function, which is actually a valid codepoint, just
one that was arrived at through unsafe methods.

_property: ethers.utils.Utf8ErrorReason.OVERRUN
The string does not have enough characters remaining for the
length of this sequence.

_property: ethers.utils.Utf8ErrorReason.UNEXPECTED_CONTINUE
This error is similar to BAD_PREFIX, since a continuation byte
cannot begin a valid sequence, but many may wish to process this
differently. However, most developers would want to trap this
and perform the same operation as a BAD_PREFIX.

_property: ethers.utils.Utf8ErrorReason.UTF16_SURROGATE
The computed codepoint represents a value reserved for
UTF-16 surrogate pairs.
This reason will pass the computed surrogate half
//badCountpoint// into the custom error function.



_heading: Provided UTF-8 Error Handling Functions

There are already several functions available for the  most common
situations.

_property: ethers.utils.Utf8ErrorFuncs.error @<strings--Utf8Error> @SRC<strings/utf8:errorFunc>
The will throw an error on **any** error with a UTF-8 sequence, including
invalid prefix bytes, overlong sequences, UTF-16 surrogate pairs.

_property: ethers.utils.Utf8ErrorFuncs.ignore @<strings--Utf8Ignore> @SRC<strings/utf8:ignoreFunc>
This will drop all invalid sequences (by consuming invalid prefix bytes and
any following continuation bytes) from the final string as well as permit
overlong sequences to be converted to their equivalent string.

_property: ethers.utils.Utf8ErrorFuncs.replace @<strings--Utf8Replace> @SRC<strings/utf8:replaceFunc>
This will replace all invalid sequences (by consuming invalid prefix bytes and
any following continuation bytes) with the
[UTF-8 Replacement Character](link-wiki-utf8-replacement),
(i.e. U+FFFD).