Normalization and Case Folding

One issue that recently came up during implementation is the interaction between NFKC and case folding. For a small number of characters, case folding is not closed under NFKC. Suppose Q(X) is a function defined by:

Q(X) = NFKC(CaseFold(X))

The following property is NOT true for all strings:

Q(Q(X)) == Q(X)

Unicode standard recommends that additional mapping be performed in order to guarantee the function to be closed [1]. These mappings are well-defined and are given in the data file http://www.unicode.org/Public/3.2-Update/DerivedNormalizationProps-3.2.0.txt under the FC_NFKC_Closure mapping. These mappings are also specified in Stringprep (RFC3454) processing, and are in use by IDNA Nameprep.

For XRI, it means that an application wishing to accurately test for authority equivalence should apply the additional mapping when performing case folding. If this is not followed, a pair of equivalent strings that only differ in case may be falsely treated as different strings even though case folding has been applied. Perhaps we should add an additional recommendation to perform the additional mapping when doing case-less matching of authority in section 2.5 of xri-syntax-V2.0-cd02 “Normalization and Comparison”.

wil.

[1] http://www.unicode.org/versions/Unicode4.0.0/ch05.pdf#G21180

xri message