Quickjs.UnicodeUnicode utilities from QuickJS's libunicode
This module provides Unicode character classification, case conversion, and normalization functions. It uses the same battle-tested Unicode tables as QuickJS's ES2023-compliant JavaScript engine.
val normalize : normalization -> string -> string optionnormalize form str normalizes a UTF-8 string to the specified form. Returns None on memory allocation failure or invalid input.
Example:
let composed = Unicode.normalize NFC "cafe\xcc\x81" in
(* composed = Some "café" (e + combining accent composed to é) *)
let decomposed = Unicode.normalize NFD "café" in
(* decomposed = Some "cafe\xcc\x81" *)
ignore (composed, decomposed)lowercase str converts a UTF-8 string to lowercase. Handles Unicode characters like "ÉCOLE" → "école".
uppercase str converts a UTF-8 string to uppercase. Handles special cases like "ß" → "SS".
fold_case str applies Unicode full case folding to a UTF-8 string, e.g. "Straße" → "strasse" and "ΣΤΙΓΜΑΣ" → "στιγμασ". Case folding is context-independent (no Final_Sigma rule) and locale-independent; two strings are caseless-equal when their foldings are equal. This is the same folding QuickJS uses for case-insensitive matching.
lowercase_char c returns the lowercase form of a code point. Returns a list because some characters expand (though lowercase rarely does).
uppercase_char c returns the uppercase form of a code point. Returns a list because some characters expand, e.g., 'ß' → 'S'; 'S'.
fold_case_char c returns the Unicode full case folding of a code point. Returns a list because some characters expand, e.g., 'ß' → 's'; 's'.
val is_cased : Uchar.t -> boolis_cased c returns true if the character has uppercase/lowercase forms. Examples: 'a', 'A', 'é' are cased; '1', '!' are not.
val is_case_ignorable : Uchar.t -> boolis_case_ignorable c returns true if the character is ignored during case mapping operations (e.g., combining marks).
val is_id_start : Uchar.t -> boolis_id_start c returns true if the character can start a JavaScript/Unicode identifier (letters, $, _).
val is_id_continue : Uchar.t -> boolis_id_continue c returns true if the character can continue a JavaScript/Unicode identifier (letters, digits, $, _, combining marks).
val is_whitespace : Uchar.t -> boolis_whitespace c returns true if the character is Unicode whitespace. Includes ASCII space, tab, newline, and Unicode spaces like U+00A0 (NBSP).
Lookups into the Unicode property tables QuickJS uses for regexp \p{...} escapes: Script, Script_Extensions, General_Category and the binary properties. Names are the canonical Unicode names and their aliases, matched case-sensitively, exactly like JavaScript's \p{...}.
module CharSet : sig ... endA set of Unicode code points, stored as sorted disjoint ranges.
val script : ?extensions:bool -> string -> CharSet.t optionscript ?extensions name returns the set of code points whose Script property is name, or None if name is not a known script. Accepts long and short names ("Greek" or "Grek"). With ~extensions:true the Script_Extensions property is used instead, like JavaScript's \p{Script_Extensions=...}.
Example:
let greek = Option.get (Unicode.script "Greek") in
assert (Unicode.CharSet.mem (Uchar.of_int 0x03B1) greek)
(* α *)val general_category : string -> CharSet.t optiongeneral_category name returns the set of code points whose General_Category is name, or None if name is not a known category. Accepts short and long names ("Lu" or "Uppercase_Letter"), including the grouped categories ("L", "Letter", ...), like JavaScript's \p{L}.
val binary_property : string -> CharSet.t optionbinary_property name returns the set of code points with the given binary property, or None if name is not a known property. Accepts names like "Alphabetic", "White_Space", "Emoji" and their aliases, like JavaScript's \p{Alphabetic}.