Ensure proper UTF-8 encoding (1 to 4 bytes).
Handle invalid encodings (return 0xFFFD and consume a single byte)
Individually encoded surrogate code points are accepted.
- add `utf8_scan()` to analyze a byte array for UTF-8 contents
detects invalid encoding, computes number of codepoints and content kind:
plain ASCII, 8-bit, 16-bit or larger codepoints.
- add `utf8_encode_len(c)` to compute the number of bytes to encode `c`
- rename `unicode_to_utf8` as `utf8_encode`
- rename `unicode_from_utf8` as `utf8_decode`
- add `utf8_decode_buf8(dest, size, src, len)` to decode a UTF-8 encoded
byte array known to contain only ASCII and 8-bit codepoints.
- add `utf8_decode_buf16(dest, size, src, len)` to decode a UTF-8 encoded
byte array into an array of 16-bit codepoints using UTF-16 surrogate pairs
for non-BMP1 codepoints.
- add `utf8_encode_buf8(dest, size, src, len)` to encode an array of 8-bit
codepoints as a UTF-8 encoded null terminated string
- add `utf16_encode_buf8(dest, size, src, len)` to decode an array of 16-bit
codepoints (including surrogate pairs) as a UTF-8 encoded null terminated string
- detect invalid UTF-8 encoding in RegExp parser
- simplify `JS_AtomGetStrRT`, `JS_NewStringLen` using the above functions
- simplify UTF-8 decoding and error testing
The shell script runs the tests and diffs stdout against v8.txt.
Lines added/removed means tests were broken/fixed.
Future work is to a) fix failing tests, and b) enable more tests.
A number are disabled for various reasons and mjsunit subdirectories are
currently skipped. Need to decide on a case-by-case basis what is and
isn't relevant to us.
At the moment about 430 tests are run of which approx. 80% pass.
JSValueConst was only used for the now removed CONFIG_CHECK_JSVALUE
build mode. It is kept around as an alias for JSValue in quickjs.h to
avoid breaking everyone's source builds but remove it everywhere else.
The default 256 kb stack is too small to run some of the test262 tests
when ASAN is enabled.
Double it to 512 kb and ensure threads created by quickjs have big
enough stacks.
The removal of the high-precision but non-standard clock source in
commit 5af98ca broke microbench because Date.now() is not granular
enough for the benchmark runner to make forward progress.
This commit adds a new method to the os module that returns time
with microsecond precision.
