This commit adds the HBXX64 and HBXX128 classes for use as a drop-in
replacement for the slow LLMD5 hashing class, where speed matters and
backward compatibility (with standard hashing algorithms) and/or
cryptographic hashing qualities are not required.
It also replaces LLMD5 with HBXX* in a few existing hot (well, ok, just
"warm" for some) paths meeting the above requirements, while paving the way for
future use cases, such as in the DRTVWR-559 and sibling branches where the slow
LLMD5 is used (e.g. to hash materials and vertex buffer cache entries), and
could be use such a (way) faster algorithm with very significant benefits and
no negative impact.
Here is the comment I added in indra/llcommon/hbxx.h:
// HBXXH* classes are to be used where speed matters and cryptographic quality
// is not required (no "one-way" guarantee, though they are likely not worst in
// this respect than MD5 which got busted and is now considered too weak). The
// xxHash code they are built upon is vectorized and about 50 times faster than
// MD5. A 64 bits hash class is also provided for when 128 bits of entropy are
// not needed. The hashes collision rate is similar to MD5's.
// See https://github.com/Cyan4973/xxHash#readme for details.
This commit adds the HBXX64 and HBXX128 classes for use as a drop-in
replacement for the slow LLMD5 hashing class, where speed matters and
backward compatibility (with standard hashing algorithms) and/or
cryptographic hashing qualities are not required.
It also replaces LLMD5 with HBXX* in a few existing hot (well, ok, just
"warm" for some) paths meeting the above requirements, while paving the way for
future use cases, such as in the DRTVWR-559 and sibling branches where the slow
LLMD5 is used (e.g. to hash materials and vertex buffer cache entries), and
could be use such a (way) faster algorithm with very significant benefits and
no negative impact.
Here is the comment I added in indra/llcommon/hbxx.h:
// HBXXH* classes are to be used where speed matters and cryptographic quality
// is not required (no "one-way" guarantee, though they are likely not worst in
// this respect than MD5 which got busted and is now considered too weak). The
// xxHash code they are built upon is vectorized and about 50 times faster than
// MD5. A 64 bits hash class is also provided for when 128 bits of entropy are
// not needed. The hashes collision rate is similar to MD5's.
// See https://github.com/Cyan4973/xxHash#readme for details.
Openjpeg was modified: "p_max_len -= l_nb_bytes_read;" was causing an overflow.
I'm not sure if I did something incorectly in opj_skip/opj_seek viewer side, but seems like openjpeg should have been checking remaining space in p_max_len either way.
P.S. Many thanks to Chafey and Neopallium for implementing openjpeg's partial bitstream support
Brad Linden