Move hexdump() and hexmix() stream formatters to new hexdump.h for potential
use by other tests.
In toPythonUsing() helper function, add a temp file to receive Python script
debug output, and direct debug output to that file. On test failure, dump the
contents of that file to the log.
Give NamedTempFile::peep() an optional target std::ostream; refactor
implementation as peep_via() that accepts a callable to process each text
line. Add operator<<() to stream the contents of a NamedTempFile object to
ostream -- but don't use that with LL_DEBUGS(), as it flattens the file
contents into a single log line. Instead add peep_log(), which streams each
individual text line to LL_DEBUGS().
It's cool to be able to write 'arg1 << "stuff" << var ...;' for a lambda
accepting a std::ostream reference, but cascading compile errors mean it's no
longer worth trying to make that work -- given actual C++ lambdas.
Also clean up a lingering BOOST_FOREACH() and a boost::bind() while at it.
It seems the problem addressed by aab769e wasn't some synergy between
Boost.Phoenix and Boost.Function, but rather the lack of a Phoenix header file
introducing operator<<().
Newer C++ compilers have different semantics around LLSDArray's special copy
constructor, which was essential to proper LLSD nesting. In short, we can no
longer trust LLSDArray to behave correctly. Now that we have variadic
functions, get rid of LLSDArray and replace every reference with llsd::array().
When sending multiple LEAP packets in the same file (for testing convenience),
use a length prefix instead of delimiting with '\n'. Now that we allow a
serialization format that includes an LLSD format header (e.g.
"<?llsd/binary?>"), '\n' is part of the packet content. But in fact, testing
binary LLSD means we can't pick any delimiter guaranteed not to appear in the
packet content.
Using a length prefix also lets us pass a specific max_bytes to the subject
C++ LLSD parser.
Make llleap_test.cpp use new freestanding Python llsd package when available.
Update Python-side LEAP protocol code to work directly with encoded bytes
stream, avoiding bytes<->str encoding and decoding, which breaks binary LLSD.
Make LLSDSerialize::deserialize() recognize LLSD format header case-
insensitively. Python emits and checks for "llsd/binary", while LLSDSerialize
emits and checks for "LLSD/Binary". Once any of the headers is recognized,
pass corrected max_bytes to the specific parser.
Make deserialize() more careful about the no-header case: preserve '\n' in
content. Introduce debugging code (disabled) because it's a little tricky to
recreate.
Revert LLLeap child process stdout parser from LLSDSerialize::deserialize() to
the specific LLSDNotationParser(), as at present: the generic parser fails one
of LLLeap's integration tests for reasons that remain mysterious.
Since parsing binary LLSD is faster than parsing notation LLSD, send data from
the viewer to the LEAP plugin child process's stdin in binary instead of
notation.
Similarly, instead of parsing the child process's stdout using specifically a
notation parser, use the generic LLSDSerialize::deserialize() LLSD parser.
Add more LLSDSerialize Python compatibility tests.
LLSDSerialize::serialize() emits a header string, e.g. "<? llsd/notation ?>"
for notation format. Until now, LLSDSerialize::deserialize() has required that
header to properly decode the input stream.
But none of LLSDBinaryFormatter, LLSDXMLFormatter or LLSDNotationFormatter
emit that header themselves. Nor do any of the Python llsd.format_binary(),
format_xml() or format_notation() functions. Until now, you could not use
LLSD::deserialize() to parse an arbitrary-format LLSD stream serialized by
anything but LLSDSerialize::serialize().
Change LLSDSerialize::deserialize() so that if no header is recognized,
instead of failing, it attempts to parse as notation. Add tests to exercise
this case.
The tricky part about this processing is that deserialize() necessarily reads
some number of bytes from the input stream first, to try to recognize the
header. If it fails to do so, it must prepend the bytes it has already read to
the rest of the input stream since they're probably the beginning of the
serialized data.
To support this use case, introduce cat_streambuf, a std::streambuf subclass
that (virtually) concatenates other std::streambuf instances. When read by a
std::istream, the sequence of underlying std::streambufs appears to the
consumer as a single continuous stream.
This changeset makes it possible to build the Second Life viewer using
Python 3. It is designed to be used with an equivalent Autobuild branch
so that a developer can compile without needing Python 2 on their
machine.
Breaking change: Python 2 support ending
Rather than supporting two versions of Python, including one that was
discontinued at the beginning of the year, this branch focuses on
pouring future effort into Python 3 only. As a result, scripts do not
need to be backwards compatible. This means that build environments,
be they on personal computers and on build agents, need to have a
compatible interpreter.
Notes
- SLVersionChecker will still use Python 2 on macOS
- Fixed the message template url used by template_verifier.py
LLSDNotationFormatter (also LLSDNotationStreamer that uses it, plus
operator<<(std::ostream&, const LLSD&) that uses LLSDNotationStreamer) is most
useful for displaying LLSD to a human, e.g. for logging. Having the default
dump raw binary bytes into the log file is not only suboptimal, it can
truncate the output if one of those bytes is '\0'. (This is a problem with the
logging subsystem, but that's a story for another day.)
Use OPTIONS_PRETTY_BINARY wherever there is a default LLSDFormatter
::EFormatterOptions argument.
Also, allow setting LLSDFormatter subclass boolalpha(), realFormat() and
format(options) using optional constructor arguments. Naturally, each subclass
that supports this must accept and forward these constructor arguments to its
LLSDFormatter base class constructor.
Fix a couple bugs in LLSDNotationFormatter::format_impl() for an LLSD::Binary
value with OPTIONS_PRETTY_BINARY:
- The code unconditionally emitted a b(len) type prefix followed by either raw
binary or hex, depending on the option flag. OPTIONS_PRETTY_BINARY caused it
to emit "0x" before the hex representation of the data. This is wrong in
that it can't be read back by either the C++ or the Python LLSD parser.
Correct OPTIONS_PRETTY_BINARY formatting consists of b16"hex digits" rather
than b(len)"raw bytes".
- Although the code did set hex mode, it didn't set either the field width or
the fill character, so that a byte value less than 16 would emit a single
digit rather than two.
Instead of having one LLSDFormatter::format() method with an optional options
argument, declare two overloads. The format() overload without options passes
the mOptions data member to the overload accepting options.
Refactor the LLSDFormatter family, hoisting the recursive format_impl() method
(accepting level) to a pure virtual method at LLSDFormatter base-class level.
Most subclasses therefore need not override either base-class format() method,
only format_impl(). In fact the short format() overload isn't even virtual.
Consistently use LLSDFormatter::EFormatterOptions enum as the options
parameter wherever such options are accepted.
Use LLStringUtil::getenv() or getoptenv() whenever we fetch a string that will
be used as a pathname.
Use LLFile::tmpdir() instead of getenv("TEMP").
As an added extra-special bonus, finally clean up $TMP/llcontrol-test-zzzzzz
directories that have been accumulating every time we run a local build!
These are mostly things that were in fact erroneous, but accepted by older
compilers.
This changeset has not yet been built with Visual Studio 2013 or Linux gcc,
even with -std=c++11.
This changeset has not been built *without* -std=c++11. It should be used in
conjunction with a corresponding change to LL_BUILD_DARWIN_BASE_SWITCHES in
viewer-build-variables/variables.
This is a work in progress. We do not assert that this changeset completes the
work needed to turn on -std=c++11, even on the Mac.
We use boost::phoenix::placeholders::arg1 to imply a whole lambda expression,
replacing boost::lambda. But to bind a plain function in a more
straightforward way, seems classic boost::bind() works while
boost::phoenix::bind() does not.
https://svn.boost.org/trac/boost/ticket/10864
I've used boost::lambda with boost::function in a number of creative ways over
the years. But the clang 6 shipped with Xcode 6 seems to have somehow broken
lambda + function in Boost 1.57. boost::phoenix is a partial workaround.
Sadly, lambda's comma-operator overload doesn't seem to be supported,
necessitating a couple ugly workarounds.
With real lambdas now supported by current compilers, I'm sure the Boost
community has little incentive to repair the lambda + function problem.
Presumably we'll be able to use such features ourselves Real Soon Now...
SH-4346 FIX: Interesting: some integer Statistics are displayed as floating point after crossing region boundary
made llerrs/infos/etc properly variadic wrt tags
LL_INFOS("A", "B", "C") works, for example
fixed unit tests
remove llsimplestat
Using a Params block gives compile-time checking against attribute typos. One
might inadvertently set myLLSD["autofill"] = false and only discover it when
things behave strangely at runtime; but trying to set myParams.autofill will
produce a compile error.
However, it's excellent that the same LLProcess::create() method can accept
either LLProcess::Params or a properly-constructed LLSD block.
LLProcessLauncher had the somewhat fuzzy mandate of (1) accumulating
parameters with which to launch a child process and (2) sometimes tracking the
lifespan of the ensuing child process. But a valid LLProcessLauncher object
might or might not have ever been associated with an actual child process.
LLProcess specifically tracks a child process. In effect, it's a fairly thin
wrapper around a process HANDLE (on Windows) or pid_t (elsewhere), with
lifespan management thrown in. A static LLProcess::create() method launches a
new child; create() accepts an LLSD bundle with child parameters. So building
up a parameter bundle is deferred to LLSD rather than conflated with the
process management object.
Reconcile all known LLProcessLauncher consumers in the viewer code base,
notably the class unit tests.
Specifically:
Introduce ManageAPR class in indra/test/manageapr.h. This is useful for a
simple test program without lots of static constructors.
Extract NamedTempFile from llsdserialize_test.cpp to indra/test/
namedtempfile.h. Refactor to use APR file operations rather than platform-
dependent APIs.
Use NamedTempFile for llprocesslauncher_test.cpp.
Instead of low-level open(O_CREAT | O_EXCL) loop on all platforms, use
GetTempFileName() on Windows and mkstemp() elsewhere.
Don't append a final newline to NamedTempFile: use caller's data literally.
Tweak a couple comments.
Consider this pathname for llsdserialize_test.cpp:
C:\nats\indra\llcommon\tests\llsdserialize_test.cpp
Embed that in a Python string literal:
'C:\nats\indra\llcommon\tests\llsdserialize_test.cpp'
and you get a string containing:
C:
ats\indra\llcommon ests\llsdserialize_test.cpp
where the \n became a newline and the \t became a tab character.
Hopefully Python raw-string syntax r'C:\etc\etc' works better.
In this case, the Python code in question is being written from a C++ string
literal to a temp script file in a platform-dependent temp directory -- so the
Python __file__ value tells you nothing about the location of the repository
checkout. Embedding __FILE__ from the containing C++ source file works better.
And at that point, the Python logic needed to bring in the llsd module is big
enough to warrant capturing it in a separate string variable common to
multiple tests.
Ansariel