Using Sync with multiple threads is trickier than with coroutines. In
particular, Sync::bump() was racy (get() and set() as two different
operations), and threads were proceeding when they should have waited.
Fortunately LLCond, on which Sync is based, already supports atomic update
operations. Use that for bump().
But to nail things down even more specifically, add set(n) to complement
yield_until(n). Using those methods, there should be no ambiguity about which
call in one thread synchronizes with which call in the other thread.
Sometimes it's useful to be able to temporarily override an existing LOGFAIL
setting in the current environment. It's far more convenient to prepend
LOGFAIL='' to a command than to 'unset LOGFAIL' as a whole separate command --
and then remember to restore its previous value.
Introduce LLCoros::Stop exception, with subclasses Stopping, Stopped and
Shutdown. Add LLCoros::checkStop(), intended to be called periodically by any
coroutine with nontrivial lifespan. It checks the LLApp status and, unless
isRunning(), throws one of these new exceptions.
Make LLCoros::toplevel() catch Stop specially and log forcible coroutine
termination.
Now that LLApp status matters even in a test program, introduce a trivial
LLTestApp subclass whose sole function is to make isRunning() true.
(LLApp::setStatus() is protected: only a subclass can call it.) Add LLTestApp
instances to lleventcoro_test.cpp and lllogin_test.cpp.
Make LLCoros::toplevel() accept parameters by value rather than by const
reference so we can continue using them even after context switches.
Make private LLCoros::get_CoroData() static. Given that we've observed some
coroutines living past LLCoros destruction, making the caller call
LLCoros::instance() is more dangerous than encapsulating it within a static
method -- since the encapsulated call can check LLCoros::wasDeleted() first
and do something reasonable instead. This also eliminates the need for both a
const and non-const overload.
Defend LLCoros::delete_CoroData() (cleanup function for fiber_specific_ptr for
CoroData, implicitly called after coroutine termination) against calls after
~LLCoros().
Add a status string to coroutine-local data, with LLCoro::setStatus(),
getStatus() and RAII class TempStatus.
Add an optional 'when' string argument to LLCoros::printActiveCoroutines().
Make ~LLCoros() print the coroutines still active at destruction.
No one uses LLEventQueue to defer posted events until the next mainloop tick
-- and with LLCoros moving to Boost.Fiber, cross-coroutine event posting works
that way anyway, making LLEventQueue pretty unnecessary.
The static RegisterFlush instance in llevents.cpp was used to call
LLEventPumps::flush() once per mainloop tick, which in turn called flush() on
every registered LLEventPump. But the only reason for that mechanism was to
support LLEventQueue. In fact, when LLEventMailDrop overrode its flush()
method for something quite different, it was startling to find that the new
flush() override was being called once per frame -- which caused at least one
fairly mysterious bug. Remove RegisterFlush. Both LLEventPumps::flush() and
LLEventPump::flush() remain for now, though intended usage is unclear.
Eliminating LLEventQueue means we must at least repurpose
LLEventPumps::mQueueNames, a map intended to make LLEventPumps::obtain()
instantiate an LLEventQueue rather than the default LLEventPump. Replace it
with mFactories, a map from desired instance name to a callable returning
LLEventPump*. New map initialization syntax plus lambda support allows us to
populate that map at compile time with little lambdas returning the correct
subclass instance.
Similarly, LLLeapListener::newpump() used to check the ["type"] entry in the
LLSD request specifically for "LLEventQueue". Introduce another such map in
llleaplistener.cpp for potential future extensibility.
Eliminate the LLEventQueue-specific test.
LLEventDetail::visit_and_connect() promised special treatment for the
specific case when an LLEventPump::listen() listener was composed of (possibly
nested) boost::bind() objects storing boost::weak_ptr values -- specifically
boost::bind() rather than std::bind or lambdas, specifically boost::weak_ptr
rather than std::weak_ptr.
Outside of self-tests, it does not appear that anyone actually uses that
support.
There is good reason not to: it's a silent side effect of a complicated
compile-time inspection that could be silently derailed by use of std::bind()
or a lambda or a std::weak_ptr. Can you be sure you've engaged that promise?
How?
A more robust guarantee can be achieved by storing an LLTempBoundConnection in
the transient object itself. When the object is destroyed, the listener is
disconnected. Normal C++ rules around object destruction guarantee it. This
idiom is widely used.
There are a couple good reasons to remove the visit_and_connect() machinery:
* boost::bind() and boost::weak_ptr do not constitute the wave of the future.
Preferring those constructs to lambdas and std::weak_ptr penalizes new code,
whether by silently failing or by discouraging use of modern idioms.
* The visit_and_connect() machinery was always complicated, and apparently
never very robust. Most of its promised features have been commented out
over the years. Making the code base simpler, clearer and more maintainable
is always a useful effect.
LLEventDetail::visit_and_connect() was also used by the four
LLNotificationChannelBase::connectMumble() methods. Streamline those as well.
Of course, remove related test code.
The comments within indra/test/test.cpp promise that --debug is, in fact, like
LOGTEST=DEBUG. Until now, that was a lie. LOGTEST=level displayed log output
on stderr as well as in testprogram.log, while --debug did not.
Add LLError::logToStderr() function, and make initForApplication() (i.e.
commonInit()) call that instead of instantiating RecordToStderr inline. Also
call it when test.cpp recognizes --debug switch.
Remove the mFileRecorder, mFixedBufferRecorder and mFileRecorderFileName
members from SettingsConfig. That tactic doesn't scale.
Instead, add findRecorder<RECORDER>() and removeRecorder<RECORDER>() template
functions to locate (or remove) a RecorderPtr to an object of the specified
subclass. Both are based on an underlying findRecorderPos<RECORDER>() template
function. Since we never expect to manage more than a handful of RecorderPtrs,
and since access to the deleted members is very much application setup rather
than any kind of ongoing access, a search loop suffices.
logToFile() uses removeRecorder<RecordToFile>() rather than removing
mFileRecorder (the only use of mFileRecorder).
logToFixedBuffer() uses removeRecorder<RecordToFixedBuffer>() rather than
removing mFixedBufferRecorder (the only use of mFixedBufferRecorder).
Make RecordToFile store the filename with which it was instantiated. Add a
getFilename() method to retrieve it. logFileName() is now based on
findRecorder<RecordToFile>() instead of mFileRecorderFileName (the only use of
mFileRecorderFileName).
Make RecordToStderr::mUseANSI a simple bool rather than a three-state enum,
and set it immediately on construction. Apparently the reason it was set
lazily was because it consults its own checkANSI() method, and of course
'this' doesn't acquire the leaf class type until the constructor has completed
successfully. But since nothing in checkANSI() depends on anything else in
RecordToStderr, making it static solves that problem.
Sync is specifically intended for test programs. It is based on an
LLScalarCond<int>. The idea is that each of two coroutines can watch for the
other to get a chance to run, indicated by incrementing the wrapped int and
notifying the wrapped condition_variable. This is less hand-wavy than calling
llcoro::suspend() and hoping that the other routine will have had a chance to
run.
Use Sync in lleventcoro_test.cpp.
Also refactor lleventcoro_test.cpp so that instead of a collection of static
data requiring a clear() call at start of each individual test function, the
relevant data is all part of the test_data struct common to all test
functions. Make the helper coroutine functions members of test_data too.
Introduce llcoro::logname(), a convenience function to log the name of the
currently executing coroutine or "main" if in the thread's main coroutine.
This allows the io.cpp test to listen only on the localhost loopback, avoiding
the macOS 10.13.6 "allow listening for incoming connections" popup while
running build-time tests that might halt an unattended TeamCity build.
Longtime fans will remember that the "dcoroutine" library is a Google Summer
of Code project by Giovanni P. Deretta. He originally called it
"Boost.Coroutine," and we originally added it to our 3p-boost autobuild
package as such. But when the official Boost.Coroutine library came along
(with a very different API), and we still needed the API of the GSoC project,
we renamed the unofficial one "dcoroutine" to allow coexistence.
The "dcoroutine" library had an internal low-level API more or less analogous
to Boost.Context. We later introduced an implementation of that internal API
based on Boost.Context, a step towards eliminating the GSoC code in favor of
official, supported Boost code.
However, recent versions of Boost.Context no longer support the API on which
we built the shim for "dcoroutine." We started down the path of reimplementing
that shim using the current Boost.Context API -- then realized that it's time
to bite the bullet and replace the "dcoroutine" API with the Boost.Fiber API,
which we've been itching to do for literally years now.
Naturally, most of the heavy lifting is in llcoros.{h,cpp} and
lleventcoro.{h,cpp} -- which is good: the LLCoros layer abstracts away most of
the differences between "dcoroutine" and Boost.Fiber.
The one feature Boost.Fiber does not provide is the ability to forcibly
terminate some other fiber. Accordingly, disable LLCoros::kill() and
LLCoprocedureManager::shutdown(). The only known shutdown() call was in
LLCoprocedurePool's destructor.
We also took the opportunity to remove postAndSuspend2() and its associated
machinery: FutureListener2, LLErrorEvent, errorException(), errorLog(),
LLCoroEventPumps. All that dual-LLEventPump stuff was introduced at a time
when the Responder pattern was king, and we assumed we'd want to listen on one
LLEventPump with the success handler and on another with the error handler. We
have never actually used that in practice. Remove associated tests, of course.
There is one other semantic difference that necessitates patching a number of
tests: with "dcoroutine," fulfilling a future IMMEDIATELY resumes the waiting
coroutine. With Boost.Fiber, fulfilling a future merely marks the fiber as
ready to resume next time the scheduler gets around to it. To observe the test
side effects, we've inserted a number of llcoro::suspend() calls -- also in
the main loop.
For a long time we retained a single unit test exercising the raw "dcoroutine"
API. Remove that.
Eliminate llcoro_get_id.{h,cpp}, which provided llcoro::get_id(), which was a
hack to emulate fiber-local variables. Since Boost.Fiber has an actual API for
that, remove the hack.
In fact, use (new alias) LLCoros::local_ptr for LLSingleton's dependency
tracking in place of llcoro::get_id().
In CMake land, replace BOOST_COROUTINE_LIBRARY with BOOST_FIBER_LIBRARY. We
don't actually use the Boost.Coroutine for anything (though there exist
plausible use cases).
On Windows, when logged in with a non-ASCII username, every one of the three
documented APIs -- SHGetSpecialFolderPath(), SHGetFolderPath() and
SHGetKnownFolderPath() -- fails to retrieve any pathname at all. We cannot
account for the fact that the oldest of these continues to work with the
release viewer and within a Python script (though not, curiously, from a
Python interactive session). With a non-ASCII username, they consistently fail
when called from an Alex Ivy viewer build: "The filename, directory name, or
volume label syntax is incorrect."
Empirically, with a non-ASCII username, the preset APPDATA and LOCALAPPDATA
environment variables are also useless, e.g. c:\Users\??????\AppData\Roaming
where those are, yup, actual question marks.
Empirically, the VMP is able to successfully call SHGetFolderPath() to
retrieve both AppData\Roaming and AppData\Local. Therefore, we make the VMP
set the APPDATA and LOCALAPPDATA environment variables to the UTF-8 encoded
correct pathnames. Instead of calling SHGetSomethingFolderPath() at all, make
LLDir_Win32 retrieve those environment variables.
Make LLFile::mkdir() treat "directory already exists" as a success case. Every
single call fell into one of two categories: either it didn't check success at
all, or it tested specially to exempt errno == EEXIST. Migrate that test into
mkdir(); eliminate it from call sites.
Make LLDir::append() and add() convenience functions accept variadic
arguments. Replace add(add()...) constructs, as well as clumsy concatenations
of directory names and getDirDelimiter(), with simple variadic add() calls.
When std::istream::good() returns false, presumably we can no longer rely on
get() returning valid data. Certain streamtools tests were assuming that get()
would return the empty string at EOF, but in fact it appears that it left the
previous buffer contents unmodified.
The LLApp API used to consist of init(), mainLoop(), cleanup() methods. This
makes sense -- but on Mac that structure was being subverted. The method
called mainLoop() was in fact being called once per frame. There was
initialization code in the method, which (on Mac) needed to be skipped with an
already-initialized bool. There was a 'while' loop which (on Mac) needed to be
turned into an 'if' instead so the method would return after every frame.
Rename LLApp::mainLoop() to frame(). Propagate through subclasses LLAppViewer
and LLCrashLogger. Document the fact that frame() returns true to mean "done."
(This was always the case, but had to be inferred from the code.)
Rename the Mac Objective-C function mainLoop to oneFrame. Rename the C++ free
function it calls from runMainLoop() to pumpMainLoop(). Add comments to
llappdelegate-objc.mm explaining (inferred) control flow.
Change the Linux viewer main() and the Windows viewer WINMAIN() from a single
LLAppViewer::mainLoop() call to repeatedly call frame() until it returns true.
Move initialization code from the top of LLAppViewer::frame() to the init()
method, where it more properly belongs. Remove corresponding
mMainLoopInitialized flag (and all references) from LLAppViewer.
Remove 'while (! LLApp::isExiting())' (or on Mac, 'if (! LLApp::isExiting())')
from LLAppViewer::frame() -- thus unindenting the whole body of the 'while'
and causing many lines of apparent change. (Apologies to reviewers.)
There are four LLApp states: APP_STATUS_RUNNING, APP_STATUS_QUITTING,
APP_STATUS_STOPPED and APP_STATUS_ERROR. Change LLAppViewer::frame() return
value from (isExiting()) (QUITTING or ERROR) to (! isRunning()). I do not know
under what circumstances the state might transition to STOPPED during a
frame() call, but I'm quite sure that if it does, we don't want to call
frame() again. We only want a subsequent call if the state is RUNNING.
Also rename mainLoop() method in LLCrashLogger subclasses
LLCrashLoggerWindows, LLCrashLoggerMac, LLCrashLoggerLinux. Of course it's
completely up to the frame() method whether to yield control; none of those in
fact do. Honor protocol by returning true (frame() is done), even though each
one's main() caller ignores the return value.
In fact LLCrashLoggerWindows::mainLoop() wasn't using the return protocol
correctly anyway, returning wParam or 0 or 1 -- possibly because the return
protocol was never explicitly documented. It should always return true: "I'm
done, don't call me again."
Nat Goodspeed