connections. Convert background and foreground fetches, both
items and folders/inventory and library, to use new HTTP.
Non-fetch inventory operations continue to use LLHTTPClient
(at least for now). Error handling and retry on fetches wasn't
100% previously and that's still the case. I'll rip through
this again to clean that up. Cleaned up logging in much of
the inventory code with consistent labels on logging events
and correct macros (removed deprecation warnings).
This started as an attempt to get libcurl to do pipelining
on POSTs and PUTs. Discovered that this is going to be
very difficult to support in general in libcurl. May
look at that again in the future.
some problems disabling pipelining on a multi handle with outstanding
requests so build a more conservative system that allows requests
to drain before setting curl multi options. Would rather not have
this but it is significantly safer. "HttpPipelining" debug setting
is now fully dynamic. Connection limits can also be made dynamic
in the near future. Upped the default connection count back to 8 for
now but will revisit this in the tuning phase. It might be time to
combine mesh and textures into a single asset class. For normal
server operations that would be a clear path, but for server under
load, the current scheme may be better. Minor cleanup in logging
to elminate some redundant strings. Might add some more tracing to the
stall logic 'just in case'.
data so their isn't an opportunity for gaps over overruns (init_data).
Start some preliminary tweaking of policy class numbers. It looks
like I can easily drop the default connection count to '4' and
still hit the throttles. Did some experiments running pipeline
deeper which was mostly fine for textures but tended to slow
meshes. Reason uncertain but a depth of '5' seems generally healthy
for mesh. I had one run of 52.6S with a theoretical minimum of 51.2S.
That's as good as I've ever seen.
GetTexture and GetMesh2 at a pipeline depth of 5. Create
global debug option, HttpPipelining, to enable and disable
HTTP pipelining (defaults to true). Tweak texture and
mesh low- and high-water request levels based on pipelining
status and depth. Fixup texture console which was damaged
in a recent release. Split logging of the no-request
HTTP error case into two cases: one for missing URL in
HTTP request, one for HTTP request not created. A refactor
in llcorehttp is coming: I will be moving all libcurl-
using code into libcurl-specific modules.
Last bit for this release. Describe stream adapters and how
to select a policy class. Slight changes to setup code to
make reality reflect documentation.
While linking GetMesh2 to the old setting was simpler from a user
point-of-view, they really shouldn't be linked and the old one will
go away. This one may be renamed to AssetMaxConcurrentRequests or
something similar if we get to the mesh/texture unification step.
This really extended into the client-side request throttling.
Moved this from llmeshrepository (which doesn't really want
to do connection management) into llcorehttp. It's now a
class option with configurable rate. This still isn't the
right thing to do as it creates coupling between viewer
and services. When we get to pipelining, this notion becomes
invalid.
Much improved. Unified the global and class options into a single
option list. Implemented static and dynamic setting paths as much
as possible. Dynamic path does require packet/RPC but otherwise
there's near unification. Dynamic modes can't get values back yet
due to the response/notifier scheme but this doesn't bother me.
Flatten global and class options into simpler struct-like entities.
Setter/getter available on these when needed (external APIs) but code
can otherwise fiddle directly when it knows what to do. Much duplicated
options/state removed from HttpPolicy. Comments cleaned up. Threads
better described and consistently mentioned in API docs. Integration
test extended for 503 responses with Reply-After headers.
Taught llappcorehttp to register signals on the settings values
that chagne behavior. Have initialization and settings changes
sweep through settings and change them. Dynamic changes are tried
but have no effect (produce a warning message) as dynamic settings
still aren't supported but the plumbing is now connected. Just
need to change llcorehttp. Bounced the 'teleport started' signal
around and it ended up back where it started with some cleanup.
This is making me less angry...
Mesh repo is using three policy classes now: one for
large objects, one for GetMesh2 regions, one for
GetMesh regions. It's also detecting the presence
of the cap and using the correct class. Class
initialization cleaned up significantly in llappcorehttp
using data-directed code. Pulled in the changes to
HttpHeader done for sunshine-internal then did a
refactoring pass on the header callback which now
uses a unified approach to clean up and deliver
header information to all interested parties. Added
support for using Retry-After header information on
503 retries.
Added second mesh class as well as an asset upload class.
Refactored initialization to use less code and more data to
cleanly get http started. Modified mesh to use the new
http class for large requests (>2MB for now). Added additional
timeout setting to llcorehttp to distinguish connection timeout
from transport timeout and are now using transport timeout
values for large asset downloads that may need more time.
Bleh, had some old initialization code in place that meant I was using
32 connections. (Always verify with 'netstat'...) Logic is now to
use 1/4 of MeshMaxConncurrentRequests to head in the direction of 8
at a time. Full count is used to implement a high-water level keeping
llcorehttp in work.
Initial work completed on linux, moving over to windows to do debug
and refinement. This includes 5/6 handlers based on existing responders
and use of llcorehttp for the mesh header fetch.
llcorehttp implements a nice trace facility but it hasn't been attached to
an external control to date. This hands over control to a *non-persistent*
setting that can be used for QA or field diagnostics.
Isolate llcorehttp initialization into a utility class (LLAppCoreHttp)
that provides glue between app and library (sets up policies, handles
notifications). Introduce 'TextureFetchConcurrency' debug setting to
provide some field control when absolutely necessary.
Monty Brandenberg