clang correctly notes that such tests add nothing to the runtime behavior of a
program. Unfortunately, clang notes that in the form of a compile error.
Under pipelining, requests were given a 5x timeout factor due to the
way that the timeout clock works in libcurl. Under CDN load,
connections were not being torn down quickly and it was only
this timer that led to disconnect and retry. So, we want to
break a connection that isn't making progress but that isn't
immediately possible. We'll compromise with a 60S timeout that
(we hope) will be neither too long for stalled connections nor
too short for large asset transfer requests.
Incorporate the new libcurl 7.38.0 build with curl bug 1420
workaround. Add developer-centric testing code to evaluate
the workaround or a future fix for 1420.
on the HTTP requests for inventory. We'll benchmark with that
and see how it goes. Document some of the history of the
background fetcher for future devs. Suggest some future
projects to make things faster. Pointers on using LLSD with
the llcorehttp library in the readme. And restructured
the LLSD onCompleted() processing phases using do{}while(false)
which produced a code flow that is fairly attractive.
as transfers can appear delayed with deep pipelining and more
requests in the pool. Added bad HTTP status error (typically
getting a 0 back as HTTP status from libcurl) to the list of
retryable errors. There's a response stream problem with libcurl
and pipelining that induces this problem. Retrying helps but
may not be entirely safe. Watch bug 1420 on the libcurl sourceforge
bug tracker. Extend options of test/example program to include
un-ranged requests. Document the excessive data transfer induced
when ranged requests are disabled. This is an abnormal mode for
very rare users so we'll just eat that for now.
some problems disabling pipelining on a multi handle with outstanding
requests so build a more conservative system that allows requests
to drain before setting curl multi options. Would rather not have
this but it is significantly safer. "HttpPipelining" debug setting
is now fully dynamic. Connection limits can also be made dynamic
in the near future. Upped the default connection count back to 8 for
now but will revisit this in the tuning phase. It might be time to
combine mesh and textures into a single asset class. For normal
server operations that would be a clear path, but for server under
load, the current scheme may be better. Minor cleanup in logging
to elminate some redundant strings. Might add some more tracing to the
stall logic 'just in case'.
GetTexture and GetMesh2 at a pipeline depth of 5. Create
global debug option, HttpPipelining, to enable and disable
HTTP pipelining (defaults to true). Tweak texture and
mesh low- and high-water request levels based on pipelining
status and depth. Fixup texture console which was damaged
in a recent release. Split logging of the no-request
HTTP error case into two cases: one for missing URL in
HTTP request, one for HTTP request not created. A refactor
in llcorehttp is coming: I will be moving all libcurl-
using code into libcurl-specific modules.
Start using DNS cache in legacy LLCurl code. Go to 15 seconds
particularly as we're using threaded resolver at this point.
Documentation cleanup. Add libcurl status checking and logging
for curl_easy_setopt() operations that fail. Shouldn't happen
and we'll just continue anyway but there's info in the logs to
track these down now. Cleaned up logic around FASTTIMER enable
defines used to evaluate pipeline stalls in main thread.
Removed long-standing thread race around caps strings and
URL construction. Not a significant risk but refactoring the
code to get rid of them removed one huge eyesore. It can be
made even slicker if desired (see notes).
Last bit for this release. Describe stream adapters and how
to select a policy class. Slight changes to setup code to
make reality reflect documentation.
First edit complete. Use the library in 15 minutes. Describe the
code. Refinements to the initial try. Describe that code. Still
to do: more refinements, how to choose a policy class, FAQ.
Added toTerseString() conversion on HttpStatus to generate a string
that's more descriptive than the hex value of the HttpStatus value
but still forms a short, searchable token (e.g. "Http_503" or
"Core_7"). Using this throughout the viewer now, no live cases
of toHex(), I believe.
Added 'MeshUseGetMesh1' and 'MeshUseHttpRetryAfter' debug settings
to control mesh transport behavior. First forces the use of the
legacy mesh fetch style with high concurrency and connection churn.
The second, on by default, honors Retry-After values if they are
reasonable. If off or unreasonable, internal delay times are used.
This really extended into the client-side request throttling.
Moved this from llmeshrepository (which doesn't really want
to do connection management) into llcorehttp. It's now a
class option with configurable rate. This still isn't the
right thing to do as it creates coupling between viewer
and services. When we get to pipelining, this notion becomes
invalid.
earlier maintenance branch but the code never showed up. I'll do
it again. Spam is still available by bumping 'CoreHttp' tag up
to DEBUGS level logging. Needed for QA. Can also get this data
from tracing.
Simple change dropped this value by 7-10mS or so. Any time we
complete an operation on a transport pass, arrange to skip the
run-loop sleep so that we fill a possible empty slot as quickly
as possible. With pipelining, this kind of thing should become
unnecessary but for now, we'll do this to increase throughput.
Generally sorted the mesh timeout parameters for maximum
transport time (staying with default 30 for connect). 60S
for normal meshes, 600S for large. Also documented default
option values in httpoptions.h. Useful to have these. In
the future, the timeouts might go into standard llsd options
where they can be tracked a bit more.
Much improved. Unified the global and class options into a single
option list. Implemented static and dynamic setting paths as much
as possible. Dynamic path does require packet/RPC but otherwise
there's near unification. Dynamic modes can't get values back yet
due to the response/notifier scheme but this doesn't bother me.
Flatten global and class options into simpler struct-like entities.
Setter/getter available on these when needed (external APIs) but code
can otherwise fiddle directly when it knows what to do. Much duplicated
options/state removed from HttpPolicy. Comments cleaned up. Threads
better described and consistently mentioned in API docs. Integration
test extended for 503 responses with Reply-After headers.
termination test. Sheesh. Also get some more numbers out of the
example/load test program which drives traffic. This should
give some useful insights into how the current http throttle
works (or doesn't) with varying client demands.
Pay correct attention to status codes coming back from services. Generate
better and consistent error messages when problems arise. There's more to
do in error handling, need a way to cleanly fail all request types, only
have that for LOD at this point. Do better keeping the HTTP pipeline between
the low and high water marks. This was made challenging because the outer
most code couldn't really see what's going on internally (whose actions are
delayed in a worker thread). More to do here, the debug-like requests don't
honor limits, that will come later. Made retry counts available from llcorehttp
which can be used by the throttle-anticipating logic to advance the count.
It helps but it reinforces the coupling between viewer and server which I
do not like.
Mesh repo is using three policy classes now: one for
large objects, one for GetMesh2 regions, one for
GetMesh regions. It's also detecting the presence
of the cap and using the correct class. Class
initialization cleaned up significantly in llappcorehttp
using data-directed code. Pulled in the changes to
HttpHeader done for sunshine-internal then did a
refactoring pass on the header callback which now
uses a unified approach to clean up and deliver
header information to all interested parties. Added
support for using Retry-After header information on
503 retries.
Tonya Souther