Mesh repo is using three policy classes now: one for
large objects, one for GetMesh2 regions, one for
GetMesh regions. It's also detecting the presence
of the cap and using the correct class. Class
initialization cleaned up significantly in llappcorehttp
using data-directed code. Pulled in the changes to
HttpHeader done for sunshine-internal then did a
refactoring pass on the header callback which now
uses a unified approach to clean up and deliver
header information to all interested parties. Added
support for using Retry-After header information on
503 retries.
Added second mesh class as well as an asset upload class.
Refactored initialization to use less code and more data to
cleanly get http started. Modified mesh to use the new
http class for large requests (>2MB for now). Added additional
timeout setting to llcorehttp to distinguish connection timeout
from transport timeout and are now using transport timeout
values for large asset downloads that may need more time.
When releasing HTTP waiters, avoid unnecessary sort activity.
For Content-Type in responses, let libcurl do the work and removed
my parsing of headers. Drop Content-Encoding as libcurl will deal
with that. If anyone is interested, they can parse.
HttpResponse object now has two strings for these content headers.
Either or both may be empty. Tidied up the cross-platform string
code and got more defensive about the length of a header line.
Integration test for the new response object.
Pretty straightforward. Still don't like how I'm managing
the options block. Struct? Accessors? Can't decide. But
the options now speed up the unit test runs even as I add
tests.
Non-LLThread-based threads need to do some registration or LLMutex locks taken out in these
threads will not work as expected (SH-3154). We'll get a better solution later, this fixes
some things for now. Tracing of operations now supported. Global and per-request (via
HttpOptions) tracing levels of [0..3]. The 2 and 3 levels use libcurl's VERBOSE mode
combined with CURLOPT_DEBUGFUNCTION to stream high levels of detail into the log. *Very*
laggy but useful. Simple GET request supported (no Range: header). Really just a
degenrate case of a ranged get but supplied an API anyway. Global option to use the
LLProxy interface to setup CURL handles for either socks5 or http proxy usage. This
isn't really the most encapsulated way to do this but a better solution will have to
come later. The wantHeaders and tracing options are now supported in HttpOptions giving
per-request controls. Big refactoring of the HTTP resource waiter in lltexturefetch.
What I was doing before wasn't correct. Instead, I'm implementing the resource wait
after the Semaphore model (though not using system semaphores). So instead of having
a sequence like: SEND_HTTP_REQ -> WAIT_HTTP_RESOURCE -> SEND_HTTP_REQ, we now
do WAIT_HTTP_RESOURCE -> WAIT_HTTP_RESOURCE2 (actual wait) -> SEND_HTTP_REQ. Works
well but the prioritized filling of the corehttp library needs some performance
work later.
Implemented/modified PUT & POST to not used chunked encoding for the request.
Made the unit test much happier and probably a better thing for the pipeline.
Have a cheesy static & dynamic proxy capability using both local options and
a way to wire into LLProxy in llmessages. Not a clean thing but it will get
the proxy path working with both socks5 & http proxies. Refactoring to get
rid of unneeded library handler and unified an HttpStatus return for all
requests. Big batch of code removed as a result of that and more is possible
as well as some syscall avoidance with a bit more work. Boosted the unit
tests for simple PUT & POST test which revealed the test harness does *not*
like chunked encoding so we'll avoid it for now (and don't really need it
in any of our schemes).
surprised me. Added a retry queue similar to ready queue to the
policy object which is sorted by retry time. Currently do five
retries (after the initial try) delayed by .25, .5, 1, 2 and 5
seconds. Removed the retry logic from the lltexturefetch module.
Upped the waiting time in the unit test for the retries. People
won't like this but tough, need tests.
Identified and reacted to the priority inversion problem we
have in texturefetch. Includes the introduction of a priority_queue
for the requests that are ready. Start some parameterization in
anticipation of having policy_class everywhere. Removed _assert.h
which isn't really needed in indra codebase. Implemented async
setPriority request (which I hope I can get rid of eventually along
with all priorities in this library). Converted to using unsigned
int for priority rather than float. Implemented POST and did
groundwork for PUT.
This is the first functional viewer pass with the HTTP work of the texture fetch
code performed by the llcorehttp library. Not exactly a 'drop-in' replacement
but a work-alike with some changes (e.g. handler notification in consumer
thread versus responder notification in worker thread).
This also includes some temporary changes in the priority scheme to prevent
the kind of priority inversion found in VWR-28996. Scheme used here does
provide liveness if not optimal responsiveness or order-of-operation.
The llcorehttp library at this point is far from optimally performing.
Its worker thread is making relatively poor use of cycles it gets and
it doesn't idle or sleep intelligently yet. This early integration step
helps shake out the interfaces, implementation niceties will be covered
soon.
while allocating the first easy handle in a descent of the global initiailization
code but that doesn't seem to be a problem on TC machines. Perhaps the static
linking is creating multiple data copies. More work needed.
Andrey Lihatskiy