672 lines
27 KiB
Plaintext
672 lines
27 KiB
Plaintext
|
|
|
|
|
|
1. HTTP Fetching in 15 Minutes
|
|
|
|
Let's start with a trivial working example. You'll need a throwaway
|
|
build of the viewer. And we'll use indra/newview/llappviewer.cpp as
|
|
the host module for these hacks.
|
|
|
|
First, add some headers:
|
|
|
|
|
|
#include "httpcommon.h"
|
|
#include "httprequest.h"
|
|
#include "httphandler.h"
|
|
|
|
|
|
You'll need to derive a class from HttpHandler (not HttpHandle).
|
|
This is used to deliver notifications of HTTP completion to your
|
|
code. Place it near the top, before LLDeferredTaskList, say:
|
|
|
|
|
|
class MyHandler : public LLCore::HttpHandler
|
|
{
|
|
public:
|
|
MyHandler()
|
|
: LLCore::HttpHandler()
|
|
{}
|
|
|
|
virtual void onCompleted(LLCore::HttpHandle /* handle */,
|
|
LLCore::HttpResponse * /* response */)
|
|
{
|
|
LL_INFOS("Hack") << "It is happening again." << LL_ENDL;
|
|
|
|
delete this; // Last statement
|
|
}
|
|
};
|
|
|
|
|
|
Add some statics up there as well:
|
|
|
|
|
|
// Our request object. Allocate during initialiation.
|
|
static LLCore::HttpRequest * my_request(NULL);
|
|
|
|
// The policy class for HTTP traffic.
|
|
// Use HttpRequest::DEFAULT_POLICY_ID, but DO NOT SHIP WITH THIS VALUE!!
|
|
static LLCore::HttpRequest::policy_t my_policy(LLCore::HttpRequest::DEFAULT_POLICY_ID);
|
|
|
|
// Priority for HTTP requests. Use 0U.
|
|
static LLCore::HttpRequest::priority_t my_priority(0U);
|
|
|
|
|
|
In LLAppViewer::init() after mAppCoreHttp.init(), create a request object:
|
|
|
|
|
|
my_request = new LLCore::HttpRequest();
|
|
|
|
|
|
In LLAppViewer::mainLoop(), just before entering the while loop,
|
|
we'll kick off one HTTP request:
|
|
|
|
|
|
// Construct a handler object (we'll use the heap this time):
|
|
MyHandler * my_handler = new MyHandler;
|
|
|
|
// Issue a GET request to 'http://www.example.com/' kicking off
|
|
// all the I/O, retry logic, etc.
|
|
LLCore::HttpHandle handle;
|
|
handle = my_request->requestGet(my_policy,
|
|
my_priority,
|
|
"http://www.example.com/",
|
|
NULL,
|
|
NULL,
|
|
my_handler);
|
|
if (LLCORE_HTTP_HANDLE_INVALID == handle)
|
|
{
|
|
LL_WARNS("Hack") << "Failed to launch HTTP request. Try again."
|
|
<< LL_ENDL;
|
|
}
|
|
|
|
|
|
Finally, arrange to periodically call update() on the request object
|
|
to find out when the request completes. This will be done by
|
|
calling the onCompleted() method with status information and
|
|
response data from the HTTP operation. Add this to the
|
|
LLAppViewer::idle() method after the ping:
|
|
|
|
|
|
my_request->update(0);
|
|
|
|
|
|
That's it. Build it, run it and watch the log file. You should get
|
|
the "It is happening again." message indicating that the HTTP
|
|
operation completed in some manner.
|
|
|
|
|
|
2. What Does All That Mean
|
|
|
|
MyHandler/HttpHandler. This class replaces the Responder-style in
|
|
legacy code. One method is currently defined. It is used for all
|
|
request completions, successful or failed:
|
|
|
|
|
|
void onCompleted(LLCore::HttpHandle /* handle */,
|
|
LLCore::HttpResponse * /* response */);
|
|
|
|
|
|
The onCompleted() method is invoked as a callback during calls to
|
|
HttpRequest::update(). All I/O is completed asynchronously in
|
|
another thread. But notifications are polled by calling update()
|
|
and invoking a handler for completed requests.
|
|
|
|
In this example, the invocation also deletes the handler (which is
|
|
never referenced by the llcorehttp code again). But other
|
|
allocation models are possible including handlers shared by many
|
|
requests, stack-based handlers and handlers mixed in with other,
|
|
unrelated classes.
|
|
|
|
LLCore::HttpRequest(). Instances of this class are used to request
|
|
all major functions of the library. Initialization, starting
|
|
requests, delivering final notification of completion and various
|
|
utility operations are all done via instances. There is one very
|
|
important rule for instances:
|
|
|
|
Request objects may NOT be shared between threads.
|
|
|
|
my_priority. The APIs support the idea of priority ordering of
|
|
requests but it hasn't been implemented and the hope is that this
|
|
will become useless and removed from the interface. Use 0U except
|
|
as noted.
|
|
|
|
my_policy. This is an important one. This library attempts to
|
|
manage TCP connection usage more rigorously than in the past. This
|
|
is done by issuing requests to a queue that has various settable
|
|
properties. These establish connection usage for the queue as well
|
|
as how queues compete with one another. (This is patterned after
|
|
class-based queueing used in various networking stacks.) Several
|
|
classes are pre-defined. Deciding when to use an existing class and
|
|
when to create a new one will determine what kind of experience
|
|
users have. We'll pick up this question in detail below.
|
|
|
|
requestGet(). Issues an ordinary HTTP GET request to a given URL
|
|
and associating the request with a policy class, a priority and an
|
|
response handler. Two additional arguments, not used here, allow
|
|
for additional headers on the request and for per-request options.
|
|
If successful, the call returns a handle whose value is other than
|
|
LLCORE_HTTP_HANDLE_INVALID. The HTTP operation is then performed
|
|
asynchronously by another thread without any additional work by the
|
|
caller. If the handle returned is invalid, you can get the status
|
|
code by calling my_request->getStatus().
|
|
|
|
update(). To get notification that the request has completed, a
|
|
call to update() will invoke onCompleted() methods.
|
|
|
|
|
|
3. Refinements, Necessary and Otherwise
|
|
|
|
MyHandler::onCompleted(). You'll want to do something useful with
|
|
your response. Distinguish errors from successes and getting the
|
|
response body back in some form.
|
|
|
|
Add a new header:
|
|
|
|
|
|
#include "bufferarray.h"
|
|
|
|
|
|
Replace the existing MyHandler::onCompleted() definition with:
|
|
|
|
|
|
virtual void onCompleted(LLCore::HttpHandle /* handle */,
|
|
LLCore::HttpResponse * response)
|
|
{
|
|
LLCore::HttpStatus status = response->getStatus();
|
|
if (status)
|
|
{
|
|
// Successful request. Try to fetch the data
|
|
LLCore::BufferArray * data = response->getBody();
|
|
|
|
if (data && data->size())
|
|
{
|
|
// There's some data. A BufferArray is a linked list
|
|
// of buckets. We'll create a linear buffer and copy
|
|
// the data into it.
|
|
size_t data_len = data->size();
|
|
char * data_blob = new char [data_len + 1];
|
|
data->read(0, data_blob, data_len);
|
|
data_blob[data_len] = '\0';
|
|
|
|
// Process the data now in NUL-terminated string.
|
|
// Needs more scrubbing but this will do.
|
|
LL_INFOS("Hack") << "Received: " << data_blob << LL_ENDL;
|
|
|
|
// Free the temporary data
|
|
delete [] data_blob;
|
|
}
|
|
}
|
|
else
|
|
{
|
|
// Something went wrong. Translate the status to
|
|
// a meaningful message.
|
|
LL_WARNS("Hack") << "HTTP GET failed. Status: "
|
|
<< status.toTerseString()
|
|
<< ", Reason: " << status.toString()
|
|
<< LL_ENDL;
|
|
}
|
|
|
|
delete this; // Last statement
|
|
}
|
|
|
|
|
|
HttpHeaders. The header file "httprequest.h" documents the expected
|
|
important headers that will go out with the request. You can add to
|
|
these by including an HttpHeaders object with the requestGet() call.
|
|
These are typically setup once as part of init rather than
|
|
dynamically created.
|
|
|
|
Add another header:
|
|
|
|
|
|
#include "httpheaders.h"
|
|
|
|
|
|
In LLAppViewer::mainLoop(), add this alongside the allocation of
|
|
my_handler:
|
|
|
|
|
|
// Additional headers for all requests
|
|
LLCore::HttpHeaders * my_headers = new LLCore::HttpHeaders();
|
|
my_headers->append("Accept", "text/html, application/llsd+xml");
|
|
|
|
|
|
HttpOptions. Options are similar and include a mix of value types.
|
|
One interesting per-request option is the trace setting. This
|
|
enables various debug-type messages in the log file that show the
|
|
progress of the request through the library. It takes values from
|
|
zero to three with higher values giving more verbose logging. We'll
|
|
use '2' and this will also give us a chance to verify that
|
|
HttpHeaders works as expected.
|
|
|
|
Same as above, a new header:
|
|
|
|
|
|
#include "httpoptions.h"
|
|
|
|
|
|
And in LLAppView::mainLoop():
|
|
|
|
|
|
// Special options for requests
|
|
LLCore::HttpOptions * my_options = new LLCore::HttpOptions();
|
|
my_options->setTrace(2);
|
|
|
|
|
|
Now let's put that all together into a more complete requesting
|
|
sequence. Replace the existing invocation of requestGet() with this
|
|
slightly more elaborate block:
|
|
|
|
|
|
LLCore::HttpHandle handle;
|
|
handle = my_request->requestGet(my_policy,
|
|
my_priority,
|
|
"http://www.example.com/",
|
|
my_options,
|
|
my_headers,
|
|
my_handler);
|
|
if (LLCORE_HTTP_HANDLE_INVALID == handle)
|
|
{
|
|
LLCore::HttpStatus status = my_request->getStatus();
|
|
|
|
LL_WARNS("Hack") << "Failed to request HTTP GET. Status: "
|
|
<< status.toTerseString()
|
|
<< ", Reason: " << status.toString()
|
|
<< LL_ENDL;
|
|
|
|
delete my_handler; // No longer needed.
|
|
my_handler = NULL;
|
|
}
|
|
|
|
|
|
Build, run and examine the log file. You'll get some new data with
|
|
this run. First, you should get the www.example.com home page
|
|
content:
|
|
|
|
|
|
----------------------------------------------------------------------------
|
|
2013-09-17T20:26:51Z INFO: MyHandler::onCompleted: Received: <!doctype html>
|
|
<html>
|
|
<head>
|
|
<title>Example Domain</title>
|
|
|
|
<meta charset="utf-8" />
|
|
<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
|
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
|
<style type="text/css">
|
|
body {
|
|
background-color: #f0f0f2;
|
|
margin: 0;
|
|
padding: 0;
|
|
font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
|
|
|
|
}
|
|
div {
|
|
width: 600px;
|
|
margin: 5em auto;
|
|
padding: 50px;
|
|
background-color: #fff;
|
|
border-radius: 1em;
|
|
}
|
|
a:link, a:visited {
|
|
color: #38488f;
|
|
text-decoration: none;
|
|
}
|
|
@media (max-width: 700px) {
|
|
body {
|
|
background-color: #fff;
|
|
}
|
|
div {
|
|
width: auto;
|
|
margin: 0 auto;
|
|
border-radius: 0;
|
|
padding: 1em;
|
|
}
|
|
}
|
|
</style>
|
|
</head>
|
|
|
|
<body>
|
|
<div>
|
|
<h1>Example Domain</h1>
|
|
<p>This domain is established to be used for illustrative examples in documents. You may use this
|
|
domain in examples without prior coordination or asking for permission.</p>
|
|
<p><a href="http://www.iana.org/domains/example">More information...</a></p>
|
|
</div>
|
|
</body>
|
|
</html>
|
|
----------------------------------------------------------------------------
|
|
|
|
|
|
You'll also get a detailed trace of the HTTP operation itself. Note
|
|
the HEADEROUT line which shows the additional header added to the
|
|
request.
|
|
|
|
|
|
----------------------------------------------------------------------------
|
|
HttpService::processRequestQueue: TRACE, FromRequestQueue, Handle: 086D3148
|
|
HttpLibcurl::addOp: TRACE, ToActiveQueue, Handle: 086D3148, Actives: 0, Readies: 0
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: TEXT, Data: About to connect() to www.example.com port 80 (#0)
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: TEXT, Data: Trying 93.184.216.119...
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: TEXT, Data: Connected to www.example.com (93.184.216.119) port 80 (#0)
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: TEXT, Data: Connected to www.example.com (93.184.216.119) port 80 (#0)
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: HEADEROUT, Data: GET / HTTP/1.1 Host: www.example.com Accept-Encoding: deflate, gzip Connection: keep-alive Keep-alive: 300 Accept: text/html, application/llsd+xml
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: HEADERIN, Data: HTTP/1.1 200 OK
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: HEADERIN, Data: Accept-Ranges: bytes
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: HEADERIN, Data: Cache-Control: max-age=604800
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: HEADERIN, Data: Content-Type: text/html
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: HEADERIN, Data: Date: Tue, 17 Sep 2013 20:26:56 GMT
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: HEADERIN, Data: Etag: "3012602696"
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: HEADERIN, Data: Expires: Tue, 24 Sep 2013 20:26:56 GMT
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: HEADERIN, Data: Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: HEADERIN, Data: Server: ECS (ewr/1590)
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: HEADERIN, Data: X-Cache: HIT
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: HEADERIN, Data: x-ec-custom-error: 1
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: HEADERIN, Data: Content-Length: 1270
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: HEADERIN, Data:
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: DATAIN, Data: 256 Bytes
|
|
HttpOpRequest::debugCallback: TRACE, LibcurlDebug, Handle: 086D3148, Type: TEXT, Data: Connection #0 to host www.example.com left intact
|
|
HttpLibcurl::completeRequest: TRACE, RequestComplete, Handle: 086D3148, Status: Http_200
|
|
HttpOperation::addAsReply: TRACE, ToReplyQueue, Handle: 086D3148
|
|
----------------------------------------------------------------------------
|
|
|
|
|
|
4. What Does All That Mean, Part 2
|
|
|
|
HttpStatus. The HttpStatus object encodes errors from libcurl, the
|
|
library itself and HTTP status values. It does this to avoid
|
|
collapsing all non-HTTP error into a single '499' HTTP status and to
|
|
make errors distinct.
|
|
|
|
To aid programming, the usual bool conversions are available so that
|
|
you can write 'if (status)' and the expected thing will happen
|
|
whether it's an HTTP, libcurl or library error. There's also
|
|
provision to override the treatment of HTTP errors (making 404 a
|
|
success, say).
|
|
|
|
Share data, don't copy it. The library was started with the goal of
|
|
avoiding data copies as much as possible. Instead, read-only data
|
|
sharing across threads with atomic reference counts is used for a
|
|
number of data types. These currently are:
|
|
|
|
* BufferArray. Linked list of data blocks/HTTP bodies.
|
|
* HttpHeaders. Shared headers for both requests and responses.
|
|
* HttpOptions. Request-only data modifying HTTP behavior.
|
|
* HttpResponse. HTTP response description given to onCompleted.
|
|
|
|
Using objects of these types requires a few rules:
|
|
|
|
* Constructor always gives a reference to caller.
|
|
* References are dropped with release() not delete.
|
|
* Additional references may be taken out with addRef().
|
|
* Unless otherwise stated, once an object is shared with another
|
|
thread it should be treated as read-only. There's no
|
|
synchronization on the objects themselves.
|
|
|
|
HttpResponse. You'll encounter this mainly in onCompleted() methods.
|
|
Commonly-used interfaces on this object:
|
|
|
|
* getStatus() to return the final status of the request.
|
|
* getBody() to retrieve the response body which may be NULL or
|
|
zero-length.
|
|
* getContentType() to return the value of the 'Content-Type'
|
|
header or an empty string if none was sent.
|
|
|
|
This is a reference-counted object so you can call addRef() on it
|
|
and hold onto the response for an arbitrary time. But you'll
|
|
usually just call a few methods and return from onCompleted() whose
|
|
caller will release the object.
|
|
|
|
BufferArray. The core data representation for request and response
|
|
bodies. In HTTP responses, it's fetched with the getBody() method
|
|
and may be NULL or non-NULL with zero length. All successful data
|
|
handling should check both conditions before attempting to fetch
|
|
data from the object. Data access model uses simple read/write
|
|
semantics:
|
|
|
|
* append()
|
|
* size()
|
|
* read()
|
|
* write()
|
|
|
|
(There is a more sophisticated stream adapter that extends these
|
|
methods and will be covered below.) So, one way to retrieve data
|
|
from a request is as follows:
|
|
|
|
|
|
LLCore::BufferArray * data = response->getBody();
|
|
if (data && data->size())
|
|
{
|
|
size_t data_len = data->size();
|
|
char * data_blob = new char [data_len + 1];
|
|
data->read(0, data_blob, data_len);
|
|
|
|
|
|
HttpOptions and HttpResponse. Really just simple containers of POD
|
|
and std::string pairs. But reference counted and the rule about not
|
|
modifying after sharing must be followed. You'll have the urge to
|
|
change options dynamically at some point. And you'll try to do that
|
|
by just writing new values to the shared object. And in tests
|
|
everything will appear to work. Then you ship and people in the
|
|
real world start hitting read/write races in strings and then crash.
|
|
Don't be lazy.
|
|
|
|
HttpHandle. Uniquely identifies a request and can be used to
|
|
identify it in an onCompleted() method or cancel it if it's still
|
|
queued. But as soon as a request's onCompleted() invocation
|
|
returns, the handle becomes invalid and may be reused immediately
|
|
for new requests. Don't hold on to handles after notification.
|
|
|
|
|
|
5. And Still More Refinements
|
|
|
|
(Note: The following refinements are just code fragments. They
|
|
don't directly fit into the working example above. But they
|
|
demonstrate several idioms you'll want to copy.)
|
|
|
|
LLSD, std::streambuf, std::iostream. The read(), write() and
|
|
append() methods may be adequate for your purposes. But we use a
|
|
lot of LLSD. Its interfaces aren't particularly compatible with
|
|
BufferArray. And so two adapters are available to give
|
|
stream-like behaviors: BufferArrayStreamBuf and BufferArrayStream,
|
|
which implement the std::streambuf and std::iostream interfaces,
|
|
respectively.
|
|
|
|
A std::streambuf interface isn't something you'll want to use
|
|
directly. Instead, you'll use the much friendlier std::iostream
|
|
interface found in BufferArrayStream. This adapter gives you all
|
|
the '>>' and '<<' operators you'll want as well as working
|
|
directly with the LLSD conversion operators.
|
|
|
|
Some new headers:
|
|
|
|
|
|
#include "bufferstream.h"
|
|
#include "llsdserialize.h"
|
|
|
|
|
|
And an updated fragment based on onCompleted() above:
|
|
|
|
|
|
// Successful request. Try to fetch the data
|
|
LLCore::BufferArray * data = response->getBody();
|
|
LLSD resp_llsd;
|
|
|
|
if (data && data->size())
|
|
{
|
|
// There's some data and we expect this to be
|
|
// LLSD. Checking of content type and validation
|
|
// during parsing would be admirable additions.
|
|
// But we'll forgo that now.
|
|
LLCore::BufferArrayStream data_stream(data);
|
|
LLSDSerialize::fromXML(resp_llsd, data_stream);
|
|
}
|
|
LL_INFOS("Hack") << "LLSD Received: " << resp_llsd << LL_ENDL;
|
|
}
|
|
else
|
|
{
|
|
|
|
|
|
Converting an LLSD object into an XML stream stored in a
|
|
BufferArray is just the reverse of the above:
|
|
|
|
|
|
BufferArray * data = new BufferArray();
|
|
LLCore::BufferArrayStream data_stream(data);
|
|
|
|
LLSD src_llsd;
|
|
src_llsd["foo"] = "bar";
|
|
|
|
LLSDSerialize::toXML(src_llsd, data_stream);
|
|
|
|
// 'data' now contains an XML payload and can be sent
|
|
// to a web service using the requestPut() or requestPost()
|
|
// methods.
|
|
... requestPost(...);
|
|
|
|
// And don't forget to release the BufferArray.
|
|
data->release();
|
|
data = NULL;
|
|
|
|
|
|
LLSD will often go hand-in-hand with BufferArray and data
|
|
transport. But you can also do all the streaming I/O you'd expect
|
|
of a std::iostream object:
|
|
|
|
|
|
BufferArray * data = new BufferArray();
|
|
LLCore::BufferArrayStream data_stream(data);
|
|
|
|
data_stream << "Hello, World!" << 29.4 << '\n';
|
|
std::string str;
|
|
data_stream >> str;
|
|
std::cout << str << std::endl;
|
|
|
|
data->release();
|
|
// Actual delete will occur when 'data_stream'
|
|
// falls out of scope and is destructed.
|
|
|
|
|
|
Scoping objects and cleaning up. The examples haven't bothered
|
|
with cleanup of objects that are no longer needed. Instead, most
|
|
objects have been allocated as if they were global and eternal.
|
|
You'll put the objects in more appropriate feature objects and
|
|
clean them up as a group. Here's a checklist for actions you may
|
|
need to take on cleanup:
|
|
|
|
* Call delete on:
|
|
o HttpHandlers created on the heap
|
|
o HttpRequest objects
|
|
* Call release() on:
|
|
o BufferArray objects
|
|
o HttpHeaders objects
|
|
o HttpOptions objects
|
|
o HttpResponse objects
|
|
|
|
On program exit, as threads wind down, the library continues to
|
|
operate safely. Threads don't interact via the library and even
|
|
dangling references to HttpHandler objects are safe. If you don't
|
|
call HttpRequest::update(), handler references are never
|
|
dereferenced.
|
|
|
|
You can take a more thorough approach to wind-down. Keep a list
|
|
of HttpHandles (not HttpHandlers) of outstanding requests. For
|
|
each of these, call HttpRequest::requestCancel() to cancel the
|
|
operation. (Don't add the cancel requests' handled to the list.)
|
|
This will cancel the outstanding requests that haven't completed.
|
|
Canceled or completed, all requests will queue notifications. You
|
|
can now cycle calling update() discarding responses. Continue
|
|
until all requests notify or a few seconds have passed.
|
|
|
|
Global startup and shutdown is handled in the viewer. But you can
|
|
learn about it in the code or in the documentation in the headers.
|
|
|
|
|
|
6. Choosing a Policy Class
|
|
|
|
Now it's time to get rid of the default policy class. Take a look
|
|
at the policy class definitions in newview/llappcorehttp.h.
|
|
Ideally, you'll find one that's compatible with what you're doing.
|
|
Some of the compatibility guidelines are:
|
|
|
|
* Destination: Pair of host and port. Mixing requests with
|
|
different destinations may cause more connection setup and tear
|
|
down.
|
|
|
|
* Method: http or https. Usually moot given destination. But
|
|
mixing these may also cause connection churn.
|
|
|
|
* Transfer size: If you're moving 100MB at a time and you make your
|
|
requests to the same policy class as a lot of small, fast event
|
|
information that fast traffic is going to get stuck behind you
|
|
and someone's experience is going to be miserable.
|
|
|
|
* Long poll requests: These are long-lived, must- do operations.
|
|
They have a special home called AP_LONG_POLL.
|
|
|
|
* Concurrency: High concurrency (5 or more) and large transfer
|
|
sizes are incompatible. Another head-of-the-line problem. High
|
|
concurrency is tolerated when it's desired to get maximal
|
|
throughput. Mesh and texture downloads, for example.
|
|
|
|
* Pipelined: If your requests are not idempotent, stay away from
|
|
anything marked 'soon' or 'yes'. Hidden retries may be a
|
|
problem for you. For now, would also recommend keeping PUT and
|
|
POST requests out of classes that may be pipelined. Support for
|
|
that is still a bit new.
|
|
|
|
If you haven't found a compatible match, you can either create a
|
|
new class (llappcorehttp.*) or just use AP_DEFAULT, the catchall
|
|
class when all else fails. Inventory query operations might be a
|
|
candidate for a new class that supported pipelining on https:.
|
|
Same with display name lookups and other bursty-at-login
|
|
operations. For other things, AP_DEFAULT will do what it can and
|
|
will, in some way or another, tolerate any usage. Whether the
|
|
users' experiences are good are for you to determine.
|
|
|
|
|
|
7. FAQ
|
|
|
|
Q1. What do these policy classes achieve?
|
|
|
|
A1. Previously, HTTP-using code in the viewer was written as if
|
|
it were some isolated, local operation that didn't have to
|
|
consider resources, contention or impact on services and the
|
|
larger environment. The result was an application with on the
|
|
order of 100 HTTP launch points in its codebase that could create
|
|
dozens or even 100's of TCP connections zeroing in on grid
|
|
services and disrupting networking equipment, web services and
|
|
innocent users. The use of policy classes (modeled on
|
|
http://en.wikipedia.org/wiki/Class-based_queueing) is a means to
|
|
restrict connection concurrency, good and necessary in itself. In
|
|
turn, that reduces demands on an expensive resource (connection
|
|
setup and concurrency) which relieves strain on network points.
|
|
That enables connection keepalive and opportunites for true
|
|
improvements in throughput and user experience.
|
|
|
|
Another aspect of the classes is that they give some control over
|
|
how competing demands for the network will be apportioned. If
|
|
mesh fetches, texture fetches and inventory queries are all being
|
|
made at once, the relative weights of their classes' concurrency
|
|
limits established that apportioning. We now have an opportunity
|
|
to balance the entire viewer system.
|
|
|
|
Q2. How's that data sharing with refcounts working for you?
|
|
|
|
A2. Meh. It does reduce memory churn and the frequency at which
|
|
free blocks must be moved between threads. But it's also a design
|
|
for static configuration and dynamic reconfiguration (not
|
|
requiring a restart) is favored. Creating new options for every
|
|
request isn't too bad, it a sequence of "new, fill, request,
|
|
release" for each requested operation. That in contrast to doing
|
|
the "new, fill, release" at startup. The bad comes in getting at
|
|
the source data. One rule in this work was "no new thread
|
|
problems." And one source for those is pulling setting values out
|
|
of gSettings in threads. None of that is thread safe though we
|
|
tend to get away with it.
|
|
|
|
Q3. What needs to be done?
|
|
|
|
A3. There's a To-Do list in _httpinternal.h. It has both large
|
|
and small projects here if someone would like to try changes.
|