Java JNI Interface
First, we analyze the request flow at the application layer. The Java sample code is as follows:
:::Java
Executor executor = Executors.newSingleThreadExecutor();
UrlRequest.Callback callback = new SimpleUrlRequestCallback();
UrlRequest.Builder builder =
new UrlRequest.Builder(url, callback, executor, mCronetEngine);
applyPostDataToUrlRequestBuilder(builder, executor, postData);
builder.build().start();
From the code above, we can see that, based on the builder pattern, UrlRequest.Builder produces a UrlRequest object. This object provides a start() method to initiate the request. After the request is executed, the corresponding callback methods of the UrlRequest.Callback object are invoked on the executor thread, completing a single resource request flow.
Since UrlRequest is in fact only an abstract interface, its actual implementation is in the CronetUrlRequest class. In the start() method implemented by this class, it ultimately uses nativeStart() to invoke a C++ native function and start the URL request process. This nativeStart() method is implemented in CronetUrlRequest_jni.h; it is an auto-generated interface function whose role is to translate a Java call into the corresponding C++ call. In practice, the component that adapts the upper-layer Java code is the CronetURLRequestAdapter class defined in C++ code. At this point, execution is handed off from the upper layer to the lower-level C++ part. Next, we begin analyzing the actual workflow of URLRequest.
C++ Adaptation
At the C++ layer, a single network request corresponds strictly to a URLRequest object. Its constructor is private, which means it can only be constructed by the corresponding builder. In fact, “URLRequests are always created by calling URLRequestContext::CreateRequest”, and the URLRequestContext object also has a corresponding Java wrapper, so we do not need to consider too many details here.
Inside CronetURLRequestAdapter, it directly holds a pointer to this object. Its Start() method POSTs its own StartOnNetworkThread() method to the network thread for execution. Then, in StartOnNetworkThread(), what it does is set some properties on the real core URLRequest object and then call Start() on it to initiate the request.
URLRequest Request Flow
After the URLRequest object calls Start(), it uses URLRequestJobManager's CreateJob() method to create a URLRequestJob object based on the type of the requested resource—i.e., a task entity. In most cases, the resulting object is URLRequestHttpJob. The actual inheritance relationship is shown below.
This object is passed as a parameter to call URLRequest's StartJob() method, but this is really just an extra layer of wrapping; in essence, it is equivalent to calling the Start() method on the URLRequest(Http)Job object. In the body of that method, it mainly performs work such as setting Headers and Cookies. Note that setting Cookies is asynchronous, so operations to be executed after certain tasks complete are wrapped into a set of Closures to be executed as callbacks.
In any case, what we need to focus on here is URLRequestHttpJob::OnCookiesLoaded. In its method body, it actually calls DoStartTransaction() to begin the real content transfer process. However, this method is also just a wrapper: internally it first checks whether the current request has been canceled; if so, it stops further execution. Otherwise, it calls the real StartTransaction() method. From this we can also see an important characteristic of Chromium's asynchronous programming model: you only need to check whether the current task has been canceled, and whether it should continue, each time an asynchronous callback is executed.
StartTransaction() performs some delegate notification operations and then continues by calling StartTransactionInternal(). In this method, it first checks whether an HttpTransaction object was created previously. If so, it indicates the previous request failed and now needs to be retried. If not, it uses the HttpTransactionFactory class's CreateTransaction() method to create a new HttpTransaction object. This uses the factory pattern; the created object is generally an HttpNetworkTransaction. The actual inheritance relationship is shown below:
After that, control flow transfers to the Start() method of the HttpNetworkTransaction object. This method sets some parameters and then enters DoLoop(). At this point, we have entered the HTTP state machine, and what follows will be executed based on this state machine.
For this state machine, there are some details worth noting. Each time execution reaches a blocking operation, it returns an ERR_IO_PENDING status code, which causes the state machine loop to break directly. So how do states continue to transition and execute? In fact, the HttpNetworkTransaction class provides a callback OnIOComplete. After the blocking call finishes, it calls DoLoop() again, thus continuing to the next state.
HTTP State Machine
In the Chromium network stack, all HTTP-related states are rigorously defined as follows:
:::C
enum State {
STATE_NOTIFY_BEFORE_CREATE_STREAM,
STATE_CREATE_STREAM,
STATE_CREATE_STREAM_COMPLETE,
STATE_INIT_STREAM,
STATE_INIT_STREAM_COMPLETE,
STATE_GENERATE_PROXY_AUTH_TOKEN,
STATE_GENERATE_PROXY_AUTH_TOKEN_COMPLETE,
STATE_GENERATE_SERVER_AUTH_TOKEN,
STATE_GENERATE_SERVER_AUTH_TOKEN_COMPLETE,
STATE_GET_TOKEN_BINDING_KEY,
STATE_GET_TOKEN_BINDING_KEY_COMPLETE,
STATE_INIT_REQUEST_BODY,
STATE_INIT_REQUEST_BODY_COMPLETE,
STATE_BUILD_REQUEST,
STATE_BUILD_REQUEST_COMPLETE,
STATE_SEND_REQUEST,
STATE_SEND_REQUEST_COMPLETE,
STATE_READ_HEADERS,
STATE_READ_HEADERS_COMPLETE,
STATE_READ_BODY,
STATE_READ_BODY_COMPLETE,
STATE_DRAIN_BODY_FOR_AUTH_RESTART,
STATE_DRAIN_BODY_FOR_AUTH_RESTART_COMPLETE,
STATE_NONE
};
Among them, states with the "COMPLETE" suffix indicate that the previous operation succeeded and that corresponding callback operations, etc., need to be executed. Therefore, if we need to do HTTP-layer data collection, we need to map the relevant phases onto this state machine, collect the corresponding data, and finally map it back to the URLRequest object in some way. Below, we focus on several phases related to performance metrics and the states they correspond to.
STATE_CREATE_STREAM
This state corresponds to HttpNetworkTransaction's DoCreateStream() method. Depending on the protocol, this method requests different HttpStreams from HttpStreamFactory (actually HttpStreamFactoryImpl), again using the factory pattern. Note that HttpStream is also an abstract class; what is actually involved here is HttpBasicStream. Their inheritance relationship is shown below:
At this point, control flow transfers to the HttpStreamFactoryImpl class. We will briefly focus only on RequestStream(), which continues by directly calling RequestStreamInternal(). Inside this method, it creates an HttpStreamFactoryImpl::Job object to abstractly represent a task of requesting an HttpStream, and calls Start() on this object, so control flow transfers again.
Within the HttpStreamFactoryImpl::Job object, the method that actually takes effect is RunLoop(). It is divided into two parts: the first half runs a state machine DoLoop() used to allocate resources and create an HttpStream; the second half checks whether the creation process succeeded and performs different actions based on different errors. Issues such as SSL exceptions generally prevent opening a Stream, so they are reflected in this state machine and the corresponding error codes. In this state machine, all states are defined as follows:
:::C
enum State {
STATE_START,
STATE_RESOLVE_PROXY,
STATE_RESOLVE_PROXY_COMPLETE,
STATE_WAIT_FOR_JOB,
STATE_WAIT_FOR_JOB_COMPLETE,
STATE_INIT_CONNECTION,
STATE_INIT_CONNECTION_COMPLETE,
STATE_WAITING_USER_ACTION,
STATE_RESTART_TUNNEL_AUTH,
STATE_RESTART_TUNNEL_AUTH_COMPLETE,
STATE_CREATE_STREAM,
STATE_CREATE_STREAM_COMPLETE,
STATE_DRAIN_BODY_FOR_AUTH_RESTART,
STATE_DRAIN_BODY_FOR_AUTH_RESTART_COMPLETE,
STATE_DONE,
STATE_NONE
};
We focus on a subset of these states.
STATE_INIT_CONNECTION
This state corresponds to the HttpStreamFactoryImpl::Job::DoInitConnection() method. Its goal is to initialize one of its member variables of type ClientSocketHandle. What it does is basically set a set of parameters based on the current request configuration. Ultimately, the method it actually calls is InitSocketPoolHelper() in the same class. This function likewise performs various parameter configuration work, and then control flow transfers to ClientSocketHandle::Init() to initialize this ClientSocketHandle object. This object attempts to request a valid Socket from the Socket Pool via ClientSocketPool::RequestSocket(). Note that Chromium actually contains various types of Socket Pools, and their inheritance relationship is as follows:
For a typical HTTP request, the Socket Pool actually used is TransportClientSocketPool. In addition, Socket Pools have the concept of a “group”. This “group” is essentially the mapping between different domain names and different Socket sets. In other words, if you access "www.baidu.com", then when requesting a Socket from the Socket Pool, the group name queried is "www.baidu.com:80". This is because connection_group is generated like this:
:::C++
// Build the string used to uniquely identify connections of this type.
// Determine the host and port to connect to.
std::string connection_group = origin_host_port.ToString();
Back to the main point: the RequestSocket() call above goes through multiple layers, and finally control flow transfers to ClientSocketPoolBaseHelper::RequestSocket(), then enters RequestSocketInternal() to perform the real Socket allocation task. Of course, it is very possible that we cannot find a reusable idle Socket; in that case, we need to use ConnectJobFactory to create a ConnectJob object and call its Connect() method to open a new Socket, i.e., establish the corresponding connection. Note a detail here: if connect_job->Connect() returns ERR_IO_PENDING—meaning the IO process has been initiated but has not completed immediately—Chromium will determine whether it is allowed to enable a backup connection. If allowed, it creates another connection task that starts after 250ms. The role of this task is that if the SYN handshake packet of the first connection is unexpectedly dropped, then during the wait for TCP timeout, this backup connection can be established earlier and take over immediately, significantly reducing connection latency.
In addition, note that ConnectJob is also a pure virtual base class; what we actually use is TransportConnectJob, and its inheritance relationship is as follows:
At this point, control flow is transferred along the Connect() method to TransportConnectJobHelper::DoConnectInternal(), and here it enters the object's state machine DoLoop(). This state machine is very simple; its states are defined as follows:
:::C
enum State {
STATE_RESOLVE_HOST,
STATE_RESOLVE_HOST_COMPLETE,
STATE_TRANSPORT_CONNECT,
STATE_TRANSPORT_CONNECT_COMPLETE,
STATE_NONE,
};
So it is really just two states: resolve the Host first, then initiate a TCP Connect at the transport layer. The Host resolution process here is performed by TransportConnectJobHelper::DoResolveHost(), which calls SingleRequestHostResolver::Resolve() to complete resolution, and what this method actually executes is HostResolverImpl::Resolve(). The inheritance relationship is shown below:
At this point, for the detailed DNS flow that follows, you can refer to my previous post Chromium DNS Flow Analysis.
STATE_CREATE_STREAM
This function is used to open a stream object of the actual type HttpBasicStream, and it creates the HttpBasicState object belonging to that object.
STATE_INIT_STREAM
This state corresponds to HttpNetworkTransaction::DoInitStream(). Compared with the previous state, what it does is relatively simple: it only sets some parameters, such as reading the server IP of the connection and initializing the HttpBasicState object contained in HttpBasicStream. Of course, an important point is that HttpStreamParser is created by the HttpBasicState object in this state.
STATE_INIT_REQUEST_BODY
STATE_BUILD_REQUEST
STATE_SEND_REQUEST
This state corresponds to HttpNetworkTransaction::DoSendRequest(). In practice, it reaches HttpStreamParser::SendRequest() via HttpBasicStream::SendRequest(). It then calls the DoLoop() method contained within HttpStreamParser itself, whose corresponding state machine is defined as follows:
:::C
enum State {
STATE_NONE,
STATE_SEND_HEADERS,
STATE_SEND_HEADERS_COMPLETE,
STATE_SEND_BODY,
STATE_SEND_BODY_COMPLETE,
STATE_SEND_REQUEST_READ_BODY_COMPLETE,
STATE_READ_HEADERS,
STATE_READ_HEADERS_COMPLETE,
STATE_READ_BODY,
STATE_READ_BODY_COMPLETE,
STATE_DONE
};
From this we can see that this object mainly prepares data, and in methods such as HttpStreamParser::DoSendHeaders(), writes data into the StreamSocket, completing the send. Note that StreamSocket is an abstract class; in most cases, what we use is TCPClientSocket. Their inheritance relationship is shown below:
