class pipeline :
A pipeline is a queue of HTTP calls to perform
pipeline object is a FIFO queue of HTTP calls. It is called
"pipeline" because it is processed asynchronously: Requests may be
sent to the HTTP server independently of whether responses of the
previous requests already arrived or not.
pipeline object may keep connections to several
servers at once. (More exactly, it has a FIFO queue for every
server it is connected with.)
pipeline object keeps track what is happening, so you need
not to care about the details of communications. The API is
simple: Create a
pipeline object, do some setup (add authentication
methods; configure the proxy to use), add the requests, and
run the pipeline. The rest is done automatically. To get the results,
you can either memorize the requests you wanted to know yourself
and ask every request object about the reply of the server; or
you can specify that a callback function should be called once
the request is processed (with positive or negative result).
It is possible to add further requests to the pipeline from within
these callback functions.
If you want to have several pipelines, or some cooperation with
other network services, you may specify a
For example, to have two pipelines working concurrently:
let ues = Unixqueue.create_unix_event_system() in let p1 = new pipeline in let p2 = new pipeline in p1 # set_event_system ues; p2 # set_event_system ues; Unixqueue.run ues (* run p1 and p2 in parallel *)
This works not only with pipelines, but with every network client
or server which is compatible with the
method event_system :
Returns the event system
method set_event_system :
Unixqueue.event_system -> unit
Sets the event system. Must be called before the first call is added
method connection_cache :
The current connection cache. By default, a private restrictive cache is used.
method set_connection_cache :
connection_cache -> unit
Set the connection cache. This must happen before the first call is added.
method add_auth_handler :
auth_handler -> unit
adds a new-style authentication handler
method set_proxy :
string -> int -> unit
set_proxy name port:
sets that an HTTP proxy
name listening on
port is to be used
method set_proxy_auth :
insecure:bool -> string -> string -> unit
sets user and password for the proxy. Works for both "digest"
and "basic" mechanisms (the latter only if the
flag is set). Any realm is acceptable.
method avoid_proxy_for :
string list -> unit
sets a list of host names or domain suffixes for which no proxy
should be used.
method set_proxy_from_environment :
insecure:bool -> unit -> unit
Inspect the environment variables
and set the proxy options from them. If
insecure is set,
basic authentication is enabled (otherwise only secure auth
methods like Digest).
method set_socks5_proxy :
string -> int -> unit
Sets that a SOCKS version 5 proxy is used at this host and port.
There is no authentication. The
avoid_proxy_for setting is
method configure_transport :
transport_channel_type -> unit
configure_transport id transport: Configures that messages with
transport layer ID
id are exchanged on
By default, there is only a configuration for
Nethttp_client.http_trans_id, i.e. for normal unencrypted channels.
method set_tls_cache :
tls_cache -> unit
Sets the TLS cache (NB. The default cache is
method set_transport_proxy :
int -> (string * string * bool) option -> proxy_type -> unit
set_transport_proxy id host port auth ptype: Sets a special
proxy for the transport identified by
id. This overrides
set_socks5_proxy for the
auth triple contains
method set_transport_proxy_from_environment :
insecure:bool -> (string * transport_layer_id) list -> unit
set_proxy_from_environment, this method inspects environment
variables and configures the proxy settings. This function, however,
is more flexible, and can use different environment variables for
The argument list has pairs
(var_name, id) meaning that the
var_name configures the proxy for
[("http_proxy", http_trans_id); ("https_proxy", https_trans_id)]
means that these two variables are used for the respective transports.
"no_proxy" is interpreted anyway.
insecure: whether basic authentication is enabled.
method reset :
unit -> unit
Empties the pipeline and inactivates any open connection.
The currently active operation is interrupted, and every request
with response is set to
No_reply (i.e. you get the exception
No_reply if you try to access the response).
If there are callbacks for these requests, the callback
functions are invoked.
The queues of open requests and replies are cleared. All
connections to all servers are inactivated.
Inactivation means that open connections are given back to the connection cache for further reuse if the state of the connection allows this; otherwise the connections are closed.
method add :
http_call -> unit
Adds the call to the end of the pipeline. One must not add calls that have already been served.
Adds the call to the end of the pipeline.
After the call has been processed, the callback function
is called. This function is called for every call that
leaves the pipeline, it does not matter whether processing
was successful or not. Invoke
status on the message
to get what happened; either some status information from the
server is available (perhaps OK status), or an exception is
method add_e :
http_call -> http_call Uq_engines.engine
The same as engine: The call
c is added to the pipeline, and
when it is processed, the returned engine transitions to the
method proxy_type :
string -> proxy_type option
proxy_type url returns
Some pt if a proxy would be used for this
None if a direct connection would be made.
method proxy_type_of_call :
http_call -> proxy_type option
Same for an already created call object
method transport_layer :
http_call -> transport_layer_id
Reports the current transport of this call
method run :
unit -> unit
Runs through the requests in the pipeline. If a request can be
fulfilled, i.e. the server sends a response, the state of the
request is set and the request is removed from the pipeline.
If a request cannot be fulfilled (no response, bad response,
network error), the exception is stored in the
object and will be raised once the state of the object is
Under certain conditions (serious network errors)
not catch the exception; it simply cleans its own state up
(aborting the errorneous network connection). In this case,
run again to continue.
run terminates normally if the pipeline becomes empty.
The engine handles the following HTTP return codes itself:
All other return codes remain uninterpreted, it is up to the caller of this function to react on them.
method get_options :
method set_options :
http_options -> unit
Get/Set the available options for the HTTP engine. The new options will take into effect immediately.
method number_of_open_messages :
Returns the number of messages which are still in the pipeline.
method number_of_open_connections :
Returns the number of connections which are open at the same time and currently being used by this object (i.e. connections returned to the cache do not count)
method connections :
(string * int * int) list
Reports which connections exist:
(host, port, queue_length)
method cnt_new_connections :
Counts new connections (or better: attempts to establish connections)
method cnt_timed_out_connections :
Counts connections given up because of timeouts
method cnt_crashed_connections :
Counts connections with network or protocol errors
method cnt_server_eof_connections :
Counts connections the server terminated with EOF
method cnt_successful_connections :
Counts connections closed because pipelines become empty
method cnt_failed_connections :
Counts totally failed connections (no more reconnects allowed)
method reset_counters :
unit -> unit
cnt_new_connections: Is increased when a new connection attempt is started (that may fail or timeout in the future). Reconnects do not count.
cnt_timed_out_connections: Is increased whenever an established connection times out. Usually, it is immediately reconnected.
cnt_crashed_connections: Is increased whenever an established connection crashes. Usually, it is immediately reconnected.
cnt_failed_connections: Is increased when a timed out or crashed connection exceeds the maximum number of errors, and it is not tried to reconnect.
cnt_successful_connections: Is increased when all HTTP calls have been replied.
When the client has done all of its jobs, we have
cnt_new_connections = cnt_failed_connections + cnt_successful_connections