Ocamlnet 3 Reference Manual

Ocamlnet is an enhanced system platform library for Ocaml. As the name suggests, large parts of it have to do with network programming, but it is actually not restricted to this. Other parts deal with the management of multiple worker processes, and the interaction with other programs running on the same machine. You can also view Ocamlnet as an extension of the system interface as provided by the Unix module of the standard library.

So, which kind of applications can profit from Ocamlnet?

• Web applications can use the netcgi2 library which provides all core functionality like connecting to web servers, decomposing web requests, and emitting web data. With the nethttpd library Ocamlnet even includes a little web server of its own, so stand-alone Ocaml programs can respond to HTTP requests. This is especially interesting for providing high-speed HTTP-based APIs like REST interfaces.
• Client/server architectures can be built with the included SunRPC support. This is a very robust and mature implementation of this binary RPC protocol, both on the client and the server side. It is wire-compatible with C implementations of SunRPC. There is a generator for transforming SunRPC IDL files into Ocaml modules. Authentication and encryption are also supported (using GSS-API, or the non-standard SSL extension).
• Compute jobs profit from Ocamlnet because the netmulticore library allows it to run compute tasks on as many cores of the machine as needed. The tasks can communicate with message passing and exchange data via shared memory. Of course, netmulticore is strictly typed.
• Any kind of application can cherry-pick the parts of Ocamlnet that are considered useful. Especially, Ocamlnet includes a number of network clients (for HTTP, FTP, Telnet, SMTP, POP), and a large number of network-related string processing functions (e.g. for URL's, Base64, UTF-8, mail headers). For interacting with the system there are functions to invoke subcommands (shell library), and also for globbing, sending email, and logging. Local and remote filesystems can be accessed with the Netfs framework. The netsys library wraps lots of additional system calls that are missing in the Ocaml standard library.

A number of ideas and concepts are used throughout the libraries:

• Netchannels are a way to abstract sequential I/O channels
• The event queues provided by Equeue and Unixqueue make asynchronous protocol interpreters possible
• The netplex library manages worker subprocesses, e.g. for accepting network connections or for doing computations.

Introductory chapters

Related external articles:

• Ocamlnet 3 finally released
• Stranger in a strange land, by Gerd Stolpmann. The Win32 port in Ocamlnet 3.

Contents by library

• Base Libraries
  • The netstring library: string processing functions
  • The netzip library: compression for object channels
  • The equeue library: concurrent execution flows via event queues
  • The netplex library: generic server framework
  • The netshm library: manage shared memory
  • The netsys library: system interfaces
• Web Programming
  • The netcgi2 library: web application framework
  • The netcgi2-apache library: run web apps inside Apache
  • The netcgi2-plex library: run web apps with Netplex
  • The nethttpd library: standalone web apps
• Client/Server Programming
  • The rpc library: SunRPC
  • The rpc-auth-local extension: authentication for local sockets
  • The rpc-auth-dh extension: Diffie-Hellman authentication
  • The rpc-xti extension: System V helpers
  • The rpc_ssl extension: SSL encryption
  • The netgssapi library with the GSS-API framework
  • The netmech-scram library for SCRAM
• Parallel Computing
  • The netcamlbox library: message passing
  • The netmulticore library: compute jobs
• Network Protocol Libraries
  • The netclient library: HTTP, FTP, Telnet
  • The pop library: POP protocol
  • The smtp library: SMTP protocol

Base Libraries

External articles

• Equeue compared to Lwt, by Jake Donham.
• Parallelizing with Ocamlnet, by Gerd Stolpmann. Demonstrates how to parallelize matrix multiplications with Netplex-controlled worker processes.
• Mastering Multi-processing, by Gerd Stolpmann. About the new synchronization primitives in Ocamlnet 3

The netstring library: string processing functions

netstring focuses on string processing functions, and base definitions for other libraries.

Netconversion	Conversion between character encodings
Netunichar	Unicode character information
Netchannels	Object-oriented I/O: Basic types and classes
Netstream	A netstream is an input channel that is read block by block.
Mimestring	Low-level functions to parse and print mail and MIME messages
Netmime	Netmime contains high-level classes and functions to process mail and MIME messages.
Netsendmail	Functions to compose and send electronic mails
Neturl	Uniform Resource Locators (URLs)
Netsockaddr	Parsing of socket addresses
Netaddress	Parsing of mail addresses
Netbuffer	A Netbuffer.t is a buffer that can grow and shrink dynamically.
Netpagebuffer	Buffer for page-aligned I/O
Netdate	Support for common date/time parsing and formatting.
Netencoding	Base64, Quoted Printable, URL encoding, HTML escaping
Netulex	Support module for Alain Frisch's ulex lexer generator
Netaccel	Accelerators for bytecode
Netaccel_link	Enables accelerator module Netaccel
Nethtml	Parsing of HTML
Nethttp	Basic definitions for the HTTP protocol
Netfs	Class type stream_fs for filesystems with stream access to files
Netglob	Globbing
Netauth	Some primitives for authentication
Netcompression	Registry for compression algorithms
Netstring_str	Wrapper for regexps with Str syntax
Netmappings	Internal access to the character conversion database
Netaux	Internal auxiliary functions
Netnumber	Binary encodings of numbers
Rtypes	Binary encodings of numbers (Legacy)
Xdr	External Data Representation
Xdr_mstring	Managed Strings

The netstring-pcre library: additions for PCRE

Netstring_pcre

Wrapper for regexps with PCRE syntax

The netzip library: compression for object channels

Support for (un)compressing data on the fly with object channels. Requires ocamlzip.

Netgzip

Gzip object channels

The equeue library: concurrent execution flows via event queues

equeue is a fundamental library for event queues. It is mainly used by a number of other libraries of Ocamlnet to parallelize network code (so-called multiplexing).

Equeue	Equeue implements generic event queues.
Unixqueue	Unixqueues are one of the two forms of system event loops provided by Ocamlnet.
Unixqueue_pollset	Unixqueue implementation on top of Netsys_pollset
Unixqueue_select	This the old Unix.select-based imeplementation of event systems which was the default one until Ocamlnet-2.2.
Uq_engines	An engine performs a certain task in an autonomous way.
Uq_socks5	This module implements a SOCKS version 5 client (see RFC 1928) for use with the Uq_engines.connector, Uq_engines.listener, and Uq_engines.datagram_provider engine factories.
Uq_resolver	Support for pluggable resolvers
Uq_io	Unified engines for stream I/O
Uq_lwt	Compatibility with Lwt
Uq_libevent	Use Libevent as event loop
Uq_mt	Using engines in multi-threaded programs

The equeue-gtk1 and equeue-gtk2 extensions

Extensions for equeue to integrate the event queue into user interfaces made with lablgtk and lablgtk2

Uq_gtk

Integration with lablgtk/lablgtk2 event systems

The equeue-ssl extension

Extensions for equeue to support SSL

Uq_ssl	Asynchronous SSL connections
Https_client	HTTPS extension to Http_client

The equeue-tcl extension

Extension for equeue to integrate the event queue into user interfaces made with labltk

Uq_tcl

Integration with the labltk event system

The netplex library: generic server framework

The netplex library is a protocol-independent server framework. Especially, it can be used in conjunction with nethttpd to build web servers, and with rpc to build RPC servers.

Netplex_types	Types for Netplex
Netplex_config	Read the configuration file
Netplex_controller	Controller
Netplex_container	Containers
Netplex_sockserv	Socket service creation
Netplex_workload	Workload management
Netplex_kit	Netplex toolkit
Netplex_cenv	Container environment
Netplex_log	Loggers
Netplex_main	Main program for Netplex servers
Netplex_mp	Multi-processing provider
Netplex_mt	Multi-threading provider
Netplex_mutex	Netplex-wide mutexes
Netplex_semaphore	Netplex-wide semaphores
Netplex_sharedvar	Netplex-wide variables
Netplex_mbox	Netplex message boxes
Netplex_encap	Type-safe marshalling between processes of the same executable
Rpc_netplex	Netplex support for RPC servers (TCP only)

The shell library: start external commands

The shell library allows you to start external commands. It is integrated into equeue.

Shell	Calls external programs, creates pipelines, etc.
Shell_sys	Calls external programs, creates pipelines, etc.
Shell_uq	Run shell commands within Unixqueues
Shell_fs	Shell filesystem

The netshm library: manage shared memory

The netshm library manages a shared memory object either as hash table or array. It is designed to be used in multi-processing program architectures. Note that there is now also the much better The netmulticore library: compute jobs library.

Netshm	Shared memory for O'Caml programs using multi-processing
Netshm_data	Data representation for shared memory
Netshm_hashtbl	Hash tables in shared memory
Netshm_array	Arrays in shared memory

The netsys library: system interfaces

The netsys library contains a number of low-level functions used by other Ocamlnet libraries.

Platform
Netexn	Exception registry
Netlog	Basic logging facility
Netsys	System calls missing in the Unix module
Netsys_posix	POSIX-specific system calls missing in the Unix module, and further API's from POSIX-style operating systems.
Netsys_oothr	Object-oriented thread API
Netsys_signal	Signal handler framework
Netsys_tmp	Temporary files
Netsys_mem	Bigarrays as memory buffers
Netsys_sem	Generic anonymous semaphores
Netsys_pollset	Sets of file descriptors for polling
Netsys_pollset_generic	Returns a good standard implementation of pollset for this platform.
Netsys_pollset_posix	Pollsets for POSIX operating systems
Netsys_pollset_win32	Pollsets for Win32
Netsys_win32	Primitives for Win32
Netsys_rng	Random-number generator

Web Programming

The netcgi1 library

The netcgi1 library is no longer supported in Ocamlnet 3. Please switch to netcgi2.

The netcgi2 library: web application framework

This is the revised library for Web applications, now called netcgi2.

Netcgi	Common data-structures for CGI-like connectors.
Netcgi_common	Functions to develop new connectors.
Netcgi_cgi	Classical CGI connector.
Netcgi_fcgi	FastCGI connector.
Netcgi_ajp	Apache JServ Protocol (AJP) 1.3 connector.
Netcgi_scgi	SCGI connector.
Netcgi_test	Connector for testing your code.
Netcgi_dbi	Pools of connections for the ocamldbi generic database interface.
Netcgi1_compat	Compatibility module with the previous version of Netcgi.

The netcgi2-apache library: run web apps inside Apache

This is an Apache connector for the netcgi2 library.

Netcgi_apache	Netcgi Apache "mod" connector.
Netcgi_modtpl

The netcgi2-plex library: run web apps with Netplex

Netplex support for running Netcgi2 connectors:

Netcgi_plex

Netplex support for FastCGI, SCGI and AJP connectors

The nethttpd library: standalone web apps

The nethttpd library is a Web server component written in O'Caml. For a full web server, you'll also need netplex.

Nethttpd_types	Type definitions for the HTTP daemon
Nethttpd_kernel	The protocol kernel of the HTTP daemon
Nethttpd_reactor	The reactive encapsulation of the HTTP daemon
Nethttpd_engine	The event-based encapsulation of the HTTP daemon
Nethttpd_services	Service Providers for HTTP daemon
Nethttpd_plex	Netplex support
Nethttpd_util	Utility functions

Client/Server Programming

External articles

• The next server, please!, by Gerd Stolpmann. About highly-available RPC.

The rpc library: SunRPC

This library implements OncRPC (alias SunRPC).

Rpc	Common types and exceptions
Rpc_program	RPC programs
Rpc_client	RPC clients
Rpc_simple_client	Synchronous API for RPC clients
Rpc_proxy	RPC proxies
Rpc_server	RPC servers
Rpc_auth_sys	Authentication module AUTH_SYS
Rpc_portmapper	Portmapper version 2
Rpc_portmapper_aux
Rpc_portmapper_clnt
Rpc_transport	Low-level RPC transporters
Rpc_packer	The first n bytes of the packed value
Rpc_auth_gssapi	GSS-API for RPC authentication

The rpc-auth-local extension: authentication for local sockets

Authentication for local socket connections.

Rpc_auth_local

Return the authentication method AUTH_LOCAL.

The rpc-auth-dh extension: Diffie-Hellman authentication

This is Diffie-Hellman authentication for OncRPC (also known as "Secure RPC").

Rpc_auth_dh	Diffie-Hellman authentication (AUTH_DH alias AUTH_DES)
Rpc_key_service	Contact the keyserv daemon to encrypt/decrypt data with the common key.
Rpc_time	Get the time of the server (using the RFC 868 netdate protocol)

The rpc-xti extension: System V helpers

System V provides a network API called XTI in addition to the socket API. This library allows it to connect to RPC services that can only be reached over a local XTI connection ("cots" connection).

Rpc_xti_client

Minimal support for TI-RPC over the XTI API

The rpc_ssl extension: SSL encryption

This library allows you to tunnel RPC through SSL/TLS.

Rpc_ssl

Securing RPC by SSL

The netgssapi library with the GSS-API framework

Netgssapi

GSS-API Definition

The netmech-scram library for SCRAM

Implements SCRAM (RFC 5802). Right now only for GSS-API.

Netmech_scram	SCRAM mechanism for authentication (RFC 5802)
Netmech_scram_gssapi	The SCRAM security mechanism for GSS-API

Parallel Computing

Important change since OCaml-4.01: This OCaml version changed the semantics of the built-in primitives caml_modify and caml_initialize. Essentially, it is no longer possible to modify OCaml values residing outside the regular OCaml heap. As we do this inside Netcamlbox and Netmulticore, this change affects these libraries. Fortunately, there is a workaround on systems supporting weak symbols (all ELF systems and OS X): Here, caml_modify and caml_initialize are overridden by Ocamlnet so that they are again compatible. Note that this is a global modification of the runtime system!

The netcamlbox library: message passing

Camlbox is a fast message-passing system between processes running on the same machine. It uses shared memory for communication. A message sent to a Camlbox is only copied once, not twice.

Netcamlbox

Camlboxes are a fast IPC mechanism to send Ocaml values from one process to another.

The netmulticore library: compute jobs

Netmulticore is an experimental framework for managing multiple processes, and sending messages between them.

Netmcore	Multi-processing for compute jobs
Netmcore_process	Statically typed processes
Netmcore_mempool	Memory pools
Netmcore_heap	Shared heaps of structured values
Netmcore_array	Shared arrays
Netmcore_buffer	Shared buffer
Netmcore_hashtbl	Shared hashtables
Netmcore_matrix	Shared 2-dimensional arrays (matrices)
Netmcore_queue	Shared queues
Netmcore_ref	Shared mutable variables
Netmcore_mutex	Mutexes
Netmcore_sem	Semaphores
Netmcore_condition	Condition variables
Netmcore_camlbox	Camlboxes for use in netmulticore programs

Network Protocol Libraries

The netclient library: HTTP, FTP, Telnet

This library includes clients for HTTP, Telnet, and FTP.

Http_client	HTTP 1.1 client
Http_client_conncache	Connection cache
Http_fs	HTTP filesystem
Ftp_client	FTP client
Ftp_data_endpoint	Senders and receivers for the FTP data connection
Ftp_fs	FTP filesystem
Telnet_client	Telnet client

The pop library: POP protocol

This is a simple POP 3 client.

Netpop

This is an interface for the Post Office Protocol - Version 3 (POP3) as specifed by RFC 1939.

The smtp library: SMTP protocol

This is a simple SMTP client.

Netsmtp

This is an interface for the Simple Mail Tranfer Protocol (SMTP) as specified by RFC 2821.

Trailer

Index

• Index of types
• Index of exceptions
• Index of values
• Index of class attributes
• Index of class methods
• Index of classes
• Index of class types
• Index of modules
• Index of module types

Authors

The Ocamlnet libraries have been initially written by Gerd Stolpmann and Patrick Doane.

Contributions by

• Nicolas George: Netdate
• Eric Stokes: FastCGI support
• Pierre Habouzit: SMTP client
• Christophe Troestler: Revised version of netcgi
• Deokhwan Kim: Support for EUC-KR

Sponsorships by

• Baretta s.r.l: Nethttpd
• California State University: SSL support for RPC, Netplex

This web site is published by Informatikbüro Gerd Stolpmann