Plasma GitLab Archive
Projects Blog Knowledge

Ocamlnet 4 Reference Manual

Ocamlnet is an enhanced system platform library for Ocaml. As the name suggests, large parts of it have to do with network programming, but it is actually not restricted to this. Other parts deal with the management of multiple worker processes, and the interaction with other programs running on the same machine. You can also view Ocamlnet as an extension of the system interface as provided by the Unix module of the standard library.

So, which kind of applications can profit from Ocamlnet?

  • Web applications can use the netcgi2 library which provides all core functionality like connecting to web servers, decomposing web requests, and emitting web data. With the nethttpd library Ocamlnet even includes a little web server of its own, so stand-alone Ocaml programs can respond to HTTP requests. This is especially interesting for providing high-speed HTTP-based APIs like REST interfaces.
  • Client/server architectures can be built with the included SunRPC support. This is a very robust and mature implementation of this binary RPC protocol, both on the client and the server side. It is wire-compatible with C implementations of SunRPC. There is a generator for transforming SunRPC IDL files into Ocaml modules. Authentication and encryption are also supported (using GSS-API, or the non-standard SSL extension).
  • Compute jobs profit from Ocamlnet because the netmulticore library allows it to run compute tasks on as many cores of the machine as needed. The tasks can communicate with message passing and exchange data via shared memory. Of course, netmulticore is strictly typed.
  • Any kind of application can cherry-pick the parts of Ocamlnet that are considered useful. Especially, Ocamlnet includes a number of network clients (for HTTP, FTP, Telnet, SMTP, POP), and a large number of network-related string processing functions (e.g. for URL's, Base64, UTF-8, mail headers). For interacting with the system there are functions to invoke subcommands (shell library), and also for globbing, sending email, and logging. Local and remote filesystems can be accessed with the Netfs framework. The netsys library wraps lots of additional system calls that are missing in the Ocaml standard library.
A number of ideas and concepts are used throughout the libraries:

  • Netchannels are a way to abstract sequential I/O channels
  • The event queues provided by Equeue and Unixqueue make asynchronous protocol interpreters possible
  • The netplex library manages worker subprocesses, e.g. for accepting network connections or for doing computations.

Introductory chapters

Guides

  • Foreword: How Ocamlnet is organized
  • Platform: Which functions are available on which platform (POSIX/Win32). Also hints for portable programming.
  • Ipv6: The state of IPv6 support
  • Tls: TLS support
  • Crypto: Cryptography
  • Regexp: Regular expression backends

Related external articles:

Contents by library

Base Libraries

Guides

External articles

The netstring library: string processing functions

netstring focuses on string processing functions, and base definitions for other libraries.


Netconversion
Conversion between character encodings
Netunichar
Unicode character information
Netchannels
Object-oriented I/O: Basic types and classes
Netchannels_crypto
Crypto extensions for Netchannels
Netstream
A netstream is an input channel that is read block by block.
Mimestring
Low-level functions to parse and print mail and MIME messages
Netmime
Netmime contains high-level classes and functions to process mail and MIME messages.
Netsendmail
Functions to compose and send electronic mails
Neturl
Uniform Resource Locators (URLs)
Netsockaddr
Parsing of socket addresses
Netaddress
Parsing of mail addresses
Netbuffer
A Netbuffer.t is a buffer that can grow and shrink dynamically.
Netpagebuffer
Buffer for page-aligned I/O
Netdate
Support for common date/time parsing and formatting.
Netencoding
Base64, Quoted Printable, URL encoding, HTML escaping
Netulex
Support module for Alain Frisch's ulex lexer generator
Netaccel
Accelerators for bytecode
Netaccel_link
Enables accelerator module Netaccel
Nethtml
Parsing of HTML
Nethttp
Basic definitions for the HTTP protocol
Netfs
Class type stream_fs for filesystems with stream access to files
Netglob
Globbing
Netauth
Some primitives for authentication
Netcompression
Registry for compression algorithms
Netstring_str
Wrapper for regexps with Str syntax
Netmappings
Internal access to the character conversion database
Netaux
Internal auxiliary functions
Netnumber
Binary encodings of numbers
Nettls_support
Support types and functions for TLS
Netoid
X.500 Object Identifiers
Netx509
X.509 certificates
Netdn
X.500 distinguished names
Netasn1
ASN.1 support functions
Netgssapi_support
Support functions for GSS-API
Netmech_scram
SCRAM mechanism for authentication (RFC 5802)
Netmech_scram_gssapi
The SCRAM security mechanism for GSS-API
Rtypes
Binary encodings of numbers (Legacy)
Xdr
External Data Representation
Xdr_mstring
Managed Strings

The netstring-pcre library: additions for PCRE


Netstring_pcre
Wrapper for regexps with PCRE syntax

The netzip library: compression for object channels

Support for (un)compressing data on the fly with object channels. Requires ocamlzip.


Netgzip
Gzip object channels

The equeue library: concurrent execution flows via event queues

equeue is a fundamental library for event queues. It is mainly used by a number of other libraries of Ocamlnet to parallelize network code (so-called multiplexing).


Equeue
Equeue implements generic event queues.
Unixqueue
Unixqueues are one of the two forms of system event loops provided by Ocamlnet.
Unixqueue_pollset
Unixqueue implementation on top of Netsys_pollset
Unixqueue_select
This the old Unix.select-based imeplementation of event systems which was the default one until Ocamlnet-2.2.
Uq_engines
An engine performs a certain task in an autonomous way.
Uq_socks5
This module implements a SOCKS version 5 client (see RFC 1928) for use with the Uq_engines.connector, Uq_engines.listener, and Uq_engines.datagram_provider engine factories.
Uq_resolver
Support for pluggable resolvers
Uq_io
Unified engines for stream I/O
Uq_client
Support for socket clients
Uq_server
Server sockets
Uq_multiplex
Multiplex Controllers
Uq_transfer
Transfer engines
Uq_lwt
Compatibility with Lwt
Uq_libevent
Use Libevent as event loop
Uq_mt
Using engines in multi-threaded programs

The equeue-gtk1 and equeue-gtk2 extensions

Extensions for equeue to integrate the event queue into user interfaces made with lablgtk and lablgtk2


Uq_gtk
Integration with lablgtk/lablgtk2 event systems

The equeue-tcl extension

Extension for equeue to integrate the event queue into user interfaces made with labltk


Uq_tcl
Integration with the labltk event system

The netplex library: generic server framework

The netplex library is a protocol-independent server framework. Especially, it can be used in conjunction with nethttpd to build web servers, and with rpc to build RPC servers.


Netplex_types
Types for Netplex
Netplex_config
Read the configuration file
Netplex_controller
Controller
Netplex_container
Containers
Netplex_sockserv
Socket service creation
Netplex_workload
Workload management
Netplex_kit
Netplex toolkit
Netplex_cenv
Container environment
Netplex_log
Loggers
Netplex_main
Main program for Netplex servers
Netplex_mp
Multi-processing provider
Netplex_mt
Multi-threading provider
Netplex_mutex
Netplex-wide mutexes
Netplex_semaphore
Netplex-wide semaphores
Netplex_sharedvar
Netplex-wide variables
Netplex_mbox
Netplex message boxes
Netplex_encap
Type-safe marshalling between processes of the same executable
Rpc_netplex
Netplex support for RPC servers (TCP only)

The shell library: start external commands

The shell library allows you to start external commands. It is integrated into equeue.


Shell
Calls external programs, creates pipelines, etc.
Shell_sys
Calls external programs, creates pipelines, etc.
Shell_uq
Run shell commands within Unixqueues
Shell_fs
Shell filesystem

The netshm library: manage shared memory

The netshm library manages a shared memory object either as hash table or array. It is designed to be used in multi-processing program architectures. Note that there is now also the much better The netmulticore library: compute jobs library.


Netshm
Shared memory for O'Caml programs using multi-processing
Netshm_data
Data representation for shared memory
Netshm_hashtbl
Hash tables in shared memory
Netshm_array
Arrays in shared memory

The netsys library: system interfaces

The netsys library contains a number of low-level functions used by other Ocamlnet libraries.


Platform
Netexn
Exception registry
Netlog
Basic logging facility
Netsys
System calls missing in the Unix module
Netsys_posix
POSIX-specific system calls missing in the Unix module, and further API's from POSIX-style operating systems.
Netsys_oothr
Object-oriented thread API
Netsys_signal
Signal handler framework
Netsys_tmp
Temporary files
Netsys_mem
Bigarrays as memory buffers
Netsys_sem
Generic anonymous semaphores
Netsys_pollset
Sets of file descriptors for polling
Netsys_pollset_generic
Returns a good standard implementation of pollset for this platform.
Netsys_pollset_posix
Pollsets for POSIX operating systems
Netsys_pollset_win32
Pollsets for Win32
Netsys_win32
Primitives for Win32
Netsys_rng
Random-number generator
Netsys_crypto_types
Types for crypto providers
Netsys_crypto
Cryptographic providers
Netsys_crypto_modes
Helpers for crypto modes
Netsys_tls
User-level TLS API
Netsys_ciphers
Symmetric cryptographic ciphers
Netsys_digests
Cryptographic digests (hashes)
Netsys_gssapi
GSS-API Definition

Web Programming

Guides

The netcgi1 library

The netcgi1 library is no longer supported in Ocamlnet 3. Please switch to netcgi2.

The netcgi2 library: web application framework

This is the revised library for Web applications, now called netcgi2.


Netcgi
Common data-structures for CGI-like connectors.
Netcgi_common
Functions to develop new connectors.
Netcgi_cgi
Classical CGI connector.
Netcgi_fcgi
FastCGI connector.
Netcgi_ajp
Apache JServ Protocol (AJP) 1.3 connector.
Netcgi_scgi
SCGI connector.
Netcgi_test
Connector for testing your code.
Netcgi_dbi
Pools of connections for the ocamldbi generic database interface.
Netcgi1_compat
Compatibility module with the previous version of Netcgi.

The netcgi2-apache library: run web apps inside Apache

This is an Apache connector for the netcgi2 library.


Netcgi_apache
Netcgi Apache "mod" connector.
Netcgi_modtpl

The netcgi2-plex library: run web apps with Netplex

Netplex support for running Netcgi2 connectors:


Netcgi_plex
Netplex support for FastCGI, SCGI and AJP connectors

The nethttpd library: standalone web apps

The nethttpd library is a Web server component written in O'Caml. For a full web server, you'll also need netplex.


Nethttpd_types
Type definitions for the HTTP daemon
Nethttpd_kernel
The protocol kernel of the HTTP daemon
Nethttpd_reactor
The reactive encapsulation of the HTTP daemon
Nethttpd_engine
The event-based encapsulation of the HTTP daemon
Nethttpd_services
Service Providers for HTTP daemon
Nethttpd_plex
Netplex support
Nethttpd_util
Utility functions

Client/Server Programming

Guides

External articles

The rpc library: SunRPC

This library implements OncRPC (alias SunRPC).


Rpc
Common types and exceptions
Rpc_program
RPC programs
Rpc_client
RPC clients
Rpc_simple_client
Synchronous API for RPC clients
Rpc_proxy
RPC proxies
Rpc_server
RPC servers
Rpc_auth_sys
Authentication module AUTH_SYS
Rpc_portmapper
Portmapper version 2
Rpc_portmapper_aux
Rpc_portmapper_clnt
Rpc_transport
Low-level RPC transporters
Rpc_packer
The first n bytes of the packed value
Rpc_auth_gssapi
GSS-API for RPC authentication

The rpc-auth-local extension: authentication for local sockets

Authentication for local socket connections.


Rpc_auth_local
Return the authentication method AUTH_LOCAL.

The rpc-auth-dh extension: Diffie-Hellman authentication

This is Diffie-Hellman authentication for OncRPC (also known as "Secure RPC").


Rpc_auth_dh
Diffie-Hellman authentication (AUTH_DH alias AUTH_DES)
Rpc_key_service
Contact the keyserv daemon to encrypt/decrypt data with the common key.
Rpc_time
Get the time of the server (using the RFC 868 netdate protocol)

The rpc-xti extension: System V helpers

System V provides a network API called XTI in addition to the socket API. This library allows it to connect to RPC services that can only be reached over a local XTI connection ("cots" connection).


Rpc_xti_client
Minimal support for TI-RPC over the XTI API

Parallel Computing

Guides

Important change since OCaml-4.01: This OCaml version changed the semantics of the built-in primitives caml_modify and caml_initialize. Essentially, it is no longer possible to modify OCaml values residing outside the regular OCaml heap. As we do this inside Netcamlbox and Netmulticore, this change affects these libraries. Fortunately, there is a workaround on systems supporting weak symbols (all ELF systems and OS X): Here, caml_modify and caml_initialize are overridden by Ocamlnet so that they are again compatible. Note that this is a global modification of the runtime system!

The netcamlbox library: message passing

Camlbox is a fast message-passing system between processes running on the same machine. It uses shared memory for communication. A message sent to a Camlbox is only copied once, not twice.


Netcamlbox
Camlboxes are a fast IPC mechanism to send Ocaml values from one process to another.

The netmulticore library: compute jobs

Netmulticore is an experimental framework for managing multiple processes, and sending messages between them.


Netmcore
Multi-processing for compute jobs
Netmcore_process
Statically typed processes
Netmcore_mempool
Memory pools
Netmcore_heap
Shared heaps of structured values
Netmcore_array
Shared arrays
Netmcore_buffer
Shared buffer
Netmcore_hashtbl
Shared hashtables
Netmcore_matrix
Shared 2-dimensional arrays (matrices)
Netmcore_queue
Shared queues
Netmcore_ref
Shared mutable variables
Netmcore_mutex
Mutexes
Netmcore_sem
Semaphores
Netmcore_condition
Condition variables
Netmcore_camlbox
Camlboxes for use in netmulticore programs

Network Protocol Libraries

Guides

The netclient library: HTTP, FTP, Telnet

This library includes clients for HTTP, Telnet, and FTP.


Http_client
HTTP 1.1 client
Http_client_conncache
Connection cache
Http_fs
HTTP filesystem
Ftp_client
FTP client
Ftp_data_endpoint
Senders and receivers for the FTP data connection
Ftp_fs
FTP filesystem
Telnet_client
Telnet client

The pop library: POP protocol

This is a simple POP 3 client.


Netpop
This is an interface for the Post Office Protocol - Version 3 (POP3) as specifed by RFC 1939.

The smtp library: SMTP protocol

This is a simple SMTP client.


Netsmtp
This is an interface for the Simple Mail Tranfer Protocol (SMTP) as specified by RFC 2821.

Trailer

Index

Authors

The Ocamlnet libraries have been initially written by Gerd Stolpmann and Patrick Doane.

Contributions by

  • Nicolas George: Netdate
  • Eric Stokes: FastCGI support
  • Pierre Habouzit: SMTP client
  • Christophe Troestler: Revised version of netcgi
  • Deokhwan Kim: Support for EUC-KR
Sponsorships by

  • Baretta s.r.l: Nethttpd
  • California State University: SSL support for RPC, Netplex

This web site is published by Informatikbüro Gerd Stolpmann
Powered by Caml