Module Netftp_client

module Netftp_client: sig .. end

FTP client

Currently implements:

Core FTP (RFC 959), except compressed transfer modes, and except page files
Negotiation of FTP extensions (RFC 2389)
Common FTP extensions (RFC 3659)
IPv6 (RFC 2428)
Internationalization (RFC 2640)
Directory walking (NVFS) and direct access (TVFS)
TLS (it is required, though, that a TLS provider is initialized, see Tls for more information)
GSSAPI (RFC 2228)

The client is written in asynchronous style (using Uq_engines).

The interface of this module was changed in Ocamlnet-3.3. Before this release, the module was marked as "experimental". This is no longer the case, as the interface has been updated, and the missing features have been added (e.g. STOR support).

exception FTP_error of exn

Something went wrong, often on socket level

exception FTP_protocol_violation of string

The server violates the FTP specification

exception FTP_timeout of string

A timeout on the control or data connection (this is a fatal error)

exception GSSAPI_error of string

An error on GSSAPI level

type cmd_state = [ `Auth_data
       | `Init
       | `Not_connected
       | `Pass_acct_seq
       | `Perm_failure
       | `Preliminary
       | `Proto_error
       | `Rename_seq
       | `Restart_seq
       | `Success
       | `Temp_failure
       | `User_acct_seq
       | `User_pass_seq ]

The command state:

`Not_connected: Not connected.
`Init: Just connected, no greeting message arrived yet
`Success: Got the greeting message/last command was successful
`Proto_error: currently unused
`Temp_failure: Last command was not successful, and the code was between 400 and 499
`Perm_failure: Last command was not successful, and the code was between 500 and 599
`Rename_seq: Used instead of `Success after the RNFR command
`Restart_seq: Used instead of `Success after the REST command
`User_pass_seq: Used instead of `Success after the USER command when a password must be typed in
`User_acct_seq: Used instead of `Success after the USER command when an account ID must be typed in
`Pass_acct_seq: Used instead of `Success after the PASS command when an account iD must be typed in
`Preliminary: a reply with code 100 to 199. There will be another final reply for the command
`Auth_data: an ADAT reply inidicating that another round of authentication is necessary.

type port = [ `Active of string * int * Unix.file_descr
       | `Ext_active of string * int * Unix.file_descr
       | `Ext_passive of int
       | `Passive of string * int
       | `Unspecified ]

The port of the data connection: `Active means that the server initiates the data connection to the listening client, and in the case of `Passive the client initiates the data connection to the listening server. The string argument is the IP address as dotted quad, the int argument is the port number, and the descriptor is the listening master socket.

type form_code = [ `ASA | `Non_print | `Telnet ]

The form_code has a meaning when FTP is used to print files:

`Non_print: This is not the case.
`Telnet: Telnet control codes are used for vertical movement
`ASA: ASA (Fortran) control codes are used for vertical movement

type representation = [ `ASCII of form_code option
       | `EBCDIC of form_code option
       | `Image ]

The representation of the transferred file:

`ASCII: An ASCII variant is used, i.e. the server sends files in ASCII encoding with CR/LF as end-of-line marker. Supported by all servers.
`EBCDIC: An EBCDIC variant is used, i.e. the server sends files in EBCDIC encoding with NEL as end-of-line marker
`Image: The file is transferred in its original binary representation. Supported by all servers.

"Local" representations are not supported.

This FTP client does not recode the files such that they match the selected representation. When files are downloaded, they are stored as they are received. When files are uploaded, they are sent as they are. The user of this client must do recodings when necessary (the class Netftp_data_endpoint.data_converter may be useful for this).

If no representation is selected, FTP servers assume `ASCII None as default.

type structure = [ `File_structure | `Record_structure ]

The client supports two structures:

`File_structure: Files are simply contiguous streams of bytes
`Record_structure: Files are sequences of records. FTP does not make a difference between variable and fixed length records. It is not forbidden that the records are themselves structured into lines, in fact it can happen that end-of-line markers are contained in binary records. Operating systems that support record-structured files often store text files in this format, i.e. every line is stored in its own record, without end-of-line marker. If record structure is selected by a STRU command, it is recommended to use the classes Netftp_data_endpoint.out_record_channel and Netftp_data_endpoint.in_record_channel for the local representation of the files, otherwise the records may be incorrectly mapped to the local conventions.

Page-structured files (i.e. indexed files) are not supported.

If no structure is selected, FTP servers will assume file structure as default.

type transmission_mode = [ `Block_mode | `Stream_mode ]

The transmission mode selects how the data are encoded in the data connection.

`Stream_mode: This is the simple format that is responsible for all the failed FTP downloads. It is supported by all FTP servers, actually, you cannot assume a better transmission mode from an unknown FTP server. It is unreliable because it cannot distinguish between a transmission failure and the regular end-of-file condition.
`Block_mode: This is an improved format using frames to protect the transmitted data. Unfortunately, almost no FTP server supports it.

Both modes are compatible with both structures, i.e. you can transfer a record-structured file in stream mode and a flat file in block mode. However, in practice this is not the case. Servers that only know flat files are likely to only support stream mode, and servers implementing record structure imply that block transfers base on the record format. So the advice is to use stream mode for flat files, and block mode for record files.

type ftp_auth = [ `GSSAPI | `None | `TLS ]

type ftp_data_prot = [ `C | `E | `P | `S ]

Meaning:

`C: no protection (clear)
`S: integrity protection
`E: encryption without integrity protection
`P: integrity protection and encryption (= privacy)

type support_level = [ `If_possible | `None | `Required ]

type ftp_state = {

`cmd_state : cmd_state;`	`(*`	the command state	`*)`
`ftp_connected : bool;`	`(*`	whether connected with the server	`*)`
`ftp_data_conn : bool;`	`(*`	whether there is a clean data conn	`*)`
`ftp_user : string option;`	`(*`	successfully sent user identifier	`*)`
`ftp_password : string option;`	`(*`	successfully sent password	`*)`
`ftp_account : string option;`	`(*`	successfully sent account identifier	`*)`
`ftp_logged_in : bool;`	`(*`	whether the user is logged in	`*)`
`ftp_host : string;`	`(*`	Host name	`*)`
`ftp_port : port;`	`(*`	the selected port	`*)`
`ftp_repr : representation;`	`(*`	the selected representation	`*)`
`ftp_structure : structure;`	`(*`	the selected structure	`*)`
`ftp_trans : transmission_mode;`	`(*`	the selected trans mode	`*)`
`ftp_dir : string list;`	`(*`	The current directory, expressed as list of CWD changes minus CDUP changes. This is only reasonable if CWD does not include slashes. The list is in reverse order, i.e. deepest directory first.	`*)`
`ftp_features : (string * string option) list option;`	`(*`	The list of features returned by the last FEAT command. `None` means that no FEAT command was yet tried. `Some []` means that there are no features (either FEAT returned an empty list, or the FEAT command is not implemented by the server). Otherwise the list enumerates pairs `(label,param)` where `label` is the case-sensitive feature label and `param` the optional parameter. There is no defined order for the list of features.	`*)`
`ftp_options : (string * string option) list;`	`(*`	Remembers the OPTS commands sent to the server. The list enumerates pairs `(command,optionparam)`, where `command` is the uppercase command name the option refers to. Only the last negotiated `optionparam` for the command is remembered.	`*)`
`ftp_auth : ftp_auth;`	`(*`	Authentication/privacy mode (AUTH command)	`*)`
`ftp_auth_data : string option;`	`(*`	The data from the last ADAT reply, already base64-decoded	`*)`
`ftp_data_prot : ftp_data_prot;`	`(*`	Security protocol for data connections (PROT command)	`*)`
`ftp_data_pbsz : int;`	`(*`	protection buffer size (PBSZ command)	`*)`
`ftp_prot : Netftp_data_endpoint.ftp_protector option;`	`(*`	a security layer (RFC 2228)	`*)`

}

The ftp_state reflects the knowledge of the client about what has been agreed upon with the server.

type cmd = [ `ACCT of string
       | `ADAT of string
       | `ALLO of int * int option
       | `APPE of
           string * (ftp_state -> Netftp_data_endpoint.local_sender)
       | `AUTH of string
       | `CDUP
       | `CWD of string
       | `Connect of string * int
       | `DELE of string
       | `Disconnect
       | `Dummy
       | `EPRT
       | `EPSV of [ `AF of Unix.socket_domain | `ALL ] option
       | `FEAT
       | `HELP of string option
       | `LANG of string option
       | `LIST of
           string option *
           (ftp_state -> Netftp_data_endpoint.local_receiver)
       | `MDTM of string
       | `MKD of string
       | `MLSD of
           string option *
           (ftp_state -> Netftp_data_endpoint.local_receiver)
       | `MLST of string option
       | `MODE of transmission_mode
       | `NLST of
           string option *
           (ftp_state -> Netftp_data_endpoint.local_receiver)
       | `NOOP
       | `OPTS of string * string option
       | `PASS of string
       | `PASV
       | `PBSZ of int
       | `PORT
       | `PROT of ftp_data_prot
       | `PWD
       | `QUIT
       | `REIN
       | `REST of string
       | `RETR of
           string * (ftp_state -> Netftp_data_endpoint.local_receiver)
       | `RMD of string
       | `RNFR of string
       | `RNTO of string
       | `SITE of string
       | `SIZE of string
       | `SMNT of string
       | `STAT of string option
       | `STOR of
           string * (ftp_state -> Netftp_data_endpoint.local_sender)
       | `STOU of ftp_state -> Netftp_data_endpoint.local_sender
       | `STRU of structure
       | `SYST
       | `Start_TLS of (module Netsys_crypto_types.TLS_CONFIG)
       | `Start_protection of Netftp_data_endpoint.ftp_protector
       | `TYPE of representation
       | `USER of string ]

An FTP command. Not all commands are implemented by all servers.

`Start_TLS is a pseudo command - at this point the TLS handshake starts.

type reply = int * string

Reply code plus text

class type ftp_client_pi = object .. end

The client protocol interpreter...

type ftp_method = ftp_client_pi -> unit Uq_engines.engine

An ftp_method is a small procedure doing some task

exception FTP_method_temp_failure of int * string

exception FTP_method_perm_failure of int * string

exception FTP_method_unexpected_reply of int * string

These exceptions may be raised during execution by the FTP method. The int is the unexpected FTP control code and the string the corresponding text. A temporary failure has a code between 400 and 499, and a permanent failure has a code between 500 and 599.

val connect_method : host:string -> ?port:int -> unit -> ftp_method

This method connects to the host

val login_method : user:string ->
       get_password:(unit -> string) ->
       get_account:(unit -> string) -> unit -> ftp_method

This FTP method logs the user in. get_password is called when the FTP server asks for the password (may be skipped). get_account is called when the server asks for the account ID (may be skipped).

val quit_method : unit -> ftp_method

Quits and disconnects

val tls_method : config:(module Netsys_crypto_types.TLS_CONFIG) ->
       required:bool -> unit -> ftp_method

This FTP method negotiates the use of TLS. If required, it is an error if TLS is not supported. Otherwise, it is ok to omit the TLS protection.

val gssapi_method : config:Netsys_gssapi.client_config ->
       required:bool -> (module Netsys_gssapi.GSSAPI) -> ftp_method

This method negotiates the use of GSSAPI authentication and security. You need to pass the GSSAPI provider (e.g. Netgss.System).

The config can often simply be created with Netsys_gssapi.create_client_config(), as normally reasonably defaults are assumed by the GSSAPI provider. See Netsys_gssapi.create_client_config for options.

If required, it is an error if the server doesn't support GSSAPI authentication. Otherwise, this method is a no-op in this case.

Note that you cannot combine gssapi_method with tls_method. Although the gssapi_method authenticates the user, you still need to log in, although without password (basically, GSSAPI just gives you permissions to be somebody, but you still need to select who you want to be).

val walk_method : [ `Dir of string | `File of string | `Stay ] -> ftp_method

This FTP method walks to the target directory:

`File name: The name is interpreted as slash-separated path. It is always interpreted relative to the home directory of the user (i.e. the directory after login), even if it begins with a slash. The FTP command walks to the directory containing name.
`Dir name: The FTP command walks to the directory name (same syntax as for `File).
`Stay: The FTP command does nothing (stays in the current directory).

type filename = [ `NVFS of string | `TVFS of string | `Verbatim of string ]

There are several methods how to specify filenames:

`NVFS name: The "Network Virtual File System" is the normal way of accessing FTP files. The client walks into the directory containing the file using CWD and CDUP commands, and calls the file operation from this directory. For simplicity, this client interprets slashes in name as path component separators. The FTP server will never see these slashes.
`TVFS name: The optional "Trivial Network File System" avoids the CWD and CDUP commands. The tagged name is normalized (double slashed removed etc.), and is passed to the server as-is. Before using the faster TVFS one should check whether it is supported (feature "TVFS"). Note that even for TVFS there are no special files "." and ".."!
`Verbatim name: The string name is passed to the server without transforming it in any way.

val get_method : file:filename ->
       representation:representation ->
       store:(ftp_state -> Netftp_data_endpoint.local_receiver) ->
       unit -> ftp_method

This FTP method walks to the right directory and gets file from the server. The file is stored in the local_receiver that can be obtained by calling the store function. The selected representation remains unchanged.

val put_method : ?meth:[ `APPE | `STOR ] ->
       file:filename ->
       representation:representation ->
       store:(ftp_state -> Netftp_data_endpoint.local_sender) ->
       unit -> ftp_method

This FTP method walks to the right directory and puts file to the server. The file is taken from the local_sender that can be obtained by calling the store function. The selected representation remains unchanged.

meth selects the method to use (default `STOR).

val invoke_method : command:cmd -> unit -> ftp_method

This FTP method simply invokes command.

val set_structure_method : structure -> ftp_method

Requests a certain structure for future file transfers

val set_mode_method : transmission_mode -> ftp_method

Requests a certain mode for future file transfers

val rename_method : file_from:filename ->
       file_to:filename -> unit -> ftp_method

Renames the file_from into file_to.

Both file names must be of the same type, either `NVFS or `Verbatim. If `NVFS, both names must be in the same directory.

val mkdir_method : filename -> ftp_method

Creates the named directory

val rmdir_method : filename -> ftp_method

Deletes the named directory

val delete_method : filename -> ftp_method

Deletes the named file

val list_method : dir:filename ->
       representation:representation ->
       store:(ftp_state -> Netftp_data_endpoint.local_receiver) ->
       unit -> ftp_method

Lists the contents of the directory dir using the LIST command. The representation must not be `Image.

val nlst_method : dir:filename ->
       representation:representation ->
       store:(ftp_state -> Netftp_data_endpoint.local_receiver) ->
       unit -> ftp_method

Lists the contents of the directory dir using the NLST command The representation must not be `Image.

val parse_nlst_document : string -> string list

Returns the filenames contained in the output of `NLST

val mdtm_method : file:filename ->
       process_result:(float -> unit) -> unit -> ftp_method

Determines the date and time of the last modification of file. On success, process_result is called.

val size_method : file:filename ->
       representation:representation ->
       process_result:(int64 -> unit) -> unit -> ftp_method

Determines the size of file. On success, process_result is called. The size depends on representation.

val feat_method : ?process_result:((string * string option) list -> unit) ->
       unit -> ftp_method

Get the list of feature tokens (see also Netftp_client.ftp_state.ftp_features)

type entry = string * (string * string) list

A file entry (name, facts). The facts are given as pairs (factname,value) where factname is always lowercase. For parsers for common facts see below.

val mlst_method : file:filename ->
       process_result:(entry list -> unit) ->
       unit -> ftp_method

Get the file entry for file.

val mlsd_method : dir:filename ->
       store:(ftp_state -> Netftp_data_endpoint.local_receiver) ->
       unit -> ftp_method

Gets the entries for this directory.

val parse_mlsd_document : string -> entry list

Returns the entries contained in the output of `MLSD

type entry_type = [ `Cdir | `Dir | `File | `Other | `Pdir ]

Types:

`File: entry refers to file
`Cdir: entry refers to the directory being listed
`Pdir: entry is a parent of the directory being listed
`Dir: entry refers to directory
`Other: entry is neither file nor directory

type entry_perm = [ `Append
       | `Create
       | `Delete
       | `Delete_member
       | `Enter
       | `List
       | `Mkdir
       | `Read
       | `Rename
       | `Write ]

Permissions:

`Append: append to file possible
`Create: file can be created in this dir
`Delete: file or dir can be deleted
`Enter: dir can be entered
`Rename: file or dir can be renamed
`List: dir can be listed
`Mkdir: subdir can be created in this dir
`Delete_member: a file or dir can be deleted in this directory
`Read: a file can be retrieved
`Write: a file can be stored

The following functions extract commonly used facts from entries. They may raise Not_found.

val get_name : entry -> string

val get_size : entry -> int64

val get_modify : entry -> float

val get_create : entry -> float

val get_type : entry -> entry_type

val get_unique : entry -> string

val get_perm : entry -> entry_perm list

val get_lang : entry -> string

val get_media_type : entry -> string

val get_charset : entry -> string

val get_unix_mode : entry -> int

val get_unix_uid : entry -> string

val get_unix_gid : entry -> string

class ftp_client : ?event_system:Unixqueue.event_system -> unit -> object .. end

The ftp client is a user session that may even span several connections.

Examples and Discussion

To download a single flat file from a server:

 
   let buffer = Buffer.create 1000 in
   let ch = new Netchannels.output_buffer buffer in
   let client = new ftp_client() in
   client # exec (connect_method ~host:"127.0.0.1" ());
   client # exec (login_method ~user:"foo"
                                ~get_password:(fun () -> "password")
                                ~get_account:(fun () -> "foo") ());
   client # exec (get_method ~file:(`NVFS "path/to/file")
                             ~representation:`Image
                             ~store:(fun _ -> `File_structure ch) ());

The file is stored in buffer. By using a different netchannel, it can be stored whereever wanted.

To download a record-structured text file, use a store like:

 
    let ch = (as above) in
    let rec_ch = new Netftp_data_endpoint.write_out_record_channel
                       ~repr:(`ASCII_unix `Enc_iso88591)
                       ch
    ...
    ... ~store:(fun _ -> `Record_structure rec_ch)

Here, the end-of-record is transcoded to newline. Note that the ASCII variant (`Enc_iso88591) is ignored by write_out_record_channel. Open: How to select record structure using an FTP method.

Character conversions: To convert an EBCDIC file to ASCII, use something like

 
    let ch = (as above) in
    let converter = new Netftp_data_endpoint.data_converter
                         ~fromrepr:(`EBCDIC `Enc_cp1047)
                         ~torepr:(`ASCII_unix `Enc_iso88591) in
    let ch_ebcdic = new Netchannels.output_filter converter ch in
    ...
    ... ~representation:(`EBCDIC None)
    ... ~store:(fun _ -> `File_structure ch_ebcdic)

The class data_converter also performs the transformation of the end-of-line convention, unlike the similar class Netconversion.conversion_pipe.

Debugging

module Debug: sig .. end

This web site is published by Informatikbüro Gerd Stolpmann

Plasma	GitLab	Archive
Projects	Blog	Knowledge