Plasma GitLab Archive
Projects Blog Knowledge

Class type Mapred_fs.filesystem

class type filesystem = object .. end
Abstract access to filesystems

Methods borrowed from

Netfs.stream_fs
method path_encoding : Netconversion.encoding option
Provided for compatibility with Netfs.stream_fs. Plasma does not care about encodings.
method path_exclusions : (int * int) list
Provided for compatibility with Netfs.stream_fs. Plasma does not care about this.
method nominal_dot_dot : bool
Provided for compatibility with Netfs.stream_fs. Plasma does not care about this.
method read : read_flag list -> string -> Netchannels.in_obj_channel
read flags filename: Opens the file filename for reading, and returns the input stream. Flags:
  • `Skip n: Skips the first n bytes of the file. On many filesystems this is more efficient than reading n bytes and dropping them; however, there is no guarantee that this optimization exists.
  • `Binary: Opens the file in binary mode (if there is such a distinction)
  • `Streaming for network filesystems: If possible, open the file in streaming mode, and avoid to copy the whole file to the local disk before returning the Netchannels.in_obj_channel. Streaming mode is faster, but has also downsides. Especially, the implementation of read can do less to recover from transient network problems (like retrying the whole download). Support for this flag is optional, and it is ignored if there is no extra streaming mode.

method read_file : read_file_flag list -> string -> local_file
read_file flags filename: Opens the file filename for reading, and returns the contents as a local_file. Use the method filename to get the file name of the local file. The file may be temporary, but this is not required. The method close of the returned object should be called when the file is no longer needed. In case of a temporary file, the file can then be deleted.

Of course, the idea is that if the backing filesystem is PlasmaFS (or any other remote fs) the requested file is copied to a local temporary file, and on close the file is deleted. If the backing filesystem is the local filesystem, no copy is done, but the real file is directly returned (w/o deletion on close).

Flags:

  • `Binary: Opens the file in binary mode (if there is such a distinction)
  • `Temp(dir,prefix): If a temporary file is created, the local directory dir will contain it, and the file will have this prefix. (If not passed, defaults will be used.)
  • `Destination p: Request that the file is made available as permanent local file p (i.e. close will not delete it). (If the filesystem is the local one, simply a symlink to the real file is created.)

method write : write_flag list -> string -> Netchannels.out_obj_channel
write flags filename: Opens (and optionally creates) the filename for writing, and returns the output stream. Flags:
  • `Create: If the file does not exist, create it
  • `Truncate: If the file exists, truncate it to zero before writing
  • `Exclusive: The `Create is done exclusively
  • `Binary: Opens the file in binary mode (if there is such a distinction)
  • `Streaming: see read (above) for explanations
  • `Repl n: the file is created with replication factor n. n=0 means to use the default.
  • `Location s: the location s is the preferred physical place for the file. This is only a suggestion. It is filesystem-dependent how locations are named. The location "_local" is reserved, and means the location where the write is done. This configuration remains (at least) in effect while this filesystem object is used to write the file (including further write and copy_in_from_buf calls).
Some filesystems refuse this operation if neither `Create nor `Truncate is specified because overwriting an existing file is not supported. There are also filesystems that cannot even modify files by truncating them first, but only allow to write to new files.

It is unspecified whether the file appears in the directory directly after calling write or first when the stream is closed.

method write_file : write_file_flag list -> string -> local_file -> unit
write_file flags filename localfile: Opens the file filename for writing, and copies the contents of the localfile to it. It is ensured that the method close of localfile is called once the operation is finished (whether successful or not). Flags:
  • `Create: If the (remote) file does not exist, create it
  • `Truncate: If the file exists, truncate it to zero before writing
  • `Exclusive: The `Create is done exclusively
  • `Binary: Opens the file in binary mode (if there is such a distinction)
  • `Link: Allows that the destination file is created as a hard link of the original file. This is tried whatever other mode is specified. If not successful, a copy is done instead.
  • `Repl n: the file is created with replication factor n. n=0 means to use the default.
  • `Location s: the location s is the preferred physical place for the file. This is only a suggestion. It is filesystem-dependent how locations are named. The location "_local" is reserved, and means the location where the write is done.

method size : size_flag list -> string -> int64
Returns the size of a file. Note that there is intentionally no distinction between text and binary mode - implementations must always assume binary mode.
method test : test_flag list -> string -> test_type -> bool
Returns whether the test is true. For filesystems that know symbolic links, the test operation normally follows symlinks (except for the `N and `H tests). By specifying the `Link flag symlinks are not followed.
method test_list : test_flag list -> string -> test_type list -> bool list
Similar to test but this function performs all tests in the list at once, and returns a bool for each test.
method remove : remove_flag list -> string -> unit
Removes the file or symlink. Implementation are free to also support the removal of empty directories.

Flags:

  • `Recursive: Remove the contents of the non-empty directory recursively. This is an optional feature. There needs not to be any protection against operations done by other processes that affect the directory tree being deleted.

method rename : rename_flag list -> string -> string -> unit
Renames the file. Plasma requires the following semantics:
  • renames must be atomic
  • if the new file exists and is regular, it is atomically replaced

method symlink : symlink_flag list -> string -> string -> unit
symlink flags oldpath newpath: Creates a symlink. This is an exclusive create, i.e. the operation fails if newpath already exists.
method readdir : readdir_flag list -> string -> string list
Reads the contents of a directory. Whether "." and ".." are returned is platform-dependent. The entries can be returned in any order.
method readlink : readlink_flag list -> string -> string
Reads the target of a symlink
method mkdir : mkdir_flag list -> string -> unit
Creates a new directory. Flags:
  • `Path: Creates missing parent directories. This is an optional feature. (If not supported, ENOENT is reported.)
  • `Nonexcl: Non-exclusive create.

method rmdir : rmdir_flag list -> string -> unit
Removes an empty directory
method copy : copy_flag list -> string -> string -> unit
Copies a file to a new name. This does not descent into directories. Also, symlinks are resolved, and the linked file is copied.

Flags:

  • `Repl n: the copied file is created with replication factor n. n=0 means to use the default.
  • `Location s: the location s is the preferred physical place for the new file. This is only a suggestion. It is filesystem-dependent how locations are named. The location "_local" is reserved, and means the location where the write is done.

method cancel : unit -> unit
Cancels any ongoing write. The user must also call the close_out method after cancelling. The effect is that after the close no more network activity will occur.

Extension Methods

method local_root : string -> string option
If the passed path can be mapped to the local filesystem, the corresponding path in the local filesystem is returned (w/o tree prefix)
method blocksize : string -> int
Returns the block size for this path
method copy_out_to_buf : string -> int64 -> Netsys_mem.memory -> int -> unit
copy_out_to_buf path pos mem len: XXX

Supports partial evaluation in order to factor the file lookup out:

 let f = fs # copy_out_to_buf path 

Now, f pos mem len can be called several times to access the same file.

method copy_in_from_buf : string -> int64 -> Netsys_mem.memory -> int -> unit
copy_in_from_buf path pos mem len: XXX

Also supports partial evaluation like copy_out_to_buf.

method commit_files : (string * string) list -> unit
Renames a couple of files at once. If any of the new file names already exists, this function will fail with EEXIST. The renames may be non-atomic, but the exclusive creation of the new files names must be atomic.
method location_info : string -> (Unix.inet_addr * string) list
Returns for the given path the association on which node a location resides.
method blocklist : string -> Plasma_rpcapi_aux.blockinfo list
Returns the blocklist of a file like PlasmaFS does
method have_link : string -> bool
whether link is supported for this path
method link : link_flag list -> string -> string -> unit
link flags old_name new_name
method set_aside : unit -> unit
This filesystem connection will not be used for some time, and resources should be given back to the OS if possible. The resources will be automatically allocated again when needed, so this is transparent to the user.
method close : unit -> unit
This filesystem will not be used anymore
method open_cluster : string -> Unixqueue.event_system -> Plasma_client.plasma_cluster option
Returns a new plasma cluster for this path, if available

The cluster object is configured so it can authenticate. This is the only configuration, though.

method open_filesystem : unit -> filesystem
Returns an independent access object for the same filesystem. This object can e.g. be used for newly started threads.
This web site is published by Informatikbüro Gerd Stolpmann
Powered by Caml