Module Nn_db

module Nn_db: sig .. end

Namenode access routines to database

class type ro_transaction = object .. end

class type transaction = object .. end

type inodeinfo = Pfs_rpcapi_aux.inodeinfo

type allocinfo = {

`ai_inode :int64;`	`(*`	inode number	`*)`
`ai_index :int64;`	`(*`	index position in the file	`*)`
`ai_identity :string;`	`(*`	identity of the datanode	`*)`
`ai_block :int64;`	`(*`	block number of the datanode	`*)`

}

The allocinfo records contains information about selected blocks of a file. Usually, the blocks are remembered that are useful to know for extending the file at the end. These are the last blocks on each datanode where the file has blocks.

val update_allocinfo : int64 ->
       allocinfo list ->
       Nn_blocklist.blocklist -> (string, allocinfo) Hashtbl.t

update_allocinfo inode ailist bilist: Returns a hash table ht which maps each datanode identity to the selected allocinfo block. In ailist the old list of allocinfo block is passed. This list is updated with the information from bilist, which are the just allocated blocks.

val init : Pfs_db.db_config -> Unixqueue.event_system -> unit

The esys is used for ro transactions

val ro_esys : unit -> Unixqueue.event_system

Returns the esys for ro transactions

val commit_all_prepared_transactions : unit -> unit

Commits all prepared transactions at system startup

val transact : Unixqueue.event_system -> int -> transaction

Finds an unused connection, and start a transaction. The int is the commit_id

val ro_transact : unit -> ro_transaction Uq_engines.engine

Same for read-only transactions. Note that this is an engine: If the number of connections has reached the limit, it is waited until an existing connection becomes unused.

val with_ro_trans : (Pfs_db.ro_async_connection -> Unixqueue.event_system -> 'a Uq_engines.engine) ->
       'a Uq_engines.engine

with_ro_trans f: Runs e = f conn esys for a shared read-only transaction and returns e

val datastore_list_e : Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> Nn_datastores.datastore list Uq_engines.engine

Gets a list of all datastores

val datastore_list_ro_e : unit -> Nn_datastores.datastore list Uq_engines.engine

same, using an ro transaction

val datastore_find_e : identity:string ->
       Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> Nn_datastores.datastore option Uq_engines.engine

Find a datastore by identity

val datastore_find_ro_e : identity:string -> Nn_datastores.datastore option Uq_engines.engine

same, using an ro transaction

val datastore_upd_e : id:int ->
       identity:string ->
       size:int64 ->
       enabled:bool ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Updates the datastore table. If the record is new, it is added.

The blockalloc table is updated, too: For new stores, the rows are added. If the size of the existing store is increased, further rows are added.

It is an error to decrease the size.

val datastore_del_e : id:int ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Deletes the datastore with this ID and all rows referencing it

type ds_init_source = [ `Datastores of Nn_datastores.datastore list | `From_datastore_news_channel ]

val ds_cache_init : ds_init_source -> unit

This module provides an internal cache of the datastore ID to datastore identity mapping. One can choose from where to initialize this cache:

`From_datastore_news_channel: Calls Nn_datastore_news.list to get a list of datastores, and automatically fetches any updates. This is the right setting for the coordinator.
`Datastores l: Uses the mapping l. This is the right setting for slaves

The internal cache is used for all the following query/update functions that need this mapping. The datastore_* functions above do not use the cache (they can be called to initialize the cache), and also do neither set nor update the cache.

val ds_cache_contents : unit -> Nn_datastores.datastore list

Gets the contents of the cache

val revision_get_e : Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> string Uq_engines.engine

Get the maximum revision string

val revision_get_ro_e : unit -> string Uq_engines.engine

same, using an ro transaction

val revision_get_checksum_e : Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> string Uq_engines.engine

Get the checksum of all revision strings

val revision_upd_e : revstr:string ->
       commit_id:int ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Inserts or updates the revision string of a committer

val revision_clear_e : Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Deletes all revisions

val blockalloc_list_e : datastore:int ->
       blkidx_min:int64 ->
       blkidx_max:int64 ->
       Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> (int64 * string) list Uq_engines.engine

Reads the blockalloc table for this datastore ID. Returns the table as pairs (blkidx,blkmap). Only pairs are returned with blkidx_min <= blkidx <= blkidx_max.

val blockalloc_list_ro_e : datastore:int ->
       blkidx_min:int64 ->
       blkidx_max:int64 -> (int64 * string) list Uq_engines.engine

same, using an ro transaction

val blockalloc_upd_e : datastore:int ->
       blkidx:int64 ->
       blkmap:string ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Updates the blockalloc table

val inode_listall_e : Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> int64 list Uq_engines.engine

list all inode numbers

val inode_get_e : id:int64 ->
       Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> inodeinfo option Uq_engines.engine

Loads the inode data if existing

val inode_get_ro_e : id:int64 -> inodeinfo option Uq_engines.engine

same, using an ro transaction

val inode_maxid_e : Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> int64 Uq_engines.engine

Returns the max ID

val inode_maxid_ro_e : unit -> int64 Uq_engines.engine

same, using an ro transaction

val inode_ins_e : id:int64 ->
       inodeinfo ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Inserts a new row into inode

val inode_upd_e : id:int64 ->
       inodeinfo ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Updates a row in inode. filetype cannot be updated, and changes are silently ignored.

val inode_upd_time_e : id:int64 ->
       mtime:Pfs_rpcapi_aux.time option ->
       ctime:Pfs_rpcapi_aux.time option ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Sets the mtime and/or ctime. It is not an error if the inodes does not exist anymore.

val inode_set_anonymous_e : id:int64 ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Sets the anonymous flag of the inode

val inode_del_e : id:int64 ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Deletes a row from inode

val inode_get_anonymous_e : n:int ->
       Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> int64 list Uq_engines.engine

Returns n inodes with set anonymous field

val inode_get_anonymous_ro_e : n:int -> int64 list Uq_engines.engine

val inodeblocks_get_e : inode:int64 ->
       blkidx:int64 ->
       len:int64 ->
       Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> Nn_blocklist.blocklist Uq_engines.engine

Gets the block information for inode in the range from index blkidx to blkidx+len-1.

Note that the returned blocks use an empty ticket not granting any access!

val inodeblocks_get_ro_e : inode:int64 ->
       blkidx:int64 -> len:int64 -> Nn_blocklist.blocklist Uq_engines.engine

same, using an ro transaction

val inodeblocks_getall_e : inode:int64 ->
       dsid:int ->
       Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> (int64 * int64 * int64) list Uq_engines.engine

Returns list of (blkidx,block,length) for this inode and this datastore dsid (for fsck)

val inodeblocks_del_e : inode:int64 ->
       blkidx:int64 ->
       len:int64 ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Delete the blocks for inode in the range from index blkidx to blkidx+len-1.

val inodeblocks_ins_e : inode:int64 ->
       Nn_blocklist.blocklist ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Insert these blocks. Also, the allocinfo table is updated

val allocinfo_get_e : inode:int64 ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> allocinfo list Uq_engines.engine

Retrieves the complete allocinfo for this inode

val allocinfo_get_ro_e : inode:int64 -> allocinfo list Uq_engines.engine

same, using an ro transaction

val names_get_e : dir_inode:int64 ->
       name:string ->
       Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> int64 option Uq_engines.engine

Look up inode by dir_inode and name

val names_get_ro_e : dir_inode:int64 -> name:string -> int64 option Uq_engines.engine

same, using an ro transaction

val names_rev_get_e : inode:int64 ->
       Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> (int64 * string) list Uq_engines.engine

Look up names by inode. The return values are the inode of the directory and the member name. (No complete paths.)

val names_rev_get_ro_e : inode:int64 -> (int64 * string) list Uq_engines.engine

same, using an ro transaction

val names_get_parent_e : inode:int64 ->
       Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> int64 option Uq_engines.engine

Look up the parent inode of an inode. This works only reliably for directories - file inodes can have multiple parents, and in this case None is returned.

val names_get_parent_ro_e : inode:int64 -> int64 option Uq_engines.engine

same, using an ro transaction

val names_count_e : inode:int64 ->
       Pfs_db.ro_async_connection -> Unixqueue.event_system -> int Uq_engines.engine

Count names of an inode

val names_count_ro_e : inode:int64 -> int Uq_engines.engine

same, using an ro transaction

val names_list_e : dir_inode:int64 ->
       Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> (string * int64) list Uq_engines.engine

Lists the contents of the directory identified by dir_inode. The contents are the member names and the inodes of the members.

val names_list_ro_e : dir_inode:int64 -> (string * int64) list Uq_engines.engine

same, using an ro transaction

val names_ins_e : dir_inode:int64 ->
       name:string ->
       inode:int64 ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Insert this name to inode mapping

val names_del_e : dir_inode:int64 ->
       name:string ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Delete this name

val admin_table_get_e : key:string ->
       Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> string Uq_engines.engine

Gets the contents of the admin table key

val admin_table_get_ro_e : key:string -> string Uq_engines.engine

same, using an ro transaction

val admin_table_exists_e : key:string ->
       Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> bool Uq_engines.engine

Just return whether the key exists

val admin_table_exists_ro_e : key:string -> bool Uq_engines.engine

same, using an ro transaction

val admin_table_put_e : key:string ->
       contents:string ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Overwrite this admin table

val read_ug_admin_e : Pfs_db.ro_async_connection ->
       Unixqueue.event_system -> (string * string) list Uq_engines.engine

Two-phase commit support

type modification = [ `Admin_table_put of string * string
       | `Blockalloc_upd of int * int64 * string
       | `Datastore_del of int
       | `Datastore_upd of int * string * int64 * bool
       | `Inode_del of int64
       | `Inode_ins of int64 * inodeinfo
       | `Inode_set_anonymous of int64
       | `Inode_upd of int64 * inodeinfo
       | `Inode_upd_time of
           int64 * Pfs_rpcapi_aux.time option * Pfs_rpcapi_aux.time option
       | `Inodeblocks_del of int64 * int64 * int64
       | `Inodeblocks_ins of int64 * Nn_blocklist.blocklist
       | `Names_del of int64 * string
       | `Names_ins of int64 * string * int64
       | `Revision_clear
       | `Revision_upd of int * string ]

Database modification operations that are covered by the 2-phase commit protocol. The names of the operations correspond to those in Nn_db and Nn_slave.

val exec_e : modification ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Performs the passed modification

val consistency_check_e : modification list ->
       string ->
       Pfs_db.rw_async_connection ->
       Unixqueue.event_system -> unit Uq_engines.engine

Performs a quite costly consistency check. Especially, it is checked whether the updates on the blocklists for the inode are compatible with the updates on the blockmaps.

The string is the name of the transaction.

If the check fails, the function enters error state

This web site is published by Informatikbüro Gerd Stolpmann

Plasma	GitLab	Archive
Projects	Blog	Knowledge