Plasma GitLab Archive
Projects Blog Knowledge

Module Nn_db


module Nn_db: sig .. end
Namenode access routines to database

class type ro_transaction = object .. end
class type transaction = object .. end
type inodeinfo = Pfs_rpcapi_aux.inodeinfo 

type allocinfo = {
   ai_inode : int64; (*inode number*)
   ai_index : int64; (*index position in the file*)
   ai_identity : string; (*identity of the datanode*)
   ai_block : int64; (*block number of the datanode*)
}
The allocinfo records contains information about selected blocks of a file. Usually, the blocks are remembered that are useful to know for extending the file at the end. These are the last blocks on each datanode where the file has blocks.
val update_allocinfo : int64 ->
allocinfo list ->
Nn_blocklist.blocklist -> (string, allocinfo) Hashtbl.t
update_allocinfo inode ailist bilist: Returns a hash table ht which maps each datanode identity to the selected allocinfo block. In ailist the old list of allocinfo block is passed. This list is updated with the information from bilist, which are the just allocated blocks.
val init : Pfs_db.db_config -> Unixqueue.event_system -> unit
The esys is used for ro transactions
val ro_esys : unit -> Unixqueue.event_system
Returns the esys for ro transactions
val commit_all_prepared_transactions : unit -> unit
Commits all prepared transactions at system startup
val transact : Unixqueue.event_system -> int -> transaction
Finds an unused connection, and start a transaction. The int is the commit_id
val ro_transact : unit -> ro_transaction Uq_engines.engine
Same for read-only transactions. Note that this is an engine: If the number of connections has reached the limit, it is waited until an existing connection becomes unused.
val with_ro_trans : (Pfs_db.ro_async_connection -> Unixqueue.event_system -> 'a Uq_engines.engine) ->
'a Uq_engines.engine
with_ro_trans f: Runs e = f conn esys for a shared read-only transaction and returns e
val datastore_list_e : Pfs_db.ro_async_connection ->
Unixqueue.event_system -> Nn_datastores.datastore list Uq_engines.engine
Gets a list of all datastores
val datastore_list_ro_e : unit -> Nn_datastores.datastore list Uq_engines.engine
same, using an ro transaction
val datastore_find_e : identity:string ->
Pfs_db.ro_async_connection ->
Unixqueue.event_system -> Nn_datastores.datastore option Uq_engines.engine
Find a datastore by identity
val datastore_find_ro_e : identity:string -> Nn_datastores.datastore option Uq_engines.engine
same, using an ro transaction
val datastore_upd_e : id:int ->
identity:string ->
size:int64 ->
enabled:bool ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Updates the datastore table. If the record is new, it is added.

The blockalloc table is updated, too: For new stores, the rows are added. If the size of the existing store is increased, further rows are added.

It is an error to decrease the size.

val datastore_del_e : id:int ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Deletes the datastore with this ID and all rows referencing it
type ds_init_source = [ `Datastores of Nn_datastores.datastore list | `From_datastore_news_channel ] 
val ds_cache_init : ds_init_source -> unit
This module provides an internal cache of the datastore ID to datastore identity mapping. One can choose from where to initialize this cache:
  • `From_datastore_news_channel: Calls Nn_datastore_news.list to get a list of datastores, and automatically fetches any updates. This is the right setting for the coordinator.
  • `Datastores l: Uses the mapping l. This is the right setting for slaves
The internal cache is used for all the following query/update functions that need this mapping. The datastore_* functions above do not use the cache (they can be called to initialize the cache), and also do neither set nor update the cache.
val ds_cache_contents : unit -> Nn_datastores.datastore list
Gets the contents of the cache
val revision_get_e : Pfs_db.ro_async_connection ->
Unixqueue.event_system -> string Uq_engines.engine
Get the maximum revision string
val revision_get_ro_e : unit -> string Uq_engines.engine
same, using an ro transaction
val revision_get_checksum_e : Pfs_db.ro_async_connection ->
Unixqueue.event_system -> string Uq_engines.engine
Get the checksum of all revision strings
val revision_upd_e : revstr:string ->
commit_id:int ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Inserts or updates the revision string of a committer
val revision_clear_e : Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Deletes all revisions
val blockalloc_list_e : datastore:int ->
blkidx_min:int64 ->
blkidx_max:int64 ->
Pfs_db.ro_async_connection ->
Unixqueue.event_system -> (int64 * string) list Uq_engines.engine
Reads the blockalloc table for this datastore ID. Returns the table as pairs (blkidx,blkmap). Only pairs are returned with blkidx_min <= blkidx <= blkidx_max.
val blockalloc_list_ro_e : datastore:int ->
blkidx_min:int64 ->
blkidx_max:int64 -> (int64 * string) list Uq_engines.engine
same, using an ro transaction
val blockalloc_upd_e : datastore:int ->
blkidx:int64 ->
blkmap:string ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Updates the blockalloc table
val inode_listall_e : Pfs_db.ro_async_connection ->
Unixqueue.event_system -> int64 list Uq_engines.engine
list all inode numbers
val inode_get_e : id:int64 ->
Pfs_db.ro_async_connection ->
Unixqueue.event_system -> inodeinfo option Uq_engines.engine
Loads the inode data if existing
val inode_get_ro_e : id:int64 -> inodeinfo option Uq_engines.engine
same, using an ro transaction
val inode_maxid_e : Pfs_db.ro_async_connection ->
Unixqueue.event_system -> int64 Uq_engines.engine
Returns the max ID
val inode_maxid_ro_e : unit -> int64 Uq_engines.engine
same, using an ro transaction
val inode_ins_e : id:int64 ->
inodeinfo ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Inserts a new row into inode
val inode_upd_e : id:int64 ->
inodeinfo ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Updates a row in inode. filetype cannot be updated, and changes are silently ignored.
val inode_upd_time_e : id:int64 ->
mtime:Pfs_rpcapi_aux.time option ->
ctime:Pfs_rpcapi_aux.time option ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Sets the mtime and/or ctime. It is not an error if the inodes does not exist anymore.
val inode_set_anonymous_e : id:int64 ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Sets the anonymous flag of the inode
val inode_del_e : id:int64 ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Deletes a row from inode
val inode_get_anonymous_e : n:int ->
Pfs_db.ro_async_connection ->
Unixqueue.event_system -> int64 list Uq_engines.engine
Returns n inodes with set anonymous field
val inode_get_anonymous_ro_e : n:int -> int64 list Uq_engines.engine
val inodeblocks_get_e : inode:int64 ->
blkidx:int64 ->
len:int64 ->
Pfs_db.ro_async_connection ->
Unixqueue.event_system -> Nn_blocklist.blocklist Uq_engines.engine
Gets the block information for inode in the range from index blkidx to blkidx+len-1.

Note that the returned blocks use an empty ticket not granting any access!

val inodeblocks_get_ro_e : inode:int64 ->
blkidx:int64 -> len:int64 -> Nn_blocklist.blocklist Uq_engines.engine
same, using an ro transaction
val inodeblocks_getall_e : inode:int64 ->
dsid:int ->
Pfs_db.ro_async_connection ->
Unixqueue.event_system -> (int64 * int64 * int64) list Uq_engines.engine
Returns list of (blkidx,block,length) for this inode and this datastore dsid (for fsck)
val inodeblocks_del_e : inode:int64 ->
blkidx:int64 ->
len:int64 ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Delete the blocks for inode in the range from index blkidx to blkidx+len-1.
val inodeblocks_ins_e : inode:int64 ->
Nn_blocklist.blocklist ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Insert these blocks. Also, the allocinfo table is updated
val allocinfo_get_e : inode:int64 ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> allocinfo list Uq_engines.engine
Retrieves the complete allocinfo for this inode
val allocinfo_get_ro_e : inode:int64 -> allocinfo list Uq_engines.engine
same, using an ro transaction
val names_get_e : dir_inode:int64 ->
name:string ->
Pfs_db.ro_async_connection ->
Unixqueue.event_system -> int64 option Uq_engines.engine
Look up inode by dir_inode and name
val names_get_ro_e : dir_inode:int64 -> name:string -> int64 option Uq_engines.engine
same, using an ro transaction
val names_rev_get_e : inode:int64 ->
Pfs_db.ro_async_connection ->
Unixqueue.event_system -> (int64 * string) list Uq_engines.engine
Look up names by inode. The return values are the inode of the directory and the member name. (No complete paths.)
val names_rev_get_ro_e : inode:int64 -> (int64 * string) list Uq_engines.engine
same, using an ro transaction
val names_get_parent_e : inode:int64 ->
Pfs_db.ro_async_connection ->
Unixqueue.event_system -> int64 option Uq_engines.engine
Look up the parent inode of an inode. This works only reliably for directories - file inodes can have multiple parents, and in this case None is returned.
val names_get_parent_ro_e : inode:int64 -> int64 option Uq_engines.engine
same, using an ro transaction
val names_count_e : inode:int64 ->
Pfs_db.ro_async_connection -> Unixqueue.event_system -> int Uq_engines.engine
Count names of an inode
val names_count_ro_e : inode:int64 -> int Uq_engines.engine
same, using an ro transaction
val names_list_e : dir_inode:int64 ->
Pfs_db.ro_async_connection ->
Unixqueue.event_system -> (string * int64) list Uq_engines.engine
Lists the contents of the directory identified by dir_inode. The contents are the member names and the inodes of the members.
val names_list_ro_e : dir_inode:int64 -> (string * int64) list Uq_engines.engine
same, using an ro transaction
val names_ins_e : dir_inode:int64 ->
name:string ->
inode:int64 ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Insert this name to inode mapping
val names_del_e : dir_inode:int64 ->
name:string ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Delete this name
val admin_table_get_e : key:string ->
Pfs_db.ro_async_connection ->
Unixqueue.event_system -> string Uq_engines.engine
Gets the contents of the admin table key
val admin_table_get_ro_e : key:string -> string Uq_engines.engine
same, using an ro transaction
val admin_table_exists_e : key:string ->
Pfs_db.ro_async_connection ->
Unixqueue.event_system -> bool Uq_engines.engine
Just return whether the key exists
val admin_table_exists_ro_e : key:string -> bool Uq_engines.engine
same, using an ro transaction
val admin_table_put_e : key:string ->
contents:string ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Overwrite this admin table
val read_ug_admin_e : Pfs_db.ro_async_connection ->
Unixqueue.event_system -> (string * string) list Uq_engines.engine

Two-phase commit support


type modification = [ `Admin_table_put of string * string
| `Blockalloc_upd of int * int64 * string
| `Datastore_del of int
| `Datastore_upd of int * string * int64 * bool
| `Inode_del of int64
| `Inode_ins of int64 * inodeinfo
| `Inode_set_anonymous of int64
| `Inode_upd of int64 * inodeinfo
| `Inode_upd_time of
int64 * Pfs_rpcapi_aux.time option * Pfs_rpcapi_aux.time option
| `Inodeblocks_del of int64 * int64 * int64
| `Inodeblocks_ins of int64 * Nn_blocklist.blocklist
| `Names_del of int64 * string
| `Names_ins of int64 * string * int64
| `Revision_clear
| `Revision_upd of int * string ]
Database modification operations that are covered by the 2-phase commit protocol. The names of the operations correspond to those in Nn_db and Nn_slave.
val exec_e : modification ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Performs the passed modification
val consistency_check_e : modification list ->
string ->
Pfs_db.rw_async_connection ->
Unixqueue.event_system -> unit Uq_engines.engine
Performs a quite costly consistency check. Especially, it is checked whether the updates on the blocklists for the inode are compatible with the updates on the blockmaps.

The string is the name of the transaction.

If the check fails, the function enters error state

This web site is published by Informatikbüro Gerd Stolpmann
Powered by Caml