Plasma GitLab Archive
Projects Blog Knowledge

Index of class methods


A
abort [Mapred_io.record_writer]
Drops resources - intended to be used for error cleanup
abort [Mapred_io.record_reader]
Drops resources - intended to be used for error cleanup
access [Nn_blockmap_box.blockmap_client_t]
Access a blockmap stored in DB row id and for datastore identity, with an initial size.
active [Nn_blockmap.blockmap_t]
whether this object refers to an existing row in the datastore table.
add_group [Plasma_ug.ug_admin]
Add a group.
add_user [Plasma_ug.ug_admin]
Add a user.
admin_table_put_e [Nn_state.ds_view_t]
Write an admin table (this does not fit anywhere else)
admin_tables [Plasma_ug.ug_admin_ro]
The contents of this object as admin tables.
admin_tables [Plasma_ug.ug_admin]
The contents of this object as admin tables.
allocinfo_get_e [Nn_state.inode_view_t]
Gets the allocinfo list for this inode
as_owner [Nn_state.inode_view_t]
The owner tuple used by this view
auth_ticket [Mapred_def.mapred_env]
The authentication ticket

B
bigblock_size [Mapred_def.mapred_job_config]
Map/reduce processes files in units of bigblocks.
bigblock_size [Mapred_io.record_config]
blockmap_alloc_e [Nn_state.bm_view_t]
Reserve blocks and allocate at commit time.
blockmap_allocsize [Nn_state.bm_view_t]
Returns the allocation size
blockmap_dealloc_e [Nn_state.bm_view_t]
Mark blocks and deallocate at commit time.
blockmap_get_e [Nn_state.bm_view_t]
Get the contents of the blockalloc table for this datastore identity, wrapped into a blockmap_t object
blockmap_get_e [Nn_state.shared_state_t]
If the blockmap exists, return it.
blockmap_load_e [Nn_state.bm_view_t]
Ensures that all known blockmaps are loaded
blockmap_pin [Nn_state.bm_view_t]
Pin these blocks until commit/rollback.
blockmap_serialized [Nn_state.shared_state_t]
This is used to synchronize accesses to the in-memory blockmap with the per-inode blocklists in the db.
blockmaps [Nn_state.shared_state_t]
All blockmaps so far loaded
book_allocation [Plasma_shm.shm_manager]
Books this ID with this (total) size, and pushes the new entry
bpos_out [Mapred_io.record_writer]
The byte position of the record that will be written next

C
cauth [Nn_state.shared_state_t]
Return the cauth record
check_config [Mapred_def.mapred_job]
Check the config.
close [Nn_blockmap_box.blockmap_client_t]
Close the client (deallocate the message box)
close_in [Mapred_io.record_reader]
Releases resources (e.g.
close_out [Mapred_io.record_writer]
Releases resources (e.g.
clustername [Plasma_client_config.client_config]
The clustername
commit [Nn_blockmap.blockmap_t]
commit owner: commits owner's reservations, and makes them permanent.
commit [Nn_db.transaction]
Commits the transaction (prepared or not), and returns
condition [Plasma_shm.shm_manager]
Estimate where we are
config [Mapred_def.mapred_env]
The map/reduce server configuration
config_file [Mapred_def.mapred_env]
The config file containing config
connection [Nn_db.transaction]
The connection.
connection [Nn_db.ro_transaction]
The connection.
continue_with [Pfs_db.async_connection]
continue_with f: Set f as the continuation.
copy_to_local_directory [Mapred_taskfiles.taskfile_manager]
Copies the mentioned file to the task nodes, and places them into the local_directory.
cur_rev_ident [Nn_state.shared_state_t]
Returns current revision identifier
custom [Mapred_def.mapred_job_config]
Get a custom parameter or raise Not_found
custom_params [Mapred_def.mapred_job]
The list of allowed custom parameters

D
datastore_del_e [Nn_state.ds_view_t]
Delete this row in the datastore table
datastore_list_e [Nn_state.ds_view_t]
Get the contents of the datastore table
datastore_max_id [Nn_state.shared_state_t]
Maximum ID used in the datastore table (None if use_id was never called)
datastore_new_id_e [Nn_state.ds_view_t]
The secret random number used for securing transactional writes
datastore_upd_e [Nn_state.ds_view_t]
Change the row in datastore identified by id so that the fields identity, size, and enabled are updated.
datastore_use_id [Nn_state.shared_state_t]
Record that this ID is now also used in a transaction
db_dbname [Pfs_db.db_config]
db_host [Pfs_db.db_config]
db_hostaddr [Pfs_db.db_config]
db_password [Pfs_db.db_config]
db_port [Pfs_db.db_config]
db_ro_connections_max [Pfs_db.db_config]
db_user [Pfs_db.db_config]
del_group [Plasma_ug.ug_admin]
Delete the group.
del_user [Plasma_ug.ug_admin]
Delete the user
delayed_allocation [Plasma_shm.shm_manager]
A delayed allocation is not immediately pushed to the shared book.
delayed_inode_deletes [Nn_state.inode_view_t]
Inodes to be deleted at the end of the transaction.
delete_local_directory [Mapred_taskfiles.taskfile_manager]
Deletes the local directory on the task nodes.
disable_master [Nn_state.shared_state_t]
Disable this master
disable_slave [Nn_state.shared_state_t]
disable_slave name
dn_blocksize [Nn_config.nn_node_config]
Expected blocksize of datanodes
dn_blocksize [Dn_config.dn_config]
The blocksize.
dn_blocksize [Dn_shm.dn_shm_config]
The blocksize.
dn_blocksize [Dn_store.dn_store_config]
The blocksize.
dn_clustername [Dn_config.dn_config]
The clustername
dn_directory [Dn_config.dn_config]
The directory containing the store.
dn_directory [Dn_store.dn_store_config]
The directory containing the store.
dn_io_processes [Dn_config.dn_config]
The number of dn_io processes.
dn_max_errors [Nn_config.nn_node_config]
For the liveliness monitoring: how many errors (missing response messages) are at most acceptable.
dn_multicast_addr [Nn_config.nn_node_config]
The address to use for multicasting.
dn_multicast_period [Nn_config.nn_node_config]
Every how many seconds a new multicast messge is emitted
dn_multicast_ttl [Nn_config.nn_node_config]
The number of hops (routers+1) for multicasting.
dn_node_timeout [Nn_config.nn_node_config]
The timeout for connects and RPC calls to data nodes
dn_port [Nn_config.nn_node_config]
The data node port
dn_security_level [Nn_config.nn_node_config]
The security level the datanodes expect
dn_shm_queue_length [Dn_config.dn_config]
The size of the shm queue, in blocks.
dn_shm_queue_length [Dn_shm.dn_shm_config]
The size of the shm queue, in blocks.
dn_sync_period [Dn_config.dn_config]
How often the file is synced to disk
dump [Nn_blockmap.blockmap_t]
Get a dump of the internal state
dump [Plasma_shm.shm_manager]
Current sums for `Shm and `Buf as debug string

E
enhanced_mapping [Mapred_def.mapred_job_config]
If >0, enhanced map tasks are created.
enough_slaves [Nn_state.shared_state_t]
Whether there are still enough enabled slaves
event_system [Nn_state.shared_state_t]
The common event system
event_system [Nn_db.transaction]
The event system this transaction uses
event_system [Nn_db.ro_transaction]
The event system this transaction uses
event_system [Plasma_inodecache.lazy_transaction]
extract_key [Mapred_def.mapred_job]
Extracts the key from a record.

F
filename_count_e [Nn_state.inode_view_t]
The count is the number of names for an inode (same as length of list returned by filename_rev_get_e)
filename_get_e [Nn_state.inode_view_t]
Returns the inode of the member name of directory dir_inode
filename_link_e [Nn_state.inode_view_t]
adds this filename.
filename_list_e [Nn_state.inode_view_t]
Lists subnames of a potential directory name, together with inodes
filename_parent_dir_e [Nn_state.inode_view_t]
Returns the parent directory
filename_rename_e [Nn_state.inode_view_t]
filename_rev_get_dir_e [Nn_state.inode_view_t]
Works only for directories, and obtains an existence lock for the directory.
filename_rev_get_e [Nn_state.inode_view_t]
filename_rev_get does not acquire locks.
filename_unlink_e [Nn_state.inode_view_t]
deletes this filename from this directory.
fill_blockmap [Nn_blockmap.blockmap_t]
fill_blockmap index data: Fills the blockmap at index with data, a string consisting of 0 and 1.
fill_done [Nn_blockmap.blockmap_t]
Set that the map is filled, i.e.
fill_from_db [Nn_blockmap.blockmap_t]
fill_from_db index n: Fills the n rows at index from the db (synchronously)
filled [Nn_blockmap.blockmap_t]
whether the object is completely filled with data
finish_transaction_serialized [Nn_state.shared_state_t]
finish_transaction_serialized f: Run f esys in a serialized way.
flush [Mapred_io.record_writer]
Flushes the records from the buffer to disk
forget_identity [Nn_state.shared_state_t]
Forget everything about this datastore identity
free [Nn_blockmap.blockmap_t]
free l owner: deallocates these blocks.
from_fd_e [Mapred_io.record_writer]
Outputs the records coming from this file.

G
get_blockmap [Nn_state.shared_state_t]
Get the blockmap for this identity, or Not_found
get_changes [Nn_blockmap.blockmap_t]
get_changes owner: Returns the changes made by owner.
get_tid_e [Plasma_inodecache.lazy_transaction]
getgrgid [Plasma_ug.ug_admin_ro]
Find a group entry by name or by gid
getgrgid [Plasma_ug.ug_admin]
Find a group entry by name or by gid
getgrnam [Plasma_ug.ug_admin_ro]
getgrnam [Plasma_ug.ug_admin]
getgroups [Plasma_ug.ug_admin_ro]
getgroups user: Returns the set of effective group memberships for user
getgroups [Plasma_ug.ug_admin]
getgroups user: Returns the set of effective group memberships for user
getpwnam [Plasma_ug.ug_admin_ro]
getpwnam [Plasma_ug.ug_admin]
getpwuid [Plasma_ug.ug_admin_ro]
Find a passwd entry by name or by uid
getpwuid [Plasma_ug.ug_admin]
Find a passwd entry by name or by uid
groups [Plasma_ug.ug_admin_ro]
Return the users and groups
groups [Plasma_ug.ug_admin]
Return the users and groups

H
has_free_blocks [Nn_blockmap.blockmap_t]
whether there are still free blocks

I
id [Nn_blockmap.blockmap_t]
the datastore db table id
identity [Nn_blockmap.blockmap_t]
This is the blockmap for this datastore identity
idle [Pfs_db.async_connection]
whether nothing is being executed
inactivate [Nn_blockmap_box.blockmap_client_t]
Inactivate
inactivate [Nn_blockmap.blockmap_t]
Sets active to false
inode_alloc_e [Nn_state.inode_view_t]
Inserts a new inode and returns the ID.
inode_dealloc_e [Nn_state.inode_view_t]
deletes the inode.
inode_get_blocks_e [Nn_state.inode_view_t]
returns blocks.
inode_get_e [Nn_state.inode_view_t]
inode_get_new_id_serialized [Nn_state.shared_state_t]
Serializer for getting a new ID from the inode table
inode_get_with_lock_e [Nn_state.inode_view_t]
like inode_get_e but immediately acquires a write lock for the inode
inode_max_id [Nn_state.shared_state_t]
Maximum ID used in the inode table (None if use_id was never called)
inode_upd_blocks_e [Nn_state.inode_view_t]
First delete the blocks in the del range, then add the blocks from add.
inode_upd_e [Nn_state.inode_view_t]
Updates the inode.
inode_upd_time [Nn_state.inode_view_t]
This update of mtime/ctime is lock-free
inode_use_id [Nn_state.shared_state_t]
Record that this ID is now also used in a transaction
input_dir [Mapred_def.mapred_job_config]
This plasma directory contains the input files
input_record [Mapred_io.record_reader]
Reads one record, and advances the cursor.
input_records [Mapred_io.record_reader]
Reads some records and appends them to the passed queue.
is_slave_disabled [Nn_state.shared_state_t]
Whether this slave is disabled

J
job_id [Mapred_def.mapred_job_config]
An automatically generated name.

L
local_directory [Mapred_taskfiles.taskfile_manager]
The directory path on the task nodes for files that are needed by tasks
lock_filename [Nn_state.shared_state_t]
lock_filename (dir_inode,name) lock_type owner: Locks the name member of directory dir_inode.
lock_inode [Nn_state.shared_state_t]
Returns Some unlock if successful where calling unlock removes the lock, or reverts to the previous state (1-step undo)
log_dir [Mapred_def.mapred_job_config]
This plasma directory is used for log files.
log_directory [Mapred_taskfiles.taskfile_manager]
The directory path on the task nodes where the tasks can put files, and these files are finally copied to PlasmaFS

M
map [Mapred_def.mapred_job]
The mapper reads records, maps them, and writes them into a second file.
map_tasks [Mapred_def.mapred_job_config]
The number of map tasks that should not be exceeded.
master [Nn_state.shared_state_t]
The master in host:port syntax
master_enabled [Nn_state.shared_state_t]
Whether the master (i.e.
merge_limit [Mapred_def.mapred_job_config]
How many files are merged at most by a shuffle task
minimum_slaves [Nn_state.shared_state_t]
How many enabled slaves we need at minimum
mr_buf_high [Mapred_config.mapred_config]
High watermark for buffer memory.
mr_buf_high_factor [Mapred_config.mapred_config]
Alternate way for setting the high watermark as fraction of available physical RAM.
mr_buf_low [Mapred_config.mapred_config]
Low watermark for buffer memory.
mr_buf_low_factor [Mapred_config.mapred_config]
Alternate way for setting the low watermark as fraction of available physical RAM.
mr_buf_max [Mapred_config.mapred_config]
Maximum for buffer memory.
mr_buf_max_factor [Mapred_config.mapred_config]
Alternate way for setting the maximum as fraction of available physical RAM.
mr_buffer_size [Mapred_io.record_config]
mr_buffer_size [Mapred_config.mapred_config]
The normal size of I/O buffers.
mr_buffer_size_tight [Mapred_io.record_config]
mr_buffer_size_tight [Mapred_config.mapred_config]
The size of I/O buffers when RAM is tight.
mr_shm_high [Mapred_config.mapred_config]
High watermark for shared memory.
mr_shm_high_factor [Mapred_config.mapred_config]
Alternate way for setting the high watermark as fraction of available shared memory.
mr_shm_low [Mapred_config.mapred_config]
Low watermark for shared memory.
mr_shm_low_factor [Mapred_config.mapred_config]
Alternate way for setting the low watermark as fraction of available shared memory.
mr_shm_max [Mapred_config.mapred_config]
Maximum for shared memory.
mr_shm_max_factor [Mapred_config.mapred_config]
Alternate way for setting the maximum as fraction of available shared memory.
mr_sort_size [Mapred_config.mapred_config]
The size of the buffers for sorting.
mr_task_load_limit [Mapred_config.mapred_config]
Load limit per task server (in number of tasks).
mr_task_nodes [Mapred_config.mapred_config]
Task nodes (only hostname)
mr_task_port [Mapred_config.mapred_config]
The port number
mr_task_tmpdir [Mapred_config.mapred_config]
A directory where to put executables, logs, etc.
multi_serialized [Pfs_condition.int64_serializer_t]
let se = serialized il f: Same as serializing over all values of il (in a nested way)

N
name [Mapred_def.mapred_job_config]
A name for identifying this job, e.g.
new_id [Plasma_shm.shm_manager]
allocates a new ID for an shm or buf object
new_rev_ident [Nn_state.shared_state_t]
Creates a new revision identifier
next [Pfs_db.async_connection]
If there is a delayed continuation execute it now
nn_clustername [Mapred_config.mapred_config]
The clustername
nn_clustername [Nn_config.nn_min_config]
nn_clustername [Nn_config.nn_node_config]
The clustername
nn_elect_timeout [Nn_config.nn_node_config]
For how many seconds to wait for neighbor namenodes
nn_hostname [Nn_config.nn_node_config]
The hostname of this node.
nn_inodecaches [Nn_config.nn_node_config]
Which inodecaches are announced to the clients
nn_lock_timeout [Nn_config.nn_node_config]
After these many seconds transactions are not retried on ECONFLICT
nn_node_alive_min [Nn_config.nn_node_config]
How many name nodes need to be alive to continue with operations
nn_node_alive_min_startup [Nn_config.nn_node_config]
How many name nodes need to be alive at cluster startup
nn_node_timeout [Nn_config.nn_min_config]
nn_node_timeout [Nn_config.nn_node_config]
The timeout for connects and RPC calls to name nodes
nn_nodes [Mapred_config.mapred_config]
The name nodes in "host:port" syntax
nn_nodes [Nn_config.nn_min_config]
nn_nodes [Nn_config.nn_node_config]
The name nodes in "host:port" syntax
nn_nodes [Plasma_client_config.client_config]
The name nodes in "host:port" syntax
nn_pool_size [Nn_config.nn_node_config]
How much pool RAM is allocated (shared memory)
nn_rank [Nn_config.nn_node_config]
The rank of this node compared to other nodes.
nn_replication [Nn_config.nn_node_config]
Default replication factor
node_config [Nn_state.shared_state_t]
the configuration

O
open_cluster [Mapred_def.mapred_env]
Opens the cluster filesystem
output_dir [Mapred_def.mapred_job_config]
This plasma directory will get output files.
output_record [Mapred_io.record_writer]
Outputs these records.

P
partition [Mapred_def.task_info]
The partition.
partition_of_key [Mapred_def.mapred_job]
Determines the partition of a key.
partitions [Mapred_def.mapred_job_config]
The number of partitions = number of reduce tasks
peek_record [Mapred_io.record_reader]
Reads the record, but does not advance the cursor, i.e.
pin [Nn_blockmap.blockmap_t]
pin l owner: marks these blocks as "to keep" even if a competing transaction frees them.
plasma_cluster [Mapred_io.record_writer]
The PlasmaFS cluster
plasma_cluster [Mapred_io.record_reader]
The PlasmaFS cluster
pos_in [Mapred_io.record_reader]
The ordinal number of the record that will be read next.
pos_out [Mapred_io.record_writer]
The ordinal number of the record that will be written next.
post_commit [Nn_state.view_t]
Needs to be called after a commit.
post_job_finish [Mapred_def.mapred_job]
This is run by the job process after the last task is finished
pre_commit [Nn_state.view_t]
Prepares the commit, and returns the journal to commit
pre_job_start [Mapred_def.mapred_job]
This is run by the job process before the first task is started
prepare [Nn_db.transaction]
Prepares the transaction for commit
push [Plasma_shm.shm_manager]
Push allocations to the shared book.

R
reduce [Mapred_def.mapred_job]
The reducer reads all the records of one partition, and puts them into an output file.
release [Nn_state.bm_view_t]
Release the pinned blocks
release [Nn_blockmap.blockmap_t]
release owner: Unpins the blocks of this owner.
release_e [Plasma_inodecache.lazy_transaction]
req_id [Mapred_def.task_info]
The request ID.
reserve [Nn_blockmap.blockmap_t]
reserve n owner ri: finds n free blocks and returns their indices.
return [Nn_db.transaction]
return the transaction to the pool or closes it
return [Nn_db.ro_transaction]
return the transaction to the pool
ro_connection [Nn_db.transaction]
The same connection coerced to read-only
rollback [Nn_state.view_t]
Rolls this view back
rollback [Nn_blockmap.blockmap_t]
rollback owner: unreserves owner's blocks.
rollback [Nn_db.transaction]
Rolls the transaction back, and returns

S
send [Pfs_db.async_connection]
Unless noprepare is set, prepare the query first, and execute it then.
serialized [Pfs_condition.int64_serializer_t]
let se = serialized i f: If already a serializer is running for the passed value of i, it is waited until the previous engines reach a final state, and then it is run e = f esys.
set_rev_ident [Nn_state.shared_state_t]
Sets the identitifer returned by cur_rev_ident
shutdown [Nn_blockmap_box.blockmap_client_t]
Shut the process down
size [Nn_blockmap.blockmap_t]
Size of the blockmap in blocks
slaves [Nn_state.shared_state_t]
Connections to the name slaves as triples (name,client,enabled).
split_limit [Mapred_def.mapred_job_config]
How many files are created by a shuffle task.
stats [Nn_blockmap.blockmap_t]
Number of used, transitional, free blocks

T
task_delete_log_directory [Mapred_taskfiles.taskfile_manager]
Deletes the entire log directory.
task_files [Mapred_def.mapred_job_config]
These files are copied at job start to the "local" directory for this job on all task nodes.
task_files_log_directory [Mapred_taskfiles.taskfile_manager]
Returns the files that are still in the log directory
task_prefix [Mapred_def.task_info]
This filename prefix is used for log files
task_save_log_directory [Mapred_taskfiles.taskfile_manager]
Copies these files of the log directory into PlasmaFS.
to_any_e [Mapred_io.record_reader]
to_any_e dest: like to_fd_e but the data is not written to a file descriptor.
to_fd_e [Mapred_io.record_reader]
to_fd_e fd: The records are written to fd.

U
unlock_filenames [Nn_state.shared_state_t]
unlock all filenames of this owner
unlock_inodes [Nn_state.shared_state_t]
unlock all inodes of this owner
users [Plasma_ug.ug_admin_ro]
users [Plasma_ug.ug_admin]

W
wm [Plasma_shm.shm_manager]
Effective watermarks
work_dir [Mapred_def.mapred_job_config]
This plasma directory is used for temporary files.

This web site is published by Informatikbüro Gerd Stolpmann
Powered by Caml