Plasma GitLab Archive
Projects Blog Knowledge

Index of class methods


A
abort [Mapred_io.record_writer]
Drops resources - intended to be used for error cleanup
abort [Mapred_io.record_reader]
Drops resources - intended to be used for error cleanup
active [Nn_blockmap.blockmap_t]
whether this object refers to an existing row in the datastore table.

B
bigblock_size [Mapred_def.mapred_job_config]
Map/reduce processes files in units of bigblocks.
bigblock_size [Mapred_io.record_config]
Same as Mapred_def.mapred_job.bigblock_size.
blockmap_alloc_e [Nn_state.bm_view_t]
Reserve blocks and allocate at commit time.
blockmap_dealloc_e [Nn_state.bm_view_t]
Mark blocks and deallocate at commit time.
blockmap_get_e [Nn_state.bm_view_t]
Get the contents of the blockalloc table for this datastore identity, wrapped into a blockmap_t object
blockmap_get_e [Nn_state.shared_state_t]
If the blockmap exists, return it.
blockmap_keep [Nn_state.bm_view_t]
Marks these blocks as "to keep" until commit/rollback.
blockmap_load_e [Nn_state.bm_view_t]
Ensures that all known blockmaps are loaded
blockmap_prioritizer [Nn_state.shared_state_t]
Used to prioritize commits before other accesses to the blockmap.
blockmaps [Nn_state.shared_state_t]
All blockmaps
book_allocation [Plasma_shm.shm_manager]
Books this ID with this (total) size
bpos_out [Mapred_io.record_writer]
The byte position of the record that will be written next

C
check_config [Mapred_def.mapred_job]
Check the config.
close_in [Mapred_io.record_reader]
Releases resources (e.g.
close_out [Mapred_io.record_writer]
Releases resources (e.g.
clustername [Plasma_client_config.client_config]
The clustername
commit [Nn_blockmap.blockmap_t]
commit owner: commits owner's reservations, and makes them permanent.
commit [Nn_db.transaction]
Commits the transaction (prepared or not), and returns
config [Mapred_def.mapred_env]
The map/reduce server configuration
config_file [Mapred_def.mapred_env]
The config file containing config
connection [Nn_db.transaction]
The connection.
connection [Nn_db.ro_transaction]
The connection.
continue_with [Pfs_db.async_connection]
continue_with f: Set f as the continuation.
copy_to_local_directory [Mapred_taskfiles.taskfile_manager]
Copies the mentioned file to the task nodes, and places them into the local_directory.
create_blockmap [Nn_state.shared_state_t]
create_blockmap m: Creates a map of free/used/reserved blocks.
cur_rev_ident [Nn_state.shared_state_t]
Returns current revision identifier
custom [Mapred_def.mapred_job_config]
Get a custom parameter or raise Not_found
custom_params [Mapred_def.mapred_job]
The list of allowed custom parameters

D
datastore_del_e [Nn_state.ds_view_t]
Delete this row in the datastore table
datastore_list_e [Nn_state.ds_view_t]
Get the contents of the datastore table
datastore_max_id [Nn_state.shared_state_t]
Maximum ID used in the datastore table (None if use_id was never called)
datastore_new_id_e [Nn_state.ds_view_t]
The secret random number used for securing transactional writes
datastore_upd_e [Nn_state.ds_view_t]
Change the row in datastore identified by id so that the fields identity, size, and enabled are updated.
datastore_use_id [Nn_state.shared_state_t]
Record that this ID is now also used in a transaction
db_dbname [Pfs_db.db_config]
db_host [Pfs_db.db_config]
db_hostaddr [Pfs_db.db_config]
db_password [Pfs_db.db_config]
db_port [Pfs_db.db_config]
db_ro_connections_max [Pfs_db.db_config]
db_user [Pfs_db.db_config]
delayed_inode_deletes [Nn_state.inode_view_t]
Inodes to be deleted at the end of the transaction
delete_local_directory [Mapred_taskfiles.taskfile_manager]
Deletes the local directory on the task nodes.
dn_blocksize [Nn_config.nn_node_config]
Expected blocksize of datanodes
dn_blocksize [Dn_config.dn_config]
The blocksize.
dn_blocksize [Dn_shm.dn_shm_config]
The blocksize.
dn_blocksize [Dn_store.dn_store_config]
The blocksize.
dn_clustername [Dn_config.dn_config]
The clustername
dn_directory [Dn_config.dn_config]
The directory containing the store.
dn_directory [Dn_store.dn_store_config]
The directory containing the store.
dn_io_processes [Dn_config.dn_config]
The number of dn_io processes.
dn_node_timeout [Nn_config.nn_node_config]
The timeout for connects and RPC calls to data nodes
dn_nodes [Nn_config.nn_node_config]
The data nodes in "host:port" syntax
dn_shm_queue_length [Dn_config.dn_config]
The size of the shm queue, in blocks.
dn_shm_queue_length [Dn_shm.dn_shm_config]
The size of the shm queue, in blocks.
dn_sync_period [Dn_config.dn_config]
How often the file is synced to disk
dump [Nn_blockmap.blockmap_t]
Get a dump of the internal state

E
enough_slaves [Nn_state.shared_state_t]
Whether there are enough slaves
event_system [Nn_state.shared_state_t]
The common event system
event_system [Nn_db.transaction]
The event system this transaction uses
event_system [Nn_db.ro_transaction]
The event system this transaction uses
exists_blockmap [Nn_state.shared_state_t]
Whether we have a blockmap for this identity
extract_key [Mapred_def.mapred_job]
Extracts the key from a record.

F
filename_count_e [Nn_state.inode_view_t]
The count is the number of names for an inode (same as length of list returned by filename_rev_get_e)
filename_get_e [Nn_state.inode_view_t]
Returns the inode of the member name of directory dir_inode
filename_link_e [Nn_state.inode_view_t]
adds this filename.
filename_list_e [Nn_state.inode_view_t]
Lists subnames of a potential directory name, together with inodes
filename_parent_dir_e [Nn_state.inode_view_t]
Returns the parent directory
filename_rename_e [Nn_state.inode_view_t]
filename_rev_get_dir_e [Nn_state.inode_view_t]
Works only for directories, and obtains an existence lock for the directory.
filename_rev_get_e [Nn_state.inode_view_t]
filename_rev_get does not acquire locks.
filename_unlink_e [Nn_state.inode_view_t]
deletes this filename from this directory
fill_blockmap [Nn_blockmap.blockmap_t]
fill_blockmap index data: Fills the blockmap at index with data, a string consisting of 0 and 1.
finish_transaction_serialized [Nn_state.shared_state_t]
finish_transaction_serialized f: Run f esys in a serialized way.
flush [Mapred_io.record_writer]
Flushes the records from the buffer to disk
forget_identity [Nn_state.shared_state_t]
Forget everything about this datastore identity
free [Nn_blockmap.blockmap_t]
free l owner: deallocates these blocks.
from_fd_e [Mapred_io.record_writer]
Outputs the records coming from this file.

G
get_blockmap [Nn_state.shared_state_t]
Get the blockmap for this identity, or Not_found
get_changes [Nn_blockmap.blockmap_t]
get_changes owner: Returns the changes made by owner.
get_tid_e [Plasma_inodecache.lazy_transaction]

H
has_free_blocks [Nn_blockmap.blockmap_t]
whether there are still free blocks

I
id [Nn_blockmap.blockmap_t]
the datastore db table id
identity [Nn_blockmap.blockmap_t]
This is the blockmap for this datastore identity
idle [Pfs_db.async_connection]
whether nothing is being executed
inactivate [Nn_blockmap.blockmap_t]
Sets active to false
inode_alloc_e [Nn_state.inode_view_t]
Inserts a new inode and returns the ID.
inode_dealloc_e [Nn_state.inode_view_t]
deletes the inode.
inode_get_blocks_e [Nn_state.inode_view_t]
returns blocks.
inode_get_e [Nn_state.inode_view_t]
inode_get_new_id_serialized [Nn_state.shared_state_t]
Serializer for getting a new ID from the inode table
inode_get_with_lock_e [Nn_state.inode_view_t]
like inode_get_e but immediately acquires a write lock for the inode
inode_max_id [Nn_state.shared_state_t]
Maximum ID used in the inode table (None if use_id was never called)
inode_upd_blocks_e [Nn_state.inode_view_t]
First delete the blocks in the del range, then add the blocks from add.
inode_upd_e [Nn_state.inode_view_t]
Updates the inode.
inode_upd_time [Nn_state.inode_view_t]
This update of mtime/ctime is lock-free
inode_use_id [Nn_state.shared_state_t]
Record that this ID is now also used in a transaction
input_dir [Mapred_def.mapred_job_config]
This plasma directory contains the input files
input_record [Mapred_io.record_reader]
Reads one record, and advances the cursor.

J
job_id [Mapred_def.mapred_job_config]
An automatically generated name.

K
keep [Nn_blockmap.blockmap_t]
keep l owner: marks these blocks as "to keep" even if a competing transaction frees them.

L
local_directory [Mapred_taskfiles.taskfile_manager]
The directory path on the task nodes for files that are needed by tasks
lock_filename [Nn_state.shared_state_t]
lock_filename (dir_inode,name) lock_type owner: Locks the name member of directory dir_inode.
lock_inode [Nn_state.shared_state_t]
Returns Some unlock if successful where calling unlock removes the lock, or reverts to the previous state (1-step undo)
log_dir [Mapred_def.mapred_job_config]
This plasma directory is used for log files.
log_directory [Mapred_taskfiles.taskfile_manager]
The directory path on the task nodes where the tasks can put files, and these files are finally copied to PlasmaFS

M
map [Mapred_def.mapred_job]
The mapper reads records, maps them, and writes them into a second file.
map_tasks [Mapred_def.mapred_job_config]
The number of map tasks that should not be exceeded.
master [Nn_state.shared_state_t]
The master in host:port syntax
master_enabled [Nn_state.shared_state_t]
Whether the master (i.e.
merge_limit [Mapred_def.mapred_job_config]
How many files are merged at most by a shuffle task
mr_buffer_size [Mapred_io.record_config]
mr_buffer_size [Mapred_config.mapred_config]
The normal size of I/O buffers.
mr_buffer_size_tight [Mapred_io.record_config]
mr_buffer_size_tight [Mapred_config.mapred_config]
The size of I/O buffers when RAM is tight.
mr_shm_high [Mapred_config.mapred_config]
High watermark for shared memory.
mr_shm_low [Mapred_config.mapred_config]
Low watermark for shared memory.
mr_sort_size [Mapred_config.mapred_config]
The size of the buffers for sorting.
mr_task_load_limit [Mapred_config.mapred_config]
Load limit per task server (in number of tasks).
mr_task_nodes [Mapred_config.mapred_config]
Task nodes (only hostname)
mr_task_port [Mapred_config.mapred_config]
The port number
mr_task_tmpdir [Mapred_config.mapred_config]
A directory where to put executables, logs, etc.

N
n_free [Nn_blockmap.blockmap_t]
Number of used, transitional, free blocks
n_trans [Nn_blockmap.blockmap_t]
n_used [Nn_blockmap.blockmap_t]
name [Mapred_def.mapred_job_config]
A name for identifying this job, e.g.
new_id [Plasma_shm.shm_manager]
allocates a new ID for an shm object
new_rev_ident [Nn_state.shared_state_t]
Creates a new revision identifier
next [Pfs_db.async_connection]
If there is a delayed continuation execute it now
nn_clustername [Mapred_config.mapred_config]
The clustername
nn_clustername [Nn_config.nn_node_config]
The clustername
nn_elect_timeout [Nn_config.nn_node_config]
For how many seconds to wait for neighbor namenodes
nn_hostname [Nn_config.nn_node_config]
The hostname of this node.
nn_inodecaches [Nn_config.nn_node_config]
Which inodecaches are announced to the clients
nn_node_alive_min [Nn_config.nn_node_config]
How many name nodes need to be alive to continue with operations
nn_node_alive_min_startup [Nn_config.nn_node_config]
How many name nodes need to be alive at cluster startup
nn_node_timeout [Nn_config.nn_node_config]
The timeout for connects and RPC calls to name nodes
nn_nodes [Mapred_config.mapred_config]
The name nodes in "host:port" syntax
nn_nodes [Nn_config.nn_node_config]
The name nodes in "host:port" syntax
nn_nodes [Plasma_client_config.client_config]
The name nodes in "host:port" syntax
nn_rank [Nn_config.nn_node_config]
The rank of this node compared to other nodes.
nn_replication [Nn_config.nn_node_config]
Default replication factor
node_config [Nn_state.shared_state_t]
the configuration

O
open_cluster [Mapred_def.mapred_env]
Opens the cluster filesystem
output_dir [Mapred_def.mapred_job_config]
This plasma directory will get output files.
output_record [Mapred_io.record_writer]
Outputs these records.

P
partition [Mapred_def.task_info]
The partition.
partition_of_key [Mapred_def.mapred_job]
Determines the partition of a key.
partitions [Mapred_def.mapred_job_config]
The number of partitions = number of reduce tasks
peek_record [Mapred_io.record_reader]
Reads the record, but does not advance the cursor, i.e.
plasma_cluster [Mapred_io.record_writer]
The PlasmaFS cluster
plasma_cluster [Mapred_io.record_reader]
The PlasmaFS cluster
pos_in [Mapred_io.record_reader]
The ordinal number of the record that will be read next.
pos_out [Mapred_io.record_writer]
The ordinal number of the record that will be written next.
post_commit [Nn_state.view_t]
Needs to be called after a commit.
post_job_finish [Mapred_def.mapred_job]
This is run by the job process after the last task is finished
pre_commit [Nn_state.view_t]
Prepares the commit, and returns the journal to commit
pre_job_start [Mapred_def.mapred_job]
This is run by the job process before the first task is started
prepare [Nn_db.transaction]
Prepares the transaction for commit

R
reduce [Mapred_def.mapred_job]
The reducer reads all the records of one partition, and puts them into an output file.
release_e [Plasma_inodecache.lazy_transaction]
req_id [Mapred_def.task_info]
The request ID.
reserve [Nn_blockmap.blockmap_t]
reserve n owner: finds n free blocks and returns their indices.
return [Nn_db.transaction]
return the transaction to the pool or closes it
return [Nn_db.ro_transaction]
return the transaction to the pool
ro_connection [Nn_db.transaction]
The same connection coerced to read-only
rollback [Nn_state.view_t]
Rolls this view back
rollback [Nn_blockmap.blockmap_t]
rollback owner: unreserves owner's blocks.
rollback [Nn_db.transaction]
Rolls the transaction back, and returns

S
set_rev_ident [Nn_state.shared_state_t]
Sets the identitifer returned by cur_rev_ident
shm_condition [Plasma_shm.shm_manager]
Estimate where we are
size [Nn_blockmap.blockmap_t]
Size of the blockmap in blocks
slaves [Nn_state.shared_state_t]
Connections to the name slaves as triples (name,client,enabled).
split_limit [Mapred_def.mapred_job_config]
How many files are created at most by a shuffle task

T
task_delete_log_directory [Mapred_taskfiles.taskfile_manager]
Deletes the entire log directory.
task_files [Mapred_def.mapred_job_config]
These files are copied at job start to the "local" directory for this job on all task nodes.
task_files_log_directory [Mapred_taskfiles.taskfile_manager]
Returns the files that are still in the log directory
task_prefix [Mapred_def.task_info]
This filename prefix is used for log files
task_save_log_directory [Mapred_taskfiles.taskfile_manager]
Copies these files of the log directory into PlasmaFS.
to_fd_e [Mapred_io.record_reader]
to_fd_e esys fd: The remaining records are written to fd.

U
unlock_filenames [Nn_state.shared_state_t]
unlock all filenames of this owner
unlock_inodes [Nn_state.shared_state_t]
unlock all inodes of this owner

W
work_dir [Mapred_def.mapred_job_config]
This plasma directory is used for temporary files.

This web site is published by Informatikbüro Gerd Stolpmann
Powered by Caml