A | |
| abort [Mapred_io.record_writer] |
Drops resources - intended to be used for error cleanup
|
| abort [Mapred_io.record_reader] |
Drops resources - intended to be used for error cleanup
|
| active [Nn_blockmap.blockmap_t] |
whether this object refers to an existing row in the
datastore
table.
|
| allocinfo_get_e [Nn_state.inode_view_t] |
Gets the allocinfo list for this inode
|
B | |
| bigblock_size [Mapred_def.mapred_job_config] |
Map/reduce processes files in units of bigblocks.
|
| bigblock_size [Mapred_io.record_config] |
Same as
Mapred_def.mapred_job.bigblock_size.
|
| blockmap_alloc_e [Nn_state.bm_view_t] |
Reserve blocks and allocate at commit time.
|
| blockmap_allocsize [Nn_state.bm_view_t] |
Returns the allocation size
|
| blockmap_dealloc_e [Nn_state.bm_view_t] |
Mark blocks and deallocate at commit time.
|
| blockmap_get_e [Nn_state.bm_view_t] |
Get the contents of the
blockalloc table for this datastore
identity, wrapped into a blockmap_t object
|
| blockmap_get_e [Nn_state.shared_state_t] |
If the blockmap exists, return it.
|
| blockmap_keep [Nn_state.bm_view_t] |
Marks these blocks as "to keep" until commit/rollback.
|
| blockmap_load_e [Nn_state.bm_view_t] |
Ensures that all known blockmaps are loaded
|
| blockmap_prioritizer [Nn_state.shared_state_t] |
Used to prioritize commits before other accesses to the blockmap.
|
| blockmaps [Nn_state.shared_state_t] |
All blockmaps
|
| book_allocation [Plasma_shm.shm_manager] |
Books this ID with this (total) size, and pushes the new entry
|
| bpos_out [Mapred_io.record_writer] |
The byte position of the record that will be written next
|
C | |
| check_config [Mapred_def.mapred_job] |
Check the config.
|
| close_in [Mapred_io.record_reader] |
Releases resources (e.g.
|
| close_out [Mapred_io.record_writer] |
Releases resources (e.g.
|
| clustername [Plasma_client_config.client_config] |
The clustername
|
| commit [Nn_blockmap.blockmap_t] | commit owner: commits owner's reservations, and makes them
permanent.
|
| commit [Nn_db.transaction] |
Commits the transaction (prepared or not), and returns
|
| condition [Plasma_shm.shm_manager] |
Estimate where we are
|
| config [Mapred_def.mapred_env] |
The map/reduce server configuration
|
| config_file [Mapred_def.mapred_env] |
The config file containing
config
|
| connection [Nn_db.transaction] |
The connection.
|
| connection [Nn_db.ro_transaction] |
The connection.
|
| continue_with [Pfs_db.async_connection] | continue_with f: Set f as the continuation.
|
| copy_to_local_directory [Mapred_taskfiles.taskfile_manager] |
Copies the mentioned file to the task nodes, and places them into
the
local_directory.
|
| create_blockmap [Nn_state.shared_state_t] | create_blockmap m: Creates a map of free/used/reserved
blocks.
|
| cur_rev_ident [Nn_state.shared_state_t] |
Returns current revision identifier
|
| custom [Mapred_def.mapred_job_config] |
Get a custom parameter or raise
Not_found
|
| custom_params [Mapred_def.mapred_job] |
The list of allowed custom parameters
|
D | |
| datastore_del_e [Nn_state.ds_view_t] |
Delete this row in the
datastore table
|
| datastore_list_e [Nn_state.ds_view_t] |
Get the contents of the
datastore table
|
| datastore_max_id [Nn_state.shared_state_t] |
Maximum ID used in the datastore table (None if use_id was never called)
|
| datastore_new_id_e [Nn_state.ds_view_t] |
The secret random number used for securing transactional writes
|
| datastore_upd_e [Nn_state.ds_view_t] |
Change the row in
datastore identified by id so that the
fields identity, size, and enabled are updated.
|
| datastore_use_id [Nn_state.shared_state_t] |
Record that this ID is now also used in a transaction
|
| db_dbname [Pfs_db.db_config] | |
| db_host [Pfs_db.db_config] | |
| db_hostaddr [Pfs_db.db_config] | |
| db_password [Pfs_db.db_config] | |
| db_port [Pfs_db.db_config] | |
| db_ro_connections_max [Pfs_db.db_config] | |
| db_user [Pfs_db.db_config] | |
| delayed_allocation [Plasma_shm.shm_manager] |
A delayed allocation is not immediately pushed to the shared
book.
|
| delayed_inode_deletes [Nn_state.inode_view_t] |
Inodes to be deleted at the end of the transaction
|
| delete_local_directory [Mapred_taskfiles.taskfile_manager] |
Deletes the local directory on the task nodes.
|
| dn_blocksize [Nn_config.nn_node_config] |
Expected blocksize of datanodes
|
| dn_blocksize [Dn_config.dn_config] |
The blocksize.
|
| dn_blocksize [Dn_shm.dn_shm_config] |
The blocksize.
|
| dn_blocksize [Dn_store.dn_store_config] |
The blocksize.
|
| dn_clustername [Dn_config.dn_config] |
The clustername
|
| dn_directory [Dn_config.dn_config] |
The directory containing the store.
|
| dn_directory [Dn_store.dn_store_config] |
The directory containing the store.
|
| dn_io_processes [Dn_config.dn_config] |
The number of dn_io processes.
|
| dn_node_timeout [Nn_config.nn_node_config] |
The timeout for connects and RPC calls to data nodes
|
| dn_nodes [Nn_config.nn_node_config] |
The data nodes in "host:port" syntax
|
| dn_shm_queue_length [Dn_config.dn_config] |
The size of the shm queue, in blocks.
|
| dn_shm_queue_length [Dn_shm.dn_shm_config] |
The size of the shm queue, in blocks.
|
| dn_sync_period [Dn_config.dn_config] |
How often the file is synced to disk
|
| dump [Nn_blockmap.blockmap_t] |
Get a dump of the internal state
|
| dump [Plasma_shm.shm_manager] |
Current sums for `Shm and `Buf as debug string
|
E | |
| enough_slaves [Nn_state.shared_state_t] |
Whether there are enough slaves
|
| event_system [Nn_state.shared_state_t] |
The common event system
|
| event_system [Nn_db.transaction] |
The event system this transaction uses
|
| event_system [Nn_db.ro_transaction] |
The event system this transaction uses
|
| exists_blockmap [Nn_state.shared_state_t] |
Whether we have a blockmap for this identity
|
| extract_key [Mapred_def.mapred_job] |
Extracts the key from a record.
|
F | |
| filename_count_e [Nn_state.inode_view_t] |
The count is the number of names for an inode (same as length of
list returned by
filename_rev_get_e)
|
| filename_get_e [Nn_state.inode_view_t] |
Returns the inode of the member
name of directory dir_inode
|
| filename_link_e [Nn_state.inode_view_t] |
adds this filename.
|
| filename_list_e [Nn_state.inode_view_t] |
Lists subnames of a potential directory name, together with inodes
|
| filename_parent_dir_e [Nn_state.inode_view_t] |
Returns the parent directory
|
| filename_rename_e [Nn_state.inode_view_t] | |
| filename_rev_get_dir_e [Nn_state.inode_view_t] |
Works only for directories, and obtains an existence lock for the
directory.
|
| filename_rev_get_e [Nn_state.inode_view_t] |
filename_rev_get does not acquire locks.
|
| filename_unlink_e [Nn_state.inode_view_t] |
deletes this filename from this directory
|
| fill_blockmap [Nn_blockmap.blockmap_t] | fill_blockmap index data: Fills the blockmap at index
with data, a string consisting of 0 and 1.
|
| finish_transaction_serialized [Nn_state.shared_state_t] | finish_transaction_serialized f: Run f esys in a serialized
way.
|
| flush [Mapred_io.record_writer] |
Flushes the records from the buffer to disk
|
| forget_identity [Nn_state.shared_state_t] |
Forget everything about this datastore identity
|
| free [Nn_blockmap.blockmap_t] | free l owner: deallocates these blocks.
|
| from_fd_e [Mapred_io.record_writer] |
Outputs the records coming from this file.
|
G | |
| get_blockmap [Nn_state.shared_state_t] |
Get the blockmap for this identity, or
Not_found
|
| get_changes [Nn_blockmap.blockmap_t] | get_changes owner: Returns the changes made by owner.
|
| get_tid_e [Plasma_inodecache.lazy_transaction] | |
H | |
| has_free_blocks [Nn_blockmap.blockmap_t] |
whether there are still free blocks
|
I | |
| id [Nn_blockmap.blockmap_t] |
the datastore db table id
|
| identity [Nn_blockmap.blockmap_t] |
This is the blockmap for this datastore identity
|
| idle [Pfs_db.async_connection] |
whether nothing is being executed
|
| inactivate [Nn_blockmap.blockmap_t] |
Sets
active to false
|
| inode_alloc_e [Nn_state.inode_view_t] |
Inserts a new inode and returns the ID.
|
| inode_dealloc_e [Nn_state.inode_view_t] |
deletes the inode.
|
| inode_get_blocks_e [Nn_state.inode_view_t] |
returns blocks.
|
| inode_get_e [Nn_state.inode_view_t] | |
| inode_get_new_id_serialized [Nn_state.shared_state_t] |
Serializer for getting a new ID from the inode table
|
| inode_get_with_lock_e [Nn_state.inode_view_t] |
like
inode_get_e but immediately acquires a write lock for the
inode
|
| inode_max_id [Nn_state.shared_state_t] |
Maximum ID used in the inode table (None if use_id was never called)
|
| inode_upd_blocks_e [Nn_state.inode_view_t] |
First delete the blocks in the
del range, then add
the blocks from add.
|
| inode_upd_e [Nn_state.inode_view_t] |
Updates the inode.
|
| inode_upd_time [Nn_state.inode_view_t] |
This update of mtime/ctime is lock-free
|
| inode_use_id [Nn_state.shared_state_t] |
Record that this ID is now also used in a transaction
|
| input_dir [Mapred_def.mapred_job_config] |
This plasma directory contains the input files
|
| input_record [Mapred_io.record_reader] |
Reads one record, and advances the cursor.
|
| input_records [Mapred_io.record_reader] |
Reads some records and appends them to the passed queue.
|
J | |
| job_id [Mapred_def.mapred_job_config] |
An automatically generated name.
|
K | |
| keep [Nn_blockmap.blockmap_t] | keep l owner: marks these blocks as "to keep" even if a
competing transaction frees them.
|
L | |
| local_directory [Mapred_taskfiles.taskfile_manager] |
The directory path on the task nodes for files
that are needed by tasks
|
| lock_filename [Nn_state.shared_state_t] | lock_filename (dir_inode,name) lock_type owner: Locks the name
member of directory dir_inode.
|
| lock_inode [Nn_state.shared_state_t] |
Returns
Some unlock if successful where calling unlock removes
the lock, or reverts to the previous state (1-step undo)
|
| log_dir [Mapred_def.mapred_job_config] |
This plasma directory is used for log files.
|
| log_directory [Mapred_taskfiles.taskfile_manager] |
The directory path on the task nodes where the tasks can put
files, and these files are finally copied to PlasmaFS
|
M | |
| map [Mapred_def.mapred_job] |
The mapper reads records, maps them, and writes them into a
second file.
|
| map_tasks [Mapred_def.mapred_job_config] |
The number of map tasks that should not be exceeded.
|
| master [Nn_state.shared_state_t] |
The master in host:port syntax
|
| master_enabled [Nn_state.shared_state_t] |
Whether the master (i.e.
|
| merge_limit [Mapred_def.mapred_job_config] |
How many files are merged at most by a shuffle task
|
| mr_buf_high [Mapred_config.mapred_config] |
High watermark for buffer memory.
|
| mr_buf_high_factor [Mapred_config.mapred_config] |
Alternate way for setting the high watermark as fraction of
available physical RAM.
|
| mr_buf_low [Mapred_config.mapred_config] |
Low watermark for buffer memory.
|
| mr_buf_low_factor [Mapred_config.mapred_config] |
Alternate way for setting the low watermark as fraction of
available physical RAM.
|
| mr_buf_max [Mapred_config.mapred_config] |
Maximum for buffer memory.
|
| mr_buf_max_factor [Mapred_config.mapred_config] |
Alternate way for setting the maximum as fraction of
available physical RAM.
|
| mr_buffer_size [Mapred_io.record_config] | |
| mr_buffer_size [Mapred_config.mapred_config] |
The normal size of I/O buffers.
|
| mr_buffer_size_tight [Mapred_io.record_config] | |
| mr_buffer_size_tight [Mapred_config.mapred_config] |
The size of I/O buffers when RAM is tight.
|
| mr_shm_high [Mapred_config.mapred_config] |
High watermark for shared memory.
|
| mr_shm_high_factor [Mapred_config.mapred_config] |
Alternate way for setting the high watermark as fraction of
available shared memory.
|
| mr_shm_low [Mapred_config.mapred_config] |
Low watermark for shared memory.
|
| mr_shm_low_factor [Mapred_config.mapred_config] |
Alternate way for setting the low watermark as fraction of
available shared memory.
|
| mr_shm_max [Mapred_config.mapred_config] |
Maximum for shared memory.
|
| mr_shm_max_factor [Mapred_config.mapred_config] |
Alternate way for setting the maximum as fraction of
available shared memory.
|
| mr_sort_size [Mapred_config.mapred_config] |
The size of the buffers for sorting.
|
| mr_task_load_limit [Mapred_config.mapred_config] |
Load limit per task server (in number of tasks).
|
| mr_task_nodes [Mapred_config.mapred_config] |
Task nodes (only hostname)
|
| mr_task_port [Mapred_config.mapred_config] |
The port number
|
| mr_task_tmpdir [Mapred_config.mapred_config] |
A directory where to put executables, logs, etc.
|
N | |
| n_free [Nn_blockmap.blockmap_t] |
Number of used, transitional, free blocks
|
| n_trans [Nn_blockmap.blockmap_t] | |
| n_used [Nn_blockmap.blockmap_t] | |
| name [Mapred_def.mapred_job_config] |
A name for identifying this job, e.g.
|
| new_id [Plasma_shm.shm_manager] |
allocates a new ID for an shm or buf object
|
| new_rev_ident [Nn_state.shared_state_t] |
Creates a new revision identifier
|
| next [Pfs_db.async_connection] |
If there is a delayed continuation execute it now
|
| nn_clustername [Mapred_config.mapred_config] |
The clustername
|
| nn_clustername [Nn_config.nn_node_config] |
The clustername
|
| nn_elect_timeout [Nn_config.nn_node_config] |
For how many seconds to wait for neighbor namenodes
|
| nn_hostname [Nn_config.nn_node_config] |
The hostname of this node.
|
| nn_inodecaches [Nn_config.nn_node_config] |
Which inodecaches are announced to the clients
|
| nn_node_alive_min [Nn_config.nn_node_config] |
How many name nodes need to be alive to continue with operations
|
| nn_node_alive_min_startup [Nn_config.nn_node_config] |
How many name nodes need to be alive at cluster startup
|
| nn_node_timeout [Nn_config.nn_node_config] |
The timeout for connects and RPC calls to name nodes
|
| nn_nodes [Mapred_config.mapred_config] |
The name nodes in "host:port" syntax
|
| nn_nodes [Nn_config.nn_node_config] |
The name nodes in "host:port" syntax
|
| nn_nodes [Plasma_client_config.client_config] |
The name nodes in "host:port" syntax
|
| nn_rank [Nn_config.nn_node_config] |
The rank of this node compared to other nodes.
|
| nn_replication [Nn_config.nn_node_config] |
Default replication factor
|
| node_config [Nn_state.shared_state_t] |
the configuration
|
O | |
| open_cluster [Mapred_def.mapred_env] |
Opens the cluster filesystem
|
| output_dir [Mapred_def.mapred_job_config] |
This plasma directory will get output files.
|
| output_record [Mapred_io.record_writer] |
Outputs these records.
|
P | |
| partition [Mapred_def.task_info] |
The partition.
|
| partition_of_key [Mapred_def.mapred_job] |
Determines the partition of a key.
|
| partitions [Mapred_def.mapred_job_config] |
The number of partitions = number of reduce tasks
|
| peek_record [Mapred_io.record_reader] |
Reads the record, but does not advance the cursor, i.e.
|
| plasma_cluster [Mapred_io.record_writer] |
The PlasmaFS cluster
|
| plasma_cluster [Mapred_io.record_reader] |
The PlasmaFS cluster
|
| pos_in [Mapred_io.record_reader] |
The ordinal number of the record that will be read next.
|
| pos_out [Mapred_io.record_writer] |
The ordinal number of the record that will be written next.
|
| post_commit [Nn_state.view_t] |
Needs to be called after a commit.
|
| post_job_finish [Mapred_def.mapred_job] |
This is run by the job process after the last task is finished
|
| pre_commit [Nn_state.view_t] |
Prepares the commit, and returns the journal to commit
|
| pre_job_start [Mapred_def.mapred_job] |
This is run by the job process before the first task is started
|
| prepare [Nn_db.transaction] |
Prepares the transaction for commit
|
| push [Plasma_shm.shm_manager] |
Push allocations to the shared book.
|
R | |
| reduce [Mapred_def.mapred_job] |
The reducer reads all the records of one partition, and puts them
into an output file.
|
| release_e [Plasma_inodecache.lazy_transaction] | |
| req_id [Mapred_def.task_info] |
The request ID.
|
| reserve [Nn_blockmap.blockmap_t] | reserve n owner ri: finds n free blocks and returns their indices.
|
| return [Nn_db.transaction] |
return the transaction to the pool or closes it
|
| return [Nn_db.ro_transaction] |
return the transaction to the pool
|
| ro_connection [Nn_db.transaction] |
The same connection coerced to read-only
|
| rollback [Nn_state.view_t] |
Rolls this view back
|
| rollback [Nn_blockmap.blockmap_t] | rollback owner: unreserves owner's blocks.
|
| rollback [Nn_db.transaction] |
Rolls the transaction back, and returns
|
S | |
| send [Pfs_db.async_connection] |
Unless
noprepare is set, prepare the query first, and execute it
then.
|
| set_rev_ident [Nn_state.shared_state_t] |
Sets the identitifer returned by
cur_rev_ident
|
| size [Nn_blockmap.blockmap_t] |
Size of the blockmap in blocks
|
| slaves [Nn_state.shared_state_t] |
Connections to the name slaves as triples (name,client,enabled).
|
| split_limit [Mapred_def.mapred_job_config] |
How many files are created by a shuffle task.
|
T | |
| task_delete_log_directory [Mapred_taskfiles.taskfile_manager] |
Deletes the entire log directory.
|
| task_files [Mapred_def.mapred_job_config] |
These files are copied at job start to the "local" directory
for this job on all task nodes.
|
| task_files_log_directory [Mapred_taskfiles.taskfile_manager] |
Returns the files that are still in the log directory
|
| task_prefix [Mapred_def.task_info] |
This filename prefix is used for log files
|
| task_save_log_directory [Mapred_taskfiles.taskfile_manager] |
Copies these files of the log directory into PlasmaFS.
|
| to_any_e [Mapred_io.record_reader] | to_any_e dest: like to_fd_e but the data is not written to
a file descriptor.
|
| to_fd_e [Mapred_io.record_reader] | to_fd_e esys fd: The records are written to fd.
|
U | |
| unlock_filenames [Nn_state.shared_state_t] |
unlock all filenames of this owner
|
| unlock_inodes [Nn_state.shared_state_t] |
unlock all inodes of this owner
|
W | |
| wm [Plasma_shm.shm_manager] |
Effective watermarks
|
| work_dir [Mapred_def.mapred_job_config] |
This plasma directory is used for temporary files.
|
