A | |
| abort [Mapred_io.record_writer] |
Drops resources - intended to be used for error cleanup
|
| abort [Mapred_io.record_reader] |
Drops resources - intended to be used for error cleanup
|
| active [Nn_blockmap.blockmap_t] |
whether this object refers to an existing row in the
datastore
table.
|
| arguments [Mapred_def.mapred_env] |
Job arguments
|
B | |
| blockmap_alloc_e [Nn_state.bm_view_t] |
Reserve blocks and allocate at commit time.
|
| blockmap_dealloc_e [Nn_state.bm_view_t] |
Mark blocks and deallocate at commit time.
|
| blockmap_get_e [Nn_state.bm_view_t] |
Get the contents of the
blockalloc table for this datastore
identity, wrapped into a blockmap_t object
|
| blockmap_get_e [Nn_state.shared_state_t] |
If the blockmap exists, return it.
|
| blockmap_keep [Nn_state.bm_view_t] |
Marks these blocks as "to keep" until commit/rollback.
|
| blockmap_load_e [Nn_state.bm_view_t] |
Ensures that all known blockmaps are loaded
|
| blockmap_prioritizer [Nn_state.shared_state_t] |
Used to prioritize commits before other accesses to the blockmap.
|
| blockmaps [Nn_state.shared_state_t] |
All blockmaps
|
| book_allocation [Plasma_shm.shm_manager] |
Books this ID with this (total) size
|
| bpos_out [Mapred_io.record_writer] |
The byte position of the record that will be written next
|
C | |
| check_config [Mapred_def.mapred_job] |
Check the config.
|
| close_in [Mapred_io.record_reader] |
Releases resources (e.g.
|
| close_out [Mapred_io.record_writer] |
Releases resources (e.g.
|
| clustername [Plasma_client_config.client_config] |
The clustername
|
| commit [Nn_blockmap.blockmap_t] | commit owner: commits owner's reservations, and makes them
permanent.
|
| commit [Nn_db.transaction] |
Commits the transaction (prepared or not), and returns
|
| config_file [Mapred_def.mapred_env] |
The config file
|
| connection [Nn_db.transaction] |
The connection.
|
| connection [Nn_db.ro_transaction] |
The connection.
|
| continue_with [Pfs_db.async_connection] | continue_with f: Set f as the continuation.
|
| create_blockmap [Nn_state.shared_state_t] | create_blockmap m: Creates a map of free/used/reserved
blocks.
|
| cur_rev_ident [Nn_state.shared_state_t] |
Returns current revision identifier
|
D | |
| datastore_del_e [Nn_state.ds_view_t] |
Delete this row in the
datastore table
|
| datastore_list_e [Nn_state.ds_view_t] |
Get the contents of the
datastore table
|
| datastore_max_id [Nn_state.shared_state_t] |
Maximum ID used in the datastore table (None if use_id was never called)
|
| datastore_new_id_e [Nn_state.ds_view_t] |
The secret random number used for securing transactional writes
|
| datastore_upd_e [Nn_state.ds_view_t] |
Change the row in
datastore identified by id so that the
fields identity, size, and enabled are updated.
|
| datastore_use_id [Nn_state.shared_state_t] |
Record that this ID is now also used in a transaction
|
| db_dbname [Pfs_db.db_config] | |
| db_host [Pfs_db.db_config] | |
| db_hostaddr [Pfs_db.db_config] | |
| db_password [Pfs_db.db_config] | |
| db_port [Pfs_db.db_config] | |
| db_ro_connections_max [Pfs_db.db_config] | |
| db_user [Pfs_db.db_config] | |
| delayed_inode_deletes [Nn_state.inode_view_t] |
Inodes to be deleted at the end of the transaction
|
| dn_blocksize [Nn_config.nn_node_config] |
Expected blocksize of datanodes
|
| dn_blocksize [Dn_config.dn_config] |
The blocksize.
|
| dn_blocksize [Dn_shm.dn_shm_config] |
The blocksize.
|
| dn_blocksize [Dn_store.dn_store_config] |
The blocksize.
|
| dn_clustername [Dn_config.dn_config] |
The clustername
|
| dn_directory [Dn_config.dn_config] |
The directory containing the store.
|
| dn_directory [Dn_store.dn_store_config] |
The directory containing the store.
|
| dn_io_processes [Dn_config.dn_config] |
The number of dn_io processes.
|
| dn_node_timeout [Nn_config.nn_node_config] |
The timeout for connects and RPC calls to data nodes
|
| dn_nodes [Nn_config.nn_node_config] |
The data nodes in "host:port" syntax
|
| dn_shm_queue_length [Dn_config.dn_config] |
The size of the shm queue, in blocks.
|
| dn_shm_queue_length [Dn_shm.dn_shm_config] |
The size of the shm queue, in blocks.
|
| dn_sync_period [Dn_config.dn_config] |
How often the file is synced to disk
|
| dump [Nn_blockmap.blockmap_t] |
Get a dump of the internal state
|
E | |
| enough_slaves [Nn_state.shared_state_t] |
Whether there are enough slaves
|
| event_system [Nn_state.shared_state_t] |
The common event system
|
| event_system [Nn_db.transaction] |
The event system this transaction uses
|
| event_system [Nn_db.ro_transaction] |
The event system this transaction uses
|
| exists_blockmap [Nn_state.shared_state_t] |
Whether we have a blockmap for this identity
|
| extract_key [Mapred_def.mapred_job] |
Extracts the key from a record
|
F | |
| filename_count_e [Nn_state.inode_view_t] |
The count is the number of names for an inode (same as length of
list returned by
filename_rev_get_e)
|
| filename_get_e [Nn_state.inode_view_t] | exists_lock: whether to create an `Exists lock
|
| filename_link_e [Nn_state.inode_view_t] |
adds this filename.
|
| filename_list_e [Nn_state.inode_view_t] |
Lists subnames of a potential directory name, together with inodes
|
| filename_rev_get_e [Nn_state.inode_view_t] | |
| filename_unlink_e [Nn_state.inode_view_t] |
deletes this filename
|
| fill_blockmap [Nn_blockmap.blockmap_t] | fill_blockmap index data: Fills the blockmap at index
with data, a string consisting of 0 and 1.
|
| finish_transaction_serialized [Nn_state.shared_state_t] | finish_transaction_serialized f: Run f esys in a serialized
way.
|
| flush [Mapred_io.record_writer] |
Flushes the records from the buffer to disk
|
| forget_identity [Nn_state.shared_state_t] |
Forget everything about this datastore identity
|
| free [Nn_blockmap.blockmap_t] | free l owner: deallocates these blocks.
|
G | |
| get_blockmap [Nn_state.shared_state_t] |
Get the blockmap for this identity, or
Not_found
|
| get_changes [Nn_blockmap.blockmap_t] | get_changes owner: Returns the changes made by owner.
|
H | |
| has_free_blocks [Nn_blockmap.blockmap_t] |
whether there are still free blocks
|
I | |
| id [Nn_blockmap.blockmap_t] |
the datastore db table id
|
| identity [Nn_blockmap.blockmap_t] |
This is the blockmap for this datastore identity
|
| idle [Pfs_db.async_connection] |
whether nothing is being executed
|
| inactivate [Nn_blockmap.blockmap_t] |
Sets
active to false
|
| inode_alloc_e [Nn_state.inode_view_t] |
Inserts a new inode and returns the ID.
|
| inode_dealloc_e [Nn_state.inode_view_t] |
deletes the inode.
|
| inode_get_blocks_e [Nn_state.inode_view_t] |
returns blocks.
|
| inode_get_e [Nn_state.inode_view_t] | |
| inode_get_new_id_serialized [Nn_state.shared_state_t] |
Serializer for getting a new ID from the inode table
|
| inode_max_id [Nn_state.shared_state_t] |
Maximum ID used in the inode table (None if use_id was never called)
|
| inode_upd_blocks_e [Nn_state.inode_view_t] |
First delete the blocks in the
del range, then add
the blocks from add.
|
| inode_upd_e [Nn_state.inode_view_t] |
Updates the inode.
|
| inode_upd_time [Nn_state.inode_view_t] |
This update of mtime/ctime is lock-free
|
| inode_use_id [Nn_state.shared_state_t] |
Record that this ID is now also used in a transaction
|
| input_dir [Mapred_def.mapred_job] |
This plasma directory contains the input files
|
| input_record [Mapred_io.record_reader] |
Reads one record, and advances the cursor.
|
K | |
| keep [Nn_blockmap.blockmap_t] | keep l owner: marks these blocks as "to keep" even if a
competing transaction frees them.
|
L | |
| lock_filename [Nn_state.shared_state_t] |
Lock types:
`Link: States that the file is created, and must not be
created by another transaction. A `Link lock is exlusive. , `Unlink: States that the file is removed. Several transactions
are allowed to remove the same file without conflict., `Exists: States that the file exists. Several transactions can
request this. (Used for ensuring that parent directories exist
until the end of the transaction.)
Certain lock type changes are allowed provided this is ok with the
protected operations (e.g.
|
| lock_inode [Nn_state.shared_state_t] |
Returns
Some unlock if successful where calling unlock removes
the lock, or reverts to the previous state (1-step undo)
|
M | |
| map [Mapred_def.mapred_job] |
The mapper reads records, maps them, and writes them into a
second file.
|
| map_tasks [Mapred_def.mapred_job] |
The number of map tasks that should not be exceeded.
|
| master [Nn_state.shared_state_t] |
The master in host:port syntax
|
| master_enabled [Nn_state.shared_state_t] |
Whether the master (i.e.
|
| merge_limit [Mapred_def.mapred_job] |
How many files are merged at most by a shuffle task
|
| mr_shm_high [Mapred_config.mapred_config] |
High watermark for shared memory
|
| mr_shm_low [Mapred_config.mapred_config] |
Low watermark for shared memory
|
| mr_task_load_limit [Mapred_config.mapred_config] |
Load limit per task server (in number of tasks)
|
| mr_task_nodes [Mapred_config.mapred_config] |
Task nodes (only hostname)
|
| mr_task_port [Mapred_config.mapred_config] |
The port number
|
| mr_task_tmpdir [Mapred_config.mapred_config] |
A directory where to put executables, logs, etc.
|
N | |
| n_free [Nn_blockmap.blockmap_t] |
Number of used, transitional, free blocks
|
| n_trans [Nn_blockmap.blockmap_t] | |
| n_used [Nn_blockmap.blockmap_t] | |
| name [Mapred_def.mapred_job] |
A name for identifying this job, e.g.
|
| new_id [Plasma_shm.shm_manager] |
allocates a new ID for an shm object
|
| new_rev_ident [Nn_state.shared_state_t] |
Creates a new revision identifier
|
| next [Pfs_db.async_connection] |
If there is a delayed continuation execute it now
|
| nn_clustername [Mapred_config.mapred_config] |
The clustername
|
| nn_clustername [Nn_config.nn_node_config] |
The clustername
|
| nn_elect_timeout [Nn_config.nn_node_config] |
For how many seconds to wait for neighbor namenodes
|
| nn_hostname [Nn_config.nn_node_config] |
The hostname of this node.
|
| nn_inodecaches [Nn_config.nn_node_config] |
Which inodecaches are announced to the clients
|
| nn_node_alive_min [Nn_config.nn_node_config] |
How many name nodes need to be alive to continue with operations
|
| nn_node_alive_min_startup [Nn_config.nn_node_config] |
How many name nodes need to be alive at cluster startup
|
| nn_node_timeout [Nn_config.nn_node_config] |
The timeout for connects and RPC calls to name nodes
|
| nn_nodes [Mapred_config.mapred_config] |
The name nodes in "host:port" syntax
|
| nn_nodes [Nn_config.nn_node_config] |
The name nodes in "host:port" syntax
|
| nn_nodes [Plasma_client_config.client_config] |
The name nodes in "host:port" syntax
|
| nn_rank [Nn_config.nn_node_config] |
The rank of this node compared to other nodes.
|
| nn_replication [Nn_config.nn_node_config] |
Default replication factor
|
| node_config [Nn_state.shared_state_t] |
the configuration
|
O | |
| open_cluster [Mapred_def.mapred_env] |
Opens the cluster filesystem
|
| output_dir [Mapred_def.mapred_job] |
This plasma directory will get output files.
|
| output_record [Mapred_io.record_writer] |
Outputs these records.
|
P | |
| partition_of_key [Mapred_def.mapred_job] |
Determines the partition of a key.
|
| partitions [Mapred_def.mapred_job] |
The number of partitions = number of reduce tasks
|
| peek_record [Mapred_io.record_reader] |
Reads the record, but does not advance the cursor, i.e.
|
| pos_in [Mapred_io.record_reader] |
The ordinal number of the record that will be read next.
|
| pos_out [Mapred_io.record_writer] |
The ordinal number of the record that will be written next.
|
| post_commit [Nn_state.view_t] |
Needs to be called after a commit.
|
| pre_commit [Nn_state.view_t] |
Prepares the commit, and returns the journal to commit
|
| prepare [Nn_db.transaction] |
Prepares the transaction for commit
|
R | |
| reduce [Mapred_def.mapred_job] |
The reducer reads all the records of one partition, and puts them
into an output file.
|
| reserve [Nn_blockmap.blockmap_t] | reserve n owner: finds n free blocks and returns their indices.
|
| return [Nn_db.transaction] |
return the transaction to the pool or closes it
|
| return [Nn_db.ro_transaction] |
return the transaction to the pool
|
| ro_connection [Nn_db.transaction] |
The same connection coerced to read-only
|
| rollback [Nn_state.view_t] |
Rolls this view back
|
| rollback [Nn_blockmap.blockmap_t] | rollback owner: unreserves owner's blocks.
|
| rollback [Nn_db.transaction] |
Rolls the transaction back, and returns
|
S | |
| set_rev_ident [Nn_state.shared_state_t] |
Sets the identitifer returned by
cur_rev_ident
|
| shm_condition [Plasma_shm.shm_manager] |
Estimate where we are
|
| size [Nn_blockmap.blockmap_t] |
Size of the blockmap in blocks
|
| slaves [Nn_state.shared_state_t] |
Connections to the name slaves as triples (name,client,enabled).
|
| sort_limit [Mapred_def.mapred_job] |
How big the data chunks are that can be sorted in-memory
|
| split_limit [Mapred_def.mapred_job] |
How many files are created at most by a shuffle task
|
U | |
| unlock_filenames [Nn_state.shared_state_t] |
unlock all filenames of this owner
|
| unlock_inodes [Nn_state.shared_state_t] |
unlock all inodes of this owner
|
W | |
| work_dir [Mapred_def.mapred_job] |
This plasma directory is used for temporary files.
|
