Plasma_release: Release notes
Plasma_release_0_1: Old release notes for 0.1Plasma_release_0_2: Old release notes for 0.2Plasma_release_0_3: Old release notes for 0.3Plasma_release_0_4: Old release notes for 0.4Plasma_license: License
PlasmaFS Documentation
PlasmaFS is the distributed transactional filesystem. It is implemented in user space, and can be accessed via RPC calls, or via NFS.
Plasmafs_start: Feature set, theory of operationPlasmafs_deployment: Deploying PlasmaFSPlasmafs_authentication: AuthenticationPlasmafs_dn_admin: Datanode administrationPlasmafs_nn_admin: Namenode administrationPlasmafs_protocol: The PlasmaFS protocolPlasmafs_and_hdfs: Feature comparison of PlasmaFS and HDFS
Cmd_plasma: The plasma utilityCmd_plasmad: The plasmad daemon for datanodes and namenodesCmd_plasma_datanode_init: The plasma_datanode_init utility for
initializing datanodesCmd_plasma_admin: The plasma_admin utility for doing namenode
administrationCmd_nfs3d: The nfs3d daemon for the NFS bridgeClient applications can only link with plasmasupport and plasmaclient.
plasmaclient: RPC client
| Plasma_client |
Client access to the Plasma Filesystem
|
| Plasma_client_config |
Configuration of clients
|
| Plasma_shm |
Management of large buffers incl.
|
| Plasma_inodecache |
The inodecache stores the metadata found in inodes, namely the
inodeinfo struct, and subsets of the block list.
|
| Plasma_filebuf |
Buffer for
read and write
|
| Plasma_blocks |
Helpers for analyzing blocklists
|
| Plasma_netfs |
Accessing PlasmaFS via the
Netfs.stream_fs interface
|
These are the mappings of the XDR definition to Ocaml (as used in the client):
| Plasma_rpcapi_aux | |
| Plasma_rpcapi_clnt |
plasmasupport: Support library
| Plasma_rng |
Random numbers
|
| Plasma_ug |
Users and groups
|
| Plasma_platform |
Some C functions
|
PlasmaFS RPC protocol definition
These are XDR definitions:
Pfs_types: Types for all RPC definitionsPfs_nn_fsys: Filesystem accessPfs_nn_coord: Determining the coordinatorPfs_nn_dnadmin: Administration of the known datanodesPfs_nn_internal: Internal interfaces used between PlasmaFS RPC serversPfs_datanode: Datanode accessPfs_dn_internal: Internal interfaces used between PlasmaFS RPC serversPfs_nfs3: NFS version 3Pfs_server: All RPC definitions for server contextPfs_client: All RPC definitions for client contextThe following interfaces exist only within the server.
pfs_support: Support library
| Pfs_db |
Utility functions for database accesses
|
| Pfs_auth |
Authentication in server context
|
| Pfs_condition |
Condition variables
|
| Pfs_pmanage |
Manager for potentially persisting objects
|
These are the mappings of the XDR definition to Ocaml (as used in the server):
| Pfs_rpcapi_aux | |
| Pfs_rpcapi_clnt | |
| Pfs_rpcapi_srv |
pfs_datanode: Datanode server
| Dn_config |
The clustername
|
| Dn_store |
The directory containing the store.
|
| Dn_shm |
Shared memory for transmitting data from the main process to the
I/O processes and vice versa
|
| Dn_io |
Returns this as factory for inclusion into
Netplex_main.startup
|
| Dn_manager |
Returns this as factory for inclusion into
Netplex_main.startup
|
pfs_namenode: Namenode server
| Nn_config |
Configuration of name nodes
|
| Nn_db |
Namenode access routines to database
|
| Nn_blocklist |
Support module for blocklists
|
| Nn_blockmap |
Already allocated blocks of this inode.
|
| Nn_blockmap_box |
Fast RPC to the blockmap server via camlboxes
|
| Nn_alloc_types | |
| Nn_alloc_trivial |
Block allocation algorithm
|
| Nn_alloc_balanced |
Block allocation algorithm
|
| Nn_datanode_ctrl |
Manage tickets and push ticket secrets to the datanodes
|
| Nn_datastores |
Datastore types
|
| Nn_datastore_news |
Getting news about datastores and their availability
|
| Nn_state |
State manager: Provide the logical view of the state, and forward changes
to the database and to the nameslaves (2-phase commit)
|
| Nn_commit |
Two-phase commit protocol
|
| Nn_elect |
Election of the coordinator
|
| Nn_dentrycache |
The dentrycache checks whether directory metadata is still up to date.
|
| Nn_inodecache |
The inodecache checks whether inode metadata is still up to date.
|
| Nn_ug |
Users and groups
|
| Nn_monitor |
Management of datastores, i.e.
|
| Nn_push |
Push updates to the slaves (client)
|
| Nn_slave |
The implementation of the namenode component for slave nodes
|
| Nn_coord |
The implementation of the namenode component for the coordinator
node
|
| Nn_authticket |
Management of authentication tickets
|
pfs_nfs3: NFS server
| Nfs3_aux | |
| Nfs3_clnt | |
| Nfs3_srv |
Plasma MapReduce
Plasma MapReduce is the compute framework for Map/Reduce-style data transformations. It uses PlasmaFS for storing the data:
Plasmamr_start: What is Map/Reduce?Plasmamr_howto: How to run a Map/Reduce jobPlasmamr_task_types: The tasks a Map/Reduce job is composed ofPlasmamr_and_hadoop: Feature comparison of PlasmaMR and Hadoop
mr_framework: Framework
| Mapred_io |
Utility library for record-based I/O
|
| Mapred_config |
Configuration of mapred servers
|
| Mapred_job_config |
Extract job configuration, and marshalling
|
| Mapred_def |
Defining the map/reduce job
|
| Mapred_split |
Splits a line into key and value.
|
| Mapred_sort | |
| Mapred_taskfiles |
Copy files to and from the task nodes
|
| Mapred_tasks |
Representation of tasks
|
| Mapred_sched |
Scheduler
|
| Mapred_task_exec |
Executes this task
|
| Mapred_task_server |
Factory for the processor "mapred_task_server"
|
| Mapred_job_exec |
Execute a scheduled job by submitting tasks to servers
|
| Mapred_streaming |
Support for streaming
|
| Mapred_main |
Main program for the task server and job controller
|
| Mapred_rpc_aux | |
| Mapred_rpc_clnt | |
| Mapred_rpc_srv |
mr_platform: Platform-specific functions
| Platform |
Platform-specific functions
|
Plasma MapReduce RPC protocol definition
These are XDR definitions:
Pmr_task: The RPC definition of the task executorKey/value databases are just files mapping keys to arbitrarily large values. The files live, of course, inside PlasmaFS. In transactional mode, the databases are ACID-compliant and allow high concurrency.
Plasmakv_intro: Plasma Key/Value Databases
Cmd_plasma_kv: The plasma_kv utility
| Pkv_api |
Plasma Key Value API
|
