Plasma GitLab Archive
Projects Blog Knowledge

Plasma Documentation

Plasma release: 0.6 "Rechenknecht". This is a beta release, intended for broader testing. This release contains: General documents:

Transcripts of sample sessions:

Web resources:

PlasmaFS Documentation

PlasmaFS is the distributed transactional filesystem. It is implemented in user space, and can be accessed via RPC calls, or via NFS.

PlasmaFS Commands

PlasmaFS Client Interfaces

Client applications can only link with plasmasupport and plasmaclient.

plasmaclient: RPC client


Plasma_client
Client access to the Plasma Filesystem
Plasma_client_config
Configuration of clients
Plasma_shm
Management of large buffers incl.
Plasma_inodecache
The inodecache stores the metadata found in inodes, namely the inodeinfo struct, and subsets of the block list.
Plasma_filebuf
Buffer for read and write
Plasma_blocks
Helpers for analyzing blocklists
Plasma_netfs
Accessing PlasmaFS via the Netfs.stream_fs interface

These are the mappings of the XDR definition to Ocaml (as used in the client):


Plasma_rpcapi_aux
Plasma_rpcapi_clnt

plasmasupport: Support library


Plasma_rng
Random numbers
Plasma_ug
Users and groups
Plasma_platform
Some C functions

PlasmaFS RPC protocol definition

These are XDR definitions:

PlasmaFS Internal Server Interfaces

The following interfaces exist only within the server.

pfs_support: Support library


Pfs_db
Utility functions for database accesses
Pfs_auth
Authentication in server context
Pfs_condition
Condition variables
Pfs_pmanage
Manager for potentially persisting objects

These are the mappings of the XDR definition to Ocaml (as used in the server):


Pfs_rpcapi_aux
Pfs_rpcapi_clnt
Pfs_rpcapi_srv

pfs_datanode: Datanode server


Dn_config
The clustername
Dn_store
The directory containing the store.
Dn_shm
Shared memory for transmitting data from the main process to the I/O processes and vice versa
Dn_io
Returns this as factory for inclusion into Netplex_main.startup
Dn_manager
Returns this as factory for inclusion into Netplex_main.startup

pfs_namenode: Namenode server


Nn_config
Configuration of name nodes
Nn_db
Namenode access routines to database
Nn_blocklist
Support module for blocklists
Nn_blockmap
Already allocated blocks of this inode.
Nn_blockmap_box
Fast RPC to the blockmap server via camlboxes
Nn_alloc_types
Nn_alloc_trivial
Block allocation algorithm
Nn_alloc_balanced
Block allocation algorithm
Nn_datanode_ctrl
Manage tickets and push ticket secrets to the datanodes
Nn_datastores
Datastore types
Nn_datastore_news
Getting news about datastores and their availability
Nn_state
State manager: Provide the logical view of the state, and forward changes to the database and to the nameslaves (2-phase commit)
Nn_commit
Two-phase commit protocol
Nn_elect
Election of the coordinator
Nn_dentrycache
The dentrycache checks whether directory metadata is still up to date.
Nn_inodecache
The inodecache checks whether inode metadata is still up to date.
Nn_ug
Users and groups
Nn_monitor
Management of datastores, i.e.
Nn_push
Push updates to the slaves (client)
Nn_slave
The implementation of the namenode component for slave nodes
Nn_coord
The implementation of the namenode component for the coordinator node
Nn_authticket
Management of authentication tickets

pfs_nfs3: NFS server


Nfs3_aux
Nfs3_clnt
Nfs3_srv

Plasma MapReduce

Plasma MapReduce is the compute framework for Map/Reduce-style data transformations. It uses PlasmaFS for storing the data:

Plasma MapReduce Commands

Plasma MapReduce Interfaces

mr_framework: Framework


Mapred_io
Utility library for record-based I/O
Mapred_io_bgreader
Background reader for Mapred_io
Mapred_config
Configuration of mapred servers
Mapred_job_config
Extract job configuration, and marshalling
Mapred_def
Defining the map/reduce job
Mapred_split
Splits a line into key and value.
Mapred_sort
Mapred_sorters
Sorter classes
Mapred_taskfiles
Copy files to and from the task nodes
Mapred_tasks
Representation of tasks
Mapred_sched
Scheduler
Mapred_task_exec
Executes this task
Mapred_task_server
Factory for the processor "mapred_task_server"
Mapred_job_exec
Execute a scheduled job by submitting tasks to servers
Mapred_job
Run a map/red job
Mapred_streaming
Support for streaming
Mapred_main
Main program for the task server and job controller, for this single job only
Mapred_stats
Mapred_toolkit
Map/reduce toolkit
Mapred_fields
Access to record fields
Mapred_rfun
Registered functions (low-level)
Mapred_fs
Filesystem abstraction
Mapred_auth
Authentication in server context
Mapred_rpc_aux
Mapred_rpc_clnt
Mapred_rpc_srv

mr_platform: Platform-specific functions


Platform
Platform-specific functions

Plasma MapReduce RPC protocol definition

These are XDR definitions:

  • Pmr_task: The RPC definition of the task executor

PlasmaKV Documentation

Key/value databases are just files mapping keys to arbitrarily large values. The files live, of course, inside PlasmaFS. In transactional mode, the databases are ACID-compliant and allow high concurrency.

PlasmaKV commands

PlasmaKV Interfaces


Pkv_api
Plasma Key Value API

This web site is published by Informatikbüro Gerd Stolpmann
Powered by Caml