Plasma GitLab Archive
Projects Blog Knowledge

Plasmafs_and_hdfs



Feature comparison of PlasmaFS and HDFS

HDFS is another user-space filesystem that was originally developed for map/reduce.

Feature				PlasmaFS		HDFS
---------------------------------------------------------------------------
Supported blocksizes		any			any
				recommended 64K-1M	recommended >= 64M

Blocksize can be set for
  each file separately		no			yes

System can allocate blocks
  in contiguous regions		yes (all blocks are	no (each block is a
     				stored in a single	separate file)
				file)

Number of datanodes is
  limited by RAM in namenode	yes			yes

Number of files is limited
  by RAM in namenode		no			yes

Replication can be set for
  each file separately		yes			yes

Client communicates directly
  with datanodes		yes			yes

Block checksums			no			yes

Random read access to files	yes			yes

Random write access to files	yes			no
				(blocks can only be	(files immutable after
				replaced but not	creation)
				overwritten)

Directory hierarchy		yes			yes

Symbolic links			yes			no

POSIX file semantics		yes (few exceptions)	no

Authentication system		yes			no

Encrypted data communication	optional		no

Authorization system		yes			yes
							(based on fake
							authentication)

Several namenode operations
  can be bundled in an
  atomic transaction		yes			no

Namenode crashes can lead to
  data loss			no (2-phase commit)	yes
       	     	       		
Datanode crashes are handled
  automatically (fail-over)	yes			yes

Namenode crashes are handled
  automatically (fail-over)	not yet			no
  				(but planned)
				so far: auto-selection
				of live coordinator at
				startup time

Datanode configuration can
  be changed w/o restart
  (e.g. add node, del node)	yes			no

Namenodes profit from special
  hardware			yes (SSDs)		no

Filesystem can be mounted	yes (NFS bridge)	no

Rebalancing    	  		not yet			yes
				(but planned)

Communication to local
  datanode servers via shared
  memory			yes			no

Primary access method		SunRPC from any		ad-hoc protocol
	       			language    		(undocumented)

Clients available		Ocaml			Java
				Access from any
				language via NFS


This web site is published by Informatikbüro Gerd Stolpmann
Powered by Caml