Plasma GitLab Archive
Projects Blog Knowledge

Module Nn_config


module Nn_config: sig .. end
Configuration of name nodes

class type nn_node_config = object .. end
val extract_node_config : Netplex_types.config_file -> nn_node_config
Config section must look like (see Nn_config.nn_node_config for explanations):

         netplex {
           ...
           namenodes {
             clustername = "cluster1";  (* required *)
             node { addr = "hostname:port" };
             node { addr = "hostname:port" };
             ...
             hostname = "hostname";     (* optional *)
             timeout = 60.0;            (* optional *)
             alive_min = 1;             (* optional *)
             alive_min_startup = 1;     (* optional *)
             elect_timeout 300.0;       (* optional *)
             rank = "0";                (* mandatory *)
             replication = 3;           (* optional *)
             inodecache { port = <port> };
           }
           datanodes {
             node { addr = "hostname:port" }
             node { addr = "hostname:port" }
             ...
             timeout = 60.0;            (* optional *)
             blocksize = 65536;         (* mandatory *)
           }
           ...
         }
      

The parameter rank can also be given by running a script that outputs the rank to stdout. This script must be given as parameter rank_script.

Instead of describing the nodes directly in the config file using node subsections, one can also point to external file:

 
         node_list = "filename";
         port = <default_port>;
      

These two parameters can replace both the node subsections for namenodes and for datanodes. The external file enumerates the hosts, one hostname a line (with shell-style comments).

Special values:

  • alive_min and alive_min_startup can also be set to "all" (in double quotes). This means all configured name nodes must be available
  • replication can also be set to "max" (in double quotes). This means that the replication is set to the number of data nodes.

val bmaprowsize : int
block map row size - how many bits are in one row of the block map. Changing this value also affects the database.
This web site is published by Informatikbüro Gerd Stolpmann
Powered by Caml