Plasma GitLab Archive
Projects Blog Knowledge

Module Mapred_job_exec


module Mapred_job_exec: sig .. end
Execute a scheduled job by submitting tasks to servers

type runtime_job_config 
val create_runtime_job_config : ?map_weight:float * float ->
?sort_weight:float * float ->
?shuffle_weight:float * float ->
?reduce_weight:float * float ->
?dump_plan_when_complete:bool ->
?shm_low:int64 ->
?shm_high:int64 ->
io_load:float ->
cpu_load:float ->
Mapred_job_config.m_job_config -> runtime_job_config
Especially how many tasks are started per task server. Each task counts with a certain weight, given as (io_weight,cpu_weight)- The sum of the I/O weights must not exceed io_load, and the sum of the CPU weights must not exceed cpu_load.
type running_job 
val start : Unixqueue.event_system ->
Mapred_sched.plan ->
Mapred_def.mapred_job ->
runtime_job_config -> int -> running_job
Starts the job which is running asynchronously with the event system
val kill : running_job -> unit
Kills the job (also async)
val cleanup : running_job -> unit
Deletes all temp files (sync)
type status = [ `Errors of string list | `Killed | `Running | `Successful ] 
val status : running_job -> status
This web site is published by Informatikbüro Gerd Stolpmann
Powered by Caml