Package org.apache.storm.container
Class DefaultResourceIsolationManager
- java.lang.Object
-
- org.apache.storm.container.DefaultResourceIsolationManager
-
- All Implemented Interfaces:
ResourceIsolationInterface
- Direct Known Subclasses:
CgroupManager
public class DefaultResourceIsolationManager extends Object implements ResourceIsolationInterface
This is the default class to manage worker processes, including launching, killing, profiling and etc.
-
-
Constructor Summary
Constructors Constructor Description DefaultResourceIsolationManager()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description booleanareAllProcessesDead(String user, String workerId)Check if all the processes are dead.voidcleanup(String user, String workerId, int port)This function will be called when the worker needs to shutdown.voidforceKill(String user, String workerId)Kill the given worker forcefully.protected Set<Long>getAllPids(String workerId)Get all the pids that are a part of the container.longgetMemoryUsage(String user, String workerId, int port)Get the current memory usage of the a given worker.longgetSystemFreeMemoryMb()Get the amount of free memory in MB.booleanisResourceManaged()This class doesn't really manage resources.voidkill(String user, String workerId)Kill the given worker.voidlaunchWorkerProcess(String user, String topologyId, Map<String,Object> topoConf, int port, String workerId, List<String> command, Map<String,String> env, String logPrefix, ExitCodeCallback processExitCallback, File targetDir)After reserving resources for the worker (i.e.voidprepare(Map<String,Object> conf)Called when starting up.voidreserveResourcesForWorker(String workerId, Integer workerMemory, Integer workerCpu, String numaId)This function should be used prior to starting the worker to reserve resources for the worker.booleanrunProfilingCommand(String user, String workerId, List<String> command, Map<String,String> env, String logPrefix, File targetDir)Run profiling command.
-
-
-
Method Detail
-
prepare
public void prepare(Map<String,Object> conf) throws IOException
Description copied from interface:ResourceIsolationInterfaceCalled when starting up.- Specified by:
preparein interfaceResourceIsolationInterface- Parameters:
conf- the cluster config- Throws:
IOException- on any error.
-
reserveResourcesForWorker
public void reserveResourcesForWorker(String workerId, Integer workerMemory, Integer workerCpu, String numaId)
Description copied from interface:ResourceIsolationInterfaceThis function should be used prior to starting the worker to reserve resources for the worker.- Specified by:
reserveResourcesForWorkerin interfaceResourceIsolationInterface- Parameters:
workerId- worker id of the worker to startworkerMemory- the amount of memory for the worker or null if not enforcedworkerCpu- the amount of cpu for the worker or null if not enforcednumaId- NUMA zone if applicable the worker should be bound to
-
cleanup
public void cleanup(String user, String workerId, int port) throws IOException
Description copied from interface:ResourceIsolationInterfaceThis function will be called when the worker needs to shutdown. This function should include logic to clean up after a worker is shutdown.- Specified by:
cleanupin interfaceResourceIsolationInterface- Parameters:
user- the user of the workerworkerId- worker id to shutdown and clean up afterport- the port of the worker- Throws:
IOException
-
launchWorkerProcess
public void launchWorkerProcess(String user, String topologyId, Map<String,Object> topoConf, int port, String workerId, List<String> command, Map<String,String> env, String logPrefix, ExitCodeCallback processExitCallback, File targetDir) throws IOException
Description copied from interface:ResourceIsolationInterfaceAfter reserving resources for the worker (i.e. calling reserveResourcesForWorker), this function can be used to launch worker process.- Specified by:
launchWorkerProcessin interfaceResourceIsolationInterface- Parameters:
user- the user who runs the command astopologyId- the Id of the topologytopoConf- the topology configurationport- the port where the worker is onworkerId- the Id of the workercommand- the command to runenv- the environment to run the commandlogPrefix- the prefix to include in the logsprocessExitCallback- a callback for when the process exitstargetDir- the working directory to run the command in- Throws:
IOException- on I/O exception
-
getMemoryUsage
public long getMemoryUsage(String user, String workerId, int port) throws IOException
Description copied from interface:ResourceIsolationInterfaceGet the current memory usage of the a given worker.- Specified by:
getMemoryUsagein interfaceResourceIsolationInterface- Parameters:
user- the user that the worker is running asworkerId- the id of the workerport- the port of the worker- Returns:
- the amount of memory the worker is using in bytes or -1 if not supported
- Throws:
IOException- on I/O exception
-
getSystemFreeMemoryMb
public long getSystemFreeMemoryMb() throws IOExceptionDescription copied from interface:ResourceIsolationInterfaceGet the amount of free memory in MB. This might not be the entire box, it might be within a parent resource group.- Specified by:
getSystemFreeMemoryMbin interfaceResourceIsolationInterface- Returns:
- The amount of memory in MB that are free on the system.
- Throws:
IOException- on I/O exception
-
kill
public void kill(String user, String workerId) throws IOException
Description copied from interface:ResourceIsolationInterfaceKill the given worker.- Specified by:
killin interfaceResourceIsolationInterface- Parameters:
user- the user that the worker is running asworkerId- the id of the worker to kill- Throws:
IOException- on I/O exception
-
forceKill
public void forceKill(String user, String workerId) throws IOException
Description copied from interface:ResourceIsolationInterfaceKill the given worker forcefully.- Specified by:
forceKillin interfaceResourceIsolationInterface- Parameters:
user- the user that the worker is running asworkerId- the id of the worker to kill- Throws:
IOException- on I/O exception
-
getAllPids
protected Set<Long> getAllPids(String workerId) throws IOException
Get all the pids that are a part of the container.- Returns:
- all of the pids that are a part of this container
- Throws:
IOException
-
areAllProcessesDead
public boolean areAllProcessesDead(String user, String workerId) throws IOException
Description copied from interface:ResourceIsolationInterfaceCheck if all the processes are dead.- Specified by:
areAllProcessesDeadin interfaceResourceIsolationInterface- Parameters:
user- the user that the processes are running asworkerId- the id of the worker to kill- Returns:
- true if all the processed are dead; false otherwise
- Throws:
IOException- on I/O exception
-
runProfilingCommand
public boolean runProfilingCommand(String user, String workerId, List<String> command, Map<String,String> env, String logPrefix, File targetDir) throws IOException, InterruptedException
Description copied from interface:ResourceIsolationInterfaceRun profiling command.- Specified by:
runProfilingCommandin interfaceResourceIsolationInterface- Parameters:
user- the user that the worker is running asworkerId- the id of the workercommand- the command to runenv- the environment to run the commandlogPrefix- the prefix to include in the logstargetDir- the working directory to run the command in- Returns:
- true if succeeds; false otherwise
- Throws:
IOException- on I/O exceptionInterruptedException- if interrupted
-
isResourceManaged
public boolean isResourceManaged()
This class doesn't really manage resources.- Specified by:
isResourceManagedin interfaceResourceIsolationInterface- Returns:
- false
-
-