public class ProportionalCapacityPreemptionPolicy extends Object implements SchedulingEditPolicy, CapacitySchedulerPreemptionContext
SchedulingEditPolicy that is designed to be
paired with the CapacityScheduler. At every invocation of editSchedule() it computes the ideal amount of resources assigned to each
queue (for each queue in the hierarchy), and determines whether preemption
is needed. Overcapacity is distributed among queues in a weighted fair manner,
where the weight is the amount of guaranteed capacity for the queue.
Based on this ideal assignment it determines whether preemption is required
and select a set of containers from each application that would be killed if
the corresponding amount of resources is not freed up by the application.
If not in observeOnly mode, it triggers preemption requests via a
ContainerPreemptEvent that the ResourceManager will ensure
to deliver to the application (or to execute).
If the deficit of resources is persistent over a long enough period of time
this policy will trigger forced termination of containers (again by generating
ContainerPreemptEvent).| Modifier and Type | Class and Description |
|---|---|
static class |
ProportionalCapacityPreemptionPolicy.IntraQueuePreemptionOrderPolicy
IntraQueuePreemptionOrder will be used to define various priority orders
which could be configured by admin.
|
| Constructor and Description |
|---|
ProportionalCapacityPreemptionPolicy() |
ProportionalCapacityPreemptionPolicy(RMContext context,
CapacityScheduler scheduler,
org.apache.hadoop.yarn.util.Clock clock) |
public ProportionalCapacityPreemptionPolicy()
public ProportionalCapacityPreemptionPolicy(RMContext context, CapacityScheduler scheduler, org.apache.hadoop.yarn.util.Clock clock)
public void init(org.apache.hadoop.conf.Configuration config,
RMContext context,
ResourceScheduler sched)
init in interface SchedulingEditPolicypublic org.apache.hadoop.yarn.util.resource.ResourceCalculator getResourceCalculator()
getResourceCalculator in interface CapacitySchedulerPreemptionContextpublic void editSchedule()
SchedulingEditPolicyeditSchedule in interface SchedulingEditPolicypublic long getMonitoringInterval()
getMonitoringInterval in interface SchedulingEditPolicypublic String getPolicyName()
getPolicyName in interface SchedulingEditPolicypublic Map<RMContainer,Long> getToPreemptContainers()
public TempQueuePerPartition getQueueByPartition(String queueName, String partition)
getQueueByPartition in interface CapacitySchedulerPreemptionContextpublic Collection<TempQueuePerPartition> getQueuePartitions(String queueName)
getQueuePartitions in interface CapacitySchedulerPreemptionContextpublic CapacityScheduler getScheduler()
getScheduler in interface CapacitySchedulerPreemptionContextpublic RMContext getRMContext()
getRMContext in interface CapacitySchedulerPreemptionContextpublic boolean isObserveOnly()
isObserveOnly in interface CapacitySchedulerPreemptionContextpublic Set<org.apache.hadoop.yarn.api.records.ContainerId> getKillableContainers()
getKillableContainers in interface CapacitySchedulerPreemptionContextpublic double getMaxIgnoreOverCapacity()
getMaxIgnoreOverCapacity in interface CapacitySchedulerPreemptionContextpublic double getNaturalTerminationFactor()
getNaturalTerminationFactor in interface CapacitySchedulerPreemptionContextpublic Set<String> getLeafQueueNames()
getLeafQueueNames in interface CapacitySchedulerPreemptionContextpublic Set<String> getAllPartitions()
getAllPartitions in interface CapacitySchedulerPreemptionContextpublic int getClusterMaxApplicationPriority()
getClusterMaxApplicationPriority in interface CapacitySchedulerPreemptionContextpublic float getMaxAllowableLimitForIntraQueuePreemption()
getMaxAllowableLimitForIntraQueuePreemption in interface CapacitySchedulerPreemptionContextpublic float getMinimumThresholdForIntraQueuePreemption()
getMinimumThresholdForIntraQueuePreemption in interface CapacitySchedulerPreemptionContextpublic org.apache.hadoop.yarn.api.records.Resource getPartitionResource(String partition)
getPartitionResource in interface CapacitySchedulerPreemptionContextpublic LinkedHashSet<String> getUnderServedQueuesPerPartition(String partition)
getUnderServedQueuesPerPartition in interface CapacitySchedulerPreemptionContextpublic void addPartitionToUnderServedQueues(String queueName, String partition)
addPartitionToUnderServedQueues in interface CapacitySchedulerPreemptionContextpublic ProportionalCapacityPreemptionPolicy.IntraQueuePreemptionOrderPolicy getIntraQueuePreemptionOrderPolicy()
getIntraQueuePreemptionOrderPolicy in interface CapacitySchedulerPreemptionContextpublic boolean getCrossQueuePreemptionConservativeDRF()
getCrossQueuePreemptionConservativeDRF in interface CapacitySchedulerPreemptionContextpublic boolean getInQueuePreemptionConservativeDRF()
getInQueuePreemptionConservativeDRF in interface CapacitySchedulerPreemptionContextpublic long getDefaultMaximumKillWaitTimeout()
getDefaultMaximumKillWaitTimeout in interface CapacitySchedulerPreemptionContextCopyright © 2008–2023 Apache Software Foundation. All rights reserved.