@InterfaceAudience.Private @InterfaceStability.Unstable public class DefaultResourceCalculator extends ResourceCalculator
| Constructor and Description |
|---|
DefaultResourceCalculator() |
| Modifier and Type | Method and Description |
|---|---|
int |
compare(org.apache.hadoop.yarn.api.records.Resource unused,
org.apache.hadoop.yarn.api.records.Resource lhs,
org.apache.hadoop.yarn.api.records.Resource rhs,
boolean singleType)
On a cluster with capacity
clusterResource, compare lhs
and rhs. |
long |
computeAvailableContainers(org.apache.hadoop.yarn.api.records.Resource available,
org.apache.hadoop.yarn.api.records.Resource required)
Compute the number of containers which can be allocated given
available and required resources. |
float |
divide(org.apache.hadoop.yarn.api.records.Resource unused,
org.apache.hadoop.yarn.api.records.Resource numerator,
org.apache.hadoop.yarn.api.records.Resource denominator)
Divide resource
numerator by resource denominator
using specified policy (domination, average, fairness etc.); hence overall
clusterResource is provided for context. |
org.apache.hadoop.yarn.api.records.Resource |
divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
float denominator)
Divide-and-ceil
numerator by denominator. |
org.apache.hadoop.yarn.api.records.Resource |
divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
int denominator)
Divide-and-ceil
numerator by denominator. |
boolean |
fitsIn(org.apache.hadoop.yarn.api.records.Resource smaller,
org.apache.hadoop.yarn.api.records.Resource bigger)
Check if a smaller resource can be contained by bigger resource.
|
Set<String> |
getInsufficientResourceNames(org.apache.hadoop.yarn.api.records.Resource required,
org.apache.hadoop.yarn.api.records.Resource available)
Get insufficient resource names via comparing required resource and
capacity resource.
|
boolean |
isAllInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)
Determine if all resources are zero.
|
boolean |
isAnyMajorResourceAboveZero(org.apache.hadoop.yarn.api.records.Resource resource)
Check if resource has any major resource types (which are all NodeManagers
included) has a > 0 value.
|
boolean |
isAnyMajorResourceZeroOrNegative(org.apache.hadoop.yarn.api.records.Resource resource)
Check if resource has any major resource types (which are all NodeManagers
included) a zero value or negative value.
|
boolean |
isInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)
Determine if a resource is not suitable for use as a divisor
(will result in divide by 0, etc)
|
org.apache.hadoop.yarn.api.records.Resource |
multiplyAndNormalizeDown(org.apache.hadoop.yarn.api.records.Resource r,
double by,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
Multiply resource
r by factor by
and normalize down using step-factor stepFactor. |
org.apache.hadoop.yarn.api.records.Resource |
multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r,
double[] by,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
Multiply resource
r by factor by
and normalize up using step-factor stepFactor. |
org.apache.hadoop.yarn.api.records.Resource |
multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r,
double by,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
Multiply resource
r by factor by
and normalize up using step-factor stepFactor. |
org.apache.hadoop.yarn.api.records.Resource |
normalize(org.apache.hadoop.yarn.api.records.Resource r,
org.apache.hadoop.yarn.api.records.Resource minimumResource,
org.apache.hadoop.yarn.api.records.Resource maximumResource,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
Normalize resource
r given the base
minimumResource and verify against max allowed
maximumResource using a step factor for the normalization. |
org.apache.hadoop.yarn.api.records.Resource |
normalizeDown(org.apache.hadoop.yarn.api.records.Resource r,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
Get resource
rand normalize down using step-factor
stepFactor. |
float |
ratio(org.apache.hadoop.yarn.api.records.Resource a,
org.apache.hadoop.yarn.api.records.Resource b)
Ratio of resource
a to resource b. |
org.apache.hadoop.yarn.api.records.Resource |
roundDown(org.apache.hadoop.yarn.api.records.Resource r,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
Round-down resource
r given factor stepFactor. |
org.apache.hadoop.yarn.api.records.Resource |
roundUp(org.apache.hadoop.yarn.api.records.Resource r,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
Round-up resource
r given factor stepFactor. |
compare, divideAndCeil, divideAndCeil, divideAndCeil, divideAndCeil, divideSafelyAsFloat, roundDown, roundDown, roundUp, roundUppublic int compare(org.apache.hadoop.yarn.api.records.Resource unused,
org.apache.hadoop.yarn.api.records.Resource lhs,
org.apache.hadoop.yarn.api.records.Resource rhs,
boolean singleType)
ResourceCalculatorclusterResource, compare lhs
and rhs. Consider all resources unless singleType is set
to true. When singleType is set to true, consider only one
resource as per the ResourceCalculator implementation; the
DefaultResourceCalculator considers memory and
DominantResourceCalculator considers the dominant resource.compare in class ResourceCalculatorunused - cluster capacitylhs - First Resource to comparerhs - Second Resource to comparesingleType - Whether to consider a single resource type or all
resource typeslhs is smaller, 0 if equal and 1 if it is largerpublic long computeAvailableContainers(org.apache.hadoop.yarn.api.records.Resource available,
org.apache.hadoop.yarn.api.records.Resource required)
ResourceCalculatoravailable and required resources.computeAvailableContainers in class ResourceCalculatoravailable - available resourcesrequired - required resourcespublic float divide(org.apache.hadoop.yarn.api.records.Resource unused,
org.apache.hadoop.yarn.api.records.Resource numerator,
org.apache.hadoop.yarn.api.records.Resource denominator)
ResourceCalculatornumerator by resource denominator
using specified policy (domination, average, fairness etc.); hence overall
clusterResource is provided for context.divide in class ResourceCalculatorunused - cluster resourcesnumerator - numeratordenominator - denominatornumerator/denominator
using specific policypublic boolean isInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)
ResourceCalculatorisInvalidDivisor in class ResourceCalculatorr - resourcepublic float ratio(org.apache.hadoop.yarn.api.records.Resource a,
org.apache.hadoop.yarn.api.records.Resource b)
ResourceCalculatora to resource b.ratio in class ResourceCalculatora - resourceb - resourcea to resource bpublic org.apache.hadoop.yarn.api.records.Resource divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
int denominator)
ResourceCalculatornumerator by denominator.divideAndCeil in class ResourceCalculatornumerator - numerator resourcedenominator - denominatorpublic org.apache.hadoop.yarn.api.records.Resource divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
float denominator)
ResourceCalculatornumerator by denominator.divideAndCeil in class ResourceCalculatornumerator - numerator resourcedenominator - denominatorpublic org.apache.hadoop.yarn.api.records.Resource normalize(org.apache.hadoop.yarn.api.records.Resource r,
org.apache.hadoop.yarn.api.records.Resource minimumResource,
org.apache.hadoop.yarn.api.records.Resource maximumResource,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculatorr given the base
minimumResource and verify against max allowed
maximumResource using a step factor for the normalization.normalize in class ResourceCalculatorr - resourceminimumResource - minimum valuemaximumResource - the upper bound of the resource to be allocatedstepFactor - the increment for resources to be allocatedpublic org.apache.hadoop.yarn.api.records.Resource roundUp(org.apache.hadoop.yarn.api.records.Resource r,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculatorr given factor stepFactor.roundUp in class ResourceCalculatorr - resourcestepFactor - step-factorpublic org.apache.hadoop.yarn.api.records.Resource roundDown(org.apache.hadoop.yarn.api.records.Resource r,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculatorr given factor stepFactor.roundDown in class ResourceCalculatorr - resourcestepFactor - step-factorpublic org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r,
double by,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculatorr by factor by
and normalize up using step-factor stepFactor.multiplyAndNormalizeUp in class ResourceCalculatorr - resource to be multipliedby - multiplierstepFactor - factor by which to normalize uppublic org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r,
double[] by,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculatorr by factor by
and normalize up using step-factor stepFactor.multiplyAndNormalizeUp in class ResourceCalculatorr - resource to be multipliedby - multiplier array for all resource typesstepFactor - factor by which to normalize uppublic org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeDown(org.apache.hadoop.yarn.api.records.Resource r,
double by,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculatorr by factor by
and normalize down using step-factor stepFactor.multiplyAndNormalizeDown in class ResourceCalculatorr - resource to be multipliedby - multiplierstepFactor - factor by which to normalize downpublic boolean fitsIn(org.apache.hadoop.yarn.api.records.Resource smaller,
org.apache.hadoop.yarn.api.records.Resource bigger)
ResourceCalculatorfitsIn in class ResourceCalculatorsmaller - smaller resource.bigger - bigger resource.public org.apache.hadoop.yarn.api.records.Resource normalizeDown(org.apache.hadoop.yarn.api.records.Resource r,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculatorrand normalize down using step-factor
stepFactor.normalizeDown in class ResourceCalculatorr - resource to be multipliedstepFactor - factor by which to normalize downpublic boolean isAnyMajorResourceZeroOrNegative(org.apache.hadoop.yarn.api.records.Resource resource)
ResourceCalculatorisAnyMajorResourceZeroOrNegative in class ResourceCalculatorresource - resourcepublic boolean isAnyMajorResourceAboveZero(org.apache.hadoop.yarn.api.records.Resource resource)
ResourceCalculatorisAnyMajorResourceAboveZero in class ResourceCalculatorresource - resourcepublic Set<String> getInsufficientResourceNames(org.apache.hadoop.yarn.api.records.Resource required, org.apache.hadoop.yarn.api.records.Resource available)
ResourceCalculatorgetInsufficientResourceNames in class ResourceCalculatorrequired - - required resourceavailable - - available resourcepublic boolean isAllInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)
ResourceCalculatorisAllInvalidDivisor in class ResourceCalculatorr - resourceCopyright © 2008–2024 Apache Software Foundation. All rights reserved.