public class ZookeeperFederationStateStore extends Object implements FederationStateStore
FederationStateStore.
The znode structure is as follows:
ROOT_DIR_PATH
|--- MEMBERSHIP
| |----- SC1
| |----- SC2
|--- APPLICATION
| |----- HIERARCHIES
| | |----- 1
| | | |----- (#ApplicationId barring last character)
| | | | | |----- APP Data
| | | ....
| | |
| | |----- 2
| | | |----- (#ApplicationId barring last 2 characters)
| | | | |----- (#Last 2 characters of ApplicationId)
| | | | | |----- APP Data
|--- POLICY
| |----- QUEUE1
| |----- QUEUE1
|--- RESERVATION
| |----- RESERVATION1
| |----- RESERVATION2
|--- ROUTER_RM_DT_SECRET_MANAGER_ROOT
| |----- ROUTER_RM_DELEGATION_TOKENS_ROOT
| | |----- RM_DELEGATION_TOKEN_1
| | |----- RM_DELEGATION_TOKEN_2
| | |----- RM_DELEGATION_TOKEN_3
| |----- ROUTER_RM_DT_MASTER_KEYS_ROOT
| | |----- DELEGATION_KEY_1
| |----- ROUTER_RM_DT_SEQUENTIAL_NUMBER| Modifier and Type | Field and Description |
|---|---|
protected static Version |
CURRENT_VERSION_INFO |
protected static String |
ROOT_ZNODE_NAME_VERSION |
static String |
ROUTER_APP_ROOT_HIERARCHIES |
| Constructor and Description |
|---|
ZookeeperFederationStateStore() |
| Modifier and Type | Method and Description |
|---|---|
AddApplicationHomeSubClusterResponse |
addApplicationHomeSubCluster(AddApplicationHomeSubClusterRequest request)
Register the home
SubClusterId of the newly submitted
ApplicationId. |
AddReservationHomeSubClusterResponse |
addReservationHomeSubCluster(AddReservationHomeSubClusterRequest request)
Register the home
SubClusterId of the newly submitted
ReservationId. |
void |
close()
Perform any cleanup operations of the StateStore.
|
DeletePoliciesConfigurationsResponse |
deleteAllPoliciesConfigurations(DeletePoliciesConfigurationsRequest request)
Delete all queue-to-policy configurations.
|
DeleteApplicationHomeSubClusterResponse |
deleteApplicationHomeSubCluster(DeleteApplicationHomeSubClusterRequest request)
Delete the mapping of home
SubClusterId of a previously submitted
ApplicationId. |
DeleteSubClusterPoliciesConfigurationsResponse |
deletePoliciesConfigurations(DeleteSubClusterPoliciesConfigurationsRequest request)
Delete PoliciesConfigurations.
|
DeleteReservationHomeSubClusterResponse |
deleteReservationHomeSubCluster(DeleteReservationHomeSubClusterRequest request)
Delete the mapping of home
SubClusterId of a previously submitted
ReservationId. |
void |
deleteStateStore()
We will clear the data in stateStore through the deleteStateStore method.
|
SubClusterDeregisterResponse |
deregisterSubCluster(SubClusterDeregisterRequest request)
Deregister a subcluster identified by
SubClusterId to
change state in federation. |
GetApplicationHomeSubClusterResponse |
getApplicationHomeSubCluster(GetApplicationHomeSubClusterRequest request)
Get information about the application identified by the input
ApplicationId. |
GetApplicationsHomeSubClusterResponse |
getApplicationsHomeSubCluster(GetApplicationsHomeSubClusterRequest request)
Get the
ApplicationHomeSubCluster list representing the mapping of
all submitted applications to it's home sub-cluster. |
int |
getCurrentKeyId()
Get Current KeyId.
|
Version |
getCurrentVersion()
Get the
Version of the underlying federation state store client. |
int |
getDelegationTokenSeqNum()
Get DelegationToken SeqNum.
|
RouterMasterKeyResponse |
getMasterKeyByDelegationKey(RouterMasterKeyRequest request)
ZookeeperFederationStateStore Supports Remove MasterKey.
|
ZKFederationStateStoreOpDurations |
getOpDurations() |
GetSubClusterPoliciesConfigurationsResponse |
getPoliciesConfigurations(GetSubClusterPoliciesConfigurationsRequest request)
Get a map of all queue-to-policy configurations.
|
GetSubClusterPolicyConfigurationResponse |
getPolicyConfiguration(GetSubClusterPolicyConfigurationRequest request)
Get the policy configuration for a given queue.
|
GetReservationHomeSubClusterResponse |
getReservationHomeSubCluster(GetReservationHomeSubClusterRequest request)
Get information about the Reservation identified by the input
ReservationId. |
GetReservationsHomeSubClusterResponse |
getReservationsHomeSubCluster(GetReservationsHomeSubClusterRequest request)
Get the
ReservationHomeSubCluster list representing the mapping of
all submitted Reservations to it's home sub-cluster. |
GetSubClusterInfoResponse |
getSubCluster(GetSubClusterInfoRequest request)
Get the membership information of subcluster as identified by
SubClusterId. |
GetSubClustersInfoResponse |
getSubClusters(GetSubClustersInfoRequest request)
Get the membership information of all the subclusters that are
currently participating in federation.
|
RouterRMTokenResponse |
getTokenByRouterStoreToken(RouterRMTokenRequest request)
The Router Supports GetTokenByRouterStoreToken.
|
int |
incrementCurrentKeyId()
The Router Supports incrementCurrentKeyId.
|
int |
incrementDelegationTokenSeqNum()
Increase SequenceNum.
|
void |
init(org.apache.hadoop.conf.Configuration conf)
Initialize the FederationStore.
|
Version |
loadVersion()
Load the version information from the federation state store.
|
SubClusterRegisterResponse |
registerSubCluster(SubClusterRegisterRequest request)
Register a subcluster by publishing capabilities as represented by
SubClusterInfo to indicate participation in federation. |
RouterMasterKeyResponse |
removeStoredMasterKey(RouterMasterKeyRequest request)
ZookeeperFederationStateStore Supports Remove MasterKey.
|
RouterRMTokenResponse |
removeStoredToken(RouterRMTokenRequest request)
ZookeeperFederationStateStore Supports Remove RMDelegationTokenIdentifier.
|
void |
setDelegationTokenSeqNum(int seqNum)
Set DelegationToken SeqNum.
|
SetSubClusterPolicyConfigurationResponse |
setPolicyConfiguration(SetSubClusterPolicyConfigurationRequest request)
Set the policy configuration for a given queue.
|
protected void |
storeApplicationStateInternal(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
byte[] data) |
RouterMasterKeyResponse |
storeNewMasterKey(RouterMasterKeyRequest request)
ZookeeperFederationStateStore Supports Store NewMasterKey.
|
RouterRMTokenResponse |
storeNewToken(RouterRMTokenRequest request)
ZookeeperFederationStateStore Supports Store RMDelegationTokenIdentifier.
|
void |
storeVersion()
Store the Version information in federation state store.
|
SubClusterHeartbeatResponse |
subClusterHeartbeat(SubClusterHeartbeatRequest request)
Periodic heartbeat from a
ResourceManager participating in
federation to indicate liveliness. |
UpdateApplicationHomeSubClusterResponse |
updateApplicationHomeSubCluster(UpdateApplicationHomeSubClusterRequest request)
Update the home
SubClusterId of a previously submitted
ApplicationId. |
protected void |
updateApplicationStateInternal(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
byte[] data) |
UpdateReservationHomeSubClusterResponse |
updateReservationHomeSubCluster(UpdateReservationHomeSubClusterRequest request)
Update the home
SubClusterId of a previously submitted
ReservationId. |
RouterRMTokenResponse |
updateStoredToken(RouterRMTokenRequest request)
ZookeeperFederationStateStore Supports Update RMDelegationTokenIdentifier.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitcheckVersionprotected static final String ROOT_ZNODE_NAME_VERSION
@VisibleForTesting public static final String ROUTER_APP_ROOT_HIERARCHIES
protected static final Version CURRENT_VERSION_INFO
public void init(org.apache.hadoop.conf.Configuration conf)
throws org.apache.hadoop.yarn.exceptions.YarnException
FederationStateStoreinit in interface FederationStateStoreconf - the cluster configurationorg.apache.hadoop.yarn.exceptions.YarnException - if initialization failspublic void close()
throws Exception
FederationStateStoreclose in interface FederationStateStoreException - if cleanup failspublic AddApplicationHomeSubClusterResponse addApplicationHomeSubCluster(AddApplicationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
SubClusterId of the newly submitted
ApplicationId. Currently response is empty if the operation was
successful, if not an exception reporting reason for a failure. If a
mapping for the application already existed, the SubClusterId in
this response will return the existing mapping which might be different
from that in the AddApplicationHomeSubClusterRequest.addApplicationHomeSubCluster in interface FederationApplicationHomeSubClusterStorerequest - the request to register a new application with its home sub-cluster.AddApplicationHomeSubClusterRequest containing the home
sub-cluster of the application. Otherwise, an exception reporting
reason for a failure.org.apache.hadoop.yarn.exceptions.YarnException - indicates exceptions from yarn servers.public UpdateApplicationHomeSubClusterResponse updateApplicationHomeSubCluster(UpdateApplicationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
SubClusterId of a previously submitted
ApplicationId. Currently response is empty if the operation was
successful, if not an exception reporting reason for a failure.updateApplicationHomeSubCluster in interface FederationApplicationHomeSubClusterStorerequest - the request to update the home sub-cluster of an
application.org.apache.hadoop.yarn.exceptions.YarnException - indicates exceptions from yarn servers.public GetApplicationHomeSubClusterResponse getApplicationHomeSubCluster(GetApplicationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
ApplicationId.getApplicationHomeSubCluster in interface FederationApplicationHomeSubClusterStorerequest - contains the application queriedApplicationHomeSubCluster containing the application's home
subclusterorg.apache.hadoop.yarn.exceptions.YarnException - indicates exceptions from yarn servers.public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(GetApplicationsHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
ApplicationHomeSubCluster list representing the mapping of
all submitted applications to it's home sub-cluster.getApplicationsHomeSubCluster in interface FederationApplicationHomeSubClusterStorerequest - empty representing all applicationsorg.apache.hadoop.yarn.exceptions.YarnException - indicates exceptions from yarn servers.public DeleteApplicationHomeSubClusterResponse deleteApplicationHomeSubCluster(DeleteApplicationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
SubClusterId of a previously submitted
ApplicationId. Currently response is empty if the operation was
successful, if not an exception reporting reason for a failure.deleteApplicationHomeSubCluster in interface FederationApplicationHomeSubClusterStorerequest - the request to delete the home sub-cluster of an
application.org.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic SubClusterRegisterResponse registerSubCluster(SubClusterRegisterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationMembershipStateStoreSubClusterInfo to indicate participation in federation. This is
typically done during initialization or restart/failover of the
subcluster's ResourceManager. Upon successful registration, an
identifier for the subcluster which is unique across the federated
cluster is returned. The identifier is static, i.e. preserved across
restarts and failover.registerSubCluster in interface FederationMembershipStateStorerequest - the capabilities of the subcluster that
wants to participate in federation. The subcluster id is also
specified in case registration is triggered by restart/failoverorg.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic SubClusterDeregisterResponse deregisterSubCluster(SubClusterDeregisterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationMembershipStateStoreSubClusterId to
change state in federation. This can be done to mark the sub cluster lost,
deregistered, or decommissioned.deregisterSubCluster in interface FederationMembershipStateStorerequest - - the request to deregister the
sub-cluster from federation.org.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic SubClusterHeartbeatResponse subClusterHeartbeat(SubClusterHeartbeatRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationMembershipStateStoreResourceManager participating in
federation to indicate liveliness. The heartbeat publishes the current
capabilities as represented by SubClusterInfo of the subcluster.
Currently response is empty if the operation was successful, if not an
exception reporting reason for a failure.subClusterHeartbeat in interface FederationMembershipStateStorerequest - the capabilities of the subcluster that
wants to keep alive its participation in federationorg.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic GetSubClusterInfoResponse getSubCluster(GetSubClusterInfoRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationMembershipStateStoreSubClusterId. The membership information includes the cluster
endpoint and current capabilities as represented by SubClusterInfo.getSubCluster in interface FederationMembershipStateStorerequest - the subcluster whose information is requiredSubClusterInfo, or null if there is no mapping
for the subclusterorg.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic GetSubClustersInfoResponse getSubClusters(GetSubClustersInfoRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationMembershipStateStoreSubClusterInfo.getSubClusters in interface FederationMembershipStateStorerequest - request for sub-clusters informationSubClusterInfo keyed by the SubClusterIdorg.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic GetSubClusterPolicyConfigurationResponse getPolicyConfiguration(GetSubClusterPolicyConfigurationRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationPolicyStoregetPolicyConfiguration in interface FederationPolicyStorerequest - the queue whose SubClusterPolicyConfiguration is
requiredSubClusterPolicyConfiguration for the specified queue,
or null if there is no mapping for the queueorg.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic SetSubClusterPolicyConfigurationResponse setPolicyConfiguration(SetSubClusterPolicyConfigurationRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationPolicyStoresetPolicyConfiguration in interface FederationPolicyStorerequest - the SubClusterPolicyConfiguration with the
corresponding queueSubClusterPolicyConfiguration for the specified queueorg.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic GetSubClusterPoliciesConfigurationsResponse getPoliciesConfigurations(GetSubClusterPoliciesConfigurationsRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationPolicyStoregetPoliciesConfigurations in interface FederationPolicyStorerequest - empty to represent all configured queues in the systemorg.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic DeleteSubClusterPoliciesConfigurationsResponse deletePoliciesConfigurations(DeleteSubClusterPoliciesConfigurationsRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationPolicyStoredeletePoliciesConfigurations in interface FederationPolicyStorerequest - List containing delete queues.org.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic DeletePoliciesConfigurationsResponse deleteAllPoliciesConfigurations(DeletePoliciesConfigurationsRequest request) throws Exception
FederationPolicyStoredeleteAllPoliciesConfigurations in interface FederationPolicyStorerequest - delete request.Exception - if the request is invalid/failspublic Version getCurrentVersion()
FederationStateStoreVersion of the underlying federation state store client.getCurrentVersion in interface FederationStateStoreVersion of the underlying federation store clientpublic Version loadVersion() throws Exception
FederationStateStoreloadVersion in interface FederationStateStoreVersion of the federation state storeException - an exception occurred in load version.public void storeVersion()
throws Exception
FederationStateStorestoreVersion in interface FederationStateStoreException - an exception occurred in store version.public void deleteStateStore()
throws Exception
FederationStateStoredeleteStateStore in interface FederationStateStoreException - an exception occurred in delete store.protected void storeApplicationStateInternal(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
byte[] data)
throws Exception
Exceptionprotected void updateApplicationStateInternal(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
byte[] data)
throws Exception
Exception@VisibleForTesting public ZKFederationStateStoreOpDurations getOpDurations()
public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(AddReservationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationReservationHomeSubClusterStoreSubClusterId of the newly submitted
ReservationId. Currently response is empty if the operation was
successful, if not an exception reporting reason for a failure. If a
mapping for the Reservation already existed, the SubClusterId in
this response will return the existing mapping which might be different
from that in the AddReservationHomeSubClusterRequest.addReservationHomeSubCluster in interface FederationReservationHomeSubClusterStorerequest - the request to register a new Reservation with its home
sub-clusterAddReservationHomeSubClusterRequest containing the home
sub-cluster of the Reservation. Otherwise, an exception reporting
reason for a failureorg.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic GetReservationHomeSubClusterResponse getReservationHomeSubCluster(GetReservationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationReservationHomeSubClusterStoreReservationId.getReservationHomeSubCluster in interface FederationReservationHomeSubClusterStorerequest - contains the Reservation queriedReservationHomeSubCluster containing the Reservation's home
subclusterorg.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic GetReservationsHomeSubClusterResponse getReservationsHomeSubCluster(GetReservationsHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationReservationHomeSubClusterStoreReservationHomeSubCluster list representing the mapping of
all submitted Reservations to it's home sub-cluster.getReservationsHomeSubCluster in interface FederationReservationHomeSubClusterStorerequest - empty representing all Reservationsorg.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic DeleteReservationHomeSubClusterResponse deleteReservationHomeSubCluster(DeleteReservationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationReservationHomeSubClusterStoreSubClusterId of a previously submitted
ReservationId. Currently response is empty if the operation was
successful, if not an exception reporting reason for a failure.deleteReservationHomeSubCluster in interface FederationReservationHomeSubClusterStorerequest - the request to delete the home sub-cluster of a reservation.org.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic UpdateReservationHomeSubClusterResponse updateReservationHomeSubCluster(UpdateReservationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationReservationHomeSubClusterStoreSubClusterId of a previously submitted
ReservationId. Currently response is empty if the operation was
successful, if not an exception reporting reason for a failure.updateReservationHomeSubCluster in interface FederationReservationHomeSubClusterStorerequest - the request to update the home sub-cluster of a reservation.org.apache.hadoop.yarn.exceptions.YarnException - if the request is invalid/failspublic RouterMasterKeyResponse storeNewMasterKey(RouterMasterKeyRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
storeNewMasterKey in interface FederationDelegationTokenStateStorerequest - The request contains RouterMasterKey, which is an abstraction for DelegationKeyorg.apache.hadoop.yarn.exceptions.YarnException - if the call to the state store is unsuccessful.IOException - An IO Error occurred.public RouterMasterKeyResponse removeStoredMasterKey(RouterMasterKeyRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
removeStoredMasterKey in interface FederationDelegationTokenStateStorerequest - The request contains RouterMasterKey, which is an abstraction for DelegationKeyorg.apache.hadoop.yarn.exceptions.YarnException - if the call to the state store is unsuccessful.IOException - An IO Error occurred.public RouterMasterKeyResponse getMasterKeyByDelegationKey(RouterMasterKeyRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
getMasterKeyByDelegationKey in interface FederationDelegationTokenStateStorerequest - The request contains RouterMasterKey, which is an abstraction for DelegationKeyorg.apache.hadoop.yarn.exceptions.YarnException - if the call to the state store is unsuccessful.IOException - An IO Error occurred.public RouterRMTokenResponse storeNewToken(RouterRMTokenRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
storeNewToken in interface FederationDelegationTokenStateStorerequest - The request contains RouterRMToken (RMDelegationTokenIdentifier and renewDate)org.apache.hadoop.yarn.exceptions.YarnException - if the call to the state store is unsuccessful.IOException - An IO Error occurred.public RouterRMTokenResponse updateStoredToken(RouterRMTokenRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
updateStoredToken in interface FederationDelegationTokenStateStorerequest - The request contains RouterRMToken (RMDelegationTokenIdentifier and renewDate)org.apache.hadoop.yarn.exceptions.YarnException - if the call to the state store is unsuccessful.IOException - An IO Error occurred.public RouterRMTokenResponse removeStoredToken(RouterRMTokenRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
removeStoredToken in interface FederationDelegationTokenStateStorerequest - The request contains RouterRMToken (RMDelegationTokenIdentifier and renewDate)org.apache.hadoop.yarn.exceptions.YarnException - if the call to the state store is unsuccessful.IOException - An IO Error occurred.public RouterRMTokenResponse getTokenByRouterStoreToken(RouterRMTokenRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
getTokenByRouterStoreToken in interface FederationDelegationTokenStateStorerequest - The request contains RouterRMToken (RMDelegationTokenIdentifier and renewDate)org.apache.hadoop.yarn.exceptions.YarnException - if the call to the state store is unsuccessfulIOException - An IO Error occurredpublic int incrementDelegationTokenSeqNum()
incrementDelegationTokenSeqNum in interface FederationDelegationTokenStateStorepublic int getDelegationTokenSeqNum()
getDelegationTokenSeqNum in interface FederationDelegationTokenStateStorepublic void setDelegationTokenSeqNum(int seqNum)
setDelegationTokenSeqNum in interface FederationDelegationTokenStateStoreseqNum - sequenceNum.public int getCurrentKeyId()
getCurrentKeyId in interface FederationDelegationTokenStateStorepublic int incrementCurrentKeyId()
incrementCurrentKeyId in interface FederationDelegationTokenStateStoreCopyright © 2008–2024 Apache Software Foundation. All rights reserved.