Skip to main content

Cloudera CDP to Hadoop In-Place Migration Procedure

Classic performs an in-place migration from Cloudera CDP 7.x and CDH 6 clusters onto open-source Hadoop: the Ilum Bigtop distribution, managed by Apache Ambari (Ilum build), operated directly or through Ilum Cluster Ops. Cloudera Manager is removed. The distribution and the management plane change; the cluster topology, bare-metal footprint, data, and operational model stay. The migration is executed by Bifrost, not by a human checklist, and every step is reversible until an explicit final step.

When to Use Classic​

Classic is appropriate when:

  • The priority is removing Cloudera licensing cost while keeping the existing Hadoop architecture.
  • Regulatory or organizational constraints prevent a move to Kubernetes in the short term.
  • A weekend cutover window is required, with a well-defined rollback path.
  • The estate is tightly integrated with bare-metal infrastructure: local-disk HDFS, hardware-level rack awareness, custom tuning.

Classic does not modernize workflows, convert HBase, or move storage. For those, use Modernize after Classic, or use Direct in a single engagement.

What Classic Covers​

Classic migrates Cloudera CDP 7.x and CDH 6 estates. Every component below is carried to the target and placed under the management plane; data directories are never moved. Further services, including Kudu, Ozone, the Kafka ecosystem, Flink, and NiFi, are listed in the Support matrix.

ComponentHow it is handled
HDFS, YARN, MapReduceMigrated in place, including NameNode, JournalNode, and ResourceManager high availability. Data directories are untouched and nothing is reformatted.
Hive and TezThe Hive Metastore database is re-pointed in place, not copied. On CDH 6 sources, the metastore schema is upgraded to the Hive 3 schema.
ZooKeeperThe data directory is carried, so application znodes are preserved.
Spark 2 and Spark 3Both lines run side by side as two management plane services, each with its history server.
LivyCarried for each Spark line.
HBasePreserved on HDFS, with hbase:meta kept intact.
KafkaZooKeeper and KRaft modes are detected; offsets, topics, and ACLs are backed up; brokers are drained before the cutover and checked for topic and partition parity after it.
Solr and Infra-SolrPreserved in place.
AtlasRebuilt and adopted under the management plane, with the Hive, HBase, Sqoop, Impala, and Spark hooks carried.
Ranger and Ranger KMSPolicies, plugins, and key stores are carried; policy content is compared before and after the cutover.
ImpalaAdopted under the management plane with Ranger authorization.
Hue, Oozie, Sqoop, ZeppelinCarried, including the Hue secret key and redaction rules and the Oozie share library.
Sentry (CDH 6 sources)Sentry roles and grants are translated to Ranger policies.

Service databases (Hive Metastore, Ranger, Oozie, Hue, and others) are reused in place on PostgreSQL, Oracle, or MySQL and MariaDB. PostgreSQL databases are dumped and restored by Bifrost. Oracle databases are exported with Data Pump, and MySQL and MariaDB databases with their native tools; their in-tool restore is optional, and an external database snapshot can be declared instead.

Security settings (Kerberos with MIT KDC or Active Directory, TLS with corporate PKI, Ranger, and HDFS encryption zones) are carried as they are. See Security and encryption.

Version-Aligned Target​

By default, the component versions of the target are aligned with the source. Three consequences follow:

  • No HDFS layout upgrade runs: the NameNode metadata keeps its layout.
  • The Hive Metastore is re-pointed, not upgraded.
  • Rollback restores packages and configuration without reverting HDFS.

The component versions of the default target line, and the supported sources, operating systems, JDKs, and databases, are listed in the Support matrix.

Upgrading component versions as part of the migration is common, and other target versions are supported, for example for CDH 6 sources. They follow the upgraded-version variant of phase 7, which runs the HDFS layout upgrade. Whether an upgrade in the same window is advisable depends on the complexity of the estate: the number of services, the client applications and integrations that depend on specific versions, and the length of the maintenance window. For complex estates, a version-aligned migration followed by a separate upgrade is usually recommended.

Adopted components run on JDK 11 by default: the source's own JDK 11 when present, otherwise a JDK 11 available on every host. Another JDK can be set for all adopted components with ambari-takeover --component-java-home; JDK 8 is used only when chosen explicitly. The Ambari server runs on JDK 17.

Phase Pipeline​

Classic has 11 numbered phases (0 to 10); on the default path phase 9 runs before phase 8. Phases 0 through 2 are non-destructive and can be run on live production clusters. Phases 3 and beyond require a maintenance window. The operator sequence is:

StepCommandsPurpose
Phase 0discoverInventory of the source estate.
Phase 1extractConfiguration translation and the adoption blueprint.
Plan validationscan-class-coverage, validate-planOffline checks of the migration plan.
Phase 2validate-prePre-flight checklist.
Stagingsmoke-keytab, canary prereqs, canary seed, canary capture, stage-nodes, stage-bigtopPreparation before the window.
Phases 2a to 7migrate --executeIn-window preflight and security steps, backup, stop, package swap, configuration deployment.
Management plane setupstage-ambari, validate-ambariAmbari server installation and parity check.
Phase 9ambari-takeover --executeManagement plane takeover; services start.
Phase 8canary capture, canary verify, validate-postBefore-and-after comparison and post-migration validation.
After the cutoverreplication-activateImported replication schedules are resumed.
Phase 10finalize --confirm-irreversible --executeIrreversible close-out after the soak.

bifrost classic estate plan prints the same chain as the commands it would run. See CLI reference: Classic commands.

Phase 0: Discover​

Connects to the Cloudera Manager API and inventories the source environment: hosts, roles, rack assignments, service configurations, security assets, encryption zones, keytabs, and TLS certificates. Over SSH, discovery also captures the deployed configuration files, principals and keytabs with their key version numbers, encryption-zone key versions, the Ranger policy baseline, service database coordinates, replication schedules, Oozie coordinators, and Solr collections. Discovery writes a manifest of required inputs and a readiness probe for the credentials that must be captured from the source. Produces a complete inventory tree under inventories/<cluster>/.

Command: bifrost classic discover. See CLI reference.

Phase 1: Extract and convert​

Translates Cloudera Manager configuration into standalone Hadoop XML templates. The translation runs in three stages:

  • Raw extraction: full deployment export plus actual rendered XML from Cloudera Manager agent process directories.
  • Property translation: maps Cloudera Manager property names to standard Hadoop equivalents, expands safety valve contents, filters Cloudera-internal properties (internal metrics, validation state, Cloudera-managed paths), and flags unmapped properties for manual review.
  • Template generation: produces Jinja2 templates for every configuration file and populates them with cluster-specific variables.

After generation, Bifrost renders the templates, canonicalizes with xmlstarlet c14n, and diffs the result against the source-rendered XML. Remaining differences must be explained and approved before the migration proceeds. Extraction also produces the adoption blueprint that the management plane takeover uses in phase 9.

Command: bifrost classic extract.

Plan Validation​

bifrost classic validate-plan checks the migration plan offline, without cluster access: whether discovery captured everything the migration needs, and what will differ after the migration (configuration drift, endpoints, service coverage). It returns one verdict. A finding can be waived only with a scope and a stated reason, and a waived finding stays visible in the report. bifrost classic scan-class-coverage lists the Java classes the source configuration references that the target packages lack, so that they are backported rather than removed from the configuration. See Plan validation.

Phase 2: Validate-pre​

Runs the full pre-flight checklist. Returns PROCEED, WARN, or ABORT.

CheckAbort condition
fsimage parseable by the target Hadoop versionParse failure
Edit logs parseable by the target Hadoop versionParse failure
NameNode layout versionUnexpected value
DataNode layout versionUnexpected value
HBase HFile validityAny corrupt HFile
Hive Metastore schema validityValidation failure
Keytab accessibilityAny failure
TLS certificate validityAbort: any certificate expired or expiring during the migration window. Warn: any certificate expiring within 6 months.
Encryption zones documented with keys backed upAny undocumented zone
Service ports reachable across nodesAny unreachable
Free disk space on data volumesBelow 10 % free (warn at 20 % free)
Distribution package cache created on all nodesMissing
Restore point (LVM or VM snapshot) declared for parcel-based sourcesMissing or older than the freshness window
Hive Metastore database backup verifiedCorrupt
Policy database backup verifiedCorrupt
Source fsimage compatibility with target NameNodeParse failure

The fsimage compatibility test is the most important check. Before any production cutover, Bifrost loads the source fsimage against the target Hadoop version on isolated test infrastructure. A failure here blocks the migration; a pass confirms that the upgrade is safe.

Command: bifrost classic validate-pre.

Staging Before the Window​

The following steps prepare the cluster without interrupting it:

  • On secured clusters, smoke-keytab places the smoke-test identity and canary prereqs provisions the canary identity and trust material.
  • canary seed writes a small deterministic dataset to the source, and canary capture --snapshot cdp-before records the before-state signatures of every service.
  • stage-nodes installs only the prerequisites the nodes lack.
  • stage-bigtop stages a local package mirror of the Ilum Bigtop distribution on every node, so that the cutover installs without network access. It refuses before any transfer when a node lacks the disk space.

Phases 2a to 2c: Window Preflight and Security Steps​

The first steps of bifrost classic migrate --execute run inside the window, before the backup:

  • 2a, air-gap preflight: confirms that every package the cutover needs is available from the local mirrors.
  • 2b, encryption-zone keys: on clusters with HDFS encryption zones, the kms-key-migrate step proves every key version decryptable on the target.
  • 2c, Sentry to Ranger: on CDH 6 sources with Sentry, the sentry-migrate step translates Sentry roles and grants into Ranger policies.

See Security and encryption.

Phase 3: Backup​

Creates comprehensive rollback assets for every subsequent phase:

  • HDFS namespace checkpoint (saveNamespace) and fsimage copy.
  • JournalNode edit logs and ZooKeeper state.
  • Records the declared restore point of the NameNode metadata volume (LVM or VM snapshot).
  • Hive Metastore database dump.
  • Policy database dump.
  • Policy export via REST API (JSON).
  • Oracle and MySQL database exports, where those databases are used.
  • Distribution package cache on all nodes (for in-place rollback without internet access).
  • Persistent copies of keytabs and TLS certificates.
  • An encrypted capture of service credentials that are readable only while the source runs.
  • Baseline metrics: HDFS fsck, HDFS report, HBase status, Hive table counts.

Phase 4: Stop services​

Graceful shutdown in dependency order. Master services run with serial: 1; worker services run in batches. Every stop includes a reachability check to confirm the port has closed. After shutdown, Bifrost verifies that all Java processes are terminated and stops the cluster manager agent on each node.

Phase 5: Swap packages​

Removes source distribution packages and installs the target distribution from a local mirror. No internet access is required during the swap.

  • Data directories are never touched.
  • Compatibility symlinks are created so applications referencing the source distribution paths continue to work.
  • Each node runs an inline rescue block: if installation fails, Bifrost automatically reinstalls the source distribution and restarts the cluster manager agent on that node before failing the overall play.

Phase 6: Deploy configs​

Deploys the converted Hadoop XML templates, keytabs from backup, TLS keystores, and sets the hadoop-conf alternatives link. Includes assertion checks that confirm critical properties in the rendered XML match expected values. From phase 9 onward, the configuration is owned by the management plane.

Phase 7: Start services and HDFS upgrade​

The start sequence depends on the target.

Version-aligned target (default). No HDFS layout upgrade is needed. The services remain stopped at the end of migrate and are started by the management plane takeover in phase 9.

Upgraded-version target. Services start in reverse dependency order. The exact sequence depends on the migration strategy.

Stop-and-swap. The cluster was fully shut down. Each NameNode starts with the hdfs namenode -upgrade startup argument, which converts the fsimage to the new layout version. DataNodes start in canary-first batches (1 node, then 5, then 20 % waves).

Shrink-and-grow. The cluster stays online. Bifrost runs hdfs dfsadmin -rollingUpgrade prepare on the active NameNode, then restarts the Standby NameNode first with the -rollingUpgrade started startup argument, waits for it to catch up, triggers an HA failover, and finally restarts the previously-active NameNode with the same argument. DataNodes follow in canary-first batches.

Prerequisite for shrink-and-grow. HDFS rolling upgrade is only available when the source and target NameNode layoutVersion values match. If the target distribution has a different layout version, rolling upgrade is not supported and the migration must use stop-and-swap. Bifrost's pre-flight check fails early if a shrink-and-grow run is attempted across incompatible layout versions.

Abort gates enforce time limits: if the NameNode does not exit safe mode before its gate time, rollback triggers automatically. The gates are durations measured from the start of phase 7 and are configured per cluster:

bifrost_abort_gates:
  namenode_up_secs: 14400      # NameNode active within 4 hours
  safe_mode_exit_secs: 21600   # safe mode exited within 6 hours
  all_datanodes_secs: 36000    # all DataNodes reporting within 10 hours

Phase 8: Validate-post​

Runs the full validation suite against the pre-migration baseline. Critical checks must all pass. Performance checks warn on regressions greater than 20 %. On the default path, this phase runs after the management plane takeover in phase 9.

Critical checks include HDFS write and read cycles, HDFS fsck cleanliness, block count parity, HBase meta scan, Hive table count parity, Kafka consumer offset preservation, Kerberos authentication success, policy enforcement, and encryption zone accessibility. Performance checks run TeraSort and TestDFSIO and compare against the baseline. Smoke tests cover HDFS, HBase, Hive, Kafka, Spark, and ZooKeeper.

Before validate-post, the canary compares the after-state of every service with the before-state recorded during staging. See Before-and-after canary.

Command: bifrost classic validate-post.

Phase 9: Management Plane Takeover​

Places the migrated cluster under Apache Ambari (Ilum build), operated directly or through Ilum Cluster Ops. The takeover is part of the migration window, and it is the step that starts the services on the default path:

  • Ambari agents are installed from the local package cache, and the cluster is created without starting anything.
  • The estate's own keytabs and TLS material are placed, Kerberos is enabled, Ranger plugins and Atlas hooks are carried, and per-host configuration differences become Ambari configuration groups.
  • Every service is started explicitly, and Ambari automatic restart is enabled.
  • The adopted configuration is certified as the baseline for later drift detection.

Cloudera Manager replication schedules are imported in a paused state. After validation, bifrost classic replication-activate runs a dry run of each schedule and resumes the schedules that read their source.

The takeover plans by default and acts with --execute. ambari-takeover --un-adopt removes the Ambari cluster again; the full inverse is rollback --from-adopted. See Management plane.

Command: bifrost classic ambari-takeover.

Phase 10: Finalize​

Irreversible. The 5-day soak that precedes finalize is not a recommendation; it is the rollback window. While the soak is in progress, the full "after services started" rollback procedure remains available (roughly a 4-hour revert). Finalize closes that window. Re-run validate-post at the end of the soak: finalize accepts only a PROCEED at most seven days old by default (BIFROST_FINALIZE_VALIDATE_POST_MAX_AGE_SECONDS). Finalize also refuses until every HDFS encryption-zone key is proven decryptable on the target.

bifrost classic finalize does two things:

  1. On an upgraded-version target, runs hdfs dfsadmin -finalizeUpgrade on the NameNode, which deletes the previous/ fsimage directory on the NameNode and every DataNode. After this point, both HDFS-native rollback mechanisms become unavailable: the NameNode start-up -rollback flag (used by stop-and-swap rollback) and hdfs namenode -rollingUpgrade rollback (used by shrink-and-grow rollback). Both depend on the previous/ directory being present.
  2. Removes the Bifrost-managed rollback assets: the distribution package cache, LVM snapshots, and baseline captures.

On an upgraded-version target, HDFS stays in the upgrade-in-progress state until finalize runs, with previous/ preserved on every data volume, and bifrost classic rollback relies on that state. On a version-aligned target no layout upgrade runs; -finalizeUpgrade is recorded as not applicable and rollback does not revert HDFS. For the full stage-by-stage rollback model with timings, see Validation and rollback: Rollback (Classic).

Command: bifrost classic finalize --confirm-irreversible --execute.

Migration Strategies​

Classic supports two strategies. The choice depends on cluster size and tolerance for downtime.

Stop-and-swap​

Full cluster shutdown. Every node is processed in parallel (fork count 50 is typical). Fastest total migration time, but requires complete downtime.

Cluster sizeTypical duration
17 nodes4 to 8 hours
29 nodes8 to 16 hours

Use --strategy stop-and-swap for clusters with fewer than 30 nodes, or when the customer has a well-defined weekend window and a straightforward rollback plan.

Shrink-and-grow​

DataNodes are decommissioned in batches from the live source cluster. Each batch follows the same cycle: decommission from HDFS, decommission from YARN, stop services, swap packages, deploy configs, start services, recommission.

HDFS remains available throughout. max_fail_percentage: 0 is the default for production: any single-batch failure aborts the play.

Cluster sizeTypical durationTypical batch size
60 nodes1 to 2 days10
120 nodes2 to 4 days10
300+ nodes4 to 8 days15 to 20

Use --strategy shrink-and-grow for clusters over 30 nodes or when partial HDFS availability must be preserved.

Timing Guidance​

Plan the weekend window from the abort gates. For an upgraded-version target, a typical Stop-and-Swap gate configuration for a Saturday 20:00 start, with phase 7 starting at about midnight, is:

  • 04:00 Sunday: NameNode must be active.
  • 06:00 Sunday: Safe mode must have exited.
  • 10:00 Sunday: All DataNodes must report healthy.
  • 18:00 Sunday: Full validation must pass.

On a version-aligned target, the services are started by the management plane takeover, and the same times serve as operator targets for the takeover and for validation.

Gates exceeded during migrate revert the cluster automatically; after the takeover, revert with rollback --from-adopted, or automatically when validate-post runs with --rollback-on-abort. A cluster that is reverted exits the weekend with the original configuration intact.

Planning Considerations​

The following items are prepared before the migration window:

  • Restore point. Parcel-based sources declare a fresh restore point of the NameNode metadata volume (LVM or VM snapshot) before every window. validate-pre checks that it exists and is recent.
  • Encryption keys. A Key Trustee KMS is moved to Ranger KMS before the window. For HSM-backed key stores, the HSM client is installed on the target hosts.
  • Databases. Oracle and MySQL sources need an account with export privileges. The Hue and Cloudera Manager databases are covered by a snapshot taken by the database team.
  • Queue management. Queues are managed in the YARN Queue Manager view of Apache Ambari (Ilum build). Queue definitions and placement rules are carried.
  • Accounts and automation. Cloudera Manager users and automation that calls the Cloudera Manager API move to Ambari or Ilum Cluster Ops accounts and the Ambari REST API; the Cloudera Manager API compatibility layer of Apache Ambari (Ilum build) supports a gradual move.
  • Alerts. Alert targets are derived from the Cloudera Manager alert configuration and are armed after the cutover. See Management plane: monitoring and alerts.

What Classic Does Not Do​

Classic explicitly does not handle the following. For these, use Modernize or Direct.

  • Oozie to Airflow conversion: Classic preserves Oozie; workflow modernization is a separate workstream.
  • HBase migration: Classic preserves HBase on HDFS; HBase migration is a Modernize or Direct track.
  • Storage migration: Classic leaves HDFS in place; storage migration to object storage is Modernize or Direct.

Next Steps​

Last updated on