Safe mode: read-only until the NameNode can see enough of the cluster

The motivation: after a restart, the NameNode is half-blind. It recovers the directory tree and the file-to-block lists from FsImage plus EditLog (see ledger-persistence), but it has no block-to-DataNode map: that map lives only in the DataNodes, and it arrives piece by piece as they report in. Acting on an incomplete map would be wrong in two ways: mistaking a block for under-replicated because its other replicas have not reported yet, and triggering a wave of useless copying — or worse, deleting and repairing on the basis of information known to be incomplete.

Safe mode is read-only, and it issues no repair orders. On startup the NameNode enters safe mode automatically. In safe mode it offers only a read-only view of the file system to clients: reads of blocks already known work, but writes, deletes and renames fail. And it issues no block-replication or deletion instructions to DataNodes. It only accepts registrations and block reports, patiently rebuilding its map.

The exit condition has two parts. First, the minimal replication condition: the proportion of blocks meeting the minimum replication level (default 1, set by dfs.namenode.replication.min) must reach the threshold, dfs.namenode.safemode.threshold-pct, which defaults to 0.999 — 99.9% of blocks. Note carefully: a block counts as safe with just 1 replica, not with all 3; safe mode waits for coverage, not for full replication. Second, the extension: once the threshold is met, safe mode is extended by dfs.namenode.safemode.extension, 30,000 milliseconds by default, giving late nodes a buffer before the NameNode starts acting. Only after both does it exit automatically and begin generating replication requests for genuinely under-replicated blocks.

Getting stuck is safety by design. If nodes never come back, the threshold is never reached and the NameNode stays in safe mode. That is deliberate: Hadoop will not automatically delete blocks or trigger aggressive re-replication while DataNodes are missing. Check status with hdfs dfsadmin -safemode get. Forcing an exit with hdfs dfsadmin -safemode leave is allowed, but it is the administrator vouching that the unreported blocks really are lost and the missing nodes really are not coming back — it must not become a routine operation. An administrator can also enter safe mode manually, for example to hold the cluster still during maintenance, with hdfs dfsadmin -safemode enter.