2026-10-31·by Sijie Wang

read-path

Read path: NameNode returns ordered blocks and distance-sorted replicas, client streams packets with checksum verification, falls through replicas on failure.

Read path: ask the NameNode for addresses, take data from the DataNodes

The motivation: the client starts with only a file name. A file is cut into blocks and scattered across DataNodes, and the map from file to blocks to machines lives only in the NameNode. So the first step of any read is always to ask the ledger — and then to go directly to the machines holding the data, because data never passes through the NameNode.

The steps. First, the client (DFSClient) asks the NameNode to open the file. Second, the NameNode checks permissions and returns two things: the file's block list in order, and for each block the DataNodes holding its replicas, sorted by distance — same machine first, then same rack, then further away. Third, the client connects directly to the nearest DataNode for the first block and streams it back packet by packet, verifying each packet's checksum as it reads. Fourth, it repeats for the next block in order, until the file is read and closed.

When something goes wrong. If a packet fails its checksum, that replica is bad: the client immediately retries the next replica and reports the bad replica to the NameNode, which later removes it from the valid set and re-replicates a good copy. If the DataNode being read dies mid-read, the client switches to the next replica and continues — no restart from the beginning. If every replica of a block has been tried and still fails, the client goes back to the NameNode and asks for a fresh location list.

The one sentence to keep. Ask the NameNode for addresses, take data from the DataNodes. The NameNode talks, it never carries.

about this entry

One of sijie's wiki entries. The AI on this site is grounded in the same corpus and answers in sijie's voice, with citations back to entries like this one — answering costs sijie money, so it waits behind a code: enter an access code →