2026-10-31·作者 Sijie Wang

read-path

Read path: NameNode returns ordered blocks and distance-sorted replicas, client streams packets with checksum verification, falls through replicas on failure.

Read path: ask the NameNode for addresses, take data from the DataNodes

The motivation: the client starts with only a file name. A file is cut into blocks and scattered across DataNodes, and the map from file to blocks to machines lives only in the NameNode. So the first step of any read is always to ask the ledger — and then to go directly to the machines holding the data, because data never passes through the NameNode.

The steps. First, the client (DFSClient) asks the NameNode to open the file. Second, the NameNode checks permissions and returns two things: the file's block list in order, and for each block the DataNodes holding its replicas, sorted by distance — same machine first, then same rack, then further away. Third, the client connects directly to the nearest DataNode for the first block and streams it back packet by packet, verifying each packet's checksum as it reads. Fourth, it repeats for the next block in order, until the file is read and closed.

When something goes wrong. If a packet fails its checksum, that replica is bad: the client immediately retries the next replica and reports the bad replica to the NameNode, which later removes it from the valid set and re-replicates a good copy. If the DataNode being read dies mid-read, the client switches to the next replica and continues — no restart from the beginning. If every replica of a block has been tried and still fails, the client goes back to the NameNode and asks for a fresh location list.

The one sentence to keep. Ask the NameNode for addresses, take data from the DataNodes. The NameNode talks, it never carries.

關於這一條

sijie 的一條 wiki 條目。本站的 AI 以同一套語料庫為依據、以 sijie 的口吻作答,並附有指向此類條目的引用——回答會花 sijie 的錢,所以它在存取碼後等待: 輸入存取碼 →