Write path: one client send, a pipeline of relays, acknowledgements in reverse

The motivation: three requirements fighting each other. A write must satisfy a single writer (see writer-lease), it must land on 3 replicas, and the NameNode moves no data. So the 3 copies cannot be distributed by the NameNode, and the client cannot afford to send the same data 3 times — that would waste client bandwidth and time threefold.

The answer is a pipeline. The client sends once, and the nodes relay it down a chain.

The steps. First, the client applies to create the file and takes the lease (see writer-lease). Second, it asks the NameNode for the first block's 3 nodes, chosen rack-aware: the first as close to the writer as possible, the second on another rack, the third on the same rack as the second but a different node. Third, the client cuts the data into 64 KB packets and sends each packet only to the pipeline's first node; the first stores it and forwards it to the second, and the second stores it and forwards it to the third. Fourth, acknowledgements travel in reverse: the third acknowledges to the second, the second to the first, the first to the client. Only when the client receives the chain's acknowledgement is that packet counted as stably written on all 3 replicas. Fifth, when a block fills to 128 MB, the client asks the NameNode for the next block, a new pipeline is arranged, and writing continues. When the file is done, the client flushes the last partial packet, waits for its acknowledgements, tells the NameNode the write is complete, and the NameNode records the final length and releases the lease.

When something goes wrong. If a node dies mid-pipeline, the pipeline breaks: the NameNode removes it from the pipeline, and the client rebuilds the pipeline from the remaining nodes plus a new node. Already-acknowledged packets are not rewritten; only unacknowledged packets are resent. To the writer, that looks like a pause, not a rewrite of the whole file.

One hard rule. HDFS files are write-once, read-many: once written, a file cannot be randomly modified in the middle; it can only be appended to. That is the premise the lease and the pipeline stand on. Random multi-writer modification is simply unmanageable in this model.