Node.js theorytheory 0/50 · 0%
I/O · easy

11. Streams

Processing data piece by piece instead of loading it all in memory.

A Stream processes data incrementally as chunks arrive, instead of buffering an entire file or response in memory first. Node has four kinds: Readable, Writable, Duplex (both), and Transform (a Duplex that modifies data as it passes through).

js
import { createReadStream, createWriteStream } from "node:fs";

const input = createReadStream("./big-log.txt");
const output = createWriteStream("./filtered.txt");
input.pipe(output);

input.on("data", (chunk) => console.log("got", chunk.length, "bytes"));
input.on("end", () => console.log("done"));

Streams are why Node can serve a multi-gigabyte file, or process a long-running RPC subscription's log data, using constant memory: each chunk is handled and discarded rather than accumulated.

Check your understanding

  1. 1. Why use streams for a large file instead of readFile?

  2. 2. Which stream type both reads and transforms data as it passes through?

  3. 3. What does `.pipe()` do?