File I/O with `std::fs`

20 min read

Quick Overview

Almost every command-line tool reads or writes files: a linter slurps source, a log processor streams gigabytes, a config tool writes back settings. In Node you reach for fs.readFileSync, fs.promises.readFile, or streams; in Rust the equivalent lives in the standard library’s std::fs and std::io modules — no crate required. The two big differences for a TypeScript/JavaScript developer: every fallible operation returns a Result you must handle (no silent ENOENT), and Rust draws a sharp line between cheap whole-file helpers and explicit buffered readers/writers for streaming.

This page covers the everyday operations: reading and writing whole files, buffering with BufReader/BufWriter, and reading a file line by line. Turning a string path into a real, cross-platform Path is covered in Path handling; reading configuration from the environment is in Environment variables.

The current stable toolchain is Rust 1.96.0 on the latest stable edition (2024); cargo new selects it automatically. Everything here is in the standard library, so there are no dependencies to add.

TypeScript/JavaScript Example

Here is a small log-filter tool in Node. It reads a file, keeps the lines containing a search term, writes them to a second file, and appends a one-line summary to an audit log.

1
// filter.ts — run with: npx tsx filter.ts access.log errors.log 500
2
// Uses only Node's built-in `node:fs` — no npm install needed.
3
import { readFileSync, writeFileSync, appendFileSync } from "node:fs";
4

5
const [input, output, needle] = process.argv.slice(2);
6
if (!input || !output || !needle) {
7
  console.error("usage: filter <input> <output> <needle>");
8
  process.exit(2);
9
}
10

11
// Read the whole file into a string (UTF-8 by default).
12
const text = readFileSync(input, "utf8");
13

14
// Filter the lines.
15
const matches = text.split("\n").filter((line) => line.includes(needle));
16

17
// Write the result and append a summary.
18
writeFileSync(output, matches.join("\n") + "\n");
19
appendFileSync("audit.log", `filtered ${matches.length} lines from ${input}\n`);
20

21
console.log(`wrote ${matches.length} matching line(s) to ${output}`);

1
$ npx tsx filter.ts access.log errors.log 500
2
wrote 2 matching line(s) to errors.log

This is idiomatic Node, but it has two quiet hazards that Rust forces you to confront. First, readFileSync loads the entire file into memory — fine for a 4 KB config, a problem for a 4 GB log. Second, if input does not exist, readFileSync throws and the error propagates as an uncaught exception; nothing in the type system reminded you to handle it.

Rust Equivalent

The direct translation uses the whole-file helpers fs::read_to_string, fs::write, and OpenOptions for appending. Notice the ? after every file call and the -> io::Result<()> on main.

1
use std::env;
2
use std::fs::{self, OpenOptions};
3
use std::io::{self, Write};
4
use std::process::ExitCode;
5

6
fn run() -> io::Result<()> {
7
    let args: Vec<String> = env::args().collect();
8
    let (input, output, needle) = match args.as_slice() {
9
        [_, i, o, n] => (i, o, n),
10
        _ => {
11
            eprintln!("usage: filter <input> <output> <needle>");
12
            std::process::exit(2);
13
        }
14
    };
15

16
    // Read the whole file as a UTF-8 String. `?` returns early on any I/O error.
17
    let text = fs::read_to_string(input)?;
18

19
    // Filter the lines. `lines()` is an iterator; `collect` into a Vec<&str>.
20
    let matches: Vec<&str> = text.lines().filter(|line| line.contains(needle)).collect();
21

22
    // Write the result in one call. Joining with '\n' rebuilds the file body.
23
    fs::write(output, matches.join("\n") + "\n")?;
24

25
    // Append a summary line, creating the audit log if needed.
26
    let mut audit = OpenOptions::new().create(true).append(true).open("audit.log")?;
27
    writeln!(audit, "filtered {} lines from {input}", matches.len())?;
28

29
    println!("wrote {} matching line(s) to {output}", matches.len());
30
    Ok(())
31
}
32

33
fn main() -> ExitCode {
34
    match run() {
35
        Ok(()) => ExitCode::SUCCESS,
36
        Err(e) => {
37
            eprintln!("error: {e}");
38
            ExitCode::FAILURE
39
        }
40
    }
41
}

1
$ cargo run --quiet -- access.log errors.log 500
2
wrote 2 matching line(s) to errors.log

This is correct and concise, and it mirrors the Node version closely. But like the Node version, it reads the whole file into RAM. The Real-World Example below rewrites it to stream the file with BufReader, so memory stays flat no matter how large the input is.

Detailed Explanation

Whole-file helpers: `read_to_string`, `read`, `write`

std::fs gives you three one-call helpers that open, do the work, and close the file for you. They are the equivalent of Node’s readFileSync / writeFileSync:

1
use std::fs;
2
use std::io;
3

4
fn main() -> io::Result<()> {
5
    fs::write("notes.txt", "first line\nsecond line\nthird line\n")?;
6

7
    // Read the file as a UTF-8 String. Errors if the bytes are not valid UTF-8.
8
    let contents: String = fs::read_to_string("notes.txt")?;
9
    println!("--- read_to_string ---");
10
    print!("{contents}");
11

12
    // Read the file as raw bytes. Never fails on encoding — bytes are bytes.
13
    let bytes: Vec<u8> = fs::read("notes.txt")?;
14
    println!("--- read (bytes) ---");
15
    println!("{} bytes", bytes.len());
16

17
    // `lines()` on a String/str splits on '\n' (and trims a trailing '\r').
18
    let count = contents.lines().count();
19
    println!("line count: {count}");
20

21
    fs::remove_file("notes.txt")?;
22
    Ok(())
23
}

1
--- read_to_string ---
2
first line
3
second line
4
third line
5
--- read (bytes) ---
6
34 bytes
7
line count: 3

fs::write(path, data) accepts anything that is AsRef<[u8]> — a &str, a String, a &[u8], or a Vec<u8>. It truncates and overwrites, exactly like writeFileSync with no flag. It creates the file if it does not exist.
fs::read_to_string(path) returns io::Result<String>. It fails with ErrorKind::InvalidData if the file is not valid UTF-8 — Rust will not hand you a half-broken string. Use fs::read for arbitrary bytes.
fs::read(path) returns io::Result<Vec<u8>> and is the binary-safe counterpart, like readFileSync(path) with no encoding argument (which returns a Buffer in Node).

Note: String::lines() is the precise analogue of JavaScript’s text.split("\n"), with two refinements: it does not yield a trailing empty string when the file ends in a newline, and it strips a trailing \r so Windows \r\n files just work. We lean on that for cross-platform line handling — see Cross-platform considerations.

The `?` operator and `io::Result`

Every fallible call returns Result<T, std::io::Error>, aliased as io::Result<T>. The ? operator unwraps the Ok value or returns the Err from the current function. Because main here is declared -> io::Result<()>, an unhandled error is printed via its Debug representation and the process exits non-zero. The mechanics of ? are covered in depth in The ? operator; for an error type that adds context, see anyhow and thiserror.

Buffered I/O: `BufReader` and `BufWriter`

A raw File performs one system call per read or write. Writing 100,000 lines directly to a File means 100,000 write(2) syscalls — slow. BufWriter batches them into a memory buffer (8 KB by default) and flushes in big chunks; BufReader does the symmetric thing for reads. This is the explicit version of what Node’s stream layer does for you under the hood.

1
use std::fs::File;
2
use std::io::{self, BufRead, BufReader, BufWriter, Write};
3

4
fn main() -> io::Result<()> {
5
    // --- buffered writing ---
6
    let file = File::create("log.txt")?;
7
    let mut writer = BufWriter::new(file);
8
    for i in 1..=5 {
9
        // `writeln!` writes into the buffer, not straight to disk.
10
        writeln!(writer, "event {i}")?;
11
    }
12
    writer.flush()?; // push the buffer to disk before we read it back
13

14
    // --- buffered, line-by-line reading ---
15
    let file = File::open("log.txt")?;
16
    let reader = BufReader::new(file);
17

18
    let mut total = 0usize;
19
    for line in reader.lines() {
20
        let line = line?; // each item is io::Result<String>
21
        if line.contains('3') {
22
            println!("matched: {line}");
23
        }
24
        total += 1;
25
    }
26
    println!("read {total} lines");
27

28
    std::fs::remove_file("log.txt")?;
29
    Ok(())
30
}

1
matched: event 3
2
read 5 lines

Key points:

File::create opens for writing, truncating any existing file; File::open opens read-only and errors if the file is missing.
writeln! / write! are macros that work on any Write target (a BufWriter, a File, even Vec<u8>). They are the file-writing cousins of println!/print!. You must bring the Write trait into scope with use std::io::Write to call them — see the pitfall below.
reader.lines() requires the BufRead trait (use std::io::BufRead). It yields io::Result<String> items — each line is a freshly allocated, owned String with the line terminator stripped.

Reading lines: three approaches, three trade-offs

There is more than one way to iterate lines, and the right choice depends on file size and whether you need owned strings.

1
use std::fs::{self, File};
2
use std::io::{self, BufRead, BufReader, Write};
3

4
fn main() -> io::Result<()> {
5
    fs::write("audit.log", "boot\n")?;
6

7
    // Append mode — like fs.appendFile / { flags: "a" } in Node.
8
    let mut f = fs::OpenOptions::new().append(true).open("audit.log")?;
9
    writeln!(f, "user logged in")?;
10
    writeln!(f, "user logged out")?;
11
    drop(f);
12

13
    // Reuse one String buffer across reads to avoid a per-line allocation.
14
    let file = File::open("audit.log")?;
15
    let mut reader = BufReader::new(file);
16
    let mut buf = String::new();
17
    let mut n = 0;
18
    while reader.read_line(&mut buf)? != 0 {
19
        print!("{n}: {buf}");
20
        buf.clear(); // crucial: read_line APPENDS, it does not overwrite
21
        n += 1;
22
    }
23

24
    fs::remove_file("audit.log")?;
25
    Ok(())
26
}

1
0: boot
2
1: user logged in
3
2: user logged out

The three approaches, ranked from simplest to fastest:

fs::read_to_string(path)?.lines() — read it all, then iterate &str slices. Zero per-line allocation, but the whole file is in memory. Best for small-to-medium files.
BufReader::new(file).lines() — streams the file, allocating a new String per line. The most readable for big files; the allocation is usually negligible.
reader.read_line(&mut buf) in a loop — streams the file and reuses one buffer, the lowest-allocation option. Note read_line appends to buf (it does not clear it) and keeps the trailing \n, so you call buf.clear() each iteration. This is the hot-loop choice for multi-gigabyte inputs.

`OpenOptions`: append, and everything `File::create`/`open` cannot express

File::open and File::create are shorthands. For anything else — append mode, create-if-missing-but-don’t-truncate, create-only-if-new — use OpenOptions, the builder equivalent of Node’s fs.open(path, flags):

Node `flags`	`OpenOptions` builder
`"r"`	`OpenOptions::new().read(true)` (or just `File::open`)
`"w"`	`OpenOptions::new().write(true).create(true).truncate(true)` (or `File::create`)
`"a"`	`OpenOptions::new().append(true).create(true)`
`"wx"` (fail if exists)	`OpenOptions::new().write(true).create_new(true)`

.append(true) implies write and seeks to the end before every write — concurrent appends from multiple processes do not clobber each other on most platforms.

Key Differences

Concern	Node.js (`node:fs`)	Rust (`std::fs` / `std::io`)
Missing-file handling	Throws (sync) / rejects (async); easy to forget	Returns `Result`; the compiler warns if you ignore it
Default read result	`Buffer`, or `string` with `"utf8"`	`Vec<u8>` (`fs::read`) or `String` (`fs::read_to_string`)
Invalid UTF-8	Silently replaced with `�` in `"utf8"` mode	`read_to_string` errors with `ErrorKind::InvalidData`
Buffering	Automatic in the streams layer	Explicit: wrap in `BufReader` / `BufWriter`
Flushing	Handled by stream `end()`/GC	You call `.flush()` (or rely on `Drop`, which can hide errors)
Line splitting	`text.split("\n")` (keeps trailing empty, keeps `\r`)	`str::lines()` (drops trailing empty, strips `\r`)
Sync vs async	`readFileSync` vs `fs.promises` / streams	`std::fs` is blocking; async needs `tokio::fs`

`std::fs` is blocking — and that is fine for a CLI

Every std::fs call blocks the current thread until the OS finishes. For a typical CLI tool that does its work and exits, blocking is exactly what you want — it is simpler and faster than an async runtime. You only need tokio::fs (covered in Section 11: Async) when file I/O happens inside an async server that must keep serving other requests. Do not reach for async file I/O in a command-line tool by reflex; in JavaScript the async API is the default, in Rust the blocking API is the default for CLIs.

Common Pitfalls

Ignoring the `Result` from a write

In JavaScript, writeFileSync either works or throws; you can fire and forget. In Rust, a Result you do not use triggers a warning, because the write may have silently failed (disk full, permission denied).

1
use std::fs;
2

3
fn main() {
4
    // compiles, but with a warning (unused `Result` that must be used)
5
    fs::write("out.txt", "data");
6
    println!("done");
7
}

1
warning: unused `Result` that must be used
2
 --> src/main.rs:5:5
3
  |
4
5 |     fs::write("out.txt", "data");
5
  |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
6
  |
7
  = note: this `Result` may be an `Err` variant, which should be handled
8
  = note: `#[warn(unused_must_use)]` on by default
9
help: use `let _ = ...` to ignore the resulting value
10
  |
11
5 |     let _ = fs::write("out.txt", "data");
12
  |     +++++++

Handle it with ? (and an io::Result return type) or .expect("..."). Only use let _ = when you have genuinely decided the failure is irrelevant.

Forgetting `use std::io::Write` (or `BufRead`)

writeln! on a writer needs the Write trait in scope; .lines() / .read_line() need BufRead. Without the import, the methods appear not to exist:

1
use std::fs::File;
2
use std::io::BufWriter; // missing: use std::io::Write;
3

4
fn main() -> std::io::Result<()> {
5
    let mut w = BufWriter::new(File::create("x.txt")?);
6
    // does not compile (error[E0599]: cannot write into `BufWriter<File>`)
7
    writeln!(w, "hello")?;
8
    Ok(())
9
}

The real error names the trait you forgot (the ::: line points into your local toolchain’s copy of the standard library, so its path will differ on your machine):

1
error[E0599]: cannot write into `BufWriter<File>`
2
    --> src/main.rs:6:14
3
     |
4
   6 |     writeln!(w, "hello")?;
5
     |              ^
6
     |
7
    ::: /home/you/.rustup/toolchains/stable/lib/rustlib/src/rust/library/std/src/io/mod.rs:1950:8
8
     |
9
1950 |     fn write_fmt(&mut self, args: fmt::Arguments<'_>) -> Result<()> {
10
     |        --------- the method is available for `BufWriter<File>` here
11
     |
12
note: must implement `io::Write`, `fmt::Write`, or have a `write_fmt` method
13
    --> src/main.rs:6:14
14
     |
15
   6 |     writeln!(w, "hello")?;
16
     |              ^
17
     = help: items from traits can only be used if the trait is in scope
18
help: trait `Write` which provides `write_fmt` is implemented but not in scope; perhaps you want to import it
19
     |
20
   1 + use std::io::Write;
21
     |

The fix is exactly what the compiler suggests: add use std::io::Write;.

Dropping a `BufWriter` without flushing

A BufWriter flushes its buffer when it is dropped — but the flush at drop time cannot return an error, so a failure (disk full, broken pipe) is silently swallowed. Always call .flush()? explicitly when you care whether the bytes actually landed:

1
use std::fs::File;
2
use std::io::{self, BufWriter, Write};
3

4
fn save(path: &str, data: &[&str]) -> io::Result<()> {
5
    let mut w = BufWriter::new(File::create(path)?);
6
    for line in data {
7
        writeln!(w, "{line}")?;
8
    }
9
    w.flush()?; // surfaces any error HERE, instead of losing it at drop
10
    Ok(())
11
}
12

13
fn main() -> io::Result<()> {
14
    save("ok.txt", &["one", "two"])?;
15
    std::fs::remove_file("ok.txt")?;
16
    Ok(())
17
}

Warning: This is a real correctness bug, not a style nit. Without the explicit flush(), a program can print “Saved!” and exit 0 while the last buffered chunk never reached disk.

Expecting `read_to_string` to tolerate non-UTF-8 bytes

Node’s "utf8" mode quietly substitutes � for invalid bytes; fs::read_to_string refuses and returns an error. Read raw bytes and convert lossily if you want the Node behavior:

1
use std::fs;
2
use std::io;
3

4
fn main() -> io::Result<()> {
5
    fs::write("bin.dat", [0x68, 0x69, 0xFF])?; // 0xFF is never valid UTF-8
6

7
    match fs::read_to_string("bin.dat") {
8
        Ok(s) => println!("text: {s}"),
9
        Err(e) => println!("read_to_string failed: kind={:?}", e.kind()),
10
    }
11

12
    let raw = fs::read("bin.dat")?;
13
    // Best-effort, like Buffer.toString("utf8") with replacement chars.
14
    println!("lossy: {}", String::from_utf8_lossy(&raw));
15

16
    fs::remove_file("bin.dat")?;
17
    Ok(())
18
}

1
read_to_string failed: kind=InvalidData
2
lossy: hi�

Forgetting that `read_line` keeps the newline and appends

A surprising number of bugs come from read_line not behaving like a “give me the next line, trimmed” function. It appends to the buffer (so you must buf.clear() each loop) and it retains the trailing \n (use line.trim_end() if you need it gone). The .lines() iterator, by contrast, strips the terminator for you.

Best Practices

Match the tool to the file size. Reach for fs::read_to_string / fs::write for small files (configs, single source files). Switch to BufReader/BufWriter the moment a file could be large or unbounded, like a log stream.
Always wrap a File in BufReader/BufWriter when you read or write in a loop. Unbuffered per-iteration syscalls are the most common accidental performance cliff.
Call .flush()? explicitly on any BufWriter whose success you report to the user; do not rely on the silent drop-time flush.
Return io::Result<T> and propagate with ? rather than .unwrap() in real tools. Reserve .unwrap()/.expect() for tests and quick prototypes — see unwrap and expect.
Match on error.kind() to recover from expected conditions (a missing optional config file) while still failing loudly on unexpected ones:

1
use std::fs;
2
use std::io::ErrorKind;
3

4
fn load_config() -> String {
5
    match fs::read_to_string("config.toml") {
6
        Ok(text) => text,
7
        Err(e) if e.kind() == ErrorKind::NotFound => {
8
            // Missing config is fine — fall back to defaults.
9
            String::from("default = true")
10
        }
11
        Err(e) => {
12
            eprintln!("failed to read config: {e}");
13
            std::process::exit(1);
14
        }
15
    }
16
}
17

18
fn main() {
19
    println!("config = {:?}", load_config());
20
}

1
config = "default = true"

Tip: ErrorKind::NotFound is the moral equivalent of checking err.code === "ENOENT" in Node, but it is a typed enum variant the compiler knows about — no stringly-typed comparison.

Use fs::exists(path)? (stabilized in recent Rust) rather than the older Path::exists() when you want to distinguish “does not exist” from “exists but I lack permission to check” — fs::exists returns io::Result<bool> and surfaces the permission error instead of swallowing it.

Real-World Example

Here is the log-filter from the top, rewritten to stream through BufReader/BufWriter. It processes one line at a time, so a 50 GB log uses the same memory as a 50-byte one. It exits with a meaningful status code, so it composes in shell pipelines — exit codes are covered in Cross-platform considerations.

1
use std::env;
2
use std::fs::File;
3
use std::io::{self, BufRead, BufReader, BufWriter, Write};
4
use std::process::ExitCode;
5

6
/// Stream `input`, writing every line that contains `needle` into `output`.
7
/// Memory stays flat regardless of file size — we never hold the whole file,
8
/// only one line at a time.
9
fn filter_file(input: &str, output: &str, needle: &str) -> io::Result<usize> {
10
    let reader = BufReader::new(File::open(input)?);
11
    let mut writer = BufWriter::new(File::create(output)?);
12

13
    let mut matches = 0;
14
    for line in reader.lines() {
15
        let line = line?;
16
        if line.contains(needle) {
17
            writeln!(writer, "{line}")?;
18
            matches += 1;
19
        }
20
    }
21
    writer.flush()?; // make errors surface here, not silently on drop
22
    Ok(matches)
23
}
24

25
fn main() -> ExitCode {
26
    let args: Vec<String> = env::args().collect();
27
    if args.len() != 4 {
28
        eprintln!("usage: {} <input> <output> <needle>", args[0]);
29
        return ExitCode::from(2);
30
    }
31
    match filter_file(&args[1], &args[2], &args[3]) {
32
        Ok(n) => {
33
            println!("wrote {n} matching line(s) to {}", args[2]);
34
            ExitCode::SUCCESS
35
        }
36
        Err(e) => {
37
            eprintln!("error: {e}");
38
            ExitCode::FAILURE
39
        }
40
    }
41
}

Given an access.log of:

1
GET /index 200
2
POST /login 500
3
GET /style.css 200
4
GET /api 500

it runs as:

1
$ cargo run --quiet -- access.log errors.log 500
2
wrote 2 matching line(s) to errors.log
3

4
$ cat errors.log
5
POST /login 500
6
GET /api 500
7

8
$ cargo run --quiet -- access.log
9
usage: target/debug/probe <input> <output> <needle>
10
$ echo $?
11
2

A missing input file no longer crashes with a stack trace; it is caught by ?, formatted by the Err arm, and turned into exit code 1:

1
$ cargo run --quiet -- nope.log out.log 500
2
error: No such file or directory (os error 2)
3
$ echo $?
4
1

In a real tool you would parse these three positional arguments with clap instead of indexing args by hand — see clap derive API — and you might wrap errors with anyhow to attach the offending path to the message.

Exercises

Exercise 1: Line numbering

Difficulty: Beginner

Objective: Practice buffered reading and writing with BufReader/BufWriter.

Instructions: Write a function number_lines(input: &str, output: &str) -> io::Result<()> that copies input to output, prefixing each line with its 1-based number right-aligned in a 4-character field followed by two spaces (so line 1 becomes 1 <text>). Use buffered I/O and propagate errors with ?.

Solution

1
use std::fs::File;
2
use std::io::{self, BufRead, BufReader, BufWriter, Write};
3

4
fn number_lines(input: &str, output: &str) -> io::Result<()> {
5
    let reader = BufReader::new(File::open(input)?);
6
    let mut writer = BufWriter::new(File::create(output)?);
7
    for (i, line) in reader.lines().enumerate() {
8
        let line = line?;
9
        writeln!(writer, "{:>4}  {line}", i + 1)?;
10
    }
11
    writer.flush()
12
}
13

14
fn main() -> io::Result<()> {
15
    std::fs::write("in.txt", "alpha\nbeta\ngamma\n")?;
16
    number_lines("in.txt", "out.txt")?;
17
    print!("{}", std::fs::read_to_string("out.txt")?);
18
    std::fs::remove_file("in.txt")?;
19
    std::fs::remove_file("out.txt")?;
20
    Ok(())
21
}

1
   1  alpha
2
   2  beta
3
   3  gamma

enumerate() pairs each line with its index; {:>4} right-aligns the number in 4 columns. The final writer.flush() (whose io::Result becomes the function’s return value) guarantees everything reaches disk before number_lines returns.

Exercise 2: A tiny `wc`

Difficulty: Intermediate

Objective: Combine whole-file reading with graceful error handling on ErrorKind.

Instructions: Write count(path: &str) -> io::Result<(usize, usize, usize)> returning (lines, words, bytes) for a file. Then, in main, count a list of paths and, for any file that does not exist, print <path>: no such file to stderr and continue with the rest instead of aborting. (Hint: str::split_whitespace counts words; str::len counts bytes.)

Solution

1
use std::fs;
2
use std::io::{self, ErrorKind};
3

4
fn count(path: &str) -> io::Result<(usize, usize, usize)> {
5
    let text = fs::read_to_string(path)?;
6
    let lines = text.lines().count();
7
    let words = text.split_whitespace().count();
8
    let bytes = text.len();
9
    Ok((lines, words, bytes))
10
}
11

12
fn main() {
13
    fs::write("sample.txt", "the quick brown fox\njumps over\n").unwrap();
14
    for path in ["sample.txt", "missing.txt"] {
15
        match count(path) {
16
            Ok((l, w, b)) => println!("{l:>3} {w:>3} {b:>3} {path}"),
17
            Err(e) if e.kind() == ErrorKind::NotFound => {
18
                eprintln!("{path}: no such file");
19
            }
20
            Err(e) => eprintln!("{path}: {e}"),
21
        }
22
    }
23
    fs::remove_file("sample.txt").unwrap();
24
}

1
  2   6  31 sample.txt
2
missing.txt: no such file

The Err(e) if e.kind() == ErrorKind::NotFound guard handles the expected “file missing” case, while a final Err(e) arm still reports anything unexpected (like a permission error). Because each path is handled independently inside the loop, one missing file does not stop the others.

Exercise 3: Streaming `uniq`

Difficulty: Advanced

Objective: Process an arbitrarily large file in constant memory using read_line with a reused buffer.

Instructions: Write uniq(input: &str, output: &str) -> io::Result<usize> that copies input to output, collapsing consecutive identical lines into one (like the Unix uniq command). Return the number of lines written. Constraint: you must not load the whole file into memory — read one line at a time, comparing only against the previous line.

Solution

1
use std::fs::File;
2
use std::io::{self, BufRead, BufReader, BufWriter, Write};
3

4
/// Like `uniq`: drop a line if it is identical to the line just written.
5
/// Uses two reused buffers so memory does not grow with the file.
6
fn uniq(input: &str, output: &str) -> io::Result<usize> {
7
    let mut reader = BufReader::new(File::open(input)?);
8
    let mut writer = BufWriter::new(File::create(output)?);
9

10
    let mut current = String::new();
11
    let mut previous = String::new();
12
    let mut written = 0;
13
    let mut first = true;
14

15
    while reader.read_line(&mut current)? != 0 {
16
        if first || current != previous {
17
            write!(writer, "{current}")?; // read_line keeps the '\n'
18
            written += 1;
19
            first = false;
20
        }
21
        std::mem::swap(&mut previous, &mut current);
22
        current.clear();
23
    }
24
    writer.flush()?;
25
    Ok(written)
26
}
27

28
fn main() -> io::Result<()> {
29
    std::fs::write("dup.txt", "a\na\nb\nb\nb\na\n")?;
30
    let n = uniq("dup.txt", "uniq.txt")?;
31
    println!("kept {n} lines:");
32
    print!("{}", std::fs::read_to_string("uniq.txt")?);
33
    std::fs::remove_file("dup.txt")?;
34
    std::fs::remove_file("uniq.txt")?;
35
    Ok(())
36
}

1
kept 3 lines:
2
a
3
b
4
a

The trick is two String buffers swapped with std::mem::swap: after writing current, it becomes the new previous (no allocation, just a pointer swap), and current.clear() readies it for the next read_line. Because read_line retains the trailing \n, we use write! (not writeln!) so we do not double the newlines. Memory is bounded by the longest single line, not the file size.

File I/O with `std::fs`

Quick Overview

TypeScript/JavaScript Example

Rust Equivalent

Detailed Explanation

Whole-file helpers: `read_to_string`, `read`, `write`

The `?` operator and `io::Result`

Buffered I/O: `BufReader` and `BufWriter`

Reading lines: three approaches, three trade-offs

`OpenOptions`: append, and everything `File::create`/`open` cannot express

Key Differences

`std::fs` is blocking — and that is fine for a CLI

Common Pitfalls

Ignoring the `Result` from a write

Forgetting `use std::io::Write` (or `BufRead`)

Dropping a `BufWriter` without flushing

Expecting `read_to_string` to tolerate non-UTF-8 bytes

Forgetting that `read_line` keeps the newline and appends

Best Practices

Real-World Example

Further Reading

Official documentation

Exercises

Exercise 1: Line numbering

Exercise 2: A tiny `wc`

Exercise 3: Streaming `uniq`

File I/O with std::fs

Quick Overview

TypeScript/JavaScript Example

Rust Equivalent

Detailed Explanation

Whole-file helpers: read_to_string, read, write

The ? operator and io::Result

Buffered I/O: BufReader and BufWriter

Reading lines: three approaches, three trade-offs

OpenOptions: append, and everything File::create/open cannot express

Key Differences

std::fs is blocking — and that is fine for a CLI

Common Pitfalls

Ignoring the Result from a write

Forgetting use std::io::Write (or BufRead)

Dropping a BufWriter without flushing

Expecting read_to_string to tolerate non-UTF-8 bytes

Forgetting that read_line keeps the newline and appends

Best Practices

Real-World Example

Further Reading

Official documentation

Related sections of this guide

Exercises

Exercise 1: Line numbering

Exercise 2: A tiny wc

Exercise 3: Streaming uniq

File I/O with `std::fs`

Whole-file helpers: `read_to_string`, `read`, `write`

The `?` operator and `io::Result`

Buffered I/O: `BufReader` and `BufWriter`

`OpenOptions`: append, and everything `File::create`/`open` cannot express

`std::fs` is blocking — and that is fine for a CLI

Ignoring the `Result` from a write

Forgetting `use std::io::Write` (or `BufRead`)

Dropping a `BufWriter` without flushing

Expecting `read_to_string` to tolerate non-UTF-8 bytes

Forgetting that `read_line` keeps the newline and appends

Exercise 2: A tiny `wc`

Exercise 3: Streaming `uniq`