Verifying a download without trusting the page you got it from

You download a file, and beside the download link there is a line of hexadecimal with sha256 in front of it. This piece is about what comparing it establishes, where the check quietly stops meaning anything, and what to do when the two values do not agree.

The short version: a checksum proves the file you have is byte-for-byte identical to the file the publisher hashed. It proves that and nothing else, and the nothing else is doing a great deal of work, because the check is worth zero if the digest reached you over the same channel as the file.

What a checksum actually proves

What you get is a commitment. If the publisher publishes the digest of a file, and you compute the digest of your copy, and the two strings are equal, then your copy and the publisher's copy are the same bytes — not the same version number, not the same file size, the same bytes. A single flipped bit anywhere produces a digest that looks nothing like the published one.

What you do not get is any statement about who the publisher is, or whether the file is safe to run. A digest is not a signature. Anyone can compute the SHA-256 of a malicious installer and publish it beside the installer, and the arithmetic will be correct and the download will verify. The check tells you the file is the file that was hashed. It has no opinion on who did the hashing.

Which reduces the whole thing to one question: how did the digest reach you? If the file and its checksum both came off the same page over the same connection, then whoever can change one can change the other, and the comparison was decided before you ran it. It would pass for any file on that page, including one swapped five minutes ago. It confirms that the transfer completed, which is a real thing to know and a much smaller thing than it feels like.

A checksum verification is only ever as strong as the independence of the channel that carried it.

Why the digest is the same size every time

A hash function is built around a compression function with a fixed internal state — 256 bits for SHA-256, 512 for SHA-512 — and it walks the input through that state in fixed-size blocks. The output is that final state, written out. The length of the input changes how many times the state is churned and never how many bits come out.

So the length of the string in front of you is a property of the algorithm and nothing else. SHA-256 returns 256 bits, which is 32 bytes, which in hexadecimal is four bits per character, which is always 64 characters. SHA-1 returns 160 bits, so 40. MD5 returns 128 bits, so 32. The same three input bytes through all three:

input   abc

MD5     900150983cd24fb0d6963f7d28e17f72
SHA-1   a9993e364706816aba3e25717850c26c9cd0d89d
SHA-256 ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad

The other property that matters is that the function does not run backwards, and that its output does not vary gently with its input. Change one bit anywhere and each bit of the output flips with a probability of about one half, so roughly thirty of those sixty-four characters come out different. Two files that differ by one byte do not have digests that differ by one character. Hold on to that, because it is what lets you read a mismatch and know which kind of problem you are holding.

The algorithms that no longer qualify

MD5 and SHA-1 were both designed to be collision-resistant, and neither of them still is. A collision is a pair of different inputs with the same digest. Practical MD5 collisions have been constructible since 2004, and a chosen-prefix pair takes seconds on ordinary hardware today. For SHA-1, the SHAttered result in 2017 produced two different PDF documents with the identical digest 38762cf7f55934b34d179ae6a4c80cadccbb7f0a, at a cost its authors put at several thousand years of single-CPU computation. That is a large number, but it is a fixed and purchasable one, and it has come down since.

The attack that matters for a download is not "find a file matching this digest", which is the preimage problem and is still hard. It is "prepare two files, one that does what the reader expects and one that does not, and get them to collide before either is published". That is far cheaper, and against MD5 and SHA-1 it is solved. What those algorithms still do perfectly well is catch accidents: a dropped byte, a truncated transfer, a disc that has begun to rot. None of those are trying to collide with anything, and any hash notices a random change.

So the distinction is this. Collision resistance is about a forger and error detection is about physics. If a page hands you an MD5 and no SHA-256, it is telling you it expects to catch a broken packet and does not expect to be attacked.

The hex column is the bit count divided by four, so it is entirely predictable from the column before it. CRC-32 is in the table as calibration: it appears in file names and error messages constantly, it is excellent at spotting a corrupted transfer, and it offers no protection whatsoever against someone who wants two different files to share a checksum.

Reading a checksum line

Most checksum files are plain text, one entry per line: the digest, then a separator, then the filename. Two spaces means text mode and a space followed by an asterisk means binary mode, which on any modern system are the same thing.

Before comparing anything, count the characters, which tells you which algorithm produced the line. Then check the filename: a checksum file covering several builds for several architectures is normal, and comparing your download against the line for a different build is the most common false alarm there is. Then normalise the case, because hex compares case-insensitively and a page printing A9 where your terminal printed a9 has not found you a problem.

Running the check on your machine

Every mainstream system already ships a hash tool. On Windows that is certutil, which prints the digest in lowercase on its own line and then a confirmation line:

certutil -hashfile example-1.4.tar.gz SHA256

In PowerShell, Get-FileHash returns an object rather than a line of text: Get-FileHash .\example-1.4.tar.gz -Algorithm SHA256. On macOS, shasum has been in the base system for years and the algorithm goes after the flag:

shasum -a 256 example-1.4.tar.gz

On Linux it is sha256sum, from coreutils. All of these will also read a checksum file and check every entry in it:

sha256sum -c example-1.4.tar.gz.sha256

That last form is the one worth remembering, because it does the comparison in software rather than in your visual cortex. Reading sixty-four characters by eye is a task humans are bad at, and the failure is not carelessness: it is that you read the first eight characters, recognise them, feel satisfied, and stop. The Hash Generator will produce the same values from a file you drop on it, if you want to see them from a second implementation.

A mismatch, character by character

Here is a published digest — the real SHA-256 of the three-byte input abc — set against a digest with one character changed, which I have changed on purpose to show you what the comparison looks like:

published  ba7816bf 8f01cfea 414140de 5dae2223 b00361a3 96177a9c b410ff61 f20015ad
computed   ba7816bf 8f01cfea 414140de 5dae2223 b00361a3 96177a9c b410ff61 f20015ae
                                                                                 ^

Sixty-four characters, and the first sixty-three are identical. The difference is at position sixty-four, where the published value ends in d and the computed one ends in e. Broken into the eight-character blocks shown above, seven agree completely and the last one does not:

That is not what a corrupted download looks like. A digest that matches almost perfectly is a copying problem: a transcription slip, a truncated paste, a checksum file with something appended to the end of the line. A digest computed over a file that genuinely differs from the publisher's is not ninety-eight per cent the same string. It is unrecognisable, with about half its characters different, because of the avalanche behaviour described earlier. The shape of the difference diagnoses the problem before you know anything else about it.

What a mismatch means

A mismatch is information, not a verdict. Work through the causes in order of how boring they are, because the boring ones are the common ones.

1. You hashed the wrong file. The path was relative and you were in a different directory, or you hashed the archive when the digest is for the installer inside it, or the temporary file still has .part on the end of its name.
2. You compared against the wrong line. One checksum file covering releases for several architectures is normal, and the digest for the ARM build will not match the x86 build you downloaded. Neither of them is compromised.
3. The transfer was damaged. An interrupted download, a proxy that rewrote the body, an FTP session in the wrong mode. Fetch it again, from a different mirror if you can, and hash that.
4. The file changed after publication. A CDN serving a stale object, a build re-rolled without regenerating the checksum file, a package rebuilt for a reproducible-build fix. The published digest is the wrong one rather than your file being wrong.
5. Someone served you something else. The last conclusion to reach, not the first, and the one you cannot rule out by hashing harder — if the digest came from the same place as the file, every case above looks identical from where you are sitting.

One level up: signatures

The flaw in all of the above is that a checksum has to reach you somehow, and the internet is not a trustworthy courier. Instead of publishing a digest beside the file, a publisher can sign the file, or sign the checksum file, with a private key, and you verify with the matching public key. The hash does the same job it did before; what the signature adds is identity, because that digest could only have been produced by somebody holding the private key. With GnuPG it is one command against a detached signature that came alongside the download:

gpg --verify example-1.4.tar.gz.sig example-1.4.tar.gz

What comes back is confirmation that the signature is good and, more importantly, the fingerprint of the key that made it. That fingerprint is the entire trust story. If you have not checked it against something other than the page you downloaded from, you have moved the problem rather than solved it, because whoever can serve you a file can serve you a matching public key. Which is why a key you already hold is the only kind that helps: get it once, confirm it against the project's own documentation or a message from a maintainer, and keep it. Smaller projects sometimes use minisign or OpenBSD's signify instead, whose public keys are short enough to paste into a README, which is the part that makes verification realistic.

The rule to keep

A checksum answers one question: are these the bytes that were hashed. Whether the answer is worth anything depends entirely on where the digest came from. Same place as the file, and you have confirmed that your download finished, which is worth doing and is not a security check. Different place, and you have confirmed the file matches what the publisher had, which is the claim you wanted to check in the first place.

So get the digest from somewhere else before you compare, and make the machine do the comparing rather than your eyes. When a check fails, resist the story: almost every mismatch is a wrong path, a wrong line, or a half-finished transfer, and the way to find out which is to fix the boring possibility and hash again. If it still fails after you have re-downloaded from another mirror and confirmed you are on the right line, then you have something worth telling people about, and you will know that because you ruled out everything else first.