Reading a URL query string without guessing
A query string is key=value pairs joined by ampersands, and every value is percent-encoded. Here is how to decode one and read it without guessing.
A newsletter lands in your inbox with a link a hundred and fifty characters long, or a colleague forwards an endpoint that worked yesterday, and what you have is a string where every third character is a percent sign. Somewhere inside it there is a campaign name, a redirect target, a page number and a signature. The work is telling those apart on purpose rather than by trial and error, and it starts with knowing that a URL has named parts at all.
Here is one, close to dozens I have had to pull apart:
https://shop.example.com/orders/8812?utm_source=newsletter&utm_campaign=spring%20sale&next=%2Faccount%2Forders%3Fpage%3D2&tag=a&tag=b&ts=1749030000&sig=9f2c41ab7e
Nothing there needs decoding before you can say what it is. The path is /orders/8812, so it is a single order. The query carries the referring campaign, a destination hiding behind %2F, a repeated tag, a timestamp, and a signature that covers the rest of it. That read takes about ten seconds, and it is only possible because each piece of the URL has a name and a job.
A URL is five named parts
The general form is scheme, authority, path, query and fragment. RFC 3986 calls them components and gives each one its own grammar, which is why a URL can be taken apart mechanically instead of by eye. The authority is itself three fields stacked together — userinfo, host and port — and the userinfo half is the one nobody expects: https://user:pass@host/ is a legal URL, and a poor home for a credential, because it lands in logs, bookmarks and shell history wherever it goes.
| Part | Starts at | In the example | Reaches the server |
|---|---|---|---|
| scheme | the first colon | https | Yes |
| authority | the two slashes | shop.example.com, port 443 by default | Yes |
| path | the first slash after the host | /orders/8812 | Yes |
| query | the first question mark | utm_source=newsletter, and the rest of the pairs | Yes |
| fragment | the first hash | nothing in this URL | Never |
The last column is where most of the debugging happens. Everything from the scheme down to the query travels in the request; the fragment stays with the browser, and knowing which side of that line a value sits on answers a large share of the questions people bring to a URL.
The fragment is never sent
Everything from the first hash onward is removed by the browser before the request is written. It is not on the request line, it is not in the headers, and a server that has never seen the URL cannot know it existed. The request line for the order page above is exactly this, and no more:
GET /orders/8812?utm_source=newsletter&utm_campaign=spring%20sale&next=%2Faccount... HTTP/1.1 Host: shop.example.com
The fragment exists so a page can point at a position inside a document: the browser fetches the document, finds the element whose identifier matches the value, and scrolls to it without asking the server anything. Because the browser owns that value and can rewrite it with no round trip, a script in the page can use it as storage too, which is what hash routing does.
The trap is what follows. A key, a filter, a page number or a token placed after the hash reaches the backend as nothing: the handler sees the path and the query and has no idea the value was ever there. So when a parameter looks like it is being ignored — the page loads, the request returns 200, and the value has no effect anywhere — check which side of the hash it sits on, then check whether the value contains an unencoded hash of its own. That failure is quieter still: the parameter keeps its old value and everything after the stray hash is gone.
The query string nobody agreed on
The query is key=value pairs joined by an ampersand. That much everyone does. What the specification never settled is which characters are reserved inside which component. RFC 3986 defines the query as a run of characters from a permitted set and then leaves the internal structure to whoever writes the client and whoever writes the server. Most of the rules in daily use come from HTML form submission rather than from the URL specification, which is why an encoder usually offers two modes: encode one component, or encode a whole URL and leave its delimiters alone.
The conventions that actually bite:
- The question mark starts the query. A second question mark is an ordinary character inside a value, and most parsers treat it as one.
- The ampersand separates pairs. An ampersand inside a value has to be written %26 or it splits the pair in two.
- The equals sign separates a key from its value, and most parsers split on the first one only, which is why a value is allowed to contain more of them.
- A plus sign means a space in a query string and a literal plus in a path.
- %20 means a space on both sides of the question mark.
That fourth rule is the one that costs an afternoon. A file called My Report+Notes.csv lives at a path where the plus really is a plus. The same name inside a query parameter has to be sent as My%20Report%2BNotes.csv — the space as %20 and the plus escaped as %2B. Make the mistake one way and a query that was meant to carry a plus arrives as a space, which is visible. Make it the other way, putting a literal plus in a path where the file system expects the escaped form, and you get a 404 for a file that is plainly sitting there in the directory listing.
Percent-encoding works one byte at a time
The escape is a percent sign followed by two hexadecimal digits giving the value of one byte. That is the whole mechanism. Any byte can be written that way, so it covers the whole of UTF-8 without needing a table of characters: the encoder writes bytes and the decoder reads them back. The characters worth having in your head are the ones you meet constantly.
| Character | In a path | In a query | Why it matters |
|---|---|---|---|
| space | %20 | %20, or + for a space | Not legal raw in either; the plus form means a space only in a query |
| + | + as itself | %2B for a literal plus | The one character whose meaning changes across the question mark |
| & | & as itself | %26 | Unescaped in a query it ends the pair and starts the next one |
| = | = as itself | %3D when it is data | Splits key from value; most parsers split on the first one only |
| # | %23 | %23 | Never legal raw; it starts the fragment and the query stops there |
| % | %25 | %25 | The escape character itself, so a literal one must be escaped |
| / | / as a separator, %2F inside a segment | %2F | A slash inside a query value is data, but unescaped it looks like a path |
| ? | %3F | %3F | Starts the query wherever it appears raw, including mid-path |
| é | %C3%A9 | %C3%A9 | Two UTF-8 bytes, so two escape groups rather than one |
Read the middle two columns against each other. They agree about the percent sign, about the accented character and about the space, and disagree about nearly everything else. The plus row is the expensive one: the same character is a space on one side of the question mark and a literal plus on the other, and nothing in the URL tells you which reading the far end chose.
The accented character is the row to remember. é is U+00E9, and its UTF-8 encoding is two bytes, C3 and A9, so it travels as %C3%A9. A single escape, %E9, is the same character in Latin-1, and it is what a tool that encodes from the wrong character set will produce. When a server reads those bytes as Latin-1 while you wrote them as UTF-8, you get é — the two characters you see when C3 and A9 are each decoded on their own. That pair is the signature of a charset mismatch, and it is worth recognising on sight, because usually nothing has been lost. The bytes are correct and only the interpretation is wrong.
The percent sign itself is the landmine. It is the escape character, so a value that contains one has to send it as %25. Skip that and one of two things happens. If the two characters after the percent are valid hex digits they are decoded as a byte, and the text the reader typed is not the text that arrives: a code written into a page as 100%2B5 decodes to 100+5 at the far end, and nobody notices until the redemption fails. If they are not valid hex, the behaviour is up to the parser — some raise an error, some pass the percent through untouched, some drop it and the two characters behind it. The tell for this one is a value that looks almost right, one character different somewhere in the middle.
Lists have no standard form
There is no agreed way to put a list of values in a query string. The specification does not define one, and every framework solved it independently. All of the following are in the wild, all of them are correct within the system that produced them, and none of them is a mistake:
- ?tag=a&tag=b — the repeated key, which is what a plain parser usually expects.
- ?tag[]=a&tag[]=b — the bracket form, PHP's convention, copied since by a long list of frameworks and by plenty of services that are not PHP at all.
- ?tag=a,b — comma-joined, the default in OpenAPI, needing %2C when a value contains a comma of its own.
- ?tag[0]=a&tag[1]=b — the indexed form, read as an array by some parsers and as two keys literally named tag[0] and tag[1] by others.
The failure mode is not an error message. A parser that expects a single value and meets a repeated key will keep the last one and discard the rest, so ?tag=a&tag=b arrives as b and nothing anywhere reports that a value went missing. The same silence runs the other way: a server expecting brackets receives tag[] as a key whose name contains square brackets, looks up tag, finds nothing, and treats the parameter as absent. In both directions the request succeeds and the answer is wrong.
What to do when you cannot tell
Read the documentation. When there is none, read the code. When there is neither, ask the response: send the list one way, then send the same list the other way with values that are easy to count, and compare what comes back. If the endpoint echoes its parsed parameters anywhere — a filter summary, an error message, a debug header — that echo is the definitive answer and it costs one request.
Order stops being harmless when a signature covers it
For a plain endpoint, ?a=1&b=2 and ?b=2&a=1 are the same request. Nothing reads a query in order, because a query is a mapping and a mapping has no order. Now ask a signed URL. Presigned object storage links, webhook signatures and CDN tokens are computed over the query as it was written, and the check on the receiving end recomputes them over the query as it arrived. If anything between the two re-sorts the parameters, or adds an empty pair, or drops a redundant one, every value is still correct and the signature is wrong.
That is why an API rejecting a request for an invalid signature is so often not complaining about a wrong key. Reordering is enough by itself, and so is a proxy that normalises an escaped tilde to a raw one or sorts the keys on the way through. When you are debugging one of these, compare the raw bytes of the query, in the order they appear, rather than the parameters you believe you sent. Signing libraries usually expose the exact string they signed, for precisely this reason.
Case is the other half of it. Key names are case-sensitive: ?UserId=7 and ?userid=7 are two different keys, and a framework that folds them together is doing you a favour you did not ask for. Percent-encoding runs the other way, since %2f and %2F are the same byte and any decoder accepts both. A signature computed over the lower-case form and verified against the upper-case one will fail with no visible difference anywhere in the URL, which is the sort of mismatch that costs an hour if you do not know to look for it.
How to read one properly
Decode the query into a table, then read the table rather than the string. Reading the raw text works for short URLs and falls apart on long ones, because the eye cannot reliably pair a key with a value when several characters sit between them and the separator is a character that also occurs encoded twice inside the value itself.
The order link above decodes to utm_source, utm_campaign, next, tag, ts and sig. The utm_ keys are campaign tracking, worth knowing about and rarely worth reading. ts and sig are the pair that makes the link expire, and they are the reason a URL that worked this morning returns an error this afternoon. The keys a reader actually wants are next, which is where the click is going, and tag, which is the list.
A long tracking URL is not a puzzle to solve completely. Find the keys that carry meaning, decide what the rest are for, and move on. Nested values are the other thing to watch. If a decoded value still contains a percent sign, it was encoded twice and the inner value needs a pass of its own. next=%252Faccount decodes once to %2Faccount and once more to /account. A value that reads %7B after one decode is a JSON object that has not been unwrapped yet. Do that second decode and the string becomes this:
{"page": 3, "sort": "desc"}Which is a page number and a sort direction, not a mystery. The rule I use is to decode until the value stops containing escapes, then read it as whatever it now looks like — a URL, a JSON document, a timestamp, a serialised form field.
Where the tools fit. The URL Encoder does the first pass: paste a query in and it comes back as a table of keys and decoded values, so a doubly-encoded parameter shows up as a value that still has a percent sign in it rather than as something you have to notice by eye. It encodes the other way as well, which is how you check that a value survives a round trip before you put it in a request.
When the decoded value turns out to be JSON, the JSON Formatter is the second pass. It formats the blob, points at the line and column of any syntax error, and turns a string like the one above into something you can read at a glance instead of counting braces.
The habit worth keeping
Decode before you read, and decode as many times as the value needs. When a parameter appears to do nothing, check which side of the hash it sits on, then check whether the value contains an unescaped hash of its own. When a value arrives subtly wrong, look for a percent sign that was never escaped. When a signature fails, compare the query as bytes in order rather than as values. When a key repeats, find out which convention the endpoint uses instead of assuming the one your client sends.
None of that requires the specification memorised. It requires knowing that the query is a convention rather than a law, that the fragment belongs to the browser rather than the server, and that the percent signs are a byte encoding you can reverse by hand when you have to. Put the URL into a decoder, read the table, and trust the keys that mean something. Guessing stops being necessary once you can see what is actually in front of you.