How to Encode and Decode URLs: RFC 3986 Percent-Encoding, URLSearchParams & Polyglot Recipes
Complete guide to URL percent-encoding and decoding: RFC 3986 rules, query string construction, multi-byte UTF-8 handling, and code recipes in JS, Python, Go, and Shell.
What is URL Percent-Encoding?
Uniform Resource Identifiers (URIs) are constrained by RFC 3986 to a small subset of US-ASCII characters. Any character outside this permitted set—including spaces, control codes, non-ASCII Unicode characters (such as emojis or accented characters), and reserved delimiter symbols used out of context—must be percent-encoded.
Percent-encoding replaces each prohibited byte with a triplet consisting of the percent symbol (%) followed by two hexadecimal digits representing the byte value in base-16 (e.g. ASCII space 0x20 becomes %20, and & 0x26 becomes %26).
RFC 3986: Reserved vs Unreserved Characters
| Classification | Characters | Encoding Rule |
|---|---|---|
| Unreserved Characters | `A-Z`, `a-z`, `0-9`, `-`, `_`, `.`, `~` | Never encoded. Safe anywhere in a URI. |
| Reserved (Gen-delims) | `:`, `/`, `?`, `#`, `[`, `]`, `@` | Delimit URI scheme, authority, path, query, and fragment. Must be encoded when used as data. |
| Reserved (Sub-delims) | `!`, `$`, `&`, `'`, `(`, `)`, `*`, `+`, `,`, `;`, `=` | Delimit sub-components like query string key-value pairs. |
| Unsafe Characters | Spaces, control characters (`0x00-0x1F`), quotes, angle brackets | Must always be percent-encoded. |
Multi-Byte UTF-8 Percent-Encoding Explained
When encoding Unicode characters beyond standard ASCII (code point > 127), modern web standards require encoding the string into raw UTF-8 octets first, then percent-encoding each individual byte:
Character: 'é' (U+00E9)
UTF-8 Representation: 2 bytes -> 0xC3 0xA9
Percent-Encoded Result: "%C3%A9"
Character: '🚀' (U+1F680 Rocket Emoji)
UTF-8 Representation: 4 bytes -> 0xF0 0x9F 0x9A 0x80
Percent-Encoded Result: "%F0%9F%9A%80"URL Encoding & Decoding in JS, Python, Go & Bash
// JavaScript / TypeScript (Browser & Node.js)
const rawParam = 'hello world & CoShareX';
const encoded = encodeURIComponent(rawParam);
console.log(encoded); // "hello%20world%20%26%20CoShareX"
const decoded = decodeURIComponent(encoded);
console.log(decoded); // "hello world & CoShareX"# Python 3
import urllib.parse
raw_param = "hello world & CoShareX"
encoded = urllib.parse.quote(raw_param, safe="")
print(encoded) # "hello%20world%20%26%20CoShareX"
decoded = urllib.parse.unquote(encoded)
print(decoded) # "hello world & CoShareX"// Go (Golang)
package main
import (
"fmt"
"net/url"
)
func main() {
rawParam := "hello world & CoShareX"
encoded := url.QueryEscape(rawParam)
fmt.Println(encoded) // "hello+world+%26+CoShareX"
decoded, _ := url.QueryUnescape(encoded)
fmt.Println(decoded) // "hello world & CoShareX"
}# Linux / macOS Shell (jq or python one-liner)
echo -n "hello world & CoShareX" | jq -sRr @uri
# Output: hello%20world%20%26%20CoShareXHandling Edge Cases & Decode Errors
When decoding user-submitted URLs, applications must defend against malformed byte sequences and incomplete percent sequences:
export function safeUrlDecode(input: string): string {
try {
return decodeURIComponent(input.replace(/\+/g, ' '));
} catch (err) {
// Falls back gracefully on malformed percent sequences like '%E0%A4'
console.warn('Malformed URI sequence encountered, returning raw input:', err);
return input;
}
}Code Formatter & Converter Suite
Validate, format, minify, and convert JSON, SQL, YAML, XML, and code dialects directly on your local machine.
Frequently Asked Questions
What happens if a URL contains unencoded spaces?
Unencoded spaces are invalid in URIs. HTTP clients or browsers may attempt to auto-correct them to `%20` or `+`, but proxies and load balancers may reject the request with a 400 Bad Request error.
How do I encode an entire URL while preserving query parameters?
Do not encode the entire URL with `encodeURIComponent`. Use the standard `URL` constructor or parse query parameters into `URLSearchParams`, which guarantees that path slashes, protocol headers, and query keys are encoded with exact RFC 3986 precision.
Conclusion
For robust URL handling, construct query parameters using native URLSearchParams APIs and ensure that individual parameter keys and values are encoded with encodeURIComponent before assembly.
Related Articles
encodeURI vs encodeURIComponent: Key Differences, Character Sets, and Pitfalls
A definitive comparison of encodeURI and encodeURIComponent in JavaScript: character escape rules, query parameter handling, and modern URLSearchParams patterns.
Developer ToolsHow to Validate JSON: Syntax Diagnostics, JSON Schema (Draft 2020-12), and Runtime Validation
Learn how to validate JSON data: debugging syntax errors, defining JSON Schema (Draft 2020-12) contracts, and validating runtime payloads with Ajv, Zod, and Pydantic.
ConvertersWhat is Base64? Binary-to-Text Encoding Explained from Bitwise Roots
A deep architectural guide to Base64 encoding: 6-bit binary chunking, ASCII translation tables, padding mathematics, and URL-safe variants.