ABC Tools

String Escaping for JSON, Shells and Regex

After reading this you will know exactly which characters need escaping in a JSON string literal, a POSIX shell argument, and a regular expression, and why the three rules differ.

Escaping means rewriting text so that a parser reads it as data rather than as syntax. The same string needs three different treatments depending on who reads it next. Consider the value O'Reilly "guide" \n. Fed to three contexts, it becomes three different things:

One input, three escapes
ContextEscaped result
JSON string"O'Reilly \"guide\" \\n"
Shell argument'O'\''Reilly "guide" \n'
Regex literalO'Reilly "guide" \\n

Notice that the double quote is dangerous in JSON but harmless in the shell single-quote form, and the single quote is the opposite. Each context has its own set of characters that carry meaning, and escaping neutralises exactly that set and no more.

When to escape, and when the language already does it

Reach for manual escaping when you are building text by hand or by string concatenation: writing a JSON fixture, pasting a value into a shell command, or turning a fixed keyword into a regex pattern. In those moments there is no library between you and the parser.

Do not escape when a proper API already handles it. If you call JSON.stringify(value) in JavaScript or json.dumps in Python, the encoder escapes the string for you, and escaping again produces doubled backslashes. The same holds for parameterised SQL queries and for shell libraries that take an argument list (Python's subprocess.run(["grep", pattern]) passes each element without a shell, so no shell escaping is needed at all).

The classic bug is double escaping. If a value passes through two encoders, \n becomes \\n and the newline is lost. Escape once, at the boundary where the text enters a new context, and never inside code that a real encoder already covers.

The three rules stated precisely

Each context replaces a fixed list of characters. Learn the list and the whole scheme becomes mechanical.

JSON string (RFC 8259, section 7)
Inside a double-quoted literal you must escape the quotation mark ", the backslash \, and every control character U+0000 through U+001F. Common controls get short forms: \n, \t, \r, \b, \f. Any other control uses \u followed by four hex digits, for example \u0007 for the bell.
POSIX shell single quoting
Inside single quotes the shell treats every byte literally, with one exception: a single quote itself cannot appear. So you close the quote, emit an escaped quote, and reopen. The single quote ' becomes the four-character sequence '\''. Nothing else changes.
Regex literal
To make text match itself, backslash-protect the 14 metacharacters that otherwise carry meaning: . ^ $ * + ? ( ) [ ] { } | \. Every other character already matches literally.

The character counts are worth memorising: JSON escapes 2 fixed characters plus 32 control codes, the shell escapes exactly 1 character, and regex escapes 14. The shell rule looks strange because it works by leaving and re-entering the quoted region rather than by prefixing a backslash.

Why does regex need backslashes but the shell single-quote form does not? Because a regex has no surrounding delimiter that turns off all special meaning. The shell single quote is a hard wall: everything inside is literal except the wall character. A regex has no such wall, so each metacharacter must be disarmed on its own.

A worked example on the demo data

Escaping O'Reilly "guide" \n

Take the literal 18-character string O'Reilly "guide" \n, where the last two characters are a backslash and the letter n (not a newline). Trace each context by hand.

  1. JSON. The double quotes each become \". The single backslash before n becomes \\. The single quote and letters are untouched. Wrap the whole thing in quotes: "O'Reilly \"guide\" \\n". Character count grows from 18 to 24 inside the quotes.
  2. Shell. Wrap in single quotes, then rewrite the one embedded single quote. The ' after O becomes '\''. Result: 'O'\''Reilly "guide" \n'. The double quotes and backslash are safe inside single quotes, so they pass through unchanged.
  3. Regex. Only two characters here are metacharacters: nothing, actually, except the backslash. The parentheses, dots and quotes in this particular string happen not to appear, but the backslash does, so it becomes \\. Result: O'Reilly "guide" \\n. That pattern matches the original text exactly.

Reproduce all three at once by clicking the demo button on the tool; the field defaults feed this same string through every context.

The chart below counts how many characters each context adds for this input. Shell adds 3 (the extra \'' around one quote plus the two wrapping quotes counted separately gives the net growth shown), JSON adds 3 escape characters, and regex adds 1.

How much each escaping scheme inflates this particular input. A string full of parentheses would push the regex bar much higher.

Reading the output and reversing it

Unescaping is the inverse, and it is where mistakes hide. For JSON, a lone backslash that is not part of a known escape is invalid input, not a literal backslash. So "\d" is not a valid JSON string; you must write "\\d" to store the two characters backslash-d. This trips up anyone storing regex patterns in JSON config files.

For the shell, unescaping means running the value through the shell's own quote removal. The sequence '\'' collapses back to a single quote. If you see \n survive as backslash-n rather than a newline, that is correct: single quotes never interpret backslash escapes. Only double quotes and the $'...' form do.

To store a regex inside a JSON file, escape for regex first, then escape that result for JSON. The pattern \d+ becomes \\d+ for JSON, so the file contains "\\d+". When the program parses the JSON it recovers \d+, which the regex engine then reads correctly.

Watch the layering yourself

The pattern \d+ is 3 characters. Escaped for regex it stays \d+ only if you treat it as already-regex; escaped for JSON it becomes \\d+ (5 characters); wrapped in shell single quotes it becomes '\d+' (5 characters). Stacking JSON then shell gives '\\d+' (6 characters).

Common mistakes

Four errors account for most escaping bugs.

  • Using double quotes in the shell for arbitrary text. Double quotes still interpret $, `, \ and !. A value containing $HOME expands to your home directory. Single quoting with the '\'' rewrite is the only form safe for any input.
  • Forgetting that JSON has no \' escape. RFC 8259 defines \" but not \'. A single quote inside a JSON string needs no escaping at all; writing \' is a parse error.
  • Over-escaping regex. Escaping a letter like \w when you meant literal w changes the meaning entirely. Only the 14 metacharacters need protection.
  • Escaping data that a library already escapes. Covered above, and worth repeating because it silently corrupts newlines and tabs.

Related tools

Once your JSON string is escaped correctly, check the whole document with the JSON Formatter & Validator, or compare two versions with JSON Diff. If you are moving structured data between formats, the data converter handles JSON, YAML, TOML, XML and CSV. Build and test the pattern before you escape it with the Regex Tester. For query text, the SQL Formatter tidies statements, and the .gitignore Generator covers a different daily chore.

Frequently asked questions

Why does the shell wrap everything in single quotes instead of using backslashes?

Because single quotes disable every special character at once. A backslash-per-character scheme would need to escape spaces, dollar signs, semicolons and more, and it is easy to miss one. Single quoting is a hard wall: only the quote character itself needs the '\'' workaround.

Does JSON require escaping the forward slash?

No. RFC 8259 allows \/ as an optional escape, but a bare / is equally valid. Most encoders leave it alone. You will see \/ mainly in HTML-embedded JSON to avoid the sequence </script>.

How do I put a literal backslash in a regex?

Write two backslashes: \\. To match a Windows path like C:\Users you need the pattern C:\\Users, and if that pattern lives in a JSON file you escape again to "C:\\\\Users".

Is escaping the same as encoding?

They overlap but differ. Escaping keeps the same character set and adds delimiters or backslashes so a parser reads data as data. Encoding like percent-encoding or base64 transforms bytes into a restricted alphabet. This tool does escaping, not encoding.

Which characters does regex escaping cover?

Exactly 14: . ^ $ * + ? ( ) [ ] { } | \. Every other character already matches itself, so escaping it is unnecessary and sometimes wrong (\d means digit, not the letter d).

Try it yourself in the tool

Related tools

JSON Formatter & Validator

Format, validate, minify and query JSON with JSONPath.

JSON Schema Validator

Validate JSON data against a JSON Schema.

JSON ↔ YAML ↔ TOML ↔ XML ↔ CSV

Convert between the five common data formats.

Regex Tester

Test regular expressions live, with a plain-English explanation.

SQL Formatter

Pretty-print and minify SQL for any major dialect.

.gitignore Generator

Build a .gitignore from language, IDE, and OS templates with deduplication, stack presets, and share links.