RSA Encryption with OAEP, Explained

After reading this you will know what RSA encryption actually does to your bytes, why it can only encrypt short messages, why OAEP padding matters, and how real systems use RSA to wrap an AES key rather than encrypt data directly.

What RSA encryption does

RSA is a public-key cipher. You publish one key (the public key) and keep the other secret (the private key). Anyone can encrypt a message with your public key, but only you can decrypt it with the private key. The two keys are mathematically linked, yet deriving the private key from the public key requires factoring a large number, which is infeasible for a well-chosen 2048-bit or larger modulus.

Here is the hook. Suppose you want to send a 32-byte AES key to a server. You take the server's 2048-bit public key, run RSA-OAEP encryption, and get a 256-byte ciphertext (Base64: about 344 characters). The server decrypts it with its private key and recovers your exact 32 bytes. No shared secret was ever transmitted in the clear. That is the whole point of public-key cryptography.

The tool on this page does exactly that operation. It takes a PEM public key and a short plaintext, applies RSA-OAEP, and returns Base64 ciphertext. Paste the matching private key to reverse it.

When to use it, and when not to

Use RSA encryption to protect a small, high-value secret when the sender only has the recipient's public key. The classic case is key wrapping: encrypt a random AES key with RSA, then encrypt your real data (which may be gigabytes) with that AES key. This is called a hybrid scheme, and it is what TLS, PGP, S/MIME and age-style tools all do internally.

Do not use RSA to encrypt bulk data directly. It is slow and size-limited. A 2048-bit key can encrypt at most about 190 bytes with OAEP, and each encryption is roughly a thousand times slower than an AES block. If you need to encrypt a file, encrypt a 32-byte AES-256 key with RSA and encrypt the file with AES-GCM.

The math, with intuition

An RSA key is built from two large primes, p and q. Their product is the modulus:

Here n is the public modulus, and its bit length is what people mean by "2048-bit RSA". The public exponent e is almost always 65537. The private exponent d is chosen so that e \cdot d \equiv 1 \pmod{\lambda(n)}, where \lambda(n) is the Carmichael totient of n.

Raw (textbook) encryption of an integer message m is a single modular exponentiation:

Decryption undoes it with the private exponent:

The symbols: m is the message as a number less than n, c is the ciphertext, e and d are the public and private exponents, and n is the modulus. It works because m^{e \cdot d} \equiv m \pmod{n} by Euler's theorem.

Why textbook RSA is not enough, and what OAEP adds

The formulas above are deterministic. Encrypting the same message twice gives the same ciphertext, so an attacker can detect repeats and even build a dictionary for small message spaces. Textbook RSA is also malleable: if c encrypts m, then c \cdot 2^{e} \bmod n encrypts 2m. That is a real attack, not a curiosity.

OAEP (Optimal Asymmetric Encryption Padding, defined in RFC 8017, PKCS#1 v2.2) fixes this. Before the exponentiation, it expands your message with a random seed and two applications of a mask generation function built on a hash (typically SHA-256). The result is a randomized, full-width block. Two consequences follow:

• Encrypting the same 32 bytes twice gives two different ciphertexts, because a fresh random seed is mixed in each time.
• Any tampering with the ciphertext makes the padding check fail on decryption, so malleability attacks do not survive.

The size limit, made concrete

OAEP eats space. With SHA-256, the padding overhead is 2 \cdot h + 2 bytes, where h = 32 is the hash output length. So the maximum plaintext you can encrypt is:

Here k is the modulus size in bytes. For 2048-bit RSA, k = 256, so m_{\max} = 256 - 64 - 2 = 190 bytes. The table shows the limit at common key sizes.

Notice the ciphertext is always exactly k bytes regardless of message length, because it is a residue mod n. A 32-byte AES key and a 190-byte message both produce 256 bytes of ciphertext under a 2048-bit key.

Reading and interpreting the results

The Base64 ciphertext is not meant to be human-readable and carries no structure you can inspect. What you can verify:

Length

The decoded ciphertext is always exactly the modulus size, 256 bytes for a 2048-bit key. A different length means a different key size or a corrupted value.

Nondeterminism

Encrypting the same input twice must give different ciphertext. If it does not, the padding is broken or missing.

Round trip

Decrypting with the matching private key returns your exact input bytes. A key mismatch or corrupted ciphertext produces a decryption error, not partial output.

Common mistakes

Most RSA failures are not broken math. They are misuse.

• Encrypting large data directly. If you hit a "data too large" error, you tried to encrypt more than k - 2h - 2 bytes. Switch to hybrid encryption: RSA-wrap an AES key, then AES-encrypt the payload.
• Mismatched OAEP parameters. Encryption with SHA-256 OAEP will not decrypt if the other side expects SHA-1 OAEP or PKCS#1 v1.5. The hash and padding scheme must match on both ends.
• Using the wrong key. Encrypt with the public key, decrypt with the private key. Reversing them is signing, a different operation with different padding (RSA-PSS).
• Reusing RSA for authenticity. Successful decryption does not authenticate the sender. Pair encryption with a signature if you need to know who sent the message.
• Trusting small keys. 1024-bit RSA is deprecated. NIST SP 800-57 sets 2048 bits as the minimum for use through the near term and recommends 3072 bits for longer protection.

Related tools

To create the keys this tool consumes, use the RSA Key Pair Generator, which outputs 2048/3072/4096-bit keys in PEM format. To prove authorship rather than hide content, use RSA Sign & Verify with RSA-PSS. If you want smaller keys and faster operations for signatures, ECDSA Keys, Sign & Verify covers the elliptic-curve alternative on P-256, P-384 and P-521.

Related tools

RSA Key Pair Generator

Generate 2048/3072/4096-bit RSA keys in PEM format.

ECDSA Keys, Sign & Verify

Elliptic-curve key pairs and signatures (P-256/384/521).

Elliptic Curve Visualizer

See curve addition, scalar multiples and a toy ECDH exchange — over the reals and over F_p.

RSA Sign & Verify

Sign a message with RSA-PSS and verify the signature.

AES Encryption

Encrypt and decrypt text with AES-GCM or AES-CBC.

Cryptographic Key Generator

Random keys, IVs, salts and API tokens from a CSPRNG.