Fixing Dynamic Routing Bug in Next.js 13.5.4: The URL Encoding Issue

In Next.js 13.5.4, a crucial bug(Maybe just for me..) related to dynamic routing was fixed. This bug involved handling URL encoding and decoding, particularly when proxy servers or web servers decode URLs before passing them to the backend. I’d share what I’ve been experiencing with the above issue and what caused it.

Background

Web applications often receive requests through proxy servers like Nginx or Apache. These proxies can decode URLs before sending them to the backend server. However, inconsistencies in handling URL encoding and decoding can cause several issues:

• Proxy Server and Web Server Mismatch: When a proxy server decodes a URL, the backend server handles the decoded URL. If the backend only checks the decoded URL, it might miss the originally encoded URL sent by the client.
• Multiple Decoding Problems: URLs can be encoded and decoded multiple times through different stages, leading to issues if the server doesn’t handle nested encodings correctly.
• File System Path Mismatch: The file system path might differ based on encoding, causing the server to fail to locate the correct file.
• Security Concerns: Improper handling of encoded special characters can lead to security vulnerabilities.

What I encountered

I encountered an issue after upgrading from Next.js 13.4.1 to 13.5.1:

• Docker Compose Environment: Dynamic routing worked perfectly when I deployed the application directly using Docker Compose.
• Kubernetes Environment: However, when the application was deployed in a Kubernetes (K8s) environment behind an Nginx proxy, dynamic routing failed to work.

While troubleshooting this issue, I discovered the bug related to URL encoding and decoding.

Root Cause

The issue was primarily due to how URLs were handled in the code. Here’s a snippet of the problematic code:

if (!items.has(curItemPath) && !opts.dev) {
  curItemPath = curDecodedItemPath
}
const matchedItem = items.has(curItemPath)
if (matchedItem || opts.dev) {
  ...

In this code, only the decoded path was checked, and there was no handling for the encoded path. As a result, if the proxy server decoded the URL, the encoded path was missed, causing routing issues.

The Fix

https://github.com/vercel/next.js/pull/56187/files

In Next.js 13.5.4, the fix involved adding logic to check both the encoded and decoded paths. Here’s the revised code:

let matchedItem = items.has(curItemPath)

if (!matchedItem && !opts.dev) {
  matchedItem = items.has(curItemPath)
  if (matchedItem) curItemPath = curDecodedItemPath
  else {
    try {
      const encodedCurItemPath = encodeURI(curItemPath)
      matchedItem = items.has(encodedCurItemPath)
    } catch {}
  }
}

With this fix, Next.js now follows these steps:

1. Check Decoded Path: First, it checks the decoded path.
2. Check Encoded Path: If the decoded path doesn’t match, it checks the encoded path.
3. Recheck with Encoded Path: Finally, it encodes the path again and checks once more.

This ensures that whether the proxy server decodes the URL or not, both the encoded and decoded paths are handled correctly.

Conclusion

Next.js 13.5.4 addresses a significant bug in handling URL encoding for dynamic routing. This fix ensures that both encoded and decoded paths are properly checked, making the framework more robust in various environments, especially when dealing with proxy servers.

I hope this detailed explanation helps you understand the URL encoding issue and how it was resolved in Next.js 13.5.4.

Note for Custom Server Users

Please note that the render-server-standalone file has been removed since version 13.4.13. If you are using a custom server, be aware of this change. 🥲