Ermir Beqiraj
Backend architect. Systems, agents, infrastructure — from inside the work.
all writing

Issue! Duplicate without user-selected canonical

Oops… Google’s spider has been checking your site and found the above issue. That means at least:

  • You didn’t specify a canonical metatag or header
  • The same resource is reachable from more than one URI

Google elaborates more on this here and here.

So what’s the problem?

If the crawler finds that you don’t organize your resources per their Uniform Resource Locators spec, it may choose not to index your page.

That said, having a resource reachable from more than one URL can be legitimate. For example, consider a search on a commerce site:

  • /laptop/msi/the-id-here
  • /laptop/msi?memory=32
  • /laptop/msi?memory=32&color=black

The second and third queries might return a subset of the same products as the first. Which one is the true source? That’s what canonical URLs are for — you declare which URI is the definitive source, and the crawler respects it.

In my case the issue was simple: the same page rendered with or without a trailing slash. There’s an entire saga of to-slash-or-not-to-slash in Google docs.

The options

ASP.NET Core documentation approach

Microsoft docs covers this for IIS and Apache using the RewriteRules NuGet package (open source here).

A typical public app HTTP request goes through:

[client] → [api-gtw | cache] → [lb] → [reverse-proxy] → [application]

The sooner you let the client know about the new location, the better. I’m not using IIS or Apache, so this option doesn’t apply.

Configure Nginx to remove the trailing slash

If you’re using nginx and can modify its config, add a rewrite directive in the root location as the first rule:

location / {
    rewrite ^(/.*?)/?$ https://super-duper-domain.com$1 permanent;
}
  • Pros: Redirect happens before reaching the application
  • Cons: Nginx (infrastructure) needs to know about SEO rules
  • Gain: Saves the round-trip from nginx to application and back

In-code middleware

Performance-wise this is the worst option, but it’s the right choice when business logic is attached to the slash decision. For pure removal, ASP.NET Core middleware is clean and straightforward.

Using the RewriteRules package:

using Microsoft.AspNetCore.Rewrite;
using RewriteRules;

var app = builder.Build();

// other middlewares (e.g., forwarded headers) go before this

var options = new RewriteOptions().AddRedirectToCanonicalUrl(new CanonicalUrlOptions
{
    TrailingSlash = TrailingSlashAction.Remove
});

app.UseRewriter(options);

Or without any extra package — a tiny inline middleware:

var app = builder.Build();

// other middlewares go before this

app.Use(async (context, next) =>
{
    var path = context.Request.Path;
    if (path.HasValue && path.Value.Length > 1 && path.Value.EndsWith("/"))
    {
        var nonSlashPath = path.Value[..^1];
        if (context.Request.QueryString.HasValue)
            nonSlashPath += context.Request.QueryString.Value;

        context.Response.Redirect(nonSlashPath, permanent: true, preserveMethod: true);
        return;
    }

    await next();
});

My choice: CloudFront Functions

The only publicly accessible route to my app is through a CloudFront edge node. That makes CloudFront Functions the best candidate.

Cons:

  • Moves app routing concern to the infrastructure level

Pros:

  • Runs at the edge — the nearest network node capable of issuing the redirect
  • Serverless, scales to 10M+ requests per second
  • Attractive cost: $0.10 per million invocations after the first 2M covered under the free tier

CloudFront Function to remove a trailing slash:

function handler(event) {
    var originalUrl = event.request.uri;
    if (originalUrl.length > 1 && originalUrl.endsWith('/')) {
        var newUrl = originalUrl.substring(0, originalUrl.length - 1);

        var rawQuery = "";
        var cfQuery = event.request.querystring;
        for (var prop in cfQuery) {
            rawQuery += `${prop}=${cfQuery[prop].value}&`;
        }

        if (rawQuery.endsWith('&')) {
            rawQuery = rawQuery.substring(0, rawQuery.length - 1);
        }

        if (rawQuery) {
            newUrl += `?${rawQuery}`;
        }

        return {
            statusCode: 301,
            statusDescription: 'Found',
            headers: {
                "location": { "value": newUrl }
            }
        };
    }

    return event.request;
}

Use the built-in test functionality to verify it handles your required cases, then assign the function to your distribution.

Ermir Beqiraj is a backend architect building AI-integrated infrastructure. This is his personal writing.