Issue! Duplicate without user-selected canonical
Oops… Google’s spider has been checking your site and found the above issue. That means at least:
- You didn’t specify a canonical metatag or header
- The same resource is reachable from more than one URI
Google elaborates more on this here and here.
So what’s the problem?
If the crawler finds that you don’t organize your resources per their Uniform Resource Locators spec, it may choose not to index your page.
That said, having a resource reachable from more than one URL can be legitimate. For example, consider a search on a commerce site:
/laptop/msi/the-id-here/laptop/msi?memory=32/laptop/msi?memory=32&color=black
The second and third queries might return a subset of the same products as the first. Which one is the true source? That’s what canonical URLs are for — you declare which URI is the definitive source, and the crawler respects it.
In my case the issue was simple: the same page rendered with or without a trailing slash. There’s an entire saga of to-slash-or-not-to-slash in Google docs.
The options
ASP.NET Core documentation approach
Microsoft docs covers this for IIS and Apache using the RewriteRules NuGet package (open source here).
A typical public app HTTP request goes through:
[client] → [api-gtw | cache] → [lb] → [reverse-proxy] → [application]
The sooner you let the client know about the new location, the better. I’m not using IIS or Apache, so this option doesn’t apply.
Configure Nginx to remove the trailing slash
If you’re using nginx and can modify its config, add a rewrite directive in the root location as the first rule:
location / {
rewrite ^(/.*?)/?$ https://super-duper-domain.com$1 permanent;
}
- Pros: Redirect happens before reaching the application
- Cons: Nginx (infrastructure) needs to know about SEO rules
- Gain: Saves the round-trip from nginx to application and back
In-code middleware
Performance-wise this is the worst option, but it’s the right choice when business logic is attached to the slash decision. For pure removal, ASP.NET Core middleware is clean and straightforward.
Using the RewriteRules package:
using Microsoft.AspNetCore.Rewrite;
using RewriteRules;
var app = builder.Build();
// other middlewares (e.g., forwarded headers) go before this
var options = new RewriteOptions().AddRedirectToCanonicalUrl(new CanonicalUrlOptions
{
TrailingSlash = TrailingSlashAction.Remove
});
app.UseRewriter(options);
Or without any extra package — a tiny inline middleware:
var app = builder.Build();
// other middlewares go before this
app.Use(async (context, next) =>
{
var path = context.Request.Path;
if (path.HasValue && path.Value.Length > 1 && path.Value.EndsWith("/"))
{
var nonSlashPath = path.Value[..^1];
if (context.Request.QueryString.HasValue)
nonSlashPath += context.Request.QueryString.Value;
context.Response.Redirect(nonSlashPath, permanent: true, preserveMethod: true);
return;
}
await next();
});
My choice: CloudFront Functions
The only publicly accessible route to my app is through a CloudFront edge node. That makes CloudFront Functions the best candidate.
Cons:
- Moves app routing concern to the infrastructure level
Pros:
- Runs at the edge — the nearest network node capable of issuing the redirect
- Serverless, scales to 10M+ requests per second
- Attractive cost: $0.10 per million invocations after the first 2M covered under the free tier
CloudFront Function to remove a trailing slash:
function handler(event) {
var originalUrl = event.request.uri;
if (originalUrl.length > 1 && originalUrl.endsWith('/')) {
var newUrl = originalUrl.substring(0, originalUrl.length - 1);
var rawQuery = "";
var cfQuery = event.request.querystring;
for (var prop in cfQuery) {
rawQuery += `${prop}=${cfQuery[prop].value}&`;
}
if (rawQuery.endsWith('&')) {
rawQuery = rawQuery.substring(0, rawQuery.length - 1);
}
if (rawQuery) {
newUrl += `?${rawQuery}`;
}
return {
statusCode: 301,
statusDescription: 'Found',
headers: {
"location": { "value": newUrl }
}
};
}
return event.request;
}
Use the built-in test functionality to verify it handles your required cases, then assign the function to your distribution.