Skip to content

Updates to improve handling of 404s for bibliography entries - #355

Merged
stumbo merged 8 commits into
mainfrom
bs39_404_improvements
Nov 1, 2026
Merged

stumbo merged 8 commits into
mainfrom
bs39_404_improvements

Conversation

@stumbo

@stumbo stumbo commented Oct 25, 2026

Copy link
Copy Markdown
Member

Custom 404 message is provided for bibliography entries. If we have a record of the page being removed we supply both the page name and item title. In either case, provide a link to the bibliography, Zotero and the Bibliography intake page.

Also maps so links that Google reports as 404s to relevant pages on the website.


After reviewing the latest Google report on 404s I decided to try developing a custom 404 page. This is the result of that AI assisted effort.

Custom 404 message is provided for bibliography entries.  If we have a
record of the page being removed we supply both the page name and item
title.  In either case, provide a link to the bibliography, Zotero and
the Biblography intake page.

Also maps so links that Google reports as 404s to relevant pages
on the website.
@stumbo stumbo self-assigned this Oct 25, 2026
@interlisp-staging-bot

Copy link
Copy Markdown

Staging preview for this pull request is available at https://interlisp.github.io/Interlisp.staging/pr-355/

@pamoroso

Copy link
Copy Markdown
Member

The bibliography entry for 1108 USERS GUIDE has URL https://interlisp.org/Interlisp.staging/pr-355/history/bibliography/wxmqhqcz If I generate an incorrect URL, by, say appending garbage like this https://interlisp.org/Interlisp.staging/pr-355/history/bibliography/wxmqhqczGARBAGE I get a 404 page that differs from what the PR seems to address:

image

Am I misunderstanding what the PR is supposed to do?

@nbriggs

nbriggs commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

An example of a link that gets the custom 404 page would be helpful in the PR comments.

@hjellinek

Copy link
Copy Markdown
Contributor

I tried a similar test to @pamoroso's and got identical results.

Can you provide an image of what the custom 404 page ought to look like, and what URL prefix(es) it applies to?

@stumbo

stumbo commented Oct 25, 2026

Copy link
Copy Markdown
Member Author

Here's what it looks like when running locally:
image

A couple things to note. Bibliography pages have a specific format: /history/bibliography/f7xlxrg5/

There appear to be two points of failure that need further addressing:

  • Loosen up the rules for a bibliography entry: Currently I restrict identification of a potential bibliography entry to the expected 8 later format. Appending junk or something on to a good bibliography entry creates a bad URL, but not what I currently flag as a bad bibliography entry URL
  • Secondly, and related, URL root matching. The root for the staging site doesn't match my root URL pattern. Which is why it works fine locally but fails on the staging site.

I'll update those two things and validate that it works on the staging site. As far as some test URL, here are the ones Google is currently flagging as 404s:

More work to do...

@hjellinek

Copy link
Copy Markdown
Contributor

https://interlisp.org/Interlisp.staging/pr-355/history/bibliography/wxmqhqnz/ has an 8-letter document ID I concocted, and it produces a standard 404 error page, not a bibliography-specific one.

@interlisp-staging-bot

Copy link
Copy Markdown

Staging preview for this pull request is available at https://interlisp.github.io/Interlisp.staging/pr-355/

@stumbo
stumbo marked this pull request as draft October 26, 2026 03:46
@interlisp-staging-bot

Copy link
Copy Markdown

Staging preview for this pull request is available at https://interlisp.org/Interlisp.staging/pr-355/

@interlisp-staging-bot

Copy link
Copy Markdown

Staging preview for this pull request is available at https://interlisp.org/Interlisp.staging/pr-355/

@interlisp-staging-bot

Copy link
Copy Markdown

Staging preview for this pull request is available at https://interlisp.org/Interlisp.staging/pr-355/

@stumbo
stumbo marked this pull request as ready for review October 26, 2026 21:26
@stumbo

stumbo commented Oct 26, 2026

Copy link
Copy Markdown
Member Author

Moving this back to ready to review. The underlying problem wasn't this PR but how the staging site works. Each individual PR is based off the staging site's home. Interlisp.org/Interlisp.staging -- the problem was 404 pages are expected at the root, Interlisp.org/Interlisp.staging. This PR has no knowledge that when staged it's going to be mapped slightly differently. The fix was to update the staging site's build script to ensure the 404 page is mapped correctly.

If we have multiple PRs updating the 404 page, we will run into difficulties. But for now I think the probability is pretty low.

@interlisp-staging-bot

Copy link
Copy Markdown

Staging preview for this pull request is available at https://interlisp.org/Interlisp.staging/pr-355/

@hjellinek

Copy link
Copy Markdown
Contributor

This is an improvement, but there's still the program that only well-formed (8 character) document IDs trigger the special 404 page. A broken link is a broken link, right? Why bother checking for a special kind of brokenness?

@stumbo

stumbo commented Oct 27, 2026 •

Copy link
Copy Markdown
Member Author

This isn't a hill I'm willing to die on, but I think it's work discussing a little before making a decision.

A broken link is a broken link...

Yes and no. Yes, a broken link is broken. But, is it binary? What if we know more about the user's intentions, do we help the user and ourselves by directing them to information on our site that might get them what they're looking for?

Bibliography links follow a known pattern (well known to those that look at them), if a link comes in matching that pattern, there's a good bet the missing link was for something in our bibliography that for some reason is no longer there. It seemed reasonable to make that assumption and serve up an error page that would help the user find information in our bibliography.

In cases, where the page doesn't match the pattern for a bibliography entry, then we just fall through to our standard 404 page, making no assumptions on what the user was looking for.

@hjellinek

Copy link
Copy Markdown
Contributor

My point is it's simple to consider the pattern for a bibliography entry as anything under the /history/bibliography/ directory structure, which handles common errors like these:

Here's the document I told you about, https://interlisp.org/history/bibliography/wxmqhqnz/.

Hey Dan, check out this crazy essay about translating CLisp to APL https://interlisp.org/history/bibliography/
wxmqhqnz/

I just don't see the utility of trying to distinguish "correct" typos from "incorrect" ones.

@stumbo

stumbo commented Oct 27, 2026

Copy link
Copy Markdown
Member Author

Ah, maybe I misinterpreted your comment - you're arguing that anything under bibliography should be treated as a bibliography error and driven to the bibliography specific 404 page. Not, as I initially interpreted, everything should go to the generic Interlisp 404 page.

I can agree to that.

@hjellinek

Copy link
Copy Markdown
Contributor

Ah, maybe I misinterpreted your comment - you're arguing that anything under bibliography should be treated as a bibliography error and driven to the bibliography specific 404 page. Not, as I initially interpreted, everything should go to the generic Interlisp 404 page.

I can agree to that.

Yes, I am arguing exactly that.

@interlisp-staging-bot

Copy link
Copy Markdown

Staging preview for this pull request is available at https://interlisp.org/Interlisp.staging/pr-355/

@pamoroso

Copy link
Copy Markdown
Member

At commit 8ceb8ed I can get the expected 404 page with a link that includes the bibliography path in the URL, such as https://interlisp.org/Interlisp.staging/pr-355/history/bibliography/hiazfwryGARBAGE

@stumbo

stumbo commented Oct 29, 2026

Copy link
Copy Markdown
Member Author

@hjellinek Let me know if there's anything else we need to discuss about this PR.

@hjellinek

Copy link
Copy Markdown
Contributor

@hjellinek Let me know if there's anything else we need to discuss about this PR.

I just visited the URL in Paolo's message and got the normal 404 page, which is not what I expected.
screenshot_1398
I've gotten identical unexpected results with other garbage URLs that start with https://interlisp.org/Interlisp.staging/pr-355/history/bibliography/.

@hjellinek hjellinek changed the title Updates to improve handling of 404s for bibliography entires Updates to improve handling of 404s for bibliography entries Oct 29, 2026
@pamoroso

Copy link
Copy Markdown
Member

Now I no longer get the bibliography themed 404 page at https://interlisp.org/Interlisp.staging/pr-355/history/bibliography/hiazfwryGARBAGE either, only the generic 404 page.

@stumbo

stumbo commented Oct 30, 2026

Copy link
Copy Markdown
Member Author

From my comments earlier:

If we have multiple PRs updating the 404 page, we will run into difficulties. But for now I think the probability is pretty low.

I neglected to think that until this change is merged any other PR that hits the staging site is going to swap back in the default 404 page. And, I pushed up a draft PR earlier that did just that.

I'll reexamine the changes I made to the GitHub Actions on the staging site. After thinking about it some more, I think I have a better idea that will fix this issue and keep the staging sites 404s PR specific.

@stumbo

stumbo commented Nov 1, 2026

Copy link
Copy Markdown
Member Author

@pamoroso @hjellinek I've reworked the staging site. It now serves up PR specific 404 pages. You should now see the correct 404 pages when reviewing this PR. If you go to another PR and reference a non-existent page, you'll get the 404 for that site.

When you have a chance let me know if you run into any problems.

Also, with the changes made to the staging site, it's root page is a little more useful: https://interlisp.org/Interlisp.staging/

@pamoroso

pamoroso commented Nov 1, 2026

Copy link
Copy Markdown
Member

I confirm I see the expected bibliography 404 when reviewing the staging site of the corresponding PR.

@hjellinek

Copy link
Copy Markdown
Contributor

LGTM.

@stumbo

stumbo commented Nov 1, 2026

Copy link
Copy Markdown
Member Author

@pamoroso @hjellinek Excellent, now do either of you want to approve it?

@pamoroso pamoroso left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@stumbo
stumbo merged commit 6c3b96d into main Nov 1, 2026
6 checks passed
@stumbo
stumbo deleted the bs39_404_improvements branch November 1, 2026 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants