Please fix: Unknown or expired link.

https://news.ycombinator.com/item?id=7651301

by kijin • 12 years ago

112 75 12 years ago

news.ycombinator.com

I'm pretty sure that the HN community likes to read thoughtful, thoroughly researched comments that are somewhat longer than one-liners. But whenever I take more than a few minutes to write a comment, I am greeted with the following error message as soon as I hit "Submit":

    Unknown or expired link.

Fortunately, I can usually hit the Back button, refresh the page, and resubmit my comment without losing the contents of the textarea. Anyway, I try to be extra careful and copy the comment to the clipboard before I hit "Submit" if I feel like it's been more than a few minutes since I started writing.

But this should not be necessary in the first place.

Please increase the timeout on your CSRF tokens (or whatever it is that triggers the error message above) to something more user-friendly. I would suggest a sensible default of 86400 seconds.

Thanks.

lutusp• 12 years ago

There's a reason things are set up as they are:

* Although HTTP connections are normally stateless, HN has a login system for account holders, which means there's a state, a connection, associated with each logged-in user.

* Because of the state information associated with logins, A Web server has a limited number of login connections it can handle.

* To serve the maximum number of visitors, a server needs to recycle connections that are no longer in use.

* There's no reliable heartbeat method between browser and server that a server can use to determine whether a given connection is still in use. Also, heartbeats would represent an extra traffic load on the server. And now that tabs have become popular, there's no way to be sure that a heartbeat response isn't from an unused, forgotten tab among dozens.

* Therefore, after a connection has not seen packet requests for a set time interval, the server expires the connection to make it available to other visitors.

* The shorter the timeout, the more connections the server can handle.

I would suggest a sensible default of 86400 seconds.

Let's say the server can handle 2048 simultaneous login sessions (I don't know the actual number). According to your suggestion, after the first 2048 visitors log in, the server would refuse to accept any new logins for 24 hours. This isn't realistic.

Here's how I handle the problem -- if I decide a reply is going to take more than a few seconds, I transfer my reply to a text editor and let the connection expire. When my long-winded reply is ready, I refresh the original page, paste my reply in, and click "Add Comment".

I hope this helps.

krapp5• 12 years ago

Although - other sites can manage user state and validating posts just fine without this 'feature.'

Even assuming it's all but impossible to fix given the implementation in Arc, surely it would be possible to automatically reassociate the post with whatever the current function id is? It seems to be as much a user experience problem as anything.

jayrox2• 12 years ago

i always figured it was because they built each page and cached it to html, then that cache was set to expire after 5 minutes. then once the cache expired you couldn't go to the next page or what ever because the cached file was deleted.

IMO the reasons presented for this "Feature" should be changed to "lazy programming". cause that's all it is.

exelib• 12 years ago

Then it's broken by design. There is no sense to do so, but a great way to anger visitors and destroy UX.

nwh2• 12 years ago

I don't think this is what is going on here, it makes no sense for it to operate like this. To demonstrate this is not the case, I just momentarily unplugged my NIC and then submitted this comment. The connection was definitely dropped and re-associated, but my comment passed perfectly fine. This is more likely a very short cache time on CSRF tokens rather than anything to do with associated sockets.

phamilton2• 12 years ago

Http is stateless, so there are no "known sockets". Every request opens a new socket. Unplugging and plugging back in is a no-op in this context.

brudgers4• 12 years ago

The architectural model your comment and the initial post assumes to underlay the behavior is incorrect.

The reason for 'Unknown or Expired Link' is that HN runs on a webserver that uses continuations to maintain state. The continuations are stored in a table and based on load individual table entries get flushed to make room for new entries as needed. The message is actually always the result of an unknown link because the hash is no longer in the table.

Your experiment worked because the relevant continuation [state reference] was available after disconnect e.g. the 'route' so to speak was still valid.

Now obviously one might wonder why HN works this way. The reason is that the HN codebase came about as a way of field testing Arc [Graham's 100 year language]. Until rather recently, HN ran on a single server. It still uses relatively few lines of code.

Quai2• 12 years ago

This is not the case. Websites, including hacker news, does not handle sessions like that.

kijin3• 12 years ago

CSRF tokens != login sessions != persistent HTTP connections.

There is no need to keep an HTTP connection open in order for the server to log people in or recognize when a valid token has been submitted with a form.

grey-area4• 12 years ago

Because of the state information associated with logins, A Web server has a limited number of login connections it can handle.

You're making excuses for a particular setup - this is not something which holds true of all web servers. It's quite possible to store the state necessary to allow 10000 simultaneous logins if you use cookies and a db to store your state, rather than relying on in-memory state and fragile continuation links like this (created a few minutes ago):

https://news.ycombinator.com/x?fnid=l5qgIWgvnRbHE7owLEaPMw

or tying replies to stored state on the server which expires within minutes. Many large scale sites manage it, I suppose because they don't store data like this in memory or urls. I'm not sure about this idea of storing state in urls, it often backfires, and urls should identify resources and actions, not store state.

There is no good reason the server cannot receive a POST request from an authenticated user against a node id, and perform the requested action (post my reply please), no matter how many intervening logins/actions it has handled.

Looking at the form in question (reply forms) reveals why it does break:

    <input type="hidden" name="fnid" value="UswPVmT3a8lZnqPBtcx359">

The server stores a continuation in the comment reply form, which (like the more link above) expires very quickly. The reason it expires is presumably because the server keeps a list of these fnids in memory, and as you say the list is finite, but it doesn't have to do that. There are other ways to create a CSRF (if that's what they use this for) which won't expire so quickly, and this problem is not one which many sites experience.

It is annoying just how quickly reply links, more links etc expire on this site, and if the site used a different method to sign reply forms or construct other links they could get rid of this issue. Keeping data in memory like this will become more and more of a problem as the site scales - eventually things would break in seconds with enough load - that's clearly not acceptable.

zackbrown• 12 years ago

A login system does introduce a notion of state between the server and client, but the use of the word 'connection' to describe that is probably a misnomer. I'm unaware of any basic HTTP servers that use a persistent socket connection (which seems to be the sort of connection you're describing, given the resource constraints you suggest) with clients in order to manage authentication.

Most often, the server just needs to set a flag associating an auth token with a given user (for whom it has a persistent representation, e.g. in a database.) Lookups against this token => user hash/map/table should be very cheap, especially if it's kept in memory. Further, even for tens of thousands of concurrent users, that's not a lot of memory overhead (1KB/authed user would be a generous upper bound, and that would still only be 10MB of RAM per 10k concurrent users.) Could be that HN is running on a 1990s-era desktop with 256MB of memory in someone's office and that this is an actual resource constraint issue--but my hunch is that this problem lies on the software side, not the hardware/resource side.

newobj• 12 years ago

I'm not sure what's more disturbing; this reply, or the fact that only 2 out of 5 replies to this reply attempt to refute it.

ordinary• 12 years ago

This is largely correct. Each reply form includes a hidden input field with the name 'fnid' in their form. Many other pages include a parameter with the same name. This is the unique identifier of a closure that's associated with the page you're on. These closures are deleted after X time, at which point requests from that page can no longer be handled correctly.

Yes, obviously it would be great if X were increased. But doing so would require an increase in server resources or a significant change in the way HN's backend works; neither of these options is free. That is the reason why things are this way.

This is an old debate. See, for example: https://github.com/HackerNews/HN/issues/11

grey-area4• 12 years ago

fnids are evil, they just need to drop them and respond properly to posts without them. What more state needs to be recorded which is not already encoded in a POST reply request {user:n,post:7651593,text:'xxx'}?

The solution should be pretty straightforward:

    Reply forms -> Remove fnid, authenticate on post, add CSRF

    Flag links -> Remove fnid, authenticate on post, add CSRF

They already check authentication on these actions anyway as you'll see if you try upvote links from another user without login, all that would be required would be to use CSRF instead of these closures to handle posts like this - the closures are not a workable solution on a large site, and I'd contend are dangerous on a small site - they throw away a lot of the advantages of HTTP for no appreciable gains.

Things like the more link are particularly pointless, they could just use something like:

    https://news.ycombinator.com/x?fnid=xxx -> https://news.ycombinator.com/news/2

This would change over time like the existing home page, but there's no reason for a simple GET request like this to expire.

acdha3• 12 years ago

* Because of the state information associated with logins, A Web server has a limited number of login connections it can handle.

a) this is technically incorrect – you can trivially handle this using signed cookies to store the logged in user

b) even using traditional database or filesystem based approaches, those limits would be aggressive even if the site was hosted on an iPhone – you're talking tens of bytes per session.

To serve the maximum number of visitors, a server needs to recycle connections that are no longer in use.

Also technically wrong: if you aren't doing anything, you aren't causing any load whatsoever on the server. This would be different for something like a chat site using WebSockets but for a vanilla app like HN this is not an issue.

Also, again, even if you were using a persistent connection the absolute technical limits are much, much higher than the traffic HN sees - hundreds of thousands of simultaneous users.

There's no reliable heartbeat method between browser and server that a server can use to determine whether a given connection is still in use.

Again, technically wrong: plain HTTP sites don't maintain persistent connections but in the case of something like a WebSocket connection you'd simply use TCP keepalives, which are a mature standard feature and very low overhead.

You appear to have confused the concept of a session with something which is actually happening at the network level. Try reading something about the HTTP request cycle (e.g. http://www.slideshare.net/cczona/full-stack-full-circle-what...) for basic background and after that move on to some discussion about the work done to make servers which can handle many simultaneous connections - the C10K problem was the original work tuning operating systems to handle tens of thousands of simultaneous connections and was large a non-issue at the network / basic HTTP leel by the mid-2000s:

http://bulk.fefe.de/scalability/

By now a single server can be assumed to handle hundreds of thousands of simultaneous connections at the network level – assuming, of course, that you have capacity to actual do whatever work those connections are used for. Idle users typing text into a form wouldn't be a problem even if it did use a persistent connection.

jamesmoroni• 12 years ago

This happens with the next page button as well if I've been on the page for more than 3 minutes or so.

Quai2• 12 years ago

I see this almost every day. I have always assumed that Hacker News is too cool to fix/support navigation after the frontpage (or any other page) has changed.

judk2• 12 years ago

It's not "cool", it's that the implementation is a quick hack, not intended to be a quality product. It just needs to be good enough to make users grudgingly return and see the PR for YC startups.

midnitewarrior• 12 years ago

Ya, for a community that is supposed to value great design and technical acumen, this site looks like somebody's pet project that was abandoned.

danford• 12 years ago

I don't really mind the design. I have a feeling they purposely keep the site design simple and, what some might consider "unappealing" to the average user. Helps weed out the normies.

judk2• 12 years ago

It also weeds out the adults who think about what they read and don't have child eyes.

brudgers4• 12 years ago

It's not a priority for HN development. See Graham's 2009 essay for an explanation of what matters:

http://paulgraham.com/hackernews.html

vannevar• 12 years ago

This 'bug' has been around so long that at this point I suspect that it's really just a long-running troll.

chrismorgan• 12 years ago

This has bearing on my absolutely favourite part of the Web Content Accessibility Guidelines (WCAG) 2.0:

Section 2.2.1, Timing Adjustable (http://www.w3.org/TR/WCAG20/#time-limits-required-behaviors):

----

2.2.1 Timing Adjustable: For each time limit that is set by the content, at least one of the following is true: (Level A)

- Turn off: The user is allowed to turn off the time limit before encountering it; or

- Adjust: The user is allowed to adjust the time limit before encountering it over a wide range that is at least ten times the length of the default setting; or

- Extend: The user is warned before time expires and given at least 20 seconds to extend the time limit with a simple action (for example, "press the space bar"), and the user is allowed to extend the time limit at least ten times; or

- Real-time Exception: The time limit is a required part of a real-time event (for example, an auction), and no alternative to the time limit is possible; or

- Essential Exception: The time limit is essential and extending it would invalidate the activity; or

- 20 Hour Exception: The time limit is longer than 20 hours.

Note: This success criterion helps ensure that users can complete tasks without unexpected changes in content or context that are a result of a time limit. This success criterion should be considered in conjunction with Success Criterion 3.2.1, which puts limits on changes of content or context as a result of user action.

----

Yea, verily! HN isn’t even Level A (the most basic level that "everyone" should comply with) compliant. Through this, HN fails Accessibility 101. (Its text colour choices are also often dubious as far as contrast goes—that’s covered in WCAG 2.0 as well, of course.)

I simply don’t care about the technical reasons. The choice of coroutines to back all of these things has broken the basic accessibility of all the content.

zura• 12 years ago

The choice of coroutines to back all of these things has broken the basic accessibility of all the content.

Exactly. This is actually a good example of More is Less.

seabee• 12 years ago

The choice of coroutines is not the cause of the problem, it's the decision to store session state on the server and clearing that state before the user is done.

This same problem can happen in a PHP application, and it can be avoided in a Racket web framework application that uses continuations.

IgorPartola2• 12 years ago

Also, can someone please make the site render with a larger font on mobile? I am spending a fortune on new glasses and contact lenses, as well as monocles and loupes just to read HN when not on my computer. So many people on here have asked for this over the years. Perhaps this is a subtle attempt by the YC to make sure all the people in our industry have to use eyewear so we can recognize each other at the secret meetings. Maybe this has something to do with Gunnar Optiks marketing eyewear for geeks? Maybe YC is about to fund an optometrist practice. Or maybe someone here knows that Apple is about to release a 20 inch iPad?

Please, for the love of everything that is good and pure and true, make the font larger!

Sincerely, Blind among blind

Edit: please don't suggest one of the other HN wrappers. I know they exist. I don't want to hand over my HN password to them. My HN identity is important to me, so unless I can run my own wrapper on my own servers I don't want it.

Edit edit: I believe if HN mods/operators announced a contest for best mobile design they'd get a ton of submissions. Sadly I am not a designer and could not participate, but I am sure the amount of submissions would be huge.

loup-vaillant• 12 years ago

Can't you control the minimum font size on your mobile browser or something?

IgorPartola2• 12 years ago

I believe on iOS you can increase the font size for the entire UI, not just the browser. Not something I would want to do. No idea about Android.

maxerickson• 12 years ago

There is a new layout that is supposed to be more mobile friendly in progress. dang has (briefly) posted about it here:

https://news.ycombinator.com/item?id=7606298

saurik• 12 years ago

Edit: please don't suggest one of the other HN wrappers. I know they exist.

What about one of the many apps? Have you tried news:yc?

wfjackson• 12 years ago

In the meantime you can use this http://ihackernews.com

vosper• 12 years ago

My solution to this for my Android phone (Galaxy Nexus) was to switch to the Opera browser. Opera is one of few browsers that still supports text reflow, meaning that I can easily zoom in and have the content nicely reformatted to fit the screen. Works like a charm on HN.

pfg• 12 years ago

See [1] on why this is happening. Relevant part:

Internally, the web server associates a token called the fnid with each instance of a link or form, and records the continuation function for each token. Periodically, old fnids are deleted.

[1] http://arclanguage.github.io/ref/srv.html

kijin3• 12 years ago

Thanks for the reference.

Any idea where the fnids are stored and how they are garbage-collected? Could it be that they're just dumped into a memcached instance and older fnids get pushed out whenever the memory limit is reached?

wahnfrieden• 12 years ago

It's not the fnid that gets garbage collected, it's the coroutine itself.

gaving• 12 years ago

I'm not sure anybody particularly cares why it is happening, just that it is happening.

jmount• 12 years ago

I agree with you: HN time-outs and failures (which also occur on reading) are an unnecessary pain. This is one reason you should never write into a web text box (a rule I also fail to adhere to). But really even "modern application grade" web stuff is so flakey if you value your text you write it in a local editor and then copy/paste later.

BrandonMarc2• 12 years ago

With this rule in mind, I wonder if a textures / notes area would be a good feature to include in the web browser itself (including mobile browsers) ...

sergiotapia• 12 years ago

Public Service Announcement:

If you're on Chrome use HackerNews Enhancement Suite.

https://chrome.google.com/webstore/detail/hacker-news-enhanc...

No more ass-ugly hackernews font choices and colors.

sk5t• 12 years ago

Neat! Worth it for the option to collapse threads alone. Recently some extraordinarily lengthy, tedious threads have dominated the discussion on otherwise-interesting topics, and simply finding the end of the dreck was a problem.

cordite• 12 years ago

Yeah, it's really annoying to have to edit the URL on mobile when this happens..

throw_away128472• 12 years ago

I really do believe that HN is poorly designed for end users. The UI is not forward with intent, or pleasurable to use. It's been literally years and the expired link issue hasn't been addressed. ---No, I'll stop you right there. It's been acknowledged as an implementation bug, but not addressed. PG's lack of user empathy and stubbornness make HN worse.

The only reason people come here is the content.

btilly2• 12 years ago

The only reason people come here is the content.

I consider this a good thing. It attracts the right kind of people.

throw_away128472• 12 years ago

Are you insinuating that the key to dissuading the wrong kind of people is to make Hacker News difficult to use? The notion that good users will "power through" a bad design is dubious at best.

I hope that you're not involved in product design. This thought process is not good for anyone.

jellicle• 12 years ago

For any new readers: this is a "bug by design", it has been extensively complained about in the past, and it's marked "will not fix, stop complaining" by management.

This has been a public service announcement.

akkartik2• 12 years ago

Not true. It's an open bug. It's just really hard to fix.

https://github.com/HackerNews/HN/issues/11

kijin3• 12 years ago

@akkartik's JS redirect suggestion sounds like it will be trivial to implement. No need to fix the legacy arc code, just add a stopgap measure that will improve the UX for a lot of people :)

falcolas• 12 years ago

With all due respect, if a framework makes storing sessions for more than 30 minutes "hard", there's something fundamentally wrong with the framework. This has been a solved problem for longer than I have worked in the industry.

phamilton2• 12 years ago

One positive side effect of the architecture is proper isolation in pagination. When I fetch Page 1, the link to Page 2 will always return the same set, regardless of timing (up until it expires). You will never see the last entry on one page show up as the first entry on the next page.

grey-area4• 12 years ago

I have a modest proposal for addressing the terrible dearth of fnids on this site. Since fnids are in such high demand, and yet are expensive to keep in memory, one small change might free up lots of them - remove the use of fnids for more pages and use a static url instead

so the more link on:

https://news.ycombinator.com/news

would point to:

https://news.ycombinator.com/news/2

and so on.... This would free up many fnids for use elsewhere, and also make the secondary list pages more easily cacheable and reduce load on the server.

The same could be done for other lists which currently use fnids without any appreciable change in the user experience or probably much change in the server code.

Kiro• 12 years ago

Can someone explain why this expiration thing is happening like I'm 5? Even the most basic PHP tutorial app doesn't have this problem so I don't understand people saying it's inevitable.

BrandonMarc2• 12 years ago

So ... is this a bug or a feature?

Is the internal activity causing this unpopular behavior supposed to do this, in the first place? If the site is indeed behaving exactly as its designers want it to, then that's a rather separate debate as opposed to simply ways and means.

I'm not saying I agree with the behavior; I'm just trying to see what should be debated to start with.

Raphael• 12 years ago

Yes, the order changes over time, so the second page will often contain items from the old first page.

dang2• 12 years ago

I'll try to respond to everything in one comment.

kogir and I talked recently about getting rid of "unknown or expired link". It's on our list. There are a lot of things on the list; we don't know yet when we'll get to it.

There's work ongoing to make HN more usable on mobile. This will likely roll out sooner than the above. We intend to roll it out incrementally, starting with the front page. We'll give people lots of notice so anyone running scrapers that are affected (like me!) will have time to change.

The look and feel of HN is unlikely to change much. We can argue about that one till the cows come home. I'm tempted to—my opinions are as strong as many of yours—but that doesn't seem like the best idea right now.

Feature requests should go to https://news.ycombinator.com/item?id=363. Software bugs should go to https://github.com/HackerNews/HN/issues. That would have been the appropriate destination for the OP, and indeed this issue has been posted about there.

Threads like the current one get lots of upvotes and comments, but they've never been on topic for HN. I'm going to bury this one now.

krapp5• 12 years ago

How is it not on topic for HN?

It would be on topic to discuss the implementation of any other site.

dang2• 12 years ago

What's on topic for HN, as the guidelines say, are stories that gratify intellectual curiosity. Meta posts gratify several things, but that is not one of them. They generate masses of comments like fast-growing weeds, because we're all full of opinions about HN, but the discussions never go very deep, or really anywhere. This has been the case for many years—pretty much since the beginning of the site—and so has the policy on it.

dfc• 12 years ago

  > I'm pretty sure that the HN community likes to read thoughtful, thoroughly
  > researched comments that are somewhat longer than one-liners.

Could not agree more and this is a great side effect of the time out. It gives the commenter yet another reason to compose their comment in a real editor. Personally I find the comment text box is too small to do a decent job of assessing and editing my comment. A real editor also makes it easy to format the above blockquote[^1] or to insert[^2] logical and consistent footnotes.[^3]

[^1]: http://www.nicemice.net/par/

[^2]: http://jblevins.org/projects/markdown-mode/

[^3]: I just wish markdown-mode would renumber the footnotes if I go back and insert a new footnote. As it is I have to pipe the output through pandoc

Please fix: Unknown or expired link.

Related Stories