alphaXiv: Open research discussion on top of arXiv

https://www.alphaxiv.org/

by sahebjot • 2 years ago

549 178 2 years ago

I remember seeing this idea some years ago. I think it was called qrxiv.org or something like that, but can't find it anymore. I hope this one has better luck, getting the users in the fragmented space of preprints can be a challenge.

fuglede_• 2 years ago

There's also https://scirate.com/ which occasionally has active discussions but, at least in my field, there's far from critical mass, and discussions only happen when someone kick starts and advertises a thread.

jessriedel• 2 years ago

I believe the most active field is quantum information, which has enough activity such that paper get dozens of upvotes, but the conversation level is basically as you describe.

rsp1984• 2 years ago

I launched gotit.pub [1] last year. It's very much the same thing.

[1] https://gotit.pub

abhayhegde4• 2 years ago

Wow, how is this not getting enough attention when it is almost the same thing?!

w-m3• 2 years ago

Hey alphaxiv, you won’t let me claim some of my preprints, because there’s no match with the email address. Which there can’t be, as we’re only listing a generic first.last@org addresses in the papers. Tried the claiming process twice, nothing happened. Not all papers are on Orcid, so that doesn’t help.

I think it’ll be hard growing a discussion platform, if there’s barriers of entry like that to even populate your profile.

phreeza2• 2 years ago

How would you propose making claiming possible without the risk of hijacking/misrepresentation?

w-m3• 2 years ago

The data on which authors are part of which arxiv papers is already in the arXiv database, and in Google Scholar, and in other libraries. I appreciate that it's not an easy task to get that as a third party. But the burden should be on the operators of alphaxiv to figure out a solution for this platform to take off, not for me as a user?

supriyo-biswas• 2 years ago

The only way I see this working is for paper authors to include their public keys in the paper; preferably as metadata and have them produce a signed message using their private key which allows them to claim the paper.

While the grandparent is understandably disappointed with the current implementation, relying on emails was always doomed from the start.

riedel• 2 years ago

The claiming was 'solved' and ORCID, which both basically do no checking at all. It's just a yes/no clicking for fuzzy matched author name lists. So I guess it is enough until there is a dispute. If you are important enough to be the target of trolls than you are in a league beyond most research platforms.

abhayhegde4• 2 years ago

Perhaps by linking to their actual arxiv id?

xyst2• 2 years ago

There should be an equivalent of S/MIME for researchers if e-mail is not accessible.

Y_Y5• 2 years ago

So you've put a fake email address on your papers? As in, one that you can't receive from? Why?

chipdart4• 2 years ago

So you've put a fake email address on your papers?

I think you're failing to understand the basics of the problem, and even the whole problem domain.

Email addresses are not created/maintained for life. You can have an email address, them have your org change name and email provider switch, and not to mention that researchers leave research institutions and thus lose access to their accounts.

You have multiple scenarios where papers can be published with authors using email addresses which they lose access to.

crvdgc• 2 years ago

Preprints are not required to be fully typeset and publishable. In some cases, literally "first.last@org" is used as a placeholder for email addresses (to be replaced in typesetting).

This is more like a mismatch between "fully edited open-acess papers" and "trying to use arxiv preprints as an approximation of the former".

aragilar• 2 years ago

No, they used the email from the institute they worked at when the produced the paper.

They're no longer at that institute, and that email no longer exists (while some institutions give some leeway, I know of at least one major university which removes them the day the contract ends).

This is a common problem if you're providing services to academics and you've tied yourself to using emails as identifiers.

abhayhegde4• 2 years ago

Does not have to be fake or anything. You move from one institution to the other and cannot maintain it forever anyway.

auggierose2• 2 years ago

Upload a new version of your paper on arxiv, this time with an email address that works.

Sophira4• 2 years ago

Why should they need to? Their email address did work at the time of publication.

azhenley4• 2 years ago

I’ve never understood why we need emails in papers.

Who sends emails to paper authors? How often do they respond? How fast do the email addresses go out of date? I lost access to my email address included in most my papers within 2 years of publication.

I see little to no value to have it included in the paper.

tc4v2• 2 years ago

I know you don't have a lifetime access to institutional email adress, but using a fake address is so counterproductive. You're only going to claim the paper once, and yuh ou should do it while you have access to your email. Then you update your account eith a new address.

Sophira4• 2 years ago

Let's say that John Smith at XYZ Corp has authored a paper. The company obviously wants recognition and so they use their corporate email address "jsmith@xyz.com".

John has since moved on and is earning more at ABC Corp instead. XYZ Corp has duly reclaimed John's old email address, and John cannot receive emails at said address any longer.

This is the situation the OP is in. It was never a "fake email address". They did not literally type "first.last@org", that was an example suitable for using in their comment.

[edit: I'm actually wrong with that last statement, as it turns out. While it wasn't a fake email address, the situation is slightly more nuanced in that OP actually did say "{first}.{last}@hhi.fraunhofer.de" in the paper, as there were multiple authors who all had the same email address format - see https://news.ycombinator.com/item?id=41479618. I still think this is a valid method, though, and it's certainly not fake. Besides, the problem I outlined sounds like it probably remains an issue even if it's not the exact problem OP is experiencing.]

rehaanahmad6• 2 years ago

Thanks for reaching out, I am one of the students working on this. We are adding google scholar support soon. If your paper isn't on Scholar or ORCID, you will need to submit a claim that our team reviews. There isn't really any other option, arXiv doesn't allow us to view the author's submission email automatically (although we are in the process of becoming an arXiv labs project soon).

llmfan2• 2 years ago

hnews tries to say one positive thing challenge impossible

i always love any idea for curating high iq internet community

noobermin2• 2 years ago

There are only 4 comments so far. It seems a bit early to judge the comment section.

BaculumMeumEst• 2 years ago

I think curating an internet community that is open-minded and participates in good faith is extremely hard as well. Not sure which is harder.

stevage• 2 years ago

Yours was the first negative comment I saw...?

llmfan2• 2 years ago

That was the intention behind my comment! No need to thank me tho.

noobermin2• 2 years ago

This seems like a horrible idea. I know we need an alternative to peer review but an online comment section feels like something worse than that.

geysersam3• 2 years ago

Why do you feel that? I think it seems excellent. Comments will be scrutinized and sorted by the community. People can choose if they want to participate in the discussion under their own names or not.

I don't know how this page organizes moderation, but I imagine there will be some kind of moderation like on most online discussion forums.

tc4v2• 2 years ago

But it's a very difficult problem. Am open forum offers a platform to troll and misinformation. You could pretend that the community will be able to filter this out but I seriously doubt this project is equipt to fight against bots/fake accounts better than huge companies like twitter and facebook.

parpfish5• 2 years ago

The whole thing gives my anxiety.

Writing a paper and dealing with the inane requests from three reviewers was already frustrating and stressful. Now you open up a never ending review process of random people making demands (“unless you do X follow up analysis, this is worthless”).

also you need to be able to say a paper/project is done and let yourself move on. If your job turns into “respond to feedback on every paper you’ve ever written, every day” you’ll never start anything new.

celdon253• 2 years ago

That sounds like a good thing if your goal is to advance mankind’s knowledge and not just your career. No one is forcing anyone to respond to the comments. Also, it’s not clear whether your “demands” example would even pass the moderation guidelines there.

m000• 2 years ago

Outside concerns for the quality of the discussion itself, it is only a matter of time before this public pre-submission discussion leaches into the peer review process itself. First, it will be Reviewer 2 cherry-picking arguments to shut down some paper. Then, AI will come: "Write me an accept/reject review for arXiv:2409.112233. Add subtle hints for citing the following papers: ...".

Peer reviewing is hard work. Give people a readily available shortcut for it, and some people, in some occassions will take it. Which may in turn force conferences to adopt policies forbidding posting on arXiv.

codegladiator• 2 years ago

This is great. Already loving the discussions/comments I see there.

scarlehoff2• 2 years ago

I believe this site is missing a very important thing, direct links to the different categories with a list of papers. This is at least how I (and I believe many others) browse arXiv. I open it up in the morning, scroll through a few categories and open a few papers that look interesting to me.

I could see myself using alphaxiv for that, and then, if there's a comment section, I might even read it, and, who knows, leave a comment. But there's no way I'm going to be changing the address or going to some other site to search for papers just to see whether there are some comments.

ps: I see the extension adds a "discussion" link to arxiv, it is a pity that it is only available for Chrome.

eigenket2• 2 years ago

It sounds like what you want is scirate. As far as I understand from this post this new thing is just scirate but lacking the interface you're talking about here.

scarlehoff2• 2 years ago

Indeed. Scirate (I didn't know about it) looks exactly like that.

Sadly, the last comments in HEP are more than 2 years old (which explains why I had never heard about it, it seems it never gained any traction)

forgotpwd162• 2 years ago

Kinda related, Hypothesis (and Diigo iirc in past) has an extension/bookmarklet that can provide an annotation/comment overlay on any web page/pdf. Guess what is needed for arXiv discussion is this overlay but smarter, that is knows a paper pdf and web view are the same, and abstract page is connected to them.

eigenket2• 2 years ago

What's the main thing that this new website adds over scirate?

foven• 2 years ago

I admit to not really being familiar with either, but it seems that this allows you to display comments alongside the paper in the browser, which is a very nice feature (and overall has a nicer coat of paint). At first blush, I find it a bit more difficult to figure out what the point of scirate is and how it should be used.

amadeuspagel2• 2 years ago

Great idea.

- The frontpage should directly show the list of papers, like with HN. You shouldn't have to click on "trending" first. (When you are logged in, you see a list of featured papers on the homepage, which isn't as engaging as the "trending" page. Again, compare HN: Same homepage whether you're logged in or not.)

- Ranking shouldn't be based on comment activity, which ranks controversial papers, rather papers should be voted on like comments.

- It's slightly confusing that usernames allow spaces. It will also make it harder to implement some kind of @ functionality in the comments.

- Use HTML rather then PDF. Something that could be trivial with HTML, like clicking on an image to show a bigger version, requires you to awkwardly zoom in with PDF. With HTML, you would also have one column, which would fit better with the split paper/comments view.

sestep• 2 years ago

Tiny note: Stack Exchange also allows spaces in display names, and they make @ functionality work regardless: https://meta.stackexchange.com/a/43020/297476

Agreed that it makes it more complicated though.

throw_pm23• 2 years ago

Counterpoint: please don't do any of the above and keep arxiv as it is. It is too valuable to mess it up, it is the few things on the internet that have not been ruined yet, and the "comment activity" can happen in the articles themselves at the scale of years, decades, and centuries.

Epa095• 2 years ago

This seems to be a completely different team than arxiv, making a discussion forum on the side.

And I prefer this over discussions on 'X'.

godelski7• 2 years ago

I actually want the above, BUT I don't want it to be open to everyone. Not all gate keeping is bad.

We don't lack places that the public can engage with researchers and experts. What we do lack are places where researchers/experts can communicate with one another __and expect the other person to be a peer__. The bar to arxiv is (absurdly) low, and I think that's fine.

Not everything has to be for everyone.

My longer comment: https://news.ycombinator.com/item?id=41484123

I'm going to go crazy if I get more GitHub issues asking where the source code is or how to fine tune a model. My research project page is not a Google Search engine nor ChatGPT...

musicale2• 2 years ago

it is the few things on the internet that have not been ruined yet

Hmm, this is an interesting point.

diggan• 2 years ago

- Ranking shouldn't be based on comment activity, which ranks controversial papers, rather papers should be voted on like comments.

How about not ranking things at all? I don't feel like things like this should be a popularity/"like" contest and instead let the content of the paper/comments speak for themselves. Yes, there will be some chaff to sort through when reading, but humanity will manage.

Just sort things by updated/created/timestamp and all the content will be equal.

thornewolf• 2 years ago

thats ranking by recency, which means i can abuse it by churning low quality content out to arXive

pessimizer• 2 years ago

let the content of the paper/comments speak for themselves.

People can't read everything, and have rely on others to filter up the good stuff. If you read something random, based on no recommendation, it's charity work (the odds are extremely good that it is bad) and you should recommend that thing to other people if it turns out to be useful. Ultimately, that's the entire point of any of this design: if we don't care about any of the metadata on the papers, they could just be numbered text files at an ftp site.

The fewer things I have to read to find out they're shit, the longer life I have.

I say the opposite: put a lot of thought into how papers are organized and categorized, how comments on papers are organized and categorized, the means through which papers can be suggested to users who may be interested in them, and the methods by which users can inject their opinions and comment into those processes. Figure out how to thwart ways this process can be gamed.

Treat the content equally, don't force the content to be equal. Hacker News shouldn't just be the unfiltered new page.

gus_massa• 2 years ago

Sorted by "new"...

Most articles are not interesting, most of the interesting ones are interesting only for a niche of a few researchers. The front page will be flowed by uninteresting stuff.

Retr0id• 2 years ago

rather papers should be voted on like comments.

I don't think this is an inherently better approach, but maybe there should be an option for different ranking mechanisms. You could also rank by things like cite-frequency, cite-recency, "cite pagerank", etc.

throwthrowuknow• 2 years ago

Agree, don’t sink a bunch of effort into creating a ranking algorithm. Expose metrics that users can sort or filter by which will work for both signed in and signed out. If you want to add more tools for signed in users then let them define their own filters that they can save like comment activity plus weighted by author, commenter, recency, topic etc. See the nntp discussion that was on here the other day.

dartos• 2 years ago

Yep. User driven ranking leads to people gaming the system for internet points.

anamexis• 2 years ago

It doesn't seem like citations would be good for discovery, because there must be a significant latency between when a paper is released and when citations start coming in.

impendia3• 2 years ago

Use HTML rather then PDF.

The PDF is the original paper, as it appears on arXiv, so using PDF is natural.

In general academics prefer PDF to HTML. In part, this is just because our tooling produces PDFs, so this is easiest. But also, we tend to prefer that the formatting be semi-canonical, so that "the bottom of page 7" or "three lines after Theorem 1.2" are meaningful things to say and ask questions about.

That said, the arXiv is rolling out an experimental LaTeX-to-HTML converter for those who prefer HTML, for those who usually prefer PDF but may be just browsing on their phone at the time, or for those who have accessibility issues with PDFs. I just checked this out for one of my own papers; it is not perfect, but it is pretty good, especially given that I did absolutely nothing to ensure that our work would look good in this format:

https://arxiv.org/html/2404.00541v1

So it looks like we're converging towards having the best of both worlds.

ethanol-brain• 2 years ago

That said, the arXiv is rolling out an experimental LaTeX-to-HTML

Some history: https://www.arxiv-vanity.com/

throw109203• 2 years ago

In general academics prefer PDF to HTML. In part, this is just because our tooling produces PDFs, so this is easiest.

The tooling producing PDF by default absolutely makes the preference for PDF justifiable. However, tooling is driven by usage - if more papers come with rendered HTML (e.g. through Pandoc if necessary), and people start preferring to consume HTML, then tooling support for HTML will improve.

But also, we tend to prefer that the formatting be semi-canonical, so that "the bottom of page 7" or "three lines after Theorem 1.2" are meaningful things to say and ask questions about.

Couldn't you replace references like "the bottom of page 7" with others like "two sentences after theorem 1.2" that are layout-independent? This would also make it easier to rewrite parts of the paper without having to go back and fix all of your layout-dependent references when the layout shifts.

HTML has strong advantages for both paper and electronic reading, so I think it's worth making an effort to adopt.

When I print out a paper to take notes, the margins are usually too narrow for my note-taking, and I additionally have a preference for a narrow margin on one side and a wide margin on the other (on the same side, not alternating with page parity like a book), which virtually no paper has in its PDF representation. When I read a paper electronically, I want to eliminate pagination and read the entire thing as a single long page. Both of these things are significantly easier to do with HTML than LaTeX (and, in the case of the "eliminate pagination" case, I've never found a way to do it with LaTeX at all).

(also, in general, HTML is just far more flexible and accessible than PDF for most people to modify to suit their preferences - I think most on HN would agree with that)

gwern2• 2 years ago

I increasingly recommend against the Arxiv HTML version. I thought it had an acceptable start and they would fix the remaining problems and rapidly become on par with the PDF, but that seems to not be happening.

The HTML version is seriously buggy; and the worst part is, a lot of those bugs take the form of silently dropping or hiding content. It's bad enough when half the paper is gone, because at least you notice that quickly, but it'll also do things like silently drop sections or figures, and you won't realize that until you hit a reference like 'as discussed in Section 3.1' and you wonder how you missed that. I filed like 25 bugs on their HTML pages, concentrating on the really big issues (minor typographic & styling issues are too legion to try to report), and AFAIK, not a single one has been fixed in a year+. Whatever resources they're devoting to it, it's apparently totally inadequate to the task.

stogot• 2 years ago

I’m ok with the PDF but the title should be in HTML. The pdf failed to load for me due to tracker blockers (also why?!) so I was confused because there was no title but had comments

gradus_ad• 2 years ago

Ranking shouldn't be based on comment activity, which ranks controversial papers

But don't we want people's attention drawn to controversial/conversation generating papers? The whole point of the platform is to drive conversation

woodson• 2 years ago

The concern may be about what effect this will have on future papers (just like news headlines engineered for clickbait).

godelski7• 2 years ago

My guess is that they're trying to deal with a problem they're creating: being open to everyone. The problem with places like Twitter, Reddit, Github, HN, etc is that you don't know you're talking to a peer and the idiots asking irrelevant questions or proposing dumb things outnumber researchers a million to one. Even allowing public to upvote or affect the rankings is not beneficial to science.

I'm all for casting wide nets and making things available to everyone, but a little gate keeping is not bad (just don't gate keep by race, class, or those things). But I'm sorry, research is hard. There's a reason people spend decades researching things that at face value look trivial. Rabbit holes are everywhere and just because you don't know about them doesn't mean your opinion has equal weight.

We seriously lack areas where experts can talk to other experts.

runningmike• 2 years ago

Many people on earth have names with spaces. So good that a username can reflect a real name a person has.

rehaanahmad6• 2 years ago

Great idea, we'll look into making the home page the trending page soon.

Regarding HTMl, our original site actually only supported HTML (because it was easier to build an annotator for an HTML page). the issue is that a good ~25% of these papers don't render properly which pisses off a lot of academics. Academics spend a lot of time making their papers look nice for PDF, so when someone comes along and refactors their entire paper in HTML, not everyone is a fan.

That being said, I do think long term HTML makes a lot of sense for papers. It allows researchers to embed videos and other content (think, robotics papers!). At some point we do want to incorporate HTML papers back into the site (perhaps as a toggle).

DoctorOetker2• 2 years ago

I apologize for changing topic here:

Did you bulk download the arxiv metadata, PDF and or LaTeX files?

I am trying to figure out what the required space is for just the most recent version of the PDF's.

I can find mentions of the total size in their S3 bucket but unclear if that also includes older versions of the PDF's.

I also wonder if the Kaggle dataset is kept up to date since it states merely 1.7M articles instead of 2.4 I read elsewhere.

Edit: I just found the answers to my question here: https://info.arxiv.org/help/bulk_data_s3.html

ZeroSolstice• 2 years ago

The frontpage should directly show the list of papers, like with HN.

I disagree. There are numerous times where I have browsed the comments on a HN post where people haven't read the article and are just responding to the comment thread. The workflow for this seems a bit different in that a person would have already read a paper and wanted to read through existing discussions or respond to discussion. With that, having the search front and center would follow as the next steps for a person who read a paper and wanted to "search" for discussions related to that paper in particular.

HN is more an aimless browsing which is a bit different than researching a specific area or topic.

tinyhouse• 2 years ago

We obviously had this for many years with OpenReview, which has a different purpose, but having something for every paper is indeed needed. I have trouble opening some links, guessing it's still under heavy development. Looks nice!

dangoodmanUT• 2 years ago

advisors seem very biased to ML, Google, and Stanford

karencarits• 2 years ago

There is also https://pubpeer.com/

I worry that fragmentation of this space might not be beneficial, so it would be nice if these services could collaborate in some way, perhaps using activitypub or something

levocardia• 2 years ago

Agreed, pubpeer is already a very widely used platform in health and biology research. The PubPeer chrome extension is a must-have, in my mind, as it alerts you when a paper you find (even linked on some other website) has comments or has been retracted.

barmstrong• 2 years ago

Same with https://www.researchhub.com/

cs702• 2 years ago

How are the creators going to prevent gaming?

I ask because every system I've ever come across for discussing and ranking content without human moderation is always, sooner or later, gamed.

rehaanahmad6• 2 years ago

We have a team of enthusiastic reviewers/moderators in a couple sub-categories. We plan on growing this team out as the site continues to grow. If you'd like to be a reviewer: https://docs.google.com/forms/d/11ve-4cL0axTDcqnHF66zX6greFV...

tuxguy• 2 years ago

awesome honking idea ! please add "spaces" for biorxiv and medrxiv too !

fileyfood5002• 2 years ago

It would be amazing if bioRxiv and medRxiv were included! I was curious how bioRxiv compared in usage/size to arXiv, and dug through the stats on each site. Both sites report paper download stats, and arXiv is about 12x larger (~50+ million downloads/month vs 4 million downloads/month for bioRxiv).

Interestingly, the sites have grown at a similar rate. Going back to 2020, arXiv had 25+ million downloads/month and bioRxiv had ~2.1 million.

[1] https://arxiv.org/stats/monthly_downloads [2] https://api.biorxiv.org/reports/usage

fileyfood5002• 2 years ago

Adding submissions stats:

1. arXiv, ~21k/month

2. bioRxiv, ~5k/month

3. medRxiv, ~1.3k/month

It would be interesting to see usage of other _xiv's and other pre-print publishers!

[1] https://arxiv.org/stats/monthly_submissions [2] https://api.biorxiv.org/reports/content_summary [3] https://api.medrxiv.org/reports/content_summary

sundarurfriend• 2 years ago

I wish for either:

1) Zoom buttons just for the paper - the article text is often tiny, and zooming in with the browser messes up the page layout and makes the page practically unusable.

2) A simple direct button to download the PDF directly. This would alleviate the zoom problem since I can view it in my local PDF reader with the best settings for me. Having to go to arxiv to download the PDF for every paper would be a nuisance over time though, so a button in the top bar would make the experience a lot better.

AlexDragusin• 2 years ago

For me it always downloads the PDF, because I have disabled the View PDF in browser option (Toggle ON, on Edge: "Always download PDF files"), in browser settings, consider this as a solution.

Edit: The above is applicable to arxiv itself, I got confused, the alphaxiv.org opens the PDF in a framed way with no option to download, indeed.

rehaanahmad6• 2 years ago

Zoom is in the works! We are adding this in the coming week!

shayankh• 2 years ago

so cool!

abhayhegde4• 2 years ago

Great platform for invigorating research discussions! But seeing only AI based (or broadly CS based) research as featured papers is a bit discouraging. Perhaps there isn't enough critical mass for other topics yet.

cgshep2• 2 years ago

Use HTML rather then PDF.

Tenured prof here. Academics don't use HTML, despite its obvious advantages. The incentive system is deeply broken. No big-name journal or conference will accept a well-formatted HTML over their proprietary Latex/Word format. Latex to PDF converters generally suck.

elashri• 2 years ago

Arxiv already provide a HTML version of the articles [1]. The authors does not have to provide HTML version, it is converted by arxiv. i.e [2]

[1] https://info.arxiv.org/about/accessible_HTML.html

[2] https://arxiv.org/html/2409.00838v1

cgshep2• 2 years ago

Tenured prof here. Every paper of mine goes on Arxiv with no exceptions, published under CC BY-NC-ND licenses. Some of us are working hard to overcome the system (e.g. look at the IACR's efforts). Unfortunately, academics are still hindered by institutional inertia; in fact, many prefer the status quo, usually those who rely on prestige over actual quality to advance their careers.

michaelmior5• 2 years ago

usually those who rely on prestige over actual quality to advance their careers

Unfortunately for those of us pre-tenure, it's difficult to balance these as I'm sure you aware. We're evaluated by people who may have the best intentions, but don't work directly in our field. They then determine whether we keep our jobs. It's difficult not to consider prestige as a factor when you know those evaluating you will.

godelski7• 2 years ago

I cannot understand how people think that trying to evaluate people who are pushing the bounds of human knowledge is anything short of "a clusterfuck." I understand the appeal of metrics and prestige, as they provide some signal in an insanely noisy system, but to think they are perfectly aligned is ludicrous. Are we unwilling to just admit that the process is noisy? Is it really not okay that it is (and that we'll probably never escape this aspect)? Because I can't see us doing good science if we're not.

Ar-Curunir• 2 years ago

What do you mean by the IACR’s efforts here? In the crypto community it’s very much the norm to put everything on eprint, and it is very rare to find a crypto paper not on there

chipdart4• 2 years ago

(...) in fact, many prefer the status quo, usually those who rely on prestige over actual quality to advance their careers.

Your comment doesn't read like one from anyone with any relationship with academia. If you had, you'd know that the issue is not a vacuous "prestige" but funding being dependent on hard metrics such as impact factor, and in some cases with metrics being collected exclusively from a set of established peer-reviewed journals that must be whitelisted.

And ArXiv is not one of them.

This means that a big share of academia has their professional and future, as well as their institution's ability to raise funding, dependent on them publishing on a small set of non-open peer-reviewed journals.

Reading your post, you make it sound like anyone can just upload a random PDF to a random file server and call it a publication. That ain't it. If you fail to understand the problem, you certainly ain't the solution.

dguest• 2 years ago

I all fairness, I don't think the grandparent post disagrees with anything that is in the parent post here.

Yes, academia has tried to quantify prestige via impact factor and peer-reviewed journals. Yes, lots of people (even in Academia) feel that the system is being gamed, with by the publishing houses that own the journals being a common scapegoat.

The system isn't broken, but it also keeps its integrity through some dynamic tension: a bit of criticism is a good thing.

JadeNB• 2 years ago

And ArXiv is not one of them.

But putting your papers on the arXiv, as your parent said, doesn't mean you only put them on the arXiv. I put all my papers on the arXiv, but I also submit them for publication in journals that will help me make the case for funding and promotion.

BeetleB• 2 years ago

Your comment doesn't read like one from anyone with any relationship with academia.

Your comment reads likewise.

He didn't say he publishes them exclusively on Arxiv. It's quite common for professors to post it there as well as submit to journals. Many (most?) journals allow for it - they don't insist the ones in arxiv be taken down - as long as they're posting preprints and not the final (copyrighted) version.

As an academic, you should also know that practices vary widely with discipline. As an example:

dependent on them publishing on a small set of non-open peer-reviewed journals.

IIRC, NIH grants require publishing in open peer-reviewed journals.

Also, lots of disciplines are not heavily reliant on funding. In both universities I attended, the bulk of math professors did not even apply for grants! It's not required to get tenure (unlike engineering/physics). Also often true in some economics departments.

As an aside, your comment violates a number of HN guidelines.

godelski7• 2 years ago

  > the issue is not a vacuous "prestige" but funding being dependent on hard metrics such as impact factor

These things are not in contention.

There is no singular problem to be solved, which is why it is so difficult. No smoking gun.

  > And ArXiv is not one of them.

ArXiv has a large impact on metrics and so called impact factor. But let's also not be delusional to the fact that a paper from a prestigious institution will always receive more citations (or any other metric) than an equal quality paper from a less prestigious institution. All our metrics can be hacked through publicity.

  > Reading your post

Reading yours, it sounds like you stand in the way of resolving issues in academia. Not because you don't have issues with it that you want to solve, but because you have already found the answer.

Your comment reads like someone who has a relationship with academia.

parpfish5• 2 years ago

Tenured prof here.

Yeah, but every pre-tenure or postdoc is like “I can’t fight the system right now, I need to publish enough to still have a job two years from now”

DoctorOetker2• 2 years ago

helpful would be cheaper equipment and tools used in research, and unrestricted popular access to scientific literature

gigatexal• 2 years ago

Thank you, thank you, thank you! I've no skin in the game (not an academic and a math idiot but I've a hole in my heart for Aaron Swartz and what he stood for) but I love that there are professors like yourself that believe in the free sharing of knowledge.

godelski7• 2 years ago

  > Unfortunately, academics are still hindered by institutional inertia

As an ABD this has been a real pain point for me. Maybe I came into academia thinking what mattered most was the research. But now I'm the stereotypical PhD who passionately despises academia for its lack of being academic. I'm happy to have competition, but at the end of the day are we all not on the same team?

How the hell did we create a system where it is the norm that an advisor does not read a thesis, to read papers, to mentor? For that to be the norm among a committee? When I've had issues with getting works getting through review (even when they have high citations due to arxiv) I don't understand why it's acceptable for a response to be "keep trying" instead of "here, I read the paper and reviewer responses, let me help"[0]. It seems inefficient that we throw students into the deep end and watch them sink or learn to swim. I think there'd be a lot fewer dejected PhD students if there was a stronger focus on academics, mentorship, and collaboration over churning out ̶w̶i̶d̶g̶e̶t̶s̶ papers.

I think what pisses me off the most is thinking that research significance and success can be measured __purely__ through metrics like citation counts, H-indices, i10's, awards, etc. I'm not saying those are useless, but that we can evaluate without looking at the content? (as you say, actual quality of work) It's like we learned about Goodhart's Law and decided it was a feature not a bug.

(I know this is not always the case and there are many amazing advisors, but I'd be impressed if someone didn't know this is happening at least somewhere within their department.)

[0] If it takes a village to raise a child, it takes a department to mint a PhD. These types of things should come from committees, not just advisors. Our annual meetings and review shouldn't just be going through the motions.

gr__or• 2 years ago

I am very non-eager to help any further platform grow that has not been built on-top of sth like atproto (the BlueSky protocol), to prevent silos and the monopolist landlords that come with those.

Great idea though, would love to use sth like this, if it existed on a federalized protocol.

celdon253• 2 years ago

Can’t please everyone.

chfritz• 2 years ago

Why limit this to arxiv papers? Why not any paper published online, e.g., via https://bibbase.org? btw, very cool that you seem to have overcome the initial inertia of getting something like this going. The idea is not new, but it's a marketplace dynamic that is hard to bootstrap.

rehaanahmad6• 2 years ago

One of the co-creators of this site. A lot of great suggestions I'm reading so far, a lot of them are currently in the works (zooming in/out, infra issues for slow loading times on some papers, google scholar claiming papers).

For some more context, we are a group of 3 students with a background in AI research, and this site was initially built as an internal tool to discuss ai papers at Stanford. We've been dealing with a lot of growing pains/infra issues over the past month that we are in the process of hashing out. From there we would love to make a more concerted effort to share this in areas outside of AI. Happy to hear your thoughts here, or more formally via contact@alphaxiv.org.

I do want to highlight, our site has a team of reviewers/moderators and having folks from different subject areas is critical to making sure the site doesn't end up a cesspool, apply here: https://docs.google.com/forms/d/11ve-4cL0axTDcqnHF66zX6greFV....

musicale2• 2 years ago

Moderation is typically the thing that doesn't scale. I am not sure it's a solvable problem (see reddit, stackoverflow, youtube, quora, etc. for negative examples and anti-patterns.) Often sites start out great and then degrade when they become popular.

My main recommendation was going to be organizational: to cooperate and work with arXiv itself, rather than risking a potentially adversarial or competitive relationship.

Now that I think about it however, I am convinced by a peer comment that was basically "leave arxiv the way it is and don't mess it up." So carry on then.

bawolff• 2 years ago

In the search field, it would be kind of cool to list how many comments each paper has - e.g. if you want to find the most discussed papers on some topic.

tintor• 2 years ago

Too much annoying visual clutter on the discussion page, unlike Hacker News.

john-titor2• 2 years ago

Tried to sign up with my corporate email (life sciences, 100k+ employees worldwide with a big research arm). Says the institution is not known to the service. What's the process to get it known?

rehaanahmad6• 2 years ago

Email me at contact@alphaxiv.org, I'll add it asap!

john-titor2• 2 years ago

Thanks a lot, will do!

parpfish5• 2 years ago

Just had an idea that may help the moderation AND encourage higher levels of discourse — comments are not published immediately.

When I was doing peer reviews, it would often take a day or more to read a paper, think it through, and then write up something thoughtful and constructive.

If you introduce a mechanism to delay comments (eg, holding all messages for 24-72 hours before publishing or only releasing new comments on Monday mornings) it would:

- encourage commenters to write longer thoughtful responses rather than short quick comment threads

- reduce back and forth flame wars

- ease the burden on moderators and give them time to do batches of work

- see if multiple commenters come to the same conclusions/critiques to minimize bandwagon effects

data_maan2• 2 years ago

What I don't like about this is that they had to build a separate system.

Why wasn't it possible to contact arXiv and do this in collaboration with them?

cma• 2 years ago

Seems like it could just fill the role of the exploitative publishers if it takes off. Community does the work (scientists volunteer peer review), site gets the profit and locks everyone in due to network effects of being the hub for discussion. Eventually starts charging for boosting your paper with guaranteed return in citations etc. I'm just assuming it is venture backed due to having a team of advisors and stuff and about page being LinkedIn but sorry if I'm off on that and it is nonprofit or something.

data_maan2• 2 years ago

This is precisely my worry!

While it says "a Stanford project" at the top, this could mean anything. It could in particular allude to the fact that Stanford professors are advisor.

The advisors are well-known professors, but some, like Sebastian Thrun, have an entrepreneurial background, so probably a company sits behind this.

For me, the entire project gets a NO CONFIDENCE vote because I can't tell who they are, who pays for moderation (which they claim to do etc.).

I'd feel much more at ease if this was done by arXiv, or at least endorsed by arXiv.

godelski7• 2 years ago

Nice idea but I dislike the implementation. Honestly, I very much like OpenReview and question why we don't use that?

OpenReview has: - preprints - versioning - reviewing, with threads and latex support - ability to link websites, repos, datasets, etc - bibtex generation

But it's not as popular as arxiv, though very popular in review (conferences often do not use full features)

One thing I dislike about this is that it is open to all. Arxiv doesn't have a hard filter (you just need someone with an account to vet you, which stakes their reputation), but the existence of a filter is critical.

I don't want a place to engage with the public. We have Reddit, hacker news, Twitter, mastodon, and countless other places. I want a place for academics to talk to academics [0], researchers to researchers. There is a serious lack of spaces where serious low level in-the-weeds discussions can happen. Even fucking GitHub and hugging face are swamped by people asking dumb questions on research projects like how to install pytorch, fine tune a model, or where the source code is.

I'm really happy to include a lot of people, but I think we also need spaces where experts know they're talking to peers. Without that you have to assume you're not talking to peers because they outnumber us a few thousand to one. So that doesn't encourage engaging in research or technical discussions, it encourages talking down to peers and misinterpreting.

[0] a degree is not what makes you "an academic"

SubiculumCode• 2 years ago

This is cool. Seriously, I mean it. In too many field, the discussion of papers is so ephemeral, and this can really do something cool. ...but... I sure hope that no one submits a fake paper made by chatbots, then use chatbots to discuss the paper on alphaxiv.org ad nauseum to get it trending.

Which means they will. Sciencing is hard, especially because gatekeeping is made necessary to keep the spammers (the ones that flood my email with fake journal offers) at bay.

teleforce• 2 years ago

It a shame that arXiv now it is not what it used to be, a very useful pre-print before actual publication. It looks like it is now a pseudo journal masquerading as a pre-print server since apparently arXiv has editorial and review teams that reject papers based on their 'expertises' [1].

Perhaps they think they are reputable now just because Perelman's proof papers were published there and they want to maintain their 'reputation' [2]. The irony is that Perelman would most probably not publish in arXiv if it is in their current pseudo journal status.

[1] Editorial Advisory Council:

https://info.arxiv.org/about/people/editorial_advisory_counc...

[2] Reclusive mathematician rejected honors for solving 100-year-old math problem, but he relied on Cornell's arXiv to publish:

https://news.cornell.edu/stories/2006/09/proof-100-year-old-...

hfhan• 2 years ago

I’d love to see this being adopted by research community. Right now these discussions are spread out EVERYWHERE, twitter, LinkedIn, OpenReview, etc.

alphaXiv: Open research discussion on top of arXiv

Related Stories