Mendeley encrypts users' database after Zotero provides an importer

https://www.zotero.org/support/kb/mendeley_import

313 119 8 years ago

Luckily, my Linux desktop was still running an old version of Mendeley so I was able to make the switch.

My reason for wanting to switch was that Zotero has Google Docs integration. After making the switch, I was pleased to pay a modest amount for storage of my PDFs, which makes me feel like a customer instead of a product.

leemailll5• 8 years ago

If your institution does offer f1000 subscription, I suggest you take a look at f1000 workspace.

carbocation5• 8 years ago

Thanks for the recommendation. The features look interesting, and it supports my basic requirement (adding citations to Google Docs).

black_puppydog3• 8 years ago

Apart from it being domain specific (according to wikipedia) it is also not open source, so recommending this as a reaction to the Mendeley news is... words fail me... :|

andrepd2• 8 years ago

Hmm, I thought this would be a lesson to illustrate what happens when you depend on proprietary software...

ramraj073• 8 years ago

This is literally the worst thing you can do. Even a tenured professor doesn't seem to stay in the same institution for their lifetime nowadays so why would you go to a service that even you personally cannot pay for yourself (reasonably)?

boyband666611• 8 years ago

I gave this a try, but sadly it came up short. For writing a single paper of MSc Thesis, it might work. However it just doesn't seem suited for handling a large number of references for a variety of projects, which get called on frequently as standard references in papers, reference material, and much more.

If it helps I have just under 2000 papers in my Zotero. It isn't massive by research standards, but it is substantial.

justinclift• 8 years ago

Reading through that, it seems like Mendeley uses SQLite for it's local database.

There aren't that many SQLite encryption libraries around (generalising), so it's probably using either SQLCipher or SEE.

https://www.zetetic.net/sqlcipher

https://www.sqlite.org/see/doc/trunk/www/index.wiki

There's a reasonable chance the page size and passphrase is stored or cached on the machine. If someone (not me) takes the time and effort to trace through the application, it should be feasible to figure out what's going on. :)

shittyadmin3• 8 years ago

Yeah, this sounds like an interesting reverse engineering challenge, do you need any special accounts to use Mendeley or can it be freely installed by anyone? It'd likely just be a hard-coded key, you wouldn't want to restrict people from copying their own databases around... probably.

dunham• 8 years ago

I did this a while back with dropbox (I was curious about their database structure).

I ended up adapting some code that scans an OSX process for AES key schedules and prints any keys that it finds (caveat - sqlite3 rolls its own AES and the key schedule ends up byte-swapped in memory).

I also had to write some Go code to decrypt the sqlite3 database file. There is code to read an encrypted database buried in sqlite3's revision history, but it's an old enough version of sqlite3 that it didn't understand dropbox's schema.

Lord_Nightmare• 8 years ago

They're using SEE, see https://eighty-twenty.org/2018/06/13/mendeley-encrypted-db

shittyadmin3• 8 years ago

Unfortunately here's the catch - while it's probably trivial to reverse engineer enough to extract the key for the SEE database, it means that Zotero would need an SEE license to be able to handle the file still.

As an alternative it should be quite possible to produce a hook using Frida or similar that would disable database encryption on the Mendeley side so it can be imported through the usual channels. Not pretty though.

pjc50• 8 years ago

Ah, Elsevier.

It's probably not that difficult to reverse-engineer and extract the decryption keys, but doing so opens you up to DMCA risks.

lozenge• 8 years ago

I don't think the DMCA would apply, as presumably you own the copyright of whatever's in the database. You would need EFF or other support to prove it though...

black_puppydog3• 8 years ago

On the other hand, it's gonna be hard to even detect if you just use it to import your collection to Zotero.

pygy_• 8 years ago

Portability is explicitly mentioned as an exception to the "no RE" clauses of the DMCA, so I wouldn't expect any charge to stick, but I don't expect that fact to deter Elsevier from suing anyway, out of spite.

zwaps8• 8 years ago

Zotero has improved a lot, while Mendeley has repeatedly regressed.

Mendeley used to be quite a good program, but recently you can not export annotated PDFs meaningfully. For example, sending a folder of annotated PDFs to a co-author during a literature review is impossible. This is obviously the case since Elsevier does not want you to trade research papers, whether you have lawfull access or not.

The updates that took away features were silent. What happened to me some time ago was these updates occured during a high-stress phase with a short deadline until conference submissions (if you are a researcher, you know what I mean).

I had used Mendeley for years to annotate and categorize literature. I was now in need to send my categorized PDFs to a central repository for my co-authors to evaluate and add to. After some update, without me noticing it, it was no longer possible to export folders of PDFs or PDFs in general!

I had everything in Mendeley, weeks of work. I was completely f'ed - deadline approaching. I had to re-aquire all PDFs and go through all annotations by hand.

ELSEVIER IS SIMPLY ANTI SCIENCE. Collaboration is a key in science. Sharing results, research and literature is crucial.

Mendeley makes this impossible. It does NOT allow you to fully access your own work!

So in conclusion, USE ZOTERO. It's good now, better than before. You can use a PDF reader with annotations to open and save the PDF and Zotero will keep those annotations. You can export Bibliographies, including notes AND files. You can not do that with Mendeley.

So again, as a researcher, I emplore you to drop Mendeley completely, as I have done.

Thank you.

walrus01• 8 years ago

Elsevier is to science as Oracle is to database software licensing.

mdip3• 8 years ago

Thank you for this. As someone who does not make a habit of hanging around academia, I have very little experience with Elsevier short of what I read here.

However, as someone who worked in a large IT organization for a huge company and wrote software for license compliance tracking, I completely understand. Grab me a pitch fork, I'll march.

AnaniasAnanas• 8 years ago

I had used Mendeley for years to annotate and categorize literature. I was now in need to send my categorized PDFs to a central repository for my co-authors to evaluate and add to. After some update, without me noticing it, it was no longer possible to export folders of PDFs or PDFs in general!

This is exactly why you should not depend on proprietary software for anything even remotely important. This is not the first nor will it be the last time that something like that happens. Letting a company (or an individual) dictate how and if I can access my work is unacceptable to me.

Thankfully Zotero is FOSS, I will be staying with BibTeX though.

andrepd2• 8 years ago

The adage is certainly true: "with proprietary software, the user doesn't control the program, the program controls the users".

lou1306• 8 years ago

Personal experience: I made the switch soon after starting my PhD and never looked back. Also, their Connector plugin for Firefox is a life saver. 90% of the time I only have to lookup the title of a paper on DBLP, then export it as BibTeX and pronto, the reference is now in my library with most of the relevant metadata. Much better than Mendeley's quite opinionated plugin (which also won't work unless you log into Elsevier's servers)

zwaps8• 8 years ago

I never used the Mendeley plugin, but I imagine it must be pretty horrible to store PDFs from all the convoluted sources and proxy setups or library file-servers that universities use. It probably only works well if you logged into your Elsevier(tm) account.

In any case, the Zotero plugin works great and just saves Metadata and PDFs, as it should. For the usual preprint or paper providers, it works perfectly.

So even here, using Mendeley is just bad.

zwaps8• 8 years ago

Also, you may remember me being hesitant to switch to Zotero because in the past it lacked the ability to meaningfully edit saved citations and files, in particular, it would not detect from PDFs for files that you had already downloaded. It worked only well with the plugin, getting the file "fresh" from the publisher.

Mendeley's advantage was that you could just drag and drop a pdf in and it would add meta-data.

I am happy to report that Zotero does this as well now. And it works quite well, sometimes better then Mendeley.

So if that was your reason, like it was for me before I was hit with the "no export disaster", go ahead and switch.

Let Mendeley die the lonely death it deserves.

boyband666611• 8 years ago

Zotero has improved a lot, though sadly it still has a way to go on the user experience of Mendeley. My main gripes are

- The way that it still doesn't play nice with cloud services (syncing the directory and its just a matter of time until you get database corruption. It takes a lot of wonky setting up to get it to kind of work, which just shouldn't be the case

- The lack of developers and thus slow pace of improvement. I'm a researcher not a programmer - which I think describes most people using it. That means unfortunately we are reliant on one or two volunteers to improve the product. The pace of improvement is slow, and theres also no way to meaningfully advance it - be that through offering bounties for someone to implement certain features, just inputing lists of bugs/feature requests (the list is already v long, and doesn't move much), or anything else.

It's a really good bit of software (and I don't want to sound ungrateful), I just know it still has a lot of quirks. This means it can't always do what you want, and it isn't an easy obvious choice for new researchers - Mendeley is certainly more familiar and easier to use.

zwaps8• 8 years ago

It is a good idea to get some Zotero cloud storage. The sync works perfectly for me. Since any serious library will be over the Mendeley free limit, I think it's a fair comparision.

Of course Zoteros development is less funded and less agile. Given that, I think they have worked on many shortcomings. The interface is now good, the group-based sharing works, PDFs are read and meta-data is added well. The import plugin is better than that of Mendeley.

Mendeley has an advantage in that it has a great PDF viewer and editor. But since you can not do anything with these PDFs and annotations, like export them or send them anywhere, it's now pretty much useless.

Switching is not easy, but in the long run I don't think you'd be faced with much issues going from Mendeley to Zotero. It was certainly worth it for me.

As it stands, you can still import from Mendeley to Zotero, so I'd at least do that now, until Elsevier finds a way to shut this down completely.

Libbum2• 8 years ago

What cloud service are you attempting to connect to? I'm using my universities' box.com account - which generally has pretty appalling support (on linux at least). Zotero has managed this flawlessly.

montalbano• 8 years ago

The way that it still doesn't play nice with cloud services (syncing the directory and its just a matter of time until you get database corruption. It takes a lot of wonky setting up to get it to kind of work, which just shouldn't be the case

I'm using Zotero's $20/year for 2GB space membership and it was quick and easy to set up. Works great too.

dstillman• 8 years ago

The lack of developers and thus slow pace of improvement [...] we are reliant on one or two volunteers to improve the product

I'm not sure why you have that impression. Zotero has amazing, invaluable volunteers, but there's a paid, full-time dev team working on Zotero every day. In the last year, we've added:

- Google Docs integration [1]

- Unpaywall integration [2]

- A new, greatly improved PDF recognition system [3]

- Faster citing in large documents [3]

- A much more powerful saving interface [4]

- Mendeley import...

- ZoteroBib, a free web service for generating bibliographies [5]

- A barcode scanner for iOS [6]

- Regular updates and bug fixes [7]

[1] https://www.zotero.org/blog/google-docs-integration/

[2] https://www.zotero.org/blog/improved-pdf-retrieval-with-unpa...

[3] https://www.zotero.org/blog/zotero-5-0-36/

[4] https://twitter.com/zotero/status/991052142717886464

[5] https://www.zotero.org/blog/introducing-zoterobib/

[6] https://www.zotero.org/blog/scan-books-into-zotero-from-your...

[7] https://www.zotero.org/support/changelog

(Disclosure: Zotero developer)

burtonator5• 8 years ago

I have an app similar to Zotero (Polar) and we don't sync with cloud services either.

It's a bit of an anti-feature. We can't really control the data store if you're monkeying around with it under the hood.

Polar already supports cloud sync so we encourage users to use that. Same with Zotero I imagine.

- The lack of developers and thus slow pace of improvement.

Do you mean the dev pace of Zotero is slow? I wonder if their legacy infra is slowing them down.

bocklund3• 8 years ago

but recently you can not export annotated PDFs meaningfully

FWIW, I think they brought back exporting annotated PDFs in the most recent version, 1.19.3

At least maybe there is hope for exporting your work to Zotero.

greeneggs• 8 years ago

How easy is it to leave Zotero? (Either to move to a different paper manager, e.g., Mendeley, or to leave the paper manager ecosystem entirely.)

Can Zotero respect my filesystem organization (à la Lightroom)? I have thousands of papers organized, and don't want to move them all in (too much work, what if I don't like it?), but I also don't want to have two copies of some subset of the papers, which might not be in sync.

j88439h84• 8 years ago

Yeah it can do that, with the use of some plugins.

burtonator5• 8 years ago

Can't comment on Zotero but in Polar we have a similar issue where we need to move over the PDFs. We're going to start using hard links so at least if it's on the same drive you can have the same copy just in two different places.

natechols• 8 years ago

ELSEVIER IS SIMPLY ANTI SCIENCE.

+1. I worked in publicly-funded research labs for 15 years and there is no single organization I despise as much as Elsevier - only Springer-NPG comes close. If the company went bankrupt tomorrow it would be a great day for science.

burtonator5• 8 years ago

I've been looking at this too. I have a new app I've been working on which is similar to both Zotero and Mendeley.

https://getpolarized.io/

It looks like they're "encrypting" by saying that there is some sort of GDPR requirement when in reality it's more plausible that they're trying to lock in users.

Vinnl• 8 years ago

Note that this has been happening a while, given that there's submissions [1] [2] from 7 months ago.

[1] https://news.ycombinator.com/item?id=17302019

[2] https://news.ycombinator.com/item?id=17433880

mwexler• 8 years ago

For those wondering, here's what I gathered as some context.

Zotero = Your personal research assistant. Zotero is a free, easy-to-use tool to help you collect, organize, cite, and share research. https://www.zotero.org/

Mendeley = Reference Management Software, produced by Elsevier who also happens to be the publisher of many peer-reviewed journals. Elsevier come under fire for it's high costs and gateway actions to restrict access to information they've published in journals and host in archives. This most recent action of making the database of references in Mendeley difficult to export is a continuation of their attempt to protect what they, and some legal systems, would see as their IP. Others disagree.

The battle continues...

carbocation5• 8 years ago

Good idea re: giving context as to what these programs are. For most of us who do research, these programs are essentially interchangeable "reference managers." Here's what I mean:

I read a scientific paper or look up a citation. I add that with a click or two to my reference manager. It also stores the PDF for me.

In the future, I can easily re-read the PDF. I can annotate it and the annotations will be stored.

Critically, when writing my own paper, I can import those citations and trivially change the format to whatever the publisher wants without any effort.

To me, across most features, these two programs do exactly the same thing. When picking a citation manager, it's more about which one I trust will be around for the long haul and will not interfere with my research.

zwaps8• 8 years ago

The only difference is that Mendeley has a built-in PDF viewer with annotation. Of course since you can no longer share these annotations, nor export the pdf, that's all pointless.

Mendeley opens PDFs externally, but many PDF readers can create and save annotations.

boyband666611• 8 years ago

I think you meant to say "when my paper is rejected, I can trivially change the format of references to whatever the publisher of my second choice journal wants without any effort" ;-)

ramraj073• 8 years ago

Mendeley was originally a startup. Precisely why people even started embracing it (though I didn't, something always felt fishy for me) and then they got acquired by Elseiver.

I have nothing but good things to say about about Papers for Mac though.

biomcgary• 8 years ago

I've never used Papers, but have had colleagues that swear by it. However, it is not open source, so you still run the risk of the developers selling out just like Mendeley.

zwaps8• 8 years ago

Let me try to explain.

Mendeley and Zotero basically do the same thing, which is categorizing literature - specifically research papers. Mendeley offers a built-in PDF viewer with annotations tools, Zotero relies on external viewers for that. Both offer cloud storage.

Researchers usually have to categorize and scan over huge amount of prior literature. If you are working on a project, you usually want to have a good library or folder structure detailing relevant literature. When writing your paper, you also want to be able to cite from this library quickly, via BibTex for example.

Researchers maintain a personal library with literature they read, plan to read or have cited. When researchers collaborate, they need to merge these libraries. Since tools differ, the lowest common demoninator is often a dropbox somewhere. Sometimes everyone works on Zotero or Mendeley, such that files can be shared within the system. Usually not.

Mendeley and Zotero used to be equally open to sharing and collaborating. Mendeley was in a sense superior as it was polished, had great tools to annotate and great tools to work with Meta-data. Zotero, while FOSS, was always behind.

Then, Mendeley was bought by Elsevier, the largest publisher. Elsevier does not want people to share PDFs, because Elsevier wants everyone to pay for the priviledge to download those pdfs. Thus, Mendeley started to make it more and more difficult to collaborate. Now it is even difficult to share files within the Mendeley eco-system!

Perhaps you are using Calibre for you Ebooks. Now imagine Calibre would deactivate any way for you to view, send, export or use your files (the files that YOU put into it) outside of the Calibre Ebook viewer, and it would encrypt its database so that you would not even try. That's what happened.

Sharing research is the lifeblood of science, and Elsevier wants to destroy it.

Elsevier has done many other things that has harmed scientific progress - the majority of this undertaking you fund with you tax dollars! The way this is done is almost comically blatant. Elsevier acts like a comicbook villain. They are literally evil.

fantasticfears2• 7 years ago

Do you try tools like Scholia building bib file which is using Wikidta? https://tools.wmflabs.org/scholia/about

mdip3• 8 years ago

If I'm understanding things correctly from the reading (which, let's be honest, that's an assumption on its own), the tl;dr seems to be that Mendeley heard about an open source product working on a set of importers that would, potentially, result in Mendeley customers migrating to this free alternative[0]. To provide cover for this change, they invoked user security.

And hey, user security is a pretty good reason to encrypt a database, but if security were the only aim, they could have made the data accessible for export purposes (after all, it's accessible for the user to read).

At this point in history I'm not even sure it's worth going into all of the reasons this is a terrible thing to do on their part. At best, they'll scare off some users from migrating away briefly and buy themselves some time to figure out what they need to do to produce something that's better than the competition they're attempting thwart. More likely, they'll simply fail (quickly or slowly). It's that last point that really matters to me. If I'm buying an application that I'm going to rely on, I want comfort that the company will continue to maintain the product -- or in the case of open-source, confidence that the product is still regularly maintained (or in languages/frameworks that make it practical for me to maintain myself).

Aside from the fact that I wouldn't want that feature 'as it has been designed, today', I wouldn't want to rely on a product that's being designed by individuals who aren't well versed in the perils that DRM-like behavior causes. After all, that's what this is -- it's an attempt to keep the user from transforming their own data for the purposes of locking that customer into the product while attempting to position the lock-in as a security feature. And it failed on both accounts, Zotero worked around the encryption (which means the bad guys can to) and if people looking for a product like this care about that capability, they'll likely refine it to the point that it's as capable encrypted as it is decrypted.

Every time I've been asked to implement a DRM solution I've either outright refused or paired it down so much that the "R" didn't really exist. They're horribly complex to write, off the shelf solutions are already cracked and time would be better spent giving customers something they're willing to pay for. The thing that makes me roll my eyes every time I read one of these is "there are still engineers out there who think this is a good idea?"

I can't wrap my head around what causes this kind of thinking -- it's not a zero-sum game, there's room for more than one product in the marketplace -- even if one of them is free and open-source. Trying to beat the competition by sabotage rather than by making a better product doesn't work ... at least not for very long.

[0] And they also do my least favorite "other marketing thing" -- "Create a Free Account" and no hint of what the thing costs in any obvious place. So maybe this other product is free? I doubt it -- Like most companies, I'm assuming they're perfectly happy collecting your personal information for free (in order to send marketing to you to get you to pay for whatever service is tied to the free account). It's the little deceptive things that drive me crazy -- you didn't trick me, you only managed to make me suspicious of your product.

[1] I've gotten lucky ... usually budget and time doesn't allow for wasting energy on customer-hostile features that won't achieve the ends they're aiming for. The only case where something "DRM-like" was in an application I wrote was to be the opposite of customer-hostile. We had an app that provided information to users based on their permissions, determined by them logging in. I pushed to make the automation (sending us the login information) opt-in, with manual provisioning options. It worked well, since if they failed to true up after months of warnings, the application would deactivate (due to the size of the customers, months of warning was somewhere near a year and could also be disabled with a flag).

mdip3• 8 years ago

...I was going to edit my original comment, but it seemed more appropriate to just follow up with a reply...

I was unaware that this was an Elsevier product (another comment pointed that out).

I do not use Elsevier products (directly, anyway, maybe I do indirectly but I doubt it). I have, however, followed the various controversies surrounding the company for the past several years. Those who are with me will not be surprised that Mendeley added this sort of "feature".

Elsevier is not in the business of providing a helpful, useful, research library and tooling. Their business model is to be the only source of that information. This has been done using a combination of legal, contractual and technical means. I won't get into the whole ethical/legal argument about whether or not they should be allowed to do some of the legal/contractual things that they do to "pseudo-control" that data[0].

This smells like a feature borne out of corporate culture. Their business really is about exerting control over their customers, so it's natural (to them) that tool their customers rely on would similarly contain features related to keeping customers in the walled garden[1].

[0] Especially research done via grants paid for with public funds) -- I suspect I'd enter an echo chamber pretty quickly (and even if not, I'd be unlikely to persuade someone in a medium-length comment.

[1] There's some clever pun here about paper walls and research papers but it's too early to be that clever.

boyband666611• 8 years ago

So it was not developed originally as an Elsevier product, they bought it around 2013 I believe. By all accounts until this, they've also been fairly faithful with the way it's been managed and updated. It is certainly more shiny than Zotero, but I was too nervous about the potential for them doing something like this that I chose zotero (I was moving away from EndNote at the time, which is just too quirky).

I hate being right about such things. This is exactly what the fear was when they bought what was a fantastic piece of software, and doing really interesting things for researchers.

zwaps8• 8 years ago

It's not so much the lock-in issue, or the migration issue for me. I can pay for a good service, Mendeley used to be that, cause this is my job.

The issue is that due to Elsevier, a DRM logic was implemented that just doesn't make sense for scientists. The idea is that I can tell you about a paper, but you have to download it yourself because I can not send it to you. I can annotate the PDF, but I can not send you the annotated PDF. We have to look at the same screen to see what I wrote.

That end result means that Mendeley is no longer a good product. It makes scientific collaboration difficult to impossible. Sharing of files, ideas, results and writing is such a basic thing in todays world, that you should really be surprised how a company would think that this is a good idea, especially for scientists who almost always collaborate in loose teams.

But that is was happened, because Elsevier only makes money if the only place you can get any PDFs is their DRM walled garden.

So Mendeley is now bad. And NOW it matters that you can not get your data out of it. And the thing is, these annotations and folder categories, tags and meta-data - that's all specific to one researcher. It can not be replicated. It could be years of work in that database, and Mendeley holds it hostage.

masnick6• 8 years ago

For anyone who is interested in the different options for reference management, I've been sporadically maintaining a webpage about the various options since 2015: https://maxmasnick.com/projects/reference-managers/

I re-un-recommended Mendeley back in June 2018 when this news first broke. (I was initially too sketched out by Elsevier to recommend Mendeley, and this just confirms these suspicions.)

tl;dr Zotero is a pretty good bet for most people, especially since they added support for citation management in Google Docs at the end of last year (very important for academic writers). PaperPile (https://paperpile.com) and Papers (https://www.papersapp.com) are also worth checking out.

ramraj073• 8 years ago

Which one gives the best seamless mobile integration experience? Papers for iOS wasn't bad (if you let it finish syncing with Dropbox ) but curious about this!

masnick6• 8 years ago

Probably Papers is your best bet at this point.

Zotero will let you send PDFs to/from a tablet with the ZotFile (http://zotfile.com) plugin. This works well if your workflow is (1) find PDF on your computer; (2) read on tablet. But if you want to do anything else -- even choosing a PDF to read from your tablet without touching your computer -- then Zotero won't work. With that said, I know Zotero is working on mobile apps...not sure how far they have come though.

If you're Mac/iOS only, also check out https://www.sonnysoftware.com/bookends/bookends.html.

Cenk• 8 years ago

Very informative post, thanks. I’d love to hear your impression of Citationsy (closer to CiteThisForMe or RefMe in functionality than Zotero it EndNote).

masnick6• 8 years ago

Never heard of it before!

Looking at their website, it does seem like Citationsy is somewhere in-between a full-blown reference management application and one-off bibliography creation applications.

What Citationsy does not appear to do is manage inline citations within a Word or Google Docs document. Zotero will let you hit a keyboard shortcut, search for a reference, and insert a citation right in a paragraph (e.g. "(Smith, 2001)") . It then takes care of also adding that citation to the bibliography at the bottom of the document, and keeping the numbering in sync (if your citation style uses numbering).

I would only use a reference manager that has a word processor plugin for inline citation management -- and it doesn't look like Citationsy does.

To quickly build a bibliography, or just get a properly formatted citation for a single paper, I like https://zbib.org (also from Zotero).

Edit: clever, clever, parent poster is the creator of Citationsy. Hello! For more backstory on Citationsy see https://blog.prototypr.io/on-citationsy-4e143bbafc04. Sounds like Cenk has good taste (mentions iA Writer as inspiration). I'll be exploring Citationsy more.

erreJulian3• 8 years ago

I don’t remember ever visiting your site but it’s great. Thanks!

__jl__• 8 years ago

For Zotero users: Try the add on zotfile (zotfile.com). It allows you to extract PDF annotations and save them as notes in Zotero (among other things). Full disclosure: I am the developer.

esalman• 8 years ago

I love Zotero. Thanks.

masnick6• 8 years ago

zotfile is a must-have for Zotero! Thank you for making it.

burtonator5• 8 years ago

Hey.. I tried to reach out to you at your personal email address regarding collaboration between Zotfile and Polar and I never heard back.

https://getpolarized.io/

The idea was to have Polar also work with Zotfile since a lot of people use Zotfile.

boyband666611• 8 years ago

Genuine question - why make zotfile and not adding things to Zotero directly? After all it is open source. Zotfile adds a lot of useful things that feel like they should be in the main program, not optional extras.

I'm not a programmer however, so this may be very naive question!

12fro• 8 years ago

Another very grateful Zotfile user who wants to say thank you for your hard work. I use it every day.

minosniu• 7 years ago

Second this. I use zotfile to relocate all my PDFs into a single folder, which is Dropbox-synced. This works like a breeze for 2000+ and mounting papers.

masteranza• 8 years ago

Yep, if you're still using Mendeley, you should move to Zotero immediately. I have used Papers app for some time, but they've worked on the desktop app way too long after their merge with ReadCube and now I'm happy with my free Zotero. I've just tested ReadCube Papers with a simple pdf and it's nice and fast, but cannot resolve the metadata (even the suggestions are wrong) while Zotero can do it unambiguously on the first try.

jjoonathan2• 8 years ago

Mendeley also snitches evidence of your SciHub habit to Elsevier. It probably won't amount to anything, but do you really want to bet against Elsevier becoming exceptionally greedy, desperate, or both?

aw3c22• 8 years ago

Please provide some citations for that claim.

eythian• 8 years ago

They would, but they can't export them.

SubiculumCode• 8 years ago

I presume that the source of the file is in the metadata..it is often placed in references. Since those are uploaded to Mendeley in their cloud storage service, it would seem that thry have the data..now it comes down to eulas

jjoonathan2• 8 years ago

It uploads your PDFs to Elsevier servers for storage. By design.

Asking how that could lead to Elsevier obtaining evidence of paper piracy is like asking how using gmail could lead to google using your email to do targeted advertising. You are giving them everything they need to do it, so of course they might do it! The position that needs defending is the supposition that they can't.

phren0logy2• 8 years ago

To anyone looking for an alternative, I'd just throw out there that I've been very happy with ReadCube. It's proprietary subscription software, but that's a trade-off I can live with for a great browser plugin, mobile apps with good annotation, cloud storage of annotated articles, seamless syncing across devices, and a high-quality Word citation plugin that works on the Mac.

This company bought Papers, the Mac app, and it looks like they are finally getting ready to release the desktop client based on that acquisition.

krull10• 8 years ago

I've been looking for a replacement for Papers for years; it just doesn't work well anymore with the lack of attention they've given it since the acquisition. For me subscription pricing is a no-go. It's sad, I would happily pay for a new standalone Papers...

veddox2• 8 years ago

I used ReadCube several years ago when it was still really new (and free). Great program, very pretty interface - but it didn't work on Linux. When I switched from Windows, I had to let it go.

phren0logy2• 8 years ago

I mostly use the web version, and that seems to function well everywhere these days. Not sure about the desktop version, but after ages and ages they are finally getting the desktop application that incorporates their Papers acquisition, so it might be worth looking again if you like it. Hopefully, that does a better job on Linux, but that's always a gamble.

pytyper2• 8 years ago

These science website corporate types keep outsmarting all you scientists.

boromi• 8 years ago

Anyone else here use Jabref? I like it's simplicity.

erreJulian3• 8 years ago

Only if I need to tweak a .bib file, but I use Zotero with BetterBibTex (https://retorque.re/zotero-better-bibtex/) and it generates a .bib file of my entire database.

veddox2• 8 years ago

Mendeley is (or at least, was) a great software. It works on Linux and did exactly what I needed. Half a year ago, I suddenly couldn't open my PDFs anymore from within the desktop app. Web interface still worked, mobile still worked, just the desktop version was broken. (Which happens to be the only one I seriously use.) When I wrote to them several months later inquiring into the issue, they told me they were still working on the fix. (It's still not working on my machine.) Now this. Perhaps I really should look into Zotero...

iicc2• 8 years ago

GDPR?

TelmoMenezes• 8 years ago

Oh come on...

iicc2• 8 years ago

EU citizens have an explicit right to a direct transfer in to Zotero.

https://gdpr-info.eu/art-20-gdpr/

"Right to data portability"

The data subject shall have the right to receive the personal data concerning him or her, which he or she has provided to a controller, in a structured, commonly used and machine-readable format and have the right to transmit those data to another controller without hindrance from the controller to which the personal data have been provided, where:
the processing is based on consent pursuant to point (a) of Article 6(1) or point (a) of Article 9(2) or on a contract pursuant to point (b) of Article 6(1); and the processing is carried out by automated means.
In exercising his or her right to data portability pursuant to paragraph 1, the data subject shall have the right to have the personal data transmitted directly from one controller to another, where technically feasible.

guitarbill• 8 years ago

Ironically, GDPR is what Mendeley used to justify the encryption [0]. Obviously, complete rubbish. It would be interesting to see what happens if people start asking for their data in a portable format though.

[0] https://twitter.com/mendeley_com/status/1006915998841221120

bocklund3• 8 years ago

Changes in 1.19 encrypted the database, but I think 1.19.2 restored this feature see: https://www.mendeley.com/release-notes/v1_19_2

falien• 8 years ago

That just added the ability to save a copy of the encrypted database. It did not remove the encryption.

lvs• 8 years ago

Entertainingly, it was precisely that update that induced me to drop Mendeley and switch to Zotero. After updating to that version, every login attempt at the app failed. I guess it hadn't fully updated the database because import into Zotero worked flawlessly. I was pleasantly surprised by how much better Zotero has gotten in the last few years.

The browser and MS Word plugins actually work pretty well, although several of the citation style templates seem out-of-date.

erreJulian3• 8 years ago

I´ve been using Zotero for a long time now and I can’t recommend it enough. Along with several plugins, such as ZotFile [1] and ZoteroBetterBibTex [2] I’ve been able to build a very flexible workflow that allows me to work either in a traditional processor, in Latex and even in Markdown+pandoc on iOS. Recently they also added support for Google Docs [3] and it does seems to be working quite well. It enjoys the benefits of open software, such as an engaging community and support.

[1] http://zotfile.com [2] https://retorque.re/zotero-better-bibtex/ [3] https://www.zotero.org/support/google_docs

plg• 8 years ago

paperpile is sweet I just wish they would let iOS users add a paper to their database

laughingman2• 8 years ago

I hate to be under the thrall of Elsevier software. But the shitty tactics would force me to switch to Zotero anyways. I just wish Zotero packed in a consistent cross-platform pdf annotation tool and a mobile app. I wouldn't mind paying for these.

ismail• 8 years ago

I dropped Mendely a year ago. Because it kept crashing and they could not resolve the issue. The bug has not closed since then. I was unable to extract my highlights. I tried mucking about in their SQLite dB.

Tried zoetro but prefer using highlights app ad It saves the highlights on the PDF. And you can then export to markdown etc.

Not sure how zoetro saves highlights but I am never going to take that risk again. A years worth of research down the drain.

aasasd• 8 years ago

I wonder if they're not actually required by GDPR to provide access to data export in an open format―though it might, again, only apply to online storage. And of course, it would only be required in the EU.

burtonator5• 8 years ago

I'm the author of Polar. An app very similar to both Zotero and Mendeley:

https://getpolarized.io/

Both Mendeley and Zotero are better than Polar (for now) with bibliography management but I believe we're better at both at document annotation and knowledge management.

We're also Open Source and launched on Hacker News a few months back.

https://news.ycombinator.com/item?id=18219960

We're still rather new so not a lot of people have heard of us yet but moving forward very aggressively.

I'd also like to thank the Zotero guys for posting their notes about the Mendeley encryption issue.

I think this is a completely unacceptable situation and antithetical to what we should expect in the scientific community.

The Polar on disk repo is exactly the opposite.

We store all your data on disk and your annotations are in JSON format so they're easily hackable.

This is part of the design.

We MAY add end to end repository encryption at some point but it will be in the users control. We're not doing it to lock the user out or to prevent export.

The end to end encryption is so that you can store your repo in the cloud and not have to worry about your data being viewed by anyone other than you.

zwaps8• 8 years ago

Thank you for your work.

I will try your software soon. What I really like so far is the reading progress, which is actually very, very important for me. So far, I have to do this with manual tags in Zotero.

By the way, since this is a work tool, I (and I imagine others) are fully willing and able to pay you for support or storage or other features.

But since Mendeley, me and people I know have become very sensitive about lock-in, encryption and collaboration barriers associated with pro/non-pro accounts. It seems you are determined to take the right steps, I just want to emphasize the importance for your business.

The backlash against Mendeley isn't random. If I build my pipeline on your product, my lifelihood and hours upon hours of crucial work depend on it. This is not a casual software, where switching is inconvenient but not a big deal. Mendeley has f'ed me in a crucial deadline situation, even though I was paying them lots of money. That company is dead to me, and I will badmouth and try to destroy their business any chance I get.

snowwindwaves• 8 years ago

I have been using Qiqqa for 4 years and it is perfect for me. It allows me to perform full text search on my PDF library, performs OCR when necessary, and supports tagging, annotations, and bibtex.

http://www.qiqqa.com

It is wonderful software. It provides me with my own personal google for my PDFs. I used to use google desktop.

shittyadmin3• 7 years ago

Some quick RE work:

sqlite_rekey_v2 is at 00CA1C13 on the Windows version of 1.19.3

Hooking it with Frida or similar should allow you to drop the 3rd parameter and set pKey to NULL allowing you to create a database which can be read by standard sqlite.

Keys however appear to be account-specific - I haven't looked into the full algorithm yet, but it's fairly pointless as you'd need a copy of sqlite3 with SEE support (which costs $2600 or so) to decrypt it using that method anyways, I tried briefly but was unable to come across a stray copy of sqlite3 with SEE enabled, the above solution is probably better for now anyways.

asdff• 7 years ago

I wish there was one reference manager that wasn't so clunky and convoluted, and I've about tried them all now.

fori1to10• 7 years ago

The only reason I keep using Mendeley sometimes is because it is nice for reading on the phone, and it syncs annotations back and forth without issues. If I can set something as smooth with Zotero, that would be it.

Mendeley encrypts users' database after Zotero provides an importer

Related Stories