Field of Science

Mendeley -- Not quite ready for prime time

Prompted by Prodigal Academic, I decided to give Mendeley a shot. That is, instead of working on a long over-due draft of a paper.

Mendeley is two things. First, it is a PDF library/reader. Second, it is a citation manager.

Currently, I used Papers for the first and Endnote for the second.  Both work well enough -- if not perfectly -- but it is a pain that I have to enter every paper I want to cite into two different programs.

(Don't tell me I could export my Papers citations library to Endnote. First, I'd have to do that every time I update my library, which is annoying. Second, Papers was created by someone who clearly never cites books, book chapters, conference proceedings, etc. So I'd have to fix all of those in Endnote ... every time I export.)

(Also, don't tell me about Zotero. Maybe it's gotten better in the last year since I tried it, but it was seriously feature-deficient and buggy beyond all belief.)

First glance

At first, I was pleasantly surprised. Unlike Papers, Mendeley is free so long as you don't want to use their Cloud functionality much (I don't). Papers is convinced there are people named Marc Hauser, Marc D Hauser, M D Hauser, and M Hauser. Mendeley can be led astray but has some nice options to allow you to collapse two different author records -- or two different keywords.

(On that note, my Papers library has implicit causality, Implicit causality and Implicit Causality all as different keywords. Once Papers has decided the keyword for a paper is, say, Implicit Causality, nothing on G-d's green Earth will convince it to switch to implicit causality. And its searches are case sensitive. Mendeley has none of these "features.")

Also, Mendeley will let you annotate PDFs and export the PDFs with your annotations in a format readable by other PDF viewers (if, for instance, you wanted to share your annotated PDF with someone). That's a nice feature.

These would all be nice additional features if the the core functionality of Mendeley was there. I'm sorry to say that the product just doesn't seem to be ready for prime time.
I typed "prime time" into Flickr, and this is what it gave me. Not sure why.
photo credit here.

Second glance

The first disappointment is that Mendeley does not have smart collections. Like smart playlists in iTunes, smart collections are collections of papers defined by various search terms. If you have a smart collection that indexes all articles with the keywords "implicit causality," "psych verbs" and "to read", then whenever you add a new paper with those keywords, they automatically go into the smart collection. This is very handy, and it's an excellent feature of Papers (except that, as mentioned above, my smart folder for implicit causality searches for the keywords "implicit causality," "Implicit causality" OR "Implicit Causality").

I suspect Mendeley doesn't have smart collections because it doesn't have a serious search function. You can search for papers written by a given author or with a given keyword, but if you want to search for papers written by the conjunction of two authors or any paper on "implicit causality" written by Roger Brown, you're out of luck. Rather, it'll perform the search. It just won't find the right papers.

Third glance

That might be forgivable if the citation function in Mendeley was usable. The idea is that as you write a manuscript, when you want to cite, say, my paper on over-regularization (18 citations and counting!), you would click on a little button that takes you to Mendeley. You find my paper in your PDF library, click another button, and (Hartshorne & Ullman, 2006) appears in your Word document (or NeoOffice or whatever) and the full bibliographic reference appears in your manuscript's bibliography. You can even choose what citation style you're using (e.g., APA).


Sort of. Let's say you want to cite two different papers by Roger Brown and Deborah Fish, both published in 1983 (which, in fact, I did want to do). Here's what it looks like:
Implicit causality effects are found in both English (BrownFish, 1983) and Mandarin (BrownFish, 1983)
At least in APA style, those two papers should be listed as (BrownFish, 1983a) and (BrownFish, 1983b), because obviously otherwise nobody has any idea which paper you are citing.

This gets worse. Suppose you wrote:
Implicit causality effects have been found in multiple languages (BrownFish, 1983; BrownFish, 1983).
Correct APA 5th Ed. style is, I believe, (BrownFish, 1983a, 1983b). Actually, I'm not sure what exactly the correct style is, because Endnote always takes care of it for me.

There are other issues. Mendeley doesn't have a mechanism for suppressing the author. So you end up with:
As reported by Brown and Fish (BrownFish, 1983; BrownFish, 1983), verbs have causality implicit in their meaning.
instead of
 As reported by Brown and Fish (1983a, 1983b), verbs have causality implicit in their meaning.
Nor does Mendeley know about et al:
Hauser, Chomsky and Fitch (Hauser, ChomskyFitch, 2001) put forward a new proposal....blah blah blah...as has been reported several times in the literature (Hauser, ChomskyFish, 2001; BrownFish, 1983; BrownFish, 1983).
That is, the second time you cite a paper with more than 2 authors, it doesn't contract to (Hauser et al. 2001). Unfortunately, there is no work-around for any of these problems. In theory, you can edit the citations to make them match APA style. Within a few seconds, a friendly dialog box pops up and asks you if you really want to keep your edited citation. You can click "OK" or click "cancel," but either way it just changes your carefully-edited citation back to its default -- at least it does on my Mac (the forums suggest that this works for some people).

It's possible that people who don't use APA won't have as many of these problems. Numbered citations, for instance, probably work fine. I've never submitted a paper anywhere that used numbered citations, though. So I either need to switch professions or continue using Endnote to write my papers.

Hopefully

One can hope that Mendeley will solve some of these issues. I found discussions on their "suggested features" forum going back many months for each of the problems discussed above, which suggests I may be waiting a while for these fixes. I do understand that Mendeley is technically in beta testing. But it's been in beta testing for over two years, so that's not really an excuse at this point.

Alternatively, maybe Papers will add a good citation feature (and discover books). Or maybe Zotero will confront its own demons. I'm going to have to wait and see.

It makes one appreciate Endnote. Yes, it's a dinosaur. No, it hasn't added any really useable features since I started using it in 2000. But it worked then, and it still works now. There's something to be said for that.

10 comments:

Barry Belmont said...

An excellent review of the software. I've also been looking for that magic academic software that will make reading, collecting, and reference seamlessly incorporated. I wanted to loved Mendeley so much (cross-platform, excellent BibTeX support, cloud syncing, FREE...), but there's still a lot that needs to be hammered out.

However, I do think of all the major players (Papers, Sente, Mendeley, etc), that Mendeley has the most potential to hit that sweet spot.

Until then and at the very least, I'm glad we're at least having the conversation.

Mr. Gunn said...

Hi GGW - I'm glad you took the time to poke through Mendeley's features and write up your findings. This helps everyone who's looking for a solution to this problem that you would have thought would have been well solved by now.

A few points, however, about things you got right and things you may have overlooked.

Mendeley actually does have smart collections and fantastic search, it's just not implemented in the way you expected. In the search box at the upper right, you can use a variety of search operators, such as the ones shown here: http://www.mendeley.com/faq/#advanced-search This allows you to instantly retrieve a list of search results limited by most of the properties of a document. The drop down box on the lower left also allows you to limit your display by Author, Journal, Keyword, etc. In addition, there are some smart collections that show starred items, recently added items, and so on. Try those out and see if they don't get you the results you need.

It's true that the citation function could be improved, but many of the bugs you mention have actually been fixed in the recent release. In the developer preview at http://www.mendeley.com/download-mendeley-desktop/ you can get a totally redesigned citation insertion plugin that fixes the citation editing bug you mentioned and renders APA style more appropriately. Take that for a spin and let me know how it works for you. After all, the only way Mendeley will improve is through the detailed feedback we're lucky to get from people like you. Mendeley is also the only one I'm aware of that's working on a citation style editor, which you can see a demo version of here: http://csleditor.quist.de/csleditor/show/1/example-citation-style

Thanks again for your great review and please don't hesitate to get in touch personally.

Sean said...

At the risk of really irritating you, what exactly was buggy or feature deficient about Zotero? Zotero has been around for over 4 years and is production, non-beta software. It has had smart collections (we call them saved searches) from day one and supports exactly the kind of citation functionality you describe.

Sean Takats
Zotero Director

Robert Knight said...

Hello,

Thanks for the feedback.

Regarding citation styles:

Some of the issues mentioned with APA citations in Mac Word are resolved with the latest preview build that I tested (you can get preview builds from the bottom of the page at http://www.mendeley.com/download-mendeley-desktop/).

Regarding search:

What syntax were you using to try and find say a paper by a particular author with a particular title? Perhaps the search syntax is not what you expected. The following should work:

author:Brown title:"implicit causality"

GamesWithWords said...

@Gunn. I read the instructions on searching pretty carefully. My biggest problem was that you can search for a conjunction of keywords:

keyword:"implicit causality" keyword:"to read"

revealed no papers, whereas I know there are around a dozen of these in my library.

It turns out that

author:Brown keyword:"implicit causality"

does seem to work. I'm not sure what I was looking at before. However, I can't find all the papers written by Brown that don't have the keyword "implicit causality". Neither of the following seem to do the trick

author:Brown -keyword:"implicit causality"
author:Brown keyword:-"implicit causality"

The advanced searches do work better when you use tags instead of keywords. But I've got 1500 papers with keywords and none with tags.

@Robert Knight: I'm glad you have fixed some of the issues with APA style. Unfortunately, I've never found editors to be sympathetic to using some of APA style. So I really need it to work completely. Which features are fixed?

@Sean: Wow, you and the Mendeley folk are fast. I never tried using Zotero to store PDFs. The problem I had was with citations, similar to the problems I had with Mendeley. The main one was if I had a "Marc Hauser" and an "M Hauser" in my library, it would insist on citing

(Marc Hauser, 2007)

and I couldn't easily get rid of the "Marc". I don't remember what other citation problems I had, but that was a big one. I finally got fed up with it and went back to Endnote.

An unrelated issue was that in 2009 the quality of Firefox started deteriorating noticeably. It took forever to load and frequently crashed my Mac (not otherwise a common occurrence). I wasn't the only person having this problem, so I don't think it was specific to my computer. So I switched back to Safari. Now I use a combination of Chrome and Safari. I no longer have Firefox installed.

James said...

I have just switched to Mendeley because I was having troubles with EndNote and I have come across all these problems with it.
I have found a fix for the problem of "as observed by Smith et al. (Smith et al. 2010). Mendeley citations are editable so you can delete Smith et al. from inside the brackets and refresh. It will ask if you want to keep the change and once confirmed it should stay that way.
I use this to get around the (Smith et al. 2000a, Smith et al. 2000b, Smith et al. 2000c) problem too. Hope this helps.

GamesWithWords said...

@James:
Quoting from my post, "In theory, you can edit the citations to make them match APA style. Within a few seconds, a friendly dialog box pops up and asks you if you really want to keep your edited citation. You can click "OK" or click "cancel," but either way it just changes your carefully-edited citation back to its default -- at least it does on my Mac (the forums suggest that this works for some people)."

Glad it works for you. It doesn't work for me.

Anonymous said...

I've been using Mendeley for over and year now. I generally like it, but agree with your general assessment. The search feature can be frustrating at times.

One other thing is the usefulness of the website. Features such as Groups and lists of popular articles by discipline or keyword, seem like they could be useful. In a "crowd sourced google scholar" kind of way. If that makes sense. I don't think it's there yet but it has potential.

Liz said...

Best thing about Mendeley for me, as an undergrad: it's free. Papers isn't, and only works on a Mac; Zotero is web-based, and while I do use Firefox, I prefer to be offline when reading papers, to avoid distractions!

I mostly use Mendeley for organising papers, though. When it comes to citating papers in my lab project report next term, I might have to look up Endnote...

Mr. Gunn said...

GGW - I would suggest two things you can do to get the citation editing working better. First, I would go get the Developer Preview at http://www.mendeley.com/download-mendeley-desktop/ at the bottom of the page. We're going to be releasing that version as the official release soon, so it should work well for you and it contains a number of bugfixes, including a new citation plugin. If that doesn't solve your problem, email support@mendeley.com. They're fast and helpful.