3 new tools to try for Literature mapping — Connected Papers, Inciteful and Litmaps
Update Aug 2021 : Since I wrote this, ResearchRabbit has gone public, see my review . ResearchRabbit occupies roughly the same space as Litmaps and is good enough in my opinion to be another one of my favourites to be worth trying out.
Update Aug 2022 — Some of these tools have gone freemium. These are noted.
Tired of entering keywords and getting thousands of hits and not sure where to start your literature review? Or having the opposite problem and entering a keyword and getting few relevant results?
You are in luck, we are seeing the rise of a new class of innovative literature review mapping tools, built on the backs of increasingly open metadata and citations coupled with possibly some new machine learning techniques (particularly those that use machine learning on full text for citation contexts).
I track as many as a dozen such tools out there but here I list three of the latest and in my opinion slickest tools that are currently out there.
In rough order of complexity
- Connected Papers — Simple but powerful one shot visualization tool using one seed paper (Freemium model)
- Inciteful — Customizable tool , use multiple seed papers in an iterative process
- Litmaps —Use multiple seed papers and overlapping maps, combining search with citation relationships and visualization (Freemium model)
- Honorary mentions —Research Rabbit, CoCites(depreciated), Citation Gecko, VOSviewer, CitationChaser + more
- Citation context/sentiment tools — scite, Semantic Scholar
1. CONNECTED PAPERS — SIMPLE BUT POWERFUL ONE SHOT TOOL
Update: Since Aug 2022, you will need a premium account to generate more than 5 maps. This might be too restrictive for some if you use the free account.
Connected Papers is a easy to use and powerful tool that promises to help you quickly identify similar papers with just one “Seed paper” (a relevant paper) and further on help to detect seminal papers as well as review papers.
Find a “seed paper” and enter it into Connected Papers, and it will automatically attempt to generate for you 25 or so relevant papers that are similar to the paper you entered and arranges the papers into a force — directed graph of papers with similar papers placed close by with connecting lines. Size of the node represents number of citations and shade of color represents publication year.
Update : Since Aug 2022, Connected Papers has “multi-origin graphs”. You can still only add one seed paper at first. But once the graph is generated, you can keep adding other seed papers one at a time to refine the graph.
Important Note : Unlike other tools, this creates a similarity graph not a citation graph and connecting lines (based on the similarity metric) do not necessarily show direct citation relationships
The algorithm used to calculate similarity is based on both Co-citation and Bibliographic Coupling. Unlike tools that are based only on co-citations, this ensures that Connected Papers works even for very new papers that do not have many citations, as similarity of references (bibliographic coupling) is considered as well.
The full calculations for the similarity metric has not been publicly disclosed however it has been mentioned that the algorithm also tries to prioritize papers that are in roughly the same “generation” of papers, I.e are not too far apart in publication years.
As such, once you have generated the similarity graph of similar papers, you can click on either the “Prior Works” button or the “Derivative works” button to help detect seminal works or review/survey papers respectively.
The logic is that if Connected Papers did in fact successfully identify similar papers on a topic, the papers that are most commonly cited by those papers might be important prior papers or seminal papers. Similarly, the papers that cite a lot of these identified papers could likely be review papers and we know how useful review papers can be (See 4 ways to find review papers if you need more)
All three groups of identified papers can then be exported into most reference managers like Zotero, EndNote, Mendeley.
For any paper identified of interest you can further click on the “build a graph” button to make it the seed paper instead and generate yet another similarity graph using it as a seed paper.
Addition tip : If the paper you used has been processed by Connected Papers before, you can click on the “Created on xxxx (date)” and see historical versions of the graph.
2. INCITEFUL — CUSTOMIZABLE TOOL, USE MULTIPLE SEED PAPERS IN AN A ITERATIVE PROCESS
While Connected Papers is a easy to use and simple tool that needs only one relevant seed paper, some users might want a tool that allows more customization. For example, what if you had more than one relevant paper? Could you take advantage of that?
This is where the equally new Inciteful tool comes in.
While you can input one paper as the seed paper, the tool suggests that it works best if you include at least 5 papers, which you can import in a batch with a BibTex file.
If you have only one paper, you can still input that in, it will still generate a list of “similar papers” (see later) and you can add a few of them as additional seed papers.
Inciteful by default produces the following lists of papers
- Similar papers (uses Adamic/Adar index)
- “Most Important Papers in the Graph” (based on PageRank)
- Recent Papers by the Top 100 Authors
- The Most Important Recent Papers
and these results can be further filtered by keywords (Boolean query allowed plus subject to Porter Stemming) and distance filters (basically how many citation/reference hops from the seed papers) to look for relevant papers to add as seed paper. Or you can directly add seed papers by title or DOI
The documentation provides very good details on how the algo works.
Start here if you are new to graphs before going to details on how the network is generated and what metrics are used to generate these different lists of papers.
The author of Inciteful describes this tool as a more complicated Connected Papers. Unlike Connected Papers, which is a one shot tool, you can use Inciteful iteratively, by adding the papers it finds into the seed papers and to continue to see what it suggests.
In fact, below each of the categories of papers it suggested (e.g. Similar papers, Most Important Papers in the Graph) there is a small SQL button that when clicked shows you the exact SQL query that is used to generate the papers. You can even make some customizations to the SQL query!
You can fiddle with how the algo works by tweaking the SQL query, for more details refer to the Database Schema on what tables and fields are available.
3. LITMAPS — COMBINING SEARCH WITH CITATION RELATIONSHIPS AND VISUALIZATION
As nice as Inciteful is you may notice that it lacks any visualizations.
As of June 2023, it now has visualizations
So what if you want a tool that can do iterative building and visualization of the papers you have found? Then you can try Litmaps which create maps of your literature in a iterative fashion.
STARTING THE MAP FROM THE SCRATCH
Litmaps provides multiple ways to start. You can
- Import papers using bibtex format which you can export from most reference managers like Zotero, EndNote, Mendeley (similar to Inciteful)
- Import papers from an ORCID profile
- Do a keyword search of papers indexed in Litmaps and select papers you want
- “Build from a seed paper” — This is similar to Connected Papers where you enter one seed paper and it suggests papers to add based on “highly connected articles from adjacent literature”.
There isn’t any details on how the “highly connected articles from adjacent literature” works exactly, but the older version talks about a suggestion radar that uses a
“2º citation network search from your project articles. This means we search through the citation network to find the articles connected to your project graph by references and citations. These are the 1º citation search results. We then go one step further and find all the articles connected to those 1º articles. We then use all of those connections to give a list of the top 20 highest articles most connected to your project.”
No matter which method you use to add articles, as you add papers into your project, Litmaps will visualize these papers as nodes and their citation relationships in a timeline with papers published earlier on the left and the latest papers on the right.
As you may expect the lines or edges between nodes represent citation relationships.
Though not turned on by default, you can also set the size of each node to correspond with number of citations.
Addition tip : Go to the top left corner of the screen to change visualization settings
Each map created also offers a feature to get email updates of “emergent literature”.
CREATING AND COMPARING OVERLAPPING MAPS
One of the nicest features of Litmaps is that you can overlay different maps to look for overlaps.
In the example above, besides the sample map on Dynamic capabilities and strategic management (marked in red), I also created a new map with the seed paper Absorptive capacity: A review, reconceptualization, and extension (marked in orange).
As such, I can see at a glance which papers are shared by both maps (nodes with both red and orange), and how well connected each paper (node) is in their respective maps by hovering over individual papers (nodes).
EXPANDING THE MAPS WITH THE EXPLORE FUNCTION
Probably the most advanced feature in Litmaps is the explore function which allows you to find related papers to add to your map.
You can combine keywords (top results of these searches), papers (nodes) from existing maps and further combine that with filters like publication date.
It will then use the combined papers (nodes) to “traverse the citation tree” to find new articles which are “most highly connected to it”.
It’s unclear how it defines “highly connected” but an earlier version of Litmaps would take into account number of direct citation links to existing papers (either a citation or reference) as well as 2nd level citation relationships (say citation to citation or reference to reference or citation to reference or reference to citation).
Litmaps promote iterative searching, so as you add more papers suggested to the map , you can run the explore again to find new papers.
However, Litmaps will probably still keep suggesting some papers over and over again which can be irriating.
This is where you can use the “ignore” function so that paper will never be suggested again.
Because Litmaps uses papers to find other papers if they have citation relationships, you can close off this possibility for a paper by clicking “Avoid” and it will not use those papers as a citation paths to recommend other papers.
Honorary mentions
Most of the tools I believe have not being properly validated for effectiveness. CoCites is an exception as it is a well studied and validated tool, design for finding related articles in domains covered well by Pubmed.
In particular, it was tested to see how well it could retrieve articles identified by a random set of systematic reviews and meta-analysis.
“In a well-defined, randomly selected sample of reviews, the combined use of CoCites’ co-citation and citation searches retrieved a median of 75% of the included articles. The method performed better when the query articles were more similar and more frequently cited. CoCites’ co-citation and citation searches combined retrieved 88% of included articles when all were in PubMed.”
It also has a Firefox and Chrome extension that adds a one-click button in PubMed and Google Scholar to show frequently co-cited articles
Latest versions allow you to add multiple seed papers, export different sets of papers it finds and see what’s new since your last search. It focuses on finding related articles and currently does not have any visualization similar to Inciteful
This is one of the first tools I noticed that focused on providing literature review support by exploiting citation relationships in a nice user friendly way.
Like tools now days it allows you to search it’s index (Crossref) for seed papers to add, or import seed papers to include from bibtex files.
It has still one of the cleanest and most easily understood interfaces, with the yellow dots representing the seed papers and black dots representing either papers that cite the seed paper or are cited by the seed paper (2 views you can use) and the size of the dots reprsent the number of citations or cites.
Clearly the black dots that are the largests may be relevant and are suggested.
It also allows iterative expansion, as you add some of the suggested black dots as seed papers it will produce more suggested papers
It’s main drawback are two.
Firstly it uses Crossref as it’s only index of papers and until recently due to major holdouts from Elsevier, ACS meant that there were big gaps in it’s citation network.
Secondly, I have found this tool to be quite slow if you enter more than a few seed papers at the start.
3. VOSviewer — Bibliometric mapping tool
VOSviewer is probably the most flexible tool in the list and accepts inputs from a variety of sources.
On top of Microsoft Academic Graph (used by Inciteful and Litmaps), Semantic Scholar Open Research Corpus (used by Connected Papers) as well as PubMed (used by Cocites) it also supports input from
- Web of Science
- Scopus
- Dimensions
- Wikidata
- OCC
- COCI
- and more!
This is a generic bibliometric mapping tool and is extremely flexible allowing you to create multiple types of visualizations (citation maps, term coccurance maps) and choose the citation relations used (bibliometric coupling, cocites, direct citations) as well as the unit of analysis (e.g. paper, author, institution)
I consider bibliometric or science mapping tools like VOSviewer, Citespace etc as fundementally different from the newer tools discussed on this page.
The main difference lies in its intended audience, tools like VOSviewer are generally designed for bibliometricians who have good understanding of bibliometrics and as such the tool does not try to hide the complexity and freely uses bibliometric jargon.
While VOSviewer is probably the most user friendly of all the bibliometric mapping tools I know, (compare it to Citespace which has even more options and visualizations but is even harder to grasp) it still provides a lot of options and jargon which most researchers will need time to learn.
In comparison, the newer literature mapping tools represented by Citation Gecko, Cocites, Connected papers etc are designed to help the researcher do a literature review with minimal knowledge of bibliometrics and limits the number of possible options and customizations in particular types of visualizations.
In some cases, the full algorithms isn’t even revealed, all that matters is it works.
What if you don’t trust all this new fangled tools with fancyalgorithms (whose exact workings may or may not be disclosed) but simply want to accomplish this task — Given a set of articles, show me all citations and references of this set of papers? For example , you want to do a systematic review and do a citation check of relevant papers found to supplement your keyword searching.
Google Scholar would be an ideal source to do this on, but the lack of an API makes this quite time consuming to do in bulk. The next best thing you could do is to use a broad academic citation search index like Microsoft Academic, Semantic Scholar or Lens.org.
I have blogged quite a bit on the virtues of Lens.org but it is essentially a open, free and ne of the largest aggregation of academic content from a variety of sources including Microsoft Academic, Crossref , Pubmed , Pubmed C entral, CORE and more.
The major weakness of Lens.org is that as it is an aggregator of sources there will be a time delay (which can be substantial depending on the source), and might be slow to capture newer papers.
With Citation Chaser, you can easily load up a set of articles in the article input tab with identifers known to Lens.org such as DOI, PubMed ID, CORE ID or Microsoft Academic ID (or by title).
With another click on the References tab, you can extract all the references from that set of papers known to Lens.org.
Clicking on the “Citation tab” will do the same for citations.
There is also a network tab, which produces a visualization but it is still pretty undeveloped at time of writing.
Citation Chaser is opensource and if you run it yourself.
CONCLUSION
These are very new tools that provide a different way of exploring the literature. By using methods that go beyond just keyword searching, they avoid the problem of missing papers if you enter the wrong keyword.
One major limitation of such tools is to consider the index of papers used to construct such maps.
Many of these tools on this page are extremely new and we can expect rapid changes and improvements as time goes by… Already I see some convergence in features in this class of tools, including exporting/importing using bibtex, email alerts and more…
Want to go beyond traditional citations and look at citation context or sentiment (e.g. citation is a “supporting citation” or citation is “cited for method”? Look at smart cites by Scite (Youtube video) and Semantic Scholar (Youtube video).
Want even more options? Refer to my list of innovative literature mapping tools.
