This last week I ran a search for a rapid review which was
particularly grey literature heavy. As a result, I ended up with several
domain-specific search strings for Google. Originally, I thought I could either
copy the first 100 results into a Word document for screening, or screen the
results for relevance myself. However, the former is not the most user
friendly, and the latter would inevitably introduce some bias. Additionally,
speed was key for this rapid review!
I decided to look into how I could possibly pull all the
required information from the Google results, in a way that would enable the reviewer to efficiently screen themselves. My colleagues know that I LOVE a
Chrome extension, and now I have found yet another one in the Data Miner Scraper.
Data Miner is a really handy tool which enables you to quickly
scrape the information you want from any webpage. They have a bunch of public “Recipes”
available which can pull off a range of information depending on your needs (under the "Public" tab). In
this case, I simply needed the URL, title, and summary info (effectively what
you would get as you’re scrolling through the results normally). I found a
recipe that would do this, and simply ran it on each of the pages from my
searches.
You can then export the results into Excel. Once I had it in
this format it was also really easy to de-duplicate the results using the URLs.
So in the end, I had a much more user-friendly format, with de-duplicated
results, which would enable my colleague to screen the results more easily.
I am yet to look into whether this could be uploaded to a
reference manager so the results could all be screened in one place (seems
feasible?). However, for now I’m glad I’ve found Data Miner and I know I’ll be
using it in more searches in the future.
- You can download Data Miner here (big bonus was that I didn’t need permission from IT!).
- They also have a YouTube channel with lots of tutorials which can be accessed here.