Using AI for Systematic Reviews: Time-Saver or Risky Shortcut?
Systematic reviews are essential to drive research in the fields of healthcare and medicine. But they can be highly time-intensive tasks when done without using automation. With the advent of artificial intelligence (AI), researchers have found ways to incorporate appropriate AI tools for simplifying this laborious process. Here’s a look at some of the useful AI tools and how they can be efficiently used for systematic reviews despite their limitations.
5 Best Practices for Using Generative AI Tools for Systematic Reviews
AI for Systematic Reviews
Systematic reviews are performed in stages, and there are various AI tools that can be helpful at each stage1. It is crucial to identify the right kind of AI tool that’s suitable for your work.
Literature Search
During the search development, you need a comprehensive strategy that can help you identify relevant research from existing literature to obtain unbiased evidence for answering your research question. Platforms like searchrefiner, Scispace, and Elicit are popular among researchers. However, generative AI tools like Google Gemini and ChatGPT are also useful for enhancing the search via suitable search strings
Citation Discovery & Management
It is not just enough to identify relevant literature but also manage them efficiently. This is where tools like R Discovery2, Scite, and ResearchRabbit can be helpful. The use of machine learning in these tools helps enhance the search and discovery process, enables you to save the works of your choice for future reference, and aids in overall management of references.
Data Screening and Extraction
This is the most critical stage in systematic reviews, which can be an error-prone task if executed manually. Tools like Rayyan, Nested Knowledge, DistillerSR, Covidence, and Abstrackr utilize machine learning algorithms and natural language processing to automate the screening and analysis of data. The tools take user-provided key terms and extract data from abstracts, keywords, and titles to generate output.
Although these tools have proven that AI-assisted screening and data extraction can ensure the efficient execution of systematic reviews, they are not without limitations. This means that researchers should learn the best way to utilize AI for systematic reviews by understanding its drawbacks and maintaining the “human element” intact without over-relying on AI tools.
5 Best Practices for Using Generative AI Tools for Systematic Reviews
1. Craft the right prompts
Generative AI tools need strong, accurate prompts for maximized impact. Define your question clearly using simple, concise language. Include all relevant context within the prompt and simplify complex elements into smaller prompts that are more direct. Provide a persona for the AI tool and specify what format you wish to view the results.
2. Use AI for streamlining
AI tools benefit researchers the most when it comes to streamlining the processes involved in systematic reviews. For instance, the use of semantic search in the discovery phase of literature review can help you identify works that might have otherwise gone unnoticed. Moreover, automated data screening and extraction can help you save time in tedious tasks.
3. Beware of AI bias
The output of AI tools is only as good as the quality and diversity of data used for training them. For instance, during the literature search, AI tools may be overly biased towards research papers that are highly cited or papers published in high-impact journals, causing you to lose out on other relevant papers that are perhaps not easily discoverable. Therefore, putting in some amount of manual effort is essential to maintain the AI-human balance.
4. AI provides data, you interpret it
Although AI tools boast of advancements, they lack the ability to interpret information, especially in the case of poorly reported data. AI tools rely on pattern recognition and often fail to read context. This means that while AI tools may be timesaving, human oversight is necessary for accurate interpretation of extracted data.
5. Invest time in learning the tools
Like any other tool or software, generative AI requires you to have a fundamental understanding of how it works. Spend time in learning how the tool operates before you start integrating them into your research workflow. Watch videos and read handbooks/guidebooks that can help you understand the features of the tool. This initial effort and commitment will go a long way in simplifying the usage of tools for AI-assisted systematic reviews.
Key Takeaway
AI tools can be powerful time-savers when used responsibly and transparently. However, it can be a risky shortcut if used carelessly without human oversight.
Systematic reviews can be performed more efficiently as AI tools progress and develop over the years. As researchers, explore the options available, validate the tools, and always maintain the human element when using AI.
References
1. AI tools for systematic reviews https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-024-02682-2/tables/1
2. R Discovery https://www.editage.com/r-discovery
