A systematic literature review (SLR) in software engineering is a structured, repeatable method for identifying, evaluating and synthesising all available research relevant to a specific research question. Unlike a traditional literature review, it follows a written protocol defined before the search begins, so another researcher could repeat your work and reach the same conclusions. The approach most commonly used in computing was set out by Kitchenham and Charters in their 2007 guidelines, and it remains the reference point for postgraduate research in software engineering and artificial intelligence today.
This guide explains how to run an SLR from planning to write-up, in plain language, for Malaysian Master’s and PhD students who need a defensible methodology chapter.
What Makes a Review “Systematic”
Three things separate a systematic review from a narrative one.
A protocol written in advance. You decide your research questions, search strategy, inclusion and exclusion criteria, and quality assessment rules before you start reading. This is what stops you from unconsciously selecting only the papers that agree with you.
A documented, reproducible search. Every database, every search string, every date is recorded. A reader should be able to re-run your search and get a comparable set of results.
Transparent selection and synthesis. You report how many papers you found, how many you excluded and why, and how you combined the findings. Nothing is hidden in the gap between “I read a lot” and “therefore”.
The Three Phases of an SLR
Phase 1: Planning the review
Start by confirming the review is actually needed. If a recent, well-conducted review already answers your question, your contribution should be something else. Then write the protocol, which should contain your research questions, the databases you will search, your search strings, your inclusion and exclusion criteria, your quality assessment checklist, and your data extraction form.
Your research questions drive everything. In software engineering, a common structure is to ask what has been studied, how it has been evaluated, and what remains unresolved. Keep the number small: two to four questions is usually enough for a Master’s or PhD chapter.
Phase 2: Conducting the review
Run the search across the main digital libraries used in computing: IEEE Xplore, ACM Digital Library, Scopus, Web of Science and SpringerLink. Export everything into a reference manager so you can deduplicate cleanly. Screen by title and abstract first, then by full text. Record the count at each stage, because you will need those numbers for your PRISMA-style flow diagram.
Quality assessment comes next. Score each remaining study against a short checklist: is the research question clear, is the method described well enough to repeat, is the sample or dataset described, are threats to validity discussed, are the conclusions supported by the data. Studies that fail badly are excluded, and you report why.
Finally, extract data using a consistent form so every paper is recorded the same way: year, venue, research type, technique used, dataset, evaluation method, and the finding relevant to each research question.
Phase 3: Reporting the review
Synthesis is where the value is created. Group studies by theme, technique or outcome rather than describing them one by one. A chapter that reads as a list of paper summaries is a signal that synthesis has not happened yet. Report both what the literature agrees on and where it conflicts, and state honestly what could not be concluded from the available evidence.
Building a Search String That Works
Break your research question into concepts, list synonyms and alternative spellings for each concept, join the synonyms with OR, and join the concepts with AND. For example, a review on machine learning for defect prediction would combine a “machine learning” concept group, a “software defect” concept group and a “prediction” concept group.
Test the string on one database first. If it returns tens of thousands of results, your concepts are too broad. If it returns fewer than thirty, they are too narrow or your synonyms are incomplete. Adjust before running the full search, and record the final version of every string in an appendix.
Common Mistakes That Weaken an SLR
Searching only Google Scholar. It is useful for validation but is not a substitute for indexed digital libraries with reproducible query syntax.
Writing the protocol after the search. Examiners notice when inclusion criteria appear to have been designed around the papers already collected.
Skipping quality assessment. Without it, a weak conference paper carries the same weight as a rigorous empirical study.
Describing instead of synthesising. If your findings section could be reordered without losing meaning, it is a list, not a synthesis.
Ignoring threats to validity. Every review has them: publication bias, language restrictions, the date the search was run. State them rather than hoping nobody asks.
Is an SLR the Right Methodology for You?
An SLR suits research questions about the state of knowledge in a field, and it produces a strong first publication because the output is self-contained. It is less suitable if your contribution depends on building and evaluating a new system, in which case an experimental or design-science approach fits better. I compare the two paths in more detail in SLR vs Experimental Research: Choosing a Methodology for Your AI Postgraduate Study.
Many students do both: an SLR to map the field and identify a gap, then an experimental study addressing that gap. If you are still shaping your topic, research gaps in AI in education is a useful starting point, and how to write a strong AI research proposal shows how the review feeds into your proposal.
Frequently Asked Questions
How long does a systematic literature review take?
For a single postgraduate student, four to six months is realistic for a full SLR, with the screening stage usually taking the longest. Working in a pair speeds up screening considerably because two independent screeners are good practice anyway.
How many papers should a systematic literature review include?
There is no fixed number. A well-scoped SLR in software engineering typically ends with somewhere between 30 and 80 primary studies. What matters is that your inclusion and exclusion process is defensible, not that you reach a target count.
What is the difference between an SLR and a mapping study?
A systematic mapping study answers broader questions and classifies the literature to show where research activity is concentrated. An SLR answers narrower questions and synthesises evidence to reach a conclusion. Mapping studies are often a good first step when a field is unfamiliar.
Can I publish my systematic literature review?
Yes, and many students do. Review articles are welcome in indexed journals when they are rigorous and current. See how to publish in Scopus journals as a Malaysian postgraduate for the submission process.
Can AI tools help with a systematic literature review?
They can help with screening support, summarising abstracts and organising extracted data, which saves real time. They cannot decide inclusion for you, and any AI-assisted step must be declared in your methodology. Every claim you keep must be verified against the original paper.
Getting Supervision for Your Review
I am Dr Muhamad Hariz Adnan, a Doctor in Artificial Intelligence and Senior Lecturer at the Faculty of Computing and Meta-Technology, Universiti Pendidikan Sultan Idris. I supervise postgraduate research in artificial intelligence, educational technology and digital transformation, and systematic literature reviews are a regular part of that work.
If you are planning an SLR for your Master’s or PhD, see my postgraduate supervision areas or get in touch to discuss your topic.
Reference: Kitchenham, B. and Charters, S. (2007). Guidelines for Performing Systematic Literature Reviews in Software Engineering. EBSE Technical Report EBSE-2007-01.
