Case study research in software engineering is an empirical method for studying a software system, team or process inside its real context, where you cannot separate the phenomenon from the environment around it. Unlike an experiment, you do not control the variables. Unlike a systematic literature review, you collect fresh evidence from a live setting. It is the right choice when your research question starts with “how” or “why” and the answer depends on the organisation the software lives in.
This guide sets out how to design, run and report a case study in software engineering to a standard that survives examiner scrutiny, written for Master’s and PhD candidates in Malaysia. It is based on how I assess and supervise postgraduate work at UPSI.
When Case Study Research Is the Right Method
Choose a case study when all three of these are true. If any one fails, another method will usually serve you better.
- Your question is about how or why something happens, not how much or how often.
- You cannot control the behaviour of the people or systems involved.
- The context genuinely matters, so removing it would change the answer.
If you want to measure whether technique A outperforms technique B under controlled conditions, run an experiment instead. If you want to summarise what is already known across published studies, run a systematic literature review. The trade-offs between these routes are compared in SLR versus experimental research for AI postgraduates.
The Four Types of Case Study Purpose
- Exploratory: you are looking for what is happening and generating ideas for later study. Common at the start of a PhD.
- Descriptive: you are documenting a situation in detail without explaining causes.
- Explanatory: you are seeking to explain why something occurs, which is the most demanding and the most defensible at doctoral level.
- Improving: you are trying to change the situation and evaluate the change, which overlaps with action research.
State your purpose explicitly in the methodology chapter. Examiners frequently challenge candidates who describe an exploratory study but then make explanatory claims in the conclusion.
Designing the Case Study: Five Decisions to Document
1. Define the case and its boundary
The case is the unit of analysis: one development team, one migration project, one product line, one university department adopting a tool. Write down what is inside the boundary and what is outside it. A vague boundary is the most common weakness in student case studies.
2. Choose single or multiple case design
A single case is justified when it is critical, unique, revelatory or longitudinal. Multiple cases give stronger analytical generalisation because you can look for patterns that repeat across settings. For a Master’s dissertation one well-documented case is usually sufficient. For a PhD, two to four cases are more common.
3. Select the data sources
Case study strength comes from combining sources rather than relying on one. Typical software engineering sources include semi-structured interviews, direct observation of stand-ups or reviews, repository and issue-tracker data, internal documentation, and tool telemetry.
4. Plan the analysis before you collect
Decide in advance whether you will use thematic coding, pattern matching against a proposed explanation, or cross-case synthesis. Writing the analysis plan into the protocol prevents the common failure of ending up with 30 hours of interviews and no defensible way to interpret them.
5. Address ethics and confidentiality early
Company data, employee interviews and production logs all raise consent and confidentiality questions. Malaysian universities require ethics clearance before data collection begins, and Malaysian organisations will usually want a non-disclosure agreement. Build the approval timeline into your Gantt chart, because this step routinely delays candidates by two or three months.
Building Validity Into the Design
Case study research is often criticised for being subjective. The response is not to argue, but to build validity checks into the protocol and report them openly.
- Construct validity: use multiple sources for each key claim, and have participants review your account of what they said.
- Internal validity: for explanatory studies, test rival explanations against the evidence rather than confirming your first hypothesis.
- External validity: describe the context in enough detail that a reader can judge which other settings your findings might apply to. Case studies generalise to theory, not to populations.
- Reliability: maintain a case study database and a written protocol so another researcher could repeat your steps.
Reporting: What Examiners Look For
- A clear statement of the case, the boundary, and why this case was selected.
- A protocol summary, including interview guides and coding schemes, usually placed in the appendix.
- Enough context description for the reader to interpret the findings, while respecting confidentiality.
- A visible chain of evidence linking raw data to codes, codes to themes, and themes to conclusions.
- An honest limitations section that names the threats to validity you could not remove.
The chain of evidence is the part most candidates underestimate. If an examiner cannot trace a conclusion back to a specific quotation or log entry, the finding will be treated as opinion.
Common Mistakes in Student Case Studies
- Calling a single set of interviews a case study without defining a case at all.
- Collecting rich data then reporting only descriptive summaries, with no theoretical contribution.
- Treating one company as representative of an entire industry.
- Ignoring negative evidence that contradicts the emerging story.
- Leaving ethics approval until after data collection has started.
Frequently Asked Questions
Is case study research accepted for a PhD in software engineering?
Yes. Case study research is a recognised empirical method in software engineering and is widely used in doctoral work, provided the design is explicit and validity threats are addressed. Weak case studies are rejected for poor design, not for being case studies.
How many cases do I need?
There is no fixed number. A single case is defensible when it is critical, unique or revelatory. Multiple cases strengthen analytical generalisation, and two to four is common at doctoral level in Malaysia.
What is the difference between a case study and action research?
In a case study you observe without deliberately changing the situation. In action research you intervene and study the effect of your own intervention. If you are introducing a tool and measuring what happens, you are closer to action research.
Can I combine a case study with a systematic literature review?
Yes, and it is a strong combination. The review establishes what is already known and exposes the gap, and the case study supplies the fresh empirical evidence. Many Malaysian postgraduate theses use exactly this structure.
How many interviews are enough?
Stop when new interviews stop producing new codes, which is usually described as saturation. In practice, 8 to 15 interviews per case is a common range in software engineering, but you must justify your own stopping point with evidence.
Who is Dr Hariz?
Dr Muhamad Hariz Bin Muhamad Adnan holds a doctorate in artificial intelligence and is a Senior Lecturer at the Faculty of Computing and Meta-Technology, Universiti Pendidikan Sultan Idris (UPSI). He supervises postgraduate research in AI and computing and works as an HRD Corp certified AI trainer in Malaysia.
Getting Your Design Reviewed
Most case study problems are cheaper to fix at the protocol stage than after data collection. If you are shaping a Master’s or PhD design in AI, computing or software engineering, see the postgraduate supervision page or get in touch to talk through your research question. If you are still at the proposal stage, start with the guide to writing an AI research proposal for Master’s and PhD study in Malaysia, and plan ahead for publishing in a Scopus indexed journal.
