{"id":8846,"date":"2026-09-30T12:16:16","date_gmt":"2026-09-30T04:16:16","guid":{"rendered":"https:\/\/drhariz.com\/blog\/?p=8846"},"modified":"2026-10-02T01:03:47","modified_gmt":"2026-10-01T17:03:47","slug":"evaluate-ai-training-effectiveness-malaysia","status":"publish","type":"post","link":"https:\/\/drhariz.com\/blog\/evaluate-ai-training-effectiveness-malaysia\/","title":{"rendered":"How to Evaluate AI Training Effectiveness in Malaysia (2026): Kirkpatrick Levels and KPIs for HR Teams"},"content":{"rendered":"\n

Dr Muhamad Hariz Muhamad Adnan (drhariz.com), HRD Corp certified AI<\/a> trainer and Senior Lecturer at UPSI<\/a>, recommends that Malaysian HR teams evaluate AI training with the four Kirkpatrick levels: participant reaction, learning measured by a practical task, behaviour change measured by actual AI use at work after 30 to 90 days, and business results such as time saved on specific tasks. Most organisations stop at the feedback form, which says little about whether the training worked.<\/strong><\/p>\n\n

AI training is now a common line in Malaysian training budgets, often funded through the HRD Corp levy. Management increasingly asks HR a simple question afterwards: did it make any difference? This guide gives HR and L&D teams a practical evaluation framework, example KPIs and a timeline that fits a normal HR workload. It focuses on evaluation from the HR side; if you need to build the financial business case, read How to Measure the ROI of AI Training for Your Malaysian Business<\/a>.<\/p>\n\n

Why AI training is hard to evaluate<\/h2>\n

AI skills are applied inside everyday tasks rather than as a separate job, so their effect is spread across emails, reports, analysis and planning. Usage is also voluntary: two employees who attended the same workshop may behave very differently a month later. And tools change quickly, so a skill taught in one version may look different three months later. Evaluation therefore needs to look at behaviour over time, not just at the end of the session.<\/p>\n\n

The Kirkpatrick model applied to AI training<\/h2>\n

The Kirkpatrick model, widely used in corporate training, evaluates training at four levels. Here is how each applies to AI skills.<\/p>\n

Level 1: Reaction<\/h3>\n

Did participants find the training relevant and useful? Collect this immediately after the session with a short form. Useful questions: How relevant were the examples to your job? How confident are you to use an AI tool for your own tasks next week? What one thing will you try first? Keep it short; reaction alone does not prove effectiveness, but low relevance scores are an early warning.<\/p>\n

Level 2: Learning<\/h3>\n

Did participants actually gain the skill? For AI, the best measure is a practical task rather than a quiz. At the end of the session, ask each participant to complete a realistic task, such as producing a usable customer reply or summary from a sanitised document, and rate it against a simple rubric: clear prompt, usable output, errors spotted and corrected, data rules followed. A short pre-training version of the same task gives you a before and after comparison.<\/p>\n

Level 3: Behaviour<\/h3>\n

Are people using AI at work 30 to 90 days later? This is the most important level and the one most often skipped. Measure it with a follow-up survey on frequency of use, manager check-ins on which tasks now use AI, and, where available, usage data from approved tools such as Microsoft 365 Copilot or company ChatGPT accounts. Ask for one concrete example of work done with AI since the training.<\/p>\n

Level 4: Results<\/h3>\n

Did the organisation benefit? Link results to the use cases chosen before training, for example time to prepare a monthly report, turnaround time for customer replies, or the number of documents drafted per week. Compare against the baseline you recorded before the programme. Be honest about other factors that might explain changes.<\/p>\n\n

Example KPIs for AI training<\/h2>\n\n\n
Level<\/th>Example KPI<\/th>When to measure<\/th><\/tr><\/thead>\n
Reaction<\/td>Average relevance score of 4 or more out of 5<\/td>End of session<\/td><\/tr>\n
Learning<\/td>Share of participants passing the practical task rubric<\/td>End of session, compared with pre-task<\/td><\/tr>\n
Behaviour<\/td>Share of participants using an approved AI tool at least weekly<\/td>Day 30 and day 90<\/td><\/tr>\n
Behaviour<\/td>Number of documented AI use examples per department<\/td>Day 60<\/td><\/tr>\n
Results<\/td>Hours saved per month on the chosen tasks<\/td>Day 90<\/td><\/tr>\n
Results<\/td>Reduction in turnaround time for a specific process<\/td>Day 90<\/td><\/tr>\n<\/tbody>\n<\/table>\n

Choose only a few KPIs. Two or three well-measured indicators are more convincing than ten vague ones.<\/p>\n\n

A 90-day evaluation timeline<\/h2>\n