Measuring behavioural change – going beyond instant metrics

Measuring behavioural change – going beyond instant metrics

Tool-based metrics extend engagement and value of medical education (MedEd) activities. Simple questions associated with an educational tool can quantify behavioural change over time to measure the true impact of a deliverable.

Behavioural change is the desired outcome of any MedEd activity. Traditional, instant metrics – the type usually seen on a feedback form at the end of an event – show how well an activity is received, but not how it will influence decision making.

Example instant metrics:

  • Engagement by numbers
    • Sales
    • Event registrations
    • Website traffic
    • Number of questions asked
  • Experiential metrics
    • How relevant was the content?
    • How do you rate the speaker?
    • Will you change your practice based on the content?
  • Knowledge check quizzes

For a deeper understanding of impact, we need to find a way to engage beyond the initial delivery of MedEd.


Tools to continue engagement

Once a MedEd activity has been delivered, providing a follow-on tool is a great way to continue this engagement.

Any tool should support the MedEd objectives and, crucially, provide value to the target audience. It is this value that will encourage engagement and form the basis of any metrics.

Value-adding tools can fit a range of budgets and needs. From cutting edge digital deliverables, to low-tech ‘good-old-fashioned’ pen and paper. Each tool is tailored with the target audience in mind.


Example tools:

Online patient case profiler to provide clinicians with case studies and patient suitability for treatment

Digital and physical risk factor-based decision support tool to identify high-risk patients with specific treatment needs

Treatment diary to provide guidance and empower patients to take control of their symptoms


Quantifying behavioural change

So, how do we go beyond these instant metrics?

How do we see that an activity is making an impact?

To measure change we need to assess metrics over time. Importantly, any change should be supported by evidence.

Simply asking ‘have you changed your behaviour?’ is open to bias or interpretation.

Instead, we should ask if the MedEd audience can demonstrate outcomes:

  • Can you demonstrate ways in which you have changed practice?
  • What changes to your lifestyle can you tell us about?
  • What steps have you taken to share this information?

This nuanced change in question-asking unlocks true change and responses can be graded to give a quantifiable metric.


Evolving MedEd tactics through feedback

Measuring different levels of engagement in a step-wise approach can help identify change over time and any barriers preventing it.

  1. Change in individual behaviour
  2. Change in group behaviour
  3. Change in institutional behaviour

The types of questions and associated discussion can also spur increased engagement and dialogue between the MedEd provider and target audience. Thus, further adding value to the MedEd provider and the recipient.

The approach outlined here goes beyond how many units are sold or how many people are sat in the audience.

In the long term, the information gained can be used to evolve the delivery of activities and ensure real impact on the behaviours and practices of the MedEd audience.


Edward Rochford, HealthCare21 Communications, Macclesfield