John F. Innes
BVSc, PhD, DSAS(Orth), FRCVS
Dr. Innes graduated in veterinary sciences from the University of Liverpool in 1991. He worked at the University of Bristol for 10 years before returning to Liverpool as a professor of small animal surgery. From 2013 to 2022, Dr. Innes was on the executive committee for a large corporate group before cofounding Movement Veterinary Referrals, an independent specialty group in the U.K. Dr. Innes has published more than 100 peer-reviewed manuscripts in the area of veterinary and comparative orthopedics. He developed an interest in client-reported outcome measures during his PhD in the Bristol Medical School in the mid-1990s and he subsequently developed the Liverpool Osteoarthritis in Dogs (LOAD) tool, a validated client-reported outcome measure.
Read Articles Written by John F. InnesThe veterinary professions have seemingly never been under more strain. Under-resourcing, mergers and acquisitions, and rapid expansion of pet populations during the pandemic have left remaining veterinary team members very short on time. Dogs and cats commonly suffer from chronic or recurrent conditions, and with the backdrop of stretched veterinary services, these can be very challenging to manage beyond a response mode approach when a condition worsens.
As a profession, we owe it to our clients to try to improve our management of chronic health conditions, but one cannot improve what one does not measure. Measuring outcomes for chronic disorders can be challenging, but the science of doing this is improving and technology is helping provide solutions.
As veterinary scientists, our instinct for measuring disease is to turn to objective measures, such as blood biomarkers. However, not all problems have an obvious objective measure. For example, how does one measure pruritus or epilepsy? Additionally, a blood biomarker is usually just a surrogate outcome and what really matters in most scenarios is the health-related quality of life of the animal. In the author’s own specialty of orthopedics, for many years the gold standard outcome measure for lameness was the force platform. This is a machine that sensitively measures ground reaction forces.1 The objectivity of the force platform is very appealing, but the machines are costly, acquiring data is time-consuming, and in the face of multifocal musculoskeletal disease, there are issues with data interpretation. Furthermore, the commonly used force platform measures are somewhat unidimensional and do not capture the full impact of lameness, or joint pain, on a dog; effects of joint pain may also include inactivity stiffness, different types of pain, loss of economy of motion, and more.2 Thus, in the last 2 decades, client-reported outcome measures (CROMs) have emerged in orthopedics and other disciplines. CROMs have also been termed “clinical metrology instruments” and “owner-reported outcome measures.”3
The Evolution of CROMs
In human medicine, there is now a strong focus on patient-centered healthcare and seeking the patient’s opinion on their functional status, health-related quality of life, symptom and symptom burden, personal experience of care, and health-related behaviors such as anxiety and depression. Patient-reported outcomes are measured using tools called patient-reported outcome measures (PROMs).4 These tools are often patient-completed questionnaires. PROMs can be either general in nature or disease-specific. Broad PROMs, such as EuroQol EQ-5D (euroqol.org), examine aspects that fit a variety of different conditions and allow comparison across these various medical conditions to assist in the evaluation and the implementation of new methods of providing care. In contrast, disease-specific PROMs are designed to identify specific symptoms and their impact on the function of those specific conditions. The author’s view of the current situation is that disease-specific PROMs have greater face validity and credibility than generic PROMs.4
PROMs were initially developed for use in pharmacological and health service research. In 1975, the medical profession in Sweden established the nationwide use of PROMs for healthcare quality registers. By 2000, PROMs were introduced into some parts of the United States with the aim of extending PROMs as a reimbursement mechanism for accountability within care organizations. This increasing usage of PROMs has culminated in regulatory bodies adopting their use in clinical trials. For instance, the U.S. Food and Drug Administration has released guidelines that mandate the use of PROMs to support labeling claims.5
In veterinary medicine, given that the patient cannot communicate, we must rely on the pet owner as a proxy. There are data to support the validity of this approach in pediatrics and where human patients cannot communicate for themselves (e.g., neurodisability).6 The earlier CROMs began to appear in the veterinary literature in the late 1990s and early 2000s.7,8 Since then, a number of CROMs have been validated across a range of conditions such as chronic pain, pruritus, epilepsy, osteoarthritis, and dental disease (TABLE 1).3,9-15
Components of CROMs
There are 5 main components for good quality CROMs: item generation and subsequent selection, validity, reliability, responsiveness, and interpretability.
Item Generation
Items can be generated from preexisting CROMs, from client focus groups or surveys, and from clinicians. The language in a CROM should be clear and not technical. Once the pool of items has been created, statistical techniques can be used in order to select the most relevant items. The number of items can be reduced using principal components analysis without loss of descriptive value.10,31
Validity
Validation of a CROM is a continuous process and involves demonstrating 3 categories of validity: face, construct, and criterion. Face, or content, validity assesses whether the instrument addresses the relevant and important aspects of the condition. This is typically done through assessment by an expert panel. Construct validity assesses whether the candidate CROM accurately assesses what it purports to measure. This is typically assessed by comparison to other measures of disease such as other validated CROMs or other accepted disease measures; factor analysis can also contribute to construct validity. Criterion validity measures the correlation with an accepted gold standard measure. This is often used when an existing measure is potentially to be replaced by, for example, a more time-efficient or more cost-effective measure.
Reliability
The reliability of the candidate CROM is typically assessed by calculating internal consistency (e.g., by Cronbach’s α), which essentially assesses how the items correlate with each other. Another important assessment of reliability is a test–retest scenario. In this exercise, clients are asked to assess their animal, which should be clinically stable, at baseline and again at a minimum interval of 14 days. The intraclass correlation coefficient is then typically used as an estimate of the agreement between both assessments.
CROMs should also be assessed in territories and cultures that are different to those of the original validation. Technically, translation of the CROM into a new language requires revalidation.32,39
Interpretability
To aid using CROMs in clinical practice, veterinarians should be able to translate the CROM score to clinical meaning by knowing the minimal clinically important difference (MCID) change. The MCID is defined as “the smallest difference in score in the domain of interest which clients perceive as beneficial.”40 There are various methods that can be used to estimate the MCID and these are divided into anchor-based methods, which rely on client responses to an anchor question, and distribution-based methods, which rely on statistical methods using clinical datasets of CROM responses. MCIDs for many veterinary CROMs have not yet been estimated, but data are emerging for some.28
Clinical Use of CROMs
Given that there are several CROMs that have been developed for common, chronic conditions, it seems that one of the greatest hurdles to uptake of these tools into everyday veterinary practice is time. That is understandable to some extent, but it may also be that systems are also at fault. A traditional model of veterinary practice is that clients book an appointment when a condition worsens, are seen by a veterinarian in a compressed timeframe, and leave (FIGURE 1). This places the burden wholly on the veterinarian and, for chronic conditions, may leave clients feeling unsupported in between visits. Alternative workflows (FIGURE 1) allow for CROM completion in the clinic waiting room or with a veterinary nurse prior to, or after, seeing the veterinarian. In this way, the CROM can save the veterinarian time rather than be an extra burden. In addition, the involvement of, and appropriate delegation to, professional colleagues such as veterinary nurses frees up time for the veterinarian and adds to the job satisfaction of team members.
Other potential barriers to the collection of CROMs data are the manual steps required to acquire the data. In a chronic disease scenario, the logical thing to do is to automate data collection as much as possible. This can now be achieved using information technology solutions. For example, once an animal is diagnosed with a particular condition that will require long-term monitoring and management, they can be scheduled to receive automated surveys via email or text message at user-defined intervals (FIGURE 2). Alternatively, an appointment can be tagged for a particular body system problem (e.g., pruritus, lameness) and the client can automatically receive an appropriate survey by email or text message prior to attending the clinic. With this latter workflow, data are available to clinical colleagues prior to seeing the patient. If the client fails to complete the survey, a popup survey can easily be completed on a device using a QR code link.
Automation of CROMs collection has the additional advantage that data can be collated onto a dashboard and sliced and diced in many ways (FIGURE 3). This tool means that clinicians and clinic managers can see data at the level of individual patient, veterinarian, clinic, or organization level. Such data can be very useful for clinical audit and quality improvement processes. Data can also be exported for full statistical analysis if one is compiling a clinical research manuscript or report.
Figure 3. Screenshot from an automated client-reported outcome measure (CROM) collection dashboard (Cemplicity, cemplicity.com) for the LOAD (Liverpool Osteoarthritis in Dogs) CROM.
Summary
The last 20 years or so have seen the publication of a number of validated CROMs for use in veterinary medicine. Uptake in everyday practice has been slow, perhaps due to an adherence to traditional models of ways of working and the pressure of time on veterinarians. However, taking the time to introduce different ways of working may bring long-term benefits. The integration of CROMs into workflows using technological solutions is likely to accelerate the use of CROMs. The author suggests that the everyday use of CROMs would bring benefits to animals with chronic health conditions and improve the impact that our profession can have on animal welfare.
References
1. Conzemius MG, Torres BT, Muir P, Evans R, Krotscheck U, Budsberg S. Best practices for measuring and reporting ground reaction forces in dogs. Vet Surg. 2022;51(3):385-396. doi:10.1111/vsu.13772
2. Walton B, Cox T, Innes JF. ‘How do I know my animal got better?’ – measuring outcomes in small animal orthopaedics. In Practice. 2018;40(2):42-50. doi:10.1136/inp.k647
3. Radke H, Joeris A, Chen M. Evidence-based evaluation of owner-reported outcome measures for canine orthopedic care – a COSMIN evaluation of 6 instruments. Vet Surg. 2022;51(2):244-253. doi:10.1111/vsu.13753
4. Weldring T, Smith SMS. Patient-Reported Outcomes (PROs) and Patient-Reported Outcome Measures (PROMs). Health Serv Insights. 2013;6:61-68. doi:10.4137/HSI.S11093
5. U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research, U.S. Department of Health and Human Services FDA Center for Biologics Evaluation and Research, U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health Qual Life Outcomes. 2006;4:79. doi:10.1186/1477-7525-4-79
6. Morris C, Janssens A, Shilling V, et al. Meaningful health outcomes for paediatric neurodisability: Stakeholder prioritisation and appropriateness of patient reported outcome measures. Health Qual Life Outcomes. 2015;13:87. doi:10.1186/s12955-015-0284-7
7. Innes JF, Barr AR. Can owners assess outcome following treatment of canine cruciate ligament deficiency? J Small Anim Pract. 1998;39(8):373-378. doi:10.1111/j.1748-5827.1998.tb03735.x
8. Hielm-Bjorkman AK, Rita H, Tulamo RM. Psychometric testing of the Helsinki chronic pain index by completion of a questionnaire in Finnish by owners of dogs with chronic signs of pain caused by osteoarthritis. Am J Vet Res. 2009;70(6):727-734. doi:10.2460/ajvr.70.6.727
9. Monteiro BP, Lascelles BDX, Murrell J, Robertson S, Steagall PVM, Wright B. 2022 WSAVA guidelines for the recognition, assessment and treatment of pain. J Small Anim Pract. Published online October 27, 2022. doi:10.1111/jsap.13566
10. Wessmann A, Volk HA, Parkin T, Ortega M, Anderson TJ. Evaluation of quality of life in dogs with idiopathic epilepsy. J Vet Intern Med. 2014;28(2):510-514. doi:10.1111/jvim.12328
11. Benito J, DePuy V, Hardie E, et al. Reliability and discriminatory testing of a client-based metrology instrument, feline musculoskeletal pain index (FMPI) for the evaluation of degenerative joint disease-associated pain in cats. Vet J. 2013;196(3):368-373. doi:10.1016/j.tvjl.2012.12.015
12. Hill PB, Lau P, Rybnicek J. Development of an owner-assessed scale to measure the severity of pruritus in dogs. Vet Dermatol. 2007;18(5):301-308. doi:10.1111/j.1365-3164.2007.00616.x
13. Rybnicek J, Lau-Gillard PJ, Harvey R, Hill PB. Further validation of a pruritus severity scale for use in dogs. Vet Dermatol. 2009;20(2):115-122. doi:10.1111/j.1365-3164.2008.00728.x
14. Freeman LM, Rush JE, Helburn AM. Validation of a health-related quality of life (QoL) questionnaire for owners of dogs with heart disease. Anthrozoos. 2018;31(6):685-693. doi:10.1080/08927936.2018.1529350
15. Enlund KB, Brunius C, Hanson J, et al. Development and validation of two questionnaires: Dental home care and dental health in Swedish dogs. PLoS One. 2019;14(1):e0204581. doi:0.1371/journal.pone.0204581
16. Perez JM, Alessi C, Kittleson MD, Linares-Villalba S, Engel-Manchado J. Psychometric properties of the Spanish version of the functional evaluation of cardiac health questionnaire “FETCH-Q™” for assessing health-related quality of life in dogs with cardiac disease. Top Companion Anim Med. 2020;39:100431. doi:10.1016/j.tcam.2020.100431
17. Freeman LM, Rush JE, Farabaugh AE, Must A. Development and evaluation of a questionnaire for assessing health-related quality of life in dogs with cardiac disease. JAVMA. 2005;226(11):1864-1868. doi:10.2460/javma.2005.226.1864
18. Olivry T, Bensignor E, Favrot C, et al. Development of a core outcome set for therapeutic clinical trials enrolling dogs with atopic dermatitis (COSCAD’18). BMC Vet Res. 2018;14(1):238. doi:10.1186/s12917-018-1569-y
19. Hill P, Rybnicek J, Lau-Gillard P. Correlation between pruritus score and grossly visible erythema in dogs. Vet Dermatol. 2010;21(5):450-455. doi:10.1111/j.1365-3164.2010.00881.x
20. Niessen SJM, Powney S, Guitian J, et al. Evaluation of a quality-of-life tool for dogs with diabetes mellitus. J Vet Intern Med. 2012;26(4):953-961. doi:10.1111/j.1939-1676.2012.00947.x
21. Niessen SJM, Powney S, Guitian J, et al. Evaluation of a quality-of-life tool for cats with diabetes mellitus. J Vet Intern Med. 2010;24(25):1098-1105. doi:10.1111/j.1939-1676.2010.0579.x
22. Schofield I, O’Neill DG, Brodbelt DC, Church DB, Geddes RF, Niessen SJM. Development and evaluation of a health-related quality-of-life tool for dogs with Cushing’s syndrome. J Vet Intern Med. 2019;33(6):2595-2604. doi:10.1111/jvim.15639
23. Giuffrida MA, Farrar JT, Brown DC. Psychometric properties of the Canine Symptom Assessment Scale, a multidimensional owner-reported questionnaire instrument for assessment of physical symptoms in dogs with solid tumors. JAVMA. 2017;251(12):1405-1414. doi:10.2460/javma.251.12.1405
24. Brown DC, Bell M, Rhodes L. Power of treatment success definitions when the Canine Brief Pain Inventory is used to evaluate carprofen treatment for the control of pain and inflammation in dogs with osteoarthritis. Am J Vet Res. 2013;74(12):1467-1473. doi:10.2460/ajvr.74.12.1467
25. Brown DC, Boston RC, Farrar JT. Comparison of force plate gait analysis and owner assessment of pain using the Canine Brief Pain Inventory in dogs with osteoarthritis. J Vet Intern Med. 2013;27(1):22-30. doi:10.1111/jvim.12004
26. Brown DC, Boston R, Coyne JC, Farrar JT. A novel approach to the use of animals in studies of pain: validation of the canine brief pain inventory in canine bone cancer. Pain Med. 2009;10(1):133-142. doi:10.1111/j.1526-4637.2008.00513.x
27. Brown DC, Boston RC, Coyne JC, Farrar JT. Ability of the canine brief pain inventory to detect response to treatment in dogs with osteoarthritis. JAVMA. 2008;233(8):1278-1283. doi:10.2460/javma.233.8.1278
28. Innes J, Morton M, Lascelles BDX. Minimal clinically-important difference for ‘Liverpool Osteoarthritis in Dogs’ (LOAD) and Canine Orthopedic Index (COI). Paper presented at: European Society of Veterinary Orthopaedics and Traumatology; September 2022; Nice, France.
29. Muller C, Gaines B, Gruen M, et al. Evaluation of clinical metrology instrument in dogs with osteoarthritis. J Vet Intern Med. 2016;30(3):836-846. doi:10.1111/jvim.13923
30. Walton MB, Cowderoy E, Lascelles D, Innes JF. Evaluation of construct and criterion validity for the ‘Liverpool Osteoarthritis in Dogs’ (LOAD) clinical metrology instrument and comparison to two other instruments. PLoS One. 2013;8(3):e58125. doi:10.1371/journal.pone.0058125
31. Hercock CA, Pinchbeck G, Giejda A, Clegg PD, Innes JF. Validation of a client-based clinical metrology instrument for the evaluation of canine elbow osteoarthritis. J Small Anim Pract. 2009;50(6):266-271. doi:10.1111/j.1748-5827.2009.00765.x
32. Alves JC, Jorge P, Santos A. Initial psychometric evaluation of the Portuguese version of the Liverpool Osteoarthritis in Dogs. BMC Vet Res. 2022;18(1):367. doi:10.1186/s12917-022-03461-8
33. Brown DC. The canine orthopedic index. Step 1: devising the items. Vet Surg. 2014;43(3):232-240. doi:10.1111/j.1532-950X.2014.12142.x
34. Brown DC. The canine orthopedic index. Step 2: psychometric testing. Vet Surg. 2014;43(3):241-246. doi:10.1111/j.1532-950X.2014.12141.x
35. Brown DC. The canine orthopedic index. Step 3: responsiveness testing. Vet Surg. 2014;43(3):247-254. doi:10.1111/j.1532-950X.2014.12162.x
36. Enomoto M, Lascelles BDX, Robertson JB, Gruen ME. Refinement of the Feline Musculoskeletal Pain Index (FMPI) and development of the short-form FMPI. J Feline Med and Surg. 2022;24(2):142-151. doi:10.1177/1098612X211011984
37. Gruen ME, Griffith EH, Thomson AE, Simpson W, Lascelles BDX. Criterion validation testing of clinical metrology instruments for measuring degenerative joint disease associated mobility impairment in cats. PLoS One. 2015;10(7):e0131839. doi:10.1371/journal.pone.0131839
38. Benito J, Hansen B, Depuy V, et al. Feline musculoskeletal pain index: responsiveness and testing of criterion validity. J Vet Intern Med. 2013;27(3):474-482. doi:10.1111/jvim.12077
39. Ragetly GR, Massey L, Brown DC. Initial psychometric testing and validation of the French version of the Canine Brief Pain Inventory. Vet Anaesth Analg. 2019;46(5):667-672. doi:10.1016/j.vaa.2019.04.001
40. Jaeschke R, Singer J, Guyatt GH. Measurement of health-status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10(4):407-415. doi:10.1016/0197-2456(89)90005-6