Expanding the methodological toolkit in aphasia therapy:

dilemmas of ‘what to measure’ and ‘how to measure it’

Julia Wade* (MRCSLT), Jane Mortley** (MRCSLT), Pam Enderby ***(MRCSLT)

*Speech & Language Therapy Research Unit, North Bristol NHS Trust, Frenchay Hospital, Bristol BS16 1LE, julia@speech-therapy.org.uk

**Steps Cottage, Littleton Drew, Wiltshire, SN14 7NB jpmortley@btinternet.com

***Institute of General Practice & Primary Care, Community Sciences Centre, Northern General Hospital, Sheffield, S5 7AU p.m.enderby@sheffield.ac.uk

Purpose

This paper compares and contrasts the information obtained from two different forms of evidence used to evaluate the effects of computer therapy targeting word retrieval for people with aphasia in a research study (Mortley, Wade & Enderby, submitted for publication). A qualitative component (in-depth interviews to evaluate people’s experience of therapy) supplemented the conventional quantitative data (pre and post therapy language assessments) to give context and richness to an otherwise purely quantitative study.

Background

Quantitative and qualitative methodologies can be viewed as antithetical due to the differing philosophical positions about the nature of reality, underlying them. Quantitative research assumes the existence of an external reality ‘as read’. In qualitative research, the existence of an independent reality is questioned. What we take as truth depends on our viewpoint, so there may be as many versions of truth as there are observers, an obvious problem for the generalisability of research findings. In practice, however, all research is selective and therefore limited in generalisability: ‘All research depends on collecting particular sorts of evidence through the prism of particular methods, each of which has its strengths and weaknesses’ (Mays & Pope 1996, p11). Precisely because of these contrasting strengths and weaknesses, it is possible to view these two methodologies as complementary rather than antithetical (Parr & Byng, 2000). Methodology should be determined by the nature of the question addressed. If the aim is to determine ‘how much’ then quantitative approaches may be most appropriate. A ‘what kind’ type of question indicates that qualitative approaches may be more informative (Parr & Byng, 2000).

Qualitative research can complement quantitative research in various ways (Murphy & Dingwall, 1998). First, it can provide preliminary data via exploratory research in an area about which little is known, and generate hypotheses for subsequent testing by means of quantitative research. Second, it can supplement quantitative research as part of the triangulation or validation process, especially where quantitative research leads to unanticipated or inconclusive findings. Qualitative research is particularly useful in exploring complex phenomena, less amenable to quantitative, permitting ‘the development of concepts which help us to understand social phenomena in natural (rather than experimental) settings, giving due emphasis to the meanings, experiences, and views of all the participants’ (Pope & Mays, 1995, p43).

Applied to aphasia research, this methodology is particularly appropriate for investigating the complex and highly individual nature of functional communication in aphasia and the experience of what it is to have aphasia. Qualitative methodologies have already been applied in these areas (e.g. use of conversation analysis in general and of in-depth interviews, Parr, Byng, Gilpin & Ireland, 1997).

In this study, quantitative measures were used to determine evidence of treatment effects on word-retrieval ability under well-controlled experimental conditions. Qualitative methods supplemented quantitative measures with two aims in mind. First, the mode of therapy delivery was novel, i.e. therapy was delivered via computer without face to face contact with a therapist. The aim was to investigate how people experienced the therapy process. Second, in-depth interviews were intended to enable the researchers to supplement and contribute to validation of pre- and post-therapy language assessments by investigating the relationship between the impairment therapy and functional communication. The problems associated with attempting quantification of change in everyday communication are documented elsewhere and use of in-depth interviews suggested as an alternative (Parr & Byng 2000).

Method

A case series study using an ABA cross-over design (A being assessment without therapy and B being therapy) was conducted in which six participants received word-retrieval therapy by computer over a period of six months. Treatment items were divided into Set 1 (treated months one-three) and Set 2 (treatment delayed until months four-six and tested for generalisation effects after treatment to Set 1). Participants had word retrieval difficulties associated with aphasia following stroke and were at least two years post onset (Table 1). Pre therapy assessment included measuring word retrieval on a set of 162 words targeted in therapy, control assessment of sentence comprehension, and in-depth interviews with participants (and carers where applicable). Two baseline language measures were taken six weeks apart. An additional naming assessment was carried out half way through therapy to compare progress on treatment versus non-treatment items. Assessments were repeated after therapy was withdrawn and the naming assessment was repeated six weeks later to determine maintenance.

Table 1. Personal details of participants

Participant	1	2	3	4	5	6
Gender	M	M	M	M	F	M
Age	63yrs	53yrs	58yrs	63yrs	66yrs	63yrs
Yrs post onset	2	2	3	9	12	2
Previous computer therapy	Yes	No	No	Yes	Yes	Yes
Computer user prior to CVA	Yes	No	No	No	No	No

Analysis

Average pre and post therapy scores for naming items targeted in therapy were compared and for each comparison a z-statistic and p-value was calculated. The analytic approach used for the in-depth interviews followed the principles of Framework analysis and is described in detail elsewhere (Wade, Mortley & Enderby, in progress).

Results

Results of language assessments are shown in Table 2. Results from the pre and post naming assessments indicated that all six participants showed significant improvement in naming nouns targeted in therapy, maintained at six weeks post therapy. Four participants showed significant improvement in naming verbs, maintained at six weeks post therapy. Three participants showed generalisation effects in improvement on non-treated nouns but none showed generalisation to non-treated verbs. No significant change took place for any of the participants on control assessments. This fact and the lack of change on the baseline scores suggests that no detectable spontaneous recovery was taking place prior to therapy.

Table 2. Effects on treated items, untreated items and control assessment: pre and post therapy scores for treated nouns and verbs (Sets 1 and 2 combined); pre and post therapy (at 3 months) scores for as yet untreated nouns and verbs (Set 2 only); pre and post scores for sentence comprehension.

Participant			1		2		3		4		5		6
			Raw score	%	Raw score	%	Raw score	%	Raw score	%	Raw score	%	Raw score	%
Rx Noun N= 162		Pre	46	28	68	42	87	54	70	43	70	44	56	34
		Post	125	77	121	75	133	82	106	65	104	64	91	64
		P-value	P<0.001		P<0.001		P<0.001		P<0.001		P<0.001		P<0.001
Rx Verb N= 100		Pre	22	22	22	22	19	19	32	32	33	33	27	27
		Post	66	67	52	52	66	66	50	50	55	55	49	49
		P-value	P<0.001		P<0.001		P<0.001		Non sig		P<0.001		P<0.001
Non Rx Noun N=81		Pre	23	28	32	39	44	54	32	39	34	42	24	29
		Post	35	43	45	56	58	72	38	47	36	44	31	38
		P-value	P<0.009		P<0.006		P<0.003		Non sig		Non sig		Non sig
Non Rx Verb N=50	Pre Post P-value		11	22	11	22	10	20	16	32	NA	NA	18	36
			13	26	14	28	13	26	16	39	NA	NA	15	30
			Non sig		Non sig		Non sig		Non sig		Not treated		Non sig
Sent comp N=40 (control)	Pre		23	58	16	40	55	28	28	70	19	48	17	43
	Post		22	55	20	50	60	32	32	80	18	45	16	40
	P-vaue		Non sig		Non sig		Non sig		Non sig		Non sig		Non sig

Data from in-depth interviews revealed a broad range of perceived benefits to functional communication for all participants summarised in Table 3. All reported improved word retrieval in conversation. A wide range of other effects were described including benefits in the domains of activity and participation as defined by the ICF classification[1]and improvements to confidence and self-esteem. This was in the context of therapy directed purely at word retrieval on computer therapy tasks. Improvements in functional communication were ascribed not simply to direct effects of therapy on

language ability. Increased confidence associated with learning a new skill (i.e. computer use) was also seen as having played an important role. Seeing scores on therapy tasks increase was also important in increasing confidence. Improvements in confidence were also attributed to the degree of autonomy that this particular mode of therapy delivery provided, from both the therapist and partner. Participants believed that increased autonomy had facilitated more intensive therapy practice and hence led to substantial improvements. Interviews also revealed variations in approaches to therapy practice, generating a number of hypotheses for further investigation (Wade, Mortley and Enderby, in progress).

Table 3: Reported benefits to functional communication and examples of new activities or behaviours noted since start of therapy

Participant	Reported evidence of benefits to functional communication	Reported new activities and behaviours
1	· Able to cue self into words · Initiating communication with strangers · Increased participation in group conversations · Using phone · Shopping without a list · Writing letters	· Enrolled on computer course · Return to driving · Increased walking
2	· Taking time to find words · Initiating conversation with neighbours · Shopping alone · Answering phone · Discussing current affairs	· Preparing meals unaided · Doing car maintenance using manual · Assisting friend to rewire house · Using Internet as info source
3	· Producing more phrases · Increased participation in family conversations · Needing fewer repetitions · Using family names · Buying petrol alone	· Decorating house · Laughing at jokes on TV/ in conversations
4	· Producing more phrases · Finding words more easily · Reading the newspaper
5	· Finding words targeted in therapy more easily · Asserting self more to ensure message is understood · Increased participation in communication
6	· Finding broader range of words · More aware of errors and better able to correct · Strangers don’t always notice aphasia · Using phone · Using family names · Attempting to write	· Using Internet · Improved confidence with computers · Improved moods

Discussion

Previous research into the efficacy of word retrieval therapy has been criticised for failing to evaluate effects on functional communication. This study set out to combine very differing quantitative and qualitative methods to investigate effect on word retrieval and functional communication. The idea of combining quantitative and qualitative data in aphasia research is not novel but to our knowledge, this is the first time the method of in-depth interviews to evaluate therapy has been integrated into an experimental study evaluating impairment therapy for anomia.

Measuring change on formal language assessments provides evidence of change in well-controlled experimental conditions but tells us little about a person’s functional communication. In-depth interviewing can reveal whether people perceived a therapy effect and insight into how the effect may have occurred but cannot, in isolation, prove benefit to language. The use of in-depth interviewing was motivated by a desire to evaluate functional effects and acceptability of impairment therapy delivered via computer, phenomena that challenge satisfactory measurement using standardised quantitative assessment. In the event, data revealed a much more complex process than the simple effect of impairment therapy leading to improved language skills and benefit to functional communication. Issues of confidence, self-esteem and autonomy played a more central role than had been anticipated, and generated hypotheses for further investigation. Each measure provides only part of the picture if used in isolation. Combining systematic qualitative and quantitative methods can provide a richer evidence base, with information from one source providing verification for information from the other. We need to expand the range of methodologies available to us in our ‘research toolkit’ if we are to provide the appropriate form of evidence for what we seek to measure.

In-depth interviewing and subsequent analysis may be time-consuming, but it is possible that the principles of method and analysis can be applied in clinical practice in the same way that those of conversational analysis have been (Parr & Byng, 2000).

Conclusion

A multi-method approach, if applied systematically, can substantially strengthen the evidence base for both research and clinical practice. Aphasia therapy has the advantage of a research tradition, which had already established the principle of combining qualitative and quantitative methodologies. There is still potential for them to be combined more extensively.

Acknowledgements

The authors wish to thank all participants for their time and the Stroke Association for providing the funding to make this research possible.

References

Mays, N. & Pope, C. (1996) Rigour and qualitative research. In Mays, N. & Pope, C. (Eds) Qualitative Research in Health Care, pp10-19, London, UK: BMJ Publishing.

Mortley, J. Wade, J. & Enderby, P. (in progress). Superhighway to promoting a client-therapist partnership? Using the Internet to deliver word-retrieval computer therapy, monitored remotely with minimal speech and language therapy input. Aphasiology

Murphy, E & Dingwall, R (1998) Qualitative methods in health services research. In Black, N., Brazier, J., Fitzpatrick, R., Reeves, B. (Eds) Health Services Research Methods: a guide to best practice. London, UK: BMJ Publishing.

Parr, S. & Byng, S. (2000) Perspectives and priorities: accessing user views in functional communication assessment. In: L.E. Worrall, & C.M. Frattali (Eds) Neurogenic Communication Disorders: a functional approach. New York, U.S.: Thieme.

Parr, S., Byng, S., Gilpin, S. & Ireland, C. (1997). Talking about aphasia, Living with loss of language after stroke, Buckingham, UK: OUP.

Pope, C. & Mays, N. (1995) Reaching the parts other methods cannot reach: an introduction to qualitative methods in health and health services research. BMJ, 311, 42-45

Wade, J., Mortley, J. & Enderby (in progress) Talk about IT. Views of people with aphasia on receiving remotely-delivered computer-based word finding therapy.

_____________________________________

---

WHO (2001) International Classification of Functioning Disability and Handicap