September 2004, Volume 31, Number 5

Online Exclusive
Computerized Symptom and Quality-of-Life Assessment for Patients With Cancer Part II: Acceptability and Usability
Kristin H. Mullen, MN, ARNP, AOCN^®, Donna L. Berry, PhD, RN, AOCN^®, and Brenda K. Zierler, PhD, RN

Abstract

Purpose/Objectives: To determine the acceptability and usability of a computerized quality-of-life (QOL) and symptom assessment tool and the graphically displayed QOL and symptom output in an ambulatory radiation oncology clinic.

Design: Descriptive, cross-sectional.

Setting: Radiation oncology clinic located in an urban university medical center.

Sample: 45 patients with cancer being evaluated for radiation therapy and 10 clinicians, who submitted 12 surveys.

Methods: Acceptability of the computerized assessment was measured with an online, 16-item, Likert-style survey delivered as 45 patients undergoing radiation therapy completed a 25-item QOL and symptom assessment. Usability of the graphic output was assessed with clinician completion of a four-item paper survey.

Main Research Variables: Acceptability and usability of computerized patient assessment.

Findings: The patient acceptability survey indicated that 70% (n = 28) liked computers and 10% (n = 4) did not. The program was easy to use for 79% (n = 26), easy to understand for 91% (n = 30), and enjoyable for 71% (n = 24). Seventy-six percent (n = 25) believed that the amount of time needed to complete the computerized survey was acceptable. Sixty-six percent (n = 21) responded that they were satisfied with the program, and none of the participants chose the very dissatisfied response. Eighty-three percent (n = 10) of the clinicians found the graphic output helpful in promoting communication with patients, 75% (n = 9) found the output report helpful in identifying appropriate areas of QOL deficits or concerns, and 83% (n = 10) indicated that the output helped guide clinical interactions with patients.

Conclusions: The computer-based QOL and symptom assessment tool is acceptable to patients, and the graphically displayed QOL and symptom output is useful to radiation oncology nurses and physicians.

Implications for Nursing: Wider application of computerized patient-generated data can continue in various cancer settings and be tested for clinical and organizational outcomes.

Key Points

• Computerized quality-of-life and symptom assessment was acceptable to patients with cancer as a method of gathering patient-reported information.
• Patients of all computer use backgrounds reported high acceptability.
• The graphic display of assessment responses was useful to physicians and nurse clinicians in promoting communication about symptoms and quality of life.

Of the estimated 1,368,030 people who will be diagnosed with cancer in the United States in 2004, approximately 63% will survive more than five years (Jemal et al., 2004). Understanding the impact of cancer and the therapies designed to cure or prolong the lives of people with cancer is essential for patients and healthcare providers. Despite the apparent interest in assessing quality of life (QOL) in patients with cancer, routine evaluations of QOL are uncommon in most clinical cancer settings (Batel-Copel, Kornblith, Batel, & Holland, 1997).

Computerized administration of assessment tools is a reliable means of collecting patient data. Numerous studies have compared responses from each participant on the written version of the instrument of choice and the computerized version (Drummond, Ghosh, Ferguson, Brackenridge, & Tiplady, 1995; Lutner et al., 1991; Pouwer, Snoek, van der Ploeg, Heine, & Brand, 1998; Roizen et al., 1992; Skinner & Allen, 1983; Taenzer et al., 1997, 2000; Turner et al., 1998; Velikova et al., 1999). Most of these studies reported no noteworthy differences between modes of testing. In addition, when participants were exposed to both types of test administration (written and computerized), the majority reported a preference for electronic questionnaires. The benefits of electronic assessment have included decreased time to complete the questionnaire, ability to enter data directly into existing clinical databases, and potentially more accurate response rates. The acceptability of healthcare information technologies has been studied, and a considerable amount of literature has been published on healthcare workers’ interaction with computerized medical records and other new technologies (Dewan & Lorenzi, 2000; Kushniruk, Kaufman, Patel, Levesque, & Lottin, 1996; Kushniruk, Patel, & Cimino, 1997; Patel & Kushniruk, 1998). However, the literature contains much less information concerning the usability of medical computer programs with which patients interact.

Taenzer, Suave, Burgess, Milkavich, and Whitmore (1996) have documented that a computer interview was a feasible method for obtaining health information and that the program was very well accepted by the participants. In oncology settings, positive effects on increasing the number of QOL issues discussed between patients and healthcare providers were reported (Taenzer et al., 1996). Wilkie et al. (2001) evaluated the feasibility and acceptability of a computerized assessment of cancer-related symptoms in 41 patients. They concluded that the computer program was a highly acceptable way for the participants to report their symptoms (Wilkie et al.).

Velikova, Brown, Smith, and Selby (2002) and Taenzer et al. (2000) reported the perceptions of clinicians in the cancer care setting regarding usability of a computer-administered QOL assessment tool. The clinicians reported that the measurement tool identified areas of QOL concerns that had not been addressed previously and that the QOL data obtained enhanced communication between patients and clinicians.

One relevant study that focused on the acceptability and usability of computerized QOL screening was conducted in a cancer pain clinic (Carlson, Speca, Hagen, & Taenzer, 2001). The participants completed a computerized QOL assessment and a postsurvey paper-and-pencil questionnaire that assessed patients’ impressions of the computerized assessment. The authors concluded that patients found this computerized assessment easy to use, understandable, enjoyable, helpful, and a good use of waiting room time. The patients were satisfied with the experience, and their attitudes about computers improved after completing the computer program. Physicians and nurses who cared for patients in this setting reported that the QOL summary was useful in patient care. Although computerized screening has been reported about in other countries, none has been implemented and evaluated in a U.S. oncology ambulatory setting to screen for QOL concerns and symptomatology.

The purpose of this study was to determine the acceptability of a computerized QOL and symptom assessment survey for patients and the usability of the output for healthcare professionals. The acceptability analysis was intended to evaluate whether patient participants were able to complete the computerized program and whether they found computerized assessment acceptable. An additional aim of this study was to determine whether the graphically displayed assessment results were useful to the doctors and nurses who cared for the participants.

Design

This descriptive pilot study included two convenience sample groups and was part of a larger descriptive cross-sectional study to develop and test a computerized QOL and symptom assessment tool for patients with cancer (see part I of this article).

Setting and Sample

The study took place in an outpatient radiation oncology clinic located in an urban university medical center in the northwestern United States. The first sample group was comprised of 45 consecutive clinic patients, 26 men and 18 women, who were being evaluated for radiation therapy. Inclusion criteria were being 18 years of age or older, having a cancer diagnosis, being able to communicate in English, and having an evaluation by a radiation oncologist for radiation therapy. Exclusion criteria included receiving or being evaluated for total body irradiation, being evaluated for gamma knife stereotactic radiosurgery or for neurosurgery, or being unable to communicate in English. The ages of participants in this sample group ranged from 18–97 years (mean = 54.89 years). The educational backgrounds of the participants were relatively diverse: 9% (n = 4) had not completed high school, and 43% (n = 17) had achieved an undergraduate or graduate degree.

The second sample group was composed of 10 clinicians: 4 attending physicians, 2 resident physicians, 1 nurse practitioner, and 3 RNs who cared for the participants described in the patient sample. Although clinician participants were asked to complete the questionnaire one time only, two nurses completed it twice responding to two different patient surveys. Because of the pilot nature of the study and the small sample size, these second responses are included in the analysis. To be included in this sample, the healthcare providers must have seen the graphically displayed output from the computerized survey that one of their patients had completed. Exclusion criteria included the physicians and nurses who were not caring for the patients in the study. Fourteen clinicians were invited to participate, 13 questionnaires were returned, and one returned survey was deemed ineligible because the clinician (an attending physician) indicated on the survey that he had not seen the computer-generated patient data. Responses from 12 clinician questionnaires, which were completed by 10 different clinicians, were used for data analysis.

Instruments

Acceptability was measured using a computerized version of a questionnaire that was developed and used by Carlson et al. (2001). The tool consists of six preassessment items and 10 postassessment items using Likert-type responses. Total scores were not calculated. Carlson et al. did not report reliability or validity data. The preassessment items elicited responses pertaining to previous computer use, education, and attitudes toward computers, paper-and-pencil surveys, computer questionnaires, and face-to-face interviews. Each attitude item was scored on a 1 (dislike this method very much) to 5 (like this method very much) scale. Seven of the postassessment items addressed the experience of using the computerized program, including how easy and enjoyable the program was to use, how understandable the questions were, how helpful completing the program was, whether the participant liked the program, whether the amount of time to complete the program was acceptable, and overall satisfaction with the program. The response choices ranged from 1–5, with higher scores indicating a more positive experience. Reliability testing of these seven items revealed an alpha coefficient of 0.91. Three additional postassessment items elicited responses rating preferences of interview method, comparing face to face, written, and computerized, again using a 1–5 scale.

Clinician usability was measured with a written questionnaire consisting of four questions using Likert-type responses that also were scored on a 1–5 scale (Carlson et al., 2001). After the physician or nurse concluded the clinic visit with the patient, he or she was asked to complete the short questionnaire that same day. The questions determined whether the clinicians had viewed the assessment results before the clinical encounter and whether they found the graphically displayed results of the QOL and symptom survey useful. Carlson et al. did not report the reliability and validity of this usability tool, and the current study’s authors did not calculate these parameters in this pilot study.

Procedures

Human subject approval from the university human subjects division was granted prior to beginning the study. A trained research assistant explained the study and provided the laptop computer to patients. Patients read the consent information on the computer screen. The program was designed to be user-friendly with a touch screen and simple directions. The research assistant was available to assist patients if necessary. When assistance was needed, the type of assistance was recorded into a logbook. Prior to completing the QOL and symptom questions, participants were asked to complete the six preassessment questions. When they completed this segment, the program prompted them to complete the 25 QOL and symptom questions. At the end of the QOL and symptom questions, they were asked to complete the 10 postassessment questions. A total of 41 items were presented on the computer screen to the participants who completed the entire program. Once patients completed the QOL and symptom assessment, a color graphic display of the results was printed and given to patients’ physicians and nurses. The color printout ranked patients’ responses by level of symptom distress and QOL item score and flagged potentially troublesome levels in red.

Clinicians were asked to complete the four-item clinician survey and one additional open-ended question eliciting information about the usefulness of the graphic display of participants’ QOL and symptom assessment. Return of the questionnaire implied consent to participate. On this paper-and-pencil questionnaire, clinicians were asked to provide their job title (RN, attending physician, resident physician, or other) but no other identifiers. A collection box was left on a counter in the clinic area labeled “quality-of-life clinician survey,” and clinicians were instructed to deposit the completed surveys in the box. One e-mail reminder was sent to all physicians and nurses in the radiation oncology clinic to prompt the clinicians to complete and return the survey.

Data Analysis

Descriptive statistics (mean and standard deviation) of the sample characteristics as well as statistical analysis of the relationship between sample characteristics and computer use and computer acceptance were completed using chi-square. Preassessment acceptability responses were compared to the postassessment responses. In addition, an analysis of participant preference for computer versus paper-and-pencil assessment and degree of previous computer use were evaluated. Clinicians’ responses to the utility of the graphic output were described, and single-item frequency distributions were calculated.

Acceptability

Fifty-four patients were approached and invited to participate, seven patients declined to participate because of feeling sick, and two patients chose to have their responses recorded only for clinical use and not to be used in the research database, leaving a sample of 45 participants.

Preassessment results: Table 1 describes the level of patients’ computer use. The individuals who responded that they never used computers all were aged 54 or older. Also, none of the respondents who were older than 61 years reported using a computer frequently. A significant negative correlation (r = –0.551) was found between age and computer use (p = 0.002). As age increased, reported computer use decreased. Participants who did not complete high school used a computer less frequently than those who attended some college or technical training or had received undergraduate or graduate degrees (p = 0.002) (see Table 2).

When participants were asked to rate their attitudes about computers before completing the QOL and symptom items, 62% (n = 28) indicated that they liked computers, 18% (n = 8) were neutral, 9% (n = 4) reported that they did not like computers, and 11% (n = 5) did not respond.

Forty-one percent (n = 16) of the 39 responding participants reported that they liked paper-and-pencil surveys, 54% (n = 21) responded that they liked computer questionnaires, and 65% (n = 26) responded that they liked face-to-face interviews. Twenty-three percent (n = 9) of the participants disliked paper-and-pencil surveys; however, only 13% (n = 5) disliked computer surveys and 7% (n = 3) of 41 responders disliked face-to-face interviews.

Postassessment results: No significant difference existed in participants’ attitudes about computer questionnaires between individuals who had achieved a higher level of education or who reported more frequent computer use as compared to individuals who had achieved less than a college education and who never or rarely used computers.

When participants were asked how difficult the computer program was to use, none of the 33 who completed this question found the program to be very difficult. As seen in Figure 1, the responses were overwhelmingly positive with regard to aspects of using the program.

Participants were asked to do three different comparisons on methods of survey administration: face to face and computers, paper and pencil and computers, and paper and pencil and face to face. The responses to these questions were similar to the responses in the preassessment segment of the program. Seventy percent (n = 21) of 30 participants responded that they liked computer questionnaires, 23% (n = 7) chose the neutral response, and 7% (n = 2) responded that they did not like computers.

In the final segment of the survey, 66% (n = 21) of 32 participants responded that they were satisfied with the computerized program, 31% (n = 10) chose the neutral response, and none of the participants indicated that they were very dissatisfied. The responses regarding attitudes about computer questionnaires were very similar to the responses in the preassessment segment: 70% (n = 21) of 30 participants responded that they liked computer questionnaires, and 7% (n = 2) responded that they disliked computer questionnaires. The average time it took for a participant to complete the entire tool was about 10 minutes.

Missing data: All consenting patients who answered at least one question were included in the analysis. A considerable amount of data was missing from the final items because participants who were not able to finish the survey for one of several reasons were included in the analysis (Trigg, Berry, Karras, Austin-Seymour, & Lober, 2003). The average number of missing responses per participant in the preassessment portion of the survey was 0.6. The postassessment average number of missing responses was 2.6.

Usability

The data from clinicians who responded that they had seen the patient graphic output were analyzed for usability. The results of the clinician survey are seen in Figure 2. Because of the small sample size (N = 12) in the clinician group, differences between clinician groups, nurses, attending physicians, resident physicians, and the nurse practitioner were not calculated. However, the nurses and nurse practitioner did appear to give the most favorable reports of utility of the patient data in graph format, followed by the resident physicians and attending physicians.

This study suggests that the computerized QOL and symptom questionnaire used in a university medical center radiation oncology setting is an acceptable method of gathering patient information and that the immediate information generated by the program is clinically useful in the care of patients with cancer. Overall, the program was very well accepted, and patients who participated in the study preferred the computerized questionnaire format to paper-and-pencil and face-to-face interviews. Patient participants found the program easy to use and enjoyable and indicated that the amount of time it took to complete the program was appropriate. Nurse clinicians and resident physicians found the output to be more helpful in identifying areas of concern, promoting communication, and guiding their clinical interactions with patients than did attending physicians. In this university setting, as in many other settings, the resident physician typically conducts the history and physical examination portion of the new patient encounter before the attending physician interacts with the patient. The resident usually reports his or her findings verbally to the attending physician. Viewing the graphic output may have been redundant for the attending physician and therefore was judged less useful.

In this study, the pre- and postassessment questions and the clinician usability questionnaire were adapted with permission from Carlson et al. (2001). The current study’s results support the findings reported by that group. The sample was similar in size to Carlson et al.’s; however, only 23% of the sample group in Carlson et al.’s study used computers frequently (more than once per week) as compared to the current sample group, in which 49% reported that they used computers frequently. In the preassessment portion of both studies, participants reported very similar preferences for face-to-face and computer methods of questionnaires. In the postassessment segment, both studies reported that a majority of participants (65%–70%) found computer questionnaires acceptable. Carlson et al. demonstrated a larger change in the attitudes about computer questionnaires from the preassessment to the postassessment segment of the computer program, which they attributed to the positive experience using the program. In the current study, attitudes regarding computer questionnaires were roughly the same in the preassessment and postassessment segments of the program (70% acceptable). This may be related to the larger group of frequent computer users who participated in this study compared to Carlson et al.’s.

Some differences existed in the clinician survey portion of this study compared to Carlson et al.’s (2001). In the present study, the sample group was made up of four attending physicians, two resident physicians, one nurse practitioner, and three RNs. Because similarities were noted in the responses that each clinician group chose, the authors suspect that this is related to the type of practice that each group assumes. In the Carlson et al. study, the clinician sample group was made up of two doctors and two nurses who completed the survey in reference to the computerized QOL data obtained from 44 patients. They did not report differences in the responses by clinician group. Small sample sizes in both studies limited generalizability.

A major difference between Carlson et al.’s (2001) work and this study is the type of output the computer program generated. In the Carlson et al. study, the computer printed out a text format summary of the patients’ responses and patients were asked to hand carry the form to their clinic visit. In the current study, the computer generated a graph displaying high-level item scores with a red color on a bar chart and a list of QOL items and symptoms ordered by the severity of distress that the participant indicated on the computer program. The output page was placed on the medical chart with the laboratory and vital sign forms used for the day’s clinic visit or handed directly to the provider by the research assistant. These differences may have had an effect on the variation in the responses received from clinicians. Carlson et al. reported that 64% of clinicians agreed that the computer-generated output identified appropriate areas of QOL deficits or concerns, whereas in the current study, 77% of clinicians found this to be true.

This work also confirms the research results of other studies that focused on the feasibility of computerized screening (Taenzer et al., 2000; Velikova et al., 2002; Wilkie et al., 2001). Velikova et al. (2002) studied the acceptance and feasibility of a computer-administered QOL measurement in an oncology clinic in the United Kingdom. They concluded that the computer method was well accepted and that patients reported the information as a useful way to tell doctors about their feelings. In addition, the three physicians who participated in the evaluation of the usefulness and clinical relevance of the information obtained from the assessment found the information to be accurate and clinically relevant.

The current study was limited by the small size of the sample of clinicians and precluded statistical analysis of differences in responses by clinician background. Another issue is the amount of missing data from the postassessment queries, which affects the interpretation of the results. For example, those who did not have time to finish may have responded differently to the postassessment questions. The authors believe that one of the major reasons for the missing data in this study was related to the frequent interruptions in the cancer center lobby, inadequate amount of preappointment time available, and patient tardiness for appointments. These feasibility concerns have been reported elsewhere (Trigg et al., 2003).

The relationships among sample group demographics, such as income, race, and ethnicity, and individual responses on the instruments were not analyzed. In addition, the sample group was described, but other factors, such as types of cancer, types of cancer treatment (e.g., chemotherapy and surgery), or comorbid medical conditions, were not, which may affect the generalizability of the results. This approach to assessment could be tested in chemotherapy infusion areas or medical oncology areas for feasibility and acceptability. With this type of QOL and symptom assessment, clinicians and researchers would be able to measure the change of those factors over time as patients begin and finish cancer treatments. Future studies focused on the impact of serial screening on the levels of distress, symptoms, and QOL have the potential to provide the necessary impetus for the change that would be required to implement routine QOL and symptom assessment into clinical practice.

The data generated from this study indicate that computer-based, patient-entered QOL and symptom assessment is an acceptable means of gathering information. The main implication of this research study for clinical practice is the knowledge that this type of assessment is well accepted by patients with cancer and readily used by clinicians. However, this study indicated that 61% of patients who participated in the computerized survey were not using computers on a frequent basis. Thus, a computerized survey may not be an appropriate way to collect information from some patients outside the clinical setting. Such as system could be set up in a clinical setting, with local support made available to assist these patients so that all patients can benefit from routine screening.

When the implementation of routine screening is considered in a clinical setting, concerns about increasing workload, the utility of the information, patient acceptance, and the change of clinician practice or routine arise. This study confirms that computer-based screening with immediately available results may be a viable means of implementing this change into practice.

The authors wish to acknowledge Terri Whitney and Alexa Vellema, RN, BSN, for their excellent assistance throughout the project.

Batel-Copel, L.M., Kornblith, A.B., Batel, P.C., & Holland, J.C. (1997). Do oncologists have an increasing interest in the quality of life of their patients? A literature review of the last 15 years. European Journal of Cancer, 33, 29–32.

Carlson, L.E., Speca, M., Hagen, N., & Taenzer, P. (2001). Computerized quality-of-life screening in a cancer pain clinic. Journal of Palliative Care, 17, 46–52.

Dewan, N.A., & Lorenzi, N.M. (2000). Behavioral health information systems. Evaluating readiness and user acceptance. MD Computing, 17(4), 50–52.

Drummond, H.E., Ghosh, S., Ferguson, A., Brackenridge, D., & Tiplady, B. (1995). Electronic quality of life questionnaires: A comparison of pen-based electronic questionnaires with conventional paper in a gastrointestinal study. Quality of Life Research, 4, 21–26.

Jemal, A., Tiwari, R.C., Murray, T., Ghafoor, A., Samuels, A., Ward, E., et al. (2004). Cancer statistics, 2004. CA: A Cancer Journal for Clinicians, 54, 8–29.

Kushniruk, A.W., Kaufman, D.R., Patel, V.L., Levesque, Y., & Lottin, P. (1996). Assessment of a computerized patient record system: A cognitive approach to evaluating an emerging medical technology. MD Computing, 13, 406–415.

Kushniruk, A.W., Patel, V.L., & Cimino, J.J. (1997). Evaluation of Web-based patient information resources: Application in the assessment of a patient clinical information system. Proceedings of the American Medical Informatics Association Annual Symposium, 218–222. Retrieved February 25, 2004, from http://www.amia.org/pubs/symposia/D200732.PDF

Lutner, R.E., Roizen, M.F., Stocking, C.B., Thisted, R.A., Kim, S., Duke, P.C., et al. (1991). The automated interview versus the personal interview. Do patient responses to preoperative health questions differ? Anesthesiology, 75, 394–400.

Patel, V.L., & Kushniruk, A.W. (1998). Interface design for health care environments: The role of cognitive science. Proceedings of the American Medical Informatics Association Annual Symposium, 29–37. Retrieved February 25, 2004, from http://www.amia.org/pubs/symposia/D005235.PDF

Pouwer, F., Snoek, F.J., van der Ploeg, H.M., Heine, R.J., & Brand, A.N. (1998). A comparison of the standard and the computerized versions of the Well-Being Questionnaire (WBQ) and the Diabetes Treatment Satisfaction Questionnaire (DTSQ). Quality of Life Research, 7, 33–38.

Roizen, M.F., Coalson, D., Hayward, R.S., Schmittner, J., Thisted, R.A., Apfelbaum, J.L., et al. (1992). Can patients use an automated questionnaire to define their current health status? Medical Care, 30(5, Suppl.), MS74–84.

Skinner, H.A., & Allen, B.A. (1983). Does the computer make a difference? Computerized versus face-to-face versus self-report assessment of alcohol, drug, and tobacco use. Journal of Consulting and Clinical Psychology, 51, 267–275.

Taenzer, P., Bultz, B.D., Carlson, L.E., Speca, M., DeGagne, T., Olson, K., et al. (2000). Impact of computerized quality of life screening on physician behaviour and patient satisfaction in lung cancer outpatients. Psychooncology, 9, 203–213.

Taenzer, P., Sauve, L., Burgess, E.D., Milkavich, L., & Whitmore, B. (1996). The health interview: Automated assessment in a multidisciplinary outpatient hypertension treatment program. MD Computing, 13, 423–426.

Taenzer, P.A., Speca, M., Atkinson, M.J., Bultz, B.D., Page, S., Harasym, P., et al. (1997). Computerized quality-of-life screening in an oncology clinic. Cancer Practice, 5, 168–175.

Trigg, L., Berry, D., Karras, B., Austin-Seymour, M., & Lober, W. (2003). Feasibility of patient entered QOL assessment data in a radiation oncology clinic. In H. Marin, E. Marques, E. Hovenga, & W. Goosen (Eds.), eHealth for all: Designing nursing agenda for the future [CD-ROM]. Sau Palo, Brazil: NI 2003.

Turner, C.F., Ku, L., Rogers, S.M., Lindberg, L.D., Pleck, J.H., & Sonenstein, F.L. (1998). Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science, 280, 867–873.

Velikova, G., Brown, J.M., Smith, A.B., & Selby, P.J. (2002). Computer-based quality of life questionnaires may contribute to doctor-patient interactions in oncology. British Journal of Cancer, 86, 51–59.

Velikova, G., Wright, E.P., Smith, A.B., Cull, A., Gould, A., Forman, D., et al. (1999). Automated collection of quality-of-life data: A comparison of paper and computer touch-screen questionnaires. Journal of Clinical Oncology, 17, 998–1007.

Wilkie, D.J., Huang, H.Y., Berry, D.L., Schwartz, A., Lin, Y.C., Ko, N.Y., et al. (2001). Cancer symptom control: Feasibility of a tailored, interactive computerized program for patients. Family Community Health, 24(3), 48–62.

Kristin H. Mullen, MN, ARNP, AOCN^®, is a nurse practitioner for the Palliative Care Service in the Department of Veterans Affairs at the VA Puget Sound Health Care System in Seattle, WA; and Donna L. Berry, PhD, RN, AOCN^®, and Brenda K. Zierler, PhD, RN, are associate professors in Biobehavioral Nursing and Health Systems at the University of Washington in Seattle. At the time this study was conducted, Mullen was a graduate student at the University of Washington and received an educational scholarship from the American Cancer Society. (Submitted May 2003. Accepted for publication March 25, 2004.)

Author Contact: Kristin H. Mullen, MN, ARNP, AOCN^®, can be reached at kmullen@u.washington.edu, with copy to editor at rose_mary@earthlink.net.

Are you an ONS member? Click here to learn about the many benefits ONS has to offer, including access to the full article text of the Oncology Nursing Forum and Clinical Journal of Oncology Nursing.

To obtain a full text copy of this article, please visit the Library and Archives section of the ONS Website.