Integrative Review on the Effectiveness of Internet-Based Interactive Programs for Women With Breast Cancer Undergoing Treatment

Women with breast cancer who are undergoing active treatment, including surgery, chemotherapy, or radiation therapy, may suffer from a number of symptoms that often interfere with their lives (Cheng, Wong, Ling, Chan, & Thompson, 2009; Dodd, Cho, Cooper, & Miaskowski, 2010). Symptom management is crucial for women with breast cancer, and the reduction of symptom distress is a critical indicator of successful psychosocial support (Ruland et al., 2013). Self-efficacy is a crucial concept in symptom management because it determines how women think, feel, self-motivate, and perform (Bandura, 1977). Social support also plays an important role in how women with breast cancer deal with their diagnoses and treatments (Lou, Yates, McCarthy, & Wang, 2013; Smith et al., 2011). Patients’ symptom distress, self-efficacy, and social support are three interrelated components that influence a person’s ability to cope with chronic disease (Hunt et al., 2012; Lou et al., 2013). In addition, women with breast cancer undergoing treatment are reported to experience poor quality of life (QOL) and psychological well-being (So et al., 2010; Tsitsis & Lavdaniti, 2014). Therefore, healthcare professionals must integrate QOL and psychological well-being assessment in practice (So et al., 2010).

The National Institute of Nursing Research recommends interventions that improve self-efficacy and social support, as well as involve provider–client interaction and follow-up activities to improve patients’ symptom management in chronic conditions (Grey, Knafl, & McCorkle, 2006). However, the growing number of women with breast cancer and the increasing shortage of oncology healthcare providers pose challenges to the feasibility of offering clinic-based, face-to-face supportive interventions as recommended (Moyer, Sohl, Knapp-Oliver, & Schneider, 2009). Internet-based interactive programs (IIPs) provide an innovative, sustainable, and easily accessible platform for interventions where patients and healthcare providers can interact with each other to access or transmit health information or support (Murray, Burns, See, Lai, & Nazareth, 2005).

A Cochrane review investigating the effectiveness of computer-based interactive programs for people with chronic disease concluded that those programs could offer a more flexible and dynamic method of addressing patient needs, with significant positive effects on knowledge, social support, clinical outcomes, and continuous behavioral outcomes (Murray et al., 2005). This review focused on efficacy of Internet-based programs on chronic non-cancer populations. Another systematic literature review reported preliminary but inconclusive evidence of positive outcomes for various cancer survivors who used online support and resources (Hong, Peña-Purcell, & Ory, 2012). The review included 24 studies (37 articles) but did not specifically target breast cancer. The majority of articles did not tell whether the participants were undergoing treatment or were post-treatment. Because Internet-based support may provide the greatest benefits while patients are in the early stages of cancer diagnosis and undergoing treatment (Baker et al., 2011), women with breast cancer undergoing treatment may have different experiences than cancer survivors post-treatment.

Another systematic review found that Internet- or computer-based interactive education could increase breast cancer knowledge among patients with breast cancer (Ryhänen, Siekkinen, Rankinen, Korvenranta, & Leino-Kilpi, 2010). This review was conducted in 2008 and identified 14 articles, including 7 articles reporting on educational interactive computerized programs not requiring Internet access. There were only two randomized, controlled trials that explored the effect of an Internet-based program in this review. Ryhänen et al. (2010) suggested that computer-based programs were easy to install or implement via the Internet. With the rapid advancements in technology and increasing use of the Internet by women seeking information about breast cancer, many IIPs have been developed in the past decade to address health needs. Synthesizing the existing body of evidence, including studies published regarding the efficacy of IIPs, is important to facilitate healthcare delivery for women with breast cancer.

The previous review was limited to quantitative articles published in English (Ryhänen et al., 2010). Qualitative articles evaluating women’s experiences using IIPs may provide insight into the practicality and acceptability of IIPs (Dubois & Loiselle, 2009). Assessing the effects of online resources in other languages is also important even though English represents a majority of online resources (Hong et al., 2012). China is the most populous country in the world, and breast cancer is the most common cancer among Chinese women (Fan et al., 2014). The incorporation of articles published in Chinese will contribute to the evaluation of IIPs from a global perspective. The current review, which integrates quantitative and qualitative articles published in English or Chinese, will contribute to a deeper understanding of the effectiveness of IIPs for women with breast cancer undergoing treatment.

The aim of the integrative review was to synthesize studies published in English or Chinese regarding the effectiveness of these programs on the outcomes of symptom distress, social support, self-efficacy, QOL, and psychological well-being for women with breast cancer undergoing treatment.

Design

This was an integrative review of quantitative and qualitative empirical evidence. The review was conducted using the following five-stage approach: problem identification, literature search, data evaluation, data analysis, and presentation (Whittemore & Knafl, 2005).

Search Methods

A computerized search was performed of the literature published in English and Chinese from the beginning of each database, which ranges from 1806–2007, to April 2015. The following databases: CINAHL Complete, MEDLINE®, Mosby’s Nursing Index, PsycINFO®, Scopus, Web of Science, Joanna Briggs Institute, Cochrane Library, Embase, and China National Knowledge Infrastructure (CNKI) were searched. The keywords used for the search were “breast neoplasm OR cancer* OR tumour* OR carcinoma*” AND “interactive program OR patient education OR intervention OR support OR system OR teaching OR instruction* OR program*” AND “web OR online OR Internet OR e-health OR www OR mobile application OR e-intervention OR computer OR e-technology.” A manual search within reference lists was also performed.

An IIP is defined as an Internet-based package for patients that combines health information with at least one form of Internet-based support, such as social support, decision support, or behavioral change support (Murray et al., 2005). Articles were eligible for review with the following inclusion criteria: (a) professional- or peer-led support or a combination of these types of support, with the Internet accessed by computer or mobile phone or other mobile devices; (b) original quantitative articles, including randomized, controlled trials, quasiexperimental studies, or one-group pre-/post-test studies; (c) original qualitative articles exploring women’s experience or evaluation of an Internet-based support program; (d) evaluated outcomes of symptom distress, social support, self-efficacy, QOL, or psychological outcomes; (e) participants were women with any stage breast cancer undergoing treatment; and (f) articles were written in English or Chinese. Articles with mixed cancer populations were eligible for inclusion if the data for women with breast cancer could be separated. Because treatment can last six months to one year after breast cancer diagnosis (McCann, Illingworth, Wengström, Hubbard, & Kearney, 2010), the review included women as many as 12 months postdiagnosis of breast cancer. The exclusion criteria were as follows: (a) articles examining Internet-based programs for breast cancer screening, self-examination, mammography, prevention, detection, or genetic counseling; (b) articles examining a combination of face-to-face, telephone, and Internet-based communication; (c) articles examining the use of the computer with interactive CD-ROMs without access to the Internet; and (d) unpublished dissertations and editorials.

Search Outcome

The initial search yielded 329 articles. After duplicates were eliminated, 174 articles were retrieved. A preliminary review of the articles was undertaken by two authors, assessing the abstract, participant characteristics, and measured outcomes against the inclusion and exclusion criteria. The Chinese articles were read in their original language and discussed for eligibility by two authors who are bilingual, with Chinese as their first language. The reasons for exclusion were primarily based on participants’ characteristics (e.g., mixed cancer samples or cancer survivors not receiving any treatment) or outcomes instruments (e.g., no targeted outcomes). Twenty-three relevant articles were identified. A manual search was conducted by reviewing the reference lists of all 23 included articles, which led to an additional 16 articles. Five of 16 articles were included using the inclusion and exclusion criteria. The 28 articles were reviewed by three authors. Dubious articles were discussed among the three authors and were included if there was a consensus. Eighteen articles (15 quantitative and 3 qualitative) were selected and validated for the review (see Figure 1).

Quality Appraisal

The 15 quantitative studies were reviewed following Guo, Whittemore, and He’s (2011) framework, which was used to appraise the quality of studies on the basis of methodology (e.g., sampling, research design) (see Table 1), and the three qualitative studies were reviewed with Sandelowski and Barrosa’s (2002) framework for methodologic rigor, which followed methodologic description to assess the quality of studies (e.g., data collection, data management) (see Table 2). This evaluation system was applied to assist in analyzing data.

Data Abstraction

To accomplish the data reduction and data display, a review matrix was developed to collect structured data. The domains in the review matrix included program, studies, authors, year, country, design, sample size, comparison, outcomes measured, and key findings (see Table 3). Several articles were displayed as one study if they reported findings from different perspectives of the same study.

Data Synthesis

Data synthesis was conducted using the sequence proposed by Whittemore and Knafl (2005): data reduction, data display, data comparison, drawing of conclusions, and verification. The initial subgroup classification was based on study setting and type. The domains in the matrix were analyzed and compared to identify similarities and differences across articles.

[[{"type":"media","view_mode":"media_original","fid":"30216","attributes":{"alt":"","class":"media-image","height":"488","typeof":"foaf:Image","width":"1062"}}]]

Results

The 18 articles reviewed reported on 10 studies. Six studies were undertaken in the United States, two in mainland China, one in Canada, and one in the United Kingdom. The participant sample size ranged from 78–450 for quantitative studies and from 10–56 for qualitative studies. The mean age of the patients across all studies was 50.3 years. Seventy-four percent of patients had early-stage (stages 0–II) breast cancer, and 26% percent of patients were classified as having late-stage (stages III–IV) breast cancer. Five studies reported the participants’ medical treatments: 62% of all women received chemotherapy, and 26% of women received radiation therapy during the study periods.

[[{"type":"media","view_mode":"media_original","fid":"30221","attributes":{"alt":"","class":"media-image","height":"1065","typeof":"foaf:Image","width":"760"}}]]

[[{"type":"media","view_mode":"media_original","fid":"30226","attributes":{"alt":"","class":"media-image","height":"1062","typeof":"foaf:Image","width":"760"}}]]

[[{"type":"media","view_mode":"media_original","fid":"30231","attributes":{"alt":"","class":"media-image","height":"755","typeof":"foaf:Image","width":"760"}}]]

Intervention

The IIPs had durations from 2 weeks to 12 months. Of the included 10 studies, different designs were used to test the interventions. Six articles used a randomized, controlled trial design; five used a quasiexperimental design; and four used a one-group pre-/post-test design. All of the randomized, controlled trials were conducted in the United States.

The 10 studies reported six distinct IIPs: Comprehensive Health Enhancement Support System (CHESS), Oncology Interactive Educational Series (OIES), Advanced Symptom Management System (ASyMS), an Internet-based peer-to-peer interaction program, an Internet-based support program, and a mobile phone–based symptom management program.

CHESS is an Internet-based system combining an information service, a communication service, and an interactive coaching service (Baker et al., 2011). A CHESS staff member reads patients’ messages daily, provides advice when necessary, and intervenes when messages turn hostile or advocacy of “quack” cures exists (Hawkins et al., 2010). OIES is an eight-week multimedia informational intervention involving a one-hour information technology training session, a list of reputable cancer websites, and a specialized interactive CD-ROM (Loiselle, Edgar, Batist, Lu, & Lauzier, 2010). During the OIES program, a research staff member is available to answer any questions by telephone or email (Loiselle & Dubois, 2009). ASyMS and the mobile phone–based symptom management program were developed to monitor symptoms and support patients receiving chemotherapy with the use of a handheld computer or mobile phone at home. In those programs, healthcare providers viewed patients’ symptom reports from the website and offered tailored self-care advice for clinical decision making on a mobile phone (Maguire et al., 2005; Wang et al., 2015). The Internet-based support program was moderated by healthcare providers. The healthcare providers provided health online consultation and organized online discussion topics weekly (Huang, Hu, Lu, Huang, & Shen, 2011b). The unstructured peer-to-peer Internet interaction program did not have a professional moderator (Salzer et al., 2010).

Two of six IIPs discussed the use of a theoretical framework to support their development. OIES adopted the interactional model, where personal and contextual resources modulated women’s psychosocial adjustment to breast cancer and technology served as the contextual informational support (Loiselle et al., 2010). CHESS applied the self-determination theory, which hypothesized that QOL was largely dependent on the degree to which three basic psychological needs (i.e., autonomy, competence, and relatedness) were met (Gustafson et al., 2008).

All of the interventions were user-driven and accessible from home. The interventions used were described as a computer-based interactive program or a mobile phone–based symptom management system. Interaction was available in all six programs: four allowed feedback from healthcare providers (CHESS, ASyMS, Internet-based support program, mobile phone–based symptom management program), one offered CD-ROM feedback (OIES), and one provided feedback from peers (Internet-based peer-to-peer interaction program).

Effectiveness of the Interventions

Outcomes were measured pre- and post-test at different points in the studies. Timing of post-test assessments ranged from 2 weeks to 12 months postintervention. IIPs were tested or observed for their effectiveness on symptom distress, self-efficacy, social support, psychosocial well-being, and QOL.

Symptom distress: Three studies examined the effect of IIPs on symptom distress. The mobile phone–based symptom management program was reported to significantly improve symptom distress (p < 0.05 during four cycles of chemotherapy) (Wang et al., 2015). A qualitative study reported that women undergoing chemotherapy found ASyMS to be helpful for monitoring and managing their symptoms at home (Maguire et al., 2005). However, women exposed to unmoderated Internet-based peer-to-peer interaction experienced more symptom distress than the control group (p = 0.1 at 4 months; p = 0.11 at 12 months) (Salzer et al., 2010).

Self-efficacy: Three studies tested the effectiveness of IIPs for improving self-efficacy. CHESS (p = 0.028 at nine months) and the mobile phone–based symptom management program (p < 0.05 during the four cycles of chemotherapy) significantly enhanced participants’ self-efficacy (Gustafson et al., 2008; Wang et al., 2015). Disclosure of insights (e.g., thoughts, knowledge, considerations) via online communication support led to greater improvements in self-efficacy (p < 0.05 at four months) (Shim, Cappella, & Han, 2011). However, unmoderated Internet-based peer-to-peer interaction contributed no benefit to women’s self-efficacy (p = 0.06 at 4 months; p = 0.09 at 12 months) (Salzer et al., 2010).

Social support: There were inconsistent results for the effect of the IIPs on social support. Four studies with quantitative data reported that IIPs increased social support during and after the intervention (p = 0.004 at four months and p = 0.021 at nine months [Gustafson et al., 2008], p < 0.01 at five months [Gustafson et al., 2001], p = 0.00 at five months [Gustafson et al., 2005], p < 0.05 at four months [Han et al., 2009]), improved satisfaction with the cancer information received over time (p < 0.01 at one week after intervention [Loiselle et al., 2010]), and improved perceived oncologist informational support (p < 0.001 at one week postintervention [Loiselle & Dubois, 2009]). Two studies with qualitative data reported that IIPs promoted communication with doctors and nurses (Maguire et al., 2005), facilitated information exchange, and helped patients build a network with healthcare providers or women with similar conditions (Huang, Hu, Lu, Huang, & Shen, 2011a). One study did not find a positive effect on social support by the CHESS program when it was compared with Internet access only (p = 0.28 at six weeks and p = 0.26 at three months) (Baker et al., 2011; Hawkins et al., 2011).

Psychosocial well-being: IIPs significantly improved emotional processing (p = 0.02 at three months) and positive coping (p = 0.03 at three months) (Baker et al., 2011), reduced negative emotions (p = 0.004 at five months [Gustafson et al., 2005], p < 0.05 at five months [Shaw, Hawkins, McTavish, Pingree, & Gustafson, 2006]), and decreased depression among women with breast cancer (p < 0.05 at three months) (Huang et al., 2011b). Emotional communication competence was reported to moderate the effectiveness of giving and receiving emotional support on psychological QOL and concerns related to breast cancer when receiving CHESS (Yoo et al., 2014). Two studies with qualitative data reported that women felt reassured and relaxed when using ASyMS at home (Maguire et al., 2005), and women were less anxious after receiving OIES because they knew what to expect and how to prepare for each step of the illness trajectory (Dubois & Loiselle, 2009). The unmoderated Internet-based peer-to-peer interactions did not contribute to improvements in mood (p = 0.008 at 4 months and 12 months). The intervention group did not achieve a significant difference in distress at 4 and 12 months. The control group reported marginally better scores than the intervention group on hope (p = 0.06 at 12 months) (Salzer et al., 2010). The value for the intervention group was not reported in the original article.

Quality of life: This review found inconsistent results regarding the impact of the IIPs on QOL. Gustafson et al. (2008) reported that the CHESS participants scored significantly higher on QOL than a group with Internet access alone (CHESS: p = 0.029 at two months, p = 0.047 at four months; Internet only: p = 0.84 at two months, p = 0.44 at four months). One study reported no significant effects of CHESS on QOL when CHESS was compared to Internet access alone (p = 0.42 at six months for CHESS group) (Baker et al., 2011; Hawkins et al., 2011). Women derived no benefits regarding QOL from unmoderated Internet-based peer-to-peer interaction (p = 0.05 at 4 months, p = 0.03 at 12 months), and control groups reported better scores than the intervention group (Salzer et al., 2010).

Discussion

This review explored what IIPs have been developed and tested for women with breast cancer undergoing treatment. The review identified six IIPs, which suggests that IIPs available for this group of women are limited. The limited available evidence suggested that IIPs could be effective in improving self-efficacy and psychosocial well-being and reducing symptom distress. With the increasing availability of and access to online resources, women with breast cancer are more likely to search the Internet for information and social support. Therefore, a need exists for more well-designed evidence-based IIPs to meet this population’s informational and supportive needs (Ryhänen et al., 2010).

The articles published in English were conducted in North America and Europe, which shows that IIPs have become important healthcare interventions for women with breast cancer in those regions, where breast cancer incidence is highest (Ferlay et al., 2015) and women are more efficient Internet users (Eysenbach, 2003). Although this review only identified three articles published in Chinese (Huang et al., 2011a, 2011b; Wang et al., 2015), the limited available articles on IIPs showed promise in that online resources have gained recognition as worthwhile tools for educating women on breast cancer in China.

Different designs were adopted for testing the effectiveness of the interventions. The current review found only three qualitative articles on how participants used and appraised IIPs. Relatively few randomized, controlled trials were used in the reviewed studies. All six of those trials were conducted in the United States, and five referred to the same IIP program. Financial constraints and ethical considerations may have precluded random assignment of patients in Canada and China, reducing the robustness of findings from those settings (Huang et al., 2011b; Loiselle et al., 2010). Future studies need to employ a variety of different designs to allow for the development, testing, and evaluation of interventions for individual groups of patients.

Most IIPs did not adopt a theoretical framework to develop the program or select outcomes, with the exception of CHESS and OIES, both of which aligned the interventions and outcomes with a theoretical framework (Gustafson et al., 2008; Loiselle et al., 2010). Programs lacking a theoretical framework or structure are difficult to replicate or adapt across cultures (Lustria, Cortese, Noar, & Glueckauf, 2009). None of the reviewed studies identified or reported group communication structures, such as a stated group mission and purpose, or specific content to be discussed. A group communication structure may be necessary for ensuring efficacy in group interactions and limiting the potential for harmful communications (Salzer et al., 2010).

The current review found that unmoderated Internet-based peer-to-peer interactions made no contribution to women with breast cancer who were undergoing treatment (Salzer et al., 2010), but moderated programs had positive effects on symptom distress, self-efficacy, and psychological well-being (Gustafson et al., 2005, 2008; Huang et al., 2011b; Shaw et al., 2006; Wang et al., 2015). No included studies tested how different formats (moderated or peer-led) might affect health outcomes within the same IIP. Different studies applied different programs with or without moderators, making it difficult to clarify the efficacy of moderators on health outcomes. Moderators could provide a communication structure, facilitate online discussions, and intervene appropriately if messages became hostile (Bantum et al., 2014; Hawkins et al., 2010; Klemm, 2012). Klemm (2012) found that the presence of moderators increased participation in the online discussion, but there is no significant difference on depressive symptoms for women with breast cancer by different online support group format (moderated versus peer-led). No analysis of cost-effectiveness has been reported to compare moderated IIPs with peer-led programs for women with breast cancer. More studies are warranted to pay particular attention to the effectiveness of a professional moderator in an IIP and the cost-effectiveness of professionally moderated services.

Studies included in this review reported limited follow-up periods of typically less than six months, with the exception of two studies with 9- and 12-month follow-ups (Gustafson et al., 2008; Salzer et al., 2010). The short follow-up periods hindered the ability to demonstrate the long-term effects of IIPs.

Different outcome indicators and instruments were used in the reviewed studies. Some instruments were researcher-designed instruments and applied only to the CHESS (Gustafson et al., 2005, 2008; Shim et al., 2011) or OIES (Loiselle & Dubois, 2009; Loiselle et al., 2010) programs. Comparing results is difficult with a lack of standardized outcome measurement instruments.

Symptom distress and self-efficacy are critical outcomes to improve for a successful health-management program (Ruland et al., 2013). This review found limited studies that tested the effects of moderated IIPs on symptom management and self-efficacy. Only two studies reported a positive effect of moderated IIPs on symptom distress, and they were not randomized, controlled trials (Maguire et al., 2005; Wang et al., 2015). Reliance on a quasiexperimental design or qualitative study as opposed to a randomized, controlled trial has inherent limitations (Loiselle et al., 2010). Only three reviewed studies used self-efficacy as an outcome measure for moderated IIPs (Gustafson et al., 2008; Shim et al., 2011; Wang et al., 2015). Future studies with more rigorous designs should place more importance on the impact of IIPs on symptom distress and self-efficacy.

In this review, the effectiveness of moderated IIPs on social support and QOL remained inconclusive. The majority of IIPs were found to have positive effects on social support and QOL (Gustafson et al., 2008; Han et al., 2009; Huang et al., 2011a; Loiselle & Dubois, 2009; Loiselle et al., 2010; Shaw et al., 2006; Shim et al., 2011). For the same CHESS program that was implemented and tested for more than 10 years, one study reported no significant effects on social support and QOL compared with Internet use only (Baker et al., 2011; Hawkins et al., 2011); however, an earlier study found that CHESS had positive effects when compared to Internet access only (Gustafson et al., 2008). These inconclusive findings may be because of the improvement of Internet access and resources over time; with improvements to other sites, CHESS no longer had an edge with regard to integration and ease of navigation (Baker et al., 2011). Future IIPs should be developed with features that can evolve with technology and new information to meet the increasing needs of women with breast cancer.

Psychological well-being has been widely tested in the reviewed studies. It was typical for a study to find significant differences for some, but not all, of the psychological outcomes measured. Some psychological outcomes were measured in only one study (Huang et al., 2011b; Loiselle et al., 2010). Standardizing certain outcome measurement instruments is warranted to enable comparisons across articles (Hong et al., 2012). For example, the Hospital Anxiety and Depression Scale is widely used in evaluations of psychological well-being and could be incorporated into assessments of the effectiveness of IIP (van den Berg, Gielissen, Ottevanger, & Prins, 2012).

Limitations

This review included only studies reported in the English and Chinese languages. Relevant studies published in other languages may have been omitted in this review. The reviewed studies disproportionately focused on CHESS, with half of all included studies examining different aspects and different participants of CHESS. Such unbalanced available literature could lead to difficulties in generalization. This review only selected five health outcomes to test the effectiveness of IIPs, which narrowed the literature selected. To get a more comprehensive picture of IIPs, future work could include other health outcomes to test effectiveness, such as knowledge, communication, and empowerment.

Implications for Nursing

This review suggested that IIPs have potential to support women with breast cancer undergoing chemotherapy and to improve their health outcomes. IIPs provide an easily accessible option for women with breast cancer who are unable to attend face-to-face support services. The review found limited IIPs available for women with breast cancer undergoing chemotherapy. Healthcare providers, particularly nurses, are in a prime position to develop IIPs as routine care for women. Policy makers, hospital administrators, and nurse administrators from healthcare organizations can train staff in eHealth and allocate resources to further advance this effort.

Evidence suggests that unmoderated IIPs made no contribution to women’s health outcomes (Salzer et al., 2010), but moderated IIPs have been shown to be promising interventions with consistent positive effects on symptom distress, self-efficacy, and psychological well-being for women with breast cancer undergoing treatment (Gustafson et al., 2005, 2008; Huang et al., 2011b; Shaw et al., 2006; Wang et al., 2015). Nurses working with this population could provide continuity of care through moderating IIPs. Nurses can also provide guidance and advice to women to select available IIPs that are moderated. To explore whether a healthcare provider as moderator is beneficial to ensure efficacy of group interactions, more studies are warranted to compare the effect of different formats (moderated versus peer-led) within the same program and the cost-effectiveness of professionally moderated services.

The review identified limited studies with rigorous design. Studies with more robust research designs and theoretical frameworks, conducted in different countries and cultures, are warranted to elucidate the effectiveness of IIPs.

Conclusion

IIPs are promising for the potential application in research and practice for women with breast cancer undergoing treatment. It offers an innovative and easily accessible supportive care with positive effects on symptom distress, self-efficacy, and psychological well-being for women with breast cancer. Knowledge gaps remain in existing studies regarding methodologic aspect of the design and evaluation. Theoretical frameworks are needed to help the development and assessment of IIPs, improving the practicability and credibility of IIPs across different cultures and populations.

About the Author(s)

Zhu is an associate professor in the Department of Nursing and School of Medicine at Xiamen University in Fujian, China, and a doctoral student in the School of Nursing and Midwifery at the University of Newcastle in Australia; and Ebert is the Deputy Head of School, and Chan is a professor, dean of nursing, and school head, both in the School of Nursing and Midwifery, Faculty of Health and Medicine at the University of Newcastle in Australia. This study was supported by the National Natural Science Foundation of China (71503219). Zhu and Chan completed the data collection and provided the statistical support and analysis. All of the authors contributed to the conceptualization and design and the manuscript preparation. Zhu can be reached at jiemin.zhu@uon.edu.au, with copy to editor at ONFEditor@ons.org. Submitted February 2016. Accepted for publication June 9, 2016.