Short Communication

Swallowing Exercises to Prevent Dysphagia Post-Treatment in People Treated for Advanced Stage Head and Neck Cancer; Addressing the Research Shortfalls

Alison R. Perry1* and Susan M. Cotton2,3
1Department of Clinical Therapies, Faculty of Education and Health Sciences, The University of Limerick, LIMERICK, Ireland
2Orygen, The National Centre of Excellence in Youth Mental Health, 35 Poplar Road, Parkville, Melbourne, VIC 3052, Australia
3The Centre for Youth Mental Health, University of Melbourne, 35 Poplar Road, Parkville, Melbourne, VIC 3052, Australia


*Corresponding author: Alison Perry, Faculty of Education & Health Sciences, The University of Limerick, LIMERICK, Ireland

Published: 08 Aug, 2017
Cite this article as: Perry AR, Cotton SM. Swallowing Exercises to Prevent Dysphagia Post-Treatment in People Treated for Advanced Stage Head and Neck Cancer; Addressing the Research Shortfalls. Clin Oncol. 2017; 2: 1324.

Abstract

Background: Advances in treating advanced Head and Neck Squamous Cell Cancer (HNSCC) have meant improved mortality and survival rates over the last decade, but significant morbidity (specifically, dysphagia) often persists. Therapeutic swallowing exercises are often implemented, without the evidence for such intervention being established.
Methods: This is a critique of the studies sourced for our Cochrane systematic review, which was completed to determine the benefit of therapeutic swallowing exercises undertaken before, during, and/or immediately after treatment, in people treated with curative intent for advanced stage HNSCC.
Findings: We found no evidence that therapeutic exercises were better than treatment as usual, or any other treatment, in improving the safety and efficient of oral swallowing (primary outcome). Shortfalls in the published study designs included; a lack of agreement about primary outcomes, about suitable tools used to measure them, and the choice of baselines and endpoints across studies. In this paper, we make suggestions of ways to improve the methodological quality of future studies.
Keywords: Cancer of the head and neck; Squamous cell carcinoma; Deglutition disorders; Oropharyngeal dysphagia; Swallowing rehabilitation exercises


Introduction

Worldwide, Head and Neck Cancer (HNC) accounts for more than 550,000 cases annually recorded. Over 80% of these cancers originate in the squamous cells that line the mucosal surfaces of the mouth, nose and throat. In the past decade, there has been a rise in the incidence of oropharyngeal squamous cell cancer or SCC (specifically of the lingual and palatine tonsils) associated with Human Papilloma Virus (HPV) occurring in white men, younger than 50 years, who often have no history of alcohol abuse or smoking [1].
Survival rates of HNSCC have improved slightly in the last decade, but survival remains poor as people (mainly men) often present late to treatment. Five year relative survival across Europe ranges from 20% in Slovakia to 51% in the Netherlands [2]. Despite greater chances of survival from more aggressive treatments, regrettably morbidity has increased alongside the improved survival rates.
Treatments for advanced HNC include surgery and radiotherapy, surgery alone, or chemoradiotherapy without surgery. The benefits of concurrent chemotherapy and radiation therapy on survival were confirmed by a meta-analysis of results from 84 trials by Pignon et al. [3]. Currently, goals of treatment are to improve survival outcomes, but also to maintain organ preservation. Unfortunately, preserving an organ does not always result in preservation of its function, so functional morbidity is common during and after treatment for advanced (T3,T4) HNSCC.
Morbidity includes dysphagia (swallowing disability) which may be due to the cancer itself or may occur during, or after, aggressive HNC treatment. Dysphagia presents differently at each stage of the care pathway.
Pre-treatment dysphagia may be due to a (large) tumour compressing oral/ pharyngeal anatomical structures or because of comorbid cranial nerve damage (e.g., of IX, X, XI and/or XII cranial nerves). Dysphagia is not always signified by clinical signs (such as coughing, choking, or reports of food ‘sticking’ when swallowing) as people may adapt to a gradual onset of dysphagia and not be aware of its severity/consequences.
Peri-treatment dysphagia is often acute and is related to radiotherapy treatment side-effects, such as xerostomia, mucositis, or candida. Although often transitory, such painful side-effects may compromise nutritional intake and then alternative feeding (via a nasogastric or a percutaneous endoscopic gastrostomy - PEG - tube) may be implemented. If this continues for an extended period, patients may have difficulty in re-commencing oral feeding, so a structured dysphagia management programme, in consultation with a dietician and a Speech and Language Therapist (SLT), will need to be implemented to reassure the patient and provide a safe, staged return to oral feeding.
Post-treatment dysphagia occurs due to tissue fibrosis (postradiotherapy) and/or a stricture formation. To prevent such negative sequelae from treatment, many SLTs have adopted a philosophy of, ‘use it or lose it’ and insist that patients continue oral swallowing (albeit with modification to the texture and/ or volume of foods, fluids, consumed) throughout (often painful) cancer treatment. Additionally, some clinicians promote ‘swallowing exercises’ for patients throughout radiotherapy treatment in an endeavour to prevent/reduce post-treatment dysphagia.
To reduce the survivorship burden for patients and families, as well as costs to healthcare systems, it is essential to prevent/reduce treatment-related dysphagia in people treated for advanced HNC. However, work to date has been focussed on designing suitable rehabilitation (rather than preventative) programmes with adults who have acquired dysphagia.
In 1999, Logemann [4] introduced and developed the concept of therapeutic swallowing exercises. These are direct techniques, implemented to either improve swallowing safety (i.e., reduce/ eliminate aspiration of food, fluids, swallowed) or to improve swallowing efficiency (by increasing the speed and the amount of a bolus swallowed). Such exercises change swallowing physiology, either by improving sensory-motor integration or by gaining voluntary control over the timing or co-ordination of selected oropharyngeal movements during swallowing.
Swallowing therapy may also involve neuromuscular exercises [4] to increase tongue strength, endurance, power, and range of motion. Strength is achieved by exercises using high levels of resistance (isometric exercises). Endurance is achieved by repeated performance of low resistance exercises, and power is achieved by using exercises that focus on the speed of muscle contraction [5]. Exercises to maintain or increase a tongue’s range of motion may be used to keep the tongue flexible during treatment [6] and tongue base exercises are used to mitigate against the stiffening or fibrosis that can result from radiotherapy and/or from surgery [7].
In our recently published Cochrane systematic review [8] we examined the effect of undertaking such therapeutic swallowing exercises before, during, or immediately after HNC treatment, on safe oral swallowing, on aspiration and on adverse events, such as chest infections, aspiration pneumonia and profound weight loss, in people treated with curative intent for advanced stage (T3,T4) HNSCC.
The purpose of this paper is to elaborate on the findings from that review and to highlight the methodological deficiencies in published studies. We propose improvements to study designs and measurements that will enhance the rigour of future studies of people with dysphagia associated with HNC.


Materials and Methods

A standard Cochrane review methodology was applied to examine the effect of therapeutic exercises for affecting post-treatment swallowing in people treated for advanced stage HNC [8]. This paper addresses the shortcomings of the studies reviewed, specifically focussing on issues of weak methodology and of bias.
Only Randomised Controlled Trials (RCTs) were included. We included studies involving adults with a clinical and histological diagnosis of large (T3,T4) HNC who received any type of cancer treatment (surgery and radiotherapy, surgery and chemoradiotherapy, or chemo-radiotherapy without surgery) who were deemed at risk of, or presented with, dysphagia. We included all levels of dysphagia and set no age limits.
We examined direct therapeutic techniques involving swallowing and neuromuscular exercises. Therapy programmes had to last for more than one session, could have been provided by one or more disciplines, might be delivered pre-, peri- or immediately post-HNC treatment, and were designed to either reduce or prevent dysphagia from occurring.
The main comparison was therapeutic exercises versus Treatment As Usual (TAU). Other comparisons included therapeutic exercises versus sham exercises; sham exercises versus TAU; and therapeutic exercises plus TAU versus TAU.
Primary outcomes were safety and efficiency of oral swallowing, respectively measured by reduced/no aspiration, assessed using a standardised scale [9] and Oropharyngeal Swallowing Efficiency (OPSE) which is a measure of the amount of a bolus (usually a measured 3 ml or 5 ml spoonful of food or fluid) swallowed, divided by the time taken to swallow it [10]. Both measures are taken from Video-Fluoroscopic Swallowing Studies (VFSSs).
Secondary outcomes included self-reported changes to healthrelated or swallowing-related Quality Of Life (QoL); changes to psychological wellbeing (stress, anxiety, depression); patient satisfaction with intervention; patient compliance with intervention; and cost-effectiveness of the intervention.
We used the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) approach [11] to rate the overall quality of evidence. This reflects the extent to which we are confident that an estimate of effect is correct and we applied this to the interpretation of results. There are four possible ratings; high, moderate, low and very low.


Results and Discussion

We retrieved a total of 627 records through database searching which, when 184 duplicated were removed, became 443 papers. Screening out non-relevant references left 25 papers for further consideration. After closer scrutiny, 11 were discarded and 4 studies were still recruiting participants. Two researchers then independently assessed the full texts of the 10 remaining papers; three were excluded as they were not RCTs and one was excluded because the interventions were intended to be rehabilitative rather than preventative. After consultation with a third researcher, we included six studies [12-17] that met our inclusion criteria for the Cochrane review [8].
The six studies had a total of 326 participants who undertook swallowing exercises before, during, and/or immediately after HNC treatment. We could not combine the results of the studies because of the variation in participants’ cancers, their treatments, outcomes measured and tools used to assess them, as well as differing time points chosen for testing.
Researchers compared therapeutic exercises versus Treatment As Usual (TAU); therapeutic exercises versus sham therapy; therapeutic exercises plus TAU versus TAU. The exercises varied considerably in their design, timing and intensity. TAU involved managing patients’ dysphagia when it occurred and, when severe, this might involve inserting either a nasogastric (NG) or a Percutaneous Endoscopic Gastroscopy (PEG) for non-oral feeding.
There is presently insufficient evidence to draw clear conclusions about the effects of undertaking therapeutic swallowing exercises before, during, or immediately after HNC treatment to prevent/ reduce dysphagia. Studies had low participant numbers, used complex (often poorly tolerated) interventions, and varied in the choice of primary and secondary outcomes, making it difficult to draw reliable conclusions.
Although there were no reported adverse events that could be directly attributed to the therapeutic intervention (swallowing exercises), the high attrition rates across all studies indicates participants’ difficulty in tolerating and accepting the exercise regimens.
We suggest that addressing the following issues would enhance the design of future studies, and hence lead to improved study outcomes.
There is a need to more clearly operationalise terms. First, ‘TAU’ was poorly defined and included large variation in the amount of oral intake that was encouraged/tolerated. This may have a significant impact on swallowing outcome, so should be documented/reported in research studies.
Second, assessing ‘swallowing’ by applying a non-standardised scale is not sufficient, as patients may be silently aspirating a bolus or experiencing pooling of a bolus in the oropharynx, during or after swallowing, that renders them at risk of post-swallow aspiration (which can be silent). Therefore, ‘oral swallowing’ by itself does not necessarily equate to a positive outcome. Only a video-fluoroscopic swallowing study (VFSS) – an X-ray technique used to document a person swallowing (liquid and solid) boluses of known size, consistency - enables instrumental assessment of swallowing risk and swallowing efficiency.
There is an urgent need for better designed interventions – a feasible, ‘easy to do’ exercise programme, based on a preventative, rather than a rehabilitative, dysphagia paradigm needs to be designed, then tested in a series of vanguard studies, and modified accordingly.
Patient involvement is crucial at the stage of designing such interventions, but in the papers we reviewed there were no examples of patient/participant opinion directing the design or the ‘dosage’ of intervention, which may explain the high dropout rate and lack of tolerance for swallowing exercise regimens which were evident across all studies.
Any swallowing exercise programme needs to be tolerated and accepted by HNC patients. The programme’s ‘dosage’ should also be underpinned by principles of exercise physiology (in terms of timing, amount, of exercise) such as those provided by the American College of Sports Medicine [18].
Primary outcomes should include determining the safety and the efficiency of swallowing. Both are assessed by use of a VFSS [4], the criterion standard methodology. When examining swallowing safety, a widely used, well validated scale, the penetration/aspiration scale [9] should beapplied to score each VFSS. This provides a quantifiable baseline score to compare with post-intervention ratings.
Efficiency of swallowing may be determined from a VFSS by first assessing the percentage of a bolus swallowed (subtracting the estimated percent of bolus residue and aspirate from a possible 100% bolus). This amount is then divided by the time taken to swallow the bolus, using a video timer to assess the ‘swallow’ in seconds – i.e., the time from initial tongue elevation to propel bolus backwards through the oral cavity to the time that the tail of that same bolus transfers through the cricopharyngeus.
The formula percentage of bolus swallowed, divided by time taken to swallow it determines an Oro-Pharyngeal Swallowing Efficiency (OPSE) score. People who swallow normally have an OPSE score of 100 (100% bolus swallowed, divided by a time of 1 second or less), whereas HNC patients with dysphagia may have an OPSE score of 39 or lower [14]. A measure of OPSE provides a quantifiable data point that is sensitive to change over time [14].
The use of a surrogate measure of swallowing efficiency, such as an estimate of post-swallow residue, is not advisable as there are no psychometrically sound measures of residue. There are also no normative measures for this domain, although we know from VFSSs that people without dysphagia also show post-swallow bolus residue to a variable degree. For these reasons, post-swallow residue alone is not a suitable surrogate measure of swallowing efficiency, as recognised by Lazarus et al. [14].
For Phase II studies, primary outcomes should include documenting adverse events, such as chest infections, aspiration pneumonia, and /or profound weight loss.
Secondary Outcomes should include the time to return to function (oral swallowing); self-reported changes to QoL; changes to psychological wellbeing (depression, anxiety, stress). Patient compliance is very important to document, as is patient satisfaction with the intervention and its cost-effectiveness.
In considering the secondary outcomes outlined above, there are valid and reliable tools available for assessing each of these variables with HNC patients. We suggest that it is preferable to have a selfreport of health-related QoL given by each patient, rather than by use of a clinician-rated tool, as only the person themselves can accurately describe this domain [19]. The Depression, Anxiety and Stress Scale (DASS) [20] is a well validated tool that enables comparison with age and gender matched norm data when assessing psychological wellbeing.
Choice of measurement tools is important and these should, where possible, be psychometrically sound – aka provide stable, valid and reliable measurements of a HNC population.
All published studies we examined were of weak methodological quality. Some deficits in study design could be easily addressed as follows;
A clinical trial methodologist/biostatistician needs to be part of the research team, as well as a health economist. This enables proactive thinking about the translation of study results, and clear planning of how to move from a Phase III trial into a later Phase.
Good Clinical Practice Guidelines (CPG) [21] provide standards for the conduct of trials with human participants. The Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guidelines [22], which provide an evidence-based set of items to address, should be used in the development of protocols, and the Consolidated Standards of Reporting Trials (CONSORT) [23] used for reporting. CONSORT offers a standard way for authors to report trial findings, facilitating complete and transparent reporting, reducing the influence of bias on results, and assisting with critical appraisal and interpretation.
Sample size calculations need to be provided, based on anticipated effects for the primary outcome, type I error level (α), power (1- β), and consideration of potential attrition rates. There is a need to recruit larger participant numbers (by use of cross-centre studies where possible) so that sub-stratification of patients can occur, to account for population heterogeneity and to reduce the probability of sample bias.
The randomisation schedule should be computerised, remotely generated and clearly described. There needs to be consideration of how the allocation sequence is concealed. There may also be the need to stratify samples according to factors such as gender, age, smoking and HPV status.
Acknowledging that performance bias is difficult to reduce, as patients and treating clinicians will be aware of treatment allocation, nevertheless selection bias can be reduced by having two statisticians or researchers, one of whom is aware of group allocation and one who is blinded to this (Good Clinical Practice Guideline) [23] and both undertake the analyses.
There needs to be agreed timelines and time points for assessment. Baseline (pre-HNC treatment) measures should be taken (to account for any pre-existing dysphagia) and then all outcome measures repeatedly taken immediately post-HNC treatment and again at 6 and 12 months post- treatment completion. Most HNC patients optimise their swallowing ability by 12 months post-treatment [24] so this would be an suitable time point for measurement.
Primary and secondary endpoints need to be clearly differentiated, with the primary endpoint being that for which subjects are randomised and the trial is powered. Secondary endpoints are those which are analysed post hoc, for which the trial may not be powered nor randomised.
High attrition rates and incomplete outcome data are the biggest problems across all of the published studies; these introduce bias and a reduction in methodological quality.
Attrition may be due to many causes, but some of the causes can be informative regarding intervention effects. For example, if there are different reasons for dropout in an intervention versus a control group, as well as differential rates of loss, then this can introduce bias into the analyses, so should be examined and reported.
A study does not have a low risk of bias simply be having the number of dropouts reported; the reasons for dropouts or ‘missingness’ can have a significant impact on outcomes and these, in turn, may result in over/under estimating a treatment effect. There may be different reasons for dropout in a treatment and a control group (the control group may have improved, for example), so these need to be accounted for and reported [25].
The issue of high attrition rates across all published studies needs attention, as it implies that either the regimen (‘dosage’) or/and the type of swallowing exercises are not feasible for HNC patients to undertake.
In examining the exercises, these were based on a paradigm of dysphagia rehabilitation, rather than dysphagia prevention. Exercises were chosen based on published studies of (often small numbers) HNC patients whose dysphagia was being managed by SLTs. However, the principle of rehabilitation (for which these exercises were designed) is different from prevention - and it is not clear that this distinction was accounted for.
If we are assisting patients to prevent post-treatment fibrosis and/or a stricture from developing, then principles of sports exercise programmes (which rely on building muscle speed, strength and/or flexibility) could be used to underpin the intervention (s).
Using different exercises to obtain different effects in muscles is well recognised – a sprinter will be exercising very differently from a marathon runner, for example. We need to adopt such specificity when designing swallowing exercises for HNC patients.
The tongue consists predominantly of Type II muscle fibres (which quickly fatigue) so it may be optimal to undertake exercises to enhance its range of motion and/or strength over short periods, but very frequently, throughout the day. Additionally, less onerous exercise regimes that can be incorporated into daily activities and be ‘done anywhere’ may be more palatable for patients, improving compliance. Patient involvement and advice needs to be sought ahead of designing such intervention(s).
In terms of standardising the exercises, further work is needed. For example, there was no evidence of researchers using a written manual, a CD, or mobile phone app to illustrate the exercises for patients to ‘check’ that their exercises were being undertaken in the way intended.
We need quality vanguard studies to ascertain the optimal intervention (type and dosage of exercises) as well as ascertaining ways of improving standardisation (delivery) of exercises in order to improve compliance and reduce dropout. These issues need to be addressed before any large randomised clinical trial is undertaken.
Reliance on self-report of compliance introduces a further bias into reporting. Nowadays, it is possible to use technology (such as a mobile phone app) to more accurately document practise.


Conclusions

There needs to be more rigorous work undertaken on preventative interventions to provide clear evidence for the benefit of swallowing exercises in a series of carefully designed vanguard studies. Until the methodological issues described in this paper are addressed, and we have more definitive studies published in this area, swallowing dysfunction (dysphagia) after HNC treatment will continue to be ‘… one of the most important and clinically relevant side effects of curative radiotherapy or chemo-radiotherapy[26]. HNC patients deserve better...


Acknowledgements

This paper is based on our recently published Cochrane review and we acknowledge the assistance of the UK Cochrane ENT review group and of our two co-authors on that review, Professor Catriona Kennedy and Dr Siew Lee, both at Robert Gordon University, Aberdeen.
Ethical Committee approval was not necessary for this study, which was based on our Cochrane Systematic Review (2016) [8].
We received no specific grant from any funding agency, commercial or not-for-profit sectors, for this work.


References

  1. Marur S, D’Souza G, Westra WH, Forestiere AA. HPV-associated head and neck cancer: a virus-related cancer epidemic. Lancet Oncology. 2010; 11(8): 781-789.
  2. Jemal A, Bray F, Center MM, Ferlay J, Ward E, Forman D. Global cancer statistics. CA Cancer J Clin. 2011; 61(2): 69-90.
  3. Pignon JP, le Maitre A, Maillard E, Bourhis J. MACH-NC Collaborative Group. Meta-analysis of chemotherapy in head and neck cancer (MACH-NC) an update on 93 randomised trials and 17,346 patients. Radiotherapy and Oncology. 2009; 92: 4-14.
  4. Logemann JA. Behavioural management for oropharyngeal dysphagia. Folia Phoniatrica et Logopaedia 1999; 51(4-5): 199-212.
  5. Clark HM. Neuromuscular treatments for speech and swallowing: a tutorial. Am J Speech Lang Pathol. 2003; 12(4): 400-415.
  6. Appleton J, Machin J. Working with Oral Cancer. Oxon, UK. Winslow Press 1994.
  7. Fujio M, Logemann JA. Effects if tongue holding manoeuvre on posterior pharyngeal wall movement during deglutition. American Journal of Speech-Language Pathology. 1996; 5: 23-30.
  8. Perry A, Lee SH, Cotton S, Kennedy C. Therapeutic exercises for affecting post-treatment dysphagia in people treated for advanced stage head and neck cancer. Cochrane Database Syst Rev. 2016; (8): CD011112.
  9. Rosenbek JC, Robbins JA, Roecker EB, Coyle JL, Wood JL. A penetration-aspiration scale. Dysphagia 1996; 11(2): 93-98.
  10. Lazarus CL, Logemann JA, Pauloski BR, Colangelo LA, Kahrilas PJ, Mittal BB, et al. Swallowing disorders in head and neck cancer patients treated with radiotherapy and adjuvant chemotherapy. Laryngoscope.1996; 106: 1157-1166.
  11. Dijkers M. Introducing GRADE: a systematic approach to rating evidence in systematic reviews and to guideline development Presented in KT Update(Vol. 1, No. 5 -August 2013) An e-newsletter from: the Center on Knowledge Translation for Disability and Rehabilitation Research.
  12. Carnaby-Mann G, Crary MA, Schmalfuss I, Amdur R. “Pharyngocise”: randomized controlled trial of preventative exercises to maintain muscle structure and swallowing function during head-and-neck chemoradiotherapy. Int J Radiat Oncol Biol Phys. 2012; 83(1): 210-209.
  13. Kotz T, Federman AD, Kao J, Milman L, Packer S, Lopez-Prieto C, et al. Prophylactic swallowing exercises in patients with head and neck cancer undergoing chemoradiation: a randomized trial. Arch Otolaryngol Head Neck Surg. 2012; 138(4): 376-382.
  14. Lazarus CL, Husainib H, Falcigliac D, DeLacurec M, Branskid RC, Krause D, et al. Effects of exercise on swallowing and tongue strength in patients with oral and oropharyngeal cancer treated with primary radiotherapy with or without chemotherapy. Int J Oral Maxillofac Surg. 2014; 43(5): 523-530.
  15. Mortensen HR, Jensen J, Aksglaede K, Lambertsen K, Eriksen E, Grau C. Prophylactic swallowing exercises in head and neck cancer radiotherapy. Dysphagia. 2015; 30: 304-314.
  16. Van den Berg MGA, Kalf JG, Hendriks JCM, Takes RP, van Herpen CML, Wanten GJA, et al. Normalcy of food intake in patients with head and neck cancer supported by combined dietary counselling and swallowing therapy: a randomized clinical trial. Head Neck. 2016; 38 Suppl 1: E198-E206.
  17. van der Molen L, van Rossum MA, Burkhead LM, Smeele LE, Rasch CRN, Hilgers FJM. A randomized preventative rehabilitation trial in advanced HN cancer patients treated with chemoradiotherapy: feasibility, compliance and short-term effect. Dysphagia.2011; 26: 155-170.
  18. American College of Sports Medicine Position Stand. The recommended quantity and quality of exercise for developing and maintaining cardiorespiratory and muscular fitness, and flexibility in healthy adults. Med Sci Sports Exerc. 1998; 30(6): 975-991.
  19. Murphy B, Ridner S, Wells N, Dietrich M. Quality of life research in head and neck cancer: A review of the current state of the science. Critical Reviews in Oncology/Hematology. 2007; 62(3): 251-267.
  20. Lovibond SH, Lovibond PF. Manual for the Depression Anxiety Stress Scales. Psychology Foundation, Sydney; 1995.
  21. Guideline for Good Clinical Practice; ICH Topic E 6 (R1).
  22. Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT).
  23. Moher D,  Schulz KF,  Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. The Lancet. 2001; 357(9263): 1191–1194.
  24. Perry A, Shaw M. The evaluation of functional outcomes (speech and swallowing) after head and neck cancer treatment; results and analysis. J Laryngol Otol. 2003; 117(5): 368-381.
  25. Portney LG and Watkins MP. Foundations of clinical research: applications to practice. New Jersey: Prentice Hall; 2000. 2nd ed.
  26. Langendijk JA. New development in radiotherapy of head and neck cancer: higher precision with less patient discomfort. Radiotherapy and Oncology. 2007; 85: 4-5.