Short Communication
Swallowing Exercises to Prevent Dysphagia Post-Treatment in People Treated for Advanced Stage Head and Neck Cancer; Addressing the Research Shortfalls
Alison R. Perry1* and Susan M. Cotton2,3
1Department of Clinical Therapies, Faculty of Education and Health Sciences, The University of Limerick, LIMERICK, Ireland
2Orygen, The National Centre of Excellence in Youth Mental Health, 35 Poplar Road, Parkville, Melbourne, VIC 3052, Australia
3The Centre for Youth Mental Health, University of Melbourne, 35 Poplar Road, Parkville, Melbourne, VIC 3052, Australia
*Corresponding author: Alison Perry, Faculty of Education & Health Sciences, The University of Limerick, LIMERICK, Ireland
Published: 08 Aug, 2017
Cite this article as: Perry AR, Cotton SM. Swallowing
Exercises to Prevent Dysphagia
Post-Treatment in People Treated
for Advanced Stage Head and Neck
Cancer; Addressing the Research
Shortfalls. Clin Oncol. 2017; 2: 1324.
Abstract
Background: Advances in treating advanced Head and Neck Squamous Cell Cancer (HNSCC)
have meant improved mortality and survival rates over the last decade, but significant morbidity
(specifically, dysphagia) often persists. Therapeutic swallowing exercises are often implemented,
without the evidence for such intervention being established.
Methods: This is a critique of the studies sourced for our Cochrane systematic review, which was
completed to determine the benefit of therapeutic swallowing exercises undertaken before, during,
and/or immediately after treatment, in people treated with curative intent for advanced stage
HNSCC.
Findings: We found no evidence that therapeutic exercises were better than treatment as usual, or
any other treatment, in improving the safety and efficient of oral swallowing (primary outcome).
Shortfalls in the published study designs included; a lack of agreement about primary outcomes,
about suitable tools used to measure them, and the choice of baselines and endpoints across studies.
In this paper, we make suggestions of ways to improve the methodological quality of future studies.
Keywords: Cancer of the head and neck; Squamous cell carcinoma; Deglutition disorders;
Oropharyngeal dysphagia; Swallowing rehabilitation exercises
Introduction
Worldwide, Head and Neck Cancer (HNC) accounts for more than 550,000 cases annually
recorded. Over 80% of these cancers originate in the squamous cells that line the mucosal surfaces of
the mouth, nose and throat. In the past decade, there has been a rise in the incidence of oropharyngeal
squamous cell cancer or SCC (specifically of the lingual and palatine tonsils) associated with Human
Papilloma Virus (HPV) occurring in white men, younger than 50 years, who often have no history
of alcohol abuse or smoking [1].
Survival rates of HNSCC have improved slightly in the last decade, but survival remains poor as
people (mainly men) often present late to treatment. Five year relative survival across Europe ranges
from 20% in Slovakia to 51% in the Netherlands [2]. Despite greater chances of survival from more
aggressive treatments, regrettably morbidity has increased alongside the improved survival rates.
Treatments for advanced HNC include surgery and radiotherapy, surgery alone, or chemoradiotherapy
without surgery. The benefits of concurrent chemotherapy and radiation therapy on
survival were confirmed by a meta-analysis of results from 84 trials by Pignon et al. [3]. Currently,
goals of treatment are to improve survival outcomes, but also to maintain organ preservation.
Unfortunately, preserving an organ does not always result in preservation of its function, so
functional morbidity is common during and after treatment for advanced (T3,T4) HNSCC.
Morbidity includes dysphagia (swallowing disability) which may be due to the cancer itself or
may occur during, or after, aggressive HNC treatment. Dysphagia presents differently at each stage
of the care pathway.
Pre-treatment dysphagia may be due to a (large) tumour
compressing oral/ pharyngeal anatomical structures or because of
comorbid cranial nerve damage (e.g., of IX, X, XI and/or XII cranial
nerves). Dysphagia is not always signified by clinical signs (such as
coughing, choking, or reports of food ‘sticking’ when swallowing) as
people may adapt to a gradual onset of dysphagia and not be aware of
its severity/consequences.
Peri-treatment dysphagia is often acute and is related to
radiotherapy treatment side-effects, such as xerostomia, mucositis,
or candida. Although often transitory, such painful side-effects may
compromise nutritional intake and then alternative feeding (via
a nasogastric or a percutaneous endoscopic gastrostomy - PEG -
tube) may be implemented. If this continues for an extended period,
patients may have difficulty in re-commencing oral feeding, so a
structured dysphagia management programme, in consultation with
a dietician and a Speech and Language Therapist (SLT), will need to
be implemented to reassure the patient and provide a safe, staged
return to oral feeding.
Post-treatment dysphagia occurs due to tissue fibrosis (postradiotherapy)
and/or a stricture formation. To prevent such negative
sequelae from treatment, many SLTs have adopted a philosophy of,
‘use it or lose it’ and insist that patients continue oral swallowing
(albeit with modification to the texture and/ or volume of foods,
fluids, consumed) throughout (often painful) cancer treatment.
Additionally, some clinicians promote ‘swallowing exercises’ for
patients throughout radiotherapy treatment in an endeavour to
prevent/reduce post-treatment dysphagia.
To reduce the survivorship burden for patients and families, as
well as costs to healthcare systems, it is essential to prevent/reduce
treatment-related dysphagia in people treated for advanced HNC.
However, work to date has been focussed on designing suitable
rehabilitation (rather than preventative) programmes with adults
who have acquired dysphagia.
In 1999, Logemann [4] introduced and developed the concept
of therapeutic swallowing exercises. These are direct techniques,
implemented to either improve swallowing safety (i.e., reduce/
eliminate aspiration of food, fluids, swallowed) or to improve
swallowing efficiency (by increasing the speed and the amount of a
bolus swallowed). Such exercises change swallowing physiology, either
by improving sensory-motor integration or by gaining voluntary
control over the timing or co-ordination of selected oropharyngeal
movements during swallowing.
Swallowing therapy may also involve neuromuscular exercises [4]
to increase tongue strength, endurance, power, and range of motion.
Strength is achieved by exercises using high levels of resistance
(isometric exercises). Endurance is achieved by repeated performance
of low resistance exercises, and power is achieved by using exercises
that focus on the speed of muscle contraction [5]. Exercises to
maintain or increase a tongue’s range of motion may be used to keep
the tongue flexible during treatment [6] and tongue base exercises are
used to mitigate against the stiffening or fibrosis that can result from
radiotherapy and/or from surgery [7].
In our recently published Cochrane systematic review [8] we
examined the effect of undertaking such therapeutic swallowing
exercises before, during, or immediately after HNC treatment, on safe
oral swallowing, on aspiration and on adverse events, such as chest
infections, aspiration pneumonia and profound weight loss, in people
treated with curative intent for advanced stage (T3,T4) HNSCC.
The purpose of this paper is to elaborate on the findings from
that review and to highlight the methodological deficiencies in
published studies. We propose improvements to study designs and
measurements that will enhance the rigour of future studies of people
with dysphagia associated with HNC.
Materials and Methods
A standard Cochrane review methodology was applied to
examine the effect of therapeutic exercises for affecting post-treatment
swallowing in people treated for advanced stage HNC [8]. This paper
addresses the shortcomings of the studies reviewed, specifically
focussing on issues of weak methodology and of bias.
Only Randomised Controlled Trials (RCTs) were included. We
included studies involving adults with a clinical and histological
diagnosis of large (T3,T4) HNC who received any type of cancer
treatment (surgery and radiotherapy, surgery and chemoradiotherapy,
or chemo-radiotherapy without surgery) who were
deemed at risk of, or presented with, dysphagia. We included all levels
of dysphagia and set no age limits.
We examined direct therapeutic techniques involving swallowing
and neuromuscular exercises. Therapy programmes had to last for
more than one session, could have been provided by one or more
disciplines, might be delivered pre-, peri- or immediately post-HNC
treatment, and were designed to either reduce or prevent dysphagia
from occurring.
The main comparison was therapeutic exercises versus Treatment
As Usual (TAU). Other comparisons included therapeutic exercises
versus sham exercises; sham exercises versus TAU; and therapeutic
exercises plus TAU versus TAU.
Primary outcomes were safety and efficiency of oral swallowing,
respectively measured by reduced/no aspiration, assessed using a
standardised scale [9] and Oropharyngeal Swallowing Efficiency
(OPSE) which is a measure of the amount of a bolus (usually a
measured 3 ml or 5 ml spoonful of food or fluid) swallowed, divided
by the time taken to swallow it [10]. Both measures are taken from
Video-Fluoroscopic Swallowing Studies (VFSSs).
Secondary outcomes included self-reported changes to healthrelated
or swallowing-related Quality Of Life (QoL); changes
to psychological wellbeing (stress, anxiety, depression); patient
satisfaction with intervention; patient compliance with intervention;
and cost-effectiveness of the intervention.
We used the Grades of Recommendation, Assessment,
Development and Evaluation (GRADE) approach [11] to rate the
overall quality of evidence. This reflects the extent to which we are
confident that an estimate of effect is correct and we applied this to
the interpretation of results. There are four possible ratings; high,
moderate, low and very low.
Results and Discussion
We retrieved a total of 627 records through database searching
which, when 184 duplicated were removed, became 443 papers.
Screening out non-relevant references left 25 papers for further
consideration. After closer scrutiny, 11 were discarded and 4 studies
were still recruiting participants. Two researchers then independently
assessed the full texts of the 10 remaining papers; three were excluded
as they were not RCTs and one was excluded because the interventions
were intended to be rehabilitative rather than preventative. After
consultation with a third researcher, we included six studies [12-17]
that met our inclusion criteria for the Cochrane review [8].
The six studies had a total of 326 participants who undertook
swallowing exercises before, during, and/or immediately after HNC
treatment. We could not combine the results of the studies because
of the variation in participants’ cancers, their treatments, outcomes
measured and tools used to assess them, as well as differing time
points chosen for testing.
Researchers compared therapeutic exercises versus Treatment As
Usual (TAU); therapeutic exercises versus sham therapy; therapeutic
exercises plus TAU versus TAU. The exercises varied considerably in
their design, timing and intensity. TAU involved managing patients’
dysphagia when it occurred and, when severe, this might involve
inserting either a nasogastric (NG) or a Percutaneous Endoscopic
Gastroscopy (PEG) for non-oral feeding.
There is presently insufficient evidence to draw clear conclusions
about the effects of undertaking therapeutic swallowing exercises
before, during, or immediately after HNC treatment to prevent/
reduce dysphagia. Studies had low participant numbers, used
complex (often poorly tolerated) interventions, and varied in the
choice of primary and secondary outcomes, making it difficult to
draw reliable conclusions.
Although there were no reported adverse events that could
be directly attributed to the therapeutic intervention (swallowing
exercises), the high attrition rates across all studies indicates
participants’ difficulty in tolerating and accepting the exercise
regimens.
We suggest that addressing the following issues would enhance
the design of future studies, and hence lead to improved study
outcomes.
There is a need to more clearly operationalise terms. First, ‘TAU’
was poorly defined and included large variation in the amount of oral
intake that was encouraged/tolerated. This may have a significant
impact on swallowing outcome, so should be documented/reported
in research studies.
Second, assessing ‘swallowing’ by applying a non-standardised
scale is not sufficient, as patients may be silently aspirating a bolus
or experiencing pooling of a bolus in the oropharynx, during or after
swallowing, that renders them at risk of post-swallow aspiration
(which can be silent). Therefore, ‘oral swallowing’ by itself does not
necessarily equate to a positive outcome. Only a video-fluoroscopic
swallowing study (VFSS) – an X-ray technique used to document
a person swallowing (liquid and solid) boluses of known size,
consistency - enables instrumental assessment of swallowing risk and
swallowing efficiency.
There is an urgent need for better designed interventions – a
feasible, ‘easy to do’ exercise programme, based on a preventative,
rather than a rehabilitative, dysphagia paradigm needs to be designed,
then tested in a series of vanguard studies, and modified accordingly.
Patient involvement is crucial at the stage of designing such
interventions, but in the papers we reviewed there were no examples
of patient/participant opinion directing the design or the ‘dosage’ of
intervention, which may explain the high dropout rate and lack of
tolerance for swallowing exercise regimens which were evident across
all studies.
Any swallowing exercise programme needs to be tolerated and
accepted by HNC patients. The programme’s ‘dosage’ should also be
underpinned by principles of exercise physiology (in terms of timing,
amount, of exercise) such as those provided by the American College
of Sports Medicine [18].
Primary outcomes should include determining the safety and the
efficiency of swallowing. Both are assessed by use of a VFSS [4], the
criterion standard methodology. When examining swallowing safety,
a widely used, well validated scale, the penetration/aspiration scale
[9] should beapplied to score each VFSS. This provides a quantifiable
baseline score to compare with post-intervention ratings.
Efficiency of swallowing may be determined from a VFSS by
first assessing the percentage of a bolus swallowed (subtracting the
estimated percent of bolus residue and aspirate from a possible 100%
bolus). This amount is then divided by the time taken to swallow the
bolus, using a video timer to assess the ‘swallow’ in seconds – i.e., the
time from initial tongue elevation to propel bolus backwards through
the oral cavity to the time that the tail of that same bolus transfers
through the cricopharyngeus.
The formula percentage of bolus swallowed, divided by time taken
to swallow it determines an Oro-Pharyngeal Swallowing Efficiency
(OPSE) score. People who swallow normally have an OPSE score of
100 (100% bolus swallowed, divided by a time of 1 second or less),
whereas HNC patients with dysphagia may have an OPSE score of 39
or lower [14]. A measure of OPSE provides a quantifiable data point
that is sensitive to change over time [14].
The use of a surrogate measure of swallowing efficiency, such
as an estimate of post-swallow residue, is not advisable as there
are no psychometrically sound measures of residue. There are also
no normative measures for this domain, although we know from
VFSSs that people without dysphagia also show post-swallow bolus
residue to a variable degree. For these reasons, post-swallow residue
alone is not a suitable surrogate measure of swallowing efficiency, as
recognised by Lazarus et al. [14].
For Phase II studies, primary outcomes should include
documenting adverse events, such as chest infections, aspiration
pneumonia, and /or profound weight loss.
Secondary Outcomes should include the time to return to
function (oral swallowing); self-reported changes to QoL; changes
to psychological wellbeing (depression, anxiety, stress). Patient
compliance is very important to document, as is patient satisfaction
with the intervention and its cost-effectiveness.
In considering the secondary outcomes outlined above, there are
valid and reliable tools available for assessing each of these variables
with HNC patients. We suggest that it is preferable to have a selfreport
of health-related QoL given by each patient, rather than by use
of a clinician-rated tool, as only the person themselves can accurately
describe this domain [19]. The Depression, Anxiety and Stress Scale
(DASS) [20] is a well validated tool that enables comparison with
age and gender matched norm data when assessing psychological
wellbeing.
Choice of measurement tools is important and these should,
where possible, be psychometrically sound – aka provide stable, valid
and reliable measurements of a HNC population.
All published studies we examined were of weak methodological
quality. Some deficits in study design could be easily addressed as
follows;
A clinical trial methodologist/biostatistician needs to be part
of the research team, as well as a health economist. This enables
proactive thinking about the translation of study results, and clear
planning of how to move from a Phase III trial into a later Phase.
Good Clinical Practice Guidelines (CPG) [21] provide standards
for the conduct of trials with human participants. The Standard
Protocol Items: Recommendations for Interventional Trials (SPIRIT)
guidelines [22], which provide an evidence-based set of items to
address, should be used in the development of protocols, and the
Consolidated Standards of Reporting Trials (CONSORT) [23]
used for reporting. CONSORT offers a standard way for authors to
report trial findings, facilitating complete and transparent reporting,
reducing the influence of bias on results, and assisting with critical
appraisal and interpretation.
Sample size calculations need to be provided, based on anticipated
effects for the primary outcome, type I error level (α), power (1-
β), and consideration of potential attrition rates. There is a need to
recruit larger participant numbers (by use of cross-centre studies
where possible) so that sub-stratification of patients can occur, to
account for population heterogeneity and to reduce the probability
of sample bias.
The randomisation schedule should be computerised, remotely
generated and clearly described. There needs to be consideration of
how the allocation sequence is concealed. There may also be the need
to stratify samples according to factors such as gender, age, smoking
and HPV status.
Acknowledging that performance bias is difficult to reduce, as
patients and treating clinicians will be aware of treatment allocation,
nevertheless selection bias can be reduced by having two statisticians
or researchers, one of whom is aware of group allocation and one who
is blinded to this (Good Clinical Practice Guideline) [23] and both
undertake the analyses.
There needs to be agreed timelines and time points for assessment.
Baseline (pre-HNC treatment) measures should be taken (to account
for any pre-existing dysphagia) and then all outcome measures
repeatedly taken immediately post-HNC treatment and again at 6
and 12 months post- treatment completion. Most HNC patients
optimise their swallowing ability by 12 months post-treatment [24]
so this would be an suitable time point for measurement.
Primary and secondary endpoints need to be clearly differentiated,
with the primary endpoint being that for which subjects are
randomised and the trial is powered. Secondary endpoints are those
which are analysed post hoc, for which the trial may not be powered
nor randomised.
High attrition rates and incomplete outcome data are the biggest
problems across all of the published studies; these introduce bias and
a reduction in methodological quality.
Attrition may be due to many causes, but some of the causes can
be informative regarding intervention effects. For example, if there
are different reasons for dropout in an intervention versus a control
group, as well as differential rates of loss, then this can introduce bias
into the analyses, so should be examined and reported.
A study does not have a low risk of bias simply be having
the number of dropouts reported; the reasons for dropouts or
‘missingness’ can have a significant impact on outcomes and these,
in turn, may result in over/under estimating a treatment effect. There
may be different reasons for dropout in a treatment and a control
group (the control group may have improved, for example), so these
need to be accounted for and reported [25].
The issue of high attrition rates across all published studies needs
attention, as it implies that either the regimen (‘dosage’) or/and the
type of swallowing exercises are not feasible for HNC patients to
undertake.
In examining the exercises, these were based on a paradigm of
dysphagia rehabilitation, rather than dysphagia prevention. Exercises
were chosen based on published studies of (often small numbers)
HNC patients whose dysphagia was being managed by SLTs.
However, the principle of rehabilitation (for which these exercises
were designed) is different from prevention - and it is not clear that
this distinction was accounted for.
If we are assisting patients to prevent post-treatment fibrosis
and/or a stricture from developing, then principles of sports exercise
programmes (which rely on building muscle speed, strength and/or
flexibility) could be used to underpin the intervention (s).
Using different exercises to obtain different effects in muscles is
well recognised – a sprinter will be exercising very differently from
a marathon runner, for example. We need to adopt such specificity
when designing swallowing exercises for HNC patients.
The tongue consists predominantly of Type II muscle fibres
(which quickly fatigue) so it may be optimal to undertake exercises
to enhance its range of motion and/or strength over short periods,
but very frequently, throughout the day. Additionally, less onerous
exercise regimes that can be incorporated into daily activities and
be ‘done anywhere’ may be more palatable for patients, improving
compliance. Patient involvement and advice needs to be sought ahead
of designing such intervention(s).
In terms of standardising the exercises, further work is needed.
For example, there was no evidence of researchers using a written
manual, a CD, or mobile phone app to illustrate the exercises for
patients to ‘check’ that their exercises were being undertaken in the
way intended.
We need quality vanguard studies to ascertain the optimal
intervention (type and dosage of exercises) as well as ascertaining
ways of improving standardisation (delivery) of exercises in order
to improve compliance and reduce dropout. These issues need to be
addressed before any large randomised clinical trial is undertaken.
Reliance on self-report of compliance introduces a further bias
into reporting. Nowadays, it is possible to use technology (such as a
mobile phone app) to more accurately document practise.
Conclusions
There needs to be more rigorous work undertaken on preventative interventions to provide clear evidence for the benefit of swallowing exercises in a series of carefully designed vanguard studies. Until the methodological issues described in this paper are addressed, and we have more definitive studies published in this area, swallowing dysfunction (dysphagia) after HNC treatment will continue to be ‘… one of the most important and clinically relevant side effects of curative radiotherapy or chemo-radiotherapy’ [26]. HNC patients deserve better...
Acknowledgements
This paper is based on our recently published Cochrane review
and we acknowledge the assistance of the UK Cochrane ENT review
group and of our two co-authors on that review, Professor Catriona
Kennedy and Dr Siew Lee, both at Robert Gordon University,
Aberdeen.
Ethical Committee approval was not necessary for this study,
which was based on our Cochrane Systematic Review (2016) [8].
We received no specific grant from any funding agency,
commercial or not-for-profit sectors, for this work.
References
- Marur S, D’Souza G, Westra WH, Forestiere AA. HPV-associated head and neck cancer: a virus-related cancer epidemic. Lancet Oncology. 2010; 11(8): 781-789.
- Jemal A, Bray F, Center MM, Ferlay J, Ward E, Forman D. Global cancer statistics. CA Cancer J Clin. 2011; 61(2): 69-90.
- Pignon JP, le Maitre A, Maillard E, Bourhis J. MACH-NC Collaborative Group. Meta-analysis of chemotherapy in head and neck cancer (MACH-NC) an update on 93 randomised trials and 17,346 patients. Radiotherapy and Oncology. 2009; 92: 4-14.
- Logemann JA. Behavioural management for oropharyngeal dysphagia. Folia Phoniatrica et Logopaedia 1999; 51(4-5): 199-212.
- Clark HM. Neuromuscular treatments for speech and swallowing: a tutorial. Am J Speech Lang Pathol. 2003; 12(4): 400-415.
- Appleton J, Machin J. Working with Oral Cancer. Oxon, UK. Winslow Press 1994.
- Fujio M, Logemann JA. Effects if tongue holding manoeuvre on posterior pharyngeal wall movement during deglutition. American Journal of Speech-Language Pathology. 1996; 5: 23-30.
- Perry A, Lee SH, Cotton S, Kennedy C. Therapeutic exercises for affecting post-treatment dysphagia in people treated for advanced stage head and neck cancer. Cochrane Database Syst Rev. 2016; (8): CD011112.
- Rosenbek JC, Robbins JA, Roecker EB, Coyle JL, Wood JL. A penetration-aspiration scale. Dysphagia 1996; 11(2): 93-98.
- Lazarus CL, Logemann JA, Pauloski BR, Colangelo LA, Kahrilas PJ, Mittal BB, et al. Swallowing disorders in head and neck cancer patients treated with radiotherapy and adjuvant chemotherapy. Laryngoscope.1996; 106: 1157-1166.
- Dijkers M. Introducing GRADE: a systematic approach to rating evidence in systematic reviews and to guideline development Presented in KT Update(Vol. 1, No. 5 -August 2013) An e-newsletter from: the Center on Knowledge Translation for Disability and Rehabilitation Research.
- Carnaby-Mann G, Crary MA, Schmalfuss I, Amdur R. “Pharyngocise”: randomized controlled trial of preventative exercises to maintain muscle structure and swallowing function during head-and-neck chemoradiotherapy. Int J Radiat Oncol Biol Phys. 2012; 83(1): 210-209.
- Kotz T, Federman AD, Kao J, Milman L, Packer S, Lopez-Prieto C, et al. Prophylactic swallowing exercises in patients with head and neck cancer undergoing chemoradiation: a randomized trial. Arch Otolaryngol Head Neck Surg. 2012; 138(4): 376-382.
- Lazarus CL, Husainib H, Falcigliac D, DeLacurec M, Branskid RC, Krause D, et al. Effects of exercise on swallowing and tongue strength in patients with oral and oropharyngeal cancer treated with primary radiotherapy with or without chemotherapy. Int J Oral Maxillofac Surg. 2014; 43(5): 523-530.
- Mortensen HR, Jensen J, Aksglaede K, Lambertsen K, Eriksen E, Grau C. Prophylactic swallowing exercises in head and neck cancer radiotherapy. Dysphagia. 2015; 30: 304-314.
- Van den Berg MGA, Kalf JG, Hendriks JCM, Takes RP, van Herpen CML, Wanten GJA, et al. Normalcy of food intake in patients with head and neck cancer supported by combined dietary counselling and swallowing therapy: a randomized clinical trial. Head Neck. 2016; 38 Suppl 1: E198-E206.
- van der Molen L, van Rossum MA, Burkhead LM, Smeele LE, Rasch CRN, Hilgers FJM. A randomized preventative rehabilitation trial in advanced HN cancer patients treated with chemoradiotherapy: feasibility, compliance and short-term effect. Dysphagia.2011; 26: 155-170.
- American College of Sports Medicine Position Stand. The recommended quantity and quality of exercise for developing and maintaining cardiorespiratory and muscular fitness, and flexibility in healthy adults. Med Sci Sports Exerc. 1998; 30(6): 975-991.
- Murphy B, Ridner S, Wells N, Dietrich M. Quality of life research in head and neck cancer: A review of the current state of the science. Critical Reviews in Oncology/Hematology. 2007; 62(3): 251-267.
- Lovibond SH, Lovibond PF. Manual for the Depression Anxiety Stress Scales. Psychology Foundation, Sydney; 1995.
- Guideline for Good Clinical Practice; ICH Topic E 6 (R1).
- Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT).
- Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. The Lancet. 2001; 357(9263): 1191–1194.
- Perry A, Shaw M. The evaluation of functional outcomes (speech and swallowing) after head and neck cancer treatment; results and analysis. J Laryngol Otol. 2003; 117(5): 368-381.
- Portney LG and Watkins MP. Foundations of clinical research: applications to practice. New Jersey: Prentice Hall; 2000. 2nd ed.
- Langendijk JA. New development in radiotherapy of head and neck cancer: higher precision with less patient discomfort. Radiotherapy and Oncology. 2007; 85: 4-5.