When to use a survey in pharmacy practice research
Understanding when surveys can be a useful research tool and considerations for survey-based research.
Surveys in the context of pharmacy practice can be best described as an act of investigating the experiences, opinions, or behaviours of a group of patients or colleagues by asking them questions. While “knowledge, attitude and practices” surveys (the KAP model) have been used extensively in social sciences for decades, surveys have also become a commonly used tool in healthcare to gather relevant information from patients as well as practitioners. A Medline search using the term “survey research in healthcare” for the period between December 2005 and December 2015 returned 45,667 entries.
The purpose of a survey
Survey research in pharmacy practice may be used for descriptive use only, for example, collecting data to describe the demographic characteristics and risk factors of patients receiving influenza vaccination in community pharmacies. It can also be used for investigating correlational or differential hypotheses — for example, Urbonas and Kubiliene used surveys to test whether a correlational relationship existed between job satisfaction and over-the-counter counselling practice at community pharmacies, and O’Brien et al. conducted a retrospective analysis of surveys to investigate possible differences in opinions regarding knowledge and skills needed for pharmacy practice between licensed pharmacists and new graduates or graduating senior pharmacy students.
Survey research can even be used to establish cause-effect relationships, for example, Gupta et al. assessed the impact of a pharmacist-led, web-based video presentation in increasing patients’ awareness of the importance of medication adherence.
However, while it is sufficient to collect survey data for descriptive, correlational and differential purposes, establishing a cause-effect relationship requires an experimental design — the presence of an active intervention by the pharmacist, and a control group with random assignment. Furthermore, while surveys can be used for collecting data both retrospectively and prospectively, experimental studies by nature are limited to prospective data collection.
Although retrospective surveys are helpful for collecting data from the past and covering long periods, the quality of data is reliant upon recall memory, which is prone to bias for events with intense experiences.
Prospective surveys can be cross-sectional or longitudinal — cross-sectional surveys collect data once at a specific point in time, while longitudinal surveys collect data multiple times over a specified period which can extend from months to years, or even decades. A cross-sectional survey allows the researcher to determine correlations or differences at a given time, while a longitudinal survey may reveal trends and provide hints on probable cause-effect relationships. For example, Papastergiou et al. collected cross-sectional survey data once from each participant at four different community pharmacy locations during a period of eight weeks, while Williemsen et al. reported a 25-year survey study, where each subject completed two or more surveys, which collected information on parameters such as physical and mental health, lifestyle, and personality.
Time and cost are the biggest disadvantages of a longitudinal survey design and, unless it is completed over a short period, this design may not be practical for a community pharmacy. Regarding retrospective versus prospective surveys, while the latter offer the advantage of being not subject to recall bias, they may face the threat of response bias and therefore care must be taken to minimise this possibility (see ‘Carrying out a survey’).
Designing, developing and validating a survey
While pharmacists may use surveys for a wide range of purposes, it is important to note that all survey research, like any other research, must meet the criteria for scientific “validity” and “reliability”. In the context of research, the concepts of validity and reliability, while related, are distinct from each other. Validity reflects the degree to which the study succeeds in accurately measuring what it set out to measure (internal validity), and to the degree the findings of the study can be applied to real world (external validity or generalisability). Reliability refers to the degree of accuracy with which a specific variable can be measured repeatedly.
To assure quality, pharmacists must know the guidelines and recommendations regarding best practices for survey research and be mindful of the rigour that must be applied in the design, conduct and reporting of survey research so that the findings credibly reflect the target population and are a true contribution to the scientific literature. The format of the survey, the number of questions and the scales of measurement should all be considered to ensure the survey is both valid and reliable.
When designing questions, it is important to be aware of the cognitive aspects involved in answering survey questions, such as making sense of the question and the role of memory and guessing in answering. It is highly recommended that pharmacists make use of the online self-guided survey development tools such as Survey Monkey, and Survey Planet.
Regarding the length of a survey, a trade off exists between the extent of information sought and the response rate. A short survey may increase participation but may not yield the required information. A longer survey covering more information may decrease the response rate below acceptable levels (see ‘Maximising response rates’).
Finally, ethical issues of confidentiality and anonymity must be considered in survey design and development.
Sampling frame and size
Potential survey participants are selected from the ‘sampling frame’, which is defined by a careful statement of inclusion and exclusion criteria. This task must be done in a manner that gives the highest probability of yielding the sample truly representative of the target population along important variables. For example, if the pharmacist wishes to target subjects with a particular characteristic such as being elderly, or being diabetic, then the sampling frame must focus on those areas where the probability of elderly or diabetic subjects is high.
The next step is to determine the sample size with sufficient statistical power. If the sample is too small, there may not be enough power to observe the desired phenomenon, but a sample larger than required is a waste of valuable resources. Study design, effect size, type of data, variance (a measurement of how far each number in a data set is from the mean) the desired confidence level (the percentage of all possible samples that can be expected to include the true population value), the power (the likelihood that a study will detect an effect where an effect exists) and the margin of error all need to be taken into account when estimating the optimal sample size. The optimal sample size is directly proportional to the confidence level, power, and variance, and inversely proportional to the margin of error.
In general, the sample size should yield at least 95% confidence in results with 5% margin of error for a type I error (i.e. false positive), and with at least 80% power to detect a type II error (i.e. false negative). A number of online tools are available where pharmacists can compute the optimal sample size by completing desired values for these factors. For example, Dupont and Plummer Jr. have provided a free interactive programme that can be used for measurements on all scales common to surveys. The programme can determine the sample size needed to detect a specified effect with the required power, the power with which a specific effect can be detected with a given sample size, or the specific effect that can be detected with a given power and sample size.
Finally, if under-representation of specific populations is expected, such as from pediatric or geriatric patients, data should be collected with alternative methods. For example, patients aged over 75 years have been shown to visit the pharmacy to get their prescriptions filled, and this opportunity for personal interaction can be used to recruit them.
Carrying out a survey
While pharmacists may administer the survey on site, published reports in healthcare settings using this mode of survey administration are few. Most studies report off-site administration via post, telephone, or electronic (email and web-based) modes, and some also describe mixed methods.
Given advances in information technology and communication capabilities, electronic modes have become most prevalent and afford advantages of reduced costs, fast delivery and response, access to large populations and, in the case of web-based applications, direct data capture. Disadvantages of electronic modes include time-consuming initial development, coding, testing, sample bias due to computer illiteracy or non-availability, and data security concerns. Emails can suffer from lack of correct email addresses, because they change more often than postal addresses, software incompatibility, and data download errors,.
Telephone surveys are the least preferable mode for pharmacists given the practical constraints of time, personnel and financial resources. Nevertheless, since majority of the adults aged 18–34 years old have mobile phones and use them extensively, random digit dialling of mobile phones was found to be a feasible methodology for surveillance of young adults.
Alternatively, respondents could fill out surveys on a computer provided by the pharmacist on site. This approach, while it eliminates software compatibility issues and permits direct data capture, adds the burden of providing computers in a private location and that of scheduling.
Finally, the old fashioned method of posting surveys is still a viable mode of survey administration and in some cases was found to be more effective in improving response rate than the electronic mode.
Maximising response rates
By nature, data from survey methodology is entirely dependent upon the subjects’ willingness to respond and poor response rates can be a serious problem for study’s validity,. While desired response rates for most research and for pharmacy are approximately 60% and 80% respectively, the average response rate in 490 survey studies published between 2000 and 2005 for journals in the US was 52.7%. Among strategies reported to improve response rates are: pre-notifying participants about the survey and personalising the survey; publicising the survey; monitoring responses and sending reminders; promising feedback on results; and using monetary or non-monetary incentives. Although shortening a lengthy questionnaire significantly increased the response, the trade-off between the value of additional questions and a larger sample should be considered. Face-to-face interaction as a primer to set up the survey was also shown to be effective.
Regarding the mode of survey administration, the latest reports tend to support best response rates with mixed-method approaches using both hard copy and electronic survey collection methods,.
Irrespective of the response rate, it is critical to either show that the non-respondents are not different from respondents in terms of important characteristics, or take into account the differences when interpreting the results. A statistical comparison of the responders and the non-responders on key parameters will reveal whether or not significant differences exist. It is also important to state the proportion of the questionnaire not completed, which is not the same as the response rate.
Asking sensitive questions
Another area requiring special attention is that of collecting data on sensitive questions such as sexual habits, illicit drug use, and other behaviors considered unlawful or socially undesirable. Typically, answers to sensitive questions are either omitted or not answered truthfully. While some studies report on possible ways of improving the response, such as using face-to-face interviews or non-randomised response techniques, it is best to keep the possibility of omission or incorrect answers in mind when interpreting the results.
With thanks to Anant Kishore for minor editing.
Citation: The Pharmaceutical Journal DOI: 10.1211/PJ.2016.20200700
Recommended from Pharmaceutical Press