|Year : 2013 | Volume
| Issue : 1 | Page : 28-31
Test-retest reliability of a questionnaire to assess the knowledge and attitude of dental students toward tobacco counseling services
AO Ehizele, CC Azodo
Department of Periodontics, University of Benin, Benin City, Edo State, Nigeria
|Date of Web Publication||17-May-2013|
C C Azodo
Department of Periodontics, New Dental Complex, University of Benin Teaching Hospital, P.M.B. 1111 Ugbowo, Benin City, Edo State - 300 001
Objective: To determine the test-retest reliability of a new questionnaire designed to assess the attitude of students in a Nigerian dental school to tobacco cessation services. Materials and Methods: A self-administered questionnaire was administered twice at 4 weeks interval to the same set of final year dental students (N = 36) in one of the Nigerian dental schools. Ten questions were randomly selected from the 35-item questionnaire. Intra-class correlation (ICC) coefficients at 95% confidence interval were computed for each selected item and the Cronbach's alpha coefficients were calculated to assess the internal consistency of the questionnaire. Results: The results showed that the least Cronbach's alpha value was 0.49 and the highest was 0.98. Only two items had less than a substantial agreement level (<0.60) and seven items had an almost perfect agreement level (>0.80). The results also indicated that the ICC coefficient is very highly statistically significant (P < 0.001). Conclusion: The questionnaire demonstrated adequate test-retest reliability to assess the attitude of dental students to tobacco cessation services.
Keywords: Dental students, questionnaires, reliability, tobacco cessation
|How to cite this article:|
Ehizele A O, Azodo C C. Test-retest reliability of a questionnaire to assess the knowledge and attitude of dental students toward tobacco counseling services. Sahel Med J 2013;16:28-31
|How to cite this URL:|
Ehizele A O, Azodo C C. Test-retest reliability of a questionnaire to assess the knowledge and attitude of dental students toward tobacco counseling services. Sahel Med J [serial online] 2013 [cited 2021 Jun 19];16:28-31. Available from: https://www.smjonline.org/text.asp?2013/16/1/28/112066
| Introduction|| |
A questionnaire is a very useful tool for data collection, especially when a large number of respondents in many locations are to be used and when the information required is straightforward. Questionnaires are economical, provide standardized answers, and encourage pre-coded answers. Although questionnaires can be open or designed to allow respondents to state their opinion in their own words, the value of the data is likely to be the greatest where respondents provide answers that fit into a range of options offered by the researcher (closed questionnaire). These allow for the speedy collation and analysis of data by the researcher. They also have an advantage for the respondents, who, instead of thinking of how to express their ideas, are faced with the relatively easy task of needing to pick one or more answers which are spelt out for them. 
However, it is of utmost importance to determine the reliability of the questionnaire. Reliability means consistency or reproducibility of measurement.  It is the degree to which an instrument will give similar results for the same individuals at different times under the same condition. A reliable questionnaire is one that would give the same results if used repeatedly with the same group. The test-retest reliability coefficient gives an estimate of the error of measurement, or the range of fluctuation likely to occur in a single individual's score as a result of irrelevant, chance factors.  A Cronbach's alpha coefficient of 0.6 and above is a desirable test-retest reliability coefficient. 
One simple way of testing this would be to apply the questionnaire twice under exactly the same conditions and see if similar results will be obtained. It has been stated that since people can change their mind over time, measurement of attitudes, beliefs, and intention on two different occasions may not be totally identical. However, when the interval is not too far apart, some degree of correlation may be expected.  Longer time interval between administration of questionnaires may affect the reliability rates of a questionnaire used to assess tobacco use.  On the other hand, a questionnaire used to gain insights into the tobacco cessation scenario among dental graduates had the reliability values acceptable for various domains of the questionnaire.  The objective of this study was to determine the test-retest reliability of a new questionnaire intended to be used to assess the attitude of Nigerian dental students to tobacco cessation services and their compliance with the 4As approach to tobacco cessation.
| Materials and Methods|| |
A 35-item questionnaire containing both open- and closed-ended questions was developed in English to assess the students' demographic variables, tobacco use, exposure to secondhand smoke, attitude to tobacco cessation services, present practice in the area of patients counseling on tobacco cessation and the perceived constraint against counseling, students' specific training on tobacco cessation, and their knowledge of the effects of tobacco on oral health. The questionnaire was reviewed for content and face validity. Required changes were made to clarify any ambiguity and to ensure comprehension of the Nigerian dental students to be studied.
This self-administered questionnaire was administered twice to the same set of final year dental students (N = 36) in one of the Nigerian dental schools. The test interval was 4 weeks. The first set of completed questionnaires was used for the test, while the second set was used for the retest. Ten questions were randomly selected from the 35-item questionnaire.
Data analysis was done with SPSS version 15.0. Intra-class correlation (ICC) coefficient at 95% confidence intervals was computed for each selected item. P value <0.05 was considered statistically significant. Cronbach's alpha coefficients were calculated to assess internal consistency of the questionnaire. The rating suggested by Landis and Koch  was used for result interpretation. An agreement level of 0.0-0.2 was considered poor, 0.2-0.4 fair, 0.4-0.6 moderate, 0.6-0.8 substantial, and from 0.8 to <1.0 almost perfect.
| Results|| |
The results showed that the strength of association between the two tests is very high. The least Cronbach's alpha value was 0.49 and the highest was 0.98. Only two items had less than a substantial agreement level (<0.60) and seven items had an almost perfect agreement level (>0.80) [Table 1].The open-ended questions on why dentists have a role in counseling had Cronbach's alpha value of 0.93 [Table 2] and that on the major constraint against counseling in dental practice had Cronbach's alpha value of 0.70 [Table 3].The results also showed that the ICC coefficient was very highly statistically significant (P < 0.05 in all cases).
| Discussion|| |
Cronbach's alpha is a statistical tool for investigating the internal consistency of a questionnaire. 
The result from this study reflects very low variability between the two tests, and so it can deduced that the questionnaires may be reliable in assessing the knowledge of dental students about tobacco cessation, as well as their attitude to it. This finding supports other studies reporting that the use of questionnaire is reliable for studies on tobacco such as tobacco dependence, , tobacco screening,  as well as substance abuse generally. 
The item that dealt with the dental students' opinion on dentists routinely advising patients who smoke to quit was the least reliable, i.e. had the lowest Cronbach's alpha value. This was probably because some dental students did not give a consistent response to this question on the two occasions. The students were perhaps not sure if a dentist could play such a role or did not feel sufficiently capable of carrying out such a role at this stage of their training. It has been reported that not all dentists proactively advise patients to give up smoking, but more dentists might do so if they had relevant local guidelines and practical training programs. 
The item that elicited self-reported tobacco use also gave a low value of Cronbach's alpha. This might be reflective of initial denial of tobacco use by some respondents. Studies have reported that some people tend to hide their smoking status because smoking is not socially accepted. ,
Open-ended questions allow people to write down whatever they like to say on a subject matter.
Although such questionnaires give richer information, more time will be needed to complete them.  There is also the likelihood that the respondents may change their minds on retesting. Inability to achieve reliable coding of open-ended questionnaire has been identified to be one of the basic measurement problems in survey research and a source of variability. , However, this study revealed excellent reliability of the open-ended questions.
In conclusion, the questionnaire demonstrated adequate test-retest reliability in most of the items involved. Acceptable reliability was got not only when dichotomous answers were required but also when respondents were required to state their opinion in their own words. This questionnaire may therefore be useful in assessing the attitude of the study population to tobacco cessation services and their compliance with the 4As approach to tobacco cessation.
| References|| |
|1.||Denscombe M. Questionnaires in The Good Research Guide for small-scale social research projects. 4 th ed. England: Open University Press; 2007. p. 155-71. |
|2.||Hopkins WG. Measures of reliability in sports medicine and science. Sports Med 2000; 30:1-15. |
|3.||Anastasi A. Reliability in pyschological testing. 6 th ed. New York: MacMillan Publishing; 1988. p. 1908-2001. |
|4.||Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-74. |
|5.||Brinkman WP. Design of a Questionnaire Instrument, Handbook of Mobile Technology Research Methods. New York: Nova Publisher; 2009. p. 31-57. |
|6.||Huerta M, Chodick G, Balicer RD, Davidovitch N, Grotto I. Reliability of self-reported smoking history and age at initial tobacco use. Prev Med 2005;41:646-50. |
|7.||Binnal A, Rajesh G, Denny C, Ahmed J. Insights into the tobacco cessation scenario among dental graduates: An Indian perspective. Asian Pacific J Cancer Prev 2012;13:2611-7. |
|8.||Bland JM, Altman DG. Cronbach's alpha. BMJ 1997;314:572. |
|9.||Pomerleau CS, Carton SM, Lutzke ML, Flessland KA, Pomerleau OF. Reliability of the fagerstrom tolerance questionnaire and the fagerstrom test for nicotine dependence. Addict Behav 1994;19:33-9. |
|10.||Heishman SJ, Singleton EG, Moolchan ET. Tobacco craving questionnaire: reliability and validity of a new multifactorial instrument. Nicotine Tob Res 2003;5:645-54. |
|11.||Kawakami N, Takatsuka N, Inaba S, Shimizu H. Development of a screening questionnaire for tobacco/nicotine dependence according to ICD-10, DSM-III-R, and DSM-IV. Addict Behav 1999;24:155-66. |
|12.||Machado Neto Ade S, Andrade TM, Fernandes GB, Zacharias HP, Carvalho FM, Machado AP, et al. Reliability of a questionnaire on substance use among adolescent students, Brazil. Rev Saude Publica 2010;44:830-9. |
|13.||Lu HX, Wong MC, Chan KF, Chan TK, Chung WY, Leung MY, et al. Perspectives of the dentists on smoking cessation in Hong Kong. Hong Kong Dent J 2011;8:79-86. |
|14.||Awidi AS. Patterns of cigarette smoking in Jordan: A study of greater Amman area. Ann Saudi Med 1991;11:144-7. |
|15.||Jarallah JS, al-Rubeaan KA, al-Nuaim AR, al-Ruhaily AA, Kalantan KA. Prevalence and determinants of smoking in three regions of Saudi Arabia. Tob Control 1999;8:53-6. |
|16.||Hruschka D, Cummings B, Cobb St John D, Moore J, Khumalo-Sakutukwa G, Carey JW. Fixed-choice and open-ended response formats: A comparison from HIV prevention research in Zimbabwe. Field Methods 2004;16:184-202. |
|17.||Leiva FM, Ríos FJ, Martínez TL. Assessment of interjudge reliability in the open-ended questions coding process. Qual Quantity 2006;40:519-37. |
[Table 1], [Table 2], [Table 3]