Developing a Checklist for Evaluating Research Articles in Applied Linguistics

Document Type : Original Article

Authors

1 Chabahar Maritime University, Chabahar, Iran

2 Department of English Language and Literature, Faculty of Humanities and Social Sciences, Golestan University, Gorgan, Iran

Abstract

Although researchers, postgraduate students, and university professors are under tremendous pressure to publish or perish, there exist few widely acknowledged rating scales or checklists for evaluating manuscripts submitted to journals in applied linguistics. To top it all, the existent peer-reviewing criteria are typically not publicly shared; thus, peer-reviewing is a closed, occluded genre. To fill this lacuna in applied linguistics, this research critically analyzed the evaluation scales, existing review criteria, and available documents from 18 journals, two publishing institutions, and APA guidelines in order to come up with a comprehensive checklist for evaluating research articles in applied linguistics. The analysis of the data through inductive content analysis revealed 43 categories, which were later classified under six main themes. It was found that researchers must be aware of the practicalities of research, include the essentials in their manuscript, and appeal to editors by adding elements of face validity. Having met these requirements, researchers must enjoy excellent composition skills and adhere to the fundamental principles in research ethics in order to be able to write a research paper having high overall value. The paper concludes with several implications for researchers and opens up some avenues for future research.    

Keywords

References

Adamson, J., & Fujimoto-Adamson, N. (2016). Sustaining reviewing quality: Induction, mentoring, and community. Journal of English Scholars Beyond Borders, 2, 29-57.
Adel, M. R., & Ghorbani Moghadam, R. (2015). A comparison of moves in conclusion sections of research articles in psychology, Persian literature and applied linguistics. Teaching English Language, 9(2), 167-191.
Ali, P. A., & Watson, R. (2016). Peer review and the publication process. Nursing Open, 3(4), 193-202.
Alinasab, M., Gholami, J., & Mohammadnia, Z. (2021). Genre-based revising strategies of graduate students in applied linguistics: Insights from term papers. Journal of English for Academic Purposes, 49, https://www.sciencedirect.com/science/article/pii/S1475158520304756
Aspinwall, K., Simkins, T., Wilkinson, J. F., & McAuley, M. G. (1992). Managing evaluation in education. London, UK: Routledge.
Babaii, E., Atai, M. R., & Mohammadi, V. (2015). Stance in English Research articles: Two disciplines of the same science, Teaching English Language, 9(1), 1-27.
Baždarić, K., Bilić-Zulle, L., Brumini, G., Petrovečki, M. (2012). Prevalence of plagiarism in recent submissions to the Croatian Medical Journal. Science and Engineering Ethics, 18, 223–239.
Beck, C. T. (1990). The research critique: General criteria for evaluating a research report. Journal of Obstetric Genecologic, and Neonatal Nursing, 19(1), 18-22.
Belcher, B. M., Rasmussen, K. E., Kemshaw, M. R., & Zornes, D. A. (2016). Defining and assessing research quality in a transdisciplinary context. Research Evaluation, 25(1), 1-17.
Belcher, D. D. (2007). Seeking acceptance in an English-only research world. Journal of Second Language Writing, 16(1), 1-22.
Bhatia. V. K. (2004). Worlds of written discourse: A genre-based view. London, UK: Continuum International.
Butler, J. (1990). Gender trouble: Feminism and the subversion of identity. New York, NY: Routledge.
Butler, D. (2010). Journals step up plagiarism policing. Nature, 466(7303), 167.
Coriat, A. (2019). PhD merit needs to be defined by more than just publications. Nature Human Behaviors, 3(10), 1007.  
Davidoff, F. (2004). Improving peer reviews: Who’s responsible? BMJ, 328, 658-659.
Devitt, A. (2015). Genre performances: Genre analysis and rhetorical-linguistic genre studies. Journal of English for Academic Purposes, 19, 44-51.
Ebrahimi, B., & Weisi, H. (2019). Genre variation in the introduction of scientific papers in Iranian and international computer science journals. Issues in Language Teaching, 8(2), 51-82. 
Egbert, J. (2007). Quality analysis of journals in TESOL and Applied Linguistics. TESOL Quarterly, 41(1), 157-171.
European Science Foundation, )2012(. Evaluation in research and research funding organizations: European practices. In: A report by the ESF Member Organization Forum on Evaluation of Publicly Funded Research., ISBN978-2-918428-83-1.
Falkenberg, L. J., & Soranno, P. A. (2018). Reviewing reviews: An evaluation of peer reviews of journal article submissions. Limnology and Oceanography Bulletin, 27(1), 1-5.
Fanelli, D. (2010). Do pressures to publish increase scientists’ bias? An empirical support from US States data, PLOS ONE, 5(4), 1-7.
Flowerdew, J. (2013). Discourse in English language education. London, UK: Routledge.
Garcia, J. A., Rodriguez-Sánchez, R., & Fdez-Valdivia, J. (2015). Bias and effort in peer review. Journal of the American Society for Information Science and Technology, 66(10), 2010-2030.
Grimes, D. R., Bauch, C. T., & Ioannidis, J. P. A. (2018). Modelling science trustworthiness under publish or perish pressure. Royal Society Open Science, 5(1), 1-14.
Hames, I. (2007). Peer review and manuscript management in scientific journals: Guidelines for good practice. Malden, MA: Blackwell.
Haven, T. L., Bouter, L. M., Smulders, Y. M., & Tijdink, J. K. (2019). Perceived publication pressure in Amsterdam: Survey of all disciplinary fields and academic ranks. PLOS ONE, 14(6), 1-12.
Jalilifar, A. (2010). Thematization in EFL students’ composition writing and its relation to academic experience. RELC Journal, 41(1), 31-45.
Jalilifar, A. (2011). World of attitudes in research article discussion sections: A cross-linguistic perspective. Journal of Technology and Education, 5(3), 177-186.
Jalilifar, A., & Shahvali, P. (2013). Writing suggestions for further research in Iranian applied linguistics theses: A generic and metadiscoursal investigation. Iranian Journal of Applied Linguistics (IJAL), 16(1), 79-105.
Karney, B. W.  (Guest Editorial) (1998). Reviewing the review process. Journal of Hydraulic Engineering, 124(6), 561-562.
Khany, R., & Abol-Nejadian, R. (2010). Iranian post-graduate students’ problems in writing academic research articles. Teaching English Language, 4(2), 1-26.
Langfeldt, L., Nedeva, M., Sörlin, S., & Thomas, D. A. (2020). Co-existing notions of research quality: A framework to study context-specific understandings of good research. Minerva, 58, 115-137.
Lazaraton, A. (2003). Evaluative criteria for qualitative research in applied linguistics: Whose criteria and whose research? The Modern Language Journal, 87(1), 1-12.
Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology 64(1), 2-17.
Maniati, M., Jalilifar, A., & Hayati, A. (2015). Iranian scholars’ revision of their submitted manuscripts: Signaling impersonality in text. Journal of Research in Applied Language Studies, 6(1), 118-140.
Marsden, J. R. (2015). How could my paper have gotten rejected? In J. Liebowitz (Ed.), A guide to publishing for academics. Inside the publish or perish phenomenon (pp. 67-78). Boca Raton, FL: CRC Press.
Mårtensson, P., Fors, U., Wallin, S.-B., Zander, U., & Nilsson, G. H. (2016). Evaluating research: A multidisciplinary approach to assessing research practice and quality. Research Policy, 45(3), 593-603.
Mehrabi, K., Jalilifar, A., Hayati, A., & White, P. (2018). Investigating action nominalization in the introduction sections of research articles: A cross-disciplinary study of hard and soft sciences. Teaching English Language, 12(2), 1-26. 
Nasseri, M. (2021). Is postgraduate English academic writing more clausal or phrasal? Syntactic complexification at the crossroads of genre, proficiency, and statistical modelling. Journal of English for Academic Purposes, 49, https://www.sciencedirect.com/science/article/pii/S1475158520304744  
Nedić, O., & Dekanski, A. (2016). Priority criteria in peer review of scientific articles. Scientometrics, 107, 15-26.
Paltridge, B. (2017). The discourse of peer review: Reviewing submissions to academic journals. London, UK: Palgrave Macmillan: London.
Pickersgill, M. (2012). The co-production of science, ethics and emotion. Science, Technology, and Human Values, 37, 579–603.
Rahimi, M.,  Yousofi, N., & Moradkhani, S. (2019). Exploring the professional identity construction of ELT researchers in higher education. Teaching English Language, 13(1), 85-117.
Raitskaya, L. & Tikhonova, E. (2020). Pressure to publish internationally: Scholarly writing coming to the fore. Journal of Language and Education, 6(1), 4-7.
Riazi, M. A. (2016). The Routledge encyclopedia of research methods in applied linguistics: Quantitative, qualitative and mixed-methods, Routledge. Basingstoke: Taylor & Francis Ltd.
Rostami, K., Khadjooi, K., Abasaeed-Elhag, R., & Ishaq, S. (2011). How to evaluate a manuscript for publication? Gastroenterology and Hepatology: From Bed to Bench, 4(2), 58-62.
Saxena, A., Thawani, V., Chakrabarty, M., & Gharpure, K. (2013). Scientific evaluation of the scholarly publications. Journal of Pharmacology and Pharmacotherapeutics, 4(2), 125-129.
Schou, L., Høstrup, H., Lyngsø, E.E., Larsen, S. & Poulsen, I. (2011). Validation of a new assessment tool for qualitative research articles. Journal of Advanced Nursing, 68(9), 2086-2094.
Schroter, S., Black, N., Evans, S., Carpenter, J., Godlee, F., & Smith, R. (2004). Effects of training on quality of peer review: Randomised controlled trial. BMJ, 328, 328-673.
Shattell, M. M., Chinn, P., Thomas, S. P., & Cowling, W. R. (2010). Authors’ and editors’ perspectives on peer review quality in three scholarly nursing journals. Journal of Nursing Scholarship, 42(1), 58-65.
Shirazizadeh, M., & Amirfazlian, R. (2021). Lexical bundles in theses, articles and textbooks of applied linguistics: Investigating intradisciplinary uniformity and variation. Journal of English for Academic Purposes, 49, https://doi.org/10.1016/j.jeap.2020.100946
Smith, R. (2006). Peer review: A flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99(4), 178-182.
Somashekhar, S. P. (2020). Art of scientific writing. Indian Journal of Gynaecologic Oncology, 18(2). https://doi.org/10.1007/s40944-020-00382-y
Soodmand Afshar, H., Doosti, M., & Movassagh, H. (2018). A genre analysis of the introduction section of applied linguistics and chemistry research articles. Iranian Journal of Applied Linguistics, 21(1), 163-214.
Strauss, S., & Feiz, P. (2014). Discourse analysis. Putting our worlds into words. New York, NY: Routledge.
Swales, J. (1990). Genre analysis: English in academic and research settings. Cambridge, UK: Cambridge University Press.
Tight, M. (2003). Reviewing the reviewers. Quality in Higher Education, 9(3), 295-303.
Zhang, Y. (2010). Chinese journal finds 31% of submissions plagiarized. Nature, 467(7312), 153.
Teaching English Language
Volume 14, Issue 2
September 2020
Pages 239-268

Files

History

  • Receive Date: 17 July 2020
  • Revise Date: 19 November 2020
  • Accept Date: 20 December 2020

Share

How to cite

Statistics