Development and Validation of a Taxonomy for Specific Questions Based on Deficiencies in Logical Reasoning
Abstract
Purpose of the study: This study aims to develop a taxonomy of specific questions based on deficiencies in three types of logical reasoning: inductive, deductive, and hypothetical reasoning. The study also seeks to validate the quality of this taxonomy.
Methodology: This study is a developmental research project that utilized a “three-level model” combining deductive approaches with empirical data analysis to develop and validate a taxonomy of questions. A convenience sampling method was employed, whereby 57 questions were selected from 1,164 posed by graduate students at a university in Japan. The question data were categorized by two raters. Descriptive statistics and the kappa coefficient method was employed to verify the quality of the categorization.
Main Findings: The taxonomy of specific questions comprised nine categories, with three categories assigned to each type of logical reasoning. The comprehensiveness assessment results showed that all questions were assigned to one category, with at least one question included in each category. The results for exclusivity and objectivity revealed high kappa coefficients, indicating a high degree of agreement between raters. However, there was some confusion among question categories within the same reasoning method during the categorization process.
Novelty/Originality of this study: The development of a taxonomy of specific questions based on logical reasoning deficiencies. This framework facilitates the design of strategies that enable learners to generate specific questions. The proposed method of assigning questions enables an objective assessment of learners’ proficiency in formulating specific questions and is expected to more effectively guide the support process.
References
T. Bielik and A. Yarden, “Promoting the asking of research questions in a high-school biotechnology inquiry-oriented program,” Int. J. STEM Educ., no. 15, 2016, doi: 10.1186/s40594-016-0048-x.
S. K. Ratan, T. Anand, and J. Ratan, “Formulation of research question-stepwise approach,” J. Indian. Assoc. Pediatr. Surg., vol. 24, no. 1, pp. 15-20, 2019, doi: 10.4103/jiaps.JIAPS_76_18.
N. M. Dah, M. S. A. Mat Noor, M. Z. Kamarudin, and M. M. Ibrahim, “Facilitation of student questioning in the malaysian secondary science classroom using the Investigable Questioning Formulation Technique (IQFT) protocol,” Asia-Pac. Sci. Educ., vol. 9, no. 1, pp. 9-43, 2023, doi: 10.1163/23641177-bja10063.
L. Qiu, F. Ikeda, and N. Yamashita, “Development of logic-based question classification methods to measure the ability to ask questions essential for setting research questions”, In. Sci. Ed. J., vol. 5, no. 2, pp. 91-100, 2024, doi: 10.37251/isej.v5i2.1003.
M. Glöggler and E. Ammenwerth, “Development and validation of a useful taxonomy of patient portals based on characteristics of patient engagement,” Methods. Inf. Med., no. 60, pp. e44-e55, 2021, doi: 10.1055/s-0041-1730284.
G. Marbach‐Ad and P. G. Sokolove, “Good science begins with good questions,” J. Coll. Sci. Teach., vol. 30, no. 3, pp. 192-195, 2000.
C. Chin and J. Osborne, “Students’questions: a potential resource for teaching and learning science,” Stud. Sci. Educ., vol. 44, no. 1, pp. 1–39, 2008, doi: 10.1080/03057260701828101.
C. Chin and J. Osborne, “Students’ questions and discursive interaction: Their impact on argumentation during collaborative group discussions in science,” J. Res. Sci. Teach., vol. 47, no. 7, pp. 883-908, 2010, doi: 10.1002/tea.20385.
E. Panadero, G. T. Brown, and J. W. Strijbos, “The future of student self-assessment: A review of known unknowns and potential directions,” Educ. Psychol. Rev., vol. 28, pp. 803–830, Dec. 2016, doi: 10.1007/s10648-015-9350-2.
I. Fitriyah and M. Jannah, “Online assessment effect in EFL classroom: An investigation on students and teachers’ perceptions,” Indonesian Journal of English Language Teaching and Applied Linguistics, vol. 5, no. 2, pp. 265-284, 2021, doi: 10.21093/ijeltal.v5i2.709.
N. P. Subheesh and S. S. Sethy, “Learning through assessment and feedback practices: A critical review of engineering education settings,” Eurasia. J. Math., Sci Tech. Ed., vol. 16, no. 3, 2020, Art no. em1829, doi: 10.29333/ejmste/114157.
A. K. Charalampous and M. Darra, “The contribution of teacher feedback to learners’work revision: A systematic literature review,” World J. Educ., vol. 13, no. 3, pp. 40-63, 2023, doi: 10.5430/wje.v13n3p40.
A. Astalini, D. Darmaji, Dwi Agus Kurniawan, H. Jaya, and Sri Muslimatul Husna, “Analysis of teacher responses to the use of web-based assessment to assess students’ attitudes towards science subjects,” In. Sci. Ed. J, vol. 3, no. 3, pp. 66-71, 2022, doi: 10.37251/isej.v3i3.282.
H. Harizon, “Description of teacher responses to the implementation of student process skills portfolio assessment,” In. Sci. Ed. J, vol. 4, no. 3, pp. 128-134, Sep. 2023, doi: https://doi.org/10.37251/isej.v4i3.731.
C. Chin, “Using self-questioning to promote pupils’ process skills thinking,” Sch. Sci. Rev., vol. 87, no. 321, pp. 113-123, 2006.
V. Urquhart and M. Monette, Teaching Writing in the Content Areas. Alexandria, VA, USA: ASCD, 2005.
D. Prayoga, H. Herlina, and E. Rosmiyati, “Journalist questions technique in wiriting ability with narrative text,” Esteem Journal of English Education Study Programme, vol. 6, no. 1, pp. 111–116, 2023, doi: 10.31851/esteem.v6i1.10223.
R.C. Richey and J.D. Klein, “Design and Development Research,” in Handbook of Research on Educational Communications and Technology. J. Spector, M. Merrill, J. Elen, M. Bishop, Eds., New York, NY. USA:Springer. 2014, doi: 10.1007/978-1-4614-3185-5_12
F. Emamjome, G. Gable, and A. Rabaa’i, “Understanding the value of social media in organisatons: a taxonomic approach,” presented at PACIS 2014 Proceedings., Chengdu, CHN, Jan. 1-1, 2014.
K. Behfar and G. A. Okhuysen, “Discovery within validation logic: deliberately surfacing, complementing, and substituting abductive reasoning in hypothetico-deductive inquiry,” Organ. Sci., vol. 29, no. 2, pp. 323-340, 2018, doi: 10.1287/orsc.2017.1193.
R. S. Nickerson, D. N. Perkins, and E. E. Smith, The Teaching of Thinking. Evanston, IL, USA:Routledge, 2014.
F. Ikeda, N. Yamashita, and L. Qiu, “Classification Method for logic-based questions leading to scientific inquiry: the art of posing questions that foster scientific creativity,” in Japan Creativity Society., Tokyo, Japan, 2023,pp.35-42.
C. S. Peirce, “Deduction, induction, and phypothesis,” Popular Science Monthly, vol. 12, pp. 470-482, 1878.
Accessed: October, 18, 2024. [Online]. Available: https://cspeirce.omeka.net/items/show/7
H. L. Andrade, “A critical review of research on student self-assessment,” Front. Educ., vol. 4, Art. 87, pp. 1-13, 2019, doi: 10.3389/feduc.2019.00087.
G. T. L. Brown, H. L. Andrade, and F. Chen, “Accuracy in student self-assessment: directions and cautions for research,” Assess. Educ.: Princ. Policy Pract., vol. 22, no. 4, pp. 444–457, 2015, doi: 10.1080/0969594X.2014.996523.
J. Golzar, S. Noor, and O. Tajik, “Convenience Sampling,” Int. J. Educ. Lang. Stud., vol. 1, no. 2, pp. 72-77, 2022, doi: 10.22034/ijels.2022.162981.
G. Rau and Y. Shih, “Evaluation of Cohen’s kappa and other measures of inter-rater agreement for genre analysis and other nominal data,” J. Engl. Acad Purp., vol. 53, 2021, doi: 10.1016/j.jeap.2021.101026.
A. I. Moreno and J. M. Swales, “Strengthening move analysis methodology towards bridging the function-form gap,” English for Specific Purposes, vol. 50, pp. 40-63, 2018, doi: 10.1016/j.esp.2017.11.006.
F. Henry et al., “Sugarcane land classification with satellite imagery using logistic regression model,” in IOP Conf. Ser.: Mater. Sci. Eng., 2017, pp. 1-6, doi: 10.1088/1757-899X/185/1/012024.
F. Ikeda, “How to evaluate the ability to ask questions: verification of the reliability of meta-model based evaluation,” J. Higher Education and Lifelong Learning, vol. 25, no. 55, pp. 55-61, 2018, doi: 10.14943/J.HighEdu.25.55.
Copyright (c) 2025 Li Qiu, Fumihito Ikeda, Naoko Yamashita
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and acknowledge that the Integrated Science Education Journal is the first publisher licensed under a Creative Commons Attribution 4.0 International License.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges and earlier and greater citation of published work.
