Validity And Reliability of Critical Thinking Skills Instrument and Deep Learning Graduates Profile in Science Learning
DOI:
https://doi.org/10.55681/nusra.v7i4.6653Keywords:
validity, reliability, critical thinking skills, graduate profile, deep learning, PjblAbstract
This study aims to test the validity and reliability of two main instruments in the research on the development of a video-assisted PJBL model in science learning in junior high schools. The first instrument is a critical thinking skills test in the form of an essay based on Ennis's taxonomy, and the second instrument is a deep learning graduate profile questionnaire that measures three-character dimensions: independence, collaboration, and creativity (27 statement items). Content validity was tested using the Content Validity Ratio (CVR) and Content Validity Index (CVI) involving 5 expert validators. The reliability of the Critical Thinking Skills instrument was tested with inter-rater reliability (Cohen's Kappa), while the graduate profile questionnaire was tested with Cronbach's Alpha on 30 respondents. The results showed: the KBK instrument obtained a CVI = 1.000 (very valid) with a Kappa coefficient κ = 0.826 (very reliable); the graduate profile instrument obtained a CVI = 0.852 (valid) with a Cronbach's Alpha α = 0.946 (very reliable). All items were declared suitable for use in research, with several items requiring minor revisions based on validator notes.
Downloads
References
Borich, G. D. (1994). Observation skills for effective teaching. Macmillan.
Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. In JB Baron & RJ
Sternberg (Eds.), Teaching thinking skills: Theory and practice (pp. 9–26). WH Freeman.
Facione, P. A. (2011). Critical thinking: What it is and why it counts. Insight Assessment.
Fraenkel, J.R., Wallen, N.E., & Hyun, H.H. (2019). How to design and evaluate research in education (10th ed.). McGraw-Hill Education.
Fullan, M., & Langworthy, M. (2014). A rich seam: How new pedagogies discover deep learning. Pearson.
George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference (4th ed.). Allyn & Bacon.
Ministry of Education, Culture, Research, and Technology. (2025). Elementary and Secondary Education Graduate Profile Framework. Ministry of Education, Culture, Research, and Technology.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. https://doi.org/10.2307/2529310
Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x
Polit, D.F., & Beck, C.T. (2006). The content validity index: Are you sure you know what's being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489–497. https://doi.org/10.1002/nur.20147
Wilson, F.R., Pan, W., & Schumsky, D.A. (2012). Recalculation of the critical values for Lawshe's content validity ratio. Measurement and Evaluation in Counseling and Development, 45(3), 197–210. https://doi.org/10.1177/0748175612440286
Zamanzadeh, V., Ghahramanian, A., Rassouli, M., Abbaszadeh, A., Alavi-Majd, H., & Nikanfar, A. R. (2015). Design and implementation content validity study: Development of an instrument for measuring patient-centered communication. Journal of Caring Sciences, 4(2), 165–178. https://doi.org/10.15171/jcs.2015.017
Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). Prentice Hall.
Azwar, S. (2012). Reliability and validity (4th ed.). Student Library.
Brookhart, S. M. (2013). How to create and use rubrics for formative assessment and grading. ASCD.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104
Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Pearson Education.
Griffin, P., & Care, E. (Eds.). (2015). Assessment and teaching of 21st century skills: Methods and approaches. Springer. https://doi.org/10.1007/978-94-017-9395-7
Hair, JF, Black, WC, Babin, BJ, & Anderson, RE (2019). Multivariate data analysis (8th ed.). Cengage Learning.
Henson, R. K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34(3), 177–189.
Hmelo-Silver, C.E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16(3), 235–266. https://doi.org/10.1023/B:EDPR.0000034022.16470.f3
Jonassen, D.H. (2011). Learning to solve problems: A handbook for designing problem-solving learning environments. Routledge.
Ministry of Education, Culture, Research, and Technology. (2022). Decree of the Minister of Education, Culture, Research, and Technology No. 56/M/2022 concerning Guidelines for Curriculum Implementation in the Context of Learning Recovery. Ministry of Education, Culture, Research, and Technology.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
Sugiyono. (2019). Quantitative, qualitative, and R&D research methods (2nd ed.). Alfabeta.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Masringgit Marwiyah Nst, Festiyed, yerimadesi, skunda Diliarosta, Febri yanto, putri Zuhra
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
