Development of a Speaking Competence Evaluation Instrument for A2-Level Students through a Corpus-Based German–Indonesian Slang Dictionary in the German Language Education Program

Tri Edliani Lestari; Ninuk Lustyantie; Fathiaty Murtadho

doi:10.55681/jige.v6i2.4005

Authors

Tri Edliani Lestari Universitas Negeri Jakarta
Ninuk Lustyantie Universitas Negeri Jakarta
Fathiaty Murtadho Universitas Negeri Jakarta

DOI:

https://doi.org/10.55681/jige.v6i2.4005

Keywords:

Students’ speaking skill, german slang, Corpus Linguistics, lexicography, foreign language evaluation

Abstract

The ability to speak fluently and appropriately in German remains a persistent challenge for Indonesian learners at the A2 level, especially in contexts that demand spontaneous, pragmatic communication. Many existing evaluation tools focus narrowly on grammatical accuracy, often neglecting fluency, lexical diversity, and sociopragmatic competence. This study aims to develop a corpus-informed speaking evaluation instrument specifically designed for assessing A2-level German learners in Indonesian higher education. Employing a Research and Development (R&D) framework using the ADDIE model (Analysis, Design, Development, Implementation, Evaluation), the instrument was systematically constructed through needs analysis, corpus-based task design, expert validation, and iterative refinement. Data collection involved interviews, classroom observations, and document analysis, supported by expert judgment and inter-rater reliability testing. The final product includes task-based prompts grounded in authentic communicative scenarios, a Likert-based rubric covering fluency, accuracy, vocabulary range, and pragmatic appropriateness, and lexical input informed by spoken German corpora. Results demonstrate the instrument's validity and reliability, while qualitative feedback supports its pedagogical relevance. This study offers a novel contribution to corpus-informed language assessment and serves as a model for localized speaking evaluation tools aligned with CEFR standards.

Downloads

Download data is not yet available.

References

Afia, N., & Laili, M. (2023). Developing a Corpus-Based English Vocabulary Dictionary using The ADDIE Model. Nusantara Educational Review, 1(1), 56–62. https://doi.org/10.5678/ner.v1i1.123

Alderson, J. C. (2000). Assessing reading. Cambridge University Press. https://doi.org/10.1017/CBO9780511732935

Almelhi, A. M. (2021). The effect of instructional design based on the ADDIE model on developing EFL learners’ writing performance. English Language Teaching, 14(2), 20–30. https://doi.org/10.5539/elt.v14n2p20

Belz, J. A., & Vyatkina, N. (2008). Learner corpus analysis and the development of L2 pragmatic competence in networked intercultural language study. Canadian Modern Language Review, 64(4), 567–600. https://doi.org/10.3138/cmlr.64.4.567

Biber, D., Conrad, S., & Reppen, R. (1998). Corpus linguistics: Investigating language structure and use. Cambridge University Press. https://doi.org/10.1017/CBO9780511804489

Boyd, A., et al. (2014). The MERLIN corpus: Learner language and the CEFR. LREC 2014, 1281–1288.

Branch, R. M. (2009). Instructional design: The ADDIE approach. Springer. https://doi.org/10.1007/978-0-387-09506-6

Callies, M., & Götz, S. (Eds.). (2015). Learner corpora in language testing and assessment. John Benjamins. https://doi.org/10.1075/scl.70

Council of Europe. (2020). CEFR companion volume. Council of Europe Publishing.

Evert, S. (2008). A lexicographic evaluation of German adjective-noun collocations. LREC Workshop, 20–27.

Fellbaum, C., & Geyken, A. (2005). Corpus-based studies of German idioms and light verbs. International Journal of Lexicography, 19(4), 349–360. https://doi.org/10.1093/ijl/ecm030

Fulcher, G., & Davidson, F. (2007). Language testing and assessment: An advanced resource book. Routledge.

Furko, P., & Kecskes, I. (2021). The pragmatics of English as a lingua franca in intercultural communication. Intercultural Pragmatics, 18(4), 451–476. https://doi.org/10.1515/ip-2021-0009

Götz, S. (2013). Fluency in native and nonnative English speech. John Benjamins. https://doi.org/10.1075/scl.57

Hu, X., & Lu, X. (2021). Predicting CEFR levels in L2 oral speech. Language Testing, 38(3), 1–25. https://doi.org/10.1177/0265532220980423

Leńko-Szymańska, A., & Boulton, A. (Eds.). (2015). Multiple affordances of language corpora for data-driven learning. John Benjamins. https://doi.org/10.1075/scl.69

Lemmenmeier-Batinić, D. (2020). Lexical Explorer: Extending access to the Database for Spoken German for user-specific purposes. Corpora, 15(1), 55–76. https://doi.org/10.3366/cor.2020.0186

Moehrs, C., & Meliss, M. (2017). LeGeDe project: Spoken German lexical resources. Lexikographie und Wörterbuchforschung, 1(1), 45–60.

Molenda, M. (2003). In search of the elusive ADDIE model. Performance Improvement, 42(5), 34–36. https://doi.org/10.1002/pfi.4930420508

Norris, J. M., Brown, J. D., Hudson, T., & Yoshioka, J. (2002). Designing second language performance assessments. University of Hawai‘i Press.

Schmidt, T. (2014). The Database for Spoken German – DGD2. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14) (pp. 1451–1457).

Sharwood Smith, M. (2013). Rethinking the role of consciousness in second language learning. In VanPatten, B. & Benati, A. (Eds.), Key terms in second language acquisition (pp. 27–29). Bloomsbury.

Taguchi, N., & Roever, C. (2017). Second language pragmatics. Oxford University Press.

Timpe-Laughlin, V., Wain, J., & Schmidgall, J. (2015). Defining and operationalizing the construct of interactional competence. TOEFL Research Report Series, 2015(2), 1–57. https://doi.org/10.1002/ets2.12059

Tono, Y. (2012). Designing and implementing a corpus-based dictionary of collocations for learners of Japanese. International Journal of Lexicography, 25(4), 396–433. https://doi.org/10.1093/ijl/ecs017

Tracy-Ventura, N., & Huensch, A. (2022). Measuring fluency across tasks: What matters and what doesn’t. Language Testing, 39(2), 211–235. https://doi.org/10.1177/02655322211035121

Weber, T. (2018). Grammatik und Lernerkorpora: Eine korpusorientierte Untersuchung von Präpositionalphrasen im deutschen MERLIN-Korpus. In E. Fuß et al. (Eds.), Grammar and Corpora 2016 (pp. 415–424). Heidelberg University Publishing. https://doi.org/10.17885/heiup.361.c4714

Weiss, Z., & Meurers, D. (2019). Broad linguistic modeling is beneficial for German L2 proficiency assessment. In Widening the scope of learner corpus research (pp. 419–435). Presses Universitaires de Louvain.

Wisniewski, K. (2020). SLA developmental stages in the CEFR-related learner corpus MERLIN: Inversion and verb-end structures in German A2 and B1 learner texts. International Journal of Learner Corpus Research, 6(1), 1–17. https://doi.org/10.1075/ijlcr.18008.wis

Xi, X. (2017). Validity and validation in language testing. In E. Shohamy, I. G. Or, & S. May (Eds.), Language testing and assessment (pp. 1–14). Springer. https://doi.org/10.1007/978-3-319-02261-1_1