Rancang Bangun Virtual Boiler Assistant Berbasis RAG Menggunakan Large Language Model untuk Taruna Politeknik Pelayaran

Dirhamsyah Dirhamsyah; Tika Septia

doi:10.55681/sentri.v5i7.6992

Authors

Dirhamsyah Dirhamsyah Program Studi Teknologi Teknik Mesin, Politeknik Pelayaran Surabaya, Jawa Timur, Indonesia
Tika Septia Program Studi Teknologi Teknik Mesin, Politeknik Pelayaran Surabaya, Jawa Timur, Indonesia

DOI:

https://doi.org/10.55681/sentri.v5i7.6992

Keywords:

RAG, Virtual Assistant, Boiler, NLP, Maritime Education

Abstract

This study presents the design and development of a Retrieval-Augmented Generation (RAG)-based Virtual Boiler Assistant (VBA) integrated with a Large Language Model (LLM) to support boiler-system learning for cadets at Politeknik Pelayaran Surabaya. The system uses FastAPI as the backend framework, ChromaDB as a local vector database, Gemini 2.5 Flash as the generative LLM, and React as the frontend interface. A total of 192 text chunks were extracted from four official training documents used in the steam plant laboratory and Computer-Based Training (CBT) simulator. For each query, the RAG pipeline retrieves the four most semantically relevant chunks using cosine similarity over Gemini embeddings, then injects that context into the LLM prompt to produce accurate responses with source citations. The system was deployed on a VPS with Server-Sent Events (SSE) streaming, achieving an average response time of 15 seconds and an average semantic-similarity score of 0.73. Accuracy was measured using 20 validated ground-truth questions covering four domains: startup procedures, fuel systems, safety mechanisms, and monitoring software. Each response was assessed with a three-point rubric: 0 for incorrect answers, 1 for partially correct answers, and 2 for fully correct answers, all compared against reference answers derived directly from institutional teaching documents. The system achieved 30 of 40 points, or 75%, exceeding the minimum threshold of 70% set at the start of the study. These results indicate that a RAG-based virtual assistant, at least at this testing scale, can serve as a practical supplementary learning tool for cadets in maritime technical training.

Downloads

Download data is not yet available.

References

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., ... & Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877-1901.

Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. IEEE Access, 8, 75264-75278. https://doi.org/10.1109/ACCESS.2020.2988510

Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL-HLT, 4171-4186.

Gao, Y., Xiong, Y., Gong, X., Jia, W., Wang, J., Bi, Y., ... & Wang, H. (2024). Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997.

Hafizh, M. (2024). Pengembangan chatbot menggunakan metode SEMMA dan model Transformer untuk layanan akademik. Jurnal Teknik Informatika, 12(3), 45-56.

Hamzani, A., et al. (2024). Implementasi chatbot berbasis AI dan NLP untuk proses penerimaan mahasiswa baru. Jurnal Pendidikan Teknologi Informasi, 8(2), 112-125.

International Maritime Organization. (2017). International Convention on Standards of Training, Certification and Watchkeeping for Seafarers (STCW). IMO Publishing.

Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., ... & Yih, W. T. (2020). Dense passage retrieval for open-domain question answering. Proceedings of EMNLP, 6769-6781.

Kasneci, E., Sessler, K., Kuchemann, S., Bannert, M., Dementieva, D., Fischer, F., ... & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., ... & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33, 9459-9474.

Marine Insight Academy. (2025). Maritime AI Assistant for COC preparation. https://www.marineinsight.com

Nye, B. D., Graesser, A. C., & Hu, X. (2014). AutoTutor and family: A review of 17 years of natural language tutoring. International Journal of Artificial Intelligence in Education, 24(4), 427-469. https://doi.org/10.1007/s40593-014-0029-5

Primo Nautic. (2024). Maritime GPT: AI model for maritime education. Maritime Technology Review, 15(1), 22-31.

PT Tecno Multi Utama. (n.d.). Petunjuk pengoperasian monitoring boiler: Laboratorium steam plant. Politeknik Pelayaran Surabaya.

Puspitasari, D., et al. (2024). Pengembangan chatbot website menggunakan NLP dengan algoritma Neural Network. Jurnal Informatika, 11(1), 78-89.

Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using Siamese BERT-networks. Proceedings of EMNLP-IJCNLP, 3982-3992.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30, 5998-6008.

Wahyuwono, A., Oktavia, R., & Kartikasari, D. (2024). Virtual assistant berbasis web menggunakan NLP dan LSTM untuk layanan informasi akademik. Jurnal Teknologi Pendidikan, 9(2), 201-215.

Woodruff, E. B., Lammers, H. B., & Lammers, T. F. (2017). Steam plant operation (10th ed.). McGraw-Hill Education.

Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., ... & Wen, J. R. (2023). A survey of large language models. arXiv preprint arXiv:2303.18223.

Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023). Revolutionizing education with AI: Exploring the transformative potential of ChatGPT. Contemporary Educational Technology, 15(3), Article ep429. https://doi.org/10.30935/cedtech/13152

Albadarin, Y., Saqr, M., Pope, N., & Tukiainen, M. (2024). A systematic literature review of empirical research on ChatGPT in education. Discover Education, 3, Article 60. https://doi.org/10.1007/s44217-024-00138-2

Ali, D., Fatemi, Y., Boskabadi, E., Nikfar, M., Ugwuoke, J., & Ali, H. (2024). ChatGPT in teaching and learning: A systematic review. Education Sciences, 14(6), Article 643. https://doi.org/10.3390/educsci14060643

Asai, A., Wu, Z., Wang, Y., Sil, A., & Hajishirzi, H. (2024). Self-RAG: Learning to retrieve, generate, and critique through self-reflection. In Proceedings of the 12th International Conference on Learning Representations. https://openreview.net/forum?id=hSyW5go0v8

Borgeaud, S., Mensch, A., Hoffmann, J., Cai, T., Rutherford, E., Millican, K., et al. (2022). Improving language models by retrieving from trillions of tokens. In Proceedings of the 39th International Conference on Machine Learning (pp. 2206-2240). PMLR.

Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., et al. (2023). Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv preprint arXiv:2303.12712. https://arxiv.org/abs/2303.12712

Chan, C. K. Y., & Hu, W. (2023). Students' voices on generative AI: Perceptions, benefits, and challenges in higher education. International Journal of Educational Technology in Higher Education, 20, Article 43. https://doi.org/10.1186/s41239-023-00411-8

Crompton, H., & Burke, D. (2023). Artificial intelligence in higher education: The state of the field. International Journal of Educational Technology in Higher Education, 20, Article 22. https://doi.org/10.1186/s41239-023-00392-8

Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., et al. (2024). The Llama 3 herd of models. arXiv preprint arXiv:2407.21783. https://arxiv.org/abs/2407.21783

Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., et al. (2023). So what if ChatGPT wrote it? Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI. International Journal of Information Management, 71, Article 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642

Es, S., James, J., Espinosa-Anke, L., & Schockaert, S. (2024). RAGAS: Automated evaluation of retrieval augmented generation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations (pp. 150-158). https://aclanthology.org/2024.eacl-demo.16/

Feuerriegel, S., Hartmann, J., Janiesch, C., & Zschech, P. (2024). Generative AI. Business & Information Systems Engineering, 66, 111-126. https://doi.org/10.1007/s12599-023-00834-7

Google DeepMind, Anil, R., Borgeaud, S., Alayrac, J. B., Yu, J., Soricut, R., et al. (2024). Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. https://arxiv.org/abs/2312.11805

Guizani, S., Mazhar, T., Shahzad, T., Ahmad, W., Bibi, A., & Hamam, H. (2025). A systematic literature review to implement large language model in higher education: Issues and solutions. Discover Education, 4, Article 35. https://doi.org/10.1007/s44217-025-00424-7

Guu, K., Lee, K., Tung, Z., Pasupat, P., & Chang, M. W. (2020). REALM: Retrieval-augmented language model pre-training. In Proceedings of the 37th International Conference on Machine Learning (pp. 3929-3938). PMLR.

Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., & Liu, T. (2025). A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43(2), Article 42. https://doi.org/10.1145/3703155

Izacard, G., & Grave, E. (2021). Leveraging passage retrieval with generative models for open domain question answering. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (pp. 874-880). https://doi.org/10.18653/v1/2021.eacl-main.74

Izacard, G., Lewis, P., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., et al. (2023). Atlas: Few-shot learning with retrieval augmented language models. Journal of Machine Learning Research, 24(251), 1-43.

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), Article 248. https://doi.org/10.1145/3571730

Li, J., Cheng, X., Zhao, W. X., Nie, J. Y., & Wen, J. R. (2023). HaluEval: A large-scale hallucination evaluation benchmark for large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 6449-6464). https://doi.org/10.18653/v1/2023.emnlp-main.397

Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), Article 410. https://doi.org/10.3390/educsci13040410

Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D., & Hajishirzi, H. (2023). When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (pp. 9802-9822). https://doi.org/10.18653/v1/2023.acl-long.546

Mialon, G., Dessi, R., Lomeli, M., Nalmpantis, C., Pasunuru, R., Raileanu, R., et al. (2023). Augmented language models: A survey. Transactions on Machine Learning Research. https://openreview.net/forum?id=jh7wH2AzKK

Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W. T., Koh, P. W., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023). FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 12076-12100). https://doi.org/10.18653/v1/2023.emnlp-main.741

Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., et al. (2021). WebGPT: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332. https://arxiv.org/abs/2112.09332

OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., et al. (2023). GPT-4 technical report. arXiv preprint arXiv:2303.08774. https://arxiv.org/abs/2303.08774

Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., et al. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35, 27730-27744.

Ram, O., Levine, Y., Dalmedigos, I., Muhlgay, D., Shashua, A., Leyton-Brown, K., & Shoham, Y. (2023). In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11, 1316-1331. https://doi.org/10.1162/tacl_a_00605

Shuster, K., Poff, S., Chen, M., Kiela, D., & Weston, J. (2021). Retrieval augmentation reduces hallucination in conversation. In Findings of the Association for Computational Linguistics: EMNLP 2021 (pp. 3784-3803). https://doi.org/10.18653/v1/2021.findings-emnlp.320

Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10, Article 15. https://doi.org/10.1186/s40561-023-00237-x

Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M. A., Lacroix, T., et al. (2023). LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. https://arxiv.org/abs/2302.13971

UNESCO. (2023). Guidance for generative AI in education and research. United Nations Educational, Scientific and Cultural Organization. https://unesdoc.unesco.org/ark:/48223/pf0000386693

Walter, Y. (2024). Embracing the future of artificial intelligence in the classroom: The relevance of AI literacy, prompt engineering, and critical thinking in modern education. International Journal of Educational Technology in Higher Education, 21, Article 15. https://doi.org/10.1186/s41239-024-00448-3

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., et al. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35, 24824-24837.

Yan, L., Sha, L., Zhao, L., Li, Y., Martinez-Maldonado, R., Chen, G., et al. (2024). Practical and ethical challenges of large language models in education: A systematic scoping review. British Journal of Educational Technology, 55(1), 90-112. https://doi.org/10.1111/bjet.13370

Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., et al. (2023). Siren's song in the AI ocean: A survey on hallucination in large language models. arXiv preprint arXiv:2309.01219. https://arxiv.org/abs/2309.01219