Development and characterization of large language models for knowledge retrieval and safety measure generation in construction safety management

초록

This study investigates how different Large Language Model (LLM) fine-tuning strategies perform on two representative cognitive functions within the construction-safety decision process that LLM-based systems can directly support: (1) retrieving relevant regulatory safety guidelines and (2) generating tailored accident prevention measures. Three domain-adapted LLM configurations-LoRA-based (LM1), full-parameter (LM2), and layer-swapped (LM3)-are developed using a dataset of 11,172 entries from safety guideline documents and are compared against a Retrieval-Augmented Generation (RAG) system and GPT-4. Their performance is evaluated using both general benchmarks (Open Ko-LLM Leaderboard, multiple-choice accuracy) and task-specific similarity metrics (cosine similarity, BLEU, ROUGE). The results show that the RAG system achieves the highest knowledge retrieval accuracy (70.83%), while LM3 generates the most contextually relevant and lexically aligned preventive measures. By situating the evaluation within these two cognitive functions, the study provides a systematic comparison of fine-tuning strategies for LLMs in construction safety and offers empirical insights into how different approaches complement one another in supporting safety workflows.

키워드