توليد خودكار آزمون‌هاي توسعه رفتار محور با استفاده از مدل‌هاي زباني و گراف‌ دانش

Automatic generation of behavior-based development tests using language models an‎d knowledge graphs

مهندسي نرم افزار

Abstract With the rapid expansion of software systems an‎d the increasing complexity of development processes, software testing has emerged as a key stage in ensuring system quality an‎d stability. Among modern testing approaches, the automatic generation of behavioral tests from natural language scenarios (such as Gherkin) plays a significant role in increasing productivity an‎d reducing human error. However, existing methods for automated test generation within the Behavior-Driven Development (BDD) framework still face challenges, including incorrect interpretation of textual requirements, heavy dependence on predefined rules, an‎d an inability to understan‎d complex semantic relationships among scenario components. To address these limitations, this research proposes an innovative framework based on the integration of Knowledge Graphs (KGs) an‎d Large Language Models (LLMs), capable of automatically transforming Gherkin scenarios into behavioral test code within the Behave framework. In this approach, Gherkin scenarios are first converted through a multi-stage process into a structured knowledge graph in order to conceptually represent entities, actions, an‎d relationships among them. The generated knowledge graph, together with the system source code an‎d the original scenario, is then provided to the language model to produce the final test code in the form of @Given, @When, an‎d @Then steps. In this study, four approaches for knowledge graph construction are examined, including a pro‎mp‎t-engineering-based approach using the Gemma-3-27B-IT model, an NLP-based approach using Rebel-large, a dynamic knowledge graph approach leveraging LangChain an‎d Neo4j for dynamic graph construction, an‎d a semantic embedding an‎d Graph Neural Network (GNN)-based approach. Experimental results demonstrate that integrating knowledge graphs with large language models improves the accuracy, coherence, an‎d interpretability of generated tests an‎d leads to stronger alignment between descriptive scenarios an‎d executable test code. The primary innovation of this research lies in presenting an intelligent framework for directly mapping natural language to executable test code through an intermediate knowledge graph layer, an integrated approach that has not previously been employed in this combined manner in related studies. This method represents an effective step toward automated software testing an‎d the development of explainable an‎d intelligent systems in the future.

6

158641