Skip to main content

Intelligent Test Automation and Optimization

We let learning-based agents operate a system directly to find defects, and use execution traces and learning data to infer why a defect happened, cutting the cost of designing tests by hand.

Background​

Designing test cases by hand does not scale once the input space opens up. In a game, which states are reachable depends on the order of the player's actions, so the state space is effectively unbounded, and a scripted automation only ever walks the paths its author imagined.

Grading programming exercises hits the same wall. Comparing only the final output leaves no room for partial credit, and it says nothing about where or why the learner went wrong. What the two problems share is that judgement requires looking at the process, not at a single answer.

Approach​

Reinforcement Learning Playtesting​

An agent plays the game itself and looks for defects. The core of it is rewarding error detection so that exploration drifts toward defects, and every defect found is recorded as a bug report together with the input-sequence log needed to reproduce it. We implemented a tool for Unity platformer games.

Automatic Grading from Execution Logs​

Instead of comparing output, we log the order of function calls and their arguments and judge from that. Courses like Arduino practice, where results only appear if the hardware is present, collect their logs in a virtual execution environment. We widen grading coverage with concolic testing and follow the control flow of test cases to report where a submission failed. The exercise problems themselves are generated automatically from mutants.

Predicting Learner Error Causes​

Rather than looking only at submitted code, we take the learner's cognitive level as an additional input to predict the cause of an error. The point is to estimate why it was written that way, not to point at what to fix.

Applied AI Models and Their Verification​

We do not work only on testing techniques. We also build the AI models themselves on domain data: photovoltaic output prediction and panel fault detection, waste plastic sorting, and drawing-based psychological analysis.

Verification follows from that. When training data is imbalanced or distorted during augmentation, an accuracy figure stops representing model quality. We separate augmentation from validation so that data quality can be assessed on its own.

Selected Records​

The list below is a selection. Some records sit on the boundary between areas, so it is not exhaustive; the full list is in the Archive. Titles and author names are kept in their original language.

  • Predicting Rough Error Causes in Novice Programmers Using Cognitive LevelInternational ConferenceCounted as journal

    Deok Yeop Kim, Woo Jin Lee

    pp. 341-350, 2024

    DOI ↗

  • 강화학습을 이용한 게임 테스팅 자동화 도구MS Thesis

    박세찬

    경북대학교, 2024

    Full text ↗

  • 피지컬 컴퓨팅 교육을 위한 함수 실행 로그 기반 자동 채점 시스템PhD Thesis

    서강복

    경북대학교, 2024

    Full text ↗

  • UnityPGTA: A Unity Platformer Game Testing Automation Tool Using Reinforcement LearningDomestic Journal

    Se-chan Park, Deock-Yeop Kim, Woo-Jin Lee

    Journal of KIISE, Vol. 51, No. 2, pp. 149-156, 2024-2-29

    DOI ↗

  • A Convergence Study on the Image Augmentation and Validation Method for Quality Improvement of Image-Based AI ModelDomestic Journal

    Jung Min Seo, Woo Jin Lee

    The Korean Society of Science & Art, Vol. 41, No. 4, pp. 205-215, 2023-9-30

    DOI ↗

  • Arduino practice judgment system based on function execution log in virtual execution environmentInternational Journal

    Kangbok Seo, Junhyung Kim, Woojin Lee

    Computer Applications in Engineering Education, Vol. 32, No. 2, 2023-11-2

    DOI ↗

  • Function Execution Log Based Judgment System for Arduino Learning PracticeInternational ConferenceCounted as journal

    Kangbok Seo, Woojin Lee

    pp. 17-23, 2022

    DOI ↗

  • 시뮬레이터를 이용한 임베디드 소프트웨어 자동 채점 시스템 및 방법Patent GrantedDomestic

    이우진, 김덕엽, 정원웅, 서강복

    Granted 10-2373133 (2022-03-14)

    Full text ↗

  • A Study on Improvement of Scoring in Programming Practice Questions Using Concolic Testing TechniqueDomestic Journal

    Kangbok Seo, Sunghee Lee, Deokyeop Kim, Woojin Lee

    Journal of KIISE, Vol. 46, No. 11, pp. 1133-1139, 2019-11-30

    DOI ↗

  • 프로그래밍 훈련 지원을 위한 테스트케이스의 제어흐름에 기반한 프로그래밍 실패 피드백 시스템 설계Domestic Journal

    이성희, 김덕엽, 서강복, 이우진

    정보처리학회논문지. 소프트웨어 및 데이터공학, Vol. 8, No. 8, pp. 317-322, 2019-08

    DOI ↗

  • 소스코드 수행 실습문항의 뮤턴트 기반 자동생성 기법MS Thesis

    곽용섭

    경북대학교, 2017

    Full text ↗

  • 뮤턴트 기반의 문항 생성 장치 및 방법 (Mutant-based Test Problem Generating Apparatus and Method thereof)Patent FiledDomestic

    곽용섭, 이성희, 이우진

    Filed 10-2017-0061824 (2017-05-18)

    Full text ↗

  • 프로그래밍 학습을 위한 뮤턴트 기반의 실습 문항 생성기의 구조 설계Domestic Conference

    곽용섭, 이성희, 이우진

    한국정보처리학회 2017년 춘계학술발표대회, pp. 649-652, 2017-04

    DOI ↗

  • Development of a System for Predicting Photovoltaic Power Generation and Detecting Defects Using Machine LearningDomestic Journal

    Seungmin Lee, Woo Jin Lee

    KIPS Transactions on Computer and Communication Systems, Vol. 5, No. 10, pp. 353-360, 2016-10-31

    DOI ↗

  • 고품위 재활용 플라스틱 소재 확보를 위한 AI 활용 선별률 97% 및 선별용량 30 ton/day 이상의 폐플라스틱 고효율 선별공정 개발Research Project

    2025-06 ~ 2028-12

  • 머신러닝 기반의 그림 심리분석 인공지능 시스템 개발Research Project

    2017-05 ~ 2018-05