Introduction

This post is based on the original publication [1].

Background

Design of experiments (DoE) gives designers a guidance to define an adequate set of tests. Defining tests to evaluate a system comprehensively is an important step in a system design. As a complimentary to DoE, we would like to show a way to enumerate similarity of system's responses during these tests and restructure tests with respect to the similarity.

Idea

As a first step, we run a lot of simulations and monitor some selected signals. This becomes training data.

Data is fed to a variational auto-encoder (VAE) to learn both reduced representation of signals and reconstruction of inputs from the reduced representation.

When VAE is trained successfully, we have following tools for our test design:

Encoder to convert high dimensional inputs to low dimensional representation
If you feed two signals to Encoder, they will become two points in the low dimensional space. The distance between these two points gives a similarity of two signals by means of features that VAE learned.
Cost function that gives a high value when a give signal is not alike training data set.
By feeding any new signal to Encoder, the cost function tells if the new signal is alike training signals or not.

With these two tools, we can try to search for test cases which are not covered well in the original tests, and will be able to improve test quality by adding these tests and eliminate redundant tests.

Success of this process does depend on the coverage of the original tests. This is in a way alike our learning process where we depend on our experiences as a clue to understand new concepts.

Example

1. Simple Hydraulic arm with three points to be monitored

Three hydraulic pressure will be monitored during tests.

2. Test patterns

Three types of test patterns are defined simply with respect to the arm move.

Test outputs look like:

3. VAE network training

A VAE network was trained on these hydraulic pressure signals.

4. Encoding test outputs

After training the VAE, all tests defined at 2 are encoded to four dimensional latent variables. Visual representation of the encoded signals are given below (3 components of 4-D data is shown):

Purple Green, and Yellow color represent points corresponding encoded outputs from Periodic, Down, and UP test patterns.

5. Test reduction

Test cases result in similar latent variables may also be redundant since we would like to find tests stimulate systems to result in wide variety of states. If we set a threshold value of distance and eliminate similar tests, it result in a reduced test set. In the figure below, points are reduced if the distance in the 4-D latent space is less than 0.5.

References:

[1] Yasunori Yokojima and Toshihiko Nakazawa, Applying deep learning to test design process in hydraulic systems design 深層学習を応用した油圧システム設計におけるテスト設計プロセス. Journal of the Japan Fluid Power System Society 49, 71-74, 2018.