Generative models based on deep neural networks often have a high-dimensional latent space, ranging sometimes to a few hundred dimensions or even higher, which typically makes them hard for a user to explore directly. We propose differential subspace search to allow efficient iterative user exploration in such a space, without relying on domain- or data-specific assumptions. We develop a general framework to extract low-dimensional subspaces based on a local differential analysis of the generative model, such that a small change in such a subspace would provide enough change in the resulting data. We do so by applying singular value decomposition to the Jacobian of the generative model and forming a subspace with the desired dimensionality spanned by a given number of singular vectors stochastically selected on the basis of their singular values, to maintain ergodicity. We use our framework to present 1D subspaces to the user via a 1D slider interface. Starting from an initial location, the user finds a new candidate in the presented 1D subspace, which is in turn updated at the new candidate location. This process is repeated until no further improvement can be made. Numerical simulations show that our method can better optimize synthetic black-box objective functions than the alternatives that we tested. Furthermore, we conducted a user study using complex generative models and the results show that our method enables more efficient exploration of high-dimensional latent spaces than the alternatives.

Human-in-the-loop optimization, dimensionality reduction, generative models

We thank the anonymous reviewers for their insightful suggestions and discussions. This work was supported in part by a grant from JST CREST, JPMJCR17A1 (HCI for Machine Learning), Japan, a JSPS Grant-in-Aid for Scientific Research (A) 18H04106, Japan, and grants from the Ministry of Science and Technology of Taiwan, MOST-107- 2221-E-011-114-MY2 and MOST-107-2221-E-011-112-MY2.