DiverXplorer: Stock Image Exploration via Diversity Adjustment for Graphic Design

Paper (arXiv) | DOI

Antonio Tejero-de-Pablos, Sichao Song, Naoto Ohsaka, Mayu Otani, Shin'ichi Satoh

Accepted at CHI EA 2026 (Extended Abstracts of the CHI Conference on Human Factors in Computing Systems), Barcelona, Spain.

Overview

Graphic designers explore large stock image collections during open-ended or early-stage design tasks, yet common tools (e.g., Adobe Stock, Pixta, Pinterest) emphasize relevance and similarity, limiting designers' ability to overview the design space or discover visual patterns.

DiverXplorer is an image exploration prototype that enables stepwise adjustment of diversity, allowing users to transition from diverse overviews to increasingly focused subsets during exploration. The system divides image exploration into multiple steps, where the displayed images are optimized to meet a diversity-to-similarity ratio that gradually shifts from diverse to similar.

Figure 1: DiverXplorer divides image exploration into multiple steps (three in the figure), for which the visualized images are optimized to meet a diversity:similarity ratio that gradually shifts from diverse to similar. The blue arrow represents the selected image on each step.

Approach

• Diversity control via DPP: We implement diversity control via determinantal point process (DPP)-based sampling, which selects subsets that are maximally spread out in a CLIP embedding space. By dynamically constraining the similarity distance of the sampled images, our technique enables explicit control over the tradeoff between diversity and relevance.
• Stepwise refinement: Exploration proceeds iteratively—users select an image at each step, which conditions the next subset. Diversity constraints are gradually relaxed according to an exponential decay schedule, resulting in increasingly similar image sets as exploration progresses.
• Diversity-similarity transitions: Rather than presenting a fixed ranking or similarity structure, DiverXplorer exposes diversity-similarity tradeoffs through interaction, treating diversity as an interactive dimension rather than a fixed hyperparameter.

User Study

We conducted a user study with 22 professional graphic designers (16 in the main study) comparing DiverXplorer to three baselines inspired by current tools: relevance-ranked scrolling (Scroll), diversified re-ranking with scrolling (Scroll+div), and stepwise similarity refinement via clustering (Clustering).

Figure 5: Ranking results. DiverXplorer was ranked 1st by more than 60% of the designers.

Key findings:

• Stepwise diversity control supports early-stage sensemaking and comparison of visual patterns.
• Diversity aids discovery and reduces backtracking, but becomes less desirable as exploration progresses.
• DiverXplorer was ranked 1st by more than 60% of the participating designers, becoming the preferred tool over all baselines.
• Designers valued the ability to transition between divergent (exploratory) and convergent (comparative) modes fluidly, which baseline tools did not support.

Quick Start

Prerequisites

• Python 3.9-3.12
• UV package manager (recommended)

Installation & Run

1. Clone and install:

   git clone https://github.com/CyberAgentAILab/DiverXplorer.git
   cd DiverXplorer
   uv sync

2. Run the demo:

   uv run python main.py

3. Open your browser:

   • Navigate to http://localhost:8080
   • Select "Mother's Day" from the dropdown
   • Start exploring!

Demo Features

Interactive Search Interface

• 4-step progressive exploration with exponential diversity decay
• Grid-based image selection (4x4 layout per step)
• Smart reranking that balances diversity and relevance
• Similarity convergence in final step for refined selection

Pre-loaded Dataset

This demo includes data for one of the four design tasks used in the user study:

Design task: A design to celebrate Mother's Day aimed at young mothers with the feeling of "spending happy times with their children"

Search query in Pixabay: mother's day child

• 50 free Pixabay images from the Mother's Day task
• CLIP embeddings (512-dimensional feature vectors)
• Pre-computed similarity scores between image features and the task description

Note: To keep the repository size manageable, this demo includes only 50 images. The user study in the paper used 600 images per task, where the progressive diversity narrow-down effect is much more pronounced. You can create your own dataset with more images to experience the full effect.

How It Works

Search Algorithm

The system uses progressive diversity-aware reranking:

1. Step 1-3: Diversity exploration with exponential decay

   • progress = exp(-decay_rate * step)
   • score = w_d * diversity + w_r * relevance

2. Step 4: Similarity convergence

   • Focus on images similar to your selection
   • Refined results for final choice

Key Parameters

• Steps: 4 total exploration steps
• Diversity weight: 1.0 (balances exploration)
• Relevance weight: 12.0 (emphasizes query match)
• Decay rate: 8.0 (exponential diversity reduction)
• Grid size: 4x4 images per step

Project Structure

├── main.py                     # Main demo application
├── diversity_search.py         # Diversity reranking algorithms
├── diversity_metrics.py        # Diversity measurement functions
├── data_loader.py              # Data loading utilities
├── extract_features.py         # CLIP feature extraction for new images
├── text_search.py              # Text-to-image similarity scoring
├── pyproject.toml              # Dependencies (UV format)
├── uv.lock                     # Lockfile for reproducible builds
├── images/
│   └── mothers_day/            # 50 demo images
├── embeddings/
│   └── mothers_day/clip/       # CLIP embeddings + mapping
└── config/
    └── scores/mothers_day/     # Similarity scores + indices

Technical Details

Dependencies

• PyWebIO: Web interface framework
• PyTorch: Deep learning framework
• PIL/Pillow: Image processing
• NumPy: Numerical computations

Data Format

• Images: JPG/PNG format, 640px resolution
• Embeddings: PyTorch tensors (.pt files)
• Scores: NumPy arrays (.npy files)
• Metadata: JSON configuration files

Usage Tips

1. Start Broad: In early steps, select images that represent different styles/themes
2. Explore Diversity: The algorithm will show you varied options to expand your search
3. Refine Selection: Later steps focus on similarity to help narrow down choices
4. Navigation: Use "Back" and "To top" buttons to revisit previous steps
5. Final Choice: Click "Choose this" when you find the perfect image

Customization

Adding Your Own Images

To use your own dataset:

1. Place your images in ./images/<class_name>/ (e.g., ./images/mothers_day/)

2. Extract CLIP embeddings:

   uv run python extract_features.py --class-name <class_name>  # e.g., mothers_day

3. Compute similarity scores for your search query:

   uv run python text_search.py --query "your search query" --class-name <class_name>

4. Add the new task to the dropdown and query mappings in main.py

Adjusting Search Parameters

Modify these variables in main.py (lines 385-392):

• S: Number of exploration steps
• w_d: Diversity weight
• w_r: Relevance weight
• decay_rate: Diversity decay speed
• N, M_values: Grid dimensions

Citation

If you use this work, please cite:

@inproceedings{tejero2026diverxplorer,
  title={DiverXplorer: Stock Image Exploration via Diversity Adjustment for Graphic Design},
  author={Tejero-de-Pablos, Antonio and Song, Sichao and Ohsaka, Naoto and Otani, Mayu and Satoh, Shin'ichi},
  booktitle={Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA '26)},
  year={2026},
  publisher={ACM},
  doi={10.1145/3772363.3799327}
}

License

This project is part of CyberAgent AI Lab research. Please refer to the LICENSE file for usage terms.

Acknowledgments

• OpenAI CLIP: For robust image-text embeddings
• PyWebIO: For the web interface framework
• CyberAgent AI Lab: For supporting this research