# Reviewer kUm8
## Strengths:
The paper is clearly written and easy to follow.
The theoretical analysis of PoolDPP is a strong part of the paper.
The evaluation is reasonably diverse, makes the empirical claims more convincing than evaluation on a single task type.
## Weaknesses:
1. The method is limited to post-hoc selection from a fixed pool, does not provide a mechanism for steering generation toward new regions of the output space. This limits its practical usefulness. The paper would benefit from a more explicit discussion of when this selection-based setup is practically preferable to directly modifying the sampler.
2. It is unclear whether embedding-space diversity corresponds to meaningful solution diversity. The DPP objective encourages diversity through the determinant of an embedding-based kernel. This is natural for semantic diversity, but may be less reliable for structured reasoning tasks such as math and code. In these domains, small algorithmic changes can completely alter correctness, while surface-level or syntactic changes may leave the actual solution strategy unchanged. Conversely, two solutions may be algorithmically very different while appearing close under a sentence embedding model. This raises the question of whether the selected samples are genuinely diverse in reasoning structure or merely diverse in wording. The paper should analyze how sensitive Qwen3-Embedding-0.6B is to small but correctness-critical changes, such as changing one operation in a math derivation or modifying one line in a program. Without such analysis, it is hard to tell whether PoolDPP captures algorithmic diversity or only textual/semantic variation.
3. The usefulness of dual sampling is not fully clear in the reported setting. The paper motivates dual sampling as a way to reduce the complexity of DPP inference in large-pool settings. However, in the experiments, the embedding dimension is 1024, while the candidate pool size appears to be around (N=400). The paper should clarify in which regimes dual sampling provides a practical advantage, and report wall-clock or scaling comparisons showing when it becomes beneficial.
## Questions:
Q1: W2.
Q2: W3.
Q3: could output length itself be a strong proxy for selection quality in the current setup?

# Response to Reviewer kUm8

Thank you for the constructive and thorough review. We also appreciate the recognition of the theoretical contribution and our efforts to provide a comprehensive evaluation. Below, we address each of your points and present new experiments to support our responses.

## W1. Post-hoc selection only, no mechanism to steer generation

We agree that steering methods are more broadly applicable, and we are already exploring an autoregressive method that uses the DPP objective, usable to boost diverse rollouts during GRPO training or for test-time exploration. However, characterizing the diversity model correctly and measuring its upper bound was the necessary first step: a steering method needs a target to approximate, and defining that target is this paper's main contribution.

There are also potential applications where post-hoc selection is preferable, and we will include the discussion in the revision. For example, when evaluating black-box or API-only LMs, where no steering is possible but we still want a glimpse of the available range of outputs. Another example is within Best-of-K evaluation, where it is cheap to plug the DPP objective instead of the usual scorer. This applies to all evaluation methods where a pool is already present: self-consistency, verifier reranking, or MBR have already paid for the pool.

Existing decoding-time methods do not trace the quality-diversity frontier well, precisely because it is unclear what the diversity target should be. We added Stochastic Beam Search, Conditional Poisson SBS, and Determinantal Beam Search, among the submitted baselines, and found that they all fall short of the Pareto frontier (see answer to reviewer ZHAe).

## W2 / Q1. Does embedding-space diversity correspond to solution diversity? Sensitivity of embeddings to correctness-critical edits

Thank you for pointing out a relevant metric that we did not test and should be included. To evaluate the sensitivity of the embedding to correctness-critical edits, we created a collection of edits in GSM8K and Ambrosia and measured whether the edits changed the final answer or the execution result. We then measured the distance in embedding space between the original and edited items, to test whether the correctness-critical edit is further away than a surface-only edit.

SQL edits are labeled by execution: a "semantic" edit is kept only if the executed result signature changes, and a "surface" edit only if the executed result is byte-identical. GSM8K "surface" edits are verified to preserve both the final answer and the multiset of stated (a, op, b, c) arithmetic facts, and the correctness-critical edits are of exactly the kind you name, one operation in a derivation and (as a minimal control) the single token that states the answer.

The table shows the P(distance of the correctness-critical edit > distance of the surface-only edit). Critical changes like flipping GSM8K's answer token require more change in the embedding space, as expected, but we see how **better embedders increasingly separate semantics from surface-level changes**. 

| embedder | GSM8K, flip the answer token | GSM8K, flip semantic operator | SQL, semantic edit |
|---|---|---|---|
| MiniLM-L6-v2 *(the Ambrosia kernel)* | 0.004 [0.001, 0.009] | 0.106 [0.088, 0.125] | 0.965 [0.922, 1.000] |
| Qwen3-Embedding-0.6B *(submitted GSM8K kernel)* | 0.042 [0.033, 0.053] | 0.318 [0.288, 0.347] | 0.760 [0.729, 0.789] |
| Qwen3-Embedding-0.6B (instructed) | 0.166 [0.144, 0.188] | 0.817 [0.793, 0.840] | 0.894 [0.869, 0.918] |

n = 572 / 538 / 470 pairs, bootstrap 95% CIs. 

We approached the experiments as a validation, not as a extensive tune for each domain. After submission, realized that simple improvements to the embedder really influences the quality of the selected sets. As seen in the instructed row, modern embedders can be instructed in one line to focus on the relevant semantic features, which improves substantially the marginal gains with respect to the submitted GSM8K kernel.

The encoder is flexible beyond instructions, and structured tasks are a good example. Any property you can extract (the executed behavior of a program or the arithmetic steps of a derivation) can be one-hot embedded and appended to the neural embedding, and the determinant reads it as an explicit diversity signal. On GSM8K, one-hot dimensions for each chain's extracted final answer recover essentially the whole pool ceiling (pass@K 0.972 of 0.974, +4.3 points, p = 0.004 question-paired). On Ambrosia, appending execution signatures to the text embedding raises the coverage gain over ancestral from +2.4 to +6.2 points (p = 1e-5). We did not include these results since they are not general-purpose, but they address the point that structured information can be very easily incorporated into the kernel.

We will state the submitted GSM8K result as a limitation of the supplied kernel, report the probe as the diagnostic for choosing one, and update the GSM8K figure to the instructed kernel.

## W3 / Q2. Usefulness of dual sampling at D=1024, N=400

The dual representation earns its place at large pool sizes. The pool-size ablation shows the kernel objective improving monotonically with N, so during development the method was run on the largest feasible pools, N = 10^4, where the dual is the only practical representation. Measured wall-clock for exact k-DPP sampling (K=10, single thread):

| D | N | dual (ms) | primal eigendecomposition (ms) |
|---|---|---|---|
| 384 | 400 | 26 | 16 |
| 384 | 2,000 | 35 | 790 |
| 384 | 10,000 | **145** | **115,044** |
| 1024 | 400 | 200 | 17 |
| 1024 | 10,000 | **559** | **115,550** |

At N = 10^4 the dual is a 790x speedup. At reduced pool sizes (N = 100 to 400) your cost analysis is correct and the primal is cheaper, but the dual has the advantage of being computanionally feasible at all pool sizes, while D rarely exceeds a few thousand. However, it is a fair point that we should explicitly discuss the tradeoff and the regimes where each representation is preferable, so we thank you for raising this issue and will add it to the revision. 

## Q3. p(y) is not length-normalized, is length a proxy for selection quality?

We confirm p(y) is the raw and principled sequence probability, with no length normalization applied, so log p is coupled to length (pool-level Pearson r from -0.37 at Ambrosia sT=1.0, -0.65 on GSM8K at sT=1.2). However, we also confirm that log p is not a length proxy: the correlation is a property of a degenerate long tail, not of the score. Stratifying each prompt's pool by length:

| setting | quality by length quintile (short -> long) |
|---|---|
| GSM8K sT=1.2, fraction correct | 0.65, 0.71, 0.73, 0.70, 0.34 |
| Ambrosia sT=1.5, fraction executable | 0.72, 0.68, 0.49, 0.19, **0.008** |

In the hottest Ambrosia pools, 99% of the longest quintile fails to execute at all; log p correctly downweights it. Within the shortest half of every pool the length coupling nearly vanishes (Spearman rho: -0.33 to +0.01 on GSM8K). So log p penalizes a genuinely low-quality long tail and retains its quality signal when length variation is removed. This effect of longest generations being low-quality is also seen in the post-training literature during rollout generation.