WHERE TO STEER · W2S

Where to Steer:
Input-Dependent Layer Selection for Steering Improves LLM Alignment

Soham Gadgil*, Chris Lin*, Su-In Lee

Department of Computer Science · University of Washington

* Equal contribution

Read the paper ↗Code ↗BibTeX ↓

The best layer to steer depends on the input.

W2S learns a mapping from prompt representations to intervention layers, adding an adaptive choice of where to apply a steering vector.

W2S framework: train a layer predictor using prompt embeddings and optimal-layer labels; at inference, predict a layer and inject a steering vector there.
W2S uses a frozen prompt encoder and a lightweight layer predictor to select an intervention layer for each input. Figure 2 from the paper.

Abstract

Steering vectors have emerged as a lightweight and effective approach for aligning large language models (LLMs) at inference time, enabling modulation of model behaviors by shifting intermediate representations toward a target behavior. Many existing steering methods apply interventions at globally fixed locations in an LLM, effectively treating the optimal intervention location as input-invariant. However, representations relevant to a target behavior can be encoded at different LLM layers depending on the input. Theoretically, we show that different inputs may require steering at different layers to achieve alignment with a given behavior. Empirically, we observe that the most effective layer for steering varies substantially between inputs. Motivated by these observations, we introduce Where to Steer (W2S), a general framework that adaptively selects intervention layers conditioned on the inputs by learning a mapping from input representations to steering layers. Across multiple LLMs and alignment behaviors, W2S consistently outperforms fixedlayer baselines in both in-distribution and outof-distribution settings. Our findings suggest that input-dependent layer selection is a key design dimension complementary to methods for steering vector extraction.

THE APPROACH

Learn where to intervene

01

Find training targets

Sweep across layers for each training prompt to identify the layer that maximizes steerability.

02

Learn a layer predictor

Encode each prompt with a frozen encoder, then train a shallow MLP to predict its optimal steering layer.

03

Steer each input

At inference time, predict a layer and apply the steering vector at that layer’s last-token representation.

Boxplots show gains in steerability from optimal layers and variation in optimal layer indices across 13 behaviors for Llama and Qwen.
Optimal intervention layers vary substantially across inputs. These oracle results motivate learning a predictor; they are distinct from W2S performance. Figure 1 from the paper.

EXPERIMENTS

Adaptive layer selection improves steering

In-distribution results averaged across 13 target behaviors. Values are means with 95% confidence intervals across five runs (Table 1).

Model / methodSteerability ↑Steerable examples ↑
Llama-2-7B-Chat
CAA · fixed layer1.259 ± 0.01475.4% ± 0.5%
CAA + W2S1.502 ± 0.01984.6% ± 1.2%
L2S · fixed layer2.098 ± 0.00989.9% ± 0.1%
L2S + W2S2.363 ± 0.05191.8% ± 1.1%
Qwen-1.5-14B-Chat
CAA · fixed layer1.493 ± 0.01183.3% ± 0.4%
CAA + W2S1.675 ± 0.01585.4% ± 0.4%
L2S · fixed layer1.888 ± 0.01587.5% ± 0.4%
L2S + W2S2.071 ± 0.03591.8% ± 1.0%

Steerability measures how the log-probability preference for the target behavior changes with steering strength. Positive values indicate steering in the intended direction.

Open-ended generation

We evaluate truthfulness on TruthfulQA and safety on PKU-SafeRLHF. The in-distribution results below report means with 95% confidence intervals across five runs.

Steerability for open-ended generation (Table 3).
Model / methodTruthfulness ↑Safety ↑
Llama-2-7B-Chat
CAA · fixed layer0.715 ± 0.092−0.045 ± 0.066
CAA + W2S0.830 ± 0.0430.857 ± 0.260
L2S · fixed layer0.886 ± 0.055−0.059 ± 0.119
L2S + W2S3.175 ± 0.1550.328 ± 0.102
Qwen-1.5-14B-Chat
CAA · fixed layer0.715 ± 0.0925.044 ± 0.124
CAA + W2S0.830 ± 0.0335.982 ± 0.217
L2S · fixed layer2.722 ± 0.3885.147 ± 0.421
L2S + W2S2.575 ± 0.4116.378 ± 0.151
Proportion of steerable examples for open-ended generation (Table 4).
Model / methodTruthfulness ↑Safety ↑
Llama-2-7B-Chat
CAA · fixed layer63.8% ± 1.8%47.7% ± 2.2%
CAA + W2S64.7% ± 0.7%55.9% ± 7.6%
L2S · fixed layer69.1% ± 1.2%43.6% ± 1.6%
L2S + W2S71.1% ± 1.5%47.3% ± 1.6%
Qwen-1.5-14B-Chat
CAA · fixed layer62.3% ± 1.0%66.7% ± 1.4%
CAA + W2S66.6% ± 5.6%69.0% ± 1.5%
L2S · fixed layer66.5% ± 3.1%68.8% ± 1.9%
L2S + W2S65.7% ± 2.9%69.2% ± 1.1%

W2S improves mean steerability in seven of eight model–method–task comparisons and changes negative mean safety steerability to positive values for both methods on Llama-2-7B-Chat. For L2S on Qwen truthfulness, both mean steerability and the proportion of steerable examples decrease; the confidence intervals overlap.

Beyond the training distribution

W2S improves aggregate performance under four controlled prompt perturbations. Refer to the paper for full out-of-distribution results, including open-ended generation.

Citation

Provisional citation; update with the official proceedings entry when available.

@misc{gadgil_where_to_steer,
  title = {Where to Steer: Input-Dependent Layer Selection for Steering Improves LLM Alignment},
  author = {Gadgil, Soham and Lin, Chris and Lee, Su-In},
  note = {Publication details to be added}
}