Tests the ability to find a probability distribution satisfying precise dual KL divergence constraints…
Tests the ability to find a probability distribution satisfying precise dual KL divergence constraints…: a task in terminal-bench-2-1-apptainer-v1: Terminal-Bench 2.1 (offline Apptainer, v1) (Harbor dataset). The confidence of a token prediction in an LLM can be quantified using different metrics…
The task
The confidence of a token prediction in an LLM can be quantified using different metrics. This implementation focuses on two metrics based on KL divergence from the uniform distribution: