Why VLMs say ‘yes’ to invisible objects—and how a Jacobian lens reveals the truth
Using Anthropic's J‑lens, the author probes LLaVA‑1.5‑7B’s internal visual workspace and finds the model silently knows an object is absent, yet answers “yes” to yes/no questions. Switching to a forced‑choice format restores accuracy, suggesting hallucinations stem from question framing rather than missing knowledge.