Open Domain QA with Conflicting Contexts

25% of unambiguous, open domain questions can lead to conflicting contexts when retrieved using Google Search.

We study how open-domain question answering systems handle conflicting contexts retrieved from the web. Our findings reveal that a surprisingly large fraction of straightforward questions lead to conflicting evidence when using real search engines.