When asked ‘Which US state has a city served by Contour Airlines that features a tall metal arch on a river?’, you would have to combine visual information with texts and tables. Quite a challenge, requiring multimodal analysis. Our method EDMA addresses exactly this. The paper is accepted for NeurIPS 2026!

EDMA first finds the photo showing the arch in St. Louis, then uses that to pick the right row from the airline’s destination table, relates St. Louis to its state, and answers ‘Missouri’. It also reports that the photo did most of the work, cutting the remaining uncertainty by more than half, so a user can see exactly why the answer is what it is.

The method answers complex questions by combining different kinds of information at once, such as images, tables and text reports, and, unlike current AI systems, it shows step by step which source contributed how much to the answer. It also comes with a built-in confidence signal: when the method is less certain, the answer is measurably more likely to be wrong, so the system can flag when a human should take a closer look. On two challenging benchmarks, EDMA gave clearly better answers than the best existing systems, such as chain-of-thought, by roughly 20 points on the standard score, and it works with a wide range of AI models, including open-source ones that can run on our own infrastructure.

To answer complex questions, it applies Multimodal LLMs (MLLMs) by iterative logical-entropy reduction. It extracts answer requirements, constructs modality-specific partitions, performs cross-modal conditioning, enumerates answering trajectories, and selects the trajectory with the lowest terminal entropy. Experiments on MultiModalQA and ManyModalQA report average gains of 12.8 F1 over the strongest prompting baseline and 19.8 F1 over prior systems. The core idea is interesting and reasonably interpretable: logical entropy provides a model-agnostic way to quantify how different modalities narrow the candidate space, without relying on calibrated model probabilities. The paper also goes beyond aggregate accuracy by examining entropy–error correlations and alternative trajectory choices.

EDMA: Entropy-Driven Multimodal Answering. Emanuele Mezzi, Gertjan Burghouts, Fabio Massacci, Mengyuan Zhang. NeurIPS 2026.