Structure- and Semantic-based Rationale Distillation: Table and Chart Question Answering in Scientific Documents

초록

Visual structured information, such as tables and charts, is critical for interpreting scientific articles, where figures convey analyses and results. Effective understanding requires both structural reasoning (e.g., rows, columns, or axes) and semantic reasoning (e.g., identifying key findings or trends), often supported by captions and surrounding text. We propose Structure Semantic Rationale Distillation (SSRD), which employs two rationales for table and chart question answering: a structural rationale that linearizes data into text-based tables, and a semantic rationale that summarizes measures and findings. A compact student model conditions on the image, question, and context, and is trained with objectives over , , and using token-level cross entropy, structural consistency, and cosine alignment. On the SPIQA benchmark, training with a 27.8 % rationale-augmented subset shows that captions provide reliable cross-context gains, and SSRD improves performance over ANS-only and single-rationale baselines (e.g., BLEU 0.5532 to 0.5653, ROUGE-1 0.4368 to 0.4533, BERTScore 0.9057 to 0.9114). These results indicate that combining structural and semantic guidance enhances the comprehension of scientific structured data.

키워드