Materials Genome Engineering Advances Decision Sciences (miscellaneous)
DataEX‐Sc: A Specialized Parsing Model for Materials Science Scatter Plots Based on Multi‐Strategy Fusion and Bidirectional Optimization
Abstract
ABSTRACT Materials science literature often presents data as scatter plots, making reuse difficult. Extracting structured data from visual information is essential, yet existing tools lack a balance of convenience and accuracy. To address this, we develop DataEX‐Sc, a specialized model with five coordinated modules refined through dual loops. In the inner loop, DataEX‐Sc optimizes its internal modules using a data‐driven strategy combined with a sequence of computer vision methods to ensure robustness and accuracy. In the outer loop, a co‐evolution mechanism emerges between DataEX‐Sc and general‐purpose multimodal large language models (MLLMs). DataEX‐Sc provides high‐quality image‐text pairs for fine‐tuning MLLMs, improving their ability to understand images and code. In return, the refined MLLMs optimize DataEX‐Sc's algorithms and parameters using Cursor, enhancing its generalization and accuracy across complex varied inputs. DataEX‐Sc is architected uniquely to fully automate scatter plot parsing while allowing seamless user correction at any stage. It improves accuracy by more than 0.23 (absolute F 1 gain when relative error threshold = 0.02) over existing fully automatic tools and boosts efficiency by 80% over semi‐automatic tools. It is deployed online to accelerate the construction of materials databases. The tool's methodology, which integrates rule‐based specialized processing and general cognitive capabilities, is broadly applicable.