Chemical and Biological Technologies in Agriculture Biotechnology
Cross-scale multimodal phenotyping for sorghum stress early warning and yield prediction
Abstract
Background Reliable crop-stress diagnosis and yield forecasting require observations that capture both short-term canopy responses and seasonal crop development. Proximal sensors resolve canopy structure and physiology, whereas satellite time series provide phenological context; their integration is complicated by unequal footprints, asynchronous acquisition, variable quality, and missing observations. We developed CausalScale-Former, a multimodal phenotyping framework that aligns proximal and satellite observations by footprint and crop stage, learns from masked observations, and weights modalities using recorded quality. Results Two TERRA-REF sorghum seasons were combined with Sentinel-2 time series, environmental records, and a weak crop prior from the USDA Cropland Data Layer. All records from each plot remained in one partition. Evaluation included bidirectional seasonal transfer, held-out genotype and irrigation groups, spatial blocks, and natural or simulated data gaps. Stress Macro-F1 was 0.842 in plot-grouped testing, 0.755 and 0.766 in the two seasonal-transfer tests, and 0.738 in the spatial-block analysis (Table 13). Event-level F1 was 0.716, 0.742, and 0.721 at 7, 10, and 14 days, respectively (Table 11). At the final pre-harvest cutoff, yield prediction achieved and RMSE of 0.82 t ha (Table 23, Panel A). Conclusions Within this two-season, single-site experiment, scale- and phenology-aware multimodal phenotyping supported joint crop-stress diagnosis, early warning, and probabilistic yield prediction under incomplete sensing. External multi-site and multi-year validation is required before operational deployment. The causality-inspired regularizer does not identify causal effects. Graphical abstract A schematic links satellite and field sensor data to a central processing symbol, then to a plant, a clock, and a crop chart with a rising curve and a downward arrow.