[DeepTensor.AI] MGLEP: Multi-Modal Data for Accurate Pandemic Forecasting and Analysis

Emerging pandemics pose a significant threat to global health, and accurate forecasting and analysis are essential for effective public health management and decision-making. Traditional approaches to pandemic forecasting primarily rely on epidemiological data, overlooking valuable sources of information that could act as sensors or indicators of pandemic patterns. In this paper, we propose a novel framework, MGLEP, that integrates temporal graph neural networks and multi-modal data for learning and forecasting emerging pandemics. By incorporating crowd reactions from social media, MGLEP can update its forecasting models, making it applicable for real-world complex pandemics.

Figure 1: Examples showing different stances on social media reacting to the pandemic and government regulations.

Figure 1 illustrates four distinct correlations between government policies and public reactions: (1) support, (2) opposition, (3) concern, and (4) neglect regarding the pandemic. These public reactions to government policies directly influence the spread of the virus as:

  • Supportive reactions may lead to better compliance and reduced transmission
  • Opposition can result in lower compliance and higher spread,
  • Concern might drive individuals to take additional precautions beyond official guidelines
  • Neglect can exacerbate the spread by ignoring necessary precautions.

Therefore, capturing these complex correlations is crucial for modeling emerging pandemics accurately and performing reliable forecasting. That’s the reason why we started to work on MGLEP method a few years back.

The MGLEP Framework

The first figure describes our proposed framework, MGLEP, harnesses the power of temporal graph neural networks and multi-modal data to provide accurate forecasting and analysis of emerging pandemics. The framework incorporates big data sources, such as social media content, by utilizing specific pre-trained language models and discovering the underlying graph structure among users. This integration provides rich indicators of pandemic dynamics through learning with temporal graph neural networks.

Effectiveness of MGLEP

We experimented in a number of datasets and they all showed the effectiveness of our framework in pandemic forecasting and analysis. It outperforms baseline methods across different areas, pandemic situations, and prediction horizons. The fusion of temporal graph learning and multi-modal data enables a comprehensive understanding of the pandemic landscape with less time lag, cheap cost, and more potential information indicators. Here are some main highlights of our work:

Short-term vs. Long-term Predictions:

  • For short-term predictions, our models underperform compared to the LSTM baseline. However, performance improves significantly on the New York dataset, particularly for a 7-day horizon, with lower standard deviations and higher stability across various metrics.
  • In long-term predictions, our models outperform all other methods across multiple horizons, achieving lower Mean Absolute Error (MAE) by significant margins. This indicates the models’ robustness and effectiveness in providing valuable long-term forecasts for policy planning.

Model Stability and Generalization:

  • The consistency and stability of our models are evident across different datasets, metrics, and scenarios. This is highlighted by lower standard deviations in predictions, demonstrating that our approach is less prone to overfitting compared to baselines like LSTM.
  • Comprehensive ablation studies reveal that the performance degrades as the number of nodes (social media users) in the input graph decreases, underscoring the importance of sufficient input data for capturing detailed public sentiment and behavior patterns.

Impact of Graph Neural Networks:

  • Our method, MGLEP, incorporates graph neural networks with spatial-temporal characteristics, and surpasses transformer-based approaches that do not utilize correlation matrices. This highlights the effectiveness of our approach in handling multi-modality data domains.
  • The models’ ability to capture and process complex interactions within social media data contributes significantly to their superior performance, particularly in constructing input graph structures and learning algorithms for COVID-19 prediction.

A quote from our paper:

“Our proposed method, MGLEP and its variants incorporate multiple information sources effectively, leading to better performances, lower errors, and sustained accuracy, particularly in long-term predictions, compared to other popular forecasting models. MGLEP enjoys these benefits due to it being able to learn the dynamic relationships between various factors that affect the spread of COVID-19, such as official government policy, and social stances against the pandemic or situation. Moreover, our ablation results clearly demonstrate that the availability of more information significantly enhances the reliability of our forecasting models.”

Tl,dr; Take away messages:

  1. MGLEP is a novel framework that integrates temporal graph neural networks and multi-modal data for learning and forecasting emerging pandemics.
  2. The framework uses big data sources, such as social media content, and pre-trained language models to discover the underlying graph structure among users, providing rich indicators of pandemic dynamics through learning with temporal graph neural networks.
  3. Experiments demonstrate the effectiveness of MGLEP in pandemic forecasting and analysis, outperforming baseline methods across various areas, especially in long-term forecasts.
  4. Not only utilized for COVID-19 forecasting, but also for other purposes, MGLEP serves as a versatile tool for various downstream applications, such as election monitoring and media coverage analysis.

Last but not least:

For any questions or collaborations, please feel free to contact us for more information.