Mohith-akash/Global-News-Intel-Platform
📦 Open Source ProjectMohith-akash
An AI-powered geopolitical intelligence platform processing 100K+ daily events with a zero-cost data engineering stack.
The Global-News-Intel-Platform is a sophisticated end-to-end data pipeline designed to transform raw geopolitical event data into actionable intelligence. At its core, the system automates the ingestion of massive datasets from GDELT, ensuring that 100,000+ daily events are processed, cleaned, and stored efficiently. The architecture relies on Dagster for robust workflow orchestration, ensuring reliable data movement and transformation.
Storage is handled by MotherDuck, providing a serverless DuckDB experience that balances performance with cost-efficiency. A standout feature is the integrated AI chat interface, which utilizes Text-to-SQL logic to bridge the gap between complex database schemas and end-user queries. This allows analysts to perform deep-dive research into global events without needing to write manual SQL queries. The project is built with a modern Python-centric stack, incorporating CI/CD best practices and Streamlit for interactive dashboarding, making it an ideal reference architecture for developers building cost-effective, AI-augmented data analytics platforms.
💡Highlights
- ├─100K+ daily event ingestion
- ├─Serverless Text-to-SQL interface
- └─Dagster-orchestrated pipeline
🎯For
- ├─Data Engineers
- ├─AI Developers
- └─Geopolitical Analysts