benoitc/erlang-python
🏗️ Frameworkbenoitc
Seamlessly execute Python code within Erlang and Elixir applications using high-performance, GIL-aware dirty NIFs.
The erlang-python project addresses the common challenge of interoperability between the highly concurrent BEAM ecosystem and the vast Python data science landscape. At its core, the project utilizes C-based dirty NIFs to execute Python code, which allows the BEAM scheduler to offload heavy computations without stalling the entire VM.
Key technical innovations include:
- GIL-Aware Execution: Carefully manages Python's Global Interpreter Lock to ensure that concurrent Python tasks do not degrade Erlang's real-time performance.
- Free-Threading Support: Designed to take advantage of modern Python's ability to run without the GIL, maximizing multi-core utilization.
- Rate Limiting: Built-in flow control mechanisms to prevent Python-side execution from overwhelming system resources.
- Low-Latency Interop: Provides a direct path for passing data between Erlang terms and Python objects, making it ideal for real-time inference, embedding generation, and complex mathematical processing within production Erlang/Elixir systems.
💡Highlights
- ├─Dirty NIFs for non-blocking execution
- ├─GIL-aware with free-threading support
- └─Built-in rate limiting for stability
🎯For
- ├─Erlang Developers
- ├─Elixir Engineers
- └─AI/ML Infrastructure Architects