microsoft/SynapseML
🏗️ Frameworkmicrosoft
Simple and distributed machine learning on Apache Spark.
SynapseML (formerly SparkML) is a Microsoft-led open-source framework that extends Apache Spark's MLlib with simple, composable APIs for deep learning, cognitive services, and model deployment. Key features include seamless integration with Azure Cognitive Services for vision, text, and speech; support for LightGBM, ONNX, and OpenCV; HTTP-based connectors for external web services; and automatic distributed hyperparameter tuning. SynapseML allows data scientists and engineers to build end-to-end ML pipelines using familiar Spark DataFrame APIs, without needing to manage complex infrastructure. It supports Scala, Python (PySpark), and R, making it accessible across the Spark ecosystem. The library is widely used for large-scale AI workloads in Azure Synapse, Databricks, and Hadoop clusters.
💡Highlights
- ├─Distributed ML on Spark
- ├─Prebuilt cognitive services
- └─LightGBM and ONNX support
🎯For
- ├─Data scientists
- ├─Spark users
- └─ML engineers