Performance Evaluation of AutoML Frameworks for Deployment of AI Models in Serverless Cloud Architectures

Michael Ende Kerstin

Authors

Michael Ende Kerstin Cloud AI Infrastructure Engineer, United Kingdom. Author

Keywords:

AutoML, serverless computing, model deployment, cloud computing, AI infrastructure, latency optimization, H2O.ai, Google AutoML, Auto-sklearn

Abstract

The growing complexity of machine learning (ML) model development has led to the emergence of Automated Machine Learning (AutoML) frameworks that simplify model creation, hyperparameter tuning, and deployment. In parallel, serverless cloud architectures have gained traction for their scalability and cost-efficiency. This paper evaluates the performance of prominent AutoML frameworks when integrated into serverless environments for AI model deployment. Key performance indicators (KPIs) include latency, cold start times, cost efficiency, and model accuracy across AWS Lambda, Google Cloud Functions, and Azure Functions. Using benchmark datasets and consistent deployment configurations, we investigate the suitability of AutoML frameworks—such as Google AutoML, Auto-sklearn, and H2O.ai—for real-time inference tasks in serverless settings. The results highlight trade-offs between deployment simplicity and execution efficiency, offering practical insights for ML engineers and cloud architects.

References

Feurer, Matthias, et al. “Efficient and Robust Automated Machine Learning.” Advances in Neural Information Processing Systems, vol. 28, Neural Information Processing Systems Foundation, 2015, pp. 2962–2970.

Hutter, Frank, Lars Kotthoff, and Joaquin Vanschoren. Automated Machine Learning: Methods, Systems, Challenges. Springer, 2019.

Thornton, Chris, et al. “Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms.” Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2013, pp. 847–855.

Kamadi, S. (2023). Identity-Driven Zero Trust Automation in GitOps: Policy-as-Code Enforcement for Secure Code Deployments. International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 9(3), 893-902. https://doi.org/10.32628/CSEIT235148

Wang, Liang, et al. “Peeking Behind the Curtains of Serverless Platforms.” USENIX Annual Technical Conference, USENIX Association, 2018, pp. 133–146.

McGrath, Garrett, and Paul R. Brenner. “Serverless Computing: Design, Implementation, and Performance.” IEEE Cloud Computing, vol. 4, no. 2, 2017, pp. 46–56.

Jonas, Eric, et al. “Cloud Programming Simplified: A Berkeley View on Serverless Computing.” arXiv preprint arXiv:1902.03383, 2019.

Sculley, D., et al. “Hidden Technical Debt in Machine Learning Systems.” Advances in Neural Information Processing Systems, vol. 28, Neural Information Processing Systems Foundation, 2015, pp. 2503–2511.

Kotthoff, Lars, et al. “AutoML: A Survey of the State-of-the-Art.” ACM Computing Surveys, vol. 54, no. 6, ACM, 2021, pp. 1–36.

Baldini, Ioana, et al. “Serverless Computing: Current Trends and Open Problems.” Research Advances in Cloud Computing, Springer, 2017, pp. 1–20.

Lynn, Theo, et al. “A Preliminary Review of Enterprise Serverless Cloud Computing (Function-as-a-Service) Platforms.” Proceedings of the International Conference on Cloud Computing and Big Data, IEEE, 2017, pp. 162–169.

Breck, E., et al. “The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction.” Proceedings of the IEEE International Conference on Big Data, IEEE, 2017, pp. 1123–1132.

Zaharia, Matei, et al. “Accelerating the Machine Learning Lifecycle with MLflow.” IEEE Data Engineering Bulletin, vol. 41, no. 4, 2018, pp. 39–45.