Google has open-sourced the XLA Serving Engine, or XSE, a C++ runtime that serves compiled machine learning models without a Python interpreter. Released under the Apache 2.0 license, the engine already handles more than six million queries per second across internal Google products, including YouTube Shorts and Google Ads. According to Google, XSE delivers two to five times the throughput of the best-optimized Python servers and up to 36 times that of basic setups. It also uses 30% to 70% less CPU per prediction and roughly 75% less memory, while allowing the same exported artifact to run on CPUs, NVIDIA or AMD GPUs, and Google Cloud TPUs.
Source: Read the original article
Disclaimer: this content is for information purposes only and is not financial advice. Cryptocurrencies are highly volatile: you may lose all of your capital. Always do your own research. Legal notice
