Nvidia has launched PAIR (Personal AI Router), a free, open-source software that connects multiple home computers on a local network to share GPU power for AI inference without using the cloud. The tool functions as a load balancer and is compatible with Ollama, LM Studio, GeForce RTX 20-series cards and newer, as well as Apple M4 chips and newer. A DGX Spark cluster can handle models up to 200 billion parameters when models are quantized. This initiative is part of Nvidia’s broader strategy called AI Grid, aimed at reducing cloud latency and costs. For cloud providers like AWS, Google Cloud, and Azure, this move represents a potential threat as it shifts some inference workload to consumer hardware.
Source: Read the original article
