Artificial Analysis, an independent AI benchmarking firm, has launched the Decision Index to evaluate decision models on three criteria: accuracy, speed and cost. These models, which make choices from pre-defined options, achieve a median latency of approximately 148 ms at a cost of about $0.11 per 1,000 requests. They can be 60 to 400 times cheaper than large language models for comparable tasks while performing worse on tasks requiring broad world knowledge or complex reasoning. Key providers included TypeSafe AI’s Jev, Cloudflare’s Clef, Perplexity’s pplx-decider and OpenAI’s Decisions API. For companies running AI at scale, these models could offer a cost-effective alternative for repetitive, bounded tasks, while larger language models would remain reserved for work requiring reasoning capabilities.
Source: Read the original article
