How DeepSeek Cuts Costs: Apple’s AI Experts Spill the Beans

Apple researchers have already figured out the ‘secret sauce’ behind their DeepSeek AI, and surprise, surprise, it’s all about ‘sparsity.’

Thanks to a report from ZDNet that uses explainers from Apple’s research team, we now have a clearer picture as to how DeepSeek achieved its results as well as why investors panicked earlier this week.

This highfalutin term essentially means they’ve mastered the art of doing less with more. Instead of lighting up the entire neural network, they cleverly shut off the unnecessary parts, much like turning off the lights in an unused room to cut down on the electric bill. It’s a strategy that screams efficiency—if by efficiency, you mean cutting corners to save on costs.

Sparsity in AI refers to techniques that reduce the computational load by eliminating unnecessary elements in neural networks. Here are some common ways sparsity is implemented:

1. Pruning: This involves removing less important connections (weights) in a neural network. By cutting out these “unimportant” weights, the network becomes sparser and requires less computation. Pruning can be done in a structured way (removing entire filters or channels) or unstructured (removing individual weights).
2. Sparse Matrices: In this approach, matrices used in computations are filled with many zeros. By skipping operations involving these zeros, computational efficiency is improved. This is particularly useful in large-scale models where many elements are zero.
3. Sparse Tensors: Similar to sparse matrices, sparse tensors exploit the presence of zero values to skip computations and save memory. Techniques like HighLight and Tailors and Swiftiles have been developed to efficiently process these sparse tensors.
   • HighLight
   • HighLight is a technique developed to efficiently process sparse tensors by focusing on the nonzero values. Sparse tensors are data structures used in AI models where many of the values are zero. HighLight helps in identifying and processing only the nonzero values, which saves computation time and memory. This technique is particularly useful for high-performance computing tasks like graph analytics or generative AI.
   • Tailors
   • Tailors is another technique designed to handle sparse tensors. It addresses the challenge of varying nonzero values in different regions of the tensor. By efficiently finding the nonzero values and managing the memory allocation, Tailors ensures that the storage buffer is utilized effectively, reducing off-chip memory traffic and energy consumption.
   • Swiftiles
   • Swiftiles complements the Tailors technique by further optimizing the processing of sparse tensors. It helps in situations where the data do not fit in memory, increasing the utilization of the storage buffer and reducing off-chip memory traffic. This results in significant improvements in performance and energy efficiency for AI models.
   • Together, these techniques—HighLight, Tailors, and Swiftiles—boost the performance of demanding machine-learning tasks by efficiently exploiting sparsity in tensors. They represent a significant advancement in the field of AI, making it possible to achieve high accuracy with less computing power.
4. Dynamic Sparsity: This method involves dynamically adjusting the sparsity of the network during training or inference. It allows the network to adapt its level of sparsity based on the task, potentially improving performance without a significant loss in accuracy.
5. Low-Rank Approximations: This technique approximates large matrices with smaller, more manageable ones by exploiting the low-rank structure of the data. This reduces the number of parameters and computations needed.
6. Quantization: While not strictly sparsity, quantization reduces the precision of the weights, which can be combined with sparsity techniques to further reduce computational requirements.

These methods help in making AI models more efficient, reducing both computational costs and energy consumption, while maintaining acceptable levels of accuracy.

By focusing only on the most crucial parts of the network, they manage to reduce computing power, which is fantastic for the bottom line but raises eyebrows about just how much performance and accuracy they’re sacrificing. It’s akin to using a Swiss army knife: sure, it has all the tools you might need, but it’s not exactly specialized.

DeepSeek AI’s approach is basically an elaborate dance around the limitations of traditional AI. Instead of brute-forcing their way through problems with massive computational power, they’re opting for a minimalist approach. But this minimalism comes with a catch. While it might work wonders in some scenarios, it leaves you wondering if you’re settling for a half-baked solution. It’s like buying a knockoff handbag—it looks the part and might function well enough for everyday use, but don’t be surprised if it falls apart when you need it the most.

Multi-Head Latent Attention (MLA) in DeepSeek AI is another clever trick up their sleeve. Instead of the traditional attention mechanism that processes all input data at once, MLA breaks the data into multiple ‘heads’ that can focus on different parts of the input simultaneously. This allows the model to capture a richer and more nuanced understanding of the data. But here’s the kicker: MLA compresses the Key-Value (KV) cache into a latent vector, significantly reducing the memory footprint and computational load. It’s like having multiple people working on different parts of a project at the same time, but only using a fraction of the resources you’d normally need.

This combination of sparsity and MLA means DeepSeek AI can perform complex tasks with less computational power and memory, making it a cost-effective solution. But it also leaves you wondering just how much they’re skimping on to achieve these efficiencies. It’s a balancing act between performance and frugality, and only time will tell if they’ve struck the right balance.

And perhaps this is why Apple seemed a bit gun-shy about handing over piles of cash to OpenAI, unlike other big tech firms who are all too eager to leverage OpenAI’s Large Language Models (LLMs). Instead of throwing money at the problem like everyone else, Apple decided to reinvent the wheel with their frugal, sparse approach. Who needs the vast, complex, and undoubtedly expensive LLMs when you can achieve ‘good enough’ results on a budget? After all, if you can’t beat them, why not just do less and call it a day?”

So, while Apple touts this as a breakthrough in AI research, let’s not kid ourselves. This so-called ‘innovation’ is just a sophisticated way of admitting that they can’t, or won’t, invest the resources needed for a truly comprehensive AI solution. It’s a clever disguise for cost-cutting measures, wrapped up in the glamour of tech innovation.

Microsoft’s Small Language Models (SLMs), such as the Phi series, could be seen as a strategic pivot in light of DeepSeek. These models, including the latest Phi-4, are designed to deliver high-quality results with fewer parameters, making them more efficient and cost-effective compared to larger models. This efficiency is crucial as it allows Microsoft to reduce its reliance on OpenAI’s large language models (LLMs), which have been a significant part of their AI offerings.

The recent emergence of DeepSeek, a competitor in the AI space, is challenging the traditional approach of investing heavily in massive models. DeepSeek’s success demonstrates that smaller, more efficient models can achieve comparable or even superior performance, which questions the necessity of the “bigger is better” mindset in AI development.

By focusing on SLMs, Microsoft can diversify its AI portfolio and reduce dependency on any single partner, such as OpenAI. This move not only strengthens Microsoft’s position in the AI market but also aligns with a more sustainable and cost-effective approach to AI development, as highlighted by DeepSeek’s approach.