InfoSecBulletin Cybersecurity for mankind
Researchers at the University of California, Berkeley, claims they’ve managed to reproduce the core technology behind DeepSeek’s at a total cost of roughly $30.
The news raises questions about whether developing advanced AI requires huge budgets or if cheaper alternatives have been ignored by major tech companies. DeepSeek recently launched R1, an AI model that aims to replicate ChatGPT and similar systems at a significantly lower training cost.
Russia detects first SuperCard malware attacks via NFC
Income Property Investments exposes 170,000+ Individuals record
ALERT (CVE: 2023-28771)
Zyxel Firewalls Under Attack via CVE-2023-28771 by 244 IPs
CISA Flags Active Exploits in Apple iOS and TP-Link Routers
10K Records Allegedly from Mac Cloud Provider’s Customers Leaked Online
Canada 2nd largest airlines “WestJet” investigates cyberattack disrupting internal systems
Paraguay 7.4 Million Citizen Records Leaked on Dark Web
High-Severity Flaw in HashiCorp Nomad Allows Privilege Escalation
SoftBank: Over 137,000 personal info leaked
Alert
Trend Micro Apex One Flaw Allow Attackers to Inject Malicious Code
The Berkeley team’s response? To do it even more cheaply. Led by PhD candidate Jiayi Pan, the researchers created a smaller-scale version, dubbed “TinyZero,” and released it on GitHub for public experimentation. Though it lacks the massive 671-billion-parameter heft of DeepSeek’s main offering, Pan says TinyZero captures the core behaviors seen in DeepSeek’s so-called “R1-Zero” model.
Pan’s approach uses reinforcement learning, where AI begins with random guesses and improves its answers by exploring different solutions. He mentioned the Countdown game, a British TV puzzle that challenges players to combine numbers to reach a target. Pan noted that while the AI initially produced inaccurate results, it eventually learned to correct its mistakes.
The bigger revelation:
The notion that just a few days of work and a small amount of money can achieve a key AI capability surprises many in the industry. This challenges the common belief that significant AI advancements need large data centers, expensive GPUs, and massive funding.
DeepSeek made waves in the tech industry by claiming that training its main model costs only a few million dollars, much lower than many U.S. companies spend on AI. Pan and his team stated it can be done for just $30 on a small scale.
Skeptics urge caution, noting that DeepSeek’s reported affordability may not reflect the whole truth, as the company could be using alternative resources or proprietary distillation techniques.
Pan’s “TinyZero” demonstrates that advanced reinforcement learning is possible on a budget, but it may not match the range of tasks that the larger DeepSeek system can perform. TinyZero might be more of a basic proof-of-concept than a serious competitor.
The demonstration suggests a significant change in the AI landscape. If open-source developers can create advanced features with limited resources, it questions why major companies like OpenAI, Google, and Microsoft spend so much on their platforms. While developing scale and advanced capabilities is expensive, it raises concerns about potential cost inflation in the industry. Open-source projects could potentially compete with these tech giants by working with smaller budgets.
“TinyZero” and DeepSeek’s R1 indicate a growing appetite for compact, resource-friendly AI models. Many had assumed that cutting-edge breakthroughs required billions in expenditure. Now, it seems a bright graduate student or a scrappy startup might surprise the world with innovation on the cheap. Whether this ultimately reshapes the future of AI infrastructure or remains an exciting anomaly, the conversation around affordable, powerful AI is only beginning.
Full experiment log: https://wandb.ai/jiayipan/TinyZero
GitHub: https://github.com/Jiayi-Pan/TinyZero
Source: interestingengineering.com
