Scaling law

The rule of thumb is that as the amount of model, data, and calculation increases, performance predictably improves. This is the theoretical basis for AI investment competition.

The scaling law is an empirical rule that states that as the three elements of an AI model—parameters, learning data, and computation volume—are increased, performance improves predictably and follows a certain pattern. A similar relationship has been observed in AI learning, as in farming, when you increase the field size, seeds, and labor, the yield increases to a certain degree.

Thanks to this rule, companies were able to estimate in advance how much better it would be if they invested in a larger model, and this became the theoretical basis for the AI investment race, pouring astronomical amounts of money into GPUs and data centers. Recently, scaling of the reasoning stage, which increases not only learning but also thinking time when answering, has been attracting attention as a new axis.

However, this is a rule of thumb, not a law of physics, and there continues to be debate that the expansion of existing methods has reached its limit due to the depletion of high-quality data and rapid increases in costs.

✅ Why it matters

• Provides a basis for predicting the performance of AI investments in advance
• It is the key to understanding competition in AI infrastructure such as GPUs and data centers
• It is used as a guideline for optimizing resource allocation in model development

⚠️ Limits and debates

• It is uncertain how long it will remain valid as it is an empirical rule rather than a physical law
• The depletion of high-quality training data is pointed out as a bottleneck in expansion
• There is a problem with the cost increasing exponentially compared to the amount of performance improvement.