Information Compression as Intelligence: A Deep Decoding of Large Language Models' Mathematical Logical Reasoning Capabilities

The 'Compression as Intelligence' (CAI) hypothesis, stemming from the theoretical framework of OPC-103, 'Mathematical Foundations and Logical Arguments for Compression as Intelligence,' posits that the core of intelligence lies in the most efficient compression of information. This implies that intelligent systems do not merely store data but rather discover, learn, and internalize deep-seated patterns and structures within the data, enabling them to represent information in the most concise way and perform effective prediction and reasoning based on it. For large language models, this compression capability may manifest in their mastery of the underlying semantic and syntactic structures of massive textual data, allowing them to generate coherent, meaningful text and solve complex problems.

The primary objective of this research is to validate the effectiveness of the CAI hypothesis in the domain of mathematical logical reasoning. We propose the main research hypothesis (H1): in the domain of mathematical logic, there is a significant positive correlation between a model's compression capability on relevant datasets and its reasoning performance in that domain. Specifically, H1a predicts that the lower the perplexity of a large language model on a specific mathematical logical dataset (representing stronger compression capability), the better its performance in reasoning tasks within that domain. Furthermore, H1b infers that a higher compression rate of mathematical logical text datasets by traditional compression algorithms might indirectly reflect the structural information of the dataset and be correlated with the LLM's reasoning ability. If these hypotheses do not hold true, it would support the null hypothesis (H0), indicating no significant positive correlation or a negative correlation between the two.

To rigorously validate the CAI hypothesis, we have designed key metrics to quantify 'compression capability' and 'intelligence performance.' The independent variable 'compression capability' will be measured in two aspects: first, traditional data compression rate, utilizing standard algorithms such as `gzip` and `zstd` to compress mathematical logical textual data (problem descriptions, solutions) and calculate their physical compression ratio; second, LLM perplexity (PPL), which is the average perplexity calculated by large language models on unseen mathematical logical textual datasets. Lower perplexity indicates that the model's 'compression' of internal data structures and statistical regularities is more effective, its predictive power is stronger, and it better reflects its deep understanding capability of information.

The dependent variable 'intelligence performance' will primarily be quantified by 'mathematical logical reasoning task scores,' which include the model's accuracy or other relevant metrics in solving problems on standardized mathematical logical benchmark datasets (e.g., GSM8K, MATH). These scores will directly reflect the model's practical application capabilities when dealing with complex mathematical and logical problems. We anticipate that models with lower perplexity will be able to solve these reasoning tasks more accurately.

The rigor of the experiment also relies on the careful design of control variables. This includes ensuring consistency in the domain, difficulty, and format of the datasets used for compression capability evaluation and intelligence performance assessment, to avoid data bias; recording or controlling the underlying architecture and parameter scales of different LLMs to enable fairer inter-model comparisons; standardizing the hardware environment and parameter settings for all LLM inference to reduce external interference; and adopting a unified prompting strategy for all LLM reasoning tasks to mitigate the impact of prompt engineering on results. Additionally, we will introduce traditional compression algorithms and weaker LLMs as baselines, as well as (if feasible) rule-symbolic systems as controls, to provide multi-dimensional reference points.

The CAP-1 experiment will employ multiple publicly available mathematical logical datasets to ensure the comprehensiveness and universality of the evaluation. These datasets include: GSM8K, a benchmark comprising elementary school math problems, used to assess LLM's basic reasoning capabilities; MATH, a challenging set covering higher-difficulty math problems such as high school algebra and geometry; AQuA, which focuses on abstract reasoning abilities involving numbers, words, and logical relationships; and Proof-Pile (or similar datasets), used to examine the model's compression efficiency and reasoning ability on more formal, structured mathematical proofs and logical texts. All datasets will undergo strict pre-processing before being fed into compression tools and LLMs, unified into plain text format with irrelevant metadata removed, to ensure input consistency.

The implementation path of the experiment has been meticulously designed using `dsh` (a universal shell environment) and Python scripts to ensure its operability and reproducibility. The technology stack centers on Python 3.10+, supplemented by industry-standard libraries such as PyTorch, Hugging Face Transformers, and Datasets. The core model selected is Qwen3.6-35B-Chat (or another base version), which will run on an NVIDIA GPU with at least 24GB VRAM, employing 4-bit quantization if necessary to accommodate single-card environments. Specific execution steps cover environment preparation, dataset download and pre-processing, traditional compression rate measurement (using `gzip` and `zstd`), as well as Qwen3.6-35B model loading, perplexity calculation, and mathematical logical reasoning task execution. These steps ensure a complete chain from raw data to final results.

The experiment will strictly follow these executable steps: first, environment setup and dataset pre-processing will be completed to ensure all data formats are consistent; subsequently, traditional compression rate measurements will be performed on the processed datasets to obtain physical compression efficiency data. Next, the Qwen3.6-35B model will be loaded, and its average perplexity on each dataset will be calculated, quantifying its 'perception' of the text's internal structure. After this, the model will execute mathematical logical reasoning tasks, outputting answers and calculating accuracy. Finally, all data (traditional compression rates, LLM perplexity, LLM inference scores, and control group data) will be collected for statistical analysis, such as calculating Pearson or Spearman correlation coefficients, and visualized using scatter plots, bar charts, and other methods to visually present the relationship between compression capability and intelligence performance, thereby validating or refuting the core hypothesis.

This research (CAP-1) on validating the 'Compression as Intelligence' hypothesis in the domain of mathematical logic holds significant theoretical and practical implications. From a theoretical perspective, it will offer new perspectives and empirical support for understanding the nature of large language model intelligence, deepening our understanding of the complex concept of 'intelligence.' If experimental results show a significant positive correlation between LLM perplexity and reasoning ability, it will strongly support the CAI hypothesis, indicating that information compression is not merely a technical means but one of the deep mechanisms of intelligence emergence. This will help bridge information theory, cognitive science, and artificial intelligence research, promoting interdisciplinary development.

On a practical level, the results of this study will provide clear guidance for the design and optimization of future LLMs. If compression capability is indeed a key driver of intelligent performance, then improving the model's compression efficiency, for example through better architectural design, training objectives, or training data selection, will become an important avenue for enhancing its reasoning capabilities. Furthermore, by quantifying model perplexity in specific domains (such as mathematical logic), it can serve as a prospective evaluation metric to predict the model's potential performance in that domain, thereby more effectively screening and deploying models. This will not only contribute to developing more efficient, intelligent AI systems but also provide new, explainable dimensions for model evaluation.

Looking ahead, this research lays an empirical foundation for the CAI hypothesis, opening up broader avenues for exploration. Further studies can be extended to other domains (e.g., scientific discovery, artistic creation) to validate the universality of CAI. Simultaneously, deeply investigating how different model architectures and training paradigms affect their compression capability, and how this capability translates into specific intelligent behaviors, will be a key task for the next stage. Ultimately, understanding 'compression as intelligence' may lead AI towards a more fundamental and unified theoretical framework, propelling agent design to new heights.