update readme's error

This commit is contained in:
gongjy 2024-08-28 18:03:24 +08:00
parent 4cda038006
commit 79a616ac15
2 changed files with 5 additions and 5 deletions

View File

@ -38,7 +38,7 @@
因此本项目的目标是把上手LLM的门槛无限降低
直接从0开始训练一个极其轻量的语言模型。
截至2024.8.27MiniMind首发包含3个型号模型最小仅需26M0.02B即可具备Amazing的对话能力
截至2024.8.28MiniMind首发包含4个型号模型最小仅需26M0.02B即可具备Amazing的对话能力
| 模型 (大小) | 速度 (Tokens/s) | 推理占用 | 训练占用(`batch_size=8`) |
|------------------------|---------------|--------|----------------------|
@ -175,7 +175,7 @@ python 2-eval.py
但MiniMind这里选择了mistral tokenizer作为分词器以保持整体参数轻量避免头重脚轻因为mistral的词表大小只有32,000。
且MiniMind在实际测试中几乎没有出现过生僻词汇解码失败的情况效果良好。
> 方便对比测试效果额外训练了一个自定义Tokenizer模型的版本**MiniMind(-T)**自定义词表压缩长度到6400使得LLM总参数进一步降低到40M左右。
> 方便对比测试效果额外训练了一个自定义Tokenizer模型的版本**MiniMind-small-T**自定义词表压缩长度到6400使得LLM总参数进一步降低到26M左右。
---

View File

@ -45,7 +45,7 @@ exacerbates the problem of finding quality content to understand LLMs, severely
Therefore, the goal of this project is to lower the barrier to entry for working with LLMs as much as possible, by
training an extremely lightweight language model from scratch.
(As of August 27, 2024) The initial release of MiniMind includes three model variants, with the smallest being just
(As of August 28, 2024) The initial release of MiniMind includes four model variants, with the smallest being just
26MB (0.02B) and still exhibiting amazing conversational capabilities!
| Model (Size) | Speed (Tokens/s) | Inference Memory | Training Memory (`batch_size=8`) |
@ -73,7 +73,7 @@ We hope this open-source project helps LLM beginners get started quickly!
👉**Recent Updates**
<details close>
<summary> <b>2024-08-27</b> </summary>
<summary> <b>2024-08-28</b> </summary>
- Project first open-sourced
</details>
@ -206,7 +206,7 @@ git clone https://github.com/jingyaogong/minimind.git
performance in practical tests, with almost no failures in decoding rare words.
> For comparison purposes, an additional custom Tokenizer version **MiniMind(-T)** was trained, reducing the
vocabulary size to 6,400, which further decreases the total model parameters to around 40M.
vocabulary size to 6,400, which further decreases the total model parameters to around 26M.
---