update readme's error
This commit is contained in:
parent
4cda038006
commit
79a616ac15
@ -38,7 +38,7 @@
|
||||
因此,本项目的目标是把上手LLM的门槛无限降低,
|
||||
直接从0开始训练一个极其轻量的语言模型。
|
||||
|
||||
(截至2024.8.27)MiniMind首发包含3个型号模型,最小仅需26M(0.02B),即可具备Amazing的对话能力!
|
||||
(截至2024.8.28)MiniMind首发包含4个型号模型,最小仅需26M(0.02B),即可具备Amazing的对话能力!
|
||||
|
||||
| 模型 (大小) | 速度 (Tokens/s) | 推理占用 | 训练占用(`batch_size=8`) |
|
||||
|------------------------|---------------|--------|----------------------|
|
||||
@ -175,7 +175,7 @@ python 2-eval.py
|
||||
但MiniMind这里选择了mistral tokenizer作为分词器以保持整体参数轻量,避免头重脚轻,因为mistral的词表大小只有32,000。
|
||||
且MiniMind在实际测试中几乎没有出现过生僻词汇解码失败的情况,效果良好。
|
||||
|
||||
> 方便对比测试效果,额外训练了一个自定义Tokenizer模型的版本**MiniMind(-T)**,自定义词表压缩长度到6400,使得LLM总参数进一步降低到40M左右。
|
||||
> 方便对比测试效果,额外训练了一个自定义Tokenizer模型的版本**MiniMind-small-T**,自定义词表压缩长度到6400,使得LLM总参数进一步降低到26M左右。
|
||||
|
||||
---
|
||||
|
||||
|
@ -45,7 +45,7 @@ exacerbates the problem of finding quality content to understand LLMs, severely
|
||||
Therefore, the goal of this project is to lower the barrier to entry for working with LLMs as much as possible, by
|
||||
training an extremely lightweight language model from scratch.
|
||||
|
||||
(As of August 27, 2024) The initial release of MiniMind includes three model variants, with the smallest being just
|
||||
(As of August 28, 2024) The initial release of MiniMind includes four model variants, with the smallest being just
|
||||
26MB (0.02B) and still exhibiting amazing conversational capabilities!
|
||||
|
||||
| Model (Size) | Speed (Tokens/s) | Inference Memory | Training Memory (`batch_size=8`) |
|
||||
@ -73,7 +73,7 @@ We hope this open-source project helps LLM beginners get started quickly!
|
||||
👉**Recent Updates**
|
||||
|
||||
<details close>
|
||||
<summary> <b>2024-08-27</b> </summary>
|
||||
<summary> <b>2024-08-28</b> </summary>
|
||||
- Project first open-sourced
|
||||
</details>
|
||||
|
||||
@ -206,7 +206,7 @@ git clone https://github.com/jingyaogong/minimind.git
|
||||
performance in practical tests, with almost no failures in decoding rare words.
|
||||
|
||||
> For comparison purposes, an additional custom Tokenizer version **MiniMind(-T)** was trained, reducing the
|
||||
vocabulary size to 6,400, which further decreases the total model parameters to around 40M.
|
||||
vocabulary size to 6,400, which further decreases the total model parameters to around 26M.
|
||||
|
||||
---
|
||||
|
||||
|
Loading…
x
Reference in New Issue
Block a user