iomgaa

iomgaa pushed to SLM at iomgaa/Minimind

2025-05-11 23:58:35 +08:00

8dd7cfaf72 修正了loss为nan的错误

iomgaa pushed to lzn at iomgaa/Minimind

2025-05-11 23:45:40 +08:00

eba4311ac5 修复了loss为nan的错误

iomgaa pushed to lzn at iomgaa/Minimind

2025-05-11 21:50:48 +08:00

iomgaa created branch lzn in iomgaa/Minimind

2025-05-11 21:50:48 +08:00

iomgaa pushed to SLM at iomgaa/Minimind

2025-05-10 20:23:58 +08:00

cb286d26d1 wandb包含config信息

iomgaa pushed to SLM at iomgaa/Minimind

2025-05-09 15:22:10 +08:00

0c8c6e5d1a 添加了忽视数据库模式

iomgaa pushed to SLM at iomgaa/Minimind

2025-05-09 15:01:15 +08:00

b6bd97aaaa 抽取self.downsample_v与self.downsample_q的共同部分，并使用可分离卷积降低参数量

iomgaa pushed to SLM at iomgaa/Minimind

2025-05-08 23:58:09 +08:00

iomgaa pushed to SLM at iomgaa/Minimind

2025-05-08 23:54:14 +08:00

6f8aff8914 Merge branch 'SLM' of http://110.42.53.85:3000/iomgaa/Minimind into SLM

253576967c 添加了train_embedding用于预训练嵌入模型

4ab8064ee0 update

a5a39d8c9b 'update'

Compare 4 commits »

iomgaa pushed to HPC at iomgaa/Minimind

2025-05-08 23:47:14 +08:00

bed6faa379 DynamicKV-LLM 1.0.1 交叉注意力添加多头；bf16代替fp16

iomgaa created branch HPC in iomgaa/Minimind

2025-05-08 23:47:14 +08:00

iomgaa pushed to SLM at iomgaa/Minimind

2025-05-08 23:41:53 +08:00

10f15724b4 添加了train_embedding用于预训练嵌入模型

iomgaa pushed to SLM at iomgaa/Minimind

2025-05-08 21:11:27 +08:00

253576967c 添加了train_embedding用于预训练嵌入模型

iomgaa pushed to SLM at iomgaa/Minimind

2025-05-08 20:15:14 +08:00

4ab8064ee0 update

iomgaa pushed to SLM at iomgaa/Minimind

2025-04-25 16:49:11 +08:00

0859f54a88 DynamicKV-LLM 1.0.0 完成了核心架构，模型可以正常训练

e3120f5e62 fix

Compare 2 commits »

iomgaa pushed to SLM at iomgaa/Minimind

2025-04-24 22:02:06 +08:00

1ddfd310ec 将Million MoE的思想加入

c55dfc0b46 添加了注释

21fdaaa59e 更新了忽视列表

7da201a944 update chat-openai-api

d9453ed9a3 update moe note

Compare 10 commits »

iomgaa created branch SLM in iomgaa/Minimind

2025-04-24 22:02:06 +08:00

iomgaa created repository iomgaa/Minimind

2025-04-24 22:01:08 +08:00