Minimind/nohup.out
2025-07-02 12:46:37 +00:00

348 lines
184 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

swanlab: \ Waiting for the swanlab cloud response.
swanlab: swanlab version 0.6.4 is available! Upgrade: `pip install -U swanlab`
swanlab: \ Getting project...
swanlab: \ Creating experiment...
swanlab: | Creating experiment...
swanlab: Tracking run with swanlab version 0.6.3
swanlab: Run data will be saved locally in /home/rwkv/RWKV-TS/RETRO_TEST/Minimind/swanlog/run-20250702_123051-d30a286e
swanlab: 👋 Hi Garylu, welcome to swanlab!
swanlab: Syncing run MiniMind-TripleExtraction-Epoch-4-BatchSize-192-LearningRate-0.0002 to the cloud
swanlab: 🏠 View project at https://swanlab.cn/@Garylu/MiniMind-TripleExtraction
swanlab: 🚀 View run at https://swanlab.cn/@Garylu/MiniMind-TripleExtraction/runs/pgnn4um8pb74vf4bpden3
[2025-07-02 12:30:52] tokens_per_iter: 98304
[2025-07-02 12:30:52] Configuration:
[2025-07-02 12:30:52] out_dir: out
[2025-07-02 12:30:52] epochs: 4
[2025-07-02 12:30:52] embedding_epoch: 2
[2025-07-02 12:30:52] batch_size: 192
[2025-07-02 12:30:52] learning_rate: 0.0002
[2025-07-02 12:30:52] dtype: bfloat16
[2025-07-02 12:30:52] use_swanlab: True
[2025-07-02 12:30:52] swanlab_project: MiniMind-TripleExtraction
[2025-07-02 12:30:52] num_workers: 1
[2025-07-02 12:30:52] accumulation_steps: 32
[2025-07-02 12:30:52] grad_clip: 1.0
[2025-07-02 12:30:52] warmup_iters: 0
[2025-07-02 12:30:52] log_interval: 50
[2025-07-02 12:30:52] save_interval: 10000
[2025-07-02 12:30:52] dim: 512
[2025-07-02 12:30:52] n_layers: 8
[2025-07-02 12:30:52] max_seq_len: 512
[2025-07-02 12:30:52] use_moe: False
[2025-07-02 12:30:52] disable_db: False
[2025-07-02 12:30:52] data_path: /home/rwkv/RWKV-TS/RETRO_TEST/extract/processed_trex_data.json
[2025-07-02 12:30:52] pretrained_embedding_path: None
[2025-07-02 12:30:52] profile: True
[2025-07-02 12:30:52] profile_interval: 10
[2025-07-02 12:30:52] use_flash_attn: True
[2025-07-02 12:30:52] knowledge_num: 960400
[2025-07-02 12:30:52] knowledge_length: 32
[2025-07-02 12:30:52] database_init_path: ./dataset/combined_prepare.json
[2025-07-02 12:30:52] fast_clustering: True
[2025-07-02 12:30:52] cluster_cache_path: ./cache/cluster_tokens_single.pt
[2025-07-02 12:30:52] recompute_clusters: False
[2025-07-02 12:30:52] memory_monitor: False
[2025-07-02 12:30:52] memory_monitor_interval: 10
[2025-07-02 12:30:52] max_targets: 5
[2025-07-02 12:30:52] temperature: 1.0
[2025-07-02 12:30:52] detailed_timing: True
[2025-07-02 12:30:52] save_dir: out
[2025-07-02 12:30:52] swanlab_run_name: MiniMind-TripleExtraction-Epoch-4-BatchSize-192-LearningRate-0.0002
[2025-07-02 12:30:52] n_heads: 32
[2025-07-02 12:30:52] n_kv_heads: 8
[2025-07-02 12:30:52] vocab_size: 6400
[2025-07-02 12:30:52] hidden_dim: None
[2025-07-02 12:30:52] multiple_of: 64
[2025-07-02 12:30:52] norm_eps: 1e-05
[2025-07-02 12:30:52] rope_theta: 1000000.0
[2025-07-02 12:30:52] dropout: 0.0
[2025-07-02 12:30:52] flash_attn: True
[2025-07-02 12:30:52] embeddings_epoch: 2
[2025-07-02 12:30:52] num_experts_per_tok: 2
[2025-07-02 12:30:52] n_routed_experts: 4
[2025-07-02 12:30:52] n_shared_experts: True
[2025-07-02 12:30:52] scoring_func: softmax
[2025-07-02 12:30:52] aux_loss_alpha: 0.1
[2025-07-02 12:30:52] seq_aux: True
[2025-07-02 12:30:52] norm_topk_prob: True
[2025-07-02 12:30:52] knowledge_dim: 128
[2025-07-02 12:30:52] max_subject_len: 8
[2025-07-02 12:30:52] max_predicate_len: 4
[2025-07-02 12:30:52] max_object_len: 8
[2025-07-02 12:30:52] return_dict: True
[2025-07-02 12:30:52] output_hidden_states: False
[2025-07-02 12:30:52] output_attentions: False
[2025-07-02 12:30:52] torchscript: False
[2025-07-02 12:30:52] torch_dtype: None
[2025-07-02 12:30:52] use_bfloat16: False
[2025-07-02 12:30:52] tf_legacy_loss: False
[2025-07-02 12:30:52] pruned_heads: {}
[2025-07-02 12:30:52] tie_word_embeddings: True
[2025-07-02 12:30:52] chunk_size_feed_forward: 0
[2025-07-02 12:30:52] is_encoder_decoder: False
[2025-07-02 12:30:52] is_decoder: False
[2025-07-02 12:30:52] cross_attention_hidden_size: None
[2025-07-02 12:30:52] add_cross_attention: False
[2025-07-02 12:30:52] tie_encoder_decoder: False
[2025-07-02 12:30:52] max_length: 20
[2025-07-02 12:30:52] min_length: 0
[2025-07-02 12:30:52] do_sample: False
[2025-07-02 12:30:52] early_stopping: False
[2025-07-02 12:30:52] num_beams: 1
[2025-07-02 12:30:52] num_beam_groups: 1
[2025-07-02 12:30:52] diversity_penalty: 0.0
[2025-07-02 12:30:52] top_k: 50
[2025-07-02 12:30:52] top_p: 1.0
[2025-07-02 12:30:52] typical_p: 1.0
[2025-07-02 12:30:52] repetition_penalty: 1.0
[2025-07-02 12:30:52] length_penalty: 1.0
[2025-07-02 12:30:52] no_repeat_ngram_size: 0
[2025-07-02 12:30:52] encoder_no_repeat_ngram_size: 0
[2025-07-02 12:30:52] bad_words_ids: None
[2025-07-02 12:30:52] num_return_sequences: 1
[2025-07-02 12:30:52] output_scores: False
[2025-07-02 12:30:52] return_dict_in_generate: False
[2025-07-02 12:30:52] forced_bos_token_id: None
[2025-07-02 12:30:52] forced_eos_token_id: None
[2025-07-02 12:30:52] remove_invalid_values: False
[2025-07-02 12:30:52] exponential_decay_length_penalty: None
[2025-07-02 12:30:52] suppress_tokens: None
[2025-07-02 12:30:52] begin_suppress_tokens: None
[2025-07-02 12:30:52] architectures: None
[2025-07-02 12:30:52] finetuning_task: None
[2025-07-02 12:30:52] id2label: {0: 'LABEL_0', 1: 'LABEL_1'}
[2025-07-02 12:30:52] label2id: {'LABEL_0': 0, 'LABEL_1': 1}
[2025-07-02 12:30:52] tokenizer_class: None
[2025-07-02 12:30:52] prefix: None
[2025-07-02 12:30:52] bos_token_id: None
[2025-07-02 12:30:52] pad_token_id: None
[2025-07-02 12:30:52] eos_token_id: None
[2025-07-02 12:30:52] sep_token_id: None
[2025-07-02 12:30:52] decoder_start_token_id: None
[2025-07-02 12:30:52] task_specific_params: None
[2025-07-02 12:30:52] problem_type: None
[2025-07-02 12:30:52] _name_or_path:
[2025-07-02 12:30:52] _commit_hash: None
[2025-07-02 12:30:52] _attn_implementation_internal: None
[2025-07-02 12:30:52] _attn_implementation_autoset: False
[2025-07-02 12:30:52] transformers_version: None
三元组提取任务头配置:
- 主语最大长度: 8
- 谓语最大长度: 4
- 宾语最大长度: 8
已冻结以下组件的权重:
- tok_embeddings
- knowledge_dataset
- layers (所有transformer层)
- output
- pos_cis
注意triple_extraction_head 保持可训练状态
[2025-07-02 12:30:53] Loading pretrained weights from /home/rwkv/RWKV-TS/RETRO_TEST/extract/Experiment_1_2_2_pretrain_512.pth
[2025-07-02 12:30:53] Successfully loaded pretrained state_dict with 143 parameters
[2025-07-02 12:30:53] Loaded 143 parameters from pretrained weights
[2025-07-02 12:30:53] Skipped 0 parameters
[2025-07-02 12:30:53] Key loaded parameters:
[2025-07-02 12:30:53] ✅ tok_embeddings.weight
[2025-07-02 12:30:53] ✅ knowledge_dataset.keys
[2025-07-02 12:30:53] ✅ knowledge_dataset.knowledge_dataset
[2025-07-02 12:30:53] ✅ knowledge_dataset.tok_embeddings.weight
[2025-07-02 12:30:53] ✅ knowledge_dataset.to_queries.0.weight
[2025-07-02 12:30:53] ... and 61 more
[2025-07-02 12:30:53] Database embeddings and sentences stored in model
[2025-07-02 12:30:53] LLM总参数量14.486 百万
[2025-07-02 12:30:53] 模型初始化完成
[2025-07-02 12:30:53] 检测到pos_cis复数张量将其设置为不参与分布式训练
[2025-07-02 12:30:53] 三元组提取训练:使用 TriplePretrainDataset
🚀 开始加载和预处理三元组数据...
📂 加载原始数据...
📊 原始数据量: 3459987 个样本
🔍 验证数据格式并选择单个target...
验证数据格式: 0%| | 0/3459987 [00:00<?, ?it/s]
验证数据格式: 2%|▏ | 61336/3459987 [00:00<00:05, 613318.18it/s]
验证数据格式: 4%|▎ | 126888/3459987 [00:00<00:05, 638119.17it/s]
验证数据格式: 6%|▌ | 192355/3459987 [00:00<00:05, 645657.67it/s]
验证数据格式: 7%|▋ | 258273/3459987 [00:00<00:04, 650978.40it/s]
验证数据格式: 9%|▉ | 324611/3459987 [00:00<00:04, 655431.95it/s]
验证数据格式: 11%|█▏ | 390155/3459987 [00:04<01:08, 44547.50it/s]
验证数据格式: 13%|█▎ | 453152/3459987 [00:04<00:47, 63255.01it/s]
验证数据格式: 15%|█▍ | 515212/3459987 [00:04<00:33, 87587.39it/s]
验证数据格式: 17%|█▋ | 577147/3459987 [00:04<00:24, 118805.12it/s]
验证数据格式: 18%|█▊ | 639703/3459987 [00:05<00:17, 157997.13it/s]
验证数据格式: 20%|██ | 702550/3459987 [00:05<00:13, 204819.76it/s]
验证数据格式: 22%|██▏ | 765705/3459987 [00:05<00:10, 257937.26it/s]
验证数据格式: 24%|██▍ | 827753/3459987 [00:05<00:08, 312671.08it/s]
验证数据格式: 26%|██▌ | 888881/3459987 [00:05<00:07, 336273.19it/s]
验证数据格式: 27%|██▋ | 943918/3459987 [00:05<00:06, 369647.65it/s]
验证数据格式: 29%|██▉ | 1001663/3459987 [00:05<00:05, 413167.10it/s]
验证数据格式: 31%|███ | 1065838/3459987 [00:05<00:05, 465803.83it/s]
验证数据格式: 33%|███▎ | 1129974/3459987 [00:05<00:04, 509255.91it/s]
验证数据格式: 35%|███▍ | 1194314/3459987 [00:05<00:04, 544291.14it/s]
验证数据格式: 36%|███▋ | 1256779/3459987 [00:06<00:03, 566121.49it/s]
验证数据格式: 38%|███▊ | 1321417/3459987 [00:06<00:03, 588556.99it/s]
验证数据格式: 40%|████ | 1386883/3459987 [00:06<00:03, 607416.03it/s]
验证数据格式: 42%|████▏ | 1451994/3459987 [00:06<00:03, 620061.28it/s]
验证数据格式: 44%|████▍ | 1517329/3459987 [00:06<00:03, 629791.22it/s]
验证数据格式: 46%|████▌ | 1582961/3459987 [00:06<00:02, 637602.03it/s]
验证数据格式: 48%|████▊ | 1648592/3459987 [00:06<00:02, 643132.12it/s]
验证数据格式: 50%|████▉ | 1713568/3459987 [00:06<00:02, 640266.70it/s]
验证数据格式: 51%|█████▏ | 1780772/3459987 [00:06<00:02, 649676.74it/s]
验证数据格式: 53%|█████▎ | 1846832/3459987 [00:06<00:02, 652914.12it/s]
验证数据格式: 55%|█████▌ | 1912986/3459987 [00:07<00:02, 655473.71it/s]
验证数据格式: 57%|█████▋ | 1978784/3459987 [00:07<00:02, 656209.75it/s]
验证数据格式: 59%|█████▉ | 2044524/3459987 [00:07<00:02, 656145.75it/s]
验证数据格式: 61%|██████ | 2110222/3459987 [00:07<00:02, 649160.25it/s]
验证数据格式: 63%|██████▎ | 2175837/3459987 [00:07<00:01, 651224.27it/s]
验证数据格式: 65%|██████▍ | 2241882/3459987 [00:07<00:01, 653962.25it/s]
验证数据格式: 67%|██████▋ | 2307318/3459987 [00:12<00:26, 43211.35it/s]
验证数据格式: 69%|██████▊ | 2371893/3459987 [00:12<00:18, 59726.31it/s]
验证数据格式: 70%|███████ | 2431231/3459987 [00:12<00:12, 79834.43it/s]
验证数据格式: 72%|███████▏ | 2495839/3459987 [00:12<00:08, 108757.67it/s]
验证数据格式: 74%|███████▍ | 2561481/3459987 [00:12<00:06, 146002.58it/s]
验证数据格式: 76%|███████▌ | 2627449/3459987 [00:12<00:04, 191487.91it/s]
验证数据格式: 78%|███████▊ | 2693385/3459987 [00:13<00:03, 244026.07it/s]
验证数据格式: 80%|███████▉ | 2759702/3459987 [00:13<00:02, 301967.39it/s]
验证数据格式: 82%|████████▏ | 2826036/3459987 [00:13<00:01, 361560.88it/s]
验证数据格式: 84%|████████▎ | 2893813/3459987 [00:13<00:01, 421848.32it/s]
验证数据格式: 86%|████████▌ | 2959395/3459987 [00:13<00:01, 469941.56it/s]
验证数据格式: 87%|████████▋ | 3026368/3459987 [00:13<00:00, 516573.27it/s]
验证数据格式: 89%|████████▉ | 3092099/3459987 [00:13<00:00, 551557.59it/s]
验证数据格式: 91%|█████████▏| 3157800/3459987 [00:13<00:00, 578917.30it/s]
验证数据格式: 93%|█████████▎| 3223451/3459987 [00:13<00:00, 599759.57it/s]
验证数据格式: 95%|█████████▌| 3289361/3459987 [00:13<00:00, 616393.98it/s]
验证数据格式: 97%|█████████▋| 3355059/3459987 [00:14<00:00, 627681.71it/s]
验证数据格式: 99%|█████████▉| 3420722/3459987 [00:14<00:00, 635877.39it/s]
验证数据格式: 100%|██████████| 3459987/3459987 [00:14<00:00, 243701.66it/s]
✅ 有效样本数: 3459987
🔤 分批tokenize目标句子...
分批tokenize目标句子: 0%| | 0/3460 [00:00<?, ?it/s]
分批tokenize目标句子: 0%| | 1/3460 [00:00<05:59, 9.61it/s]
分批tokenize目标句子: 0%| | 2/3460 [00:00<06:31, 8.83it/s]
分批tokenize目标句子: 0%| | 4/3460 [00:00<05:31, 10.43it/s]
分批tokenize目标句子: 0%| | 6/3460 [00:00<05:12, 11.05it/s]
分批tokenize目标句子: 0%| | 8/3460 [00:00<05:04, 11.33it/s]
分批tokenize目标句子: 0%| | 10/3460 [00:00<04:59, 11.52it/s]
分批tokenize目标句子: 0%| | 12/3460 [00:01<04:56, 11.63it/s]
分批tokenize目标句子: 0%| | 14/3460 [00:01<04:54, 11.70it/s]
分批tokenize目标句子: 0%| | 16/3460 [00:01<04:55, 11.65it/s]
分批tokenize目标句子: 1%| | 18/3460 [00:01<04:54, 11.68it/s]
分批tokenize目标句子: 1%| | 20/3460 [00:01<04:53, 11.73it/s]
分批tokenize目标句子: 1%| | 22/3460 [00:01<04:51, 11.78it/s]
分批tokenize目标句子: 1%| | 24/3460 [00:02<04:52, 11.73it/s]
分批tokenize目标句子: 1%| | 26/3460 [00:02<04:54, 11.65it/s]
分批tokenize目标句子: 1%| | 28/3460 [00:02<04:54, 11.63it/s]
分批tokenize目标句子: 1%| | 30/3460 [00:02<04:57, 11.54it/s]
分批tokenize目标句子: 1%| | 32/3460 [00:02<04:55, 11.60it/s]
分批tokenize目标句子: 1%| | 34/3460 [00:02<04:55, 11.59it/s]
分批tokenize目标句子: 1%| | 36/3460 [00:03<04:57, 11.49it/s]
分批tokenize目标句子: 1%| | 38/3460 [00:03<04:59, 11.43it/s]
分批tokenize目标句子: 1%| | 40/3460 [00:03<04:59, 11.41it/s]
分批tokenize目标句子: 1%| | 42/3460 [00:03<05:00, 11.39it/s]
分批tokenize目标句子: 1%|▏ | 44/3460 [00:03<05:01, 11.32it/s]
分批tokenize目标句子: 1%|▏ | 46/3460 [00:04<05:02, 11.30it/s]
分批tokenize目标句子: 1%|▏ | 48/3460 [00:04<04:59, 11.39it/s]
分批tokenize目标句子: 1%|▏ | 50/3460 [00:04<04:58, 11.44it/s]
分批tokenize目标句子: 2%|▏ | 52/3460 [00:04<04:56, 11.48it/s]
分批tokenize目标句子: 2%|▏ | 54/3460 [00:04<04:55, 11.53it/s]
分批tokenize目标句子: 2%|▏ | 56/3460 [00:04<04:54, 11.55it/s]
分批tokenize目标句子: 2%|▏ | 58/3460 [00:05<04:58, 11.39it/s]
分批tokenize目标句子: 2%|▏ | 60/3460 [00:05<04:59, 11.36it/s]
分批tokenize目标句子: 2%|▏ | 62/3460 [00:05<04:57, 11.41it/s]
分批tokenize目标句子: 2%|▏ | 64/3460 [00:05<04:56, 11.46it/s]
分批tokenize目标句子: 2%|▏ | 66/3460 [00:05<04:55, 11.48it/s]
分批tokenize目标句子: 2%|▏ | 68/3460 [00:05<04:57, 11.41it/s]
分批tokenize目标句子: 2%|▏ | 70/3460 [00:06<04:59, 11.33it/s]
分批tokenize目标句子: 2%|▏ | 72/3460 [00:06<05:02, 11.21it/s]
分批tokenize目标句子: 2%|▏ | 74/3460 [00:06<05:11, 10.86it/s]
分批tokenize目标句子: 2%|▏ | 76/3460 [00:06<05:15, 10.72it/s]
分批tokenize目标句子: 2%|▏ | 78/3460 [00:06<05:16, 10.70it/s]
分批tokenize目标句子: 2%|▏ | 80/3460 [00:07<05:13, 10.77it/s]
分批tokenize目标句子: 2%|▏ | 82/3460 [00:07<05:09, 10.90it/s]
分批tokenize目标句子: 2%|▏ | 84/3460 [00:07<05:09, 10.93it/s]
分批tokenize目标句子: 2%|▏ | 86/3460 [00:07<05:07, 10.98it/s]
分批tokenize目标句子: 3%|▎ | 88/3460 [00:07<05:06, 11.01it/s]
分批tokenize目标句子: 3%|▎ | 90/3460 [00:07<05:04, 11.08it/s]
分批tokenize目标句子: 3%|▎ | 92/3460 [00:08<05:03, 11.09it/s]
分批tokenize目标句子: 3%|▎ | 94/3460 [00:08<04:59, 11.22it/s]
分批tokenize目标句子: 3%|▎ | 96/3460 [00:08<04:56, 11.36it/s]
分批tokenize目标句子: 3%|▎ | 98/3460 [00:08<04:54, 11.41it/s]
分批tokenize目标句子: 3%|▎ | 100/3460 [00:08<04:53, 11.45it/s]
分批tokenize目标句子: 3%|▎ | 102/3460 [00:09<04:52, 11.50it/s]
分批tokenize目标句子: 3%|▎ | 104/3460 [00:09<04:53, 11.45it/s]
分批tokenize目标句子: 3%|▎ | 106/3460 [00:09<04:58, 11.24it/s]
分批tokenize目标句子: 3%|▎ | 108/3460 [00:09<04:56, 11.30it/s]
分批tokenize目标句子: 3%|▎ | 110/3460 [00:09<04:58, 11.23it/s]
分批tokenize目标句子: 3%|▎ | 112/3460 [00:09<04:58, 11.23it/s]
分批tokenize目标句子: 3%|▎ | 114/3460 [00:10<04:57, 11.23it/s]
分批tokenize目标句子: 3%|▎ | 116/3460 [00:10<04:57, 11.24it/s]
分批tokenize目标句子: 3%|▎ | 118/3460 [00:10<04:56, 11.27it/s]
分批tokenize目标句子: 3%|▎ | 120/3460 [00:10<04:56, 11.28it/s]
分批tokenize目标句子: 4%|▎ | 122/3460 [00:10<04:57, 11.22it/s]
分批tokenize目标句子: 4%|▎ | 124/3460 [00:10<05:00, 11.09it/s]
分批tokenize目标句子: 4%|▎ | 126/3460 [00:11<04:58, 11.15it/s]
分批tokenize目标句子: 4%|▎ | 128/3460 [00:11<04:56, 11.25it/s]
分批tokenize目标句子: 4%|▍ | 130/3460 [00:11<04:55, 11.28it/s]
分批tokenize目标句子: 4%|▍ | 132/3460 [00:11<04:53, 11.32it/s]
分批tokenize目标句子: 4%|▍ | 134/3460 [00:11<04:53, 11.33it/s]
分批tokenize目标句子: 4%|▍ | 136/3460 [00:12<04:55, 11.26it/s]
分批tokenize目标句子: 4%|▍ | 138/3460 [00:12<04:53, 11.32it/s]
分批tokenize目标句子: 4%|▍ | 140/3460 [00:12<04:53, 11.32it/s]
分批tokenize目标句子: 4%|▍ | 142/3460 [00:12<04:54, 11.27it/s]
分批tokenize目标句子: 4%|▍ | 144/3460 [00:12<04:54, 11.26it/s]
分批tokenize目标句子: 4%|▍ | 146/3460 [00:12<04:54, 11.26it/s]
分批tokenize目标句子: 4%|▍ | 148/3460 [00:13<04:53, 11.30it/s]
分批tokenize目标句子: 4%|▍ | 150/3460 [00:13<05:00, 11.01it/s]
分批tokenize目标句子: 4%|▍ | 152/3460 [00:13<04:55, 11.18it/s]
分批tokenize目标句子: 4%|▍ | 154/3460 [00:13<04:53, 11.27it/s]
分批tokenize目标句子: 5%|▍ | 156/3460 [00:13<04:52, 11.29it/s]
分批tokenize目标句子: 5%|▍ | 158/3460 [00:13<04:50, 11.37it/s]
分批tokenize目标句子: 5%|▍ | 160/3460 [00:14<04:49, 11.40it/s]
分批tokenize目标句子: 5%|▍ | 162/3460 [00:14<04:49, 11.39it/s]
分批tokenize目标句子: 5%|▍ | 164/3460 [00:14<04:49, 11.39it/s]
分批tokenize目标句子: 5%|▍ | 166/3460 [00:14<04:47, 11.47it/s]
分批tokenize目标句子: 5%|▍ | 168/3460 [00:14<04:44, 11.58it/s]
分批tokenize目标句子: 5%|▍ | 170/3460 [00:15<04:42, 11.63it/s]
分批tokenize目标句子: 5%|▍ | 172/3460 [00:15<04:41, 11.67it/s]
分批tokenize目标句子: 5%|▌ | 174/3460 [00:15<04:41, 11.65it/s]
分批tokenize目标句子: 5%|▌ | 176/3460 [00:15<04:41, 11.66it/s]
分批tokenize目标句子: 5%|▌ | 178/3460 [00:15<04:42, 11.60it/s]
分批tokenize目标句子: 5%|▌ | 180/3460 [00:15<04:42, 11.62it/s]
分批tokenize目标句子: 5%|▌ | 182/3460 [00:16<04:43, 11.57it/s]
分批tokenize目标句子: 5%|▌ | 184/3460 [00:16<04:42, 11.59it/s]
分批tokenize目标句子: 5%|▌ | 184/3460 [00:26<04:42, 11.59it/s]
分批tokenize目标句子: 5%|▌ | 185/3460 [00:26<1:41:51, 1.87s/it]
分批tokenize目标句子: 5%|▌ | 187/3460 [00:26<1:09:14, 1.27s/it]
分批tokenize目标句子: 5%|▌ | 189/3460 [00:26<48:20, 1.13it/s]
分批tokenize目标句子: 6%|▌ | 191/3460 [00:26<34:29, 1.58it/s]
分批tokenize目标句子: 6%|▌ | 193/3460 [00:27<25:11, 2.16it/s]
分批tokenize目标句子: 6%|▌ | 195/3460 [00:27<18:51, 2.89it/s]
分批tokenize目标句子: 6%|▌ | 197/3460 [00:27<14:31, 3.74it/s]
分批tokenize目标句子: 6%|▌ | 199/3460 [00:27<11:32, 4.71it/s]
分批tokenize目标句子: 6%|▌ | 201/3460 [00:27<09:26, 5.76it/s]
分批tokenize目标句子: 6%|▌ | 203/3460 [00:27<07:58, 6.81it/s]
分批tokenize目标句子: 6%|▌ | 205/3460 [00:28<06:57, 7.80it/s]
分批tokenize目标句子: 6%|▌ | 207/3460 [00:28<06:14, 8.68it/s]
分批tokenize目标句子: 6%|▌ | 209/3460 [00:28<05:45, 9.42it/s]
分批tokenize目标句子: 6%|▌ | 211/3460 [00:28<05:25, 9.98it/s]
分批tokenize目标句子: 6%|▌ | 213/3460 [00:28<05:10, 10.46it/s]
分批tokenize目标句子: 6%|▌ | 215/3460 [00:28<05:01, 10.78it/s]
分批tokenize目标句子: 6%|▋ | 217/3460 [00:29<04:55, 10.99it/s]
分批tokenize目标句子: 6%|▋ | 219/3460 [00:29<04:50, 11.14it/s]
分批tokenize目标句子: 6%|▋ | 221/3460 [00:29<04:47, 11.28it/s]
分批tokenize目标句子: 6%|▋ | 223/3460 [00:29<04:46, 11.29it/s]
分批tokenize目标句子: 7%|▋ | 225/3460 [00:29<04:42, 11.46it/s]
分批tokenize目标句子: 7%|▋ | 227/3460 [00:30<04:40, 11.53it/s]
分批tokenize目标句子: 7%|▋ | 229/3460 [00:30<04:42, 11.43it/s]
分批tokenize目标句子: 7%|▋ | 231/3460 [00:30<04:43, 11.37it/s]
分批tokenize目标句子: 7%|▋ | 233/3460 [00:30<04:49, 11.13it/s]
分批tokenize目标句子: 7%|▋ | 235/3460 [00:30<04:52, 11.03it/s]
分批tokenize目标句子: 7%|▋ | 237/3460 [00:30<04:48, 11.19it/s]
分批tokenize目标句子: 7%|▋ | 239/3460 [00:31<04:43, 11.35it/s]
分批tokenize目标句子: 7%|▋ | 241/3460 [00:31<04:41, 11.44it/s]
分批tokenize目标句子: 7%|▋ | 243/3460 [00:31<04:42, 11.40it/s]
分批tokenize目标句子: 7%|▋ | 245/3460 [00:31<04:40, 11.46it/s]
分批tokenize目标句子: 7%|▋ | 247/3460 [00:31<04:43, 11.33it/s]
分批tokenize目标句子: 7%|▋ | 249/3460 [00:31<04:43, 11.34it/s]
分批tokenize目标句子: 7%|▋ | 251/3460 [00:32<04:41, 11.41it/s]
分批tokenize目标句子: 7%|▋ | 253/3460 [00:32<04:42, 11.35it/s]
分批tokenize目标句子: 7%|▋ | 255/3460 [00:32<04:42, 11.35it/s]
分批tokenize目标句子: 7%|▋ | 257/3460 [00:32<04:41, 11.37it/s]
分批tokenize目标句子: 7%|▋ | 259/3460 [00:32<04:38, 11.49it/s]
分批tokenize目标句子: 8%|▊ | 261/3460 [00:33<04:42, 11.33it/s]
分批tokenize目标句子: 8%|▊ | 263/3460 [00:33<04:38, 11.48it/s]
分批tokenize目标句子: 8%|▊ | 265/3460 [00:33<04:38, 11.45it/s]
分批tokenize目标句子: 8%|▊ | 267/3460 [00:33<04:40, 11.40it/s]
分批tokenize目标句子: 8%|▊ | 269/3460 [00:33<04:37, 11.48it/s]
分批tokenize目标句子: 8%|▊ | 271/3460 [00:33<04:39, 11.39it/s]
分批tokenize目标句子: 8%|▊ | 273/3460 [00:34<04:40, 11.37it/s]
分批tokenize目标句子: 8%|▊ | 275/3460 [00:34<04:41, 11.32it/s]
分批tokenize目标句子: 8%|▊ | 277/3460 [00:34<04:39, 11.37it/s]
分批tokenize目标句子: 8%|▊ | 279/3460 [00:34<04:39, 11.37it/s]
分批tokenize目标句子: 8%|▊ | 281/3460 [00:34<04:40, 11.32it/s]
分批tokenize目标句子: 8%|▊ | 283/3460 [00:34<04:39, 11.36it/s]
分批tokenize目标句子: 8%|▊ | 285/3460 [00:35<04:38, 11.40it/s]
分批tokenize目标句子: 8%|▊ | 287/3460 [00:35<04:36, 11.46it/s]
分批tokenize目标句子: 8%|▊ | 289/3460 [00:35<04:35, 11.51it/s]
分批tokenize目标句子: 8%|▊ | 291/3460 [00:35<04:39, 11.34it/s]
分批tokenize目标句子: 8%|▊ | 293/3460 [00:35<04:39, 11.32it/s]
分批tokenize目标句子: 9%|▊ | 295/3460 [00:36<04:44, 11.13it/s]
分批tokenize目标句子: 9%|▊ | 297/3460 [00:36<04:45, 11.08it/s]
分批tokenize目标句子: 9%|▊ | 299/3460 [00:36<04:45, 11.07it/s]
分批tokenize目标句子: 9%|▊ | 301/3460 [00:36<04:45, 11.05it/s]
分批tokenize目标句子: 9%|▉ | 303/3460 [00:36<04:45, 11.07it/s]
分批tokenize目标句子: 9%|▉ | 305/3460 [00:36<04:45, 11.04it/s]
分批tokenize目标句子: 9%|▉ | 307/3460 [00:37<04:44, 11.08it/s]
分批tokenize目标句子: 9%|▉ | 309/3460 [00:37<04:45, 11.04it/s]
分批tokenize目标句子: 9%|▉ | 311/3460 [00:37<04:46, 11.00it/s]
分批tokenize目标句子: 9%|▉ | 313/3460 [00:37<04:43, 11.09it/s]
分批tokenize目标句子: 9%|▉ | 315/3460 [00:37<04:45, 11.02it/s]
分批tokenize目标句子: 9%|▉ | 317/3460 [00:37<04:43, 11.08it/s]
分批tokenize目标句子: 9%|▉ | 319/3460 [00:38<04:43, 11.08it/s]
分批tokenize目标句子: 9%|▉ | 321/3460 [00:38<04:41, 11.16it/s]
分批tokenize目标句子: 9%|▉ | 323/3460 [00:38<04:42, 11.12it/s]
分批tokenize目标句子: 9%|▉ | 325/3460 [00:38<04:41, 11.12it/s]
分批tokenize目标句子: 9%|▉ | 327/3460 [00:38<04:40, 11.16it/s]
分批tokenize目标句子: 10%|▉ | 329/3460 [00:39<04:39, 11.21it/s]
分批tokenize目标句子: 10%|▉ | 331/3460 [00:39<04:37, 11.28it/s]
分批tokenize目标句子: 10%|▉ | 333/3460 [00:39<04:36, 11.30it/s]
分批tokenize目标句子: 10%|▉ | 335/3460 [00:39<04:36, 11.30it/s]
分批tokenize目标句子: 10%|▉ | 337/3460 [00:39<04:46, 10.91it/s]
分批tokenize目标句子: 10%|▉ | 339/3460 [00:39<04:42, 11.03it/s]
分批tokenize目标句子: 10%|▉ | 341/3460 [00:40<04:42, 11.06it/s]
分批tokenize目标句子: 10%|▉ | 343/3460 [00:40<04:41, 11.09it/s]
分批tokenize目标句子: 10%|▉ | 345/3460 [00:40<04:39, 11.14it/s]
分批tokenize目标句子: 10%|█ | 347/3460 [00:40<04:37, 11.23it/s]
分批tokenize目标句子: 10%|█ | 349/3460 [00:40<04:37, 11.22it/s]
分批tokenize目标句子: 10%|█ | 351/3460 [00:41<04:35, 11.27it/s]
分批tokenize目标句子: 10%|█ | 353/3460 [00:41<04:35, 11.28it/s]
分批tokenize目标句子: 10%|█ | 355/3460 [00:41<04:33, 11.33it/s]
分批tokenize目标句子: 10%|█ | 357/3460 [00:41<04:34, 11.32it/s]
分批tokenize目标句子: 10%|█ | 359/3460 [00:41<04:33, 11.35it/s]
分批tokenize目标句子: 10%|█ | 361/3460 [00:41<04:34, 11.29it/s]
分批tokenize目标句子: 10%|█ | 363/3460 [00:42<04:33, 11.33it/s]
分批tokenize目标句子: 11%|█ | 365/3460 [00:42<04:34, 11.28it/s]
分批tokenize目标句子: 11%|█ | 367/3460 [00:42<04:35, 11.21it/s]
分批tokenize目标句子: 11%|█ | 369/3460 [00:42<04:35, 11.23it/s]
分批tokenize目标句子: 11%|█ | 371/3460 [00:42<04:33, 11.29it/s]
分批tokenize目标句子: 11%|█ | 373/3460 [00:42<04:32, 11.32it/s]
分批tokenize目标句子: 11%|█ | 375/3460 [00:43<04:37, 11.12it/s]
分批tokenize目标句子: 11%|█ | 377/3460 [00:43<04:39, 11.05it/s]
分批tokenize目标句子: 11%|█ | 379/3460 [00:43<04:36, 11.14it/s]
分批tokenize目标句子: 11%|█ | 381/3460 [00:43<04:36, 11.12it/s]
分批tokenize目标句子: 11%|█ | 383/3460 [00:43<04:34, 11.22it/s]
分批tokenize目标句子: 11%|█ | 385/3460 [00:44<04:33, 11.23it/s]
分批tokenize目标句子: 11%|█ | 387/3460 [00:44<04:33, 11.24it/s]
分批tokenize目标句子: 11%|█ | 389/3460 [00:44<04:34, 11.17it/s]
分批tokenize目标句子: 11%|█▏ | 391/3460 [00:44<04:41, 10.92it/s]
分批tokenize目标句子: 11%|█▏ | 393/3460 [00:44<04:39, 10.97it/s]
分批tokenize目标句子: 11%|█▏ | 395/3460 [00:44<04:35, 11.13it/s]
分批tokenize目标句子: 11%|█▏ | 397/3460 [00:45<04:39, 10.95it/s]
分批tokenize目标句子: 12%|█▏ | 399/3460 [00:45<04:46, 10.68it/s]
分批tokenize目标句子: 12%|█▏ | 401/3460 [00:45<04:47, 10.64it/s]
分批tokenize目标句子: 12%|█▏ | 403/3460 [00:45<04:44, 10.74it/s]
分批tokenize目标句子: 12%|█▏ | 405/3460 [00:45<04:47, 10.62it/s]
分批tokenize目标句子: 12%|█▏ | 407/3460 [00:46<04:44, 10.73it/s]
分批tokenize目标句子: 12%|█▏ | 409/3460 [00:46<04:41, 10.82it/s]
分批tokenize目标句子: 12%|█▏ | 411/3460 [00:46<04:40, 10.87it/s]
分批tokenize目标句子: 12%|█▏ | 413/3460 [00:46<04:38, 10.95it/s]
分批tokenize目标句子: 12%|█▏ | 415/3460 [00:46<04:37, 10.96it/s]
分批tokenize目标句子: 12%|█▏ | 417/3460 [00:47<04:39, 10.89it/s]
分批tokenize目标句子: 12%|█▏ | 419/3460 [00:47<04:34, 11.09it/s]
分批tokenize目标句子: 12%|█▏ | 421/3460 [00:47<04:29, 11.29it/s]
分批tokenize目标句子: 12%|█▏ | 423/3460 [00:47<04:27, 11.37it/s]
分批tokenize目标句子: 12%|█▏ | 425/3460 [00:47<04:27, 11.36it/s]
分批tokenize目标句子: 12%|█▏ | 427/3460 [00:47<04:27, 11.32it/s]
分批tokenize目标句子: 12%|█▏ | 429/3460 [00:48<04:31, 11.18it/s]
分批tokenize目标句子: 12%|█▏ | 431/3460 [00:48<04:37, 10.90it/s]
分批tokenize目标句子: 13%|█▎ | 433/3460 [00:48<04:37, 10.89it/s]
分批tokenize目标句子: 13%|█▎ | 435/3460 [00:48<04:40, 10.77it/s]
分批tokenize目标句子: 13%|█▎ | 437/3460 [00:48<04:34, 11.02it/s]
分批tokenize目标句子: 13%|█▎ | 439/3460 [00:48<04:32, 11.09it/s]
分批tokenize目标句子: 13%|█▎ | 441/3460 [00:49<04:31, 11.14it/s]
分批tokenize目标句子: 13%|█▎ | 443/3460 [00:49<04:28, 11.24it/s]
分批tokenize目标句子: 13%|█▎ | 445/3460 [00:49<04:26, 11.30it/s]
分批tokenize目标句子: 13%|█▎ | 447/3460 [00:49<04:29, 11.20it/s]
分批tokenize目标句子: 13%|█▎ | 449/3460 [00:49<04:28, 11.23it/s]
分批tokenize目标句子: 13%|█▎ | 451/3460 [00:50<04:26, 11.31it/s]
分批tokenize目标句子: 13%|█▎ | 453/3460 [00:50<04:25, 11.32it/s]
分批tokenize目标句子: 13%|█▎ | 455/3460 [00:50<04:23, 11.41it/s]
分批tokenize目标句子: 13%|█▎ | 457/3460 [00:50<04:25, 11.32it/s]
分批tokenize目标句子: 13%|█▎ | 459/3460 [00:50<04:25, 11.31it/s]
分批tokenize目标句子: 13%|█▎ | 461/3460 [00:50<04:25, 11.29it/s]
分批tokenize目标句子: 13%|█▎ | 463/3460 [00:51<04:30, 11.10it/s]
分批tokenize目标句子: 13%|█▎ | 465/3460 [00:51<04:29, 11.10it/s]
分批tokenize目标句子: 13%|█▎ | 467/3460 [00:51<04:30, 11.07it/s]
分批tokenize目标句子: 14%|█▎ | 469/3460 [00:51<04:30, 11.07it/s]
分批tokenize目标句子: 14%|█▎ | 471/3460 [00:51<04:28, 11.15it/s]
分批tokenize目标句子: 14%|█▎ | 473/3460 [00:52<04:30, 11.06it/s]
分批tokenize目标句子: 14%|█▎ | 475/3460 [00:52<04:27, 11.16it/s]
分批tokenize目标句子: 14%|█▍ | 477/3460 [00:52<04:27, 11.15it/s]
分批tokenize目标句子: 14%|█▍ | 479/3460 [00:52<04:25, 11.22it/s]
分批tokenize目标句子: 14%|█▍ | 481/3460 [00:52<04:25, 11.23it/s]
分批tokenize目标句子: 14%|█▍ | 483/3460 [00:52<04:25, 11.21it/s]
分批tokenize目标句子: 14%|█▍ | 485/3460 [00:53<04:24, 11.24it/s]
分批tokenize目标句子: 14%|█▍ | 487/3460 [00:53<04:24, 11.25it/s]
分批tokenize目标句子: 14%|█▍ | 489/3460 [00:53<04:24, 11.25it/s]
分批tokenize目标句子: 14%|█▍ | 491/3460 [00:53<04:25, 11.17it/s]
分批tokenize目标句子: 14%|█▍ | 493/3460 [00:53<04:24, 11.23it/s]
分批tokenize目标句子: 14%|█▍ | 495/3460 [00:53<04:21, 11.33it/s]
分批tokenize目标句子: 14%|█▍ | 497/3460 [00:54<04:21, 11.32it/s]
分批tokenize目标句子: 14%|█▍ | 499/3460 [00:54<04:20, 11.38it/s]
分批tokenize目标句子: 14%|█▍ | 501/3460 [00:54<04:22, 11.26it/s]
分批tokenize目标句子: 15%|█▍ | 503/3460 [00:54<04:21, 11.30it/s]
分批tokenize目标句子: 15%|█▍ | 505/3460 [00:54<04:21, 11.31it/s]
分批tokenize目标句子: 15%|█▍ | 507/3460 [00:55<04:19, 11.38it/s]
分批tokenize目标句子: 15%|█▍ | 509/3460 [00:55<04:19, 11.37it/s]
分批tokenize目标句子: 15%|█▍ | 511/3460 [00:55<04:19, 11.38it/s]
分批tokenize目标句子: 15%|█▍ | 513/3460 [00:55<04:21, 11.29it/s]
分批tokenize目标句子: 15%|█▍ | 515/3460 [00:55<04:25, 11.09it/s]
分批tokenize目标句子: 15%|█▍ | 517/3460 [00:55<04:24, 11.11it/s]
分批tokenize目标句子: 15%|█▌ | 519/3460 [00:56<04:28, 10.97it/s]
分批tokenize目标句子: 15%|█▌ | 521/3460 [00:56<04:27, 11.01it/s]
分批tokenize目标句子: 15%|█▌ | 523/3460 [00:56<04:25, 11.08it/s]
分批tokenize目标句子: 15%|█▌ | 525/3460 [00:56<04:25, 11.06it/s]
分批tokenize目标句子: 15%|█▌ | 527/3460 [00:56<04:25, 11.06it/s]
分批tokenize目标句子: 15%|█▌ | 529/3460 [00:57<04:30, 10.82it/s]
分批tokenize目标句子: 15%|█▌ | 531/3460 [00:57<04:27, 10.93it/s]
分批tokenize目标句子: 15%|█▌ | 533/3460 [00:57<04:25, 11.04it/s]
分批tokenize目标句子: 15%|█▌ | 535/3460 [00:57<04:23, 11.08it/s]
分批tokenize目标句子: 16%|█▌ | 537/3460 [00:57<04:23, 11.10it/s]
分批tokenize目标句子: 16%|█▌ | 539/3460 [00:57<04:22, 11.13it/s]
分批tokenize目标句子: 16%|█▌ | 541/3460 [00:58<04:19, 11.24it/s]
分批tokenize目标句子: 16%|█▌ | 543/3460 [00:58<04:18, 11.30it/s]
分批tokenize目标句子: 16%|█▌ | 545/3460 [00:58<04:18, 11.29it/s]
分批tokenize目标句子: 16%|█▌ | 547/3460 [00:58<04:19, 11.21it/s]
分批tokenize目标句子: 16%|█▌ | 549/3460 [00:58<04:19, 11.24it/s]
分批tokenize目标句子: 16%|█▌ | 551/3460 [00:58<04:17, 11.28it/s]
分批tokenize目标句子: 16%|█▌ | 553/3460 [00:59<04:19, 11.21it/s]
分批tokenize目标句子: 16%|█▌ | 555/3460 [00:59<04:18, 11.23it/s]
分批tokenize目标句子: 16%|█▌ | 557/3460 [00:59<04:18, 11.23it/s]
分批tokenize目标句子: 16%|█▌ | 559/3460 [00:59<04:17, 11.25it/s]
分批tokenize目标句子: 16%|█▌ | 561/3460 [00:59<04:17, 11.26it/s]
分批tokenize目标句子: 16%|█▋ | 563/3460 [01:00<04:16, 11.27it/s]
分批tokenize目标句子: 16%|█▋ | 565/3460 [01:00<04:16, 11.29it/s]
分批tokenize目标句子: 16%|█▋ | 567/3460 [01:00<04:15, 11.30it/s]
分批tokenize目标句子: 16%|█▋ | 569/3460 [01:00<04:15, 11.30it/s]
分批tokenize目标句子: 17%|█▋ | 571/3460 [01:00<04:15, 11.30it/s]
分批tokenize目标句子: 17%|█▋ | 573/3460 [01:00<04:15, 11.32it/s]
分批tokenize目标句子: 17%|█▋ | 575/3460 [01:01<04:13, 11.39it/s]
分批tokenize目标句子: 17%|█▋ | 577/3460 [01:01<04:13, 11.38it/s]
分批tokenize目标句子: 17%|█▋ | 579/3460 [01:01<04:12, 11.41it/s]
分批tokenize目标句子: 17%|█▋ | 581/3460 [01:11<1:13:14, 1.53s/it]
分批tokenize目标句子: 17%|█▋ | 583/3460 [01:11<52:25, 1.09s/it]
分批tokenize目标句子: 17%|█▋ | 585/3460 [01:11<37:55, 1.26it/s]
分批tokenize目标句子: 17%|█▋ | 587/3460 [01:11<27:45, 1.72it/s]
分批tokenize目标句子: 17%|█▋ | 589/3460 [01:11<20:40, 2.31it/s]
分批tokenize目标句子: 17%|█▋ | 591/3460 [01:12<15:42, 3.04it/s]
分批tokenize目标句子: 17%|█▋ | 593/3460 [01:12<12:13, 3.91it/s]
分批tokenize目标句子: 17%|█▋ | 595/3460 [01:12<09:46, 4.88it/s]
分批tokenize目标句子: 17%|█▋ | 597/3460 [01:12<08:04, 5.91it/s]
分批tokenize目标句子: 17%|█▋ | 599/3460 [01:12<06:53, 6.92it/s]
分批tokenize目标句子: 17%|█▋ | 601/3460 [01:12<06:04, 7.84it/s]
分批tokenize目标句子: 17%|█▋ | 603/3460 [01:13<05:29, 8.67it/s]
分批tokenize目标句子: 17%|█▋ | 605/3460 [01:13<05:03, 9.42it/s]
分批tokenize目标句子: 18%|█▊ | 607/3460 [01:13<04:49, 9.87it/s]
分批tokenize目标句子: 18%|█▊ | 609/3460 [01:13<04:35, 10.35it/s]
分批tokenize目标句子: 18%|█▊ | 611/3460 [01:13<04:26, 10.69it/s]
分批tokenize目标句子: 18%|█▊ | 613/3460 [01:13<04:18, 11.01it/s]
分批tokenize目标句子: 18%|█▊ | 615/3460 [01:14<04:15, 11.13it/s]
分批tokenize目标句子: 18%|█▊ | 617/3460 [01:14<04:13, 11.22it/s]
分批tokenize目标句子: 18%|█▊ | 619/3460 [01:14<04:12, 11.26it/s]
分批tokenize目标句子: 18%|█▊ | 621/3460 [01:14<04:09, 11.36it/s]
分批tokenize目标句子: 18%|█▊ | 623/3460 [01:14<04:07, 11.45it/s]
分批tokenize目标句子: 18%|█▊ | 625/3460 [01:15<04:07, 11.46it/s]
分批tokenize目标句子: 18%|█▊ | 627/3460 [01:15<04:06, 11.49it/s]
分批tokenize目标句子: 18%|█▊ | 629/3460 [01:15<04:07, 11.45it/s]
分批tokenize目标句子: 18%|█▊ | 631/3460 [01:15<04:07, 11.45it/s]
分批tokenize目标句子: 18%|█▊ | 633/3460 [01:15<04:05, 11.54it/s]
分批tokenize目标句子: 18%|█▊ | 635/3460 [01:15<04:05, 11.51it/s]
分批tokenize目标句子: 18%|█▊ | 637/3460 [01:16<04:06, 11.44it/s]
分批tokenize目标句子: 18%|█▊ | 639/3460 [01:16<04:07, 11.39it/s]
分批tokenize目标句子: 19%|█▊ | 641/3460 [01:16<04:06, 11.43it/s]
分批tokenize目标句子: 19%|█▊ | 643/3460 [01:16<04:08, 11.32it/s]
分批tokenize目标句子: 19%|█▊ | 645/3460 [01:16<04:09, 11.26it/s]
分批tokenize目标句子: 19%|█▊ | 647/3460 [01:16<04:08, 11.30it/s]
分批tokenize目标句子: 19%|█▉ | 649/3460 [01:17<04:07, 11.34it/s]
分批tokenize目标句子: 19%|█▉ | 651/3460 [01:17<04:05, 11.46it/s]
分批tokenize目标句子: 19%|█▉ | 653/3460 [01:17<04:09, 11.26it/s]
分批tokenize目标句子: 19%|█▉ | 655/3460 [01:17<04:08, 11.31it/s]
分批tokenize目标句子: 19%|█▉ | 657/3460 [01:17<04:06, 11.39it/s]
分批tokenize目标句子: 19%|█▉ | 659/3460 [01:18<04:08, 11.26it/s]
分批tokenize目标句子: 19%|█▉ | 661/3460 [01:18<04:08, 11.25it/s]
分批tokenize目标句子: 19%|█▉ | 663/3460 [01:18<04:11, 11.11it/s]
分批tokenize目标句子: 19%|█▉ | 665/3460 [01:18<04:11, 11.12it/s]
分批tokenize目标句子: 19%|█▉ | 667/3460 [01:18<04:09, 11.19it/s]
分批tokenize目标句子: 19%|█▉ | 669/3460 [01:18<04:11, 11.12it/s]
分批tokenize目标句子: 19%|█▉ | 671/3460 [01:19<04:13, 10.99it/s]
分批tokenize目标句子: 19%|█▉ | 673/3460 [01:19<04:15, 10.91it/s]
分批tokenize目标句子: 20%|█▉ | 675/3460 [01:19<04:10, 11.10it/s]
分批tokenize目标句子: 20%|█▉ | 677/3460 [01:19<04:11, 11.05it/s]
分批tokenize目标句子: 20%|█▉ | 679/3460 [01:19<04:11, 11.04it/s]
分批tokenize目标句子: 20%|█▉ | 681/3460 [01:20<04:10, 11.10it/s]
分批tokenize目标句子: 20%|█▉ | 683/3460 [01:20<04:09, 11.12it/s]
分批tokenize目标句子: 20%|█▉ | 685/3460 [01:20<04:08, 11.18it/s]
分批tokenize目标句子: 20%|█▉ | 687/3460 [01:20<04:05, 11.31it/s]
分批tokenize目标句子: 20%|█▉ | 689/3460 [01:20<04:04, 11.35it/s]
分批tokenize目标句子: 20%|█▉ | 691/3460 [01:20<04:02, 11.43it/s]
分批tokenize目标句子: 20%|██ | 693/3460 [01:21<04:00, 11.53it/s]
分批tokenize目标句子: 20%|██ | 695/3460 [01:21<03:58, 11.60it/s]
分批tokenize目标句子: 20%|██ | 697/3460 [01:21<03:58, 11.58it/s]
分批tokenize目标句子: 20%|██ | 699/3460 [01:21<03:57, 11.62it/s]
分批tokenize目标句子: 20%|██ | 701/3460 [01:21<03:59, 11.54it/s]
分批tokenize目标句子: 20%|██ | 703/3460 [01:21<03:59, 11.52it/s]
分批tokenize目标句子: 20%|██ | 705/3460 [01:22<03:59, 11.50it/s]
分批tokenize目标句子: 20%|██ | 707/3460 [01:22<04:06, 11.18it/s]
分批tokenize目标句子: 20%|██ | 709/3460 [01:22<04:07, 11.10it/s]
分批tokenize目标句子: 21%|██ | 711/3460 [01:22<04:08, 11.06it/s]
分批tokenize目标句子: 21%|██ | 713/3460 [01:22<04:08, 11.04it/s]
分批tokenize目标句子: 21%|██ | 715/3460 [01:23<04:15, 10.75it/s]
分批tokenize目标句子: 21%|██ | 717/3460 [01:23<04:14, 10.78it/s]
分批tokenize目标句子: 21%|██ | 719/3460 [01:23<04:11, 10.92it/s]
分批tokenize目标句子: 21%|██ | 721/3460 [01:23<04:07, 11.06it/s]
分批tokenize目标句子: 21%|██ | 723/3460 [01:23<04:07, 11.08it/s]
分批tokenize目标句子: 21%|██ | 725/3460 [01:23<04:06, 11.07it/s]
分批tokenize目标句子: 21%|██ | 727/3460 [01:24<04:09, 10.97it/s]
分批tokenize目标句子: 21%|██ | 729/3460 [01:24<04:06, 11.06it/s]
分批tokenize目标句子: 21%|██ | 731/3460 [01:24<04:04, 11.15it/s]
分批tokenize目标句子: 21%|██ | 733/3460 [01:24<04:02, 11.24it/s]
分批tokenize目标句子: 21%|██ | 735/3460 [01:24<04:01, 11.27it/s]
分批tokenize目标句子: 21%|██▏ | 737/3460 [01:25<04:04, 11.12it/s]
分批tokenize目标句子: 21%|██▏ | 739/3460 [01:25<04:04, 11.14it/s]
分批tokenize目标句子: 21%|██▏ | 741/3460 [01:25<04:09, 10.89it/s]
分批tokenize目标句子: 21%|██▏ | 743/3460 [01:25<04:09, 10.88it/s]
分批tokenize目标句子: 22%|██▏ | 745/3460 [01:25<04:07, 10.95it/s]
分批tokenize目标句子: 22%|██▏ | 747/3460 [01:25<04:05, 11.04it/s]
分批tokenize目标句子: 22%|██▏ | 749/3460 [01:26<04:09, 10.86it/s]
分批tokenize目标句子: 22%|██▏ | 751/3460 [01:26<04:04, 11.06it/s]
分批tokenize目标句子: 22%|██▏ | 753/3460 [01:26<04:04, 11.09it/s]
分批tokenize目标句子: 22%|██▏ | 755/3460 [01:26<04:05, 11.03it/s]
分批tokenize目标句子: 22%|██▏ | 757/3460 [01:26<04:02, 11.14it/s]
分批tokenize目标句子: 22%|██▏ | 759/3460 [01:27<04:02, 11.13it/s]
分批tokenize目标句子: 22%|██▏ | 761/3460 [01:27<04:03, 11.07it/s]
分批tokenize目标句子: 22%|██▏ | 763/3460 [01:27<04:07, 10.88it/s]
分批tokenize目标句子: 22%|██▏ | 765/3460 [01:27<04:06, 10.93it/s]
分批tokenize目标句子: 22%|██▏ | 767/3460 [01:27<04:03, 11.04it/s]
分批tokenize目标句子: 22%|██▏ | 769/3460 [01:27<04:02, 11.08it/s]
分批tokenize目标句子: 22%|██▏ | 771/3460 [01:28<04:01, 11.16it/s]
分批tokenize目标句子: 22%|██▏ | 773/3460 [01:28<04:02, 11.09it/s]
分批tokenize目标句子: 22%|██▏ | 775/3460 [01:28<04:00, 11.14it/s]
分批tokenize目标句子: 22%|██▏ | 777/3460 [01:28<04:00, 11.16it/s]
分批tokenize目标句子: 23%|██▎ | 779/3460 [01:28<03:58, 11.25it/s]
分批tokenize目标句子: 23%|██▎ | 781/3460 [01:28<03:56, 11.31it/s]
分批tokenize目标句子: 23%|██▎ | 783/3460 [01:29<03:56, 11.32it/s]
分批tokenize目标句子: 23%|██▎ | 785/3460 [01:29<03:56, 11.33it/s]
分批tokenize目标句子: 23%|██▎ | 787/3460 [01:29<03:56, 11.29it/s]
分批tokenize目标句子: 23%|██▎ | 789/3460 [01:29<03:56, 11.28it/s]
分批tokenize目标句子: 23%|██▎ | 791/3460 [01:29<04:00, 11.12it/s]
分批tokenize目标句子: 23%|██▎ | 793/3460 [01:30<04:00, 11.11it/s]
分批tokenize目标句子: 23%|██▎ | 795/3460 [01:30<03:59, 11.14it/s]
分批tokenize目标句子: 23%|██▎ | 797/3460 [01:30<03:58, 11.19it/s]
分批tokenize目标句子: 23%|██▎ | 799/3460 [01:30<03:56, 11.25it/s]
分批tokenize目标句子: 23%|██▎ | 801/3460 [01:30<03:56, 11.25it/s]
分批tokenize目标句子: 23%|██▎ | 803/3460 [01:30<03:55, 11.29it/s]
分批tokenize目标句子: 23%|██▎ | 805/3460 [01:31<03:54, 11.34it/s]
分批tokenize目标句子: 23%|██▎ | 807/3460 [01:31<03:53, 11.38it/s]
分批tokenize目标句子: 23%|██▎ | 809/3460 [01:31<03:52, 11.41it/s]
分批tokenize目标句子: 23%|██▎ | 811/3460 [01:31<03:51, 11.46it/s]
分批tokenize目标句子: 23%|██▎ | 813/3460 [01:31<03:52, 11.40it/s]
分批tokenize目标句子: 24%|██▎ | 815/3460 [01:31<03:52, 11.38it/s]
分批tokenize目标句子: 24%|██▎ | 817/3460 [01:32<03:52, 11.35it/s]
分批tokenize目标句子: 24%|██▎ | 819/3460 [01:32<03:52, 11.35it/s]
分批tokenize目标句子: 24%|██▎ | 821/3460 [01:32<03:53, 11.29it/s]
分批tokenize目标句子: 24%|██▍ | 823/3460 [01:32<03:53, 11.30it/s]
分批tokenize目标句子: 24%|██▍ | 825/3460 [01:32<03:51, 11.38it/s]
分批tokenize目标句子: 24%|██▍ | 827/3460 [01:33<03:50, 11.40it/s]
分批tokenize目标句子: 24%|██▍ | 829/3460 [01:33<03:51, 11.34it/s]
分批tokenize目标句子: 24%|██▍ | 831/3460 [01:33<03:51, 11.34it/s]
分批tokenize目标句子: 24%|██▍ | 833/3460 [01:33<03:51, 11.35it/s]
分批tokenize目标句子: 24%|██▍ | 835/3460 [01:33<03:52, 11.31it/s]
分批tokenize目标句子: 24%|██▍ | 837/3460 [01:33<03:48, 11.47it/s]
分批tokenize目标句子: 24%|██▍ | 839/3460 [01:34<03:49, 11.40it/s]
分批tokenize目标句子: 24%|██▍ | 841/3460 [01:34<03:48, 11.44it/s]
分批tokenize目标句子: 24%|██▍ | 843/3460 [01:34<03:50, 11.37it/s]
分批tokenize目标句子: 24%|██▍ | 845/3460 [01:34<03:49, 11.40it/s]
分批tokenize目标句子: 24%|██▍ | 847/3460 [01:34<03:49, 11.38it/s]
分批tokenize目标句子: 25%|██▍ | 849/3460 [01:34<03:49, 11.36it/s]
分批tokenize目标句子: 25%|██▍ | 851/3460 [01:35<03:49, 11.37it/s]
分批tokenize目标句子: 25%|██▍ | 853/3460 [01:35<03:50, 11.33it/s]
分批tokenize目标句子: 25%|██▍ | 855/3460 [01:35<03:50, 11.29it/s]
分批tokenize目标句子: 25%|██▍ | 857/3460 [01:35<03:49, 11.34it/s]
分批tokenize目标句子: 25%|██▍ | 859/3460 [01:35<03:48, 11.37it/s]
分批tokenize目标句子: 25%|██▍ | 861/3460 [01:36<03:46, 11.47it/s]
分批tokenize目标句子: 25%|██▍ | 863/3460 [01:36<03:50, 11.29it/s]
分批tokenize目标句子: 25%|██▌ | 865/3460 [01:36<03:47, 11.43it/s]
分批tokenize目标句子: 25%|██▌ | 867/3460 [01:36<03:45, 11.51it/s]
分批tokenize目标句子: 25%|██▌ | 869/3460 [01:36<03:43, 11.57it/s]
分批tokenize目标句子: 25%|██▌ | 871/3460 [01:36<03:43, 11.56it/s]
分批tokenize目标句子: 25%|██▌ | 873/3460 [01:37<03:43, 11.59it/s]
分批tokenize目标句子: 25%|██▌ | 875/3460 [01:37<03:44, 11.54it/s]
分批tokenize目标句子: 25%|██▌ | 877/3460 [01:37<03:44, 11.49it/s]
分批tokenize目标句子: 25%|██▌ | 879/3460 [01:37<03:44, 11.48it/s]
分批tokenize目标句子: 25%|██▌ | 881/3460 [01:37<03:44, 11.47it/s]
分批tokenize目标句子: 26%|██▌ | 883/3460 [01:37<03:44, 11.48it/s]
分批tokenize目标句子: 26%|██▌ | 885/3460 [01:38<03:43, 11.54it/s]
分批tokenize目标句子: 26%|██▌ | 887/3460 [01:38<03:42, 11.59it/s]
分批tokenize目标句子: 26%|██▌ | 889/3460 [01:38<03:41, 11.61it/s]
分批tokenize目标句子: 26%|██▌ | 891/3460 [01:38<03:40, 11.64it/s]
分批tokenize目标句子: 26%|██▌ | 893/3460 [01:38<03:40, 11.64it/s]
分批tokenize目标句子: 26%|██▌ | 895/3460 [01:38<03:41, 11.59it/s]
分批tokenize目标句子: 26%|██▌ | 897/3460 [01:39<03:40, 11.61it/s]
分批tokenize目标句子: 26%|██▌ | 899/3460 [01:39<03:39, 11.64it/s]
分批tokenize目标句子: 26%|██▌ | 901/3460 [01:39<03:40, 11.63it/s]
分批tokenize目标句子: 26%|██▌ | 903/3460 [01:39<03:40, 11.59it/s]
分批tokenize目标句子: 26%|██▌ | 905/3460 [01:39<03:41, 11.52it/s]
分批tokenize目标句子: 26%|██▌ | 907/3460 [01:40<03:41, 11.51it/s]
分批tokenize目标句子: 26%|██▋ | 909/3460 [01:40<03:45, 11.30it/s]
分批tokenize目标句子: 26%|██▋ | 911/3460 [01:40<03:46, 11.25it/s]
分批tokenize目标句子: 26%|██▋ | 913/3460 [01:40<03:45, 11.28it/s]
分批tokenize目标句子: 26%|██▋ | 915/3460 [01:40<03:46, 11.22it/s]
分批tokenize目标句子: 27%|██▋ | 917/3460 [01:40<03:48, 11.14it/s]
分批tokenize目标句子: 27%|██▋ | 919/3460 [01:41<03:47, 11.17it/s]
分批tokenize目标句子: 27%|██▋ | 921/3460 [01:41<03:45, 11.24it/s]
分批tokenize目标句子: 27%|██▋ | 923/3460 [01:41<03:45, 11.27it/s]
分批tokenize目标句子: 27%|██▋ | 925/3460 [01:41<03:44, 11.27it/s]
分批tokenize目标句子: 27%|██▋ | 927/3460 [01:41<03:47, 11.14it/s]
分批tokenize目标句子: 27%|██▋ | 929/3460 [01:41<03:48, 11.10it/s]
分批tokenize目标句子: 27%|██▋ | 931/3460 [01:42<03:47, 11.13it/s]
分批tokenize目标句子: 27%|██▋ | 933/3460 [01:42<03:49, 11.00it/s]
分批tokenize目标句子: 27%|██▋ | 935/3460 [01:42<03:49, 11.02it/s]
分批tokenize目标句子: 27%|██▋ | 937/3460 [01:42<03:50, 10.95it/s]
分批tokenize目标句子: 27%|██▋ | 939/3460 [01:42<03:49, 10.98it/s]
分批tokenize目标句子: 27%|██▋ | 941/3460 [01:43<03:46, 11.11it/s]
分批tokenize目标句子: 27%|██▋ | 943/3460 [01:43<03:45, 11.17it/s]
分批tokenize目标句子: 27%|██▋ | 945/3460 [01:43<03:45, 11.17it/s]
分批tokenize目标句子: 27%|██▋ | 947/3460 [01:43<03:44, 11.21it/s]
分批tokenize目标句子: 27%|██▋ | 949/3460 [01:43<03:43, 11.25it/s]
分批tokenize目标句子: 27%|██▋ | 951/3460 [01:43<03:42, 11.26it/s]
分批tokenize目标句子: 28%|██▊ | 953/3460 [01:44<03:42, 11.26it/s]
分批tokenize目标句子: 28%|██▊ | 955/3460 [01:44<03:42, 11.28it/s]
分批tokenize目标句子: 28%|██▊ | 957/3460 [01:44<03:39, 11.38it/s]
分批tokenize目标句子: 28%|██▊ | 959/3460 [01:44<03:38, 11.44it/s]
分批tokenize目标句子: 28%|██▊ | 961/3460 [01:44<03:38, 11.45it/s]
分批tokenize目标句子: 28%|██▊ | 963/3460 [01:45<03:38, 11.43it/s]
分批tokenize目标句子: 28%|██▊ | 965/3460 [01:45<03:37, 11.45it/s]
分批tokenize目标句子: 28%|██▊ | 967/3460 [01:45<03:36, 11.49it/s]
分批tokenize目标句子: 28%|██▊ | 969/3460 [01:45<03:36, 11.50it/s]
分批tokenize目标句子: 28%|██▊ | 971/3460 [01:45<03:36, 11.50it/s]
分批tokenize目标句子: 28%|██▊ | 973/3460 [01:45<03:36, 11.49it/s]
分批tokenize目标句子: 28%|██▊ | 975/3460 [01:46<03:36, 11.48it/s]
分批tokenize目标句子: 28%|██▊ | 975/3460 [01:56<03:36, 11.48it/s]
分批tokenize目标句子: 28%|██▊ | 976/3460 [01:56<1:18:57, 1.91s/it]
分批tokenize目标句子: 28%|██▊ | 978/3460 [01:56<53:40, 1.30s/it]
分批tokenize目标句子: 28%|██▊ | 980/3460 [01:56<37:28, 1.10it/s]
分批tokenize目标句子: 28%|██▊ | 982/3460 [01:56<26:46, 1.54it/s]
分批tokenize目标句子: 28%|██▊ | 984/3460 [01:57<19:34, 2.11it/s]
分批tokenize目标句子: 28%|██▊ | 986/3460 [01:57<14:40, 2.81it/s]
分批tokenize目标句子: 29%|██▊ | 988/3460 [01:57<11:19, 3.64it/s]
分批tokenize目标句子: 29%|██▊ | 990/3460 [01:57<09:01, 4.56it/s]
分批tokenize目标句子: 29%|██▊ | 992/3460 [01:57<07:24, 5.55it/s]
分批tokenize目标句子: 29%|██▊ | 994/3460 [01:58<06:14, 6.59it/s]
分批tokenize目标句子: 29%|██▉ | 996/3460 [01:58<05:26, 7.55it/s]
分批tokenize目标句子: 29%|██▉ | 998/3460 [01:58<04:54, 8.37it/s]
分批tokenize目标句子: 29%|██▉ | 1000/3460 [01:58<04:32, 9.04it/s]
分批tokenize目标句子: 29%|██▉ | 1002/3460 [01:58<04:15, 9.64it/s]
分批tokenize目标句子: 29%|██▉ | 1004/3460 [01:58<04:06, 9.97it/s]
分批tokenize目标句子: 29%|██▉ | 1006/3460 [01:59<03:56, 10.36it/s]
分批tokenize目标句子: 29%|██▉ | 1008/3460 [01:59<03:51, 10.60it/s]
分批tokenize目标句子: 29%|██▉ | 1010/3460 [01:59<03:48, 10.70it/s]
分批tokenize目标句子: 29%|██▉ | 1012/3460 [01:59<03:45, 10.86it/s]
分批tokenize目标句子: 29%|██▉ | 1014/3460 [01:59<03:40, 11.08it/s]
分批tokenize目标句子: 29%|██▉ | 1016/3460 [02:00<03:40, 11.07it/s]
分批tokenize目标句子: 29%|██▉ | 1018/3460 [02:00<03:40, 11.09it/s]
分批tokenize目标句子: 29%|██▉ | 1020/3460 [02:00<03:38, 11.18it/s]
分批tokenize目标句子: 30%|██▉ | 1022/3460 [02:00<03:37, 11.20it/s]
分批tokenize目标句子: 30%|██▉ | 1024/3460 [02:00<03:38, 11.13it/s]
分批tokenize目标句子: 30%|██▉ | 1026/3460 [02:00<03:37, 11.18it/s]
分批tokenize目标句子: 30%|██▉ | 1028/3460 [02:01<03:38, 11.12it/s]
分批tokenize目标句子: 30%|██▉ | 1030/3460 [02:01<03:38, 11.14it/s]
分批tokenize目标句子: 30%|██▉ | 1032/3460 [02:01<03:36, 11.20it/s]
分批tokenize目标句子: 30%|██▉ | 1034/3460 [02:01<03:37, 11.16it/s]
分批tokenize目标句子: 30%|██▉ | 1036/3460 [02:01<03:37, 11.13it/s]
分批tokenize目标句子: 30%|███ | 1038/3460 [02:02<03:36, 11.17it/s]
分批tokenize目标句子: 30%|███ | 1040/3460 [02:02<03:39, 11.03it/s]
分批tokenize目标句子: 30%|███ | 1042/3460 [02:02<03:38, 11.08it/s]
分批tokenize目标句子: 30%|███ | 1044/3460 [02:02<03:39, 11.01it/s]
分批tokenize目标句子: 30%|███ | 1046/3460 [02:02<03:37, 11.08it/s]
分批tokenize目标句子: 30%|███ | 1048/3460 [02:02<03:37, 11.11it/s]
分批tokenize目标句子: 30%|███ | 1050/3460 [02:03<03:36, 11.12it/s]
分批tokenize目标句子: 30%|███ | 1052/3460 [02:03<03:38, 11.02it/s]
分批tokenize目标句子: 30%|███ | 1054/3460 [02:03<03:36, 11.10it/s]
分批tokenize目标句子: 31%|███ | 1056/3460 [02:03<03:36, 11.08it/s]
分批tokenize目标句子: 31%|███ | 1058/3460 [02:03<03:37, 11.05it/s]
分批tokenize目标句子: 31%|███ | 1060/3460 [02:03<03:37, 11.01it/s]
分批tokenize目标句子: 31%|███ | 1062/3460 [02:04<03:37, 11.01it/s]
分批tokenize目标句子: 31%|███ | 1064/3460 [02:04<03:37, 11.03it/s]
分批tokenize目标句子: 31%|███ | 1066/3460 [02:04<03:36, 11.07it/s]
分批tokenize目标句子: 31%|███ | 1068/3460 [02:04<03:35, 11.13it/s]
分批tokenize目标句子: 31%|███ | 1070/3460 [02:04<03:35, 11.08it/s]
分批tokenize目标句子: 31%|███ | 1072/3460 [02:05<03:34, 11.14it/s]
分批tokenize目标句子: 31%|███ | 1074/3460 [02:05<03:34, 11.15it/s]
分批tokenize目标句子: 31%|███ | 1076/3460 [02:05<03:33, 11.19it/s]
分批tokenize目标句子: 31%|███ | 1078/3460 [02:05<03:33, 11.18it/s]
分批tokenize目标句子: 31%|███ | 1080/3460 [02:05<03:32, 11.18it/s]
分批tokenize目标句子: 31%|███▏ | 1082/3460 [02:05<03:34, 11.08it/s]
分批tokenize目标句子: 31%|███▏ | 1084/3460 [02:06<03:34, 11.10it/s]
分批tokenize目标句子: 31%|███▏ | 1086/3460 [02:06<03:34, 11.05it/s]
分批tokenize目标句子: 31%|███▏ | 1088/3460 [02:06<03:35, 11.01it/s]
分批tokenize目标句子: 32%|███▏ | 1090/3460 [02:06<03:33, 11.08it/s]
分批tokenize目标句子: 32%|███▏ | 1092/3460 [02:06<03:35, 11.00it/s]
分批tokenize目标句子: 32%|███▏ | 1094/3460 [02:07<03:37, 10.87it/s]
分批tokenize目标句子: 32%|███▏ | 1096/3460 [02:07<03:36, 10.91it/s]
分批tokenize目标句子: 32%|███▏ | 1098/3460 [02:07<03:34, 11.02it/s]
分批tokenize目标句子: 32%|███▏ | 1100/3460 [02:07<03:34, 11.02it/s]
分批tokenize目标句子: 32%|███▏ | 1102/3460 [02:07<03:32, 11.09it/s]
分批tokenize目标句子: 32%|███▏ | 1104/3460 [02:07<03:32, 11.08it/s]
分批tokenize目标句子: 32%|███▏ | 1106/3460 [02:08<03:31, 11.16it/s]
分批tokenize目标句子: 32%|███▏ | 1108/3460 [02:08<03:31, 11.13it/s]
分批tokenize目标句子: 32%|███▏ | 1110/3460 [02:08<03:30, 11.17it/s]
分批tokenize目标句子: 32%|███▏ | 1112/3460 [02:08<03:30, 11.15it/s]
分批tokenize目标句子: 32%|███▏ | 1114/3460 [02:08<03:27, 11.30it/s]
分批tokenize目标句子: 32%|███▏ | 1116/3460 [02:09<03:26, 11.35it/s]
分批tokenize目标句子: 32%|███▏ | 1118/3460 [02:09<03:25, 11.39it/s]
分批tokenize目标句子: 32%|███▏ | 1120/3460 [02:09<03:25, 11.38it/s]
分批tokenize目标句子: 32%|███▏ | 1122/3460 [02:09<03:24, 11.41it/s]
分批tokenize目标句子: 32%|███▏ | 1124/3460 [02:09<03:25, 11.37it/s]
分批tokenize目标句子: 33%|███▎ | 1126/3460 [02:09<03:25, 11.36it/s]
分批tokenize目标句子: 33%|███▎ | 1128/3460 [02:10<03:26, 11.31it/s]
分批tokenize目标句子: 33%|███▎ | 1130/3460 [02:10<03:25, 11.34it/s]
分批tokenize目标句子: 33%|███▎ | 1132/3460 [02:10<03:25, 11.35it/s]
分批tokenize目标句子: 33%|███▎ | 1134/3460 [02:10<03:24, 11.39it/s]
分批tokenize目标句子: 33%|███▎ | 1136/3460 [02:10<03:23, 11.41it/s]
分批tokenize目标句子: 33%|███▎ | 1138/3460 [02:10<03:23, 11.41it/s]
分批tokenize目标句子: 33%|███▎ | 1140/3460 [02:11<03:24, 11.37it/s]
分批tokenize目标句子: 33%|███▎ | 1142/3460 [02:11<03:24, 11.36it/s]
分批tokenize目标句子: 33%|███▎ | 1144/3460 [02:11<03:23, 11.36it/s]
分批tokenize目标句子: 33%|███▎ | 1146/3460 [02:11<03:23, 11.36it/s]
分批tokenize目标句子: 33%|███▎ | 1148/3460 [02:11<03:23, 11.34it/s]
分批tokenize目标句子: 33%|███▎ | 1150/3460 [02:12<03:22, 11.39it/s]
分批tokenize目标句子: 33%|███▎ | 1152/3460 [02:12<03:22, 11.37it/s]
分批tokenize目标句子: 33%|███▎ | 1154/3460 [02:12<03:23, 11.32it/s]
分批tokenize目标句子: 33%|███▎ | 1156/3460 [02:12<03:23, 11.33it/s]
分批tokenize目标句子: 33%|███▎ | 1158/3460 [02:12<03:23, 11.33it/s]
分批tokenize目标句子: 34%|███▎ | 1160/3460 [02:12<03:24, 11.26it/s]
分批tokenize目标句子: 34%|███▎ | 1162/3460 [02:13<03:23, 11.27it/s]
分批tokenize目标句子: 34%|███▎ | 1164/3460 [02:13<03:23, 11.29it/s]
分批tokenize目标句子: 34%|███▎ | 1166/3460 [02:13<03:22, 11.34it/s]
分批tokenize目标句子: 34%|███▍ | 1168/3460 [02:13<03:22, 11.34it/s]
分批tokenize目标句子: 34%|███▍ | 1170/3460 [02:13<03:21, 11.37it/s]
分批tokenize目标句子: 34%|███▍ | 1172/3460 [02:13<03:21, 11.36it/s]
分批tokenize目标句子: 34%|███▍ | 1174/3460 [02:14<03:21, 11.37it/s]
分批tokenize目标句子: 34%|███▍ | 1176/3460 [02:14<03:21, 11.32it/s]
分批tokenize目标句子: 34%|███▍ | 1178/3460 [02:14<03:20, 11.39it/s]
分批tokenize目标句子: 34%|███▍ | 1180/3460 [02:14<03:21, 11.31it/s]
分批tokenize目标句子: 34%|███▍ | 1182/3460 [02:14<03:20, 11.35it/s]
分批tokenize目标句子: 34%|███▍ | 1184/3460 [02:15<03:23, 11.19it/s]
分批tokenize目标句子: 34%|███▍ | 1186/3460 [02:15<03:22, 11.24it/s]
分批tokenize目标句子: 34%|███▍ | 1188/3460 [02:15<03:22, 11.19it/s]
分批tokenize目标句子: 34%|███▍ | 1190/3460 [02:15<03:29, 10.83it/s]
分批tokenize目标句子: 34%|███▍ | 1192/3460 [02:15<03:30, 10.78it/s]
分批tokenize目标句子: 35%|███▍ | 1194/3460 [02:15<03:27, 10.94it/s]
分批tokenize目标句子: 35%|███▍ | 1196/3460 [02:16<03:25, 10.99it/s]
分批tokenize目标句子: 35%|███▍ | 1198/3460 [02:16<03:24, 11.07it/s]
分批tokenize目标句子: 35%|███▍ | 1200/3460 [02:16<03:24, 11.03it/s]
分批tokenize目标句子: 35%|███▍ | 1202/3460 [02:16<03:23, 11.11it/s]
分批tokenize目标句子: 35%|███▍ | 1204/3460 [02:16<03:23, 11.10it/s]
分批tokenize目标句子: 35%|███▍ | 1206/3460 [02:17<03:21, 11.17it/s]
分批tokenize目标句子: 35%|███▍ | 1208/3460 [02:17<03:20, 11.22it/s]
分批tokenize目标句子: 35%|███▍ | 1210/3460 [02:17<03:19, 11.29it/s]
分批tokenize目标句子: 35%|███▌ | 1212/3460 [02:17<03:18, 11.31it/s]
分批tokenize目标句子: 35%|███▌ | 1214/3460 [02:17<03:17, 11.38it/s]
分批tokenize目标句子: 35%|███▌ | 1216/3460 [02:17<03:18, 11.29it/s]
分批tokenize目标句子: 35%|███▌ | 1218/3460 [02:18<03:18, 11.28it/s]
分批tokenize目标句子: 35%|███▌ | 1220/3460 [02:18<03:21, 11.11it/s]
分批tokenize目标句子: 35%|███▌ | 1222/3460 [02:18<03:21, 11.09it/s]
分批tokenize目标句子: 35%|███▌ | 1224/3460 [02:18<03:26, 10.83it/s]
分批tokenize目标句子: 35%|███▌ | 1226/3460 [02:18<03:24, 10.93it/s]
分批tokenize目标句子: 35%|███▌ | 1228/3460 [02:19<03:25, 10.86it/s]
分批tokenize目标句子: 36%|███▌ | 1230/3460 [02:19<03:23, 10.93it/s]
分批tokenize目标句子: 36%|███▌ | 1232/3460 [02:19<03:22, 10.98it/s]
分批tokenize目标句子: 36%|███▌ | 1234/3460 [02:19<03:21, 11.06it/s]
分批tokenize目标句子: 36%|███▌ | 1236/3460 [02:19<03:20, 11.09it/s]
分批tokenize目标句子: 36%|███▌ | 1238/3460 [02:19<03:18, 11.20it/s]
分批tokenize目标句子: 36%|███▌ | 1240/3460 [02:20<03:20, 11.07it/s]
分批tokenize目标句子: 36%|███▌ | 1242/3460 [02:20<03:19, 11.11it/s]
分批tokenize目标句子: 36%|███▌ | 1244/3460 [02:20<03:18, 11.14it/s]
分批tokenize目标句子: 36%|███▌ | 1246/3460 [02:20<03:17, 11.19it/s]
分批tokenize目标句子: 36%|███▌ | 1248/3460 [02:20<03:17, 11.17it/s]
分批tokenize目标句子: 36%|███▌ | 1250/3460 [02:20<03:17, 11.20it/s]
分批tokenize目标句子: 36%|███▌ | 1252/3460 [02:21<03:17, 11.18it/s]
分批tokenize目标句子: 36%|███▌ | 1254/3460 [02:21<03:17, 11.16it/s]
分批tokenize目标句子: 36%|███▋ | 1256/3460 [02:21<03:19, 11.03it/s]
分批tokenize目标句子: 36%|███▋ | 1258/3460 [02:21<03:19, 11.03it/s]
分批tokenize目标句子: 36%|███▋ | 1260/3460 [02:21<03:19, 11.03it/s]
分批tokenize目标句子: 36%|███▋ | 1262/3460 [02:22<03:18, 11.09it/s]
分批tokenize目标句子: 37%|███▋ | 1264/3460 [02:22<03:16, 11.16it/s]
分批tokenize目标句子: 37%|███▋ | 1266/3460 [02:22<03:14, 11.26it/s]
分批tokenize目标句子: 37%|███▋ | 1268/3460 [02:22<03:13, 11.32it/s]
分批tokenize目标句子: 37%|███▋ | 1270/3460 [02:22<03:14, 11.25it/s]
分批tokenize目标句子: 37%|███▋ | 1272/3460 [02:22<03:14, 11.25it/s]
分批tokenize目标句子: 37%|███▋ | 1274/3460 [02:23<03:13, 11.27it/s]
分批tokenize目标句子: 37%|███▋ | 1276/3460 [02:23<03:12, 11.33it/s]
分批tokenize目标句子: 37%|███▋ | 1278/3460 [02:23<03:11, 11.42it/s]
分批tokenize目标句子: 37%|███▋ | 1280/3460 [02:23<03:11, 11.39it/s]
分批tokenize目标句子: 37%|███▋ | 1282/3460 [02:23<03:11, 11.35it/s]
分批tokenize目标句子: 37%|███▋ | 1284/3460 [02:23<03:11, 11.36it/s]
分批tokenize目标句子: 37%|███▋ | 1286/3460 [02:24<03:10, 11.38it/s]
分批tokenize目标句子: 37%|███▋ | 1288/3460 [02:24<03:11, 11.36it/s]
分批tokenize目标句子: 37%|███▋ | 1290/3460 [02:24<03:10, 11.40it/s]
分批tokenize目标句子: 37%|███▋ | 1292/3460 [02:24<03:09, 11.43it/s]
分批tokenize目标句子: 37%|███▋ | 1294/3460 [02:24<03:09, 11.41it/s]
分批tokenize目标句子: 37%|███▋ | 1296/3460 [02:25<03:10, 11.38it/s]
分批tokenize目标句子: 38%|███▊ | 1298/3460 [02:25<03:09, 11.39it/s]
分批tokenize目标句子: 38%|███▊ | 1300/3460 [02:25<03:09, 11.38it/s]
分批tokenize目标句子: 38%|███▊ | 1302/3460 [02:25<03:09, 11.41it/s]
分批tokenize目标句子: 38%|███▊ | 1304/3460 [02:25<03:09, 11.38it/s]
分批tokenize目标句子: 38%|███▊ | 1306/3460 [02:25<03:09, 11.35it/s]
分批tokenize目标句子: 38%|███▊ | 1308/3460 [02:26<03:10, 11.29it/s]
分批tokenize目标句子: 38%|███▊ | 1310/3460 [02:26<03:10, 11.30it/s]
分批tokenize目标句子: 38%|███▊ | 1312/3460 [02:26<03:11, 11.22it/s]
分批tokenize目标句子: 38%|███▊ | 1314/3460 [02:26<03:09, 11.34it/s]
分批tokenize目标句子: 38%|███▊ | 1316/3460 [02:26<03:08, 11.37it/s]
分批tokenize目标句子: 38%|███▊ | 1318/3460 [02:26<03:08, 11.34it/s]
分批tokenize目标句子: 38%|███▊ | 1320/3460 [02:27<03:09, 11.29it/s]
分批tokenize目标句子: 38%|███▊ | 1322/3460 [02:27<03:09, 11.31it/s]
分批tokenize目标句子: 38%|███▊ | 1324/3460 [02:27<03:08, 11.32it/s]
分批tokenize目标句子: 38%|███▊ | 1326/3460 [02:27<03:07, 11.35it/s]
分批tokenize目标句子: 38%|███▊ | 1328/3460 [02:27<03:09, 11.26it/s]
分批tokenize目标句子: 38%|███▊ | 1330/3460 [02:28<03:09, 11.25it/s]
分批tokenize目标句子: 38%|███▊ | 1332/3460 [02:28<03:09, 11.25it/s]
分批tokenize目标句子: 39%|███▊ | 1334/3460 [02:28<03:07, 11.31it/s]
分批tokenize目标句子: 39%|███▊ | 1336/3460 [02:28<03:10, 11.18it/s]
分批tokenize目标句子: 39%|███▊ | 1338/3460 [02:28<03:12, 11.03it/s]
分批tokenize目标句子: 39%|███▊ | 1340/3460 [02:28<03:10, 11.10it/s]
分批tokenize目标句子: 39%|███▉ | 1342/3460 [02:29<03:11, 11.07it/s]
分批tokenize目标句子: 39%|███▉ | 1344/3460 [02:29<03:11, 11.07it/s]
分批tokenize目标句子: 39%|███▉ | 1346/3460 [02:29<03:11, 11.02it/s]
分批tokenize目标句子: 39%|███▉ | 1348/3460 [02:29<03:11, 11.05it/s]
分批tokenize目标句子: 39%|███▉ | 1350/3460 [02:29<03:10, 11.10it/s]
分批tokenize目标句子: 39%|███▉ | 1352/3460 [02:30<03:14, 10.86it/s]
分批tokenize目标句子: 39%|███▉ | 1354/3460 [02:30<03:11, 10.99it/s]
分批tokenize目标句子: 39%|███▉ | 1356/3460 [02:30<03:12, 10.92it/s]
分批tokenize目标句子: 39%|███▉ | 1358/3460 [02:30<03:10, 11.05it/s]
分批tokenize目标句子: 39%|███▉ | 1360/3460 [02:30<03:10, 11.04it/s]
分批tokenize目标句子: 39%|███▉ | 1362/3460 [02:30<03:09, 11.06it/s]
分批tokenize目标句子: 39%|███▉ | 1364/3460 [02:31<03:08, 11.11it/s]
分批tokenize目标句子: 39%|███▉ | 1366/3460 [02:31<03:06, 11.23it/s]
分批tokenize目标句子: 40%|███▉ | 1368/3460 [02:31<03:04, 11.32it/s]
分批tokenize目标句子: 40%|███▉ | 1370/3460 [02:31<03:03, 11.40it/s]
分批tokenize目标句子: 40%|███▉ | 1372/3460 [02:31<03:02, 11.44it/s]
分批tokenize目标句子: 40%|███▉ | 1374/3460 [02:32<03:02, 11.44it/s]
分批tokenize目标句子: 40%|███▉ | 1376/3460 [02:32<03:03, 11.39it/s]
分批tokenize目标句子: 40%|███▉ | 1378/3460 [02:32<03:01, 11.44it/s]
分批tokenize目标句子: 40%|███▉ | 1380/3460 [02:32<03:02, 11.40it/s]
分批tokenize目标句子: 40%|███▉ | 1382/3460 [02:32<03:01, 11.45it/s]
分批tokenize目标句子: 40%|████ | 1384/3460 [02:32<03:02, 11.39it/s]
分批tokenize目标句子: 40%|████ | 1386/3460 [02:33<03:03, 11.33it/s]
分批tokenize目标句子: 40%|████ | 1388/3460 [02:33<03:01, 11.41it/s]
分批tokenize目标句子: 40%|████ | 1390/3460 [02:33<03:01, 11.39it/s]
分批tokenize目标句子: 40%|████ | 1392/3460 [02:33<03:02, 11.33it/s]
分批tokenize目标句子: 40%|████ | 1394/3460 [02:33<03:01, 11.37it/s]
分批tokenize目标句子: 40%|████ | 1396/3460 [02:33<03:02, 11.28it/s]
分批tokenize目标句子: 40%|████ | 1398/3460 [02:34<03:03, 11.22it/s]
分批tokenize目标句子: 40%|████ | 1400/3460 [02:34<03:04, 11.17it/s]
分批tokenize目标句子: 41%|████ | 1402/3460 [02:34<03:04, 11.18it/s]
分批tokenize目标句子: 41%|████ | 1404/3460 [02:34<03:04, 11.17it/s]
分批tokenize目标句子: 41%|████ | 1406/3460 [02:34<03:03, 11.20it/s]
分批tokenize目标句子: 41%|████ | 1408/3460 [02:35<03:02, 11.22it/s]
分批tokenize目标句子: 41%|████ | 1410/3460 [02:35<03:02, 11.23it/s]
分批tokenize目标句子: 41%|████ | 1412/3460 [02:35<03:04, 11.12it/s]
分批tokenize目标句子: 41%|████ | 1414/3460 [02:35<03:05, 11.05it/s]
分批tokenize目标句子: 41%|████ | 1416/3460 [02:35<03:07, 10.88it/s]
分批tokenize目标句子: 41%|████ | 1418/3460 [02:35<03:06, 10.94it/s]
分批tokenize目标句子: 41%|████ | 1420/3460 [02:36<03:04, 11.04it/s]
分批tokenize目标句子: 41%|████ | 1422/3460 [02:36<03:02, 11.16it/s]
分批tokenize目标句子: 41%|████ | 1424/3460 [02:36<03:01, 11.21it/s]
分批tokenize目标句子: 41%|████ | 1426/3460 [02:36<03:00, 11.26it/s]
分批tokenize目标句子: 41%|████▏ | 1428/3460 [02:36<02:59, 11.30it/s]
分批tokenize目标句子: 41%|████▏ | 1430/3460 [02:36<02:59, 11.34it/s]
分批tokenize目标句子: 41%|████▏ | 1432/3460 [02:37<02:59, 11.33it/s]
分批tokenize目标句子: 41%|████▏ | 1434/3460 [02:37<02:58, 11.37it/s]
分批tokenize目标句子: 42%|████▏ | 1436/3460 [02:37<02:58, 11.36it/s]
分批tokenize目标句子: 42%|████▏ | 1438/3460 [02:37<02:57, 11.40it/s]
分批tokenize目标句子: 42%|████▏ | 1440/3460 [02:37<02:57, 11.36it/s]
分批tokenize目标句子: 42%|████▏ | 1442/3460 [02:38<02:57, 11.40it/s]
分批tokenize目标句子: 42%|████▏ | 1444/3460 [02:38<02:57, 11.36it/s]
分批tokenize目标句子: 42%|████▏ | 1446/3460 [02:38<02:59, 11.24it/s]
分批tokenize目标句子: 42%|████▏ | 1448/3460 [02:38<02:58, 11.24it/s]
分批tokenize目标句子: 42%|████▏ | 1450/3460 [02:38<02:58, 11.28it/s]
分批tokenize目标句子: 42%|████▏ | 1452/3460 [02:38<02:58, 11.24it/s]
分批tokenize目标句子: 42%|████▏ | 1454/3460 [02:39<02:59, 11.19it/s]
分批tokenize目标句子: 42%|████▏ | 1456/3460 [02:39<02:58, 11.26it/s]
分批tokenize目标句子: 42%|████▏ | 1458/3460 [02:39<02:57, 11.30it/s]
分批tokenize目标句子: 42%|████▏ | 1460/3460 [02:39<02:56, 11.33it/s]
分批tokenize目标句子: 42%|████▏ | 1462/3460 [02:39<02:56, 11.35it/s]
分批tokenize目标句子: 42%|████▏ | 1464/3460 [02:39<02:56, 11.33it/s]
分批tokenize目标句子: 42%|████▏ | 1466/3460 [02:49<47:58, 1.44s/it]
分批tokenize目标句子: 42%|████▏ | 1468/3460 [02:49<34:23, 1.04s/it]
分批tokenize目标句子: 42%|████▏ | 1470/3460 [02:49<24:53, 1.33it/s]
分批tokenize目标句子: 43%|████▎ | 1472/3460 [02:49<18:15, 1.81it/s]
分批tokenize目标句子: 43%|████▎ | 1474/3460 [02:49<13:39, 2.42it/s]
分批tokenize目标句子: 43%|████▎ | 1476/3460 [02:50<10:25, 3.17it/s]
分批tokenize目标句子: 43%|████▎ | 1478/3460 [02:50<08:07, 4.06it/s]
分批tokenize目标句子: 43%|████▎ | 1480/3460 [02:50<06:33, 5.03it/s]
分批tokenize目标句子: 43%|████▎ | 1482/3460 [02:50<05:27, 6.04it/s]
分批tokenize目标句子: 43%|████▎ | 1484/3460 [02:50<04:39, 7.07it/s]
分批tokenize目标句子: 43%|████▎ | 1486/3460 [02:50<04:07, 7.98it/s]
分批tokenize目标句子: 43%|████▎ | 1488/3460 [02:51<03:44, 8.78it/s]
分批tokenize目标句子: 43%|████▎ | 1490/3460 [02:51<03:27, 9.50it/s]
分批tokenize目标句子: 43%|████▎ | 1492/3460 [02:51<03:17, 9.95it/s]
分批tokenize目标句子: 43%|████▎ | 1494/3460 [02:51<03:09, 10.35it/s]
分批tokenize目标句子: 43%|████▎ | 1496/3460 [02:51<03:05, 10.59it/s]
分批tokenize目标句子: 43%|████▎ | 1498/3460 [02:51<03:01, 10.80it/s]
分批tokenize目标句子: 43%|████▎ | 1500/3460 [02:52<02:58, 10.95it/s]
分批tokenize目标句子: 43%|████▎ | 1502/3460 [02:52<02:56, 11.06it/s]
分批tokenize目标句子: 43%|████▎ | 1504/3460 [02:52<02:58, 10.94it/s]
分批tokenize目标句子: 44%|████▎ | 1506/3460 [02:52<02:55, 11.16it/s]
分批tokenize目标句子: 44%|████▎ | 1508/3460 [02:52<02:54, 11.21it/s]
分批tokenize目标句子: 44%|████▎ | 1510/3460 [02:53<02:52, 11.29it/s]
分批tokenize目标句子: 44%|████▎ | 1512/3460 [02:53<02:51, 11.33it/s]
分批tokenize目标句子: 44%|████▍ | 1514/3460 [02:53<02:54, 11.18it/s]
分批tokenize目标句子: 44%|████▍ | 1516/3460 [02:53<02:56, 11.04it/s]
分批tokenize目标句子: 44%|████▍ | 1518/3460 [02:53<02:57, 10.91it/s]
分批tokenize目标句子: 44%|████▍ | 1520/3460 [02:53<02:56, 10.97it/s]
分批tokenize目标句子: 44%|████▍ | 1522/3460 [02:54<02:57, 10.89it/s]
分批tokenize目标句子: 44%|████▍ | 1524/3460 [02:54<02:57, 10.93it/s]
分批tokenize目标句子: 44%|████▍ | 1526/3460 [02:54<02:59, 10.75it/s]
分批tokenize目标句子: 44%|████▍ | 1528/3460 [02:54<02:55, 10.99it/s]
分批tokenize目标句子: 44%|████▍ | 1530/3460 [02:54<02:54, 11.07it/s]
分批tokenize目标句子: 44%|████▍ | 1532/3460 [02:55<02:54, 11.05it/s]
分批tokenize目标句子: 44%|████▍ | 1534/3460 [02:55<02:51, 11.20it/s]
分批tokenize目标句子: 44%|████▍ | 1536/3460 [02:55<02:51, 11.20it/s]
分批tokenize目标句子: 44%|████▍ | 1538/3460 [02:55<02:50, 11.30it/s]
分批tokenize目标句子: 45%|████▍ | 1540/3460 [02:55<02:49, 11.31it/s]
分批tokenize目标句子: 45%|████▍ | 1542/3460 [02:55<02:48, 11.36it/s]
分批tokenize目标句子: 45%|████▍ | 1544/3460 [02:56<02:49, 11.31it/s]
分批tokenize目标句子: 45%|████▍ | 1546/3460 [02:56<02:50, 11.25it/s]
分批tokenize目标句子: 45%|████▍ | 1548/3460 [02:56<02:50, 11.22it/s]
分批tokenize目标句子: 45%|████▍ | 1550/3460 [02:56<02:49, 11.28it/s]
分批tokenize目标句子: 45%|████▍ | 1552/3460 [02:56<02:50, 11.21it/s]
分批tokenize目标句子: 45%|████▍ | 1554/3460 [02:57<02:50, 11.21it/s]
分批tokenize目标句子: 45%|████▍ | 1556/3460 [02:57<02:48, 11.32it/s]
分批tokenize目标句子: 45%|████▌ | 1558/3460 [02:57<02:47, 11.35it/s]
分批tokenize目标句子: 45%|████▌ | 1560/3460 [02:57<02:47, 11.31it/s]
分批tokenize目标句子: 45%|████▌ | 1562/3460 [02:57<02:46, 11.43it/s]
分批tokenize目标句子: 45%|████▌ | 1564/3460 [02:57<02:46, 11.37it/s]
分批tokenize目标句子: 45%|████▌ | 1566/3460 [02:58<02:49, 11.20it/s]
分批tokenize目标句子: 45%|████▌ | 1568/3460 [02:58<02:47, 11.27it/s]
分批tokenize目标句子: 45%|████▌ | 1570/3460 [02:58<02:46, 11.34it/s]
分批tokenize目标句子: 45%|████▌ | 1572/3460 [02:58<02:45, 11.42it/s]
分批tokenize目标句子: 45%|████▌ | 1574/3460 [02:58<02:45, 11.37it/s]
分批tokenize目标句子: 46%|████▌ | 1576/3460 [02:58<02:44, 11.44it/s]
分批tokenize目标句子: 46%|████▌ | 1578/3460 [02:59<02:45, 11.38it/s]
分批tokenize目标句子: 46%|████▌ | 1580/3460 [02:59<02:44, 11.41it/s]
分批tokenize目标句子: 46%|████▌ | 1582/3460 [02:59<02:44, 11.42it/s]
分批tokenize目标句子: 46%|████▌ | 1584/3460 [02:59<02:43, 11.48it/s]
分批tokenize目标句子: 46%|████▌ | 1586/3460 [02:59<02:42, 11.52it/s]
分批tokenize目标句子: 46%|████▌ | 1588/3460 [02:59<02:42, 11.50it/s]
分批tokenize目标句子: 46%|████▌ | 1590/3460 [03:00<02:42, 11.52it/s]
分批tokenize目标句子: 46%|████▌ | 1592/3460 [03:00<02:42, 11.52it/s]
分批tokenize目标句子: 46%|████▌ | 1594/3460 [03:00<02:42, 11.50it/s]
分批tokenize目标句子: 46%|████▌ | 1596/3460 [03:00<02:41, 11.52it/s]
分批tokenize目标句子: 46%|████▌ | 1598/3460 [03:00<02:42, 11.49it/s]
分批tokenize目标句子: 46%|████▌ | 1600/3460 [03:01<02:42, 11.44it/s]
分批tokenize目标句子: 46%|████▋ | 1602/3460 [03:01<02:42, 11.43it/s]
分批tokenize目标句子: 46%|████▋ | 1604/3460 [03:01<02:42, 11.40it/s]
分批tokenize目标句子: 46%|████▋ | 1606/3460 [03:01<02:43, 11.34it/s]
分批tokenize目标句子: 46%|████▋ | 1608/3460 [03:01<02:43, 11.33it/s]
分批tokenize目标句子: 47%|████▋ | 1610/3460 [03:01<02:42, 11.39it/s]
分批tokenize目标句子: 47%|████▋ | 1612/3460 [03:02<02:45, 11.18it/s]
分批tokenize目标句子: 47%|████▋ | 1614/3460 [03:02<02:44, 11.24it/s]
分批tokenize目标句子: 47%|████▋ | 1616/3460 [03:02<02:43, 11.30it/s]
分批tokenize目标句子: 47%|████▋ | 1618/3460 [03:02<02:42, 11.33it/s]
分批tokenize目标句子: 47%|████▋ | 1620/3460 [03:02<02:42, 11.29it/s]
分批tokenize目标句子: 47%|████▋ | 1622/3460 [03:02<02:43, 11.25it/s]
分批tokenize目标句子: 47%|████▋ | 1624/3460 [03:03<02:41, 11.34it/s]
分批tokenize目标句子: 47%|████▋ | 1626/3460 [03:03<02:42, 11.30it/s]
分批tokenize目标句子: 47%|████▋ | 1628/3460 [03:03<02:41, 11.36it/s]
分批tokenize目标句子: 47%|████▋ | 1630/3460 [03:03<02:41, 11.31it/s]
分批tokenize目标句子: 47%|████▋ | 1632/3460 [03:03<02:42, 11.27it/s]
分批tokenize目标句子: 47%|████▋ | 1634/3460 [03:04<02:42, 11.22it/s]
分批tokenize目标句子: 47%|████▋ | 1636/3460 [03:04<02:42, 11.24it/s]
分批tokenize目标句子: 47%|████▋ | 1638/3460 [03:04<02:41, 11.28it/s]
分批tokenize目标句子: 47%|████▋ | 1640/3460 [03:04<02:40, 11.31it/s]
分批tokenize目标句子: 47%|████▋ | 1642/3460 [03:04<02:40, 11.32it/s]
分批tokenize目标句子: 48%|████▊ | 1644/3460 [03:04<02:39, 11.35it/s]
分批tokenize目标句子: 48%|████▊ | 1646/3460 [03:05<02:40, 11.34it/s]
分批tokenize目标句子: 48%|████▊ | 1648/3460 [03:05<02:39, 11.39it/s]
分批tokenize目标句子: 48%|████▊ | 1650/3460 [03:05<02:37, 11.47it/s]
分批tokenize目标句子: 48%|████▊ | 1652/3460 [03:05<02:37, 11.49it/s]
分批tokenize目标句子: 48%|████▊ | 1654/3460 [03:05<02:37, 11.47it/s]
分批tokenize目标句子: 48%|████▊ | 1656/3460 [03:05<02:37, 11.48it/s]
分批tokenize目标句子: 48%|████▊ | 1658/3460 [03:06<02:37, 11.46it/s]
分批tokenize目标句子: 48%|████▊ | 1660/3460 [03:06<02:36, 11.50it/s]
分批tokenize目标句子: 48%|████▊ | 1662/3460 [03:06<02:36, 11.50it/s]
分批tokenize目标句子: 48%|████▊ | 1664/3460 [03:06<02:36, 11.46it/s]
分批tokenize目标句子: 48%|████▊ | 1666/3460 [03:06<02:36, 11.45it/s]
分批tokenize目标句子: 48%|████▊ | 1668/3460 [03:07<02:37, 11.38it/s]
分批tokenize目标句子: 48%|████▊ | 1670/3460 [03:07<02:38, 11.29it/s]
分批tokenize目标句子: 48%|████▊ | 1672/3460 [03:07<02:38, 11.26it/s]
分批tokenize目标句子: 48%|████▊ | 1674/3460 [03:07<02:37, 11.33it/s]
分批tokenize目标句子: 48%|████▊ | 1676/3460 [03:07<02:38, 11.27it/s]
分批tokenize目标句子: 48%|████▊ | 1678/3460 [03:07<02:37, 11.32it/s]
分批tokenize目标句子: 49%|████▊ | 1680/3460 [03:08<02:38, 11.26it/s]
分批tokenize目标句子: 49%|████▊ | 1682/3460 [03:08<02:40, 11.08it/s]
分批tokenize目标句子: 49%|████▊ | 1684/3460 [03:08<02:41, 11.01it/s]
分批tokenize目标句子: 49%|████▊ | 1686/3460 [03:08<02:43, 10.88it/s]
分批tokenize目标句子: 49%|████▉ | 1688/3460 [03:08<02:42, 10.90it/s]
分批tokenize目标句子: 49%|████▉ | 1690/3460 [03:09<02:41, 10.94it/s]
分批tokenize目标句子: 49%|████▉ | 1692/3460 [03:09<02:39, 11.07it/s]
分批tokenize目标句子: 49%|████▉ | 1694/3460 [03:09<02:37, 11.18it/s]
分批tokenize目标句子: 49%|████▉ | 1696/3460 [03:09<02:36, 11.25it/s]
分批tokenize目标句子: 49%|████▉ | 1698/3460 [03:09<02:36, 11.28it/s]
分批tokenize目标句子: 49%|████▉ | 1700/3460 [03:09<02:34, 11.37it/s]
分批tokenize目标句子: 49%|████▉ | 1702/3460 [03:10<02:34, 11.37it/s]
分批tokenize目标句子: 49%|████▉ | 1704/3460 [03:10<02:35, 11.29it/s]
分批tokenize目标句子: 49%|████▉ | 1706/3460 [03:10<02:36, 11.23it/s]
分批tokenize目标句子: 49%|████▉ | 1708/3460 [03:10<02:35, 11.27it/s]
分批tokenize目标句子: 49%|████▉ | 1710/3460 [03:10<02:35, 11.28it/s]
分批tokenize目标句子: 49%|████▉ | 1712/3460 [03:10<02:35, 11.26it/s]
分批tokenize目标句子: 50%|████▉ | 1714/3460 [03:11<02:35, 11.25it/s]
分批tokenize目标句子: 50%|████▉ | 1716/3460 [03:11<02:34, 11.27it/s]
分批tokenize目标句子: 50%|████▉ | 1718/3460 [03:11<02:34, 11.28it/s]
分批tokenize目标句子: 50%|████▉ | 1720/3460 [03:11<02:33, 11.31it/s]
分批tokenize目标句子: 50%|████▉ | 1722/3460 [03:11<02:33, 11.34it/s]
分批tokenize目标句子: 50%|████▉ | 1724/3460 [03:12<02:32, 11.38it/s]
分批tokenize目标句子: 50%|████▉ | 1726/3460 [03:12<02:32, 11.38it/s]
分批tokenize目标句子: 50%|████▉ | 1728/3460 [03:12<02:32, 11.38it/s]
分批tokenize目标句子: 50%|█████ | 1730/3460 [03:12<02:32, 11.37it/s]
分批tokenize目标句子: 50%|█████ | 1732/3460 [03:12<02:31, 11.40it/s]
分批tokenize目标句子: 50%|█████ | 1734/3460 [03:12<02:31, 11.41it/s]
分批tokenize目标句子: 50%|█████ | 1736/3460 [03:13<02:32, 11.34it/s]
分批tokenize目标句子: 50%|█████ | 1738/3460 [03:13<02:31, 11.37it/s]
分批tokenize目标句子: 50%|█████ | 1740/3460 [03:13<02:31, 11.37it/s]
分批tokenize目标句子: 50%|█████ | 1742/3460 [03:13<02:31, 11.33it/s]
分批tokenize目标句子: 50%|█████ | 1744/3460 [03:13<02:31, 11.32it/s]
分批tokenize目标句子: 50%|█████ | 1746/3460 [03:13<02:30, 11.36it/s]
分批tokenize目标句子: 51%|█████ | 1748/3460 [03:14<02:29, 11.44it/s]
分批tokenize目标句子: 51%|█████ | 1750/3460 [03:14<02:29, 11.44it/s]
分批tokenize目标句子: 51%|█████ | 1752/3460 [03:14<02:30, 11.39it/s]
分批tokenize目标句子: 51%|█████ | 1754/3460 [03:14<02:29, 11.38it/s]
分批tokenize目标句子: 51%|█████ | 1756/3460 [03:14<02:29, 11.42it/s]
分批tokenize目标句子: 51%|█████ | 1758/3460 [03:14<02:29, 11.41it/s]
分批tokenize目标句子: 51%|█████ | 1760/3460 [03:15<02:28, 11.42it/s]
分批tokenize目标句子: 51%|█████ | 1762/3460 [03:15<02:29, 11.37it/s]
分批tokenize目标句子: 51%|█████ | 1764/3460 [03:15<02:29, 11.36it/s]
分批tokenize目标句子: 51%|█████ | 1766/3460 [03:15<02:28, 11.41it/s]
分批tokenize目标句子: 51%|█████ | 1768/3460 [03:15<02:28, 11.38it/s]
分批tokenize目标句子: 51%|█████ | 1770/3460 [03:16<02:28, 11.36it/s]
分批tokenize目标句子: 51%|█████ | 1772/3460 [03:16<02:28, 11.36it/s]
分批tokenize目标句子: 51%|█████▏ | 1774/3460 [03:16<02:28, 11.37it/s]
分批tokenize目标句子: 51%|█████▏ | 1776/3460 [03:16<02:27, 11.38it/s]
分批tokenize目标句子: 51%|█████▏ | 1778/3460 [03:16<02:27, 11.42it/s]
分批tokenize目标句子: 51%|█████▏ | 1780/3460 [03:16<02:27, 11.41it/s]
分批tokenize目标句子: 52%|█████▏ | 1782/3460 [03:17<02:27, 11.40it/s]
分批tokenize目标句子: 52%|█████▏ | 1784/3460 [03:17<02:27, 11.39it/s]
分批tokenize目标句子: 52%|█████▏ | 1786/3460 [03:17<02:27, 11.34it/s]
分批tokenize目标句子: 52%|█████▏ | 1788/3460 [03:17<02:26, 11.39it/s]
分批tokenize目标句子: 52%|█████▏ | 1790/3460 [03:17<02:26, 11.38it/s]
分批tokenize目标句子: 52%|█████▏ | 1792/3460 [03:17<02:26, 11.36it/s]
分批tokenize目标句子: 52%|█████▏ | 1794/3460 [03:18<02:26, 11.40it/s]
分批tokenize目标句子: 52%|█████▏ | 1796/3460 [03:18<02:26, 11.38it/s]
分批tokenize目标句子: 52%|█████▏ | 1798/3460 [03:18<02:26, 11.33it/s]
分批tokenize目标句子: 52%|█████▏ | 1800/3460 [03:18<02:26, 11.33it/s]
分批tokenize目标句子: 52%|█████▏ | 1802/3460 [03:18<02:26, 11.29it/s]
分批tokenize目标句子: 52%|█████▏ | 1804/3460 [03:19<02:27, 11.21it/s]
分批tokenize目标句子: 52%|█████▏ | 1806/3460 [03:19<02:26, 11.28it/s]
分批tokenize目标句子: 52%|█████▏ | 1808/3460 [03:19<02:26, 11.30it/s]
分批tokenize目标句子: 52%|█████▏ | 1810/3460 [03:19<02:25, 11.34it/s]
分批tokenize目标句子: 52%|█████▏ | 1812/3460 [03:19<02:25, 11.33it/s]
分批tokenize目标句子: 52%|█████▏ | 1814/3460 [03:19<02:25, 11.35it/s]
分批tokenize目标句子: 52%|█████▏ | 1816/3460 [03:20<02:25, 11.33it/s]
分批tokenize目标句子: 53%|█████▎ | 1818/3460 [03:20<02:26, 11.22it/s]
分批tokenize目标句子: 53%|█████▎ | 1820/3460 [03:20<02:24, 11.34it/s]
分批tokenize目标句子: 53%|█████▎ | 1822/3460 [03:20<02:25, 11.30it/s]
分批tokenize目标句子: 53%|█████▎ | 1824/3460 [03:20<02:24, 11.35it/s]
分批tokenize目标句子: 53%|█████▎ | 1826/3460 [03:20<02:23, 11.41it/s]
分批tokenize目标句子: 53%|█████▎ | 1828/3460 [03:21<02:23, 11.41it/s]
分批tokenize目标句子: 53%|█████▎ | 1830/3460 [03:21<02:23, 11.39it/s]
分批tokenize目标句子: 53%|█████▎ | 1832/3460 [03:21<02:22, 11.39it/s]
分批tokenize目标句子: 53%|█████▎ | 1834/3460 [03:21<02:23, 11.35it/s]
分批tokenize目标句子: 53%|█████▎ | 1836/3460 [03:21<02:22, 11.37it/s]
分批tokenize目标句子: 53%|█████▎ | 1838/3460 [03:22<02:21, 11.43it/s]
分批tokenize目标句子: 53%|█████▎ | 1840/3460 [03:22<02:21, 11.44it/s]
分批tokenize目标句子: 53%|█████▎ | 1842/3460 [03:22<02:22, 11.38it/s]
分批tokenize目标句子: 53%|█████▎ | 1844/3460 [03:22<02:20, 11.48it/s]
分批tokenize目标句子: 53%|█████▎ | 1846/3460 [03:22<02:20, 11.46it/s]
分批tokenize目标句子: 53%|█████▎ | 1848/3460 [03:22<02:20, 11.45it/s]
分批tokenize目标句子: 53%|█████▎ | 1850/3460 [03:23<02:20, 11.44it/s]
分批tokenize目标句子: 54%|█████▎ | 1852/3460 [03:23<02:20, 11.43it/s]
分批tokenize目标句子: 54%|█████▎ | 1854/3460 [03:23<02:20, 11.45it/s]
分批tokenize目标句子: 54%|█████▎ | 1856/3460 [03:23<02:21, 11.36it/s]
分批tokenize目标句子: 54%|█████▎ | 1858/3460 [03:23<02:22, 11.24it/s]
分批tokenize目标句子: 54%|█████▍ | 1860/3460 [03:23<02:23, 11.13it/s]
分批tokenize目标句子: 54%|█████▍ | 1862/3460 [03:24<02:22, 11.18it/s]
分批tokenize目标句子: 54%|█████▍ | 1864/3460 [03:24<02:24, 11.02it/s]
分批tokenize目标句子: 54%|█████▍ | 1866/3460 [03:24<02:23, 11.13it/s]
分批tokenize目标句子: 54%|█████▍ | 1868/3460 [03:24<02:22, 11.15it/s]
分批tokenize目标句子: 54%|█████▍ | 1870/3460 [03:24<02:21, 11.24it/s]
分批tokenize目标句子: 54%|█████▍ | 1872/3460 [03:25<02:21, 11.23it/s]
分批tokenize目标句子: 54%|█████▍ | 1874/3460 [03:25<02:22, 11.15it/s]
分批tokenize目标句子: 54%|█████▍ | 1876/3460 [03:25<02:22, 11.11it/s]
分批tokenize目标句子: 54%|█████▍ | 1878/3460 [03:25<02:21, 11.15it/s]
分批tokenize目标句子: 54%|█████▍ | 1880/3460 [03:25<02:21, 11.17it/s]
分批tokenize目标句子: 54%|█████▍ | 1882/3460 [03:25<02:21, 11.18it/s]
分批tokenize目标句子: 54%|█████▍ | 1884/3460 [03:26<02:20, 11.21it/s]
分批tokenize目标句子: 55%|█████▍ | 1886/3460 [03:26<02:20, 11.21it/s]
分批tokenize目标句子: 55%|█████▍ | 1888/3460 [03:26<02:20, 11.20it/s]
分批tokenize目标句子: 55%|█████▍ | 1890/3460 [03:26<02:20, 11.21it/s]
分批tokenize目标句子: 55%|█████▍ | 1892/3460 [03:26<02:19, 11.24it/s]
分批tokenize目标句子: 55%|█████▍ | 1894/3460 [03:27<02:18, 11.28it/s]
分批tokenize目标句子: 55%|█████▍ | 1896/3460 [03:27<02:18, 11.29it/s]
分批tokenize目标句子: 55%|█████▍ | 1898/3460 [03:27<02:19, 11.16it/s]
分批tokenize目标句子: 55%|█████▍ | 1900/3460 [03:27<02:20, 11.12it/s]
分批tokenize目标句子: 55%|█████▍ | 1902/3460 [03:27<02:20, 11.12it/s]
分批tokenize目标句子: 55%|█████▌ | 1904/3460 [03:27<02:22, 10.88it/s]
分批tokenize目标句子: 55%|█████▌ | 1906/3460 [03:28<02:21, 11.01it/s]
分批tokenize目标句子: 55%|█████▌ | 1908/3460 [03:28<02:19, 11.09it/s]
分批tokenize目标句子: 55%|█████▌ | 1910/3460 [03:28<02:19, 11.13it/s]
分批tokenize目标句子: 55%|█████▌ | 1912/3460 [03:28<02:19, 11.11it/s]
分批tokenize目标句子: 55%|█████▌ | 1914/3460 [03:28<02:18, 11.19it/s]
分批tokenize目标句子: 55%|█████▌ | 1916/3460 [03:29<02:17, 11.23it/s]
分批tokenize目标句子: 55%|█████▌ | 1918/3460 [03:29<02:16, 11.30it/s]
分批tokenize目标句子: 55%|█████▌ | 1920/3460 [03:29<02:15, 11.34it/s]
分批tokenize目标句子: 56%|█████▌ | 1922/3460 [03:29<02:15, 11.35it/s]
分批tokenize目标句子: 56%|█████▌ | 1924/3460 [03:29<02:15, 11.35it/s]
分批tokenize目标句子: 56%|█████▌ | 1926/3460 [03:29<02:14, 11.38it/s]
分批tokenize目标句子: 56%|█████▌ | 1928/3460 [03:30<02:14, 11.39it/s]
分批tokenize目标句子: 56%|█████▌ | 1930/3460 [03:30<02:14, 11.38it/s]
分批tokenize目标句子: 56%|█████▌ | 1932/3460 [03:30<02:14, 11.35it/s]
分批tokenize目标句子: 56%|█████▌ | 1934/3460 [03:30<02:14, 11.32it/s]
分批tokenize目标句子: 56%|█████▌ | 1936/3460 [03:30<02:14, 11.33it/s]
分批tokenize目标句子: 56%|█████▌ | 1938/3460 [03:30<02:14, 11.33it/s]
分批tokenize目标句子: 56%|█████▌ | 1940/3460 [03:31<02:15, 11.23it/s]
分批tokenize目标句子: 56%|█████▌ | 1942/3460 [03:31<02:15, 11.22it/s]
分批tokenize目标句子: 56%|█████▌ | 1944/3460 [03:31<02:14, 11.24it/s]
分批tokenize目标句子: 56%|█████▌ | 1946/3460 [03:31<02:14, 11.29it/s]
分批tokenize目标句子: 56%|█████▋ | 1948/3460 [03:31<02:14, 11.27it/s]
分批tokenize目标句子: 56%|█████▋ | 1950/3460 [03:32<02:14, 11.19it/s]
分批tokenize目标句子: 56%|█████▋ | 1952/3460 [03:32<02:14, 11.24it/s]
分批tokenize目标句子: 56%|█████▋ | 1954/3460 [03:32<02:13, 11.27it/s]
分批tokenize目标句子: 57%|█████▋ | 1956/3460 [03:32<02:13, 11.26it/s]
分批tokenize目标句子: 57%|█████▋ | 1958/3460 [03:32<02:12, 11.35it/s]
分批tokenize目标句子: 57%|█████▋ | 1960/3460 [03:32<02:12, 11.33it/s]
分批tokenize目标句子: 57%|█████▋ | 1962/3460 [03:33<02:12, 11.30it/s]
分批tokenize目标句子: 57%|█████▋ | 1964/3460 [03:33<02:12, 11.32it/s]
分批tokenize目标句子: 57%|█████▋ | 1964/3460 [03:43<02:12, 11.32it/s]
分批tokenize目标句子: 57%|█████▋ | 1966/3460 [03:43<41:29, 1.67s/it]
分批tokenize目标句子: 57%|█████▋ | 1968/3460 [03:44<29:39, 1.19s/it]
分批tokenize目标句子: 57%|█████▋ | 1970/3460 [03:44<21:22, 1.16it/s]
分批tokenize目标句子: 57%|█████▋ | 1972/3460 [03:44<15:36, 1.59it/s]
分批tokenize目标句子: 57%|█████▋ | 1974/3460 [03:44<11:34, 2.14it/s]
分批tokenize目标句子: 57%|█████▋ | 1976/3460 [03:44<08:44, 2.83it/s]
分批tokenize目标句子: 57%|█████▋ | 1978/3460 [03:44<06:46, 3.65it/s]
分批tokenize目标句子: 57%|█████▋ | 1980/3460 [03:45<05:24, 4.56it/s]
分批tokenize目标句子: 57%|█████▋ | 1982/3460 [03:45<04:25, 5.56it/s]
分批tokenize目标句子: 57%|█████▋ | 1984/3460 [03:45<03:44, 6.57it/s]
分批tokenize目标句子: 57%|█████▋ | 1986/3460 [03:45<03:15, 7.53it/s]
分批tokenize目标句子: 57%|█████▋ | 1988/3460 [03:45<02:54, 8.44it/s]
分批tokenize目标句子: 58%|█████▊ | 1990/3460 [03:46<02:40, 9.14it/s]
分批tokenize目标句子: 58%|█████▊ | 1992/3460 [03:46<02:30, 9.73it/s]
分批tokenize目标句子: 58%|█████▊ | 1994/3460 [03:46<02:22, 10.26it/s]
分批tokenize目标句子: 58%|█████▊ | 1996/3460 [03:46<02:19, 10.46it/s]
分批tokenize目标句子: 58%|█████▊ | 1998/3460 [03:46<02:16, 10.72it/s]
分批tokenize目标句子: 58%|█████▊ | 2000/3460 [03:46<02:12, 10.99it/s]
分批tokenize目标句子: 58%|█████▊ | 2002/3460 [03:47<02:13, 10.95it/s]
分批tokenize目标句子: 58%|█████▊ | 2004/3460 [03:47<02:10, 11.14it/s]
分批tokenize目标句子: 58%|█████▊ | 2006/3460 [03:47<02:11, 11.06it/s]
分批tokenize目标句子: 58%|█████▊ | 2008/3460 [03:47<02:09, 11.23it/s]
分批tokenize目标句子: 58%|█████▊ | 2010/3460 [03:47<02:07, 11.33it/s]
分批tokenize目标句子: 58%|█████▊ | 2012/3460 [03:47<02:06, 11.48it/s]
分批tokenize目标句子: 58%|█████▊ | 2014/3460 [03:48<02:06, 11.47it/s]
分批tokenize目标句子: 58%|█████▊ | 2016/3460 [03:48<02:05, 11.52it/s]
分批tokenize目标句子: 58%|█████▊ | 2018/3460 [03:48<02:06, 11.37it/s]
分批tokenize目标句子: 58%|█████▊ | 2020/3460 [03:48<02:07, 11.34it/s]
分批tokenize目标句子: 58%|█████▊ | 2022/3460 [03:48<02:08, 11.17it/s]
分批tokenize目标句子: 58%|█████▊ | 2024/3460 [03:49<02:07, 11.27it/s]
分批tokenize目标句子: 59%|█████▊ | 2026/3460 [03:49<02:07, 11.27it/s]
分批tokenize目标句子: 59%|█████▊ | 2028/3460 [03:49<02:05, 11.38it/s]
分批tokenize目标句子: 59%|█████▊ | 2030/3460 [03:49<02:05, 11.41it/s]
分批tokenize目标句子: 59%|█████▊ | 2032/3460 [03:49<02:04, 11.50it/s]
分批tokenize目标句子: 59%|█████▉ | 2034/3460 [03:49<02:05, 11.41it/s]
分批tokenize目标句子: 59%|█████▉ | 2036/3460 [03:50<02:05, 11.35it/s]
分批tokenize目标句子: 59%|█████▉ | 2038/3460 [03:50<02:05, 11.34it/s]
分批tokenize目标句子: 59%|█████▉ | 2040/3460 [03:50<02:05, 11.28it/s]
分批tokenize目标句子: 59%|█████▉ | 2042/3460 [03:50<02:05, 11.27it/s]
分批tokenize目标句子: 59%|█████▉ | 2044/3460 [03:50<02:04, 11.34it/s]
分批tokenize目标句子: 59%|█████▉ | 2046/3460 [03:50<02:05, 11.25it/s]
分批tokenize目标句子: 59%|█████▉ | 2048/3460 [03:51<02:06, 11.18it/s]
分批tokenize目标句子: 59%|█████▉ | 2050/3460 [03:51<02:06, 11.15it/s]
分批tokenize目标句子: 59%|█████▉ | 2052/3460 [03:51<02:06, 11.17it/s]
分批tokenize目标句子: 59%|█████▉ | 2054/3460 [03:51<02:05, 11.18it/s]
分批tokenize目标句子: 59%|█████▉ | 2056/3460 [03:51<02:05, 11.22it/s]
分批tokenize目标句子: 59%|█████▉ | 2058/3460 [03:52<02:05, 11.16it/s]
分批tokenize目标句子: 60%|█████▉ | 2060/3460 [03:52<02:05, 11.19it/s]
分批tokenize目标句子: 60%|█████▉ | 2062/3460 [03:52<02:04, 11.23it/s]
分批tokenize目标句子: 60%|█████▉ | 2064/3460 [03:52<02:03, 11.28it/s]
分批tokenize目标句子: 60%|█████▉ | 2066/3460 [03:52<02:03, 11.28it/s]
分批tokenize目标句子: 60%|█████▉ | 2068/3460 [03:52<02:02, 11.36it/s]
分批tokenize目标句子: 60%|█████▉ | 2070/3460 [03:53<02:02, 11.36it/s]
分批tokenize目标句子: 60%|█████▉ | 2072/3460 [03:53<02:02, 11.34it/s]
分批tokenize目标句子: 60%|█████▉ | 2074/3460 [03:53<02:01, 11.42it/s]
分批tokenize目标句子: 60%|██████ | 2076/3460 [03:53<02:01, 11.41it/s]
分批tokenize目标句子: 60%|██████ | 2078/3460 [03:53<02:00, 11.43it/s]
分批tokenize目标句子: 60%|██████ | 2080/3460 [03:53<02:01, 11.40it/s]
分批tokenize目标句子: 60%|██████ | 2082/3460 [03:54<02:00, 11.44it/s]
分批tokenize目标句子: 60%|██████ | 2084/3460 [03:54<02:00, 11.43it/s]
分批tokenize目标句子: 60%|██████ | 2086/3460 [03:54<02:00, 11.42it/s]
分批tokenize目标句子: 60%|██████ | 2088/3460 [03:54<01:59, 11.44it/s]
分批tokenize目标句子: 60%|██████ | 2090/3460 [03:54<02:01, 11.31it/s]
分批tokenize目标句子: 60%|██████ | 2092/3460 [03:55<02:00, 11.35it/s]
分批tokenize目标句子: 61%|██████ | 2094/3460 [03:55<02:00, 11.30it/s]
分批tokenize目标句子: 61%|██████ | 2096/3460 [03:55<02:01, 11.23it/s]
分批tokenize目标句子: 61%|██████ | 2098/3460 [03:55<02:01, 11.18it/s]
分批tokenize目标句子: 61%|██████ | 2100/3460 [03:55<02:01, 11.15it/s]
分批tokenize目标句子: 61%|██████ | 2102/3460 [03:55<02:02, 11.13it/s]
分批tokenize目标句子: 61%|██████ | 2104/3460 [03:56<02:02, 11.10it/s]
分批tokenize目标句子: 61%|██████ | 2106/3460 [03:56<02:00, 11.28it/s]
分批tokenize目标句子: 61%|██████ | 2108/3460 [03:56<02:00, 11.25it/s]
分批tokenize目标句子: 61%|██████ | 2110/3460 [03:56<02:00, 11.23it/s]
分批tokenize目标句子: 61%|██████ | 2112/3460 [03:56<02:00, 11.23it/s]
分批tokenize目标句子: 61%|██████ | 2114/3460 [03:57<01:59, 11.25it/s]
分批tokenize目标句子: 61%|██████ | 2116/3460 [03:57<01:59, 11.22it/s]
分批tokenize目标句子: 61%|██████ | 2118/3460 [03:57<01:59, 11.25it/s]
分批tokenize目标句子: 61%|██████▏ | 2120/3460 [03:57<02:00, 11.14it/s]
分批tokenize目标句子: 61%|██████▏ | 2122/3460 [03:57<02:00, 11.07it/s]
分批tokenize目标句子: 61%|██████▏ | 2124/3460 [03:57<02:00, 11.12it/s]
分批tokenize目标句子: 61%|██████▏ | 2126/3460 [03:58<02:00, 11.06it/s]
分批tokenize目标句子: 62%|██████▏ | 2128/3460 [03:58<02:02, 10.91it/s]
分批tokenize目标句子: 62%|██████▏ | 2130/3460 [03:58<02:01, 10.99it/s]
分批tokenize目标句子: 62%|██████▏ | 2132/3460 [03:58<01:59, 11.10it/s]
分批tokenize目标句子: 62%|██████▏ | 2134/3460 [03:58<01:58, 11.15it/s]
分批tokenize目标句子: 62%|██████▏ | 2136/3460 [03:58<01:57, 11.25it/s]
分批tokenize目标句子: 62%|██████▏ | 2138/3460 [03:59<01:56, 11.35it/s]
分批tokenize目标句子: 62%|██████▏ | 2140/3460 [03:59<01:55, 11.40it/s]
分批tokenize目标句子: 62%|██████▏ | 2142/3460 [03:59<01:55, 11.38it/s]
分批tokenize目标句子: 62%|██████▏ | 2144/3460 [03:59<01:55, 11.40it/s]
分批tokenize目标句子: 62%|██████▏ | 2146/3460 [03:59<01:55, 11.43it/s]
分批tokenize目标句子: 62%|██████▏ | 2148/3460 [04:00<01:55, 11.41it/s]
分批tokenize目标句子: 62%|██████▏ | 2150/3460 [04:00<01:55, 11.37it/s]
分批tokenize目标句子: 62%|██████▏ | 2152/3460 [04:00<01:54, 11.39it/s]
分批tokenize目标句子: 62%|██████▏ | 2154/3460 [04:00<01:54, 11.38it/s]
分批tokenize目标句子: 62%|██████▏ | 2156/3460 [04:00<01:54, 11.40it/s]
分批tokenize目标句子: 62%|██████▏ | 2158/3460 [04:00<01:54, 11.36it/s]
分批tokenize目标句子: 62%|██████▏ | 2160/3460 [04:01<01:54, 11.32it/s]
分批tokenize目标句子: 62%|██████▏ | 2162/3460 [04:01<01:54, 11.34it/s]
分批tokenize目标句子: 63%|██████▎ | 2164/3460 [04:01<01:53, 11.38it/s]
分批tokenize目标句子: 63%|██████▎ | 2166/3460 [04:01<01:53, 11.35it/s]
分批tokenize目标句子: 63%|██████▎ | 2168/3460 [04:01<01:53, 11.35it/s]
分批tokenize目标句子: 63%|██████▎ | 2170/3460 [04:01<01:54, 11.31it/s]
分批tokenize目标句子: 63%|██████▎ | 2172/3460 [04:02<01:54, 11.29it/s]
分批tokenize目标句子: 63%|██████▎ | 2174/3460 [04:02<01:54, 11.26it/s]
分批tokenize目标句子: 63%|██████▎ | 2176/3460 [04:02<01:53, 11.29it/s]
分批tokenize目标句子: 63%|██████▎ | 2178/3460 [04:02<01:53, 11.33it/s]
分批tokenize目标句子: 63%|██████▎ | 2180/3460 [04:02<01:53, 11.30it/s]
分批tokenize目标句子: 63%|██████▎ | 2182/3460 [04:03<01:53, 11.29it/s]
分批tokenize目标句子: 63%|██████▎ | 2184/3460 [04:03<01:53, 11.26it/s]
分批tokenize目标句子: 63%|██████▎ | 2186/3460 [04:03<01:52, 11.35it/s]
分批tokenize目标句子: 63%|██████▎ | 2188/3460 [04:03<01:51, 11.41it/s]
分批tokenize目标句子: 63%|██████▎ | 2190/3460 [04:03<01:51, 11.41it/s]
分批tokenize目标句子: 63%|██████▎ | 2192/3460 [04:03<01:50, 11.47it/s]
分批tokenize目标句子: 63%|██████▎ | 2194/3460 [04:04<01:50, 11.50it/s]
分批tokenize目标句子: 63%|██████▎ | 2196/3460 [04:04<01:49, 11.51it/s]
分批tokenize目标句子: 64%|██████▎ | 2198/3460 [04:04<01:49, 11.51it/s]
分批tokenize目标句子: 64%|██████▎ | 2200/3460 [04:04<01:49, 11.52it/s]
分批tokenize目标句子: 64%|██████▎ | 2202/3460 [04:04<01:50, 11.36it/s]
分批tokenize目标句子: 64%|██████▎ | 2204/3460 [04:04<01:50, 11.41it/s]
分批tokenize目标句子: 64%|██████▍ | 2206/3460 [04:05<01:49, 11.48it/s]
分批tokenize目标句子: 64%|██████▍ | 2208/3460 [04:05<01:50, 11.37it/s]
分批tokenize目标句子: 64%|██████▍ | 2210/3460 [04:05<01:49, 11.38it/s]
分批tokenize目标句子: 64%|██████▍ | 2212/3460 [04:05<01:49, 11.39it/s]
分批tokenize目标句子: 64%|██████▍ | 2214/3460 [04:05<01:49, 11.36it/s]
分批tokenize目标句子: 64%|██████▍ | 2216/3460 [04:06<01:50, 11.28it/s]
分批tokenize目标句子: 64%|██████▍ | 2218/3460 [04:06<01:52, 11.08it/s]
分批tokenize目标句子: 64%|██████▍ | 2220/3460 [04:06<01:53, 10.95it/s]
分批tokenize目标句子: 64%|██████▍ | 2222/3460 [04:06<01:53, 10.91it/s]
分批tokenize目标句子: 64%|██████▍ | 2224/3460 [04:06<01:52, 10.99it/s]
分批tokenize目标句子: 64%|██████▍ | 2226/3460 [04:06<01:50, 11.13it/s]
分批tokenize目标句子: 64%|██████▍ | 2228/3460 [04:07<01:50, 11.18it/s]
分批tokenize目标句子: 64%|██████▍ | 2230/3460 [04:07<01:49, 11.27it/s]
分批tokenize目标句子: 65%|██████▍ | 2232/3460 [04:07<01:48, 11.34it/s]
分批tokenize目标句子: 65%|██████▍ | 2234/3460 [04:07<01:47, 11.38it/s]
分批tokenize目标句子: 65%|██████▍ | 2236/3460 [04:07<01:48, 11.23it/s]
分批tokenize目标句子: 65%|██████▍ | 2238/3460 [04:07<01:48, 11.22it/s]
分批tokenize目标句子: 65%|██████▍ | 2240/3460 [04:08<01:49, 11.19it/s]
分批tokenize目标句子: 65%|██████▍ | 2242/3460 [04:08<01:48, 11.19it/s]
分批tokenize目标句子: 65%|██████▍ | 2244/3460 [04:08<01:48, 11.23it/s]
分批tokenize目标句子: 65%|██████▍ | 2246/3460 [04:08<01:47, 11.30it/s]
分批tokenize目标句子: 65%|██████▍ | 2248/3460 [04:08<01:47, 11.31it/s]
分批tokenize目标句子: 65%|██████▌ | 2250/3460 [04:09<01:46, 11.36it/s]
分批tokenize目标句子: 65%|██████▌ | 2252/3460 [04:09<01:46, 11.36it/s]
分批tokenize目标句子: 65%|██████▌ | 2254/3460 [04:09<01:46, 11.37it/s]
分批tokenize目标句子: 65%|██████▌ | 2256/3460 [04:09<01:46, 11.34it/s]
分批tokenize目标句子: 65%|██████▌ | 2258/3460 [04:09<01:46, 11.31it/s]
分批tokenize目标句子: 65%|██████▌ | 2260/3460 [04:09<01:46, 11.32it/s]
分批tokenize目标句子: 65%|██████▌ | 2262/3460 [04:10<01:46, 11.27it/s]
分批tokenize目标句子: 65%|██████▌ | 2264/3460 [04:10<01:46, 11.27it/s]
分批tokenize目标句子: 65%|██████▌ | 2266/3460 [04:10<01:46, 11.26it/s]
分批tokenize目标句子: 66%|██████▌ | 2268/3460 [04:10<01:45, 11.27it/s]
分批tokenize目标句子: 66%|██████▌ | 2270/3460 [04:10<01:45, 11.26it/s]
分批tokenize目标句子: 66%|██████▌ | 2272/3460 [04:11<01:45, 11.28it/s]
分批tokenize目标句子: 66%|██████▌ | 2274/3460 [04:11<01:44, 11.32it/s]
分批tokenize目标句子: 66%|██████▌ | 2276/3460 [04:11<01:45, 11.26it/s]
分批tokenize目标句子: 66%|██████▌ | 2278/3460 [04:11<01:45, 11.16it/s]
分批tokenize目标句子: 66%|██████▌ | 2280/3460 [04:11<01:45, 11.16it/s]
分批tokenize目标句子: 66%|██████▌ | 2282/3460 [04:11<01:44, 11.23it/s]
分批tokenize目标句子: 66%|██████▌ | 2284/3460 [04:12<01:44, 11.29it/s]
分批tokenize目标句子: 66%|██████▌ | 2286/3460 [04:12<01:43, 11.33it/s]
分批tokenize目标句子: 66%|██████▌ | 2288/3460 [04:12<01:43, 11.36it/s]
分批tokenize目标句子: 66%|██████▌ | 2290/3460 [04:12<01:42, 11.40it/s]
分批tokenize目标句子: 66%|██████▌ | 2292/3460 [04:12<01:42, 11.37it/s]
分批tokenize目标句子: 66%|██████▋ | 2294/3460 [04:12<01:42, 11.34it/s]
分批tokenize目标句子: 66%|██████▋ | 2296/3460 [04:13<01:43, 11.22it/s]
分批tokenize目标句子: 66%|██████▋ | 2298/3460 [04:13<01:43, 11.23it/s]
分批tokenize目标句子: 66%|██████▋ | 2300/3460 [04:13<01:43, 11.25it/s]
分批tokenize目标句子: 67%|██████▋ | 2302/3460 [04:13<01:42, 11.25it/s]
分批tokenize目标句子: 67%|██████▋ | 2304/3460 [04:13<01:42, 11.29it/s]
分批tokenize目标句子: 67%|██████▋ | 2306/3460 [04:14<01:42, 11.21it/s]
分批tokenize目标句子: 67%|██████▋ | 2308/3460 [04:14<01:43, 11.14it/s]
分批tokenize目标句子: 67%|██████▋ | 2310/3460 [04:14<01:42, 11.19it/s]
分批tokenize目标句子: 67%|██████▋ | 2312/3460 [04:14<01:41, 11.26it/s]
分批tokenize目标句子: 67%|██████▋ | 2314/3460 [04:14<01:42, 11.19it/s]
分批tokenize目标句子: 67%|██████▋ | 2316/3460 [04:14<01:42, 11.21it/s]
分批tokenize目标句子: 67%|██████▋ | 2318/3460 [04:15<01:42, 11.16it/s]
分批tokenize目标句子: 67%|██████▋ | 2320/3460 [04:15<01:41, 11.22it/s]
分批tokenize目标句子: 67%|██████▋ | 2322/3460 [04:15<01:41, 11.23it/s]
分批tokenize目标句子: 67%|██████▋ | 2324/3460 [04:15<01:40, 11.26it/s]
分批tokenize目标句子: 67%|██████▋ | 2326/3460 [04:15<01:40, 11.31it/s]
分批tokenize目标句子: 67%|██████▋ | 2328/3460 [04:15<01:40, 11.31it/s]
分批tokenize目标句子: 67%|██████▋ | 2330/3460 [04:16<01:40, 11.20it/s]
分批tokenize目标句子: 67%|██████▋ | 2332/3460 [04:16<01:39, 11.28it/s]
分批tokenize目标句子: 67%|██████▋ | 2334/3460 [04:16<01:39, 11.30it/s]
分批tokenize目标句子: 68%|██████▊ | 2336/3460 [04:16<01:40, 11.22it/s]
分批tokenize目标句子: 68%|██████▊ | 2338/3460 [04:16<01:39, 11.26it/s]
分批tokenize目标句子: 68%|██████▊ | 2340/3460 [04:17<01:39, 11.29it/s]
分批tokenize目标句子: 68%|██████▊ | 2342/3460 [04:17<01:38, 11.30it/s]
分批tokenize目标句子: 68%|██████▊ | 2344/3460 [04:17<01:38, 11.31it/s]
分批tokenize目标句子: 68%|██████▊ | 2346/3460 [04:17<01:38, 11.29it/s]
分批tokenize目标句子: 68%|██████▊ | 2348/3460 [04:17<01:37, 11.39it/s]
分批tokenize目标句子: 68%|██████▊ | 2350/3460 [04:17<01:37, 11.43it/s]
分批tokenize目标句子: 68%|██████▊ | 2352/3460 [04:18<01:37, 11.40it/s]
分批tokenize目标句子: 68%|██████▊ | 2354/3460 [04:18<01:37, 11.40it/s]
分批tokenize目标句子: 68%|██████▊ | 2356/3460 [04:18<01:37, 11.37it/s]
分批tokenize目标句子: 68%|██████▊ | 2358/3460 [04:18<01:36, 11.40it/s]
分批tokenize目标句子: 68%|██████▊ | 2360/3460 [04:18<01:35, 11.48it/s]
分批tokenize目标句子: 68%|██████▊ | 2362/3460 [04:18<01:35, 11.48it/s]
分批tokenize目标句子: 68%|██████▊ | 2364/3460 [04:19<01:35, 11.48it/s]
分批tokenize目标句子: 68%|██████▊ | 2366/3460 [04:19<01:35, 11.45it/s]
分批tokenize目标句子: 68%|██████▊ | 2368/3460 [04:19<01:35, 11.43it/s]
分批tokenize目标句子: 68%|██████▊ | 2370/3460 [04:19<01:35, 11.37it/s]
分批tokenize目标句子: 69%|██████▊ | 2372/3460 [04:19<01:35, 11.39it/s]
分批tokenize目标句子: 69%|██████▊ | 2374/3460 [04:20<01:35, 11.42it/s]
分批tokenize目标句子: 69%|██████▊ | 2376/3460 [04:20<01:35, 11.37it/s]
分批tokenize目标句子: 69%|██████▊ | 2378/3460 [04:20<01:35, 11.38it/s]
分批tokenize目标句子: 69%|██████▉ | 2380/3460 [04:20<01:34, 11.38it/s]
分批tokenize目标句子: 69%|██████▉ | 2382/3460 [04:20<01:34, 11.36it/s]
分批tokenize目标句子: 69%|██████▉ | 2384/3460 [04:20<01:34, 11.38it/s]
分批tokenize目标句子: 69%|██████▉ | 2386/3460 [04:21<01:34, 11.40it/s]
分批tokenize目标句子: 69%|██████▉ | 2388/3460 [04:21<01:33, 11.44it/s]
分批tokenize目标句子: 69%|██████▉ | 2390/3460 [04:21<01:33, 11.42it/s]
分批tokenize目标句子: 69%|██████▉ | 2392/3460 [04:21<01:33, 11.43it/s]
分批tokenize目标句子: 69%|██████▉ | 2394/3460 [04:21<01:33, 11.40it/s]
分批tokenize目标句子: 69%|██████▉ | 2396/3460 [04:21<01:33, 11.43it/s]
分批tokenize目标句子: 69%|██████▉ | 2398/3460 [04:22<01:33, 11.42it/s]
分批tokenize目标句子: 69%|██████▉ | 2400/3460 [04:22<01:32, 11.48it/s]
分批tokenize目标句子: 69%|██████▉ | 2402/3460 [04:22<01:32, 11.44it/s]
分批tokenize目标句子: 69%|██████▉ | 2404/3460 [04:22<01:32, 11.47it/s]
分批tokenize目标句子: 70%|██████▉ | 2406/3460 [04:22<01:31, 11.49it/s]
分批tokenize目标句子: 70%|██████▉ | 2408/3460 [04:23<01:31, 11.46it/s]
分批tokenize目标句子: 70%|██████▉ | 2410/3460 [04:23<01:32, 11.39it/s]
分批tokenize目标句子: 70%|██████▉ | 2412/3460 [04:23<01:31, 11.40it/s]
分批tokenize目标句子: 70%|██████▉ | 2414/3460 [04:23<01:31, 11.42it/s]
分批tokenize目标句子: 70%|██████▉ | 2416/3460 [04:23<01:31, 11.44it/s]
分批tokenize目标句子: 70%|██████▉ | 2418/3460 [04:23<01:31, 11.43it/s]
分批tokenize目标句子: 70%|██████▉ | 2420/3460 [04:24<01:31, 11.39it/s]
分批tokenize目标句子: 70%|███████ | 2422/3460 [04:24<01:31, 11.36it/s]
分批tokenize目标句子: 70%|███████ | 2424/3460 [04:24<01:31, 11.34it/s]
分批tokenize目标句子: 70%|███████ | 2426/3460 [04:24<01:30, 11.38it/s]
分批tokenize目标句子: 70%|███████ | 2428/3460 [04:24<01:30, 11.38it/s]
分批tokenize目标句子: 70%|███████ | 2430/3460 [04:24<01:30, 11.38it/s]
分批tokenize目标句子: 70%|███████ | 2432/3460 [04:25<01:30, 11.34it/s]
分批tokenize目标句子: 70%|███████ | 2434/3460 [04:25<01:30, 11.37it/s]
分批tokenize目标句子: 70%|███████ | 2436/3460 [04:25<01:29, 11.40it/s]
分批tokenize目标句子: 70%|███████ | 2438/3460 [04:25<01:30, 11.34it/s]
分批tokenize目标句子: 71%|███████ | 2440/3460 [04:25<01:29, 11.36it/s]
分批tokenize目标句子: 71%|███████ | 2442/3460 [04:25<01:30, 11.30it/s]
分批tokenize目标句子: 71%|███████ | 2444/3460 [04:26<01:29, 11.30it/s]
分批tokenize目标句子: 71%|███████ | 2446/3460 [04:26<01:29, 11.36it/s]
分批tokenize目标句子: 71%|███████ | 2448/3460 [04:26<01:29, 11.35it/s]
分批tokenize目标句子: 71%|███████ | 2450/3460 [04:26<01:28, 11.45it/s]
分批tokenize目标句子: 71%|███████ | 2452/3460 [04:26<01:27, 11.46it/s]
分批tokenize目标句子: 71%|███████ | 2454/3460 [04:27<01:27, 11.50it/s]
分批tokenize目标句子: 71%|███████ | 2456/3460 [04:27<01:27, 11.51it/s]
分批tokenize目标句子: 71%|███████ | 2458/3460 [04:27<01:27, 11.43it/s]
分批tokenize目标句子: 71%|███████ | 2460/3460 [04:27<01:27, 11.43it/s]
分批tokenize目标句子: 71%|███████ | 2462/3460 [04:27<01:27, 11.45it/s]
分批tokenize目标句子: 71%|███████ | 2464/3460 [04:27<01:27, 11.44it/s]
分批tokenize目标句子: 71%|███████▏ | 2466/3460 [04:28<01:27, 11.36it/s]
分批tokenize目标句子: 71%|███████▏ | 2468/3460 [04:28<01:28, 11.26it/s]
分批tokenize目标句子: 71%|███████▏ | 2470/3460 [04:28<01:27, 11.31it/s]
分批tokenize目标句子: 71%|███████▏ | 2472/3460 [04:28<01:26, 11.36it/s]
分批tokenize目标句子: 72%|███████▏ | 2474/3460 [04:28<01:26, 11.38it/s]
分批tokenize目标句子: 72%|███████▏ | 2476/3460 [04:28<01:26, 11.37it/s]
分批tokenize目标句子: 72%|███████▏ | 2478/3460 [04:29<01:26, 11.42it/s]
分批tokenize目标句子: 72%|███████▏ | 2480/3460 [04:29<01:25, 11.45it/s]
分批tokenize目标句子: 72%|███████▏ | 2482/3460 [04:29<01:25, 11.42it/s]
分批tokenize目标句子: 72%|███████▏ | 2484/3460 [04:29<01:25, 11.43it/s]
分批tokenize目标句子: 72%|███████▏ | 2486/3460 [04:29<01:25, 11.39it/s]
分批tokenize目标句子: 72%|███████▏ | 2488/3460 [04:30<01:25, 11.43it/s]
分批tokenize目标句子: 72%|███████▏ | 2490/3460 [04:30<01:25, 11.40it/s]
分批tokenize目标句子: 72%|███████▏ | 2492/3460 [04:30<01:24, 11.46it/s]
分批tokenize目标句子: 72%|███████▏ | 2494/3460 [04:30<01:24, 11.45it/s]
分批tokenize目标句子: 72%|███████▏ | 2496/3460 [04:30<01:24, 11.42it/s]
分批tokenize目标句子: 72%|███████▏ | 2498/3460 [04:30<01:24, 11.34it/s]
分批tokenize目标句子: 72%|███████▏ | 2500/3460 [04:31<01:24, 11.34it/s]
分批tokenize目标句子: 72%|███████▏ | 2502/3460 [04:31<01:24, 11.34it/s]
分批tokenize目标句子: 72%|███████▏ | 2504/3460 [04:31<01:24, 11.36it/s]
分批tokenize目标句子: 72%|███████▏ | 2506/3460 [04:31<01:23, 11.38it/s]
分批tokenize目标句子: 72%|███████▏ | 2508/3460 [04:31<01:23, 11.39it/s]
分批tokenize目标句子: 73%|███████▎ | 2510/3460 [04:31<01:23, 11.41it/s]
分批tokenize目标句子: 73%|███████▎ | 2512/3460 [04:32<01:23, 11.40it/s]
分批tokenize目标句子: 73%|███████▎ | 2514/3460 [04:32<01:23, 11.38it/s]
分批tokenize目标句子: 73%|███████▎ | 2516/3460 [04:32<01:23, 11.31it/s]
分批tokenize目标句子: 73%|███████▎ | 2518/3460 [04:32<01:22, 11.36it/s]
分批tokenize目标句子: 73%|███████▎ | 2520/3460 [04:32<01:22, 11.38it/s]
分批tokenize目标句子: 73%|███████▎ | 2522/3460 [04:33<01:22, 11.33it/s]
分批tokenize目标句子: 73%|███████▎ | 2524/3460 [04:33<01:22, 11.37it/s]
分批tokenize目标句子: 73%|███████▎ | 2526/3460 [04:33<01:21, 11.40it/s]
分批tokenize目标句子: 73%|███████▎ | 2528/3460 [04:33<01:22, 11.30it/s]
分批tokenize目标句子: 73%|███████▎ | 2530/3460 [04:33<01:22, 11.31it/s]
分批tokenize目标句子: 73%|███████▎ | 2532/3460 [04:33<01:21, 11.37it/s]
分批tokenize目标句子: 73%|███████▎ | 2534/3460 [04:34<01:21, 11.33it/s]
分批tokenize目标句子: 73%|███████▎ | 2536/3460 [04:34<01:21, 11.35it/s]
分批tokenize目标句子: 73%|███████▎ | 2538/3460 [04:34<01:21, 11.29it/s]
分批tokenize目标句子: 73%|███████▎ | 2540/3460 [04:34<01:20, 11.38it/s]
分批tokenize目标句子: 73%|███████▎ | 2542/3460 [04:34<01:20, 11.39it/s]
分批tokenize目标句子: 74%|███████▎ | 2544/3460 [04:34<01:20, 11.43it/s]
分批tokenize目标句子: 74%|███████▎ | 2546/3460 [04:35<01:19, 11.46it/s]
分批tokenize目标句子: 74%|███████▎ | 2548/3460 [04:35<01:19, 11.45it/s]
分批tokenize目标句子: 74%|███████▎ | 2550/3460 [04:35<01:19, 11.45it/s]
分批tokenize目标句子: 74%|███████▍ | 2552/3460 [04:35<01:19, 11.43it/s]
分批tokenize目标句子: 74%|███████▍ | 2554/3460 [04:35<01:19, 11.37it/s]
分批tokenize目标句子: 74%|███████▍ | 2556/3460 [04:36<01:19, 11.37it/s]
分批tokenize目标句子: 74%|███████▍ | 2558/3460 [04:36<01:18, 11.42it/s]
分批tokenize目标句子: 74%|███████▍ | 2560/3460 [04:36<01:18, 11.44it/s]
分批tokenize目标句子: 74%|███████▍ | 2562/3460 [04:36<01:18, 11.49it/s]
分批tokenize目标句子: 74%|███████▍ | 2564/3460 [04:36<01:18, 11.48it/s]
分批tokenize目标句子: 74%|███████▍ | 2566/3460 [04:36<01:18, 11.45it/s]
分批tokenize目标句子: 74%|███████▍ | 2568/3460 [04:37<01:17, 11.48it/s]
分批tokenize目标句子: 74%|███████▍ | 2570/3460 [04:37<01:17, 11.43it/s]
分批tokenize目标句子: 74%|███████▍ | 2572/3460 [04:37<01:17, 11.45it/s]
分批tokenize目标句子: 74%|███████▍ | 2574/3460 [04:37<01:17, 11.41it/s]
分批tokenize目标句子: 74%|███████▍ | 2576/3460 [04:37<01:17, 11.45it/s]
分批tokenize目标句子: 75%|███████▍ | 2578/3460 [04:37<01:17, 11.43it/s]
分批tokenize目标句子: 75%|███████▍ | 2580/3460 [04:38<01:16, 11.43it/s]
分批tokenize目标句子: 75%|███████▍ | 2582/3460 [04:38<01:16, 11.42it/s]
分批tokenize目标句子: 75%|███████▍ | 2584/3460 [04:38<01:16, 11.42it/s]
分批tokenize目标句子: 75%|███████▍ | 2586/3460 [04:38<01:16, 11.44it/s]
分批tokenize目标句子: 75%|███████▍ | 2588/3460 [04:38<01:16, 11.44it/s]
分批tokenize目标句子: 75%|███████▍ | 2588/3460 [04:49<01:16, 11.44it/s]
分批tokenize目标句子: 75%|███████▍ | 2590/3460 [04:49<23:41, 1.63s/it]
分批tokenize目标句子: 75%|███████▍ | 2592/3460 [04:49<16:55, 1.17s/it]
分批tokenize目标句子: 75%|███████▍ | 2594/3460 [04:49<12:11, 1.18it/s]
分批tokenize目标句子: 75%|███████▌ | 2596/3460 [04:49<08:53, 1.62it/s]
分批tokenize目标句子: 75%|███████▌ | 2598/3460 [04:49<06:35, 2.18it/s]
分批tokenize目标句子: 75%|███████▌ | 2600/3460 [04:50<04:58, 2.88it/s]
分批tokenize目标句子: 75%|███████▌ | 2602/3460 [04:50<03:50, 3.72it/s]
分批tokenize目标句子: 75%|███████▌ | 2604/3460 [04:50<03:03, 4.66it/s]
分批tokenize目标句子: 75%|███████▌ | 2606/3460 [04:50<02:29, 5.71it/s]
分批tokenize目标句子: 75%|███████▌ | 2608/3460 [04:50<02:06, 6.76it/s]
分批tokenize目标句子: 75%|███████▌ | 2610/3460 [04:51<01:51, 7.66it/s]
分批tokenize目标句子: 75%|███████▌ | 2612/3460 [04:51<01:39, 8.51it/s]
分批tokenize目标句子: 76%|███████▌ | 2614/3460 [04:51<01:32, 9.19it/s]
分批tokenize目标句子: 76%|███████▌ | 2616/3460 [04:51<01:26, 9.75it/s]
分批tokenize目标句子: 76%|███████▌ | 2618/3460 [04:51<01:22, 10.16it/s]
分批tokenize目标句子: 76%|███████▌ | 2620/3460 [04:51<01:20, 10.47it/s]
分批tokenize目标句子: 76%|███████▌ | 2622/3460 [04:52<01:17, 10.80it/s]
分批tokenize目标句子: 76%|███████▌ | 2624/3460 [04:52<01:16, 10.98it/s]
分批tokenize目标句子: 76%|███████▌ | 2626/3460 [04:52<01:17, 10.71it/s]
分批tokenize目标句子: 76%|███████▌ | 2628/3460 [04:52<01:16, 10.87it/s]
分批tokenize目标句子: 76%|███████▌ | 2630/3460 [04:52<01:15, 11.03it/s]
分批tokenize目标句子: 76%|███████▌ | 2632/3460 [04:52<01:14, 11.15it/s]
分批tokenize目标句子: 76%|███████▌ | 2634/3460 [04:53<01:13, 11.18it/s]
分批tokenize目标句子: 76%|███████▌ | 2636/3460 [04:53<01:13, 11.20it/s]
分批tokenize目标句子: 76%|███████▌ | 2638/3460 [04:53<01:12, 11.26it/s]
分批tokenize目标句子: 76%|███████▋ | 2640/3460 [04:53<01:13, 11.23it/s]
分批tokenize目标句子: 76%|███████▋ | 2642/3460 [04:53<01:12, 11.22it/s]
分批tokenize目标句子: 76%|███████▋ | 2644/3460 [04:54<01:12, 11.27it/s]
分批tokenize目标句子: 76%|███████▋ | 2646/3460 [04:54<01:12, 11.26it/s]
分批tokenize目标句子: 77%|███████▋ | 2648/3460 [04:54<01:12, 11.17it/s]
分批tokenize目标句子: 77%|███████▋ | 2650/3460 [04:54<01:12, 11.24it/s]
分批tokenize目标句子: 77%|███████▋ | 2652/3460 [04:54<01:11, 11.23it/s]
分批tokenize目标句子: 77%|███████▋ | 2654/3460 [04:54<01:12, 11.19it/s]
分批tokenize目标句子: 77%|███████▋ | 2656/3460 [04:55<01:11, 11.20it/s]
分批tokenize目标句子: 77%|███████▋ | 2658/3460 [04:55<01:10, 11.33it/s]
分批tokenize目标句子: 77%|███████▋ | 2660/3460 [04:55<01:10, 11.36it/s]
分批tokenize目标句子: 77%|███████▋ | 2662/3460 [04:55<01:10, 11.39it/s]
分批tokenize目标句子: 77%|███████▋ | 2664/3460 [04:55<01:10, 11.36it/s]
分批tokenize目标句子: 77%|███████▋ | 2666/3460 [04:55<01:10, 11.30it/s]
分批tokenize目标句子: 77%|███████▋ | 2668/3460 [04:56<01:10, 11.26it/s]
分批tokenize目标句子: 77%|███████▋ | 2670/3460 [04:56<01:09, 11.35it/s]
分批tokenize目标句子: 77%|███████▋ | 2672/3460 [04:56<01:10, 11.24it/s]
分批tokenize目标句子: 77%|███████▋ | 2674/3460 [04:56<01:09, 11.26it/s]
分批tokenize目标句子: 77%|███████▋ | 2676/3460 [04:56<01:09, 11.34it/s]
分批tokenize目标句子: 77%|███████▋ | 2678/3460 [04:57<01:08, 11.37it/s]
分批tokenize目标句子: 77%|███████▋ | 2680/3460 [04:57<01:07, 11.47it/s]
分批tokenize目标句子: 78%|███████▊ | 2682/3460 [04:57<01:07, 11.51it/s]
分批tokenize目标句子: 78%|███████▊ | 2684/3460 [04:57<01:07, 11.52it/s]
分批tokenize目标句子: 78%|███████▊ | 2686/3460 [04:57<01:07, 11.54it/s]
分批tokenize目标句子: 78%|███████▊ | 2688/3460 [04:57<01:06, 11.57it/s]
分批tokenize目标句子: 78%|███████▊ | 2690/3460 [04:58<01:07, 11.48it/s]
分批tokenize目标句子: 78%|███████▊ | 2692/3460 [04:58<01:07, 11.46it/s]
分批tokenize目标句子: 78%|███████▊ | 2694/3460 [04:58<01:06, 11.45it/s]
分批tokenize目标句子: 78%|███████▊ | 2696/3460 [04:58<01:07, 11.38it/s]
分批tokenize目标句子: 78%|███████▊ | 2698/3460 [04:58<01:06, 11.44it/s]
分批tokenize目标句子: 78%|███████▊ | 2700/3460 [04:58<01:06, 11.39it/s]
分批tokenize目标句子: 78%|███████▊ | 2702/3460 [04:59<01:06, 11.32it/s]
分批tokenize目标句子: 78%|███████▊ | 2704/3460 [04:59<01:06, 11.39it/s]
分批tokenize目标句子: 78%|███████▊ | 2706/3460 [04:59<01:05, 11.51it/s]
分批tokenize目标句子: 78%|███████▊ | 2708/3460 [04:59<01:05, 11.55it/s]
分批tokenize目标句子: 78%|███████▊ | 2710/3460 [04:59<01:04, 11.55it/s]
分批tokenize目标句子: 78%|███████▊ | 2712/3460 [05:00<01:05, 11.42it/s]
分批tokenize目标句子: 78%|███████▊ | 2714/3460 [05:00<01:05, 11.39it/s]
分批tokenize目标句子: 78%|███████▊ | 2716/3460 [05:00<01:05, 11.43it/s]
分批tokenize目标句子: 79%|███████▊ | 2718/3460 [05:00<01:05, 11.35it/s]
分批tokenize目标句子: 79%|███████▊ | 2720/3460 [05:00<01:04, 11.43it/s]
分批tokenize目标句子: 79%|███████▊ | 2722/3460 [05:00<01:04, 11.39it/s]
分批tokenize目标句子: 79%|███████▊ | 2724/3460 [05:01<01:04, 11.38it/s]
分批tokenize目标句子: 79%|███████▉ | 2726/3460 [05:01<01:04, 11.39it/s]
分批tokenize目标句子: 79%|███████▉ | 2728/3460 [05:01<01:04, 11.43it/s]
分批tokenize目标句子: 79%|███████▉ | 2730/3460 [05:01<01:03, 11.44it/s]
分批tokenize目标句子: 79%|███████▉ | 2732/3460 [05:01<01:03, 11.48it/s]
分批tokenize目标句子: 79%|███████▉ | 2734/3460 [05:01<01:03, 11.46it/s]
分批tokenize目标句子: 79%|███████▉ | 2736/3460 [05:02<01:03, 11.40it/s]
分批tokenize目标句子: 79%|███████▉ | 2738/3460 [05:02<01:03, 11.45it/s]
分批tokenize目标句子: 79%|███████▉ | 2740/3460 [05:02<01:02, 11.49it/s]
分批tokenize目标句子: 79%|███████▉ | 2742/3460 [05:02<01:02, 11.48it/s]
分批tokenize目标句子: 79%|███████▉ | 2744/3460 [05:02<01:02, 11.42it/s]
分批tokenize目标句子: 79%|███████▉ | 2746/3460 [05:02<01:02, 11.50it/s]
分批tokenize目标句子: 79%|███████▉ | 2748/3460 [05:03<01:02, 11.44it/s]
分批tokenize目标句子: 79%|███████▉ | 2750/3460 [05:03<01:02, 11.40it/s]
分批tokenize目标句子: 80%|███████▉ | 2752/3460 [05:03<01:01, 11.45it/s]
分批tokenize目标句子: 80%|███████▉ | 2754/3460 [05:03<01:01, 11.45it/s]
分批tokenize目标句子: 80%|███████▉ | 2756/3460 [05:03<01:01, 11.44it/s]
分批tokenize目标句子: 80%|███████▉ | 2758/3460 [05:04<01:00, 11.55it/s]
分批tokenize目标句子: 80%|███████▉ | 2760/3460 [05:04<01:00, 11.56it/s]
分批tokenize目标句子: 80%|███████▉ | 2762/3460 [05:04<01:00, 11.51it/s]
分批tokenize目标句子: 80%|███████▉ | 2764/3460 [05:04<01:00, 11.46it/s]
分批tokenize目标句子: 80%|███████▉ | 2766/3460 [05:04<01:00, 11.42it/s]
分批tokenize目标句子: 80%|████████ | 2768/3460 [05:04<01:00, 11.47it/s]
分批tokenize目标句子: 80%|████████ | 2770/3460 [05:05<00:59, 11.51it/s]
分批tokenize目标句子: 80%|████████ | 2772/3460 [05:05<00:59, 11.50it/s]
分批tokenize目标句子: 80%|████████ | 2774/3460 [05:05<00:59, 11.46it/s]
分批tokenize目标句子: 80%|████████ | 2776/3460 [05:05<00:59, 11.48it/s]
分批tokenize目标句子: 80%|████████ | 2778/3460 [05:05<00:59, 11.48it/s]
分批tokenize目标句子: 80%|████████ | 2780/3460 [05:05<00:59, 11.48it/s]
分批tokenize目标句子: 80%|████████ | 2782/3460 [05:06<00:59, 11.47it/s]
分批tokenize目标句子: 80%|████████ | 2784/3460 [05:06<00:58, 11.54it/s]
分批tokenize目标句子: 81%|████████ | 2786/3460 [05:06<00:58, 11.50it/s]
分批tokenize目标句子: 81%|████████ | 2788/3460 [05:06<00:58, 11.55it/s]
分批tokenize目标句子: 81%|████████ | 2790/3460 [05:06<00:58, 11.48it/s]
分批tokenize目标句子: 81%|████████ | 2792/3460 [05:06<00:58, 11.50it/s]
分批tokenize目标句子: 81%|████████ | 2794/3460 [05:07<00:58, 11.45it/s]
分批tokenize目标句子: 81%|████████ | 2796/3460 [05:07<00:57, 11.47it/s]
分批tokenize目标句子: 81%|████████ | 2798/3460 [05:07<00:57, 11.43it/s]
分批tokenize目标句子: 81%|████████ | 2800/3460 [05:07<00:58, 11.38it/s]
分批tokenize目标句子: 81%|████████ | 2802/3460 [05:07<00:57, 11.35it/s]
分批tokenize目标句子: 81%|████████ | 2804/3460 [05:08<00:57, 11.39it/s]
分批tokenize目标句子: 81%|████████ | 2806/3460 [05:08<00:57, 11.31it/s]
分批tokenize目标句子: 81%|████████ | 2808/3460 [05:08<00:57, 11.35it/s]
分批tokenize目标句子: 81%|████████ | 2810/3460 [05:08<00:57, 11.35it/s]
分批tokenize目标句子: 81%|████████▏ | 2812/3460 [05:08<00:57, 11.26it/s]
分批tokenize目标句子: 81%|████████▏ | 2814/3460 [05:08<00:57, 11.33it/s]
分批tokenize目标句子: 81%|████████▏ | 2816/3460 [05:09<00:56, 11.31it/s]
分批tokenize目标句子: 81%|████████▏ | 2818/3460 [05:09<00:57, 11.15it/s]
分批tokenize目标句子: 82%|████████▏ | 2820/3460 [05:09<00:56, 11.23it/s]
分批tokenize目标句子: 82%|████████▏ | 2822/3460 [05:09<00:56, 11.25it/s]
分批tokenize目标句子: 82%|████████▏ | 2824/3460 [05:09<00:56, 11.27it/s]
分批tokenize目标句子: 82%|████████▏ | 2826/3460 [05:09<00:55, 11.35it/s]
分批tokenize目标句子: 82%|████████▏ | 2828/3460 [05:10<00:55, 11.32it/s]
分批tokenize目标句子: 82%|████████▏ | 2830/3460 [05:10<00:55, 11.34it/s]
分批tokenize目标句子: 82%|████████▏ | 2832/3460 [05:10<00:55, 11.37it/s]
分批tokenize目标句子: 82%|████████▏ | 2834/3460 [05:10<00:54, 11.38it/s]
分批tokenize目标句子: 82%|████████▏ | 2836/3460 [05:10<00:54, 11.41it/s]
分批tokenize目标句子: 82%|████████▏ | 2838/3460 [05:11<00:54, 11.38it/s]
分批tokenize目标句子: 82%|████████▏ | 2840/3460 [05:11<00:55, 11.21it/s]
分批tokenize目标句子: 82%|████████▏ | 2842/3460 [05:11<00:55, 11.21it/s]
分批tokenize目标句子: 82%|████████▏ | 2844/3460 [05:11<00:55, 11.08it/s]
分批tokenize目标句子: 82%|████████▏ | 2846/3460 [05:11<00:55, 11.07it/s]
分批tokenize目标句子: 82%|████████▏ | 2848/3460 [05:11<00:54, 11.18it/s]
分批tokenize目标句子: 82%|████████▏ | 2850/3460 [05:12<00:54, 11.17it/s]
分批tokenize目标句子: 82%|████████▏ | 2852/3460 [05:12<00:54, 11.12it/s]
分批tokenize目标句子: 82%|████████▏ | 2854/3460 [05:12<00:55, 10.96it/s]
分批tokenize目标句子: 83%|████████▎ | 2856/3460 [05:12<00:54, 11.06it/s]
分批tokenize目标句子: 83%|████████▎ | 2858/3460 [05:12<00:53, 11.16it/s]
分批tokenize目标句子: 83%|████████▎ | 2860/3460 [05:13<00:53, 11.26it/s]
分批tokenize目标句子: 83%|████████▎ | 2862/3460 [05:13<00:52, 11.35it/s]
分批tokenize目标句子: 83%|████████▎ | 2864/3460 [05:13<00:52, 11.42it/s]
分批tokenize目标句子: 83%|████████▎ | 2866/3460 [05:13<00:52, 11.37it/s]
分批tokenize目标句子: 83%|████████▎ | 2868/3460 [05:13<00:51, 11.41it/s]
分批tokenize目标句子: 83%|████████▎ | 2870/3460 [05:13<00:51, 11.47it/s]
分批tokenize目标句子: 83%|████████▎ | 2872/3460 [05:14<00:51, 11.48it/s]
分批tokenize目标句子: 83%|████████▎ | 2874/3460 [05:14<00:51, 11.40it/s]
分批tokenize目标句子: 83%|████████▎ | 2876/3460 [05:14<00:51, 11.38it/s]
分批tokenize目标句子: 83%|████████▎ | 2878/3460 [05:14<00:51, 11.34it/s]
分批tokenize目标句子: 83%|████████▎ | 2880/3460 [05:14<00:51, 11.35it/s]
分批tokenize目标句子: 83%|████████▎ | 2882/3460 [05:14<00:50, 11.38it/s]
分批tokenize目标句子: 83%|████████▎ | 2884/3460 [05:15<00:50, 11.43it/s]
分批tokenize目标句子: 83%|████████▎ | 2886/3460 [05:15<00:50, 11.46it/s]
分批tokenize目标句子: 83%|████████▎ | 2888/3460 [05:15<00:50, 11.42it/s]
分批tokenize目标句子: 84%|████████▎ | 2890/3460 [05:15<00:49, 11.43it/s]
分批tokenize目标句子: 84%|████████▎ | 2892/3460 [05:15<00:49, 11.50it/s]
分批tokenize目标句子: 84%|████████▎ | 2894/3460 [05:16<00:49, 11.47it/s]
分批tokenize目标句子: 84%|████████▎ | 2896/3460 [05:16<00:49, 11.50it/s]
分批tokenize目标句子: 84%|████████▍ | 2898/3460 [05:16<00:48, 11.49it/s]
分批tokenize目标句子: 84%|████████▍ | 2900/3460 [05:16<00:49, 11.41it/s]
分批tokenize目标句子: 84%|████████▍ | 2902/3460 [05:16<00:49, 11.34it/s]
分批tokenize目标句子: 84%|████████▍ | 2904/3460 [05:16<00:49, 11.31it/s]
分批tokenize目标句子: 84%|████████▍ | 2906/3460 [05:17<00:48, 11.34it/s]
分批tokenize目标句子: 84%|████████▍ | 2908/3460 [05:17<00:48, 11.37it/s]
分批tokenize目标句子: 84%|████████▍ | 2910/3460 [05:17<00:48, 11.37it/s]
分批tokenize目标句子: 84%|████████▍ | 2912/3460 [05:17<00:48, 11.36it/s]
分批tokenize目标句子: 84%|████████▍ | 2914/3460 [05:17<00:48, 11.36it/s]
分批tokenize目标句子: 84%|████████▍ | 2916/3460 [05:17<00:47, 11.35it/s]
分批tokenize目标句子: 84%|████████▍ | 2918/3460 [05:18<00:47, 11.32it/s]
分批tokenize目标句子: 84%|████████▍ | 2920/3460 [05:18<00:47, 11.29it/s]
分批tokenize目标句子: 84%|████████▍ | 2922/3460 [05:18<00:47, 11.27it/s]
分批tokenize目标句子: 85%|████████▍ | 2924/3460 [05:18<00:47, 11.30it/s]
分批tokenize目标句子: 85%|████████▍ | 2926/3460 [05:18<00:47, 11.33it/s]
分批tokenize目标句子: 85%|████████▍ | 2928/3460 [05:18<00:46, 11.34it/s]
分批tokenize目标句子: 85%|████████▍ | 2930/3460 [05:19<00:46, 11.40it/s]
分批tokenize目标句子: 85%|████████▍ | 2932/3460 [05:19<00:46, 11.42it/s]
分批tokenize目标句子: 85%|████████▍ | 2934/3460 [05:19<00:45, 11.44it/s]
分批tokenize目标句子: 85%|████████▍ | 2936/3460 [05:19<00:46, 11.32it/s]
分批tokenize目标句子: 85%|████████▍ | 2938/3460 [05:19<00:46, 11.29it/s]
分批tokenize目标句子: 85%|████████▍ | 2940/3460 [05:20<00:45, 11.36it/s]
分批tokenize目标句子: 85%|████████▌ | 2942/3460 [05:20<00:45, 11.40it/s]
分批tokenize目标句子: 85%|████████▌ | 2944/3460 [05:20<00:45, 11.46it/s]
分批tokenize目标句子: 85%|████████▌ | 2946/3460 [05:20<00:44, 11.50it/s]
分批tokenize目标句子: 85%|████████▌ | 2948/3460 [05:20<00:44, 11.51it/s]
分批tokenize目标句子: 85%|████████▌ | 2950/3460 [05:20<00:44, 11.53it/s]
分批tokenize目标句子: 85%|████████▌ | 2952/3460 [05:21<00:44, 11.36it/s]
分批tokenize目标句子: 85%|████████▌ | 2954/3460 [05:21<00:44, 11.35it/s]
分批tokenize目标句子: 85%|████████▌ | 2956/3460 [05:21<00:44, 11.33it/s]
分批tokenize目标句子: 85%|████████▌ | 2958/3460 [05:21<00:44, 11.25it/s]
分批tokenize目标句子: 86%|████████▌ | 2960/3460 [05:21<00:44, 11.29it/s]
分批tokenize目标句子: 86%|████████▌ | 2962/3460 [05:21<00:43, 11.33it/s]
分批tokenize目标句子: 86%|████████▌ | 2964/3460 [05:22<00:43, 11.32it/s]
分批tokenize目标句子: 86%|████████▌ | 2966/3460 [05:22<00:43, 11.34it/s]
分批tokenize目标句子: 86%|████████▌ | 2968/3460 [05:22<00:43, 11.37it/s]
分批tokenize目标句子: 86%|████████▌ | 2970/3460 [05:22<00:43, 11.37it/s]
分批tokenize目标句子: 86%|████████▌ | 2972/3460 [05:22<00:42, 11.35it/s]
分批tokenize目标句子: 86%|████████▌ | 2974/3460 [05:23<00:42, 11.36it/s]
分批tokenize目标句子: 86%|████████▌ | 2976/3460 [05:23<00:42, 11.35it/s]
分批tokenize目标句子: 86%|████████▌ | 2978/3460 [05:23<00:42, 11.37it/s]
分批tokenize目标句子: 86%|████████▌ | 2980/3460 [05:23<00:42, 11.32it/s]
分批tokenize目标句子: 86%|████████▌ | 2982/3460 [05:23<00:42, 11.17it/s]
分批tokenize目标句子: 86%|████████▌ | 2984/3460 [05:23<00:42, 11.20it/s]
分批tokenize目标句子: 86%|████████▋ | 2986/3460 [05:24<00:42, 11.19it/s]
分批tokenize目标句子: 86%|████████▋ | 2988/3460 [05:24<00:42, 11.19it/s]
分批tokenize目标句子: 86%|████████▋ | 2990/3460 [05:24<00:41, 11.22it/s]
分批tokenize目标句子: 86%|████████▋ | 2992/3460 [05:24<00:41, 11.28it/s]
分批tokenize目标句子: 87%|████████▋ | 2994/3460 [05:24<00:41, 11.30it/s]
分批tokenize目标句子: 87%|████████▋ | 2996/3460 [05:24<00:40, 11.36it/s]
分批tokenize目标句子: 87%|████████▋ | 2998/3460 [05:25<00:40, 11.42it/s]
分批tokenize目标句子: 87%|████████▋ | 3000/3460 [05:25<00:40, 11.38it/s]
分批tokenize目标句子: 87%|████████▋ | 3002/3460 [05:25<00:40, 11.42it/s]
分批tokenize目标句子: 87%|████████▋ | 3004/3460 [05:25<00:39, 11.46it/s]
分批tokenize目标句子: 87%|████████▋ | 3006/3460 [05:25<00:39, 11.43it/s]
分批tokenize目标句子: 87%|████████▋ | 3008/3460 [05:26<00:39, 11.43it/s]
分批tokenize目标句子: 87%|████████▋ | 3010/3460 [05:26<00:39, 11.40it/s]
分批tokenize目标句子: 87%|████████▋ | 3012/3460 [05:26<00:39, 11.36it/s]
分批tokenize目标句子: 87%|████████▋ | 3014/3460 [05:26<00:39, 11.38it/s]
分批tokenize目标句子: 87%|████████▋ | 3016/3460 [05:26<00:39, 11.24it/s]
分批tokenize目标句子: 87%|████████▋ | 3018/3460 [05:26<00:39, 11.27it/s]
分批tokenize目标句子: 87%|████████▋ | 3020/3460 [05:27<00:38, 11.34it/s]
分批tokenize目标句子: 87%|████████▋ | 3022/3460 [05:27<00:38, 11.38it/s]
分批tokenize目标句子: 87%|████████▋ | 3024/3460 [05:27<00:38, 11.41it/s]
分批tokenize目标句子: 87%|████████▋ | 3026/3460 [05:27<00:38, 11.34it/s]
分批tokenize目标句子: 88%|████████▊ | 3028/3460 [05:27<00:38, 11.30it/s]
分批tokenize目标句子: 88%|████████▊ | 3030/3460 [05:27<00:37, 11.34it/s]
分批tokenize目标句子: 88%|████████▊ | 3032/3460 [05:28<00:37, 11.34it/s]
分批tokenize目标句子: 88%|████████▊ | 3034/3460 [05:28<00:37, 11.40it/s]
分批tokenize目标句子: 88%|████████▊ | 3036/3460 [05:28<00:37, 11.37it/s]
分批tokenize目标句子: 88%|████████▊ | 3038/3460 [05:28<00:37, 11.35it/s]
分批tokenize目标句子: 88%|████████▊ | 3040/3460 [05:28<00:36, 11.36it/s]
分批tokenize目标句子: 88%|████████▊ | 3042/3460 [05:29<00:36, 11.33it/s]
分批tokenize目标句子: 88%|████████▊ | 3044/3460 [05:29<00:36, 11.35it/s]
分批tokenize目标句子: 88%|████████▊ | 3046/3460 [05:29<00:36, 11.38it/s]
分批tokenize目标句子: 88%|████████▊ | 3048/3460 [05:29<00:36, 11.34it/s]
分批tokenize目标句子: 88%|████████▊ | 3050/3460 [05:29<00:36, 11.22it/s]
分批tokenize目标句子: 88%|████████▊ | 3052/3460 [05:29<00:36, 11.14it/s]
分批tokenize目标句子: 88%|████████▊ | 3054/3460 [05:30<00:36, 11.20it/s]
分批tokenize目标句子: 88%|████████▊ | 3056/3460 [05:30<00:36, 11.16it/s]
分批tokenize目标句子: 88%|████████▊ | 3058/3460 [05:30<00:36, 11.12it/s]
分批tokenize目标句子: 88%|████████▊ | 3060/3460 [05:30<00:35, 11.14it/s]
分批tokenize目标句子: 88%|████████▊ | 3062/3460 [05:30<00:35, 11.18it/s]
分批tokenize目标句子: 89%|████████▊ | 3064/3460 [05:31<00:35, 11.27it/s]
分批tokenize目标句子: 89%|████████▊ | 3066/3460 [05:31<00:34, 11.30it/s]
分批tokenize目标句子: 89%|████████▊ | 3068/3460 [05:31<00:34, 11.33it/s]
分批tokenize目标句子: 89%|████████▊ | 3070/3460 [05:31<00:34, 11.36it/s]
分批tokenize目标句子: 89%|████████▉ | 3072/3460 [05:31<00:34, 11.32it/s]
分批tokenize目标句子: 89%|████████▉ | 3074/3460 [05:31<00:33, 11.41it/s]
分批tokenize目标句子: 89%|████████▉ | 3076/3460 [05:32<00:33, 11.45it/s]
分批tokenize目标句子: 89%|████████▉ | 3078/3460 [05:32<00:33, 11.44it/s]
分批tokenize目标句子: 89%|████████▉ | 3080/3460 [05:32<00:33, 11.48it/s]
分批tokenize目标句子: 89%|████████▉ | 3082/3460 [05:32<00:32, 11.48it/s]
分批tokenize目标句子: 89%|████████▉ | 3084/3460 [05:32<00:32, 11.42it/s]
分批tokenize目标句子: 89%|████████▉ | 3086/3460 [05:32<00:32, 11.40it/s]
分批tokenize目标句子: 89%|████████▉ | 3088/3460 [05:33<00:32, 11.38it/s]
分批tokenize目标句子: 89%|████████▉ | 3090/3460 [05:33<00:32, 11.36it/s]
分批tokenize目标句子: 89%|████████▉ | 3092/3460 [05:33<00:32, 11.40it/s]
分批tokenize目标句子: 89%|████████▉ | 3094/3460 [05:33<00:32, 11.41it/s]
分批tokenize目标句子: 89%|████████▉ | 3096/3460 [05:33<00:31, 11.38it/s]
分批tokenize目标句子: 90%|████████▉ | 3098/3460 [05:33<00:32, 11.29it/s]
分批tokenize目标句子: 90%|████████▉ | 3100/3460 [05:34<00:32, 11.23it/s]
分批tokenize目标句子: 90%|████████▉ | 3102/3460 [05:34<00:31, 11.28it/s]
分批tokenize目标句子: 90%|████████▉ | 3104/3460 [05:34<00:31, 11.32it/s]
分批tokenize目标句子: 90%|████████▉ | 3106/3460 [05:34<00:31, 11.31it/s]
分批tokenize目标句子: 90%|████████▉ | 3108/3460 [05:34<00:31, 11.31it/s]
分批tokenize目标句子: 90%|████████▉ | 3110/3460 [05:35<00:30, 11.30it/s]
分批tokenize目标句子: 90%|████████▉ | 3112/3460 [05:35<00:30, 11.30it/s]
分批tokenize目标句子: 90%|█████████ | 3114/3460 [05:35<00:30, 11.30it/s]
分批tokenize目标句子: 90%|█████████ | 3116/3460 [05:35<00:30, 11.27it/s]
分批tokenize目标句子: 90%|█████████ | 3118/3460 [05:35<00:30, 11.28it/s]
分批tokenize目标句子: 90%|█████████ | 3120/3460 [05:35<00:30, 11.23it/s]
分批tokenize目标句子: 90%|█████████ | 3122/3460 [05:36<00:30, 11.23it/s]
分批tokenize目标句子: 90%|█████████ | 3124/3460 [05:36<00:29, 11.29it/s]
分批tokenize目标句子: 90%|█████████ | 3126/3460 [05:36<00:29, 11.28it/s]
分批tokenize目标句子: 90%|█████████ | 3128/3460 [05:36<00:29, 11.35it/s]
分批tokenize目标句子: 90%|█████████ | 3130/3460 [05:36<00:28, 11.41it/s]
分批tokenize目标句子: 91%|█████████ | 3132/3460 [05:36<00:28, 11.36it/s]
分批tokenize目标句子: 91%|█████████ | 3134/3460 [05:37<00:28, 11.29it/s]
分批tokenize目标句子: 91%|█████████ | 3136/3460 [05:37<00:29, 11.14it/s]
分批tokenize目标句子: 91%|█████████ | 3138/3460 [05:37<00:29, 10.99it/s]
分批tokenize目标句子: 91%|█████████ | 3140/3460 [05:37<00:29, 10.92it/s]
分批tokenize目标句子: 91%|█████████ | 3142/3460 [05:37<00:29, 10.96it/s]
分批tokenize目标句子: 91%|█████████ | 3144/3460 [05:38<00:28, 10.93it/s]
分批tokenize目标句子: 91%|█████████ | 3146/3460 [05:38<00:28, 11.06it/s]
分批tokenize目标句子: 91%|█████████ | 3148/3460 [05:38<00:27, 11.20it/s]
分批tokenize目标句子: 91%|█████████ | 3150/3460 [05:38<00:27, 11.26it/s]
分批tokenize目标句子: 91%|█████████ | 3152/3460 [05:38<00:27, 11.32it/s]
分批tokenize目标句子: 91%|█████████ | 3154/3460 [05:38<00:27, 11.33it/s]
分批tokenize目标句子: 91%|█████████ | 3156/3460 [05:39<00:26, 11.39it/s]
分批tokenize目标句子: 91%|█████████▏| 3158/3460 [05:39<00:26, 11.36it/s]
分批tokenize目标句子: 91%|█████████▏| 3160/3460 [05:39<00:26, 11.32it/s]
分批tokenize目标句子: 91%|█████████▏| 3162/3460 [05:39<00:26, 11.33it/s]
分批tokenize目标句子: 91%|█████████▏| 3164/3460 [05:39<00:26, 11.35it/s]
分批tokenize目标句子: 92%|█████████▏| 3166/3460 [05:40<00:26, 11.26it/s]
分批tokenize目标句子: 92%|█████████▏| 3168/3460 [05:40<00:25, 11.31it/s]
分批tokenize目标句子: 92%|█████████▏| 3170/3460 [05:40<00:25, 11.32it/s]
分批tokenize目标句子: 92%|█████████▏| 3172/3460 [05:40<00:25, 11.38it/s]
分批tokenize目标句子: 92%|█████████▏| 3174/3460 [05:40<00:24, 11.46it/s]
分批tokenize目标句子: 92%|█████████▏| 3176/3460 [05:40<00:24, 11.55it/s]
分批tokenize目标句子: 92%|█████████▏| 3178/3460 [05:41<00:24, 11.54it/s]
分批tokenize目标句子: 92%|█████████▏| 3180/3460 [05:41<00:24, 11.58it/s]
分批tokenize目标句子: 92%|█████████▏| 3182/3460 [05:41<00:24, 11.48it/s]
分批tokenize目标句子: 92%|█████████▏| 3184/3460 [05:41<00:24, 11.44it/s]
分批tokenize目标句子: 92%|█████████▏| 3186/3460 [05:41<00:24, 11.39it/s]
分批tokenize目标句子: 92%|█████████▏| 3188/3460 [05:41<00:23, 11.37it/s]
分批tokenize目标句子: 92%|█████████▏| 3190/3460 [05:42<00:24, 11.15it/s]
分批tokenize目标句子: 92%|█████████▏| 3192/3460 [05:42<00:24, 11.16it/s]
分批tokenize目标句子: 92%|█████████▏| 3194/3460 [05:42<00:23, 11.08it/s]
分批tokenize目标句子: 92%|█████████▏| 3196/3460 [05:42<00:23, 11.19it/s]
分批tokenize目标句子: 92%|█████████▏| 3198/3460 [05:42<00:23, 11.26it/s]
分批tokenize目标句子: 92%|█████████▏| 3200/3460 [05:43<00:23, 11.22it/s]
分批tokenize目标句子: 93%|█████████▎| 3202/3460 [05:43<00:22, 11.22it/s]
分批tokenize目标句子: 93%|█████████▎| 3204/3460 [05:43<00:22, 11.24it/s]
分批tokenize目标句子: 93%|█████████▎| 3206/3460 [05:43<00:22, 11.17it/s]
分批tokenize目标句子: 93%|█████████▎| 3208/3460 [05:43<00:22, 11.23it/s]
分批tokenize目标句子: 93%|█████████▎| 3210/3460 [05:43<00:22, 11.23it/s]
分批tokenize目标句子: 93%|█████████▎| 3212/3460 [05:44<00:22, 11.25it/s]
分批tokenize目标句子: 93%|█████████▎| 3214/3460 [05:44<00:21, 11.21it/s]
分批tokenize目标句子: 93%|█████████▎| 3216/3460 [05:44<00:21, 11.14it/s]
分批tokenize目标句子: 93%|█████████▎| 3218/3460 [05:44<00:22, 10.93it/s]
分批tokenize目标句子: 93%|█████████▎| 3220/3460 [05:44<00:22, 10.76it/s]
分批tokenize目标句子: 93%|█████████▎| 3222/3460 [05:45<00:22, 10.71it/s]
分批tokenize目标句子: 93%|█████████▎| 3224/3460 [05:45<00:21, 10.83it/s]
分批tokenize目标句子: 93%|█████████▎| 3226/3460 [05:45<00:21, 10.94it/s]
分批tokenize目标句子: 93%|█████████▎| 3228/3460 [05:45<00:20, 11.08it/s]
分批tokenize目标句子: 93%|█████████▎| 3230/3460 [05:45<00:20, 11.16it/s]
分批tokenize目标句子: 93%|█████████▎| 3232/3460 [05:45<00:20, 11.18it/s]
分批tokenize目标句子: 93%|█████████▎| 3234/3460 [05:46<00:20, 11.25it/s]
分批tokenize目标句子: 93%|█████████▎| 3234/3460 [05:58<00:20, 11.25it/s]
分批tokenize目标句子: 93%|█████████▎| 3235/3460 [05:58<08:21, 2.23s/it]
分批tokenize目标句子: 94%|█████████▎| 3237/3460 [05:58<05:36, 1.51s/it]
分批tokenize目标句子: 94%|█████████▎| 3239/3460 [05:58<03:52, 1.05s/it]
分批tokenize目标句子: 94%|█████████▎| 3241/3460 [05:58<02:43, 1.34it/s]
分批tokenize目标句子: 94%|█████████▎| 3243/3460 [05:59<01:57, 1.85it/s]
分批tokenize目标句子: 94%|█████████▍| 3245/3460 [05:59<01:26, 2.50it/s]
分批tokenize目标句子: 94%|█████████▍| 3247/3460 [05:59<01:04, 3.28it/s]
分批tokenize目标句子: 94%|█████████▍| 3249/3460 [05:59<00:50, 4.19it/s]
分批tokenize目标句子: 94%|█████████▍| 3251/3460 [05:59<00:40, 5.20it/s]
分批tokenize目标句子: 94%|█████████▍| 3253/3460 [05:59<00:33, 6.19it/s]
分批tokenize目标句子: 94%|█████████▍| 3255/3460 [06:00<00:28, 7.16it/s]
分批tokenize目标句子: 94%|█████████▍| 3257/3460 [06:00<00:25, 8.06it/s]
分批tokenize目标句子: 94%|█████████▍| 3259/3460 [06:00<00:22, 8.78it/s]
分批tokenize目标句子: 94%|█████████▍| 3261/3460 [06:00<00:21, 9.29it/s]
分批tokenize目标句子: 94%|█████████▍| 3263/3460 [06:00<00:20, 9.65it/s]
分批tokenize目标句子: 94%|█████████▍| 3265/3460 [06:00<00:19, 10.07it/s]
分批tokenize目标句子: 94%|█████████▍| 3267/3460 [06:01<00:18, 10.25it/s]
分批tokenize目标句子: 94%|█████████▍| 3269/3460 [06:01<00:18, 10.57it/s]
分批tokenize目标句子: 95%|█████████▍| 3271/3460 [06:01<00:17, 10.70it/s]
分批tokenize目标句子: 95%|█████████▍| 3273/3460 [06:01<00:17, 10.88it/s]
分批tokenize目标句子: 95%|█████████▍| 3275/3460 [06:01<00:17, 10.86it/s]
分批tokenize目标句子: 95%|█████████▍| 3277/3460 [06:02<00:16, 11.00it/s]
分批tokenize目标句子: 95%|█████████▍| 3279/3460 [06:02<00:16, 11.03it/s]
分批tokenize目标句子: 95%|█████████▍| 3281/3460 [06:02<00:16, 11.10it/s]
分批tokenize目标句子: 95%|█████████▍| 3283/3460 [06:02<00:15, 11.15it/s]
分批tokenize目标句子: 95%|█████████▍| 3285/3460 [06:02<00:15, 11.14it/s]
分批tokenize目标句子: 95%|█████████▌| 3287/3460 [06:02<00:15, 11.24it/s]
分批tokenize目标句子: 95%|█████████▌| 3289/3460 [06:03<00:15, 11.28it/s]
分批tokenize目标句子: 95%|█████████▌| 3291/3460 [06:03<00:14, 11.29it/s]
分批tokenize目标句子: 95%|█████████▌| 3293/3460 [06:03<00:14, 11.39it/s]
分批tokenize目标句子: 95%|█████████▌| 3295/3460 [06:03<00:14, 11.39it/s]
分批tokenize目标句子: 95%|█████████▌| 3297/3460 [06:03<00:14, 11.41it/s]
分批tokenize目标句子: 95%|█████████▌| 3299/3460 [06:03<00:14, 11.34it/s]
分批tokenize目标句子: 95%|█████████▌| 3301/3460 [06:04<00:13, 11.40it/s]
分批tokenize目标句子: 95%|█████████▌| 3303/3460 [06:04<00:13, 11.28it/s]
分批tokenize目标句子: 96%|█████████▌| 3305/3460 [06:04<00:13, 11.27it/s]
分批tokenize目标句子: 96%|█████████▌| 3307/3460 [06:04<00:13, 11.32it/s]
分批tokenize目标句子: 96%|█████████▌| 3309/3460 [06:04<00:13, 11.24it/s]
分批tokenize目标句子: 96%|█████████▌| 3311/3460 [06:05<00:13, 11.18it/s]
分批tokenize目标句子: 96%|█████████▌| 3313/3460 [06:05<00:13, 11.20it/s]
分批tokenize目标句子: 96%|█████████▌| 3315/3460 [06:05<00:13, 11.06it/s]
分批tokenize目标句子: 96%|█████████▌| 3317/3460 [06:05<00:12, 11.10it/s]
分批tokenize目标句子: 96%|█████████▌| 3319/3460 [06:05<00:12, 11.13it/s]
分批tokenize目标句子: 96%|█████████▌| 3321/3460 [06:05<00:12, 11.23it/s]
分批tokenize目标句子: 96%|█████████▌| 3323/3460 [06:06<00:12, 11.21it/s]
分批tokenize目标句子: 96%|█████████▌| 3325/3460 [06:06<00:11, 11.27it/s]
分批tokenize目标句子: 96%|█████████▌| 3327/3460 [06:06<00:11, 11.25it/s]
分批tokenize目标句子: 96%|█████████▌| 3329/3460 [06:06<00:11, 11.30it/s]
分批tokenize目标句子: 96%|█████████▋| 3331/3460 [06:06<00:11, 11.22it/s]
分批tokenize目标句子: 96%|█████████▋| 3333/3460 [06:07<00:11, 11.18it/s]
分批tokenize目标句子: 96%|█████████▋| 3335/3460 [06:07<00:11, 11.21it/s]
分批tokenize目标句子: 96%|█████████▋| 3337/3460 [06:07<00:11, 11.04it/s]
分批tokenize目标句子: 97%|█████████▋| 3339/3460 [06:07<00:10, 11.12it/s]
分批tokenize目标句子: 97%|█████████▋| 3341/3460 [06:07<00:10, 11.14it/s]
分批tokenize目标句子: 97%|█████████▋| 3343/3460 [06:07<00:10, 11.31it/s]
分批tokenize目标句子: 97%|█████████▋| 3345/3460 [06:08<00:10, 11.32it/s]
分批tokenize目标句子: 97%|█████████▋| 3347/3460 [06:08<00:10, 11.17it/s]
分批tokenize目标句子: 97%|█████████▋| 3349/3460 [06:08<00:09, 11.15it/s]
分批tokenize目标句子: 97%|█████████▋| 3351/3460 [06:08<00:09, 11.21it/s]
分批tokenize目标句子: 97%|█████████▋| 3353/3460 [06:08<00:09, 11.18it/s]
分批tokenize目标句子: 97%|█████████▋| 3355/3460 [06:08<00:09, 11.29it/s]
分批tokenize目标句子: 97%|█████████▋| 3357/3460 [06:09<00:09, 11.41it/s]
分批tokenize目标句子: 97%|█████████▋| 3359/3460 [06:09<00:08, 11.43it/s]
分批tokenize目标句子: 97%|█████████▋| 3361/3460 [06:09<00:08, 11.42it/s]
分批tokenize目标句子: 97%|█████████▋| 3363/3460 [06:09<00:08, 11.45it/s]
分批tokenize目标句子: 97%|█████████▋| 3365/3460 [06:09<00:08, 11.57it/s]
分批tokenize目标句子: 97%|█████████▋| 3367/3460 [06:10<00:08, 11.49it/s]
分批tokenize目标句子: 97%|█████████▋| 3369/3460 [06:10<00:07, 11.40it/s]
分批tokenize目标句子: 97%|█████████▋| 3371/3460 [06:10<00:07, 11.36it/s]
分批tokenize目标句子: 97%|█████████▋| 3373/3460 [06:10<00:07, 11.30it/s]
分批tokenize目标句子: 98%|█████████▊| 3375/3460 [06:10<00:07, 11.37it/s]
分批tokenize目标句子: 98%|█████████▊| 3377/3460 [06:10<00:07, 11.36it/s]
分批tokenize目标句子: 98%|█████████▊| 3379/3460 [06:11<00:07, 11.31it/s]
分批tokenize目标句子: 98%|█████████▊| 3381/3460 [06:11<00:07, 11.28it/s]
分批tokenize目标句子: 98%|█████████▊| 3383/3460 [06:11<00:06, 11.27it/s]
分批tokenize目标句子: 98%|█████████▊| 3385/3460 [06:11<00:06, 11.38it/s]
分批tokenize目标句子: 98%|█████████▊| 3387/3460 [06:11<00:06, 11.25it/s]
分批tokenize目标句子: 98%|█████████▊| 3389/3460 [06:11<00:06, 11.34it/s]
分批tokenize目标句子: 98%|█████████▊| 3391/3460 [06:12<00:06, 11.13it/s]
分批tokenize目标句子: 98%|█████████▊| 3393/3460 [06:12<00:05, 11.24it/s]
分批tokenize目标句子: 98%|█████████▊| 3395/3460 [06:12<00:05, 11.34it/s]
分批tokenize目标句子: 98%|█████████▊| 3397/3460 [06:12<00:05, 11.40it/s]
分批tokenize目标句子: 98%|█████████▊| 3399/3460 [06:12<00:05, 11.35it/s]
分批tokenize目标句子: 98%|█████████▊| 3401/3460 [06:13<00:05, 11.36it/s]
分批tokenize目标句子: 98%|█████████▊| 3403/3460 [06:13<00:05, 11.38it/s]
分批tokenize目标句子: 98%|█████████▊| 3405/3460 [06:13<00:04, 11.48it/s]
分批tokenize目标句子: 98%|█████████▊| 3407/3460 [06:13<00:04, 11.43it/s]
分批tokenize目标句子: 99%|█████████▊| 3409/3460 [06:13<00:04, 11.47it/s]
分批tokenize目标句子: 99%|█████████▊| 3411/3460 [06:13<00:04, 11.47it/s]
分批tokenize目标句子: 99%|█████████▊| 3413/3460 [06:14<00:04, 11.48it/s]
分批tokenize目标句子: 99%|█████████▊| 3415/3460 [06:14<00:03, 11.47it/s]
分批tokenize目标句子: 99%|█████████▉| 3417/3460 [06:14<00:03, 11.48it/s]
分批tokenize目标句子: 99%|█████████▉| 3419/3460 [06:14<00:03, 11.47it/s]
分批tokenize目标句子: 99%|█████████▉| 3421/3460 [06:14<00:03, 11.48it/s]
分批tokenize目标句子: 99%|█████████▉| 3423/3460 [06:14<00:03, 11.45it/s]
分批tokenize目标句子: 99%|█████████▉| 3425/3460 [06:15<00:03, 11.48it/s]
分批tokenize目标句子: 99%|█████████▉| 3427/3460 [06:15<00:02, 11.49it/s]
分批tokenize目标句子: 99%|█████████▉| 3429/3460 [06:15<00:02, 11.49it/s]
分批tokenize目标句子: 99%|█████████▉| 3431/3460 [06:15<00:02, 11.48it/s]
分批tokenize目标句子: 99%|█████████▉| 3433/3460 [06:15<00:02, 11.47it/s]
分批tokenize目标句子: 99%|█████████▉| 3435/3460 [06:15<00:02, 11.49it/s]
分批tokenize目标句子: 99%|█████████▉| 3437/3460 [06:16<00:01, 11.52it/s]
分批tokenize目标句子: 99%|█████████▉| 3439/3460 [06:16<00:01, 11.54it/s]
分批tokenize目标句子: 99%|█████████▉| 3441/3460 [06:16<00:01, 11.55it/s]
分批tokenize目标句子: 100%|█████████▉| 3443/3460 [06:16<00:01, 11.54it/s]
分批tokenize目标句子: 100%|█████████▉| 3445/3460 [06:16<00:01, 11.57it/s]
分批tokenize目标句子: 100%|█████████▉| 3447/3460 [06:17<00:01, 11.52it/s]
分批tokenize目标句子: 100%|█████████▉| 3449/3460 [06:17<00:00, 11.51it/s]
分批tokenize目标句子: 100%|█████████▉| 3451/3460 [06:17<00:00, 11.37it/s]
分批tokenize目标句子: 100%|█████████▉| 3453/3460 [06:17<00:00, 11.35it/s]
分批tokenize目标句子: 100%|█████████▉| 3455/3460 [06:17<00:00, 11.44it/s]
分批tokenize目标句子: 100%|█████████▉| 3457/3460 [06:17<00:00, 11.48it/s]
分批tokenize目标句子: 100%|█████████▉| 3459/3460 [06:18<00:00, 11.38it/s]
分批tokenize目标句子: 100%|██████████| 3460/3460 [06:18<00:00, 9.15it/s]
🎉 数据预处理完成! 共处理 3459987 个样本
[2025-07-02 12:39:23,706] [INFO] [real_accelerator.py:254:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2025-07-02 12:39:24,698] [INFO] [logging.py:107:log_dist] [Rank -1] DeepSpeed info: version=0.17.0, git-hash=unknown, git-branch=unknown
[2025-07-02 12:39:24,699] [INFO] [comm.py:675:init_distributed] cdb=None
[2025-07-02 12:39:24,699] [INFO] [comm.py:690:init_distributed] Not using the DeepSpeed or dist launchers, attempting to detect MPI environment...
[2025-07-02 12:39:26,015] [INFO] [comm.py:745:mpi_discovery] Discovered MPI settings of world_rank=0, local_rank=0, world_size=1, master_addr=192.168.1.39, master_port=29500
[2025-07-02 12:39:26,016] [INFO] [comm.py:706:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
[2025-07-02 12:39:26,023] [INFO] [config.py:744:__init__] Config mesh_device None world_size = 1
[2025-07-02 12:39:26,640] [INFO] [engine.py:1313:_configure_distributed_model] ********** distributed groups summary **********
self.dp_world_size=1
self.mp_world_size=1
self.seq_dp_world_size=1
self.sequence_parallel_size=1
***********************************************
[2025-07-02 12:39:26,863] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Flops Profiler Enabled: False
[2025-07-02 12:39:26,864] [INFO] [logging.py:107:log_dist] [Rank 0] Using client Optimizer as basic optimizer
[2025-07-02 12:39:26,864] [INFO] [logging.py:107:log_dist] [Rank 0] Removing param_group that has no 'params' in the basic Optimizer
[2025-07-02 12:39:26,868] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Basic Optimizer = AdamW
[2025-07-02 12:39:26,868] [INFO] [utils.py:59:is_zero_supported_optimizer] Checking ZeRO support for optimizer=AdamW type=<class 'torch.optim.adamw.AdamW'>
[2025-07-02 12:39:26,868] [INFO] [logging.py:107:log_dist] [Rank 0] Creating torch.bfloat16 ZeRO stage 2 optimizer
[2025-07-02 12:39:26,869] [INFO] [stage_1_and_2.py:150:__init__] Reduce bucket size 500000000
[2025-07-02 12:39:26,869] [INFO] [stage_1_and_2.py:151:__init__] Allgather bucket size 500000000
[2025-07-02 12:39:26,869] [INFO] [stage_1_and_2.py:152:__init__] CPU Offload: False
[2025-07-02 12:39:26,869] [INFO] [stage_1_and_2.py:153:__init__] Round robin gradient partitioning: False
[2025-07-02 12:39:31,187] [INFO] [utils.py:781:see_memory_usage] Before initializing optimizer states
[2025-07-02 12:39:31,188] [INFO] [utils.py:782:see_memory_usage] MA 0.38 GB Max_MA 0.41 GB CA 0.43 GB Max_CA 0 GB
[2025-07-02 12:39:31,191] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory: used = 60.17 GB, percent = 27.3%
[2025-07-02 12:39:33,966] [INFO] [utils.py:781:see_memory_usage] After initializing optimizer states
[2025-07-02 12:39:33,967] [INFO] [utils.py:782:see_memory_usage] MA 0.38 GB Max_MA 0.44 GB CA 0.49 GB Max_CA 0 GB
[2025-07-02 12:39:33,967] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory: used = 60.21 GB, percent = 27.3%
[2025-07-02 12:39:33,968] [INFO] [stage_1_and_2.py:571:__init__] optimizer state initialized
[2025-07-02 12:39:36,078] [INFO] [utils.py:781:see_memory_usage] After initializing ZeRO optimizer
[2025-07-02 12:39:36,079] [INFO] [utils.py:782:see_memory_usage] MA 0.38 GB Max_MA 0.38 GB CA 0.49 GB Max_CA 0 GB
[2025-07-02 12:39:36,080] [INFO] [utils.py:789:see_memory_usage] CPU Virtual Memory: used = 60.21 GB, percent = 27.3%
[2025-07-02 12:39:36,082] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed Final Optimizer = DeepSpeedZeroOptimizer
[2025-07-02 12:39:36,082] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed using configured LR scheduler = None
[2025-07-02 12:39:36,082] [INFO] [logging.py:107:log_dist] [Rank 0] DeepSpeed LR Scheduler = None
[2025-07-02 12:39:36,083] [INFO] [logging.py:107:log_dist] [Rank 0] step=0, skipped=0, lr=[0.0], mom=[(0.9, 0.999)]
[2025-07-02 12:39:36,083] [INFO] [config.py:1014:print] DeepSpeedEngine configuration:
[2025-07-02 12:39:36,084] [INFO] [config.py:1018:print] activation_checkpointing_config {
"partition_activations": false,
"contiguous_memory_optimization": false,
"cpu_checkpointing": false,
"number_checkpoints": null,
"synchronize_checkpoint_boundary": false,
"profile": false
}
[2025-07-02 12:39:36,084] [INFO] [config.py:1018:print] aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'intra_op_parallelism': 1, 'single_submit': False, 'overlap_events': True, 'use_gds': False}
[2025-07-02 12:39:36,084] [INFO] [config.py:1018:print] amp_enabled .................. False
[2025-07-02 12:39:36,084] [INFO] [config.py:1018:print] amp_params ................... False
[2025-07-02 12:39:36,085] [INFO] [config.py:1018:print] autotuning_config ............ {
"enabled": false,
"start_step": null,
"end_step": null,
"metric_path": null,
"arg_mappings": null,
"metric": "throughput",
"model_info": null,
"results_dir": "autotuning_results",
"exps_dir": "autotuning_exps",
"overwrite": true,
"fast": true,
"start_profile_step": 3,
"end_profile_step": 5,
"tuner_type": "gridsearch",
"tuner_early_stopping": 5,
"tuner_num_trials": 50,
"model_info_path": null,
"mp_size": 1,
"max_train_batch_size": null,
"min_train_batch_size": 1,
"max_train_micro_batch_size_per_gpu": 1.024000e+03,
"min_train_micro_batch_size_per_gpu": 1,
"num_tuning_micro_batch_sizes": 3
}
[2025-07-02 12:39:36,085] [INFO] [config.py:1018:print] bfloat16_enabled ............. True
[2025-07-02 12:39:36,085] [INFO] [config.py:1018:print] bfloat16_immediate_grad_update True
[2025-07-02 12:39:36,085] [INFO] [config.py:1018:print] checkpoint_parallel_write_pipeline False
[2025-07-02 12:39:36,085] [INFO] [config.py:1018:print] checkpoint_tag_validation_enabled True
[2025-07-02 12:39:36,086] [INFO] [config.py:1018:print] checkpoint_tag_validation_fail False
[2025-07-02 12:39:36,086] [INFO] [config.py:1018:print] comms_config ................. <deepspeed.comm.config.DeepSpeedCommsConfig object at 0x7f895e31ba30>
[2025-07-02 12:39:36,086] [INFO] [config.py:1018:print] communication_data_type ...... None
[2025-07-02 12:39:36,086] [INFO] [config.py:1018:print] compile_config ............... deepcompile=False free_activation=False offload_activation=False offload_opt_states=False double_buffer=True symmetric_memory=False debug_log=False offload_parameters=False sync_before_reduce=False sync_after_reduce=False sync_before_allgather=False sync_after_allgather=False
[2025-07-02 12:39:36,086] [INFO] [config.py:1018:print] compression_config ........... {'weight_quantization': {'shared_parameters': {'enabled': False, 'quantizer_kernel': False, 'schedule_offset': 0, 'quantize_groups': 1, 'quantize_verbose': False, 'quantization_type': 'symmetric', 'quantize_weight_in_forward': False, 'rounding': 'nearest', 'fp16_mixed_quantize': False, 'quantize_change_ratio': 0.001}, 'different_groups': {}}, 'activation_quantization': {'shared_parameters': {'enabled': False, 'quantization_type': 'symmetric', 'range_calibration': 'dynamic', 'schedule_offset': 1000}, 'different_groups': {}}, 'sparse_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'row_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'head_pruning': {'shared_parameters': {'enabled': False, 'method': 'topk', 'schedule_offset': 1000}, 'different_groups': {}}, 'channel_pruning': {'shared_parameters': {'enabled': False, 'method': 'l1', 'schedule_offset': 1000}, 'different_groups': {}}, 'layer_reduction': {'enabled': False}}
[2025-07-02 12:39:36,086] [INFO] [config.py:1018:print] curriculum_enabled_legacy .... False
[2025-07-02 12:39:36,086] [INFO] [config.py:1018:print] curriculum_params_legacy ..... False
[2025-07-02 12:39:36,086] [INFO] [config.py:1018:print] data_efficiency_config ....... {'enabled': False, 'seed': 1234, 'data_sampling': {'enabled': False, 'num_epochs': 1000, 'num_workers': 0, 'pin_memory': False, 'curriculum_learning': {'enabled': False}, 'dynamic_batching': {'enabled': False, 'lr_scaling_method': 'linear', 'min_batch_size': 1, 'max_batch_size': None, 'sequence_picking_order': 'dataloader', 'verbose': False}}, 'data_routing': {'enabled': False, 'random_ltd': {'enabled': False, 'layer_token_lr_schedule': {'enabled': False}}}}
[2025-07-02 12:39:36,086] [INFO] [config.py:1018:print] data_efficiency_enabled ...... False
[2025-07-02 12:39:36,086] [INFO] [config.py:1018:print] dataloader_drop_last ......... False
[2025-07-02 12:39:36,086] [INFO] [config.py:1018:print] disable_allgather ............ False
[2025-07-02 12:39:36,086] [INFO] [config.py:1018:print] dump_state ................... False
[2025-07-02 12:39:36,086] [INFO] [config.py:1018:print] dynamic_loss_scale_args ...... None
[2025-07-02 12:39:36,086] [INFO] [config.py:1018:print] eigenvalue_enabled ........... False
[2025-07-02 12:39:36,087] [INFO] [config.py:1018:print] eigenvalue_gas_boundary_resolution 1
[2025-07-02 12:39:36,087] [INFO] [config.py:1018:print] eigenvalue_layer_name ........ bert.encoder.layer
[2025-07-02 12:39:36,087] [INFO] [config.py:1018:print] eigenvalue_layer_num ......... 0
[2025-07-02 12:39:36,087] [INFO] [config.py:1018:print] eigenvalue_max_iter .......... 100
[2025-07-02 12:39:36,088] [INFO] [config.py:1018:print] eigenvalue_stability ......... 1e-06
[2025-07-02 12:39:36,088] [INFO] [config.py:1018:print] eigenvalue_tol ............... 0.01
[2025-07-02 12:39:36,088] [INFO] [config.py:1018:print] eigenvalue_verbose ........... False
[2025-07-02 12:39:36,088] [INFO] [config.py:1018:print] elasticity_enabled ........... False
[2025-07-02 12:39:36,088] [INFO] [config.py:1018:print] flops_profiler_config ........ {
"enabled": false,
"recompute_fwd_factor": 0.0,
"profile_step": 1,
"module_depth": -1,
"top_modules": 1,
"detailed": true,
"output_file": null
}
[2025-07-02 12:39:36,088] [INFO] [config.py:1018:print] fp16_auto_cast ............... None
[2025-07-02 12:39:36,088] [INFO] [config.py:1018:print] fp16_enabled ................. False
[2025-07-02 12:39:36,088] [INFO] [config.py:1018:print] fp16_master_weights_and_gradients False
[2025-07-02 12:39:36,089] [INFO] [config.py:1018:print] global_rank .................. 0
[2025-07-02 12:39:36,089] [INFO] [config.py:1018:print] grad_accum_dtype ............. None
[2025-07-02 12:39:36,089] [INFO] [config.py:1018:print] gradient_accumulation_steps .. 32
[2025-07-02 12:39:36,089] [INFO] [config.py:1018:print] gradient_clipping ............ 1.0
[2025-07-02 12:39:36,089] [INFO] [config.py:1018:print] gradient_predivide_factor .... 1.0
[2025-07-02 12:39:36,089] [INFO] [config.py:1018:print] graph_harvesting ............. False
[2025-07-02 12:39:36,089] [INFO] [config.py:1018:print] hybrid_engine ................ enabled=False max_out_tokens=512 inference_tp_size=1 release_inference_cache=False pin_parameters=True tp_gather_partition_size=8
[2025-07-02 12:39:36,089] [INFO] [config.py:1018:print] initial_dynamic_scale ........ 1
[2025-07-02 12:39:36,090] [INFO] [config.py:1018:print] load_universal_checkpoint .... False
[2025-07-02 12:39:36,090] [INFO] [config.py:1018:print] loss_scale ................... 1.0
[2025-07-02 12:39:36,090] [INFO] [config.py:1018:print] memory_breakdown ............. False
[2025-07-02 12:39:36,090] [INFO] [config.py:1018:print] mics_hierarchial_params_gather False
[2025-07-02 12:39:36,090] [INFO] [config.py:1018:print] mics_shard_size .............. -1
[2025-07-02 12:39:36,090] [INFO] [config.py:1018:print] monitor_config ............... tensorboard=TensorBoardConfig(enabled=False, output_path='', job_name='DeepSpeedJobName') comet=CometConfig(enabled=False, samples_log_interval=100, project=None, workspace=None, api_key=None, experiment_name=None, experiment_key=None, online=None, mode=None) wandb=WandbConfig(enabled=False, group=None, team=None, project='deepspeed') csv_monitor=CSVConfig(enabled=False, output_path='', job_name='DeepSpeedJobName')
[2025-07-02 12:39:36,090] [INFO] [config.py:1018:print] nebula_config ................ {
"enabled": false,
"persistent_storage_path": null,
"persistent_time_interval": 100,
"num_of_version_in_retention": 2,
"enable_nebula_load": true,
"load_path": null
}
[2025-07-02 12:39:36,090] [INFO] [config.py:1018:print] optimizer_legacy_fusion ...... False
[2025-07-02 12:39:36,091] [INFO] [config.py:1018:print] optimizer_name ............... None
[2025-07-02 12:39:36,091] [INFO] [config.py:1018:print] optimizer_params ............. None
[2025-07-02 12:39:36,091] [INFO] [config.py:1018:print] pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0, 'pipe_partitioned': True, 'grad_partitioned': True}
[2025-07-02 12:39:36,091] [INFO] [config.py:1018:print] pld_enabled .................. False
[2025-07-02 12:39:36,091] [INFO] [config.py:1018:print] pld_params ................... False
[2025-07-02 12:39:36,091] [INFO] [config.py:1018:print] prescale_gradients ........... False
[2025-07-02 12:39:36,091] [INFO] [config.py:1018:print] scheduler_name ............... None
[2025-07-02 12:39:36,091] [INFO] [config.py:1018:print] scheduler_params ............. None
[2025-07-02 12:39:36,092] [INFO] [config.py:1018:print] seq_parallel_communication_data_type torch.float32
[2025-07-02 12:39:36,092] [INFO] [config.py:1018:print] sparse_attention ............. None
[2025-07-02 12:39:36,092] [INFO] [config.py:1018:print] sparse_gradients_enabled ..... False
[2025-07-02 12:39:36,092] [INFO] [config.py:1018:print] steps_per_print .............. inf
[2025-07-02 12:39:36,092] [INFO] [config.py:1018:print] tensor_parallel_config ....... dtype=torch.float16 autotp_size=0 tp_overlap_comm=False tensor_parallel=TPConfig(tp_size=1, tp_grain_size=1, mpu=None, tp_group=None) injection_policy_tuple=None keep_module_on_host=False replace_with_kernel_inject=False
[2025-07-02 12:39:36,092] [INFO] [config.py:1018:print] timers_config ................ enabled=True synchronized=True
[2025-07-02 12:39:36,092] [INFO] [config.py:1018:print] train_batch_size ............. 6144
[2025-07-02 12:39:36,092] [INFO] [config.py:1018:print] train_micro_batch_size_per_gpu 192
[2025-07-02 12:39:36,093] [INFO] [config.py:1018:print] use_data_before_expert_parallel_ False
[2025-07-02 12:39:36,093] [INFO] [config.py:1018:print] use_node_local_storage ....... False
[2025-07-02 12:39:36,093] [INFO] [config.py:1018:print] wall_clock_breakdown ......... False
[2025-07-02 12:39:36,093] [INFO] [config.py:1018:print] weight_quantization_config ... None
[2025-07-02 12:39:36,093] [INFO] [config.py:1018:print] world_size ................... 1
[2025-07-02 12:39:36,093] [INFO] [config.py:1018:print] zero_allow_untested_optimizer True
[2025-07-02 12:39:36,093] [INFO] [config.py:1018:print] zero_config .................. stage=2 contiguous_gradients=True reduce_scatter=True reduce_bucket_size=500000000 use_multi_rank_bucket_allreduce=True allgather_partitions=True allgather_bucket_size=500000000 overlap_comm=False load_from_fp32_weights=True elastic_checkpoint=False offload_param=DeepSpeedZeroOffloadParamConfig(device='none', nvme_path=None, buffer_count=5, buffer_size=100000000, max_in_cpu=1000000000, pin_memory=False) offload_optimizer=DeepSpeedZeroOffloadOptimizerConfig(device='none', nvme_path=None, buffer_count=4, pin_memory=False, pipeline_read=False, pipeline_write=False, fast_init=False, ratio=1.0) sub_group_size=1000000000 cpu_offload_param=None cpu_offload_use_pin_memory=None cpu_offload=None prefetch_bucket_size=50000000 param_persistence_threshold=100000 model_persistence_threshold=9223372036854775807 max_live_parameters=1000000000 max_reuse_distance=1000000000 gather_16bit_weights_on_model_save=False module_granularity_threshold=0 use_all_reduce_for_fetch_params=False stage3_gather_fp16_weights_on_model_save=False ignore_unused_parameters=True legacy_stage1=False round_robin_gradients=False zero_hpz_partition_size=1 zero_quantized_weights=False zero_quantized_nontrainable_weights=False zero_quantized_gradients=False zeropp_loco_param=None mics_shard_size=-1 mics_hierarchical_params_gather=False memory_efficient_linear=True pipeline_loading_checkpoint=False override_module_apply=True log_trace_cache_warnings=False
[2025-07-02 12:39:36,093] [INFO] [config.py:1018:print] zero_enabled ................. True
[2025-07-02 12:39:36,093] [INFO] [config.py:1018:print] zero_force_ds_cpu_optimizer .. True
[2025-07-02 12:39:36,094] [INFO] [config.py:1018:print] zero_optimization_stage ...... 2
[2025-07-02 12:39:36,094] [INFO] [config.py:1004:print_user_config] json = {
"train_batch_size": 6.144000e+03,
"train_micro_batch_size_per_gpu": 192,
"gradient_accumulation_steps": 32,
"zero_optimization": {
"stage": 2,
"offload_optimizer": {
"device": "none",
"nvme_path": null
},
"offload_param": {
"device": "none",
"nvme_path": null
},
"stage3_gather_16bit_weights_on_model_save": false
},
"gradient_clipping": 1.0,
"steps_per_print": inf,
"bf16": {
"enabled": true
},
"fp16": {
"enabled": false
},
"zero_allow_untested_optimizer": true
}