Comment
Author: Admin | 2025-04-27
Loaded 'preprocessing'tokenizer: 2 1 2I1121 06:29:49.732066 6859 model_lifecycle.cc:819] successfully loaded 'preprocessing'tokenizer: 2 1 2I1121 06:29:49.740389 6862 model_lifecycle.cc:819] successfully loaded 'preprocessing'[TensorRT-LLM][INFO] TRTGptModel maxNumSequences: 8[TensorRT-LLM][INFO] TRTGptModel maxBatchSize: 4[TensorRT-LLM][INFO] TRTGptModel enableTrtOverlap: 1[TensorRT-LLM][INFO] Loaded engine size: 32953 MiB[TensorRT-LLM][WARNING] Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.[TensorRT-LLM][INFO] TRTGptModel maxNumSequences: 8[TensorRT-LLM][INFO] TRTGptModel maxBatchSize: 4[TensorRT-LLM][INFO] TRTGptModel enableTrtOverlap: 1[TensorRT-LLM][INFO] Loaded engine size: 32953 MiB[TensorRT-LLM][WARNING] Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.[TensorRT-LLM][INFO] TRTGptModel maxNumSequences: 8[TensorRT-LLM][INFO] TRTGptModel maxBatchSize: 4[TensorRT-LLM][INFO] TRTGptModel enableTrtOverlap: 1[TensorRT-LLM][INFO] Loaded engine size: 32953 MiB[TensorRT-LLM][WARNING] Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.[TensorRT-LLM][INFO] TRTGptModel maxNumSequences: 8[TensorRT-LLM][INFO] TRTGptModel maxBatchSize: 4[TensorRT-LLM][INFO] TRTGptModel enableTrtOverlap: 1[TensorRT-LLM][INFO] Loaded engine size: 32953 MiB[TensorRT-LLM][WARNING] Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.[TensorRT-LLM][INFO] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 33240, GPU 34890 (MiB)[TensorRT-LLM][INFO] [MemUsageChange] Init cuDNN: CPU +1, GPU +10, now: CPU 33241, GPU 34900 (MiB)[TensorRT-LLM][INFO] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 33240, GPU 34890 (MiB)[TensorRT-LLM][INFO] [MemUsageChange] Init cuDNN: CPU +1, GPU +10, now: CPU 33241, GPU 34900 (MiB)archimedes-open-api-70b-1325343-5875b79c44-lzlpt:6859:6969 [0] NCCL INFO Bootstrap : Using eth0:10.156.28.900>archimedes-open-api-70b-1325343-5875b79c44-lzlpt:6859:6969 [0] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v7 symbol.archimedes-open-api-70b-1325343-5875b79c44-lzlpt:6859:6969 [0] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin v6 (v6)archimedes-open-api-70b-1325343-5875b79c44-lzlpt:6859:6969 [0] NCCL
Add Comment