极客《深入MNIST》学习及bug调试

63 阅读 0 评论 42 点赞

我是靠谱客的博主仁爱春天，最近开发中收集的这篇文章主要介绍极客《深入MNIST》学习及bug调试，觉得挺不错的，现在分享给大家，希望可以做个参考。

概述

按照官网的代码会报下面的错，应该是我的GPU内存不够，我改小了一点batch，就可以了。
极客《深入MNIST》
http://wiki.jikexueyuan.com/project/tensorflow-zh/tutorials/mnist_pros.html
报错如下：
2018-08-09 13:28:21.996117: W C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:273] Allocator (GPU_0_bfc) ran out of memory trying to allocate 957.03MiB. Current allocation summary follows.
2018-08-09 13:28:21.996318: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (256): Total Chunks: 31, Chunks in use: 31. 7.8KiB allocated for chunks. 7.8KiB in use in bin. 2.1KiB client-requested in use in bin.
2018-08-09 13:28:21.997495: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (512): Total Chunks: 1, Chunks in use: 0. 768B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-08-09 13:28:21.997608: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (1024): Total Chunks: 1, Chunks in use: 1. 1.3KiB allocated for chunks. 1.3KiB in use in bin. 1.0KiB client-requested in use in bin.
2018-08-09 13:28:21.997694: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (2048): Total Chunks: 5, Chunks in use: 5. 16.3KiB allocated for chunks. 16.3KiB in use in bin. 15.6KiB client-requested in use in bin.
2018-08-09 13:28:21.997775: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (4096): Total Chunks: 5, Chunks in use: 5. 20.0KiB allocated for chunks. 20.0KiB in use in bin. 20.0KiB client-requested in use in bin.
2018-08-09 13:28:21.997852: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (8192): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-08-09 13:28:21.997928: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (16384): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-08-09 13:28:21.998009: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (32768): Total Chunks: 5, Chunks in use: 4. 196.8KiB allocated for chunks. 160.0KiB in use in bin. 160.0KiB client-requested in use in bin.
2018-08-09 13:28:21.998093: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (65536): Total Chunks: 2, Chunks in use: 1. 160.0KiB allocated for chunks. 78.3KiB in use in bin. 78.1KiB client-requested in use in bin.
2018-08-09 13:28:21.998175: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (131072): Total Chunks: 3, Chunks in use: 3. 600.0KiB allocated for chunks. 600.0KiB in use in bin. 600.0KiB client-requested in use in bin.
2018-08-09 13:28:21.998256: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (262144): Total Chunks: 1, Chunks in use: 1. 313.5KiB allocated for chunks. 313.5KiB in use in bin. 200.0KiB client-requested in use in bin.
2018-08-09 13:28:21.998335: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (524288): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-08-09 13:28:21.998412: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (1048576): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-08-09 13:28:21.998488: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (2097152): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-08-09 13:28:21.998568: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (4194304): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-08-09 13:28:21.998649: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (8388608): Total Chunks: 2, Chunks in use: 2. 24.50MiB allocated for chunks. 24.50MiB in use in bin. 24.50MiB client-requested in use in bin.
2018-08-09 13:28:21.998729: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (16777216): Total Chunks: 3, Chunks in use: 2. 51.75MiB allocated for chunks. 35.75MiB in use in bin. 24.50MiB client-requested in use in bin.
2018-08-09 13:28:21.998810: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (33554432): Total Chunks: 1, Chunks in use: 1. 51.46MiB allocated for chunks. 51.46MiB in use in bin. 29.91MiB client-requested in use in bin.
2018-08-09 13:28:21.998887: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (67108864): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-08-09 13:28:21.998964: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (134217728): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-08-09 13:28:21.999044: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:627] Bin (268435456): Total Chunks: 1, Chunks in use: 1. 1.00GiB allocated for chunks. 1.00GiB in use in bin. 957.03MiB client-requested in use in bin.
2018-08-09 13:28:21.999118: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:643] Bin for 957.03MiB was 256.00MiB, Chunk State:
2018-08-09 13:28:21.999187: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300560000 of size 1280
2018-08-09 13:28:21.999250: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300560500 of size 256
2018-08-09 13:28:21.999312: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300560600 of size 256
2018-08-09 13:28:21.999371: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300560700 of size 256
2018-08-09 13:28:21.999445: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300560800 of size 256
2018-08-09 13:28:21.999506: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300560900 of size 4096
2018-08-09 13:28:21.999571: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300561900 of size 256
2018-08-09 13:28:21.999655: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300561A00 of size 256
2018-08-09 13:28:21.999726: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300561B00 of size 256
2018-08-09 13:28:21.999787: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300561C00 of size 3328
2018-08-09 13:28:21.999847: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300562900 of size 256
2018-08-09 13:28:21.999907: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300562A00 of size 204800
2018-08-09 13:28:21.999966: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300594A00 of size 256
2018-08-09 13:28:22.000026: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300594B00 of size 4096
2018-08-09 13:28:22.000087: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300595B00 of size 40960
2018-08-09 13:28:22.000147: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 000000130059FB00 of size 256
2018-08-09 13:28:22.000206: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 000000130059FC00 of size 256
2018-08-09 13:28:22.000266: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 000000130059FD00 of size 256
2018-08-09 13:28:22.000324: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 000000130059FE00 of size 256
2018-08-09 13:28:22.000383: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 000000130059FF00 of size 256
2018-08-09 13:28:22.000443: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005A0000 of size 256
2018-08-09 13:28:22.000503: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005A0100 of size 256
2018-08-09 13:28:22.000567: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005A0200 of size 256
2018-08-09 13:28:22.000638: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005A0300 of size 256
2018-08-09 13:28:22.000697: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005A0400 of size 256
2018-08-09 13:28:22.000756: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005A0500 of size 256
2018-08-09 13:28:22.000815: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005A0900 of size 256
2018-08-09 13:28:22.000875: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005A0A00 of size 40960
2018-08-09 13:28:22.000935: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005AAA00 of size 80128
2018-08-09 13:28:22.000995: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005D2A00 of size 256
2018-08-09 13:28:22.001055: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005D2B00 of size 4096
2018-08-09 13:28:22.001115: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005D3B00 of size 3328
2018-08-09 13:28:22.001174: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005DDB00 of size 256
2018-08-09 13:28:22.001234: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005DDC00 of size 256
2018-08-09 13:28:22.001293: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005DDD00 of size 256
2018-08-09 13:28:22.001352: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005DDE00 of size 3328
2018-08-09 13:28:22.001412: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005DEB00 of size 3328
2018-08-09 13:28:22.001472: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005DF800 of size 256
2018-08-09 13:28:22.001531: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005DF900 of size 256
2018-08-09 13:28:22.001598: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013005DFA00 of size 204800
2018-08-09 13:28:22.001662: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300611A00 of size 321024
2018-08-09 13:28:22.001724: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:670] Free at 00000013005A0600 of size 768
2018-08-09 13:28:22.001783: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:670] Free at 00000013005BE300 of size 83712
2018-08-09 13:28:22.001843: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:670] Free at 00000013005D4800 of size 37632
2018-08-09 13:28:22.001904: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001300660000 of size 16777216
2018-08-09 13:28:22.001964: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:670] Free at 0000001301860000 of size 16777216
2018-08-09 13:28:22.002023: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001302860000 of size 256
2018-08-09 13:28:22.002083: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001302860100 of size 256
2018-08-09 13:28:22.002144: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001302860200 of size 12845056
2018-08-09 13:28:22.002205: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013034A0200 of size 20708864
2018-08-09 13:28:22.002265: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001304860000 of size 4096
2018-08-09 13:28:22.002325: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001304861000 of size 4096
2018-08-09 13:28:22.002384: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001304862000 of size 40960
2018-08-09 13:28:22.002444: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 000000130486C000 of size 40960
2018-08-09 13:28:22.002504: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001304876000 of size 256
2018-08-09 13:28:22.002568: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001304876100 of size 256
2018-08-09 13:28:22.002630: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001304876200 of size 3328
2018-08-09 13:28:22.002691: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001304876F00 of size 204800
2018-08-09 13:28:22.002751: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013048A8F00 of size 12845056
2018-08-09 13:28:22.002811: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 00000013054E8F00 of size 53965056
2018-08-09 13:28:22.002872: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:661] Chunk at 0000001308B60000 of size 1073741824
2018-08-09 13:28:22.002929: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:676] Summary of in-use Chunks by size:
2018-08-09 13:28:22.002992: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:679] 31 Chunks of size 256 totalling 7.8KiB
2018-08-09 13:28:22.003054: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:679] 1 Chunks of size 1280 totalling 1.3KiB
2018-08-09 13:28:22.003115: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:679] 5 Chunks of size 3328 totalling 16.3KiB
2018-08-09 13:28:22.003176: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:679] 5 Chunks of size 4096 totalling 20.0KiB
2018-08-09 13:28:22.003244: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:679] 4 Chunks of size 40960 totalling 160.0KiB
2018-08-09 13:28:22.003311: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:679] 1 Chunks of size 80128 totalling 78.3KiB
2018-08-09 13:28:22.003372: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:679] 3 Chunks of size 204800 totalling 600.0KiB
2018-08-09 13:28:22.003433: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:679] 1 Chunks of size 321024 totalling 313.5KiB
2018-08-09 13:28:22.003495: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:679] 2 Chunks of size 12845056 totalling 24.50MiB
2018-08-09 13:28:22.003561: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:679] 1 Chunks of size 16777216 totalling 16.00MiB
2018-08-09 13:28:22.003661: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:679] 1 Chunks of size 20708864 totalling 19.75MiB
2018-08-09 13:28:22.003757: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:679] 1 Chunks of size 53965056 totalling 51.46MiB
2018-08-09 13:28:22.003849: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:679] 1 Chunks of size 1073741824 totalling 1.00GiB
2018-08-09 13:28:22.003916: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:683] Sum Total of in-use chunks: 1.11GiB
2018-08-09 13:28:22.003983: I C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:685] Stats:
Limit: 1555401932
InUse: 1192108800
MaxInUse: 1192508928
NumAllocs: 6997
MaxAllocSize: 1073741824

2018-08-09 13:28:22.004094: W C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcorecommon_runtimebfc_allocator.cc:277] **********x************************************************************************************xxxxx
2018-08-09 13:28:22.004183: W C:tf_jenkinshomeworkspacerel-winMwindows-gpuPY35tensorflowcoreframeworkop_kernel.cc:1192] Resource exhausted: OOM when allocating tensor with shape[10000,32,28,28]
Traceback (most recent call last):
File “C:UsersThink2Anaconda3libsite-packagestensorflowpythonclientsession.py”, line 1323, in _do_call
return fn(*args)
File “C:UsersThink2Anaconda3libsite-packagestensorflowpythonclientsession.py”, line 1302, in _run_fn
status, run_metadata)
File “C:UsersThink2Anaconda3libsite-packagestensorflowpythonframeworkerrors_impl.py”, line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[10000,32,28,28]
[[Node: Conv2D = Conv2D[T=DT_FLOAT, data_format=”NHWC”, padding=”SAME”, strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device=”/job:localhost/replica:0/task:0/device:GPU:0”](Reshape, Variable/read)]]
[[Node: add_3/_7 = _Recvclient_terminated=false, recv_device=”/job:localhost/replica:0/task:0/device:CPU:0”, send_device=”/job:localhost/replica:0/task:0/device:GPU:0”, send_device_incarnation=1, tensor_name=”edge_49_add_3”, tensor_type=DT_FLOAT, _device=”/job:localhost/replica:0/task:0/device:CPU:0”]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “mnist.py”, line 68, in
x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))
File “C:UsersThink2Anaconda3libsite-packagestensorflowpythonframeworkops.py”, line 570, in eval
return _eval_using_default_session(self, feed_dict, self.graph, session)
File “C:UsersThink2Anaconda3libsite-packagestensorflowpythonframeworkops.py”, line 4455, in _eval_using_default_session
return session.run(tensors, feed_dict)
File “C:UsersThink2Anaconda3libsite-packagestensorflowpythonclientsession.py”, line 889, in run
run_metadata_ptr)
File “C:UsersThink2Anaconda3libsite-packagestensorflowpythonclientsession.py”, line 1120, in _run
feed_dict_tensor, options, run_metadata)
File “C:UsersThink2Anaconda3libsite-packagestensorflowpythonclientsession.py”, line 1317, in _do_run
options, run_metadata)
File “C:UsersThink2Anaconda3libsite-packagestensorflowpythonclientsession.py”, line 1336, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[10000,32,28,28]
[[Node: Conv2D = Conv2D[T=DT_FLOAT, data_format=”NHWC”, padding=”SAME”, strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device=”/job:localhost/replica:0/task:0/device:GPU:0”](Reshape, Variable/read)]]
[[Node: add_3/_7 = _Recvclient_terminated=false, recv_device=”/job:localhost/replica:0/task:0/device:CPU:0”, send_device=”/job:localhost/replica:0/task:0/device:GPU:0”, send_device_incarnation=1, tensor_name=”edge_49_add_3”, tensor_type=DT_FLOAT, _device=”/job:localhost/replica:0/task:0/device:CPU:0”]]

Caused by op ‘Conv2D’, defined at:
File “mnist.py”, line 30, in
h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
File “mnist.py”, line 19, in conv2d
return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding=’SAME’)
File “C:UsersThink2Anaconda3libsite-packagestensorflowpythonopsgen_nn_ops.py”, line 630, in conv2d
data_format=data_format, name=name)
File “C:UsersThink2Anaconda3libsite-packagestensorflowpythonframeworkop_def_library.py”, line 787, in _apply_op_helper
op_def=op_def)
File “C:UsersThink2Anaconda3libsite-packagestensorflowpythonframeworkops.py”, line 2956, in create_op
op_def=op_def)
File “C:UsersThink2Anaconda3libsite-packagestensorflowpythonframeworkops.py”, line 1470, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[10000,32,28,28]
[[Node: Conv2D = Conv2D[T=DT_FLOAT, data_format=”NHWC”, padding=”SAME”, strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device=”/job:localhost/replica:0/task:0/device:GPU:0”](Reshape, Variable/read)]]
[[Node: add_3/_7 = _Recvclient_terminated=false, recv_device=”/job:localhost/replica:0/task:0/device:CPU:0”, send_device=”/job:localhost/replica:0/task:0/device:GPU:0”, send_device_incarnation=1, tensor_name=”edge_49_add_3”, tensor_type=DT_FLOAT, _device=”/job:localhost/replica:0/task:0/device:CPU:0”]]

如果 GPU空间不够可以改小一点batch:
for i in range(10):
batch = mnist.train.next_batch(5000)
print (“test accuracy %g”%accuracy.eval(feed_dict={
x: batch[0], y_:batch[1], keep_prob: 1.0}))

code 如下：

from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets(‘MNIST’, one_hot=True)

import tensorflow as tf
sess = tf.InteractiveSession()

x = tf.placeholder(“float”, shape=[None, 784])
y_ = tf.placeholder(“float”, shape=[None, 10])

def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)

def bias_variable(shape):
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial)

def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding=’SAME’)

def max_pool_2x2(x):
return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding=’SAME’)

W_conv1 = weight_variable([5, 5, 1, 32])
b_conv1 = bias_variable([32])

x_image = tf.reshape(x, [-1,28,28,1])

h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)

W_conv2 = weight_variable([5, 5, 32, 64])
b_conv2 = bias_variable([64])

h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)

W_fc1 = weight_variable([7 * 7 * 64, 1024])
b_fc1 = bias_variable([1024])

h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)

keep_prob = tf.placeholder(“float”)
h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)

W_fc2 = weight_variable([1024, 10])
b_fc2 = bias_variable([10])

y_conv=tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)

cross_entropy = -tf.reduce_sum(y_*tf.log(y_conv))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
correct_prediction = tf.equal(tf.argmax(y_conv,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, “float”))
sess.run(tf.initialize_all_variables())
for i in range(2000):
batch = mnist.train.next_batch(50)
if i%100 == 0:
train_accuracy = accuracy.eval(feed_dict={
x:batch[0], y_: batch[1], keep_prob: 1.0})
print (“step %d, training accuracy %g”%(i, train_accuracy))
train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})

for i in range(10):
batch = mnist.train.next_batch(5000)
print (“test accuracy %g”%accuracy.eval(feed_dict={
x: batch[0], y_:batch[1], keep_prob: 1.0}))