Tensorflow::Key Variable not found in checkpoint











up vote
-1
down vote

favorite












I am trying to restore a checkpoint but I get the following error. How should I fix it?



def image_filling(image, mask, args):

    with tf.Graph().as_default():

        dummy = tf.Variable(0)  # dummy variable !!!

        with tf.Session() as sess:

            sess.run(tf.global_variables_initializer())

            saver = tf.train.Saver()

            saver.restore(sess, "image_completion/gan.ckpt")


Error is:



$ python completion_DCGAN.py 

2018-11-19 03:36:55.472699: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA

2018-11-19 03:36:55.641138: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1405] Found device 0 with properties: 

name: Tesla P100-PCIE-12GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285

pciBusID: 0000:02:00.0

totalMemory: 11.91GiB freeMemory: 11.63GiB

2018-11-19 03:36:55.641170: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1484] Adding visible gpu devices: 0

2018-11-19 03:36:55.928819: I tensorflow/core/common_runtime/gpu/gpu_device.cc:965] Device interconnect StreamExecutor with strength 1 edge matrix:

2018-11-19 03:36:55.928849: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971]      0 

2018-11-19 03:36:55.928855: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 0:   N 

2018-11-19 03:36:55.929099: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11251 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-12GB, pci bus id: 0000:02:00.0, compute capability: 6.0)

2018-11-19 03:36:56.139821: I tensorflow/core/common_runtime/process_util.cc:69] Creating new thread pool with default inter op setting: 28. Tune using inter_op_parallelism_threads for best performance.

2018-11-19 03:36:56.162713: W tensorflow/core/framework/op_kernel.cc:1275] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key Variable not found in checkpoint

Traceback (most recent call last):

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1278, in _do_call

    return fn(*args)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1263, in _run_fn

    options, feed_dict, fetch_list, target_list, run_metadata)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _call_tf_sessionrun

    run_metadata)

tensorflow.python.framework.errors_impl.NotFoundError: Key Variable not found in checkpoint

     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]



During handling of the above exception, another exception occurred:



Traceback (most recent call last):

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1725, in restore

    {self.saver_def.filename_tensor_name: save_path})

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 877, in run

    run_metadata_ptr)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1100, in _run

    feed_dict_tensor, options, run_metadata)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1272, in _do_run

    run_metadata)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1291, in _do_call

    raise type(e)(node_def, op, message)

tensorflow.python.framework.errors_impl.NotFoundError: Key Variable not found in checkpoint

     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]



Caused by op 'save/RestoreV2', defined at:

  File "completion_DCGAN.py", line 299, in <module>

    image_filling(real_batch, mask, args)

  File "completion_DCGAN.py", line 267, in image_filling

    saver = tf.train.Saver()

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1281, in __init__

    self.build()

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1293, in build

    self._build(self._filename, build_save=True, build_restore=True)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1330, in _build

    build_save=build_save, build_restore=build_restore)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 778, in _build_internal

    restore_sequentially, reshape)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 397, in _AddRestoreOps

    restore_sequentially)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 829, in bulk_restore

    return io_ops.restore_v2(filename_tensor, names, slices, dtypes)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1463, in restore_v2

    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper

    op_def=op_def)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func

    return func(*args, **kwargs)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op

    op_def=op_def)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1717, in __init__

    self._traceback = tf_stack.extract_stack()



NotFoundError (see above for traceback): Key Variable not found in checkpoint

     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]





During handling of the above exception, another exception occurred:



Traceback (most recent call last):

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1737, in restore

    checkpointable.OBJECT_GRAPH_PROTO_KEY)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 351, in get_tensor

    status)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 519, in __exit__

    c_api.TF_GetCode(self.status.status))

tensorflow.python.framework.errors_impl.NotFoundError: Key _CHECKPOINTABLE_OBJECT_GRAPH not found in checkpoint



During handling of the above exception, another exception occurred:



Traceback (most recent call last):

  File "completion_DCGAN.py", line 299, in <module>

    image_filling(real_batch, mask, args)

  File "completion_DCGAN.py", line 268, in image_filling

    saver.restore(sess, "image_completion/gan.ckpt")

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1743, in restore

    err, "a Variable name or other graph key that is missing")

tensorflow.python.framework.errors_impl.NotFoundError: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:



Key Variable not found in checkpoint

     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]



Caused by op 'save/RestoreV2', defined at:

  File "completion_DCGAN.py", line 299, in <module>

    image_filling(real_batch, mask, args)

  File "completion_DCGAN.py", line 267, in image_filling

    saver = tf.train.Saver()

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1281, in __init__

    self.build()

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1293, in build

    self._build(self._filename, build_save=True, build_restore=True)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1330, in _build

    build_save=build_save, build_restore=build_restore)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 778, in _build_internal

    restore_sequentially, reshape)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 397, in _AddRestoreOps

    restore_sequentially)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 829, in bulk_restore

    return io_ops.restore_v2(filename_tensor, names, slices, dtypes)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1463, in restore_v2

    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper

    op_def=op_def)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func

    return func(*args, **kwargs)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op

    op_def=op_def)

  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1717, in __init__

    self._traceback = tf_stack.extract_stack()



NotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:



Key Variable not found in checkpoint

     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]





python tensorflow deep-learning generative-adversarial-network







share|improve this question




























    up vote
    -1
    down vote

    favorite












    I am trying to restore a checkpoint but I get the following error. How should I fix it?



    def image_filling(image, mask, args):
    
    with tf.Graph().as_default():
    
        dummy = tf.Variable(0)  # dummy variable !!!
    
        with tf.Session() as sess:
    
            sess.run(tf.global_variables_initializer())
    
            saver = tf.train.Saver()
    
            saver.restore(sess, "image_completion/gan.ckpt")


    Error is:



    $ python completion_DCGAN.py 
    
2018-11-19 03:36:55.472699: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
    
2018-11-19 03:36:55.641138: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1405] Found device 0 with properties: 
    
name: Tesla P100-PCIE-12GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285
    
pciBusID: 0000:02:00.0
    
totalMemory: 11.91GiB freeMemory: 11.63GiB
    
2018-11-19 03:36:55.641170: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1484] Adding visible gpu devices: 0
    
2018-11-19 03:36:55.928819: I tensorflow/core/common_runtime/gpu/gpu_device.cc:965] Device interconnect StreamExecutor with strength 1 edge matrix:
    
2018-11-19 03:36:55.928849: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971]      0 
    
2018-11-19 03:36:55.928855: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 0:   N 
    
2018-11-19 03:36:55.929099: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11251 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-12GB, pci bus id: 0000:02:00.0, compute capability: 6.0)
    
2018-11-19 03:36:56.139821: I tensorflow/core/common_runtime/process_util.cc:69] Creating new thread pool with default inter op setting: 28. Tune using inter_op_parallelism_threads for best performance.
    
2018-11-19 03:36:56.162713: W tensorflow/core/framework/op_kernel.cc:1275] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key Variable not found in checkpoint
    
Traceback (most recent call last):
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1278, in _do_call
    
    return fn(*args)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1263, in _run_fn
    
    options, feed_dict, fetch_list, target_list, run_metadata)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _call_tf_sessionrun
    
    run_metadata)
    
tensorflow.python.framework.errors_impl.NotFoundError: Key Variable not found in checkpoint
    
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
    

    
During handling of the above exception, another exception occurred:
    

    
Traceback (most recent call last):
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1725, in restore
    
    {self.saver_def.filename_tensor_name: save_path})
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 877, in run
    
    run_metadata_ptr)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1100, in _run
    
    feed_dict_tensor, options, run_metadata)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1272, in _do_run
    
    run_metadata)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1291, in _do_call
    
    raise type(e)(node_def, op, message)
    
tensorflow.python.framework.errors_impl.NotFoundError: Key Variable not found in checkpoint
    
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
    

    
Caused by op 'save/RestoreV2', defined at:
    
  File "completion_DCGAN.py", line 299, in <module>
    
    image_filling(real_batch, mask, args)
    
  File "completion_DCGAN.py", line 267, in image_filling
    
    saver = tf.train.Saver()
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1281, in __init__
    
    self.build()
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1293, in build
    
    self._build(self._filename, build_save=True, build_restore=True)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1330, in _build
    
    build_save=build_save, build_restore=build_restore)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 778, in _build_internal
    
    restore_sequentially, reshape)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 397, in _AddRestoreOps
    
    restore_sequentially)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 829, in bulk_restore
    
    return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1463, in restore_v2
    
    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    
    op_def=op_def)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func
    
    return func(*args, **kwargs)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op
    
    op_def=op_def)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1717, in __init__
    
    self._traceback = tf_stack.extract_stack()
    

    
NotFoundError (see above for traceback): Key Variable not found in checkpoint
    
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
    

    

    
During handling of the above exception, another exception occurred:
    

    
Traceback (most recent call last):
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1737, in restore
    
    checkpointable.OBJECT_GRAPH_PROTO_KEY)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 351, in get_tensor
    
    status)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 519, in __exit__
    
    c_api.TF_GetCode(self.status.status))
    
tensorflow.python.framework.errors_impl.NotFoundError: Key _CHECKPOINTABLE_OBJECT_GRAPH not found in checkpoint
    

    
During handling of the above exception, another exception occurred:
    

    
Traceback (most recent call last):
    
  File "completion_DCGAN.py", line 299, in <module>
    
    image_filling(real_batch, mask, args)
    
  File "completion_DCGAN.py", line 268, in image_filling
    
    saver.restore(sess, "image_completion/gan.ckpt")
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1743, in restore
    
    err, "a Variable name or other graph key that is missing")
    
tensorflow.python.framework.errors_impl.NotFoundError: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
    

    
Key Variable not found in checkpoint
    
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
    

    
Caused by op 'save/RestoreV2', defined at:
    
  File "completion_DCGAN.py", line 299, in <module>
    
    image_filling(real_batch, mask, args)
    
  File "completion_DCGAN.py", line 267, in image_filling
    
    saver = tf.train.Saver()
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1281, in __init__
    
    self.build()
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1293, in build
    
    self._build(self._filename, build_save=True, build_restore=True)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1330, in _build
    
    build_save=build_save, build_restore=build_restore)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 778, in _build_internal
    
    restore_sequentially, reshape)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 397, in _AddRestoreOps
    
    restore_sequentially)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 829, in bulk_restore
    
    return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1463, in restore_v2
    
    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    
    op_def=op_def)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func
    
    return func(*args, **kwargs)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op
    
    op_def=op_def)
    
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1717, in __init__
    
    self._traceback = tf_stack.extract_stack()
    

    
NotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
    

    
Key Variable not found in checkpoint
    
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]





    python tensorflow deep-learning generative-adversarial-network







    share|improve this question


























      up vote
      -1
      down vote

      favorite









      up vote
      -1
      down vote

      favorite











      I am trying to restore a checkpoint but I get the following error. How should I fix it?



      def image_filling(image, mask, args):
      
    with tf.Graph().as_default():
      
        dummy = tf.Variable(0)  # dummy variable !!!
      
        with tf.Session() as sess:
      
            sess.run(tf.global_variables_initializer())
      
            saver = tf.train.Saver()
      
            saver.restore(sess, "image_completion/gan.ckpt")


      Error is:



      $ python completion_DCGAN.py 
      
2018-11-19 03:36:55.472699: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
      
2018-11-19 03:36:55.641138: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1405] Found device 0 with properties: 
      
name: Tesla P100-PCIE-12GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285
      
pciBusID: 0000:02:00.0
      
totalMemory: 11.91GiB freeMemory: 11.63GiB
      
2018-11-19 03:36:55.641170: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1484] Adding visible gpu devices: 0
      
2018-11-19 03:36:55.928819: I tensorflow/core/common_runtime/gpu/gpu_device.cc:965] Device interconnect StreamExecutor with strength 1 edge matrix:
      
2018-11-19 03:36:55.928849: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971]      0 
      
2018-11-19 03:36:55.928855: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 0:   N 
      
2018-11-19 03:36:55.929099: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11251 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-12GB, pci bus id: 0000:02:00.0, compute capability: 6.0)
      
2018-11-19 03:36:56.139821: I tensorflow/core/common_runtime/process_util.cc:69] Creating new thread pool with default inter op setting: 28. Tune using inter_op_parallelism_threads for best performance.
      
2018-11-19 03:36:56.162713: W tensorflow/core/framework/op_kernel.cc:1275] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key Variable not found in checkpoint
      
Traceback (most recent call last):
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1278, in _do_call
      
    return fn(*args)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1263, in _run_fn
      
    options, feed_dict, fetch_list, target_list, run_metadata)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _call_tf_sessionrun
      
    run_metadata)
      
tensorflow.python.framework.errors_impl.NotFoundError: Key Variable not found in checkpoint
      
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
      

      
During handling of the above exception, another exception occurred:
      

      
Traceback (most recent call last):
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1725, in restore
      
    {self.saver_def.filename_tensor_name: save_path})
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 877, in run
      
    run_metadata_ptr)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1100, in _run
      
    feed_dict_tensor, options, run_metadata)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1272, in _do_run
      
    run_metadata)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1291, in _do_call
      
    raise type(e)(node_def, op, message)
      
tensorflow.python.framework.errors_impl.NotFoundError: Key Variable not found in checkpoint
      
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
      

      
Caused by op 'save/RestoreV2', defined at:
      
  File "completion_DCGAN.py", line 299, in <module>
      
    image_filling(real_batch, mask, args)
      
  File "completion_DCGAN.py", line 267, in image_filling
      
    saver = tf.train.Saver()
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1281, in __init__
      
    self.build()
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1293, in build
      
    self._build(self._filename, build_save=True, build_restore=True)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1330, in _build
      
    build_save=build_save, build_restore=build_restore)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 778, in _build_internal
      
    restore_sequentially, reshape)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 397, in _AddRestoreOps
      
    restore_sequentially)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 829, in bulk_restore
      
    return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1463, in restore_v2
      
    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
      
    op_def=op_def)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func
      
    return func(*args, **kwargs)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op
      
    op_def=op_def)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1717, in __init__
      
    self._traceback = tf_stack.extract_stack()
      

      
NotFoundError (see above for traceback): Key Variable not found in checkpoint
      
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
      

      

      
During handling of the above exception, another exception occurred:
      

      
Traceback (most recent call last):
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1737, in restore
      
    checkpointable.OBJECT_GRAPH_PROTO_KEY)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 351, in get_tensor
      
    status)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 519, in __exit__
      
    c_api.TF_GetCode(self.status.status))
      
tensorflow.python.framework.errors_impl.NotFoundError: Key _CHECKPOINTABLE_OBJECT_GRAPH not found in checkpoint
      

      
During handling of the above exception, another exception occurred:
      

      
Traceback (most recent call last):
      
  File "completion_DCGAN.py", line 299, in <module>
      
    image_filling(real_batch, mask, args)
      
  File "completion_DCGAN.py", line 268, in image_filling
      
    saver.restore(sess, "image_completion/gan.ckpt")
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1743, in restore
      
    err, "a Variable name or other graph key that is missing")
      
tensorflow.python.framework.errors_impl.NotFoundError: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
      

      
Key Variable not found in checkpoint
      
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
      

      
Caused by op 'save/RestoreV2', defined at:
      
  File "completion_DCGAN.py", line 299, in <module>
      
    image_filling(real_batch, mask, args)
      
  File "completion_DCGAN.py", line 267, in image_filling
      
    saver = tf.train.Saver()
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1281, in __init__
      
    self.build()
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1293, in build
      
    self._build(self._filename, build_save=True, build_restore=True)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1330, in _build
      
    build_save=build_save, build_restore=build_restore)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 778, in _build_internal
      
    restore_sequentially, reshape)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 397, in _AddRestoreOps
      
    restore_sequentially)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 829, in bulk_restore
      
    return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1463, in restore_v2
      
    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
      
    op_def=op_def)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func
      
    return func(*args, **kwargs)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op
      
    op_def=op_def)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1717, in __init__
      
    self._traceback = tf_stack.extract_stack()
      

      
NotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
      

      
Key Variable not found in checkpoint
      
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]





      python tensorflow deep-learning generative-adversarial-network







      share|improve this question















      I am trying to restore a checkpoint but I get the following error. How should I fix it?



      def image_filling(image, mask, args):
      
    with tf.Graph().as_default():
      
        dummy = tf.Variable(0)  # dummy variable !!!
      
        with tf.Session() as sess:
      
            sess.run(tf.global_variables_initializer())
      
            saver = tf.train.Saver()
      
            saver.restore(sess, "image_completion/gan.ckpt")


      Error is:



      $ python completion_DCGAN.py 
      
2018-11-19 03:36:55.472699: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
      
2018-11-19 03:36:55.641138: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1405] Found device 0 with properties: 
      
name: Tesla P100-PCIE-12GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285
      
pciBusID: 0000:02:00.0
      
totalMemory: 11.91GiB freeMemory: 11.63GiB
      
2018-11-19 03:36:55.641170: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1484] Adding visible gpu devices: 0
      
2018-11-19 03:36:55.928819: I tensorflow/core/common_runtime/gpu/gpu_device.cc:965] Device interconnect StreamExecutor with strength 1 edge matrix:
      
2018-11-19 03:36:55.928849: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971]      0 
      
2018-11-19 03:36:55.928855: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 0:   N 
      
2018-11-19 03:36:55.929099: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11251 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-12GB, pci bus id: 0000:02:00.0, compute capability: 6.0)
      
2018-11-19 03:36:56.139821: I tensorflow/core/common_runtime/process_util.cc:69] Creating new thread pool with default inter op setting: 28. Tune using inter_op_parallelism_threads for best performance.
      
2018-11-19 03:36:56.162713: W tensorflow/core/framework/op_kernel.cc:1275] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key Variable not found in checkpoint
      
Traceback (most recent call last):
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1278, in _do_call
      
    return fn(*args)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1263, in _run_fn
      
    options, feed_dict, fetch_list, target_list, run_metadata)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _call_tf_sessionrun
      
    run_metadata)
      
tensorflow.python.framework.errors_impl.NotFoundError: Key Variable not found in checkpoint
      
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
      

      
During handling of the above exception, another exception occurred:
      

      
Traceback (most recent call last):
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1725, in restore
      
    {self.saver_def.filename_tensor_name: save_path})
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 877, in run
      
    run_metadata_ptr)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1100, in _run
      
    feed_dict_tensor, options, run_metadata)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1272, in _do_run
      
    run_metadata)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1291, in _do_call
      
    raise type(e)(node_def, op, message)
      
tensorflow.python.framework.errors_impl.NotFoundError: Key Variable not found in checkpoint
      
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
      

      
Caused by op 'save/RestoreV2', defined at:
      
  File "completion_DCGAN.py", line 299, in <module>
      
    image_filling(real_batch, mask, args)
      
  File "completion_DCGAN.py", line 267, in image_filling
      
    saver = tf.train.Saver()
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1281, in __init__
      
    self.build()
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1293, in build
      
    self._build(self._filename, build_save=True, build_restore=True)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1330, in _build
      
    build_save=build_save, build_restore=build_restore)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 778, in _build_internal
      
    restore_sequentially, reshape)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 397, in _AddRestoreOps
      
    restore_sequentially)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 829, in bulk_restore
      
    return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1463, in restore_v2
      
    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
      
    op_def=op_def)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func
      
    return func(*args, **kwargs)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op
      
    op_def=op_def)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1717, in __init__
      
    self._traceback = tf_stack.extract_stack()
      

      
NotFoundError (see above for traceback): Key Variable not found in checkpoint
      
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
      

      

      
During handling of the above exception, another exception occurred:
      

      
Traceback (most recent call last):
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1737, in restore
      
    checkpointable.OBJECT_GRAPH_PROTO_KEY)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 351, in get_tensor
      
    status)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 519, in __exit__
      
    c_api.TF_GetCode(self.status.status))
      
tensorflow.python.framework.errors_impl.NotFoundError: Key _CHECKPOINTABLE_OBJECT_GRAPH not found in checkpoint
      

      
During handling of the above exception, another exception occurred:
      

      
Traceback (most recent call last):
      
  File "completion_DCGAN.py", line 299, in <module>
      
    image_filling(real_batch, mask, args)
      
  File "completion_DCGAN.py", line 268, in image_filling
      
    saver.restore(sess, "image_completion/gan.ckpt")
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1743, in restore
      
    err, "a Variable name or other graph key that is missing")
      
tensorflow.python.framework.errors_impl.NotFoundError: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
      

      
Key Variable not found in checkpoint
      
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
      

      
Caused by op 'save/RestoreV2', defined at:
      
  File "completion_DCGAN.py", line 299, in <module>
      
    image_filling(real_batch, mask, args)
      
  File "completion_DCGAN.py", line 267, in image_filling
      
    saver = tf.train.Saver()
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1281, in __init__
      
    self.build()
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1293, in build
      
    self._build(self._filename, build_save=True, build_restore=True)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1330, in _build
      
    build_save=build_save, build_restore=build_restore)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 778, in _build_internal
      
    restore_sequentially, reshape)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 397, in _AddRestoreOps
      
    restore_sequentially)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 829, in bulk_restore
      
    return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1463, in restore_v2
      
    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
      
    op_def=op_def)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 454, in new_func
      
    return func(*args, **kwargs)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3155, in create_op
      
    op_def=op_def)
      
  File "/share/pkg/tensorflow/r1.10/install/py3-gpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1717, in __init__
      
    self._traceback = tf_stack.extract_stack()
      

      
NotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
      

      
Key Variable not found in checkpoint
      
     [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]





      python tensorflow deep-learning generative-adversarial-network




      python tensorflow deep-learning generative-adversarial-network






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited 17 hours ago

























      asked 17 hours ago









      Mona Jalal

      7,56826105207




      7,56826105207





























          active

          oldest

          votes











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














           

          draft saved


          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53370770%2ftensorflowkey-variable-not-found-in-checkpoint%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown






























          active

          oldest

          votes













          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















           

          draft saved


          draft discarded



















































           


          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53370770%2ftensorflowkey-variable-not-found-in-checkpoint%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          -

          Popular posts from this blog

          Cypress Hill

          Image
          Dieser Artikel behandelt die Musikgruppe. Zum Musikalbum siehe Cypress Hill (Album). Cypress Hill Sen Dog, Bobo und B-Real Allgemeine Informationen Genre(s) Hip-Hop, Crossover, Reggae Gründung 1988 Website www.cypresshill.com Aktuelle Besetzung Rap B-Real Rap Sen Dog DJ DJ Muggs Percussion Eric Bobo (seit 1994) Cypress Hill ist eine US-amerikanische Hip-Hop-Gruppe aus Los Angeles. Sie verkauften über 18 Millionen Alben weltweit. [1] Die Gruppe besteht aus DJ Muggs , den Rappern B-Real und Sen Dog sowie seit 1994 Percussionist Bobo , der vorher im Umkreis der Beastie Boys tätig gewesen war. Sie wurden international vor allem auch wegen ihres Engagements für die Legalisierung des Cannabiskonsums bekannt. Der Name „Cypress Hill“ (Deutsch: Zypressenhügel) bezieht sich auf den Wohnort der Gründungsmitglieder, da fast alle in der Cypress Avenue in South Gate aufwuchsen. Bis auf DJ Muggs sind alle Mitglieder Hispanics. ...
          Read more

          what are some tips for doing well in the interview? [on hold]

          Image
          .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty{ margin-bottom:0; } up vote -4 down vote favorite I completed my bachelor degree of software engineer now i want to do a government job in Pakistan regarding computer software because i know very well about software and i know how to make software but i am little bit fear full regarding job interview and i am very confused and also frightened when i think about the interview. please help me about my interview and tell me what type of question were asked for the job of software engineers and tell me some tips that i can doing well in the interview. interviewing new-job software degree share | improve this question ...
          Read more

          How does a super-power salesman not get shut down for legal reasons?

          Image
          up vote 6 down vote favorite 2 I am in trouble: my boss, a multi-millionaire, has just come up with a scheme, but it has problems and he wants me to solve them. The whole thing started when he saw this picture: This gave him an idea: invent, develop, and sell "superpowers". These "powers" would not be biological (he's rich, but not that rich), but technological. Basically, he wants to have Small Hero Five-and-a-half (tm) working for him, constantly cranking out new tech. After he has his team invent the tech (with possible help from Anderson Robotics), he would start to mass-produce it (think: bat-mobile factory), and sell the gadgets (why sell the rights, when there is still more money to be made?) to would-be super-heroes. This is where the problems start: the reason no-one has done this yet, is ...
          Read more