tensorflow tf.train.batch Data batch read mode of

Time:2020-5-20

When training neural network with a large number of data, it may be necessary to read the data in batches. So I refer to this articlearticleThe result shows that the data has been output in batch loop and will not stop automatically at the end of the data.

And then I found this post saying slice_ input_ Producer () has a parameter num_ Epochs, by setting its value, you can control the output of all data cycles several times.

Then I set the following error message:


tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value input_producer/input_producer/limit_epochs/epochs

     [[Node: input_producer/input_producer/limit_epochs/CountUpTo = CountUpTo[T=DT_INT64, _class=["loc:@input_producer/input_producer/limit_epochs/epochs"], limit=2, _device="/job:localhost/replica:0/task:0/cpu:0"](input_producer/input_producer/limit_epochs/epochs)]]

After a long time, I didn’t know why I was wrong, so I had to go to slice_ input_ The source code of the producer () function. As a result, it is found in the source code that the author said this num_ Epochs is a local variable if it is not empty. You need to call global first_ variables_ Initializer() function initialization.

So after I call it, everything is normal. I hereby record it and hope that other people can find out the reason in time when they encounter it.

Ha ha, this is the first time that I have solved the problem by reading the source code. I’m still a little excited. Ah ah, far away, on the code of ultimate success:

import pandas as pd
import numpy as np
import tensorflow as tf


def generate_data():
  num = 25
  label = np.asarray(range(0, num))
  images = np.random.random([num, 5])
  print('label size :{}, image size {}'.format(label.shape, images.shape))
  return images,label

def get_batch_data():
  label, images = generate_data()
  input_queue = tf.train.slice_input_producer([images, label], shuffle=False,num_epochs=2)
  image_batch, label_batch = tf.train.batch(input_queue, batch_size=5, num_threads=1, capacity=64,allow_smaller_final_batch=False)
  return image_batch,label_batch


images,label = get_batch_data()
sess = tf.Session()
sess.run(tf.global_variables_initializer())
sess.run ( tf.local_ variables_ Initializer()) (that's the line)
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(sess,coord)
try:
  while not coord.should_stop():
    i,l = sess.run([images,label])
    print(i)
    print(l)
except tf.errors.OutOfRangeError:
  print('Done training')
finally:
  coord.request_stop()
coord.join(threads)
sess.close()

The above tensorflow tf.train.batch The data batch reading method is all the content that Xiaobian shared with you. I hope it can give you a reference, and I hope you can support developepaer more.