Reading dataset files

We read the labels file with the ReadLabels function:

void MNISTDataset::ReadLabels(const std::string& labels_file_name) {
std::ifstream labels_file(labels_file_name,
std::ios::binary | std::ios::binary);
labels_file.exceptions(std::ifstream::failbit | std::ifstream::badbit);
if (labels_file) {
uint32_t magic_num = 0;
uint32_t num_items = 0;
if (read_header(&magic_num, labels_file) &&
read_header(&num_items, labels_file)) {
labels_.resize(static_cast<size_t>(num_items));
labels_file.read(reinterpret_cast<char*>(labels_.data()), num_items);
}
}
}

This function opens the file in binary mode and reads the header records, the magic number, and the number of items in the file. It also reads all the items directly to the C++ vector. The most important part is to correctly read the header records. To do this, we can use the read_header function:

    template <class T>
bool read_header(T* out, std::istream& stream) {
auto size = static_cast<std::streamsize>(sizeof(T));
T value;
if (!stream.read(reinterpret_cast<char*>(&value), size)) {
return false;
} else {
// flip endianness
*out = (value << 24) | ((value << 8) & 0x00FF0000) |
((value >> 8) & 0X0000FF00) | (value >> 24);
return true;
}
}

This function reads the value from the input streamin our case, the file stream—and flips the endianness. This function also assumes that header records are 32-bit integer values. In a different scenario, we would have to think of other ways to flip the endianness.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.189.178.237