This is NOT a problem, but it does affect how you might write fscanf code to read this text string.Ĭonveniently (and with decent performance) read and write data from gzipfiles directly. Implemented as an FFI wrapper for zlib, including wrappers for C FILEfunctions like fscanf and fwrite. Read 200 floats from a gzipped file directly into a torch tensor: Note: compressing floats can be useful for neural nets, since similar values atsimilar orders of magnitude will often results in repeated byte patterns. To access the underlying FILE* handle, use the handle property of GZFile.Constructor: file = GZFile(filename, mode)Opens the file for reading or writing using the given mode.See zlib gzopen() documentation for supported modes.Doesn't allocate memory, just casts using ffi.Calls fwrite and returns its ret val, i.e.:read(nbytes)Reads to a buffer then turns into a lua string.Allocates memory on each call, so slightly inefficient if you do manyreads.:writebuf(ptr, nbytes)Writes data from the given buffer to the file.Returns number of bytes written.:readbuf(ptr, nbytes)Reads into the given location in memory.Returns number of bytes read.:tell()Returns the position in the file using ftell.
