File Input and Output (IO) with ndarrays

Question

NumPy binary files

Answer

In the Array Creation section, we saw how to create an array from the data included in a file stored locally. In this section, we will focus on how to input and output NumPy arrays as NumPy binary files.

Question

What are the advantages of IO with the .npy format?

Answer

There are several advantages:

Efficiency! Efficiency!! Efficiency!!! IO with NumPy arrays is easy to code and fast to operate
We do not have to care about character encoding aspects
It is possible to wrap multiple arrays up in the same file

Question

What are the IO routines for the .npy format?

Answer

There are four routines, summarized in the table below.

File Input and Output with NumPy Arrays

Routine	Synopsis
`np.load(file[, mmap_mode, allow_pickle, ...])`	Load arrays or pickled objects from `.npy`, `.npz` or pickled files
`np.save(file, arr[, allow_pickle, fix_imports])`	Save an array to a binary file in NumPy `.npy` format
`np.savez(file, args, *kwds)`	Save several arrays into a single file in uncompressed `.npz` format
`np.savez_compressed(file, args, *kwds)`	Save several arrays into a single file in compressed `.npz` format

Note: the statements included in the ‘Routine’ column assume NumPy is loaded with the np alias.

Question

Can you show me some IO examples?

Answer

Of course! The code snippet below deals with the following cases:

Single array case: lines 18 and 20 illustrate how to write and read an .npy file (accepting one and one array only!)
Multiple array case: lines 35 - 60 illustrate how to write and read an .npz file (accepting multiple arrays)

Regarding the ‘multiple array case,’ it is worth noticing that it is possible to write the arrays to a file preserving the variable names assigned to the arrays (e.g., A, B, C, see line 55). Such an option is beneficial when the variable names are meaningful and, perhaps, we expect to load the data back sometime in the future, when we may or may not remember ‘what is what.’

# import numpy with the socially accepted alias 'np'
>>> import numpy as np

# the array
>>> A = np.reshape(np.arange(100), (10, 10))
>>> A
array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
       [20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
       [30, 31, 32, 33, 34, 35, 36, 37, 38, 39],
       [40, 41, 42, 43, 44, 45, 46, 47, 48, 49],
       [50, 51, 52, 53, 54, 55, 56, 57, 58, 59],
       [60, 61, 62, 63, 64, 65, 66, 67, 68, 69],
       [70, 71, 72, 73, 74, 75, 76, 77, 78, 79],
       [80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
       [90, 91, 92, 93, 94, 95, 96, 97, 98, 99]])

# save A, delete it, and load the data back in
>>> np.save('A.npy', A)
>>> del A
>>> A = np.load('A.npy')
>>> A
array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
       [20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
       [30, 31, 32, 33, 34, 35, 36, 37, 38, 39],
       [40, 41, 42, 43, 44, 45, 46, 47, 48, 49],
       [50, 51, 52, 53, 54, 55, 56, 57, 58, 59],
       [60, 61, 62, 63, 64, 65, 66, 67, 68, 69],
       [70, 71, 72, 73, 74, 75, 76, 77, 78, 79],
       [80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
       [90, 91, 92, 93, 94, 95, 96, 97, 98, 99]])

# working w/multiple arrays
# --+ save A and its transpose into a single file in uncompressed format
>>> np.savez('AAT.npz', A, A.T)
# --+ load the arrays back
# ----+ create a NpzFile object
>>> my_arrays = np.load('AAT.npz')
# ----+ check the arrays with the file attribute files
>>> my_arrays.files
['arr_0', 'arr_1']
# ----+ fetch the data on the second item
>>> my_arrays['arr_1']
array([[ 0, 10, 20, 30, 40, 50, 60, 70, 80, 90],
       [ 1, 11, 21, 31, 41, 51, 61, 71, 81, 91],
       [ 2, 12, 22, 32, 42, 52, 62, 72, 82, 92],
       [ 3, 13, 23, 33, 43, 53, 63, 73, 83, 93],
       [ 4, 14, 24, 34, 44, 54, 64, 74, 84, 94],
       [ 5, 15, 25, 35, 45, 55, 65, 75, 85, 95],
       [ 6, 16, 26, 36, 46, 56, 66, 76, 86, 96],
       [ 7, 17, 27, 37, 47, 57, 67, 77, 87, 97],
       [ 8, 18, 28, 38, 48, 58, 68, 78, 88, 98],
       [ 9, 19, 29, 39, 49, 59, 69, 79, 89, 99]])
# --+ it is also possible to save the arrays with their original names
>>> np.savez('AAT.npz', A=A, AT=A.T)
>>> my_arrays = np.load('AAT.npz')
>>> my_arrays.files
['A', 'AT']
>>> AT = my_arrays['AT']
>>> AT
array([[ 0, 10, 20, 30, 40, 50, 60, 70, 80, 90],
       [ 1, 11, 21, 31, 41, 51, 61, 71, 81, 91],
       [ 2, 12, 22, 32, 42, 52, 62, 72, 82, 92],
       [ 3, 13, 23, 33, 43, 53, 63, 73, 83, 93],
       [ 4, 14, 24, 34, 44, 54, 64, 74, 84, 94],
       [ 5, 15, 25, 35, 45, 55, 65, 75, 85, 95],
       [ 6, 16, 26, 36, 46, 56, 66, 76, 86, 96],
       [ 7, 17, 27, 37, 47, 57, 67, 77, 87, 97],
       [ 8, 18, 28, 38, 48, 58, 68, 78, 88, 98],
       [ 9, 19, 29, 39, 49, 59, 69, 79, 89, 99]])