Array creation behaves strangely with (very) deeply nested lists #15611
Description
I'm a bit confused by how numpy creates arrays from deeply nested lists. Everything works fine up to 32 dimensions, however it seems that when there are more than 32 nested lists in the passed data a list object is inserted, instead of it being converted to a numpy dimension. Correspondingly, the dtype of the array changes to np.object.
Reproducing code example:
import numpy as np
# This is ok
test_array_32 = np.array([[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[1]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]])
print("Array initialized from 32 nested lists:")
print(test_array_32)
print("Dtype: %s" % test_array_32.dtype)
# This is strange
test_array_33 = np.array([[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[1]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]])
print("Array initialized from 33 nested lists:")
print(test_array_33)
print("Dtype: %s" % test_array_33.dtype)
# This is strange but consistent
test_array_many = np.array([[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[1]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]])
print("Array initialized from many nested lists:")
print(test_array_many)
print("Dtype: %s" % test_array_many.dtype)
# This throws an error
test_array_many_forced = np.array([[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[1]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]], dtype=np.int)
print("Array initialized from many nested lists, forced to dtype=np.int:")
print(test_array_many_forced)
print("Dtype: %s" % test_array_many_forced.dtype)Output of the above program for convenience:
Array initialized from 32 nested lists:
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[1]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]
Dtype: int64
Array initialized from 33 nested lists:
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[list([1])]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]
Dtype: object
Array initialized from many nested lists:
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[list([[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[1]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]])]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]
Dtype: object
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-117-d641b1b0ad21> in <module>
----> 1 import codecs, os;__pyfile = codecs.open('''/tmp/pysgh9na''', encoding='''utf-8''');__code = __pyfile.read().encode('''utf-8''');__pyfile.close();os.remove('''/tmp/pysgh9na''');exec(compile(__code, '''/home/davide/numpyissue.py''', 'exec'));
~/numpyissue.py in <module>
20
21 # This throws an error
---> 22 test_array_many = np.array([[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[1]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]], dtype=np.int)
23 print("Array initialized from many nested lists, forced to dtype=int:")
24 print(test_array_many)
ValueError: setting an array element with a sequence.
Numpy/Python version information:
1.17.4 3.8.1 (default, Jan 22 2020, 06:38:00)
[GCC 9.2.0]