java.io.IOException: Invalid argument when opening file created by h5py

[frankier]

I am having problems reading HDF5 file created by h5py. Here's the file: https://easyupload.io/4u3wc2 (unfortunately this link expires in 30 days -- I can reupload/resend on request).

Say I try to read it in Java (openjdk 14.0.2 2020-07-14 on Debian buster):

    try {
      System.out.println("path/to/file.h5");
      this.hdfFile = NetcdfFiles.open(path.toString());
    } catch (IOException e) {
      throw new UncheckedIOException(e);
    }

Using ucar.nc2.NetcdfFiles obtained using this in my build.gradle: compile group: "edu.ucar", name: "cdm-core", version: "5.3.3"

I get a stack trace like so:

Exception in thread "main" java.io.UncheckedIOException: java.io.IOException: java.io.IOException: Invalid argument
*SNIP*
Caused by: java.io.IOException: java.io.IOException: Invalid argument
    at ucar.nc2.NetcdfFiles.open(NetcdfFiles.java:282)
    at ucar.nc2.NetcdfFiles.open(NetcdfFiles.java:243)
    at ucar.nc2.NetcdfFiles.open(NetcdfFiles.java:228)
    at ucar.nc2.NetcdfFiles.open(NetcdfFiles.java:216)
    at org.vitrivr.cineast.core.util.pose.PreEmbeddingReader.<init>(PreEmbeddingReader.java:17)
    ... 19 more
Caused by: java.io.IOException: Invalid argument
    at java.base/java.io.RandomAccessFile.seek0(Native Method)
    at java.base/java.io.RandomAccessFile.seek(RandomAccessFile.java:593)
    at ucar.unidata.io.RandomAccessFile.read_(RandomAccessFile.java:743)
    at ucar.unidata.io.RandomAccessFile.readBuffer(RandomAccessFile.java:505)
    at ucar.unidata.io.RandomAccessFile.seek(RandomAccessFile.java:493)
    at ucar.nc2.iosp.hdf5.FractalHeap.readIndirectBlock(FractalHeap.java:453)
    at ucar.nc2.iosp.hdf5.FractalHeap.readIndirectBlock(FractalHeap.java:517)
    at ucar.nc2.iosp.hdf5.FractalHeap.<init>(FractalHeap.java:192)
    at ucar.nc2.internal.iosp.hdf5.H5objects.readGroupNew(H5objects.java:2102)
    at ucar.nc2.internal.iosp.hdf5.H5objects.access$700(H5objects.java:26)
    at ucar.nc2.internal.iosp.hdf5.H5objects$H5Group.<init>(H5objects.java:196)
    at ucar.nc2.internal.iosp.hdf5.H5objects$H5Group.<init>(H5objects.java:154)
    at ucar.nc2.internal.iosp.hdf5.H5objects.readRootSymbolTable(H5objects.java:58)
    at ucar.nc2.internal.iosp.hdf5.H5headerNew.readSuperBlock1(H5headerNew.java:350)
    at ucar.nc2.internal.iosp.hdf5.H5headerNew.read(H5headerNew.java:246)
    at ucar.nc2.internal.iosp.hdf5.H5iospNew.build(H5iospNew.java:119)
    at ucar.nc2.NetcdfFiles.build(NetcdfFiles.java:787)
    at ucar.nc2.NetcdfFiles.open(NetcdfFiles.java:726)
    at ucar.nc2.NetcdfFiles.open(NetcdfFiles.java:279)
    ... 23 more

This is odd since ncdump dumps it just fine (netcdf library version 4.7.4 of Apr 21 2020 19:56:21).
If I check in a debugger I see that this happens when FractalHeap.java tries to seek to 4776428346306319609 which is quite far beyond the end of the file(!)

[lesserwhirls]

We've been taking a look at this one, but nothing figured out quite yet. The HDF fractal heap is somewhat complex. NetCDF-Java has implemented a pure java HDF5 reader - that's why you are able to use ncdump from the C library successfully (as it sits on top of the HDF5 C library when dealing with HDF-5 related I/O), whereas we end up with a failure going through the java stack. We are reading that position from the file, but it's pretty clean that's not correct. I suspect we are getting off track when reading deeper into the fractal heap by a byte or two (or something like that) from the location we really should really be reading, and so we are getting a reading a bad value.

[JohnLCaron]

Did you upload this file somewhere?

[lesserwhirls]

You can snag a copy here.

[JohnLCaron]

Could we add a tag "problems related to the Ellen Generes show" ?

[lesserwhirls]

Possibly related to #1546. which is fixed by #1560. Closing for now.