Topics

Unable to open .zip in Docker container


David Johansen
 

I'm trying to load a .gdb.zip file in a Docker container using `fiona.listlayers()` but it says that it's not a supported format with the following exception:

```

File \"fiona/_shim.pyx\", line 83, in fiona._shim.gdal_open_vector

  File \"fiona/_err.pyx\", line 270, in fiona._err.exc_wrap_pointer

fiona._err.CPLE_OpenFailedError: '/tmp/test.gdb.zip' not recognized as a supported file format.

 

During handling of the above exception, another exception occurred:

 

Traceback (most recent call last):

  File \"/usr/src/app/numetric/connectrunner/shp_file/shp_file.py\", line 34, in _listlayers

    for layer in fiona.listlayers(filename):

  File \"/usr/src/app/numetric/connectrunner/venv/lib/python3.8/site-packages/fiona/env.py\", line 407, in wrapper

    return f(*args, **kwargs)

  File \"/usr/src/app/numetric/connectrunner/venv/lib/python3.8/site-packages/fiona/__init__.py\", line 344, in listlayers

    return _listlayers(vsi_path(pobj))

  File \"fiona/ogrext.pyx\", line 1512, in fiona.ogrext._listlayers

 

  File \"fiona/_shim.pyx\", line 90, in fiona._shim.gdal_open_vector
```

The same code works fine on my Mac with the same file, but doesn't in the `3.8.1-slim-buster` and `3.8.1-alpine3.11` images. Any ideas on why and what I can do to fix this?

Thanks,

Dave


hannes-fiona.groups.io@...
 

Hi Dave,

are you using the same versions of Fiona and GDAL?

Cheers, Hannes

On Tue, 18 Feb 2020 16:25:57 -0800
davejohansen@gmail.com wrote:

I'm trying to load a .gdb.zip file in a Docker container using
`fiona.listlayers()` but it says that it's not a supported format
with the following exception:

```

File \"fiona/_shim.pyx\", line 83, in fiona._shim.gdal_open_vector

File \"fiona/_err.pyx\", line 270, in fiona._err.exc_wrap_pointer

fiona._err.CPLE_OpenFailedError: '/tmp/test.gdb.zip' not recognized
as a supported file format.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File \"/usr/src/app/numetric/connectrunner/shp_file/shp_file.py\",
line 34, in _listlayers

for layer in fiona.listlayers(filename):

File
\"/usr/src/app/numetric/connectrunner/venv/lib/python3.8/site-packages/fiona/env.py\",
line 407, in wrapper

return f(*args, **kwargs)

File
\"/usr/src/app/numetric/connectrunner/venv/lib/python3.8/site-packages/fiona/__init__.py\",
line 344, in listlayers

return _listlayers(vsi_path(pobj))

File \"fiona/ogrext.pyx\", line 1512, in fiona.ogrext._listlayers

File \"fiona/_shim.pyx\", line 90, in fiona._shim.gdal_open_vector
```

The same code works fine on my Mac with the same file, but doesn't in
the `3.8.1-slim-buster` and `3.8.1-alpine3.11` images. Any ideas on
why and what I can do to fix this?

Thanks,

Dave



David Johansen
 

I'm using fiona 1.8.13 in both case, and for GDAL, on my Mac, it's 2.4.2, in the debian container it's 2.4.0 and in the Alpine container it's 3.0.3.


On Wed, Feb 19, 2020 at 12:18 AM <hannes-fiona.groups.io@...> wrote:
Hi Dave,

are you using the same versions of Fiona and GDAL?

Cheers, Hannes

On Tue, 18 Feb 2020 16:25:57 -0800
davejohansen@... wrote:

> I'm trying to load a .gdb.zip file in a Docker container using
> `fiona.listlayers()` but it says that it's not a supported format
> with the following exception:
>
> ```
>
> File \"fiona/_shim.pyx\", line 83, in fiona._shim.gdal_open_vector
>
> File \"fiona/_err.pyx\", line 270, in fiona._err.exc_wrap_pointer
>
> fiona._err.CPLE_OpenFailedError: '/tmp/test.gdb.zip' not recognized
> as a supported file format.
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
>
> File \"/usr/src/app/numetric/connectrunner/shp_file/shp_file.py\",
> line 34, in _listlayers
>
> for layer in fiona.listlayers(filename):
>
> File
> \"/usr/src/app/numetric/connectrunner/venv/lib/python3.8/site-packages/fiona/env.py\",
> line 407, in wrapper
>
> return f(*args, **kwargs)
>
> File
> \"/usr/src/app/numetric/connectrunner/venv/lib/python3.8/site-packages/fiona/__init__.py\",
> line 344, in listlayers
>
> return _listlayers(vsi_path(pobj))
>
> File \"fiona/ogrext.pyx\", line 1512, in fiona.ogrext._listlayers
>
> File \"fiona/_shim.pyx\", line 90, in fiona._shim.gdal_open_vector
> ```
>
> The same code works fine on my Mac with the same file, but doesn't in
> the `3.8.1-slim-buster` and `3.8.1-alpine3.11` images. Any ideas on
> why and what I can do to fix this?
>
> Thanks,
>
> Dave
>
>
>




David Johansen
 

On Wed, Feb 19, 2020 at 7:05 AM David Johansen via Groups.Io <davejohansen=gmail.com@groups.io> wrote:
I'm using fiona 1.8.13 in both case, and for GDAL, on my Mac, it's 2.4.2, in the debian container it's 2.4.0 and in the Alpine container it's 3.0.3.

On Wed, Feb 19, 2020 at 12:18 AM <hannes-fiona.groups.io@...> wrote:
Hi Dave,

are you using the same versions of Fiona and GDAL?

Cheers, Hannes

On Tue, 18 Feb 2020 16:25:57 -0800
davejohansen@... wrote:

> I'm trying to load a .gdb.zip file in a Docker container using
> `fiona.listlayers()` but it says that it's not a supported format
> with the following exception:
>
> ```
>
> File \"fiona/_shim.pyx\", line 83, in fiona._shim.gdal_open_vector
>
> File \"fiona/_err.pyx\", line 270, in fiona._err.exc_wrap_pointer
>
> fiona._err.CPLE_OpenFailedError: '/tmp/test.gdb.zip' not recognized
> as a supported file format.
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
>
> File \"/usr/src/app/numetric/connectrunner/shp_file/shp_file.py\",
> line 34, in _listlayers
>
> for layer in fiona.listlayers(filename):
>
> File
> \"/usr/src/app/numetric/connectrunner/venv/lib/python3.8/site-packages/fiona/env.py\",
> line 407, in wrapper
>
> return f(*args, **kwargs)
>
> File
> \"/usr/src/app/numetric/connectrunner/venv/lib/python3.8/site-packages/fiona/__init__.py\",
> line 344, in listlayers
>
> return _listlayers(vsi_path(pobj))
>
> File \"fiona/ogrext.pyx\", line 1512, in fiona.ogrext._listlayers
>
> File \"fiona/_shim.pyx\", line 90, in fiona._shim.gdal_open_vector
> ```
>
> The same code works fine on my Mac with the same file, but doesn't in
> the `3.8.1-slim-buster` and `3.8.1-alpine3.11` images. Any ideas on
> why and what I can do to fix this?
>
> Thanks,
>
> Dave
>
>
>

From playing around with this a bit more, I believe it's from the GDAL not being able to open the .zip file. Is there a way for me to check what compression libraries are available to GDAL? 


René Buffat
 

Under linux, you can view shared libraries linked to a binary using ldd.

E.g. ldd /usr/bin/gdalinfo


David Johansen
 

On Fri, Feb 21, 2020 at 1:56 PM René Buffat <buffat@...> wrote:
Under linux, you can view shared libraries linked to a binary using ldd.

E.g. ldd /usr/bin/gdalinfo

It looks like gdal is using minizip ( https://packages.debian.org/buster/minizip ) in the debian image and I wonder if it doesn't support the format of the file I'm using:
# ldd /usr/lib/libgdal.so | grep -i z
libsz.so.2 => /usr/lib/x86_64-linux-gnu/libsz.so.2 (0x00007fc51c3bb000)
libzstd.so.1 => /usr/lib/x86_64-linux-gnu/libzstd.so.1 (0x00007fc51b8aa000)
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fc51b882000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fc51a963000)
libminizip.so.1 => /usr/lib/x86_64-linux-gnu/libminizip.so.1 (0x00007fc51921c000)

And on Alpine it looks like it's using libz:
# ldd /usr/lib/libgdal.so | grep -i z
libz.so.1 => /lib/libz.so.1 (0x7ff3ca10f000)
liblzma.so.5 => /usr/lib/liblzma.so.5 (0x7ff3c9999000)
libbz2.so.1 => /usr/lib/libbz2.so.1 (0x7ff3c998a000)