Date   

Re: Attempting to find version of GDAL installed

Orry Messer
 

Yeah, I believe this is what has happened. On the CI/CD server, a whl was used, and locally it's using the source .tar.gz.

Thanks again.


Re: Attempting to find version of GDAL installed

Sean Gillies
 

We provide wheel distributions for only a small group of platform tags and Python versions. See https://pypi.org/project/Fiona/1.8.18/#files. I suspect your CI/CD server's pip program finds a compatible wheel, but your dev machine cannot. In that case, pip on your dev machine will take the source distribution, the .tar.gz file.


On Mon, Mar 8, 2021, 12:44 AM Orry Messer <orry@...> wrote:
Thanks Sean!

Well, it appears that the libgdal shared library which fiona is linking resides in /usr/local/lib/python3.6/site-packages/fiona/.libs, rather than /usr/lib or /usr/local/lib - which leads me to believe that fiona has obtained its GDAL dependencies from PyPI. 

What I am struggling with, is that locally, on my dev machine, fiona does not seem to be getting GDAL dependencies from PyPI, where as on my CI/CD server, it is getting its dependencies and so the result of "pip install fiona" works. Do you have any idea why this may be the case?

- Orry


Re: Attempting to find version of GDAL installed

René Buffat
 

Not sure if related, but there seems to be no macos wheel for python 3.9 on pypi: https://pypi.org/project/Fiona/#files


Re: Attempting to find version of GDAL installed

Orry Messer
 

Thanks Sean!

Well, it appears that the libgdal shared library which fiona is linking resides in /usr/local/lib/python3.6/site-packages/fiona/.libs, rather than /usr/lib or /usr/local/lib - which leads me to believe that fiona has obtained its GDAL dependencies from PyPI. 

What I am struggling with, is that locally, on my dev machine, fiona does not seem to be getting GDAL dependencies from PyPI, where as on my CI/CD server, it is getting its dependencies and so the result of "pip install fiona" works. Do you have any idea why this may be the case?

- Orry


Re: Attempting to find version of GDAL installed

Sean Gillies
 

Hi Orry,

No need to apologise for asking, the fiona and GDAL software stack is complicated. 

The fiona packages on PyPI include their own copies of GDAL and its dependencies, so you may be detecting that one. One way to know for sure is to use (on Linux) the LD_DEBUG environment variable.

LD_DEBUG=all python -c "import fiona"

will dump a wall of text into your terminal and if you scan it for libgdal, you'll see the path of the shared library that fiona links.


On Thu, Mar 4, 2021, 7:27 AM Orry Messer <orry@...> wrote:

Good day,

 

I am attempting to debug a problem I have while installing Fiona within a Docker container. Apologies if my understanding is a bit shaky – I’m very new to this.

Essentially, I am trying to understand the relationship between GDAL and Fiona. It stems from a problem to do with specifying GDAL API version in a fresh install (either by using gdal-config or GDAL_CONFIG environment variable).

I currently have a working container, with Fiona installed. However, I do not understand how it got to this state.

When I import Fiona in python, 

>>> import fiona

And then run 

>>> fiona.gdal_version

I get: GDALVersion(major=2, minor=3, revision=2).

To me, it seems that GDAL is installed within this container. However, I cannot find it anywhere in this container. Is there a way to ascertain the path Fiona is pointing to?

 

Thanks,

Orry

 


Attempting to find version of GDAL installed

Orry Messer
 

Good day,

 

I am attempting to debug a problem I have while installing Fiona within a Docker container. Apologies if my understanding is a bit shaky – I’m very new to this.

Essentially, I am trying to understand the relationship between GDAL and Fiona. It stems from a problem to do with specifying GDAL API version in a fresh install (either by using gdal-config or GDAL_CONFIG environment variable).

I currently have a working container, with Fiona installed. However, I do not understand how it got to this state.

When I import Fiona in python, 

>>> import fiona

And then run 

>>> fiona.gdal_version

I get: GDALVersion(major=2, minor=3, revision=2).

To me, it seems that GDAL is installed within this container. However, I cannot find it anywhere in this container. Is there a way to ascertain the path Fiona is pointing to?

 

Thanks,

Orry

 


Re: Cannot install Fiona on Mac OS, GDAL installed via homebrew

amit@...
 

Hi Sean, 

Thanks for your reply, and the relevant link - my issue came down to simply not having a properly updated and configured SDK version, once I took care of that, it works with out issues.

If anyone else comes across the same issue, here is what I did to solve it:

1. Download and install the lates version of [XCode](https://developer.apple.com/xcode/)
2. Browse to "/Library/Developer/CommandLineTools/SDKs"
3. Ensure that the alias "MacOSX.sdk" points to the proper SDK version (mine has been pointing to "MacOSX11.1.sdk" instead of the appropriate "MacOSX10.15.sdk" - I am not sure when how or why happened exactly)
4. Install fiona


Re: Cannot install Fiona on Mac OS, GDAL installed via homebrew

Sean Gillies
 

Hi Amit,

I haven't seen this error before, but I also stopped developing fiona on macOS two years ago. High Sierra was what I was using. https://stackoverflow.com/a/61526989/159235 looks like it could be relevant.




On Mon, Feb 22, 2021 at 9:33 AM <amit@...> wrote:
Hey,

This is my first interaction with Fiona (and geo-related platforms in general), I ended up here after failing to install `geoplot` and `geopandas` using a package manager (Poetry), and finally pinned it down to issues installing Fiona.

I cannot seem to install Fiona. This is what I am currently doing:

1. Install GDAL through hombrew (`brew install gdal`)
2. Create a new venv (named `fiona_test`), using Python 3.9.0 (managed by pyenv).
3. Activate the venv.
4. run `gdal-config --version` seems to be fine -> shows `3.2.1`
5. run `pip install fiona` -> results in error.

Running macOS Catalina Version 10.15.17.

Full error on terminal is attached.

Any help would be greatly appreciated, thanks in advance!

Amit



--
Sean Gillies


Cannot install Fiona on Mac OS, GDAL installed via homebrew

amit@...
 

Hey,

This is my first interaction with Fiona (and geo-related platforms in general), I ended up here after failing to install `geoplot` and `geopandas` using a package manager (Poetry), and finally pinned it down to issues installing Fiona.

I cannot seem to install Fiona. This is what I am currently doing:

1. Install GDAL through hombrew (`brew install gdal`)
2. Create a new venv (named `fiona_test`), using Python 3.9.0 (managed by pyenv).
3. Activate the venv.
4. run `gdal-config --version` seems to be fine -> shows `3.2.1`
5. run `pip install fiona` -> results in error.

Running macOS Catalina Version 10.15.17.

Full error on terminal is attached.

Any help would be greatly appreciated, thanks in advance!

Amit


Re: `fiona.transform.transform_geom(...)` missing gcs.csv

René Buffat
 

You are using Fiona pip wheels with conda which is not supported. If you use conda you should use the fiona-feedstock package: https://github.com/conda-forge/fiona-feedstock

lg rene


`fiona.transform.transform_geom(...)` missing gcs.csv

calebrob6@...
 

Hi all,
 
Some methods of installing fiona in a conda environment that also has GDAL result in `fiona.transform.transform_geom(src_crs, dst_crs, geom)` throwing an error, e.g.:
```
ERROR 4: Unable to open EPSG support file gcs.csv.  Try setting the GDAL_DATA environment variable to point to the directory containing EPSG csv files.
ERROR 6: No translation for an empty SRS to PROJ.4 format is known.
```
When this happens, `geom` is returned unmodified and no Python exceptions are raised (this may be a separate problem -- it seems like this should be caught/raised).
 
I think this is related to two things:
1. Installing GDAL>=2.5 through conda will set GDAL_DATA to a directory that does not contain "gcs.csv", see https://github.com/OSGeo/gdal/issues/1471.
2. If GDAL_DATA is not set when fiona (or rasterio) are imported, then a directory packaged with the library will be used instead (which will contain "gcs.csv"). See https://github.com/Toblerity/Fiona/issues/731.
 
If you install GDAL through conda, then GDAL_DATA will be set, but will not contain "gcs.csv". This is not the entire picture though, as it does not explain why Case 2 below passes.
 
 
# Reproducing
 
The following is a test script and three methods for installing fiona (all of which I expect to result in the test script passing).
 
```test_script.py
import fiona.transform
geom = {'type': 'Polygon', 'coordinates': [[(440115.0, 4317302.0), (446180.0, 4317302.0), (446180.0, 4309726.0), (440115.0, 4309726.0), (440115.0, 4317302.0)]]}
geom = fiona.transform.transform_geom("EPSG:26918", "EPSG:4326", geom)
# The expected result is approximately `{'type': 'Polygon', 'coordinates': [[(-75.69160116601654, 39.00268487761508), (-75.62156162389587, 39.00307904086472), (-75.62096527634455, 38.9348126572227), (-75.69093763250096, 38.93441944712403), (-75.69160116601654, 39.00268487761508)]]}`
```
 
 
Installation **Case 1**
```
conda create -n testfiona1 "gdal=3.1.4" "python<3.9" -c conda-forge
conda activate testfiona1
pip install fiona
python test_script.py
conda deactivate
```
This doesn't work, installing gdal through conda sets GDAL_DATA to something that doesn't have the files that fiona needs.
 
 
Installation **Case 2**
```
conda create -n testfiona2 "gdal=3.1.4" "python<3.9" fiona -c conda-forge
conda activate testfiona2
python test_script.py
conda deactivate
```
This does work, but I'm not sure why.
 
 
Installation **Case 3**
```
conda create -n testfiona3 "python<3.9" -c conda-forge
conda activate testfiona3
pip install fiona
python test_script.py
conda deactivate
```
This also works, GDAL_DATA is never set.
 
 
Can anyone shed some light on what is happening here?
 
Best,
Caleb


Re: Can Fiona recognize the AWS_PROFILE variable?

René Buffat
 

I never used Fiona with AWS, but I suppose this is mainly dependent if GDAL supports it. GDALs issue tracker seems to contain a relevant entry:  https://github.com/OSGeo/gdal/issues/2470
This issue was fixed with GDAL 3.2.0: https://github.com/OSGeo/gdal/blob/v3.2.0/gdal/NEWS

How to set credentials in Fiona is described here:  https://fiona.readthedocs.io/en/latest/manual.html#cloud-storage-credentials


Can Fiona recognize the AWS_PROFILE variable?

Tim Hopper
 

The AWS CLI, Pandas, and other tools recognize the AWS_PROFILE variable to indicate where to find credentials for accessing AWS resources (e.g. S3).

My testing of Geopandas indicates that Fiona doesn’t recognize this environmental variable. Is this correct?

Is there any alternative way to get Geopandas/Fiona to discover specific AWS from a file? My code, as of now, works with an attached IAM role on an EC2 instance, but I can’t run it locally with credentials.


NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. Any e-mail reply to this address may be subject to interception or monitoring for operational reasons or for lawful business practices


Re: installing Fiona and conflicting instillation information for Windows

René Buffat
 

At this point, I would recommend testing the Gohlke wheels on a clean Windows installation (following the instructions of Gohlkes homepage by the line and e.g. not modifying any environment variable if not required by the instruction etc.) and check if the issues persist on this system.

But I fear I can't contribute more to this discussion. I currently don't have access to a Windows computer and as I said, the last time I used these wheels is a long time ago (But then they worked quite well). 


Re: installing Fiona and conflicting instillation information for Windows

Eyal Saiet
 

cleaned some old paths and reinstalled gdal wheel, rasterio wheel, and fiona wheel.
Trying different path, and other variables in the Environment variable

Now rasterio is back working but fiona still not:
First I tried:

GDAL_DRIVER_PATH value= C:\Users\esaiet\AppData\Local\Programs\Python\Python37\Lib\site-packages\osgeo\gdalplugins

Import rasterio #worked!

Import Fiona    #failed

Created this error:

Can't load requested DLL: C:\Users\esaiet\AppData\Local\Programs\Python\Python37\Lib\site-packages\osgeo\gdalplugins\ogr_FileGDB.dll

126: The specified module could not be found.

 

Can't load requested DLL: C:\Users\esaiet\AppData\Local\Programs\Python\Python37\Lib\site-packages\osgeo\gdalplugins\ogr_FileGDB.dll

126: The specified module could not be found.

 

Can't load requested DLL: C:\Users\esaiet\AppData\Local\Programs\Python\Python37\Lib\site-packages\osgeo\gdalplugins\ogr_FileGDB.dll

126: The specified module could not be found.

 

Can't load requested DLL: C:\Users\esaiet\AppData\Local\Programs\Python\Python37\Lib\site-packages\osgeo\gdalplugins\ogr_FileGDB.dll

126: The specified module could not be found.


Then tried something different  to grab the gdal301.dll file

GDAL_DRIVER_PATH from C:\Users\esaiet\AppData\Local\Programs\Python\Python37\Lib\site-packages\osgeo

Import rasterio #worked!

Import Fiona # failed:

Got the error

Missing global # gdal: DRIVER_NAME declaration in C:\Users\esaiet\AppData\Local\Programs\Python\Python37\Lib\site-packages\osgeo\gdal_array.py

Missing global # gdal: DRIVER_NAME declaration in C:\Users\esaiet\AppData\Local\Programs\Python\Python37\Lib\site-packages\osgeo\gdal_array.py


Maybe someone can give insight into the "Missing global.. Driver_NAME"  and gdal_array.py?


I am missing how fiona and rasterio  interact   differently with GDAL.




Re: installing Fiona and conflicting instillation information for Windows

Eyal Saiet
 


Rene, I agree 100%, but I thought if  my environment variables were correct that should not have mattered.
Anyhow, I uninstall GDAL from my programs and now rasterio as well cannot find the .dll files. Clearly the environment variable path=  C:\Users\esaiet\AppData\Local\Programs\Python\Python37\Lib\site-packages\osgeo\gdal301.dll  is not working, And, clearly rasterio is not directed to the  osgeo

So I need to do more homework on what I am missing here...



Re: installing Fiona and conflicting instillation information for Windows

René Buffat
 

C:/programfiles/gdal does not look like a path from a gohlke gdal wheel. 

https://www.lfd.uci.edu/~gohlke/pythonlibs/#gdal says: "This distribution includes a complete GDAL installation. Do not use together with OSGeo4W or gdalwin32." This could be an issue.


Re: installing Fiona and conflicting instillation information for Windows

Eyal Saiet
 

I uninstalled and reinstalled  osgeo/gdal , rasterio, and fiona. This time I did not get any installation error. But get a dll error when trying to import fiona.
Previously I installed all the required dependencies.
As well added to User Enviroment Variables
Path= C:\Users\esaiet\AppData\Local\Programs\Python\Python37\Lib\site-packages\osgeo\gdal301.dll
Path= C:\Users\esaiet\AppData\Local\Programs\Python\Python37\Lib\site-packages\osgeo\data

image.png
How is rasterio working and fiona not?

Thanks for the conda suggestion, I have been successful in the past not using conda, which enables me to create small size tools.



On Wed, Nov 25, 2020 at 3:13 PM Simon Norris <snorris@...> wrote:
I have used conda on windows - it makes gdal/fiona/rasterio/geopandas installation much easier:

conda create -n myenv
conda activate myenv
conda config --env --add channels conda-forge
conda config --env --set channel_priority strict
conda install gdal
conda install fiona
conda install rasterio

https://geopandas.org/install.html

On Nov 25, 2020, at 12:12 PM, René Buffat <buffat@...> wrote:

I have to say it is a long time ago since I used these wheels on Windows. From what I remember there was no need to set any paths. However, it was important to install all the necessary dependencies (MS C++ redistributable, pip, numpy, gdal etc.). But this could have changed. 

An alternative would be to use conda. The benefit of using conda is that you can install packages over an index and you don't have to install them manually. Fiona is available in conda-forge: https://github.com/conda-forge/fiona-feedstock However, I never used conda on Windows, thus I can't say too much about it. 



--


Eyal Saiet


Re: installing Fiona and conflicting instillation information for Windows

Simon Norris
 

I have used conda on windows - it makes gdal/fiona/rasterio/geopandas installation much easier:

conda create -n myenv
conda activate myenv
conda config --env --add channels conda-forge
conda config --env --set channel_priority strict
conda install gdal
conda install fiona
conda install rasterio

On Nov 25, 2020, at 12:12 PM, René Buffat <buffat@...> wrote:

I have to say it is a long time ago since I used these wheels on Windows. From what I remember there was no need to set any paths. However, it was important to install all the necessary dependencies (MS C++ redistributable, pip, numpy, gdal etc.). But this could have changed. 

An alternative would be to use conda. The benefit of using conda is that you can install packages over an index and you don't have to install them manually. Fiona is available in conda-forge: https://github.com/conda-forge/fiona-feedstock However, I never used conda on Windows, thus I can't say too much about it. 


Re: installing Fiona and conflicting instillation information for Windows

René Buffat
 

I have to say it is a long time ago since I used these wheels on Windows. From what I remember there was no need to set any paths. However, it was important to install all the necessary dependencies (MS C++ redistributable, pip, numpy, gdal etc.). But this could have changed. 

An alternative would be to use conda. The benefit of using conda is that you can install packages over an index and you don't have to install them manually. Fiona is available in conda-forge: https://github.com/conda-forge/fiona-feedstock However, I never used conda on Windows, thus I can't say too much about it. 

21 - 40 of 115