I have spent the last three days trying to use f2py to interface fortran to python. I am working on windows using both cygwin and mingw. This post is about using cygwin, but I'm concerned about conflicts between the two. Here is the source multxy.f90:
subroutine multxy(x,y,z)
integer, parameter :: flt = selected_real_kind(15)
real(flt), intent(in) :: x,y
real(flt), intent(out) :: z
write(*,'(a,3g12.5)')'multxy'
write(*,'(a,3g12.5)')'multxy',x,y,x*y
z = x*y
end subroutine multxy
If I run f2py like all the examples I’ve seen:
f2py -m multxyC -c multxy.f90
It produces a link error. Amongst the tons of output, I get complaints about strtoflt128
. This is a function in the gnu gcc quad math library. My flt
kind is equivalent to a double in C. There is no quad math in the code, so why is it used? I tried to create an extension file to see if it would tell me anything, but it still tried to link, so no code was produced (or saved).
At this point, I built a small c program using strtofl128
. It compiles and links if I include the quadmath library: gcc main.c -lquadmath
It doesn’t run correctly, but that is something I'll look into later.
I believe the actual quadmath library file is …./cwin/lib/gcc/x86_64-pc-cygwin/8.3.0/libquadmath.a
. The output from running f2py shows this directory, so you would think it would link the library. I have tried running f2py with all sorts of variations of the -l
option. Usually it would complain about no library found. I got one variant to work, but the link still failed.
I’ve also worked with an integer function:
integer function intfunc(n)
integer, intent(in) :: n
intfunc = n*n
end function intfunc
I have been able to create a dll and call it from python without using f2py. It seem to run through f2py just fine. It creates a file intfunc.cp37-win_amd64.pyd
and a directory which contains a .dll with a huge name. I haven’t been able to call it from python yet, but I’ll keep working on that.
Questions:
- Why is f2py using quad precision? Is there an option to avoid using it?
- How do I include the correct library?
- How do I create the C extension code without invoking the linker?
- Can there be a conflict between Cygwin and MinGW? How to avoid?
- Is f2py ready for prime time? Would it be simpler to do a manual interface using the C extensions?
- Can f2py handle fortran functions? Several places have recommended using subroutines?
f2py is touted as easy to use. Perhaps it is, but only after you get it to work properly. I have been interfacing different languages for 25 years, eg. c/c++, fortran, excel/vba, matlab/octave, without nearly this much difficulty. This seems to be a program which does a lot which is hidden under the covers and that makes it difficult to troubleshoot.
One issue is that the output from running f2py is about as easy to read as a core dump. The output when I run it is:
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building extension "multxyC" sources
f2py options: []
f2py:> C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\multxyCmodule.c
creating C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7
Reading fortran codes...
Reading file 'multxy.f90' (format:free)
Post-processing...
Block: multxyC
Block: multxy
Post-processing (stage 2)...
Building modules...
Building module "multxyC"...
Constructing wrapper function "multxy"...
z = multxy(x,y)
Wrote C/API module "multxyC" to file "C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\multxyCmodule.c"
adding 'C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\fortranobject.c' to sources.
adding 'C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7' to include_dirs.
copying c:\program files\python37\lib\site-packages\numpy\f2py\src\fortranobject.c -> C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7
copying c:\program files\python37\lib\site-packages\numpy\f2py\src\fortranobject.h -> C:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7
build_src: building npy-pkg config files
running build_ext
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
customize MSVCCompiler using build_ext
get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
customize GnuFCompiler
Could not locate executable g77
Could not locate executable f77
customize IntelVisualFCompiler
Could not locate executable ifort
Could not locate executable ifl
customize AbsoftFCompiler
Could not locate executable f90
customize CompaqVisualFCompiler
Found executable C:\cwin\bin\DF.exe
customize IntelItaniumVisualFCompiler
Could not locate executable efl
customize Gnu95FCompiler
Found executable C:\cwin\bin\gfortran.exe
Using built-in specs.
COLLECT_GCC=/usr/bin/gfortran
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-cygwin/8.3.0/lto-wrapper.exe
Target: x86_64-pc-cygwin
Configured with: /cygdrive/i/szsz/tmpp/gcc/gcc-8.3.0-1.x86_64/src/gcc-8.3.0/configure --srcdir=/cygdrive/i/szsz/tmpp/gcc/gcc-8.3.0-1.x86_64/src/gcc-8.3.0 --prefix=/usr --exec-prefix=/usr --localstatedir=/var --sysconfdir=/etc --docdir=/usr/share/doc/gcc --htmldir=/usr/share/doc/gcc/html -C --build=x86_64-pc-cygwin --host=x86_64-pc-cygwin --target=x86_64-pc-cygwin --without-libiconv-prefix --without-libintl-prefix --libexecdir=/usr/lib --enable-shared --enable-shared-libgcc --enable-static --enable-version-specific-runtime-libs --enable-bootstrap --enable-__cxa_atexit --with-dwarf2 --with-tune=generic --enable-languages=ada,c,c++,fortran,lto,objc,obj-c++ --enable-graphite --enable-threads=posix --enable-libatomic --enable-libgomp --enable-libitm --enable-libquadmath --enable-libquadmath-support --disable-libssp --enable-libada --disable-symvers --with-gnu-ld --with-gnu-as --with-cloog-include=/usr/include/cloog-isl --without-libiconv-prefix --without-libintl-prefix --with-system-zlib --enable-linker-build-id --with-default-libstdcxx-abi=gcc4-compatible --enable-libstdcxx-filesystem-ts
Thread model: posix
gcc version 8.3.0 (GCC)
customize Gnu95FCompiler
Using built-in specs.
COLLECT_GCC=/usr/bin/gfortran
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-cygwin/8.3.0/lto-wrapper.exe
Target: x86_64-pc-cygwin
Configured with: /cygdrive/i/szsz/tmpp/gcc/gcc-8.3.0-1.x86_64/src/gcc-8.3.0/configure --srcdir=/cygdrive/i/szsz/tmpp/gcc/gcc-8.3.0-1.x86_64/src/gcc-8.3.0 --prefix=/usr --exec-prefix=/usr --localstatedir=/var --sysconfdir=/etc --docdir=/usr/share/doc/gcc --htmldir=/usr/share/doc/gcc/html -C --build=x86_64-pc-cygwin --host=x86_64-pc-cygwin --target=x86_64-pc-cygwin --without-libiconv-prefix --without-libintl-prefix --libexecdir=/usr/lib --enable-shared --enable-shared-libgcc --enable-static --enable-version-specific-runtime-libs --enable-bootstrap --enable-__cxa_atexit --with-dwarf2 --with-tune=generic --enable-languages=ada,c,c++,fortran,lto,objc,obj-c++ --enable-graphite --enable-threads=posix --enable-libatomic --enable-libgomp --enable-libitm --enable-libquadmath --enable-libquadmath-support --disable-libssp --enable-libada --disable-symvers --with-gnu-ld --with-gnu-as --with-cloog-include=/usr/include/cloog-isl --without-libiconv-prefix --without-libintl-prefix --with-system-zlib --enable-linker-build-id --with-default-libstdcxx-abi=gcc4-compatible --enable-libstdcxx-filesystem-ts
Thread model: posix
gcc version 8.3.0 (GCC)
customize Gnu95FCompiler using build_ext
building 'multxyC' extension
compiling C sources
creating C:\cwin\tmp\tmprix0z7i7\Release\cwin
creating C:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp
creating C:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp\tmprix0z7i7
creating C:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -IC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7 -Ic:\program files\python37\lib\site-packages\numpy\core\include -Ic:\program files\python37\include -Ic:\program files\python37\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\include -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt /TcC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\multxyCmodule.c /FoC:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\multxyCmodule.obj
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MT -IC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7 -Ic:\program files\python37\lib\site-packages\numpy\core\include -Ic:\program files\python37\include -Ic:\program files\python37\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.23.28105\include -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt /TcC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\fortranobject.c /FoC:\cwin\tmp\tmprix0z7i7\Release\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7\fortranobject.obj
compiling Fortran sources
Fortran f77 compiler: C:\cwin\bin\gfortran.exe -Wall -g -ffixed-form -fno-second-underscore -O3 -funroll-loops
Fortran f90 compiler: C:\cwin\bin\gfortran.exe -Wall -g -fno-second-underscore -O3 -funroll-loops
Fortran fix compiler: C:\cwin\bin\gfortran.exe -Wall -g -ffixed-form -fno-second-underscore -Wall -g -fno-second-underscore -O3 -funroll-loops
compile options: '-IC:\cwin\tmp\tmprix0z7i7\src.win-amd64-3.7 -Ic:\program files\python37\lib\site-packages\numpy\core\include -Ic:\program files\python37\include -Ic:\program files\python37\include -c'
gfortran.exe:f90: multxy.f90
C:\cwin\bin\gfortran.exe -Wall -g -Wall -g -shared ..\..\..\..\cwin\tmp\tmprix0z7i7\Release\multxy.o -L/usr/lib/gcc/x86_64-pc-cygwin/8.3.0 -Lc:\program files\python37\libs -Lc:\program files\python37\PCbuild\amd64 -o C:\cwin\tmp\tmprix0z7i7\Release\.libs\libmultxy.J7YCD6VUIR3DWPQ3PSLGVWZTCQ2KMJO6.gfortran-win_amd64.dll -Wl,--allow-multiple-definition -Wl,--output-def,C:\cwin\tmp\tmprix0z7i7\Release\libmultxy.J7YCD6VUIR3DWPQ3PSLGVWZTCQ2KMJO6.gfortran-win_amd64.def -Wl,--export-all-symbols -Wl,--enable-auto-import -static -mlong-double-64
/usr/lib/gcc/x86_64-pc-cygwin/8.3.0/../../../../x86_64-pc-cygwin/bin/ld: /usr/lib/gcc/x86_64-pc-cygwin/8.3.0/libgfortran.a(read.o): in function `_gfortrani_convert_real':
/usr/src/debug/gcc-8.3.0-1/libgfortran/io/read.c:173:(.text$_gfortrani_convert_real+0x85): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `strtoflt128'
/usr/lib/gcc/x86_64-pc-cygwin/8.3.0/../../../../x86_64-pc-cygwin/bin/ld: /usr/lib/gcc/x86_64-pc-cygwin/8.3.0/libgfortran.a(read.o): in function `_gfortrani_convert_infnan':
/usr/src/debug/gcc-8.3.0-1/libgfortran/io/read.c:249:(.text$_gfortrani_convert_infnan+0x5c): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `strtoflt128'
collect2: error: ld returned 1 exit status
error: Command "C:\cwin\bin\gfortran.exe -Wall -g -Wall -g -shared ..\..\..\..\cwin\tmp\tmprix0z7i7\Release\multxy.o -L/usr/lib/gcc/x86_64-pc-cygwin/8.3.0 -Lc:\program files\python37\libs -Lc:\program files\python37\PCbuild\amd64 -o C:\cwin\tmp\tmprix0z7i7\Release\.libs\libmultxy.J7YCD6VUIR3DWPQ3PSLGVWZTCQ2KMJO6.gfortran-win_amd64.dll -Wl,--allow-multiple-definition -Wl,--output-def,C:\cwin\tmp\tmprix0z7i7\Release\libmultxy.J7YCD6VUIR3DWPQ3PSLGVWZTCQ2KMJO6.gfortran-win_amd64.def -Wl,--export-all-symbols -Wl,--enable-auto-import -static -mlong-double-64" failed with exit status 1