Hi!

I hereby want to announce a small C-Extension we created for an
internal research project. It might be useful to scientists who use 1- dimensional fourier transforms frequently.
It wrapps the "Fastest Fourier Transformation in the West", or at least
a very small subset of it (currently only the complex 1D transform,
both serial and with MPI, and wisdom handling).

You can find the source code, licensed under GPLv3, here: https://git.sr.ht/~imperator/mini-mpi-fftw

I have a sourcehut mailing list for discussions, bug reports etc. ~imperator/mini-mpi-fftw-discuss@lists.sr.ht https://lists.sr.ht/~imperator/mini-mpi-fftw-discuss

...and might in the future announce new versions on this mailing list: https://lists.sr.ht/~imperator/mini-mpi-fftw-announce


Minifftw's main design goal is to make the FFTW on Linux clusters with
MPI very easy to use for people without much experience on MPI.
You can build two versions of the module
* one without MPI, wrapping the serial FFTW for usage i.e. on desktop
computers
* one with MPI, wrapping the MPI-FFTW for distributed computing.

Both versions behave exactly the same from the perspective of the
Python program. Therefore, the programmer can run the same simulation
program without changing the code on his laptop (within hardware
resource limits, of course) or on a cluster.
Another advantage is that it directly writes to and reads from the
memory of passed numpy arrays, without copying memory when entering or
leaving the API.
MFFTW aims at mostly offering the FFTW API to the Python world, so you
have the same functions (plan, execute) available and don't really need
to relearn anything when you already know FFTW.

MFFTW manually uses MPI to distribute and collect parts of a whole
contiguous array behind the scenes. That's necessary because FFTW only
supports pure distributed memory, where each process allocates and
fills its memory slice locally.

Therefore, MFFTW is intended for comparatively small transforms
(currently maximum size 2^31-1 complex numbers), which you want to
speed up using distributed computing. Our tests showed that, with the
right configuration, you can gain significant performance, even with
arrays much smaller to what distributed memory FFTW seems to be
intended for.

The disadvantage of this easy-to-use approach is that the control over
MPI is taken away from the user, therefore I don't think it will be
possible to use MFFTW and other MPI-requiring functions in the same
simulation.
Also, currently, the wrapper is very small according the implemented functionality. We maybe will implement more transforms and would accept
patches implementing those, but our own simulations only need the DFT-
1D.

The README should answer most questions about the internals, I hope.

Next steps might include packaging for Debian and Arch-Linux. I am
unsure, though, how much sense this makes, because you'd either have to
provide two packages (w/ & ^w/ MPI), or needed a runtime-flag to use
serial FFTW in a package build for MPI.
Also, I expect the wrapper to be most useful on clusters, where you
will likely have a custom build process without standard package
managers.

I hope it will be helpful to some of you.
Greetings,
P.

<P><A HREF="https://git.sr.ht/~imperator/mini-mpi-fftw
">MiniFFTW 1.0.4</A> -short
C-Extension for easy FFTW-MPI usage (11-April-21)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)