Quantcast
Channel: Active questions tagged gcc - Stack Overflow
Viewing all articles
Browse latest Browse all 22113

cython: Cythonizing a large dict lookup

$
0
0

Given a script with only a hardcoded lookup dictionary, I was wondering if compiling it using Cython gave any performance gain.

The dictionary file looks like described below and is around 5Mb. The generated c-file is around 65Mb. Compilation takes ages. Is there a way to reduce the size of the generated c-file and/or speed up the compilation? Alternatively, is there a fundamental reason why this approach would not work?

filename.py:

lookup_dict = {     0: {"Value 1"},     2: {"Value 2", "Something else"},     3: {"Some other value"},     # (...)     99996: {"one", "two", "three value"},     100000: {"Another value"},}def lookup(idx: int) -> set:    try:        return lookup_dict[idx]    except KeyError:        return set()

The Python setup.py script:

from distutils.core import setupfrom distutils.extension import Extensionfrom Cython.Build import cythonizefrom Cython.Distutils import build_extext_modules = cythonize(    [Extension("lookup_module", [r"filename.py"])], language_level="3")setup(    name="Lookup Module",    cmdclass={"build_ext": build_ext},    ext_modules=ext_modules,)

Which is then called using:

python setup.py build_ext --inplace

(gcc version 7.4.0, Ubuntu 18.04)


Viewing all articles
Browse latest Browse all 22113

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>