Given a script with only a hardcoded lookup dictionary, I was wondering if compiling it using Cython gave any performance gain.
The dictionary file looks like described below and is around 5Mb. The generated c-file is around 65Mb. Compilation takes ages. Is there a way to reduce the size of the generated c-file and/or speed up the compilation? Alternatively, is there a fundamental reason why this approach would not work?
filename.py:
lookup_dict = { 0: {"Value 1"}, 2: {"Value 2", "Something else"}, 3: {"Some other value"}, # (...) 99996: {"one", "two", "three value"}, 100000: {"Another value"},}def lookup(idx: int) -> set: try: return lookup_dict[idx] except KeyError: return set()The Python setup.py script:
from distutils.core import setupfrom distutils.extension import Extensionfrom Cython.Build import cythonizefrom Cython.Distutils import build_extext_modules = cythonize( [Extension("lookup_module", [r"filename.py"])], language_level="3")setup( name="Lookup Module", cmdclass={"build_ext": build_ext}, ext_modules=ext_modules,)Which is then called using:
python setup.py build_ext --inplace(gcc version 7.4.0, Ubuntu 18.04)