Cython Compile Tips

Cython will convert your python program to C, then you can compile it for up to 200x speed gains!

Since there is very sporadic [[working]] examples of how to compile cython correctly - this worked.

• Install cython / cythonize.

python3 -m pip install cython

• You may also need to install python-devel

sudo apt install libpython3.11*
sudo apt install parrot-meta-devel-python

Once you have that done, the simplest way is to put all your sub-files into a single large python file for minimal includes outside of the python base.

cythonize yourfile.py

It should produce a very large 'c' file, alternately if it does not work:

cython --embed yourfile.py

It was also noted that one needed to set a virtualenv inside the python3.11 to make sure it completely worked.

cd venv/bin
source activate

Advice around the internet did not work, until I found this for a working compiling option.

gcc -o yourfile yourfile.c -I/usr/include/python3.11 -lpython3.11

It was FAST. Well not quite..

The following code was tested to generate 10,000 primes.

from sys import argv
from time import time

a = 10000

def prime(i, primes):
    for prime in primes:
        if not (i == prime or i % prime):
            return False
    primes.add(i)
    return i

def historic(n):
    primes = set([2])
    i, p = 2, 0
    while True:
        if prime(i, primes):
            p += 1
            if p == n:
                return primes
        i += 1

def naive(n):
    from itertools import count, islice
    primes = (n for n in count(2) if all(n % d for d in range(2, n)))
    return islice(primes, 0, n)

def isPrime(n):
    import re
    # see http://tinyurl.com/3dbhjv
    return re.match(r'^1?$|^(11+?)\1+$', '1' * n) == None

def regexp(n):
    import sys
    N = int(a) # number of primes wanted (from command-line)
    M = 100              # upper-bound of search space
    l = list()           # result list

    while len(l) < N:
        l += filter(isPrime, range(M - 100, M)) # append prime element of [M - 100, M] to l
        M += 100                                # increment upper-bound

    return l[:N] # print result list limited to N elements

def dotime(func, n):
    print (func.__name__)
    start = time()
    a = func(n)
    #print(sorted(list(func(n))))
    print(f'Time in seconds: {func} time: {time() - start:.5f}')


if __name__ == "__main__":
    for func in naive, historic, regexp:
        dotime(func, a)

Source: https://stackoverflow.com/questions/1628949/to-find-first-n-prime-numbers-in-python

It was then compiled, in python it's timing was:

Time in seconds: <function naive at 0x7fdf514dccc0> time: 0.00001
historic
Time in seconds: <function historic at 0x7fdf51472480> time: 1.91470

Once compiled to Cython it's timing was:

naive
Time in seconds: <cyfunction naive at 0x7f06a69865a0> time: 0.00001
historic
Time in seconds: <cyfunction historic at 0x7f06a6986260> time: 3.07382

It still presented a LOT of advantages because it allowed for independent conversion of a .py to a passable executable.

The other option is to just compile .py to .exe...

python -m install pyinstaller
pyinstaller <code.py>

Producing a pile of informaiton:

270 INFO: checking Analysis
271 INFO: Building Analysis because Analysis-00.toc is non existent
271 INFO: Running Analysis Analysis-00.toc
271 INFO: Target bytecode optimization level: 0
271 INFO: Initializing module dependency graph...
272 INFO: Initializing module graph hook caches...
284 INFO: Analyzing modules for base_library.zip ...
scripts/venv/lib/python3.11/site-packages/PyInstaller/hooks/rthooks'
3285 INFO: Creating base_library.zip...
3309 INFO: Looking for dynamic libraries
3451 INFO: Warnings written to /home/c/scripts/build/prime/warn-prime.txt
3457 INFO: Graph cross-reference written to /home/c/scripts/build/prime/xref-prime.html
3462 INFO: checking PYZ
3462 INFO: Building PYZ because PYZ-00.toc is non existent
3462 INFO: Building PYZ (ZlibArchive) /home/c/scripts/build/prime/PYZ-00.pyz
3588 INFO: Building PYZ (ZlibArchive) /home/c/scripts/build/prime/PYZ-00.pyz completed successfully.
3601 INFO: checking PKG
3601 INFO: Building PKG because PKG-00.toc is non existent
3601 INFO: Building PKG (CArchive) prime.pkg
3608 INFO: Building PKG (CArchive) prime.pkg completed successfully.

It should be noted the compile sped up!

naive
Time in seconds: <function naive at 0x7f6da0b84900> time: 0.00001
historic
Time in seconds: <function historic at 0x7f6da0b844a0> time: 2.41750