Use dataclass slots to decrease memory usage

[ZetaTwo]

This PR adds the use of slots to the dataclasses which greatly reduces memory usage. From my basic test I see a about a 45% reduction in usage. The runtime is potentially slightly worse but those results are far less reliable as I had a lot of other stuff running on the system and there might be caching effects etc.

This change also removes the

super().__init__(content)

call in unitheaders.py. There seem to be no reason for that call to be there. If it needs to be there, it needs to be re-written, see the dataclass docs

Test script:

#!/usr/bin/env python3

import sys
sys.path.insert(1, '../genieutils-py/src')
from genieutils.datfile import DatFile
DatFile.parse('empires2_x2_p1.dat')

Memory Usage

Before changes:

python -m memory_profiler test1.py 
Filename: /home/zetatwo/Projects/genieutils-py-test/../genieutils-py/src/genieutils/datfile.py

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
    45   24.324 MiB   24.324 MiB           1       @classmethod
    46                                             @profile
    47                                             def parse(cls, input_file: Path | PathLike | str) -> 'DatFile':
    48   31.574 MiB    7.250 MiB           1           content = Path(input_file).read_bytes()
    49   92.594 MiB   61.020 MiB           1           data = zlib.decompress(content, wbits=-15)
    50   92.594 MiB    0.000 MiB           1           byte_handler = ByteHandler(memoryview(data))
    51 1287.094 MiB 1194.500 MiB           1           return cls.from_bytes(byte_handler)

After changes:

python -m memory_profiler test1.py
Filename: /home/zetatwo/Projects/genieutils-py-test/../genieutils-py/src/genieutils/datfile.py

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
    45   24.500 MiB   24.500 MiB           1       @classmethod
    46                                             @profile
    47                                             def parse(cls, input_file: Path | PathLike | str) -> 'DatFile':
    48   31.750 MiB    7.250 MiB           1           content = Path(input_file).read_bytes()
    49   92.762 MiB   61.012 MiB           1           data = zlib.decompress(content, wbits=-15)
    50   92.762 MiB    0.000 MiB           1           byte_handler = ByteHandler(memoryview(data))
    51  703.137 MiB  610.375 MiB           1           return cls.from_bytes(byte_handler)

Run Time

Before changes:

time python3 test1.py        
python3 test1.py  12.29s user 9.12s system 98% cpu 21.666 total
time python3 test1.py
python3 test1.py  18.51s user 0.78s system 100% cpu 19.107 total
time python3 test1.py
python3 test1.py  16.24s user 0.62s system 100% cpu 16.703 total

After changes:

time python3 test1.py
python3 test1.py  16.94s user 0.39s system 100% cpu 17.161 total
time python3 test1.py
python3 test1.py  17.13s user 0.44s system 98% cpu 17.806 total
time python3 test1.py
python3 test1.py  21.02s user 0.48s system 98% cpu 21.869 total

[ZetaTwo]

Further context for the dataclass and super issue: python/cpython#124692

[HSZemi]

Thank you! Maybe you could also add it to

genieutils-py/tests/test_common.py

Line 104 in f3f51fc

@dataclasses.dataclass

?

[ZetaTwo]

Sure, will do tonight

[ZetaTwo]

@HSZemi added slots to the test as well

[HSZemi]

Wonderful. Now let's find out if I can create a new release