This week was spent entirely running experiments and optimizing code. Cumulatively, I improved my original implementation by about 20x by manually doing bit operations. I tried using SIMD and OMP, but they will likely require a much harder refactor before becoming reliable.
I also wasted a few days trying to setup the project on Amarel. Unfortunately, the build system I’m using is a bit new, so I will need to update python before installing it. This opens a whole can of worms with getting a newer version of glibc, binutils, gcc, etc. In any case, I’ve made zero progress on this end, and will likely have to redo my build system, or ship everything through singularity to get it to work. This is probably going to be a requirement, since running tests is time consuming, but doable in parallel.
Next week is the presentation and I have pretty much no content (lol). Hopefully I can make some headway into testing the Neural Data Compression algorithms before then.