Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

At least for data processing pure python can quickly become a problem when handling somewhat large, dozens of GB, files. I've encountered this a few times and tge way to go is usually to either, if viable, port to a different language or only port the hot parts to a relevant library or Cython/Rust with pyo3.



Out of interest, did you find this was a problem that using optimised numpy still didn’t perform good enough? (depending on what you call pure python, yes I agree python loops etc are extremely slow)


I didn't find any problems with numpy based processing in practice. The issue more often is that users choose Python for ease of writing and the std lib and don't consider numpy for these cases. The performance considerarions usually only come up later.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: