arthur-st
Moreover, there are some methodological oddities. For example, JPEG XL decoding is tested in single-threaded mode when it's optimized for multi-threaded decoding (where AVIF has artifacts from tiling). Also, the tested command is "jxl_cli --speedtest", which means that the benchmarked time includes the warm-up procedure of the "--speedtest" flag, rather than the pure decoding time.
Another example of this is progressive rendering, where the comparison moment is chosen at a local maximum that favors AVIF. JPEG XL materializes a full frame much sooner than AVIF, and starting at, say, the 96,256 B mark, it looks better than AVIF until the full image is loaded.
[1] https://marketplace.visualstudio.com/items?itemName=James-Yu...
[2] https://marketplace.visualstudio.com/items?itemName=ltex-plu...
[3] https://www.texdev.net/2024/11/05/engine-news-from-the-latex...
That is not true. Companies like AWS had paid staff working as OSS Redis core maintainers before the licencing schism. This talk of "achieving their goals" is just bluster serving no reason other than damage control.
To understand shortcomings of MyPy, I strongly suggest reading pyright's documentation for how they compare: https://github.com/microsoft/pyright/blob/main/docs/mypy-com...
Quoting the pertinent part:
> Pyright was designed with performance in mind. It is not unusual for pyright to be 3x to 5x faster than mypy when type checking large code bases. Some of its design decisions were motivated by this goal.
> Pyright was also designed to be used as the foundation for a Python language server. Language servers provide interactive programming features such as completion suggestions, function signature help, type information on hover, semantic-aware search, semantic-aware renaming, semantic token coloring, refactoring tools, etc. For a good user experience, these features require highly responsive type evaluation performance during interactive code modification. They also require type evaluation to work on code that is incomplete and contains syntax errors.
> To achieve these design goals, pyright is implemented as a “lazy” or “just-in-time” type evaluator. Rather than analyzing all code in a module from top to bottom, it is able to evaluate the type of an arbitrary identifier anywhere within a module. If the type of that identifier depends on the types of other expressions or symbols, pyright recursively evaluates those in turn until it has enough information to determine the type of the target identifier. By comparison, mypy uses a more traditional multi-pass architecture where semantic analysis is performed multiple times on a module from the top to the bottom until all types converge.
> Pyright implements its own parser, which recovers gracefully from syntax errors and continues parsing the remainder of the source file. By comparison, mypy uses the parser built in to the Python interpreter, and it does not support recovery after a syntax error. This also means that when you run mypy on an older version of Python, it cannot support newer language features that require grammar changes.
Astral's type checker seems to an exercise in speeding up Pyright's approach to designing a type checker, and removing the Node dependency from it.
[1] https://evokeendurance.com/a-new-and-better-look-at-training...
What's the threat model here, that Rust is a trojan language from the feds?
Installing it via the PyPI gets you the CPU version on Windows/macOS, and CUDA 12.1 on Linux. You can specify CUDA version explicitly, which you need to get CUDA on Windows, or CUDA 11.8 on Linux, by referencing the corresponding PyTorch package index, e.g., https://download.pytorch.org/whl/cu118 - cu118 means CUDA 11.8. Doing that explicitly is the normal practice, since in ML land you'll know your hardware well - no one is asking for tooling that automatically resolves CUDA/cuDNN/etcetra compatibility.
Hence, it hopefully follows that installing this with pip is trivial, you just need to maintain multiple requirements.txt files so that you specify the correct index URL where it's necessary. What I specifically want, then, is for someone to give me a UX where I don't have to maintain these requirements.txt files. I should be able to just specify arbitrary build environments* in pyproject.toml, and override the global default package index for specific packages in [platform-]specific build environments.
*Really, we're probably talking about combining blocks of optional dependencies into an ersatz build environment, but that's what they ultimately are for.
If I can then "compile" these build environments into properly reproducible lock files, and then use the dependency manager to export these lock files into conventional requirements.txt files with exact pins, then we've arrived at what I would call a good UX for specifying dependencies in Python.
To the best of my knowledge, for 99.9% of packages this doesn't require Docker. Different parts of this process work well across the ecosystem, just not as a matter of full package. To name a few
- Poetry has many building blocks, but just has had a questionable implementation as per parent you're replying to, and also random other problems like not working at all with private indexes back when I tried, and so on;
- pip-tools does the entire pipeline well, but it quite unfortunately doesn't support cross-platform targeting;
So, quite honestly, until someone can do this correctly and in one place - and, ideally, with endorsements from PSF and PyPA - I'll just consider Python dependency management tooling to be inadequate for my use cases.
For example, it used to download wheels for all matching distributions (say pytorch for a few platforms) to get metadata. That would be like a 10 GB download for 4 wheels of pytorch, to generate the lockfile.
Another thing it used to do is ignore local cache, so for some locking operations would download the same wheel multiple times.
In general, it was not entertaining, from an ML practitioner's perspective, to infer an attitude from the Poetry maintainers that the mountain shall come to Muhammad, so to say. Their abridged position was that pytorch is not compliant with standards, when Poetry itself, e.g., is not PEP 440-compliant, in a manner pertinent to the issue at hand. [2]
I realise that my grumbling about pytorch has nothing to do with your perfectly adequate comment, that said. Please interpret it as nothing more than an expanded rationale fore being more interested in learning about Rye/uv than in revisiting Poetry, at least for the projects I've worked on.