Lightnews — Scholar-powered news

The PyPy Project @pypyproject.bsky.social · Jun 15

New blog post by @cfbolz.bsky.social: "How fast can the RPython GC allocate?" pypy.org/posts/2025/0...

While working on a paper about allocation profiling in VMProf I got curious about how quickly the RPython GC can allocate an object. I wrote a small RPython benchmark program to get an idea of the ord

pypy.org

4 6

Reposted by The PyPy Project

Stefan Marr @stefan-marr.de · Jun 9

This year's DLS Most Notable Paper award goes to

Compiling for Multi-language Task Migration
by Marc Feeley

The paper is motivated by the challenges of implementing continuations for an uncooperative environment, and worth a read!
Congratulations to the author! www.iro.umontreal.ca/~feeley/pape...

The award with the following citation:

Recent years have seen a resurgent interest in the use and implementation of first-class control, especially in the form of delimited continuations via effect handlers. This paper is motivated by the challenges of implementing continuations for an uncooperative environment and the opportunities for code migration that the implementation affords. The paper recaps the design of the Gambit Virtual Machine for efficient continuations, and it provides evidence for the ongoing effectiveness of the design.

5 11

Reposted by The PyPy Project

CF Bolz-Tereick @cfbolz.bsky.social · Jun 12

"How slow is the tracing interpreter of PyPy's meta-tracing JIT?" cfbolz.de/posts/speed-...

Spoiler: for the microbenchmark I tried, the slowdown of the meta-tracing interpreter is about 900x

How slow is the tracing interpreter of PyPy's meta-tracing JIT?

I wanted to investigate the warmup behavior of the PyPy interpreter, so I wrote a somewhat arbitrary microbenchmark: all_results = set() num = int(sys.argv[1]) class A(object): pass def main():

cfbolz.de

1 4 10

The PyPy Project @pypyproject.bsky.social · May 21

Adding heap operations to Z3 didn't actually find any bugs in the JIT optimizer for heap operations. Maybe that's not entirely surprising, because miscompilations in that area tend to lead to crashes pretty quickly. In any case, it's good to have that infrastructure for the future.

1

The PyPy Project @pypyproject.bsky.social · May 21

We also added limited support for generating random heap operations to our JIT optimizer fuzzer. This extends previous work, which supported only int operations in Z3:

pypy.org/posts/2022/1...

pypy.org

1

The PyPy Project @pypyproject.bsky.social · May 21

Christoph Jung, another student @cfbolz.bsky.social, added a memory model to our Z3 based translation validation. Now our CI checks whether our optimizer unit tests contain invalid optimizations using Z3.

pypy.org

1

The PyPy Project @pypyproject.bsky.social · May 21

Micro-benchmarks for both optimizations look great, but both are hard to see in the performance of bigger programs.

1

The PyPy Project @pypyproject.bsky.social · May 21

An unrelated optimization we also merged was to make store-to-load and load-to-load forwarding work for array reads and writes where the index is a variable. This already worked for constant indexes before.

1 1

The PyPy Project @pypyproject.bsky.social · May 21

We've already had type based alias analysis before, but only taking the RPython level types into account. Unfortunately instances of different Python classes get represented as instances of the same RPython type, making the old JIT logic useless for distinguishing them, which is now fixed.

1

The PyPy Project @pypyproject.bsky.social · May 21

We just merged an improvement to the PyPy JIT heap optimizations: type based alias analysis for Python instances. Work was done by @cfbolz.bsky.social and their student Nico Rittinghaus.

1 2 6

Reposted by The PyPy Project

Ned Batchelder @nedbat.com · Mar 9

nedbatchelder.com/blog/202503/...

After nearly two years, I think this is finally ready: coverage․py
can use sys.monitoring to more efficiently measure branch
coverage.

Faster branch coverage measurement

After nearly two years, I think this is finally ready: coverage.py can use <a rel="external noopener" href="https://docs.python.org/3/library/sys.monitoring.html">sys.monitoring</a> to more efficientl...

nedbatchelder.com

1 5 29

Reposted by The PyPy Project

matti-p.bsky.social @matti-p.bsky.social · Feb 26

Fast on the heels of the last release, I have published a PyPy v7.3.19 bug-fix release with PyPy2.7, PyPY3.10 and a beta-quality PyPy3.11. Please try it out! The complete release note is at doc.pypy.org/en/latest/re...

PyPy v7.3.19: release of python 2.7, 3.10 and 3.11 beta, released 2025-02-26 — PyPy documentation

doc.pypy.org

5 4

Reposted by The PyPy Project

CF Bolz-Tereick @cfbolz.bsky.social · Feb 25

New blog post by my excellent master student Christoph Jung:

"Low Overhead Allocation Sampling with VMProf in PyPy's GC"

pypy.org/posts/2025/0...

Hopefully the first in a series about his work on a sampling allocation profiler for PyPy

3 8

Reposted by The PyPy Project

CF Bolz-Tereick @cfbolz.bsky.social · Feb 13

Since I was now wondering, here's PyPy VS CPython 3.x binary sizes

Plot of CPython and PyPy binary sizes over time. CPython 3.7 is 35 MiB, shrinks to 22.5 MiB in 3.8, then slowly grows again until 3.11 at 25.2 MiB. Cpython 3.12 is much bigger, at 37.3 MiB. CPy 3.13 shrinks a bit again to 36.1 MiB.

PyPy tracks the ups and downs in roughly the same shape but is generally much bigger. 3.7 is at 60.5, 3.8 at 56.2, then it slowly increases to 58.8 at PyPy 3.11

1 2

The PyPy Project @pypyproject.bsky.social · Feb 13

Interestingly enough PyPy 3.x is mostly smaller than 2.7. E.g. PyPy 3.11 is 58.8 MiB, despite having a bunch more features. I never quite figure out why. Just for comparison, CPython 3.11 is 25.2 MiB.

1 2

The PyPy Project @pypyproject.bsky.social · Feb 13

This is the size of PyPy 2.7, btw. It doesn't make sense to use the 3.x variant for this, because the size of that changes due to newer Python versions being implemented over time.

1 1

The PyPy Project @pypyproject.bsky.social · Feb 13

We don't do this regularly or anything, but every couple of years @cfbolz.bsky.social looks at how the PyPy binary sizes have developed. Looks like an ok balance between occasionally cleaning something up and thereby shrinking the binary; and then slow growth or explicit time/binary-size-tradeoffs.

plot showing how the PyPy binary went from 68 MiB in early 2020 to 61 MiB today, with a few ups and downs in between. various points in time are annotated with reasons for growth and shrinkage. the biggest shrinkage is: 2022-05-10 compact jitcode liveness info

1 1 1

The PyPy Project @pypyproject.bsky.social · Feb 9

As a response to these benchmarks, @cfbolz.bsky.social optimized `itertools.islice` for the common case where no step is given, and also for converting large but not very large integers to strings (basically any int that doesn't fit into 64 bits, but is also not really huge, maybe a few words).

The PyPy Project @pypyproject.bsky.social · Feb 9

Tim Peters benchmarking Python 3.13 vs PyPy on an OEIS sequence program he wrote: discuss.python.org/t/python-3-1...

(the results are complicated, on one version Python 3.13 wins, but on others PyPy does).

Python 3.13.0 speed

This is a hearty “congratulations!” message. I often tackle a problem inspired by The On-Line Encyclopedia of Integer Sequences (OEIS). Here, sequence A378048: integers k such that k and k**2 together...

discuss.python.org

2 4

The PyPy Project @pypyproject.bsky.social · Feb 9

Tim Peters benchmarking Python 3.13 vs PyPy on an OEIS sequence program he wrote: discuss.python.org/t/python-3-1...

(the results are complicated, on one version Python 3.13 wins, but on others PyPy does).

Python 3.13.0 speed

This is a hearty “congratulations!” message. I often tackle a problem inspired by The On-Line Encyclopedia of Integer Sequences (OEIS). Here, sequence A378048: integers k such that k and k**2 together...

discuss.python.org

4

The PyPy Project @pypyproject.bsky.social · Feb 7

So far this is a rather low-level mechanism, there's no convenient pdb integration yet. We plan to add something like that later, for now you can use a third-party package such as sdb to get an actual remote debugger going: github.com/ryanpetrello...

GitHub - ryanpetrello/sdb: a socket-based remote debugger for Python

a socket-based remote debugger for Python. Contribute to ryanpetrello/sdb development by creating an account on GitHub.

github.com

1

The PyPy Project @pypyproject.bsky.social · Feb 7

The other new feature an implementation of the draft PEP 768, "Safe external debugger interface" (again, Linux only so far). PEP was written by @pablogsal.com et al. It allows you to remotely attach to a running PyPy process and inject Python code into it.

doc.pypy.org/en/latest/re...

Remote Debugging — PyPy documentation

doc.pypy.org

1 1

The PyPy Project @pypyproject.bsky.social · Feb 7

We have further features planned for VMProf, including a stochastic memory profiler hopefully coming out later this year.

1

The PyPy Project @pypyproject.bsky.social · Feb 7

Here's a screenshot of the Firefox Profiler UI showing VMProf output. The blue functions are native functions, yellow is Python functions and orange jitted functions (no example of that is in the screenshot).

A screenshot of the Firefox profiler UI. It shows a hierarchically expandable view of stack traces together with counts of which stack traces were seen how often during sampling. The top layers of a stack trace are always yellow Python functions. the bottom layers show the native functions that these Python functions call in blue. One of the functions is selected, with extra information shown in a pane on the right.

1 3 2

The PyPy Project @pypyproject.bsky.social · Feb 7

It works best by combining it with vmprof-firefox-profile, which converts vmprof output to a format the Firefox Profiler UI can read: github.com/Cskorpion/vm...
(We're still working on a PyPI release of this tool.)

GitHub - Cskorpion/vmprof-firefox-converter: Firefox-Profiler as UI for VMProf

Firefox-Profiler as UI for VMProf. Contribute to Cskorpion/vmprof-firefox-converter development by creating an account on GitHub.

github.com

1

The PyPy Project @pypyproject.bsky.social · Feb 7

VMProf on PyPy can now sample native functions and symbolify their names (Linux only at the moment). VMProf for CPython has had that feature for a while, we've now ported it to PyPy.

1 1