I’ll be honest, though, it was the Peep Show reference in the article image that got me reading!

3. Use tiered asynchronous checkpointing (DRAM -> node local storage -> shared storage) to avoid blocking GPUs

2. A fast parallel distributed file system is not needed for training, adds unnecessary complexity to the system

Interesting points I picked up

1. To keep GPUs busy in LLM training use node local SSDs to store training

I used this to do an editable pip install of a src-layout Python package I was working on in the dev container, with all the development dependencies. Now when the dev container is launched, the Python package is already installed and ready to develop/test/debug! Very handy.

A really nice feature is that dev containers make it very easy to do post-build customisation of a base image/Containerfile involving the source code being developed. You specify commands to run in the built container with your source code workspace (from the host) mounted into the container.

Quite impressed. After reading the VSCode dev containers docs (code.visualstudio.com/docs/devcont...), it didn’t take long for me to get a containerised dev environment set up…

An unordered list of interesting things I learned:

* How to write a ReFrame test
* What a roofline model is
* There are many novel/exotic hardware testbeds dotted around the UK (thanks to the ExCALIBUR H&ES programme)
* People really like to see and hold the GH200 and Grace CPU superchips