I found that implementing a local cache on the runners has been helpful. Ingress/egress on local network is hella slow, especially when each build has ~10-20GB of artifacts to manage.
ZeroFS looks really good. I know a bit about this design space but hadn't run across ZeroFS yet. Do you do testing of the error recovery behavior (connectivity etc)?
This has been mostly manual testing for now.
ZeroFS currently lacks automatic fault injection and proper crash tests, and it’s an area I plan to focus on.
SlateDB, the lower layer, already does DST as well as fault injection though.
Not 100% related but not 100% not-related either: I've got a script that generates variations of the domain names I use the most... All the most common typos/mispelling, all the "1337" variations, all the Levenhstein edit distance of 1, quite some of the 2, etc.
For example for "lillybank.com", I'll generate:
llllybank.com
liliybank.com
...
and countless others.
Hundreds of thousands of entries. They then are null-routed from my unbound DNS resolver.
My browsers are forced into "corporate" settings where they cannot use DoH/DoT: it's all, between my browsers and my unbound resolver, in the clear.
All DNS UDP traffic that contains any Unicode domain name is blocked by the firewall. No DNS over TCP is allowed (and, no, I don't care).
I also block entire countries' TLD as well as entire countries' IP blocks.
Been running a setup like that (and many killfiles, and DNS resolvers known to block all known porn and know malware sites etc.) since years now already. The Internet keeps working fine.
Forgot to say, thanks for posting this, looks quite useful for various projects that have been on my mind. At one point I was looking for a git vfs for Python (I did find one for caddy static serving specifically, but I needed it for Python) but couldn’t find much that wasn’t abandoned—- an s3 vfs might do the trick for a lot of use cases though.
[0] https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/1052
[1] https://github.com/Barre/ZeroFS
reply