I sort of commissioned the project at Citus, under the auspices of figuring out how much memory copies were costing WAL-E. The answer was: a lot. Our main goal was to touch as many risk points as possible in the design of WAL-E replacement, with emphasis on performance. It was a prototype to that end, and I rather planned to rewrite it, borrowing heavily as I saw fit: the fate of all prototypes. Various priorities got in the way of that.
It was also an opportunity to build competency in management and definition of an intern-sized project. Katie did a very good job, surpassing, by far, the amount of information I thought we could gather during her tenure.
However, unexpectedly, and for a while now, Yandex staff have really worked on the code base a rather lot, bringing it beyond merely practical. They seem to use it under similar conditions WAL-E was designed: for en-masse deployment.
That said, though, I don't think it has the end-user polish that WAL-E had (insomuch as it did) at its peak of maintenance attention. I would consider it acceptable for a programmer to use, but don't expect a sealed project. It might be suitable for those running a large operation and with willingness to get into the implementation.
If your DB is under few TB, you, probably, can go with WAL-E.
We have thousands of clusters with total amount around 1PB of data.
WAL-G is simple to set up, but current CLI experience is far from perfect.
Our goal is to make the most performant PostgreSQL backup system for cloud deployments.
WAL-G is not just fast compression tool: we parallelize serial archive\restore interface and provide very cheap delta-backups. In PostgreSQL, you usually have PITR through WAL. If you have rare backups, your restore time is slow: WAL is applied serially. With WAL-G you can have delta-backups often, they are applied in parallel and much faster than WAL.
This is important for us, because we have a bunch of distributed datacenters and from time to time we need to repair HA clusters from backups as fast as we can.
My name is Andrey, I'm an engineer. I'm contributing to PostgreSQL on behalf of Yandex.Cloud. Also, I'm working on WAL-G. I like jogging and quake[2,3].
Not sure what else defines me...
WAL-G only supports AWS, so a lot of us are still using WAL-E. If anyone is looking for a reasonably simple Go project to contribute to, have a look at adding Azure, OpenStack or GCE support to help bring it up to parity.
