In a control theory context, we're likely talking about inferring a small handful of parameters where the relationships between them are well known. In this paper they're inferring the entire dynamics of an environment from thousands of raw pixel values. This is not something that admits a tractable exact optimization

That may be fair. I only read the first paper with the spaceships and mazes, which are much more traditional problems.

It sounds like the second paper is the more interesting one from your description though, so I will give that a read.