You do need to prove that you have assigned the colors before the choice of what to reveal is made, I think. Physically this is guaranteed by the presence of the colors under the post-its (barring dynamic e-paper or something).
I'm not sure how it works with the example of the primes, I lost the link to the later pages of the game so I can't read over it again, but I think it's guaranteed because there's just one number encoding all the assignments and you just get to unlock a single pair with the key given in response to your choice. There's an assumption that there isn't enough information in the key to fake any response, it has to reveal something that was already in there.
You're exactly right. The definition via primes ensures there is only one color consistent with each number (formally, this is called a perfectly binding commitment scheme). Also, here's the link if you want to go back: rahulilango.com/coloring/zk
I'm not sure how it works with the example of the primes, I lost the link to the later pages of the game so I can't read over it again, but I think it's guaranteed because there's just one number encoding all the assignments and you just get to unlock a single pair with the key given in response to your choice. There's an assumption that there isn't enough information in the key to fake any response, it has to reveal something that was already in there.