Do yourself a favor and contact the social security administration for a replacement card. It's a lot easier to get it now and keep it somewhere safe then be screwed when you suddenly need it for a second form of ID.
I probably should although I also have Passport, RealID, and a Global Entry card--the latter of which I use for flying now given that it doesn't really matter if I lose it.
