You're still focusing on averages, which is the error inherent in the myth.
While it's true that average life expectancy has increased, the point is that it was absolutely routine for people to live into their 60s, and not uncommon for them to live into their 70s or 80s.
The average still means something. Yes a lot of people were living to 60s, 70s or 80s, but way more people than today were dying at a younger age because of a disease that can now be cured, hygiene not as good, etc.
While it's true that average life expectancy has increased, the point is that it was absolutely routine for people to live into their 60s, and not uncommon for them to live into their 70s or 80s.
See e.g. https://www.sapiens.org/biology/human-lifespan-history/