Android does not support CalDAV or CardDAV, which are used to sync calendars and contacts. However, a workaround is installing a CalDAV or CardDAV sync adapter.
We have tested and recommend DAVx⁵, which is approximately $5. Once you have added your account in DAVx⁵, you can set up calendars or contacts in the app of your choice, and the changes will sync with Fastmail.
It'll do bi-directional sync between Android contacts/calendar and Fastmail (or any other CardDav/CalDav server)
Also available on F-Droid: https://f-droid.org/en/packages/at.bitfire.davdroid/