Perhaps I'm being particular here, but that wouldn't be a cocktail - it would be a mixed drink (and is called such in the article) based on the definition of a cocktail at the time:
"a stimulating liquor comprised of spirits of any kind, with sugar, water, and bitters included"
I've never thought about this before, but I think there's a good argument that tea technically falls in the category of bitters.
After all, bitters are just an infusion of plant material. What is tea but an infusion of plant material?
On the other hand, bitters generally include many different plants and seems to be quite concentrated, while tea is just one plant and isn't as concentrated.
I guess asking whether tea is bitters is kind of like asking if hot dogs or tacos are sandwiches...
I assume that bitters must have a base of alcohol, but only because I'm not aware of any bitters that don't use alcohol as the solvent to create the extract.
I once went to a class once on bitters-making and learned that they can be made with water, alcohol, or oil to extract the flavors, and that all three versions will result in different flavors because they will extract different compounds in differening amounts.
(Also that some plants will have poisons that might be extracted in one method but not another, so you always want to research first.)
Obviously oil-based bitters aren't going to be compatible with cocktails because they won't mix in most cases.
But you can definitely make water-based bitters, if you want to avoid the flavors that alcohol extracts. But I think you'd then still add alcohol afterwards simply for preservation purposes, since you want a bottle of bitters to last for months/years and not need to bother with refrigeration.
This is a fascinating article full of history. This is the literal opposite of clickbait. There's no "preamble for engagement", the whole thing is just a really interesting story of an aspect of drinking culture in the US.
And the article makes a good case that tea punch was the first mixed drink, which you could interpret either as predating cocktails or being the first cocktail.
Do you have historical evidence to suggest another mixed drink predated it?