All I can say is I hate when projects just lazily list every pull request for its change log / release notes. It makes it very difficult to discover what breaking changes there are.
That has been my concern as well. The script I wrote tries to bucket entries into categories, including "Backward Incompatible Change" so those are easier to spot. Since it is automated I am trading some accuracy for time saved, which seemed like the only practical choice for me, since I had to backfill a lot of history, but it’s been surprisingly decent so far.
I am also planning to add some PR templates so contributors include the context up front, which should make any release note generation more accurate.
Are you using any tooling to help with changelog curration? I know towncrier is all about fragments, so contributors must had write a brief summary of their contribution, which would be more in-line with your preference.
Not related really, as you can (and should) publish BREAKING.md separately. Release notes should inform more about new stuff than the update process. Usually PRs have details, so release notes can be easily automated. Migrations, on the other hand…
+1 on separating "how to upgrade" due to breaking changes from "what’s new". A dedicated BREAKING.md / MIGRATIONS.md is a really good idea.
One thing I am trying to do is make the generator surface breaking/migration items explicitly, but I still think anything that requires human judgment (migration steps, caveats) should be hand-curated in a dedicated document like you suggested.
