Or even up or down hills. I could stand to lose quite a few pounds, and it's far cheaper to lose 20 pounds of me than 20 pounds of bicycle, and while I love my good old steel Surly, there's an obvious difference riding my aluminum road bike and a very obvious difference when I've test-ridden modern carbon bikes.
An additional couple of pounds on the bike is only a couple of %, and you're not going to notice it. Wind, tires, positioning, how you're feeling, and road surface matter more. Anything less than a pound and you're talking the difference between a full small waterbottle and an empty one.
Doubling the bike weight, you might notice in the speed, you might not. You might notice it in the handling/feel, you might not. Adding 8kg of panniers to my bike doesn't noticeably slow me down.
Personally, I'm just as fast up hills on a 12kg gravel bike with fat, supple tires (650x42), a rack and fenders as I am on the 8kg carbon 700x25c boneshaker tires.
And I'm faster going down, because I'm not shaken all to hell.