Estimation often is way off for construction projects, but would you hire a contractor who refused to hazard a guess of how long a job would take or how much it would cost?
Unless the contractor is willing to put in writing a "not to exceed" price, I've never put any faith in build estimates. I'd expect a professional to be able to give a rough estimate based on experience and scope, as well as clear communication as the project progresses. Rough estimate meaning accurate withing something like 20-30%, not a detailed breakdown by ever project and material expense.
I try to do the same in software projects. Estimating projects by hours is impossible, fibonnaci numbers are a meaningless abstraction, and spending time each sprint to discuss how it went, what the next sprint plan is and estimate every task is a waste of time.
What I can say is whether a task looks like something that could be done in an afternoon, a few days, a few weeks, or multiple months. Accuracy there comes with experience, but no one gets better by chopping the calendar year into 2 week intervals and pouring a ton of process all over it.
Would you agree to pay for any service where not even an estimate of the price could be given? I guess healthcare is the only example I can think of where anyone consents to that.
Research. We’re researching cures to cancer, nuclear fusion, space flight, etc. with no idea what the total cost of ownership will be.
Software dev can resemble research on a smaller scale. The more interesting or complex your requirements and environment, the closer it resembles research.
The original point of agile was to lean into this fact and get out of planning and predicting. Scrum culture has corrupted this goal.
Maybe I don't see the relevance of your question then.
Software developers are not negotiating fixed price contracts with their project managers every two weeks. Just how toxic such a situation would be should be obvious. Managers need to understand the context of the job and determine if they're satisfied with the productivity of their employees.
Agile is a development process not a contract pricing strategy. If anything proponents of "Agile" would be more hesitant to quote you a firm fixed price contract than "Waterfall" teams.
They don’t have to hire you every two weeks but they are paying for you and do have to decide how to allocate your time and plan various projects with interlocking dependencies. How can they do that without even a rough idea of how many people it takes for how long to accomplish anything? You are, in effect, still asking them to write you a blank check to accomplish something when they might want to change gears if apprised of how difficult it is.
If you go way back up in this thread the reason given for Scrum being bad is it features estimation, which is useless because it's inaccurate, so telling me that systems other than Scrum feature estimation is not a rebuttal.
Yes, you get paid to do research and there are theorems delivered. Just like software projects, some never get delivered and you don't know when if they're not trivial.
You won't gain that understanding until you actually complete both of those projects.
Industries and occupations that manage estimates well do so because bulk of the work is repeatable and predictable, and the people doing that work have done nearly identical kind of work before. There is some variability involved, but there's only so much of it, and it yields to tried and true classical project management techniques (not the things that pass as "project management" in software companies).
For example, even an inexperienced carpenter, asked to install a door system they haven't seen before, will be able to do it two or three times to get a feel and work the kinks out, and then when they're hired to install it in 300 flats of a new apartment complex, they'll be able to give you an accurate estimate of time and effort required.
With software, almost universally, your project is meaningfully unique, and your workers have never worked on something very similar before. The sources of variability are endless - project-unique challenges ahead, programmers never doing that particular component in that particular combination of languages, libraries and frameworks, in their specific versions, programmers and PMs never working in that specific domain before, etc. - and that's before you add stakeholders changing their requirements every other week.
The understanding one needs to have first is that the software project they're holding a stake in is typically best thought of as R&D effort.
Meh. The time it takes to complete a project is usually somewhat flexible. What really matters is how valuable the project is. That is what needs to be estimated explicitly, and not just "I really like these two ideas lets do the cheaper one".
The variation in effort is orders of magnitude smaller than that in value. Great value is very rarely canceled out by high effort and low value almost always cancel out low effort.
Relative comparison is exactly the scrum method which is why they don't estimate with time. And I don't think revenue estimation is any less subject to variance.