The reasons for this situation are many. And the most critical problem is actually that nobody really believes they're in the situation they think they're in.
Developers think they're being handed a complete list of needs from the customer. But no, they're handed a list of perceived needs. That's quite different: it's the outer peel of what's normally a pretty strong-smelling onion. And if you're not ready, a lot of tears await.
Customers think they're being clear and specific about their needs. But no, they are embedded in a business, and they're quite familiar with that business. Developers generally are not familiar, and so they have no idea why a customer has a need. They only have a declaration of that need. What are the priorities on this need? Is it properly formed, or does the customer want to dictate how the need is fulfilled (and thus avoid an extensive description of ... see next sentence)? Are the customer's needs for user interface, roles, security, auditing, are they implicit or explicit? Does the customer even have time available to describe or review or even discuss in excruciating detail, what they need? Is the customer representative even familiar, firsthand, with the existing process? Has he done the work we're about to help with?
And how can developers actually meet the customer's needs, under the lack of information, lack of experience with customer processes, and lack of collaboration dictated by the customer?
It's not a rhetorical question. It's shared by millions of developers worldwide, anyone who has developed new applications in a ground-zero environment.
And what it implies is a more serious look at a real need in application development. Programming needs to be flexible & configurable. I use these terms because all the precise terms have been commandeered. It's not a great label. It's not flashy & attractive. And it's too vague. Microsoft is likely to chuck something deceptive into the word "flexible", and give customers the bull of goods it wants to sell 'em. Be wary of labels. "Agile", "collaboration", "groupware", "generation", "DevOps", "object-oriented", "metaprogramming", "structured": they've all been commandeered.
So what would be "flexible" programming? Well it always involves listening to a customer. And it involves listening for lack -- listening for places where a customer rep is not sure. Where a customer rep waffles. Where a customer rep isn't willing to answer, or switches answers, or is overriding subordinates, who may well be right and he, wrong. It involves writing that stuff down and documenting the overridden, changed, or vague, or non, answers. And it involves programming so that those answers are easily implemented.
Is a customer rep prepared to pay for all that stuff? Maybe not. They're paying for plenty. But every single project I've been on where this is not done, has cost more and caused more pain to the customer than was needed. Our managers talk a lot about "delighting" our customers, of "serving what they need, not only what they say". But as a developer, I've found that to get a customer what they need, developers have to provide abilities the customer may not be aware of.
The people who change out your garage doors: they come prepared with lots of materials that you, a homeowner, weren't aware would be needed. The plumbers, they come with valves and pipe stock that will fit your drains & supply lines. They come with expertise to avoid flooding your home.
Application developers could do well to be prepared in this way. We've experienced where some of this customer pain comes from. And when it's our applications at fault, maybe we've hit on a few ways to ease that pain. It may cost us minimal work upfront. Or it may even simplify our own application development. But when it's minimal work, we should see the opportunity and fill it.
More on just how to do that in Notes -- to come, soon.