Tuesday, October 09, 2007

The need for a new requirements approach - and BPM's role

Ok, it has happened again. Fortunately, it doesn't happen often, but from time to time, I find myself in the unlucky projectworld of "Type 2" , described by the Standish Chaos research as "Over time, Over budget and Less Functionality". Ouch.

Remember the Standish Chaos Report? Also see for instance http://www.projectsmart.co.uk/docs/chaos-report.pdf

It lists succesrates for projects, based on research. And it lists succes and failure factors.
It's findings (So, I am part of the majority with my project :-):
- 16.2% of all projects are Resolution Type 1, or project success: The project is completed on-time and on-budget, with all features and functions as initially specified.
- 52.7% - Resolution Type 2, or project challenged: The project is completed and operational but
over-budget, over the time estimate, and offers fewer features and functions than
originally specified.
- 31.1% - Resolution Type 3, or project impaired: The project is cancelled at some point during the development cycle.

The failure factors for projects ending up as a 2 or 3:

The survey participants were also asked about the factors that cause projects to be challenged.
1. Lack of User Input 12.8%
2. Incomplete Requirements & Specifications 12.3%
3. Changing Requirements & Specifications 11.8%

And yes - in this project, for all kinds of reaons, these items are true.

It wasn't me, though, I was not working on the requirement side ;-). But a repeating lesson I learned: good requirements are essential. And in my opinion that needs process-context awareness.

Remember the old days of applicatie analysis and design? I call it feature oriented requirements engineering: talk to users, find out what's difficult in their job, and think up a set of features to cover their issues.
Example: we do sales, but miss information on our clients and status of orders.
Ah! We prototype a nice screen, build you a nice database with clients, and the order status we get through EAI from your order entry system. Problem solved!
Really? No.
You end up with a so-called data & isolated task oriented application. It's not aware of the process.
To be honest, as IT requirements person, I used to like this. Process was difficult, all those exceptions, terminology, discussions, yucky. Features where clear - a screen here, a database there, a business rule there, data and task oriented.

But in these days of evry growing complexity, task and data oriented requirements analysis will not do anymore. Why? Because it leads to suboptimal solutions, which force people to work AROUND an application, to perform the process. (And ask a random user how many systems he or she typically needs to use to execute a front to back process - usually a lot...)

Premises: EVERY application we use, we use in the context of a certain process. So, by carefully analysing the process, we can get to better requirements, more project succes (standish) and much better solutions.

There is a problem though. Most of the current Requirements frameworks are still in the Data/Task oriented mode. Think of...
- Feature driven development
- XP's User stories
- RUP's Use cases
All these items are focused on isolated functionality, that brings value to a user, in a logical transaction. Process is much more than that! So, how do we capture this.
Sure, some methods call for "business modelling", but most of these elements are, to say the least, not very developed.

BPM would be a great vehicle to improve this.
It asks us from an analysis perspective to:
- SCOPE: Define the scope for a to-be process area and supporting IT solution
- GET CONTEXT: Analyse the processes, front to back, including trigger, input, output, steps, information required in a step, data produced in a step, data flow, actors, KPI's, etc
- DEFINE HIGH LEVEL REQUIREMENTS And then dive deeper: per step, we can start thinking about the required IT support.

From this step, we can revert back to for instance use cases - to define the functionality a user needs to complete a logical process step. (So: a use case is not the same thing as a business process!)

My lesson: BPM is there for many reasons. And one of them is to help you improve your understanding of the business context and get to better requirements. Add it to the more IT driven methods, and you have a better chance of getting to project success.


James Taylor said...

Great post - would have tracked back if I could but I can't so here's a link
Author of SMart (Enough) Systems
ebizQ blog

Roeland Loggen said...

James' reaction was:

Roeland had an interesting post this week on the need for a new approach to requirements and BPM's role in that. He outlined the three classic causes of problems with projects (Lack of User Input, Incomplete Requirements & Specifications, Changing Requirements & Specifications) and discussed how a BPM approach can help with that.
Not only do I agree with him, I would argue that exactly the same rationale drives a need to keep rules out of requirements. Process definitions are not the same as requirements. They change on different schedules, have different regulators/policy setters and involve a different degree of business user involvement at various times. Similarly neither process definitions nor requirements are suitable for capturing the rules behind decisions. All of these, like data, need to be described separately and referenced to each other if you are to build a model of the solution that can be built and then modified over time.
So lay out your process, working with your users and use use cases and other requirements tools to capture requirements while keeping the rules that drive your decisions out of both but linked. If you use a BPMS to manage the process and a BRMS to manage decisions you can even ensure that the flexibility and agility you need will be there.
Scott Sehlhorst wrote a nice piece on separating rules and requirements to which I responded here.

Roeland Loggen said...

James, I partly agree. I created a new posting on Rules vs Requirements....

Scott Sehlhorst said...

Hey Roeland - I responded to your comment on my article: http://tynerblain.com/blog/2007/10/11/stakeholder-goals/#comment-165186

This is definitely an interesting conversation, looking forward to more of your thoughts!


PM Hut said...


I have published an article on the Chaos Report of 2009, you and your readers might find it useful. It also compares 2009 to previous years, you can easily see that the failure rate is actually decreasing.