Lately, I have been seeing a lot of wonderfull features of a BPM technology suite.
Point, click, ah, building applications is so easy. Point to existing web services, pull them in a BPMN flow, create Xforms that talk to web services, no pains!
Really?
Sure, it's easy to create the application, given the current requirements and services.
But what if a service change?
Suddenly I found myself back in early 90's, with 2 tier client server tooling. Oh, what a delight after the green screen software. Widgets, windows, and click click link it to a database table, an SQL query, and tada, the software worked!
Until a requirement changed, we needed to make a small change to the database, and realized 40% of our windows would not work and needed fixing (even though their functionality did not change...).
It's called coupling....
I am assuming that many of these BPM tools have this risk: the risk that we build BPMN and presentation layer stuff, directly connecting to services. Leading to heavy coupling.
The alternative is supported by most tools too, I hope: an intermediate layer, that shields presentation to back-end services. Call it controller or business objects or whatever -
but please use it...
Friday, October 26, 2007
Towards a new TLA: ITPM
Short post, it's late :-)
Interesting article at BPM enterprise about BPM for software development
(link: http://www.bpmenterprise.com/content/c071022a.asp)
What's missing is a key metric: the likelyhood/trackrecord that we deliver on scheduled date.
It's funny - all the effort in various software development methodologies.
In a way it's all (new TLA:) ITPM. IT Process management....
As a BA/BPM specialist, and 15 years of IT development/projectmanagement experience, it often happens that when starting on a new or running project, 20 - 40% of my time is actually spent on process improvement... in the project. E.g. implementing, structuring, refining and improving requirement management process, issue management, testing approach etc. :-).
Is your job also often ITPM?
Interesting article at BPM enterprise about BPM for software development
(link: http://www.bpmenterprise.com/content/c071022a.asp)
What's missing is a key metric: the likelyhood/trackrecord that we deliver on scheduled date.
It's funny - all the effort in various software development methodologies.
In a way it's all (new TLA:) ITPM. IT Process management....
As a BA/BPM specialist, and 15 years of IT development/projectmanagement experience, it often happens that when starting on a new or running project, 20 - 40% of my time is actually spent on process improvement... in the project. E.g. implementing, structuring, refining and improving requirement management process, issue management, testing approach etc. :-).
Is your job also often ITPM?
Monday, October 15, 2007
Towards a new requirements framework
James triggered me again, around Requirements in a BPM context. See http://www.ebizq.net/blogs/decision_management/2007/10/whats_the_difference_between_r.php
I notice in discussions with clients that there is a great deal of confusion about the notion of requirements. I get quite careful when clients start asking "are the requirements done?", "do we have clear requirements"?
Same with terms as "Software requirements" "Specifications" "Business Requirements". When I ask a bit further, the definition, content and context of these terms differ per person, even in the same organisation. If Standish sees 'Clear Requirements" as one of the top-3 CSF's for projects, then it would be quite nice if we all know what we mean by that. How I love clear concepts :-).
Working with a client recently, we discovered together that their change processes were, well, somewhat messy. As change as a process is clearly an element in your BPM strategy: e.g you want to have control over this process as well!, we researched the painpoints in more detail.
One central thema - you guessed it - no shared language around the concept of requirements.
If we don't share the same language on the meta-concepts and the quality criteria around requirements, how would we even start to hope that we understand the requirements in itself?
What we came up with was a high level Requirements Framework, which defines requirement area's. The following diagram is useful:
I notice in discussions with clients that there is a great deal of confusion about the notion of requirements. I get quite careful when clients start asking "are the requirements done?", "do we have clear requirements"?
Same with terms as "Software requirements" "Specifications" "Business Requirements". When I ask a bit further, the definition, content and context of these terms differ per person, even in the same organisation. If Standish sees 'Clear Requirements" as one of the top-3 CSF's for projects, then it would be quite nice if we all know what we mean by that. How I love clear concepts :-).
Working with a client recently, we discovered together that their change processes were, well, somewhat messy. As change as a process is clearly an element in your BPM strategy: e.g you want to have control over this process as well!, we researched the painpoints in more detail.
One central thema - you guessed it - no shared language around the concept of requirements.
If we don't share the same language on the meta-concepts and the quality criteria around requirements, how would we even start to hope that we understand the requirements in itself?
What we came up with was a high level Requirements Framework, which defines requirement area's. The following diagram is useful:
It defines a framework of types/areas of requirements.
The area's:
1. Business Requirements. As in: requirements you put upon your business.
Strategic Goals & Drivers
These are statements about the intent and plans of the organization. 'Be number 1 in market XYZ, region ABC". "We sell to <target audience> through channels X, Y and Z". "We want to have an operating income of at least XXXX".
Product requirements
All the properties of your products (or services). And the rules of engagement - when are you willing to sell a product? What terms and conditions?
Information requirements
What information do you need when performing your processes? Per process? Per activity? What information do you produce?
How is this information structured? What rules apply (correctness, etc)?
What information is needed to steer your organisation and it's processes?
What information is needed from external stakeholders?
Process requirements
What processes do you need to exist? To serve your customers, deal with suppliers, produce products, steer processes and change processes? What supporting processes?
Which steps in a process? Inputs, outputs, link to goals, etc.
Organisation requirements?
How do you want the organisation to be structured? Which roles? Which authorities? What skill levels?
How many people?
2. Software requirements. As in: requirements you put upon your IT systems?
Functional requirements
WHAT does the application need to be able to do.
Non-functional requirements
Think URPS. Number of users to support, performance, etc.
Architecture requirements
Often put in the same area as non-functionals, but in fact very different. These are requirements from another stakeholder group, and need another area: what (mostly longterm) structures, practices, and technology standards do you require the application to comply with.
A number of notes:
- Each area has it's own value and it's own techniques & best practices to understand, formulate and document the requirements in the area.
- Areas are interrelated. Actually, most interesting requirements you will find on the borders: when performing process A to sell product B with information do we need when in step C. When performing process B, what functionality is required from the IT system?
- In each area you will find hierarchy in terms of detail. In process, for instance, a high level services overview, then drill down in process architecture, process activity flows, etc.
- Functional Requirements is the area that gets most attention. Typical techniques to document requirements in this area are "System Should .....", use cases, business rules documents, etc.
- Note that business rules are types of requirements you will find in more area's. In information: an customer can order multiple products in one order... In process: if insured amount > XYZ manual approval by role ABC is needed. Etc.
- You could call this complete framework "business architecture". I am always a bit carefull with "architecture", since that's an even vaguer concept. My objectives are to have a framework that evryone in a company can understand (and not some ivory tower group).
- The pyramid is also a great tool for knowledge management. If you work somewhere, these are the area's you should understand, to know your own role and added value!
Observation:
I think that methodologies that play a part in changing a business, including software development life cycle methods such as RUP, SDM, etc. should be innovated to methods that understand that requirements can be defined on multiple levels (e.g. system - as in organization and in software system) and that are able to integrate into these broader frameworks. This to prevent the earlier blogged about "task oriented applications" (see http://process-transformation.blogspot.com/2007/10/need-for-new-requirements-approach-and.html).
Saturday, October 13, 2007
Requirements vs Process and Rules
James Taylor reacted to a posting about the importance of good Requirements. He sees process as something else than requirements. And he advocates a clear seperation of the concept of Rule and Requirement.
See http://www.ebizq.net/blogs/decision_management/2007/10/requirements_business_process.php
First on the process part vs requirements.
I see a process as a specific concept in requirements. It can be requirements towards a business:
- This is the way I want our people to perform this process (a business requirement)
Or it can form part of the requirements to a system (a software requirement)
- This is the process I want the system to follow when processing claims.
Or a mix of course.
So, is a process definition a software requirement? Sometimes.
Then the part of the rules: again, i don't fully agree.
First of all - YES. When we are trying to understand how an AS-IS situation is working or a TO-BE situation should be working, we can use various concepts in analysis. What processes do you have? What information flow, inputs, output? What do you do in an activity? And, what rules apply for decisions - why do you refuse this order, how do you calculate that price, why do you decide to escalate this?
Rules analysis are a great tool for understanding. And during requirements analysis, I typically seperate them out of the other requirements, simply because it's a separate set of things/concepts.
But are rules requirements or not? I think rules are, for a start, always a business requirement: "when we are doing business, I want our people to follow this rule".
And, in my opinion, when a rule will get automated, we will need a requirement defining the rule. The rule becomes a software requirement: we want to IT system to perform CHECK XYZ, and based on the results do A or B or C.
Now - before I continue, my requirement constraint: a requirement is part of the WHAT definition of a desired system, and does not define the solution.
I have the suspicion that people are separating rules out of the requirements, because they have the solution already in the back of their minds: a rules engine or BRMS.
This links back to my earlier posting on "pick the right technology for your functionality".
Rules can be put in a BRMS. Or programmed in Java. Or in Cobol. Or asked to a user (continue Y/N?). It depends on the requirements...mainly the non-functional ones.
Are the requirements that define this rules highly violatile? Or should they be maintainable by non-programmers? Well, don't put them deeeeep in code in that case.
Are rules quite fixed? Do you only have Java programmers and no money for a BRMS? Does the business delegate all IT rule keeping to the IT people? Go ahead, bury them in code (but do make sure you defined them as a seperate part in your requirements, and make them traceabile in your code...!)
/**=== Check business Rule XYZ , traceability: BR23.22 =======
if (bla.blo(var1, var2, var3)
{.....}
else
{....}
/** end check BR23.22
Conclusion: each technology has it's business case. BPMS and BRMS as well.
See http://www.ebizq.net/blogs/decision_management/2007/10/requirements_business_process.php
First on the process part vs requirements.
I see a process as a specific concept in requirements. It can be requirements towards a business:
- This is the way I want our people to perform this process (a business requirement)
Or it can form part of the requirements to a system (a software requirement)
- This is the process I want the system to follow when processing claims.
Or a mix of course.
So, is a process definition a software requirement? Sometimes.
Then the part of the rules: again, i don't fully agree.
First of all - YES. When we are trying to understand how an AS-IS situation is working or a TO-BE situation should be working, we can use various concepts in analysis. What processes do you have? What information flow, inputs, output? What do you do in an activity? And, what rules apply for decisions - why do you refuse this order, how do you calculate that price, why do you decide to escalate this?
Rules analysis are a great tool for understanding. And during requirements analysis, I typically seperate them out of the other requirements, simply because it's a separate set of things/concepts.
But are rules requirements or not? I think rules are, for a start, always a business requirement: "when we are doing business, I want our people to follow this rule".
And, in my opinion, when a rule will get automated, we will need a requirement defining the rule. The rule becomes a software requirement: we want to IT system to perform CHECK XYZ, and based on the results do A or B or C.
Now - before I continue, my requirement constraint: a requirement is part of the WHAT definition of a desired system, and does not define the solution.
I have the suspicion that people are separating rules out of the requirements, because they have the solution already in the back of their minds: a rules engine or BRMS.
This links back to my earlier posting on "pick the right technology for your functionality".
Rules can be put in a BRMS. Or programmed in Java. Or in Cobol. Or asked to a user (continue Y/N?). It depends on the requirements...mainly the non-functional ones.
Are the requirements that define this rules highly violatile? Or should they be maintainable by non-programmers? Well, don't put them deeeeep in code in that case.
Are rules quite fixed? Do you only have Java programmers and no money for a BRMS? Does the business delegate all IT rule keeping to the IT people? Go ahead, bury them in code (but do make sure you defined them as a seperate part in your requirements, and make them traceabile in your code...!)
/**=== Check business Rule XYZ , traceability: BR23.22 =======
if (bla.blo(var1, var2, var3)
{.....}
else
{....}
/** end check BR23.22
Conclusion: each technology has it's business case. BPMS and BRMS as well.
The right technology for the right functionality
A number of posts reminded me about a discussion I had some years ago.
Situation: we were introducing advanced output management technology (e.g. a tool + programming language to create applications that were able to receive XML and generate output (PDF's, prints, etc)).
This technology had a number of advantages:
- Quicker time to market
- Powerfull language
- Easier testing (what you see is what you get programming environment)
- More possibilities in layout
The objective was to reduce the code base in the legacy area considerably. In those huge Cobol programs, many were programmed to generate output (line print, so very '70-ies output). Of course, whenever output needed to change - a new product, new rules, a new contract, quote, etc, Cobol programmers would need to start hacking. They did that for years and saw no issue with that. But the business did - waiting for 2 - 3 months of implementation time, a lot of testing, and that 70-ies layout.
The Cobol group had a lot of resistance towards the new technology. One argument was: well, now we can programm the WHOLE application in one language, but with this technology, we will need multiple skills in multiple languages.
Yep. And I explained that we don't use a hammer to crush coffeebeans to make coffee. We use a grinder. And the hammer we use for nails.
The lesson: when requirements ask for a certain solution, pick the right technology for it.
The same discussion I recognize in a number of postings on BPM (Why Java developers hate BPM tools):
http://weblogs.java.net/blog/johnreynolds/archive/2007/10/why_do_java_dev_1.html
http://www.oreillynet.com/onjava/blog/2007/10/do_i_hate_bpm_no_i_hate_bpm_pr.html?CMP=OTC-FP2116136014&ATT=Do+I+hate+BPM+No+I+hate+BPM+Products
The key question is again: what are the requirements, and what is the best technology to solve something. I am sure most BPM featurs you can easily develop in Java. Hell - you can develop a database with Java, but big chance is you don't....
The key advantages with using BPM technology:
- You can change a process quickly
- The process diagram reads (most of the time) quicker than 10 pages of a programming languages, heck - maybe some people on the business side even understand!
- The whole SOA calling is hidden - it helps you to focus on the process and information flow instead of Java and XML
- Process and activities are measured for you, in a standard way.
Situation: we were introducing advanced output management technology (e.g. a tool + programming language to create applications that were able to receive XML and generate output (PDF's, prints, etc)).
This technology had a number of advantages:
- Quicker time to market
- Powerfull language
- Easier testing (what you see is what you get programming environment)
- More possibilities in layout
The objective was to reduce the code base in the legacy area considerably. In those huge Cobol programs, many were programmed to generate output (line print, so very '70-ies output). Of course, whenever output needed to change - a new product, new rules, a new contract, quote, etc, Cobol programmers would need to start hacking. They did that for years and saw no issue with that. But the business did - waiting for 2 - 3 months of implementation time, a lot of testing, and that 70-ies layout.
The Cobol group had a lot of resistance towards the new technology. One argument was: well, now we can programm the WHOLE application in one language, but with this technology, we will need multiple skills in multiple languages.
Yep. And I explained that we don't use a hammer to crush coffeebeans to make coffee. We use a grinder. And the hammer we use for nails.
The lesson: when requirements ask for a certain solution, pick the right technology for it.
The same discussion I recognize in a number of postings on BPM (Why Java developers hate BPM tools):
http://weblogs.java.net/blog/johnreynolds/archive/2007/10/why_do_java_dev_1.html
http://www.oreillynet.com/onjava/blog/2007/10/do_i_hate_bpm_no_i_hate_bpm_pr.html?CMP=OTC-FP2116136014&ATT=Do+I+hate+BPM+No+I+hate+BPM+Products
The key question is again: what are the requirements, and what is the best technology to solve something. I am sure most BPM featurs you can easily develop in Java. Hell - you can develop a database with Java, but big chance is you don't....
The key advantages with using BPM technology:
- You can change a process quickly
- The process diagram reads (most of the time) quicker than 10 pages of a programming languages, heck - maybe some people on the business side even understand!
- The whole SOA calling is hidden - it helps you to focus on the process and information flow instead of Java and XML
- Process and activities are measured for you, in a standard way.
Case management - The empowered information worker...
Recently I was at the Cordys Cordial (http://www.cordys.com/) day, a day were they presented the new version of their platform.
A funny statement by (I think John Pyke) was: processes are created when something failed. :-) - now, that's a nice start of BPM... And John also nagged towards SAP, which as an abbreviation would stand for Stone Age Processing or Spent All Profit. Hm.
A good presentation was on the new case management features of the Cordys platform.
These features were added to the platform when they discovered (while doing a project with my company) that certain processes are not that straight forward as hoped for. Aka - forget modelling it in BPMN, the diagram would be too complex and add no value. I sometimes wonder how many processes actually are really that structured, but that's another post...
A small summary on the case management features and thoughts behind Cordys:
- Where in production workflow, each process instance has a certain flow, each case will have a unique sequence of activities
- When a case is started (based on some event), certain activities are automatically opened, based on the type of the case and certain event criteria.
- A user is presented with activities, can open them (role bases), execute them, delegate them or create new activities (select or freely define).
- Status of cases and activities can be tracked, measured (monitoring)
- Three main components were presented:
+ Document matching - used to trigger creation of a case, or to be able to link an incoming document to a running case. Remark: strange - I would not have called this a document matching system. Incoming documents (and other information flows) basically define EVENTS. So - it's a CEP or Event Correlation Engine....
+ Case management - the actual presentation towards the user of all case activities, history, data, etc.
+ activity management - the ability to execute, define, delegate, etc activities.
- Another component is the maintenance area - in which case rules and activities can be defined, including roles etc.
They had a nice slide which defined a sliding scale between pure workflow and pure case management.
- 100% production workflow
- production workflow with small cases
- Flow + case
- Case management, with small flows
- Pure case management
I liked it a lot - since I do think you will encounter many processes where things are mixed. For instance a straight through processing unit, which asks for case management when exceptions occur.
Some remarks:
- Unfortunately, the "activities" are NOT integrated in the current BPM engine. E.g. when I create flows in the BPM engine, I can not mark them as optional or mandatory sets of activities in the case management component. Ideally, in the case environment, as a user (or automatic triggers by certain rules) I want to be able to pick: a listed activity, a free activity, a process fragment or a transaction. This also meant that in the current Cordys setup - the BPM inbox and the case management module are not integrated. Ouch!
- Cordys talked a lot about "the ability to measure status of all cases, allowing you to manage the workforce". Well - since Case flow can not be predicted, in my opinion, workforce management will be difficult. Sure I can see what activities are open NOW, but I have no clarity on the expected activities for these cases, and the required resources...
- What I see as a challenge is case data. When you are doing production workflow, most process instance data that you will need, is clear on design time. Each activity and decision needs certain data - and you will need to make sure it's available in the process instance. But what if you are working with cases, where you do not exactly know what information is needed to supply the knowledge worker with enough information to decide on activities and case closure.
- Another item is where to store this data? In the Cordys solution, all data is stored in unstructured documents. But what to do with all data that is stored in backend systems, which users maybe need to decide? Store in the case management system, or call/get every time the case is opened and viewed?
The demo showed a nice presentation layer which showed the case in full context. It allowed a user to understand the trigger, all done activities, notes, etc. A true process workbench.
The main lesson I started realizing, seeing the demo, was that whatever the type of process support a knowledge worker gets with his or her BPM application - the best would be to provide him or her with this full context. Don't make the mistake of underestimating your users and only give them the microscopic task view (aka inbox with single task - do this, no questions asked), without a clue about the full process!
Empowerment and process ownership are things that should be the basis for process workers..
A funny statement by (I think John Pyke) was: processes are created when something failed. :-) - now, that's a nice start of BPM... And John also nagged towards SAP, which as an abbreviation would stand for Stone Age Processing or Spent All Profit. Hm.
A good presentation was on the new case management features of the Cordys platform.
These features were added to the platform when they discovered (while doing a project with my company) that certain processes are not that straight forward as hoped for. Aka - forget modelling it in BPMN, the diagram would be too complex and add no value. I sometimes wonder how many processes actually are really that structured, but that's another post...
A small summary on the case management features and thoughts behind Cordys:
- Where in production workflow, each process instance has a certain flow, each case will have a unique sequence of activities
- When a case is started (based on some event), certain activities are automatically opened, based on the type of the case and certain event criteria.
- A user is presented with activities, can open them (role bases), execute them, delegate them or create new activities (select or freely define).
- Status of cases and activities can be tracked, measured (monitoring)
- Three main components were presented:
+ Document matching - used to trigger creation of a case, or to be able to link an incoming document to a running case. Remark: strange - I would not have called this a document matching system. Incoming documents (and other information flows) basically define EVENTS. So - it's a CEP or Event Correlation Engine....
+ Case management - the actual presentation towards the user of all case activities, history, data, etc.
+ activity management - the ability to execute, define, delegate, etc activities.
- Another component is the maintenance area - in which case rules and activities can be defined, including roles etc.
They had a nice slide which defined a sliding scale between pure workflow and pure case management.
- 100% production workflow
- production workflow with small cases
- Flow + case
- Case management, with small flows
- Pure case management
I liked it a lot - since I do think you will encounter many processes where things are mixed. For instance a straight through processing unit, which asks for case management when exceptions occur.
Some remarks:
- Unfortunately, the "activities" are NOT integrated in the current BPM engine. E.g. when I create flows in the BPM engine, I can not mark them as optional or mandatory sets of activities in the case management component. Ideally, in the case environment, as a user (or automatic triggers by certain rules) I want to be able to pick: a listed activity, a free activity, a process fragment or a transaction. This also meant that in the current Cordys setup - the BPM inbox and the case management module are not integrated. Ouch!
- Cordys talked a lot about "the ability to measure status of all cases, allowing you to manage the workforce". Well - since Case flow can not be predicted, in my opinion, workforce management will be difficult. Sure I can see what activities are open NOW, but I have no clarity on the expected activities for these cases, and the required resources...
- What I see as a challenge is case data. When you are doing production workflow, most process instance data that you will need, is clear on design time. Each activity and decision needs certain data - and you will need to make sure it's available in the process instance. But what if you are working with cases, where you do not exactly know what information is needed to supply the knowledge worker with enough information to decide on activities and case closure.
- Another item is where to store this data? In the Cordys solution, all data is stored in unstructured documents. But what to do with all data that is stored in backend systems, which users maybe need to decide? Store in the case management system, or call/get every time the case is opened and viewed?
The demo showed a nice presentation layer which showed the case in full context. It allowed a user to understand the trigger, all done activities, notes, etc. A true process workbench.
The main lesson I started realizing, seeing the demo, was that whatever the type of process support a knowledge worker gets with his or her BPM application - the best would be to provide him or her with this full context. Don't make the mistake of underestimating your users and only give them the microscopic task view (aka inbox with single task - do this, no questions asked), without a clue about the full process!
Empowerment and process ownership are things that should be the basis for process workers..
Friday, October 12, 2007
BPM technology consolidation...
Another one bites the dust...
http://www.ebizq.net/news/8554.html
Oracle to buy BEA.
Hm, interesting. Definitely a step towards further consilidation.
How many process engines will Oracle have now???
http://www.ebizq.net/news/8554.html
Oracle to buy BEA.
Hm, interesting. Definitely a step towards further consilidation.
How many process engines will Oracle have now???
Tuesday, October 09, 2007
The need for a new requirements approach - and BPM's role
Ok, it has happened again. Fortunately, it doesn't happen often, but from time to time, I find myself in the unlucky projectworld of "Type 2" , described by the Standish Chaos research as "Over time, Over budget and Less Functionality". Ouch.
Remember the Standish Chaos Report? Also see for instance http://www.projectsmart.co.uk/docs/chaos-report.pdf
It lists succesrates for projects, based on research. And it lists succes and failure factors.
It's findings (So, I am part of the majority with my project :-):
- 16.2% of all projects are Resolution Type 1, or project success: The project is completed on-time and on-budget, with all features and functions as initially specified.
- 52.7% - Resolution Type 2, or project challenged: The project is completed and operational but
over-budget, over the time estimate, and offers fewer features and functions than
originally specified.
- 31.1% - Resolution Type 3, or project impaired: The project is cancelled at some point during the development cycle.
The failure factors for projects ending up as a 2 or 3:
The survey participants were also asked about the factors that cause projects to be challenged.
1. Lack of User Input 12.8%
2. Incomplete Requirements & Specifications 12.3%
3. Changing Requirements & Specifications 11.8%
And yes - in this project, for all kinds of reaons, these items are true.
It wasn't me, though, I was not working on the requirement side ;-). But a repeating lesson I learned: good requirements are essential. And in my opinion that needs process-context awareness.
Remember the old days of applicatie analysis and design? I call it feature oriented requirements engineering: talk to users, find out what's difficult in their job, and think up a set of features to cover their issues.
Example: we do sales, but miss information on our clients and status of orders.
Ah! We prototype a nice screen, build you a nice database with clients, and the order status we get through EAI from your order entry system. Problem solved!
Really? No.
You end up with a so-called data & isolated task oriented application. It's not aware of the process.
To be honest, as IT requirements person, I used to like this. Process was difficult, all those exceptions, terminology, discussions, yucky. Features where clear - a screen here, a database there, a business rule there, data and task oriented.
But in these days of evry growing complexity, task and data oriented requirements analysis will not do anymore. Why? Because it leads to suboptimal solutions, which force people to work AROUND an application, to perform the process. (And ask a random user how many systems he or she typically needs to use to execute a front to back process - usually a lot...)
Premises: EVERY application we use, we use in the context of a certain process. So, by carefully analysing the process, we can get to better requirements, more project succes (standish) and much better solutions.
There is a problem though. Most of the current Requirements frameworks are still in the Data/Task oriented mode. Think of...
- Feature driven development
- XP's User stories
- RUP's Use cases
All these items are focused on isolated functionality, that brings value to a user, in a logical transaction. Process is much more than that! So, how do we capture this.
Sure, some methods call for "business modelling", but most of these elements are, to say the least, not very developed.
BPM would be a great vehicle to improve this.
It asks us from an analysis perspective to:
- SCOPE: Define the scope for a to-be process area and supporting IT solution
- GET CONTEXT: Analyse the processes, front to back, including trigger, input, output, steps, information required in a step, data produced in a step, data flow, actors, KPI's, etc
- DEFINE HIGH LEVEL REQUIREMENTS And then dive deeper: per step, we can start thinking about the required IT support.
From this step, we can revert back to for instance use cases - to define the functionality a user needs to complete a logical process step. (So: a use case is not the same thing as a business process!)
My lesson: BPM is there for many reasons. And one of them is to help you improve your understanding of the business context and get to better requirements. Add it to the more IT driven methods, and you have a better chance of getting to project success.
Remember the Standish Chaos Report? Also see for instance http://www.projectsmart.co.uk/docs/chaos-report.pdf
It lists succesrates for projects, based on research. And it lists succes and failure factors.
It's findings (So, I am part of the majority with my project :-):
- 16.2% of all projects are Resolution Type 1, or project success: The project is completed on-time and on-budget, with all features and functions as initially specified.
- 52.7% - Resolution Type 2, or project challenged: The project is completed and operational but
over-budget, over the time estimate, and offers fewer features and functions than
originally specified.
- 31.1% - Resolution Type 3, or project impaired: The project is cancelled at some point during the development cycle.
The failure factors for projects ending up as a 2 or 3:
The survey participants were also asked about the factors that cause projects to be challenged.
1. Lack of User Input 12.8%
2. Incomplete Requirements & Specifications 12.3%
3. Changing Requirements & Specifications 11.8%
And yes - in this project, for all kinds of reaons, these items are true.
It wasn't me, though, I was not working on the requirement side ;-). But a repeating lesson I learned: good requirements are essential. And in my opinion that needs process-context awareness.
Remember the old days of applicatie analysis and design? I call it feature oriented requirements engineering: talk to users, find out what's difficult in their job, and think up a set of features to cover their issues.
Example: we do sales, but miss information on our clients and status of orders.
Ah! We prototype a nice screen, build you a nice database with clients, and the order status we get through EAI from your order entry system. Problem solved!
Really? No.
You end up with a so-called data & isolated task oriented application. It's not aware of the process.
To be honest, as IT requirements person, I used to like this. Process was difficult, all those exceptions, terminology, discussions, yucky. Features where clear - a screen here, a database there, a business rule there, data and task oriented.
But in these days of evry growing complexity, task and data oriented requirements analysis will not do anymore. Why? Because it leads to suboptimal solutions, which force people to work AROUND an application, to perform the process. (And ask a random user how many systems he or she typically needs to use to execute a front to back process - usually a lot...)
Premises: EVERY application we use, we use in the context of a certain process. So, by carefully analysing the process, we can get to better requirements, more project succes (standish) and much better solutions.
There is a problem though. Most of the current Requirements frameworks are still in the Data/Task oriented mode. Think of...
- Feature driven development
- XP's User stories
- RUP's Use cases
All these items are focused on isolated functionality, that brings value to a user, in a logical transaction. Process is much more than that! So, how do we capture this.
Sure, some methods call for "business modelling", but most of these elements are, to say the least, not very developed.
BPM would be a great vehicle to improve this.
It asks us from an analysis perspective to:
- SCOPE: Define the scope for a to-be process area and supporting IT solution
- GET CONTEXT: Analyse the processes, front to back, including trigger, input, output, steps, information required in a step, data produced in a step, data flow, actors, KPI's, etc
- DEFINE HIGH LEVEL REQUIREMENTS And then dive deeper: per step, we can start thinking about the required IT support.
From this step, we can revert back to for instance use cases - to define the functionality a user needs to complete a logical process step. (So: a use case is not the same thing as a business process!)
My lesson: BPM is there for many reasons. And one of them is to help you improve your understanding of the business context and get to better requirements. Add it to the more IT driven methods, and you have a better chance of getting to project success.
Friday, September 28, 2007
Came across the new BPM institute magazine, http://www.bpminstitute.org/magazine.html, and found the following diagram:
Some observations:
- It's first of all a nice overview of the improvement frameworks existing today!
- For some reason I always get a bit tired of the arrogant tone of voice of the Omega-6 and CEM/CEMM supplier (now Bennugroup). Well, I guess the above diagram tells enough - no trace....
- I am glad to see Lean high on the list. In my opinion a great framework.
- It would be nice to see this broken down by industry - for some reasons, in Finance, Lean is only slowly cathing on.
- It would be nice to see this broken down by "reason/goals for improvement". My suspicion is that we can link goals and methods still quite a bit (visibility - balanced score card, EFQM - Quality, ISO - Compliance, etc)
Saturday, September 22, 2007
From organisation chart to process chart
When business analysts come into a new business, they need to quickly understand the structure, people, products and processes in the company.
One of the key things I usually ask for is the organizational chart (aka organigram, organogram).
As a side note: it's amazing how often I either get incomplete or outdated diagrams. I wonder in that case how the heck people in their own organization understand the structure, and can place themselves + link themselves to the goals of the organization. So much for knowledge management, in such a simple form!
In these process-orientation days, I have found organization charts more and more disfunctional. What do you get?
- Powerstructure/reportingstructure
- Functional area's and departments (but often functional departments support multiple processes, and only parts of it)
- Names of people
The thing is - the basic question: "What does this company do" is not easy to answer. For a process-oriented person (like me :-)), that is often difficult - I want to see the chain of actions that lead to results for customers, and their interrelation + the people responsible for the different elements in the chain. Stuff that not easily can extracted from the organization chart.
So - can we come up with a new standard? The processchart, procesogram? Some high-level, I understand this in 1 minute, easy to draw and maintain diagram?
That quickly answers:
- External stakeholders
- Key transactions
- Chains of actions/process areas that support and fullfill transactions
- Key people responsible for parts in the chain, chain accountability
I challenge the reader - do you know something like this??
Some of my thoughts:
- The dutch "Demo" method (http://www.demo.nl/) offers a very concise diagram (organisation construction diagram) - however, this misses the accountability structure.
- In the method of Cordys - there is a business model
One of the key things I usually ask for is the organizational chart (aka organigram, organogram).
As a side note: it's amazing how often I either get incomplete or outdated diagrams. I wonder in that case how the heck people in their own organization understand the structure, and can place themselves + link themselves to the goals of the organization. So much for knowledge management, in such a simple form!
In these process-orientation days, I have found organization charts more and more disfunctional. What do you get?
- Powerstructure/reportingstructure
- Functional area's and departments (but often functional departments support multiple processes, and only parts of it)
- Names of people
The thing is - the basic question: "What does this company do" is not easy to answer. For a process-oriented person (like me :-)), that is often difficult - I want to see the chain of actions that lead to results for customers, and their interrelation + the people responsible for the different elements in the chain. Stuff that not easily can extracted from the organization chart.
So - can we come up with a new standard? The processchart, procesogram? Some high-level, I understand this in 1 minute, easy to draw and maintain diagram?
That quickly answers:
- External stakeholders
- Key transactions
- Chains of actions/process areas that support and fullfill transactions
- Key people responsible for parts in the chain, chain accountability
I challenge the reader - do you know something like this??
Some of my thoughts:
- The dutch "Demo" method (http://www.demo.nl/) offers a very concise diagram (organisation construction diagram) - however, this misses the accountability structure.
- In the method of Cordys - there is a business model
Curious if other people have a better diagram technique!
Five areas you need in your process architecture
Whether you are trying to understand processes enterprise wide, or just for a small department, in my opinion, there are always five process areas that you should consider.
Here in diagram:
Here in diagram:
1. Operational Processes
Processes in this area are considered core to the organization''s goals. They make up the business model of the company and focus on making customers happy, by procuring, producing, selling, marketing and servicing products/services. The value chain of your company.
2. Supporting Processes
Although not core, these processes facilitate all the other process areas and are often of crucial importance. Think Admin, HR, IT support, Finance, Legal, QC/QA.
3. Supplier and Customer processes
From an outside-in perspective, it is very important to understand (and help improve!) the processes of your main outside stakeholders: your suppliers and customers. By understanding their processes, you are able to help improve them, which in the end, will lead to competitive advantage. Model these processes from a supply chain perspective - optimize the chain in total.
4. Management processes
These processes steer the other processes (and cover the plan, check and act of the Demming cycle). They form the process control layer in your business.
5. Change processes
Yes, change is a process to. In modern days, change will occur more and more often. A repeatable framework for change, aka a process, will help you to become more agile: when a change is needed, all people will know their role and responsibility, to enable the business to respond fast. The change process includes detecting the need for change, understanding it, designing it, and implementing it, including all impact on the people and technology in all layers.
The 5 competence-areas for real process transformation
I often use this model to explain but also check if a company is well positioned to achieve succesful process transformations - the 5 competence areas:
- Process orientation
- BPM technology
- Change management
- Process improvement
- Subject matter expertise
1. Process orientation
This competence area is what I would call the typical "BPM as a discipline". It asks for people, supported by methods and techniques to see the processes that are embedded in the organization, see the interrelation between them and the alignment with the busines goals, and see opportunities and risks in the as-is process situation. It asks for people that understand how interventions in process, people and technology can help the company to perform better. Concepts, methods and tools that people should be able to understand/use are for instance:
- Business model - linking goals, business services and processes
- Business architecture
- Process architecture
- Business case - TCO of processes, sourcing
- Process models for operational processes, supplier/client linked processes (supply chain understanding), management processes, supporting processes and change processes
- Information architecture - what information concepts are used, and in which step in what process is what information needed to perform the step and what information is delivered
- Performane measurement and process control models - what to measure when (based on CSF's and KPI's) for whom and to control what risk - and the possible steering that management can do when measures ask for action
- Business rules - what rules and knowledge drives what decisions in the processes
- Methods for analysis of processes, and problems-cause analysis and knowledge
2. BPM Technology
IT offers many powerful innovation possibilities. But, people are needed, with the ability to see the possibilities and potential benefits. This requires competences in the technology, including:
- Business process analysis technology, to support people to model / visualize processes
- BPM Suites - automated process coordination engines
- BAM - process intelligence
- WFM - Workflow management, often as part of the BPM suite, offering functionality to support people performing tasks in the context of a process
- Case management - again, functionality to support people performing tasks in processes, where the processes are more difficult to predict and model, and part of the process execution is based on people knowledge and decisions at run-time
- SOA - a way to architect automated processes, based on calling services
- ESB / EAI - Enterprise Service bus (or SOA grids) and other ways to deliver functionality and data needed to perform processes or tasks in a process
- STP - Straight through processing, where the process is coordinated and executed in a totally automated environment with no or very limited (exception handling, sample checks) human involvement
- BRE - Business rules engines (or in EDM terms: platforms for Decision Services)
Important to note it that people are needed that can think outside the "tool". E.g. we do need people that understand specific vendor technology (BEA, IBM, ARIS, Oracle, Cordys, etc), but we also need people that see beyond the limitations and design patterns of these tools, and understand the bigger "process model, execute, measure, improve" needs and features of your business.
3. Change management
Process transformation is a change. And change, asking people and their supporting systems to change, can be hard. It's an area often forgotten - many BPM projects are, unfortunately, run either from a technology perspective or an architecture/blue print perspectice. Change is not following a design-implement paradigma, and asks for people oriented interventions - two-way communication, training, coaching, steering, etc. It asks for people with competence in:
- Change management
- People management, development of people
- Project management
- Requirements management (as-is, to-be)
- Design, simulate, evaluate, test, learn
4. Process improvement
I am very happy that process improvement is slowing turning into a art and science on it's own. In fact, improvement can come in a number of impact/scope combinations:
- BPI - Business process improvement : small changes within the current processes, e.g. removing or reshuffling tasks, tuning business rules and tuning work/workforce-assignments
- BPT - Business process transformations: medium to large changes, were processes are analyzed and thoroughly adjusted, including BPM/Case management/STP automation
- BPR - Full scale process redesign, with often a total new look at product-market-channels orientation.
Beware - do not let a simple business analist or architect design process improvements - it will lead to nothing. Process improvement asks for people with skills in:
- Outside in thinking
- Measurement & benchmarking, statistical process control
- Lean
- Six-Sigma
- TOC
- Operations Research and modern Logistical concepts (JIT, Kaizen, Flow, Pull, Agility)
- Simulation
5. Subject Matter
Whenever you start a process transformation programme, make sure you involve the subject matters. The people you need, are people with extensive knowledge of
- Your company's internal structure, network, power grid and culture
- Your current process portfolio
- Your products
- Your customers
- Your suppliers
- Your legal and compliance environment
I hope this list can help you, as a checklist. Try to make sure you covers these skill area's. Whatever you miss - will mean risk to your transformation project. Risk is okay, as long as you are aware of it.
Being goal and result oriented
Today I saw an add, with a sentence that lingered for a while:
"Innovation is an illusion, until you actually do something about it".
I remember, while studying, that I read the excellent book by Goldratt, "The Goal", about the Theory of Constraints. The thing that stuck was actually a more basic concept: whatever you do, be aware how it relates to your overal goal. Something that in modern management sometimes is called Alignment.
I have been involved with many projects and transformations, and I have always tried this. When I was still more involved into IT/software development, my motto was: "In the end, it's about working software for a happy business".
To my surprise in many situations, I had to fight for this motto. The countless meetings, email conversations and phone conferences where people forgot what we were really striving for, it is amazing.
So, ask yourself: what is it that you REALLY try to achieve. What's your goal?
Are you in IT development? Working software, for a happy business!
Are you in IT support? Working software and empowered users, for a happy business
Are you in process improvement? Better processes!
Are you in change? Real change!
All the rest, the blabla, the politics, the countless reports, the discussions, sure - they are often needed. But they are not the goal. So make sure - know your goal, and, whatever you do, ask yourself - is it helping us forward to our goals?
What's you goal? And what are you doing? Aligned?
"Innovation is an illusion, until you actually do something about it".
I remember, while studying, that I read the excellent book by Goldratt, "The Goal", about the Theory of Constraints. The thing that stuck was actually a more basic concept: whatever you do, be aware how it relates to your overal goal. Something that in modern management sometimes is called Alignment.
I have been involved with many projects and transformations, and I have always tried this. When I was still more involved into IT/software development, my motto was: "In the end, it's about working software for a happy business".
To my surprise in many situations, I had to fight for this motto. The countless meetings, email conversations and phone conferences where people forgot what we were really striving for, it is amazing.
So, ask yourself: what is it that you REALLY try to achieve. What's your goal?
Are you in IT development? Working software, for a happy business!
Are you in IT support? Working software and empowered users, for a happy business
Are you in process improvement? Better processes!
Are you in change? Real change!
All the rest, the blabla, the politics, the countless reports, the discussions, sure - they are often needed. But they are not the goal. So make sure - know your goal, and, whatever you do, ask yourself - is it helping us forward to our goals?
What's you goal? And what are you doing? Aligned?
A personal transformation
It's been a while since I blogged, but that has to do with a large personal transformation. One that will definitely impact all my personal processes :-)
September 16th our first child Hannah Rosa was born. Mother and daughter are doing fine!
September 16th our first child Hannah Rosa was born. Mother and daughter are doing fine!
Wednesday, September 05, 2007
The challenge of being a BPM specialist
As a "process therapist" I encounter many processes in trouble.
The most difficult one is the "But we have always done it like this" or "still in the dark" processes.
The trouble with this is to make people understand that there is a whole new world out there, which will soon replace your business, because customers will simply switch. A world of new practices, new technology, and new demands.
I often encounter these processes in organizations with an almost monopolistic nature, where customers are almost forced to accept the service, simply because of the trouble to switch.
Think - utilities companies - water, electricity, national phone company, internet provider, mobile phone company with the 1 year subscription lock-in, your pension provider, the law, mortgage handler, insurance company, car repair dealer.
Sometimes arrogant, mostly naive. "We have always done it like this".
E.g. it's normal that....
- Customers need to wait weeks, sometimes months on requests
- Things get lost, and are found again
- No insight in metrics such as progress, workload, speed
- If customers call, they need to call back later, because it takes a lot of time to findout the current status (and we can't call them)
- Decisions are taken, but no one knows or tracks why, or even evaluates the business case behind the decision
- Processes and channels are there for the company's efficiency, not for the customer "openinghours: 9:00 - 17:00, closed in the weekend", "no, we can not react to email" "please press XYZ, for ..... The waiting time is YY. This call will cost you XX per minute. There are 40 people before you in queue" "yes, you will need to fill out this form, even though we have all your data already"
We all know these types of companies, and those poor processes in there. And the even more poor customers having to deal with this.
I'm often amazed - how can the people working there, have become used to this, and think it's "normal". The boiling frog metaphore comes to mind... (cook them slowly and they won't notice and get used....until it's too late)
What to do? Can BPM be an answer? The answer is a complex and yet simple one:
Yes, but you need leadership from local managers that understand that they have a growing issue and have the courage and committment to really do something about it.
Without this, forget it.
So if, as a BPM specialist, you encounter the "we have always done it like this" type of process, find the leadership, grow it, test it, and let go if it's not there...There are always other companies that do understand the need for process improvement, and your BPM expertise is very welcome there!
The most difficult one is the "But we have always done it like this" or "still in the dark" processes.
The trouble with this is to make people understand that there is a whole new world out there, which will soon replace your business, because customers will simply switch. A world of new practices, new technology, and new demands.
I often encounter these processes in organizations with an almost monopolistic nature, where customers are almost forced to accept the service, simply because of the trouble to switch.
Think - utilities companies - water, electricity, national phone company, internet provider, mobile phone company with the 1 year subscription lock-in, your pension provider, the law, mortgage handler, insurance company, car repair dealer.
Sometimes arrogant, mostly naive. "We have always done it like this".
E.g. it's normal that....
- Customers need to wait weeks, sometimes months on requests
- Things get lost, and are found again
- No insight in metrics such as progress, workload, speed
- If customers call, they need to call back later, because it takes a lot of time to findout the current status (and we can't call them)
- Decisions are taken, but no one knows or tracks why, or even evaluates the business case behind the decision
- Processes and channels are there for the company's efficiency, not for the customer "openinghours: 9:00 - 17:00, closed in the weekend", "no, we can not react to email" "please press XYZ, for ..... The waiting time is YY. This call will cost you XX per minute. There are 40 people before you in queue" "yes, you will need to fill out this form, even though we have all your data already"
We all know these types of companies, and those poor processes in there. And the even more poor customers having to deal with this.
I'm often amazed - how can the people working there, have become used to this, and think it's "normal". The boiling frog metaphore comes to mind... (cook them slowly and they won't notice and get used....until it's too late)
What to do? Can BPM be an answer? The answer is a complex and yet simple one:
Yes, but you need leadership from local managers that understand that they have a growing issue and have the courage and committment to really do something about it.
Without this, forget it.
So if, as a BPM specialist, you encounter the "we have always done it like this" type of process, find the leadership, grow it, test it, and let go if it's not there...There are always other companies that do understand the need for process improvement, and your BPM expertise is very welcome there!
Saturday, September 01, 2007
SIG Pam articles - open
I came accross this site through Sandy (column2.com).
A recent entree with some great research articles, freely published:
http://www.sigpam.org/2007/08/31/papers-presented-at-the-amcis-2007-minitrack-on-business-process-automation-and-management/
A recent entree with some great research articles, freely published:
http://www.sigpam.org/2007/08/31/papers-presented-at-the-amcis-2007-minitrack-on-business-process-automation-and-management/
Thursday, August 30, 2007
Package Vendors: Free your functionality, free your processes
ebizq published a good whitepaper (
http://www.ebizq.net/to/BIANWP082907W, direct link to http://www.ebizq.net/views/download_raw?metadata_id=8345&what=white_paper) written by Tom Dwyer.
Where we want to go to, is this: if packages only support parts in the end-to-end processes in our companies, we do not want to interface. We want to create the overall process, and link it to services in all the apps we need for that process.
In the following diagram I show a typical example, that you will often encounter:
(click to enlarge)
From a IT perspective, this leads to:
extra technology, extra maintenance burden.
From a business perspective, if we look at this process from a LEAN point of view (http://www.lean.org/), we see there is waste:
- Extra tasks, that do not add value (starting B, nightly batch, rekey activity)
- Time waste, waiting a night before we can continue to serve this customer
What we want is shown in the following diagram:
A key question that is hidden in Tom's article is this:
Can you, as a package vendor, still offer packages that:
- Are only task based (e.g. no process awareness)?
- Offer limited proprietary process technology?
- Which therefore result in difficult integration and suboptimal business and IT solutions?
I think not...
Sure, one last statement - we are not there yet. A recent IDC study also showed that BPM engines are mainly implemented on a project basis, not from an infrastructural basis. This worries me, because we will end up as customers with many many BPM pockets of technology, each requiring licences, knowledge to support, etc. A great role for Enterprise Architecture to help prevent this.... with some help from package vendors that is...
http://www.ebizq.net/to/BIANWP082907W, direct link to http://www.ebizq.net/views/download_raw?metadata_id=8345&what=white_paper) written by Tom Dwyer.
It's key points conform to my point of view, which, in a nutshell, is "free your functionality, free your processes".
Sadly, Tom's view at this point only advice package vendors to select a BPM vendor and implement it's technology, and not look further for an open standard solution for process technology.
Back to basics.
In many current software packages (COTS) that you buy, you will get great functionality and in some cases even automated process support (in the form of workflow and possible straight through processing capability).
We have come a long way from packages which stored their data in closed proprietary persistence layers. Now, many vendors agree with the adagium "Set your data free!". Many vendors have taken the step to become database neutral - e.g. you can stick any major database brand underneath these packages - Oracle, SQLserver, DB2 to MySql, etc. Great news for technology consilidation - vendors finally started to understand that customers did not like to keep thousands of databases running, based on many different database technologies (why? think high cost of licenses, knowledge of staff, lack of centralized monitoring, difficulties interfacing between databases, etc).
The next step they have started is publishing their data store structure or (better) have implemented decoupled logical views, so that we can access data in the application, directly through database calls or predefined API's. A good step but not enough...
The next step they have started is publishing their data store structure or (better) have implemented decoupled logical views, so that we can access data in the application, directly through database calls or predefined API's. A good step but not enough...
The next step should be that vendors set their functionality and processes free.
:-) I can also see those nice pieces of functionality, and processes, sitting there, in their jails of closed code. Very sad indeed.
So free them? How? Like this...
Functionality not locked up, only to be used from user interfaces or very limited API's, but free, in the form of services. We can see a number of vendors that have been doing this, or are working hard on it now. It's basically opening up the business logic (and data) in the form of services, the SOA-fication of your functionality stack.
And the same for processes. First from a implementation perspective: What I mean is being able to put in ANY (standard) BPM engine. Away from proprietary process technology, towards standard BPEL engines, from for instance IBM, BEA, Appian, etc.
Again the business case is clear here: one BPM technology. In addition, the process technology many package vendors are offering, is very limited. No process modelling, but XML script hacking. No freely defined BAM functionality, but only what they provide.
Second is from a logical perspective: either being able to startup a process through a standard interface, or being able to simple reuse the process definition from the package, but place it as part of the total process. I still have thinking to do here, because how much influence do I want to give to a package, to determine part of my process view?Where we want to go to, is this: if packages only support parts in the end-to-end processes in our companies, we do not want to interface. We want to create the overall process, and link it to services in all the apps we need for that process.
In the following diagram I show a typical example, that you will often encounter:
(click to enlarge)
From a IT perspective, this leads to:
extra technology, extra maintenance burden.
From a business perspective, if we look at this process from a LEAN point of view (http://www.lean.org/), we see there is waste:
- Extra tasks, that do not add value (starting B, nightly batch, rekey activity)
- Time waste, waiting a night before we can continue to serve this customer
What we want is shown in the following diagram:
A key question that is hidden in Tom's article is this:
Can you, as a package vendor, still offer packages that:
- Are only task based (e.g. no process awareness)?
- Offer limited proprietary process technology?
- Which therefore result in difficult integration and suboptimal business and IT solutions?
I think not...
Sure, one last statement - we are not there yet. A recent IDC study also showed that BPM engines are mainly implemented on a project basis, not from an infrastructural basis. This worries me, because we will end up as customers with many many BPM pockets of technology, each requiring licences, knowledge to support, etc. A great role for Enterprise Architecture to help prevent this.... with some help from package vendors that is...
Tuesday, August 28, 2007
Additional thoughts on BPM architecture
James Taylor from EDM blog (http://www.edmblog.com/weblog/2007/08/process-transfo.html) and Phil Ayres of Improving NAO posted a reaction Improving New Account Opening: Component stack - a simplified architecture for applications - Solving complex business problems with financial services technology to my earlier post on an architecture around BPM.
A number of thoughts:
First on "liked his architecture except that I think the Decision Platform he identifies also needs to be able to support both the CEP and BAM components. After all, determining which process to trigger, which action to take or when to inform someone may be a non-trivial business decision and so require decision management. I also think that a decision platform needs both rules and predictive analytics, as you might expect, and that it needs to be available to all the various components that might need decisions (typically implemented through decision services perhaps as a decision service hub as Neil and I discussed in Smart (Enough) Systems). "
Yes - I agree on the coupling between CEP and Decision platform. Good point - knowing what to correlate when, and what process needs to be (re)started could be a complex task. I can see scenario's where for instance correlated events might (or might not) start up a fraud research process.
It's actually interesting to see that many BPM platforms currently have no to only limited inter-process synchronisation. This could also be a great area where the CEP & decision services could help out. For instance: deciding that a certain running process should wait, until a new event is handled by a new separate process. Think of for instance updating a customers address, while at the same time doing a claim process. You don't want to end up sending stuff to the old address (that just burned down!).
On the link between CEP and BAM I am still a bit puzzled. Possible one could see certain BAM data, that points to a certain event, which needs a decision? Not sure. And maybe only usefull in quite advanced BPM maturity environments (which, unfortunately also is true for the notion of CEP and Decision Services - I am not seeing them yet in client implementations - which will in the future lead to additional maintenance...)
For me a decision platform definitely contains BRM technology. What I am still thinking of is - what if a certain decision "service" actually needs some human knowledge? Does it mean that from the presentation layer we will get some possibility of opening a "decision to-be-taken inbox"? Or will decision services actually lead to a normal task (decide on XYZ and let the process know?".
Another thing I am still wondering is the balance between putting decision logic in components, or centralizing it all in a decision services hub. Sure, centralizing it sounds like a estethic architecture thing to do. But the amount of overhead is large. For every IF statement, a component would have to go centrally. And why? So that we can maintain it easier? A trade-off is needed.
A presentation I saw from the Business Rules Platform in the Netherlands talked about this as well. They proposed a mixed model where a decision service hub/business rules hub could either:
- Work as a central service
- Distribute the logic towards the components.
- Do both, but synchronize the logic
Interesting to see which model will emerge.
Phil proposed "James suggests some enhancements to the architecture, largely to ensure that the Decision Platform can interact with the Complex Event Processing (CEP) and Business Activity Monitoring (BAM) components. I agree with his rationale, and would even take it a step further - every component in an architecture (that isn't pure technology) should be able to interact with every other, ensuring that really advanced business requirements can be considered, offering more and more business value."
Hm. Not sure that I agree. Sure, interoperatability is a great thing. And yes, in my picture, although not shown, most components will be able to contact other services.
However, what we should not forget is, again, the notion of coupling. Providing a integration platform does allow for flexible communication between components, or call it services. However, this does not take away the notion of coupling. When I, as a service A, am using service B, I do know about it! I have knowledge of the provided functionality and data, so that I can use it and add value to it, for a certain requirement. Knowing about it is nice, because then I can make use of it. But it also leads to dependencies - changing B will need carefull consideration.
The more connections, the more "services governance" you need.
In advance one cannot predict the business requirements, so true: you would want an architecture that supports easy connectivity. But the objective of my architecture drawing was also maintainability and manageability. Based on an (okay, not disclosed :-) set of requirements, you would want to find the architecture coupling that is on one hand supporting the requirements and provide flexibility towards change cases, but on the other hand limits the coupling/knowledge. I wanted to create an architecture where most components know eachother on a lean, need-to-know basis.
Gentlemen, thanks for your feedback and reactions welcome!
Regards,
Roeland
A number of thoughts:
First on "liked his architecture except that I think the Decision Platform he identifies also needs to be able to support both the CEP and BAM components. After all, determining which process to trigger, which action to take or when to inform someone may be a non-trivial business decision and so require decision management. I also think that a decision platform needs both rules and predictive analytics, as you might expect, and that it needs to be available to all the various components that might need decisions (typically implemented through decision services perhaps as a decision service hub as Neil and I discussed in Smart (Enough) Systems). "
Yes - I agree on the coupling between CEP and Decision platform. Good point - knowing what to correlate when, and what process needs to be (re)started could be a complex task. I can see scenario's where for instance correlated events might (or might not) start up a fraud research process.
It's actually interesting to see that many BPM platforms currently have no to only limited inter-process synchronisation. This could also be a great area where the CEP & decision services could help out. For instance: deciding that a certain running process should wait, until a new event is handled by a new separate process. Think of for instance updating a customers address, while at the same time doing a claim process. You don't want to end up sending stuff to the old address (that just burned down!).
On the link between CEP and BAM I am still a bit puzzled. Possible one could see certain BAM data, that points to a certain event, which needs a decision? Not sure. And maybe only usefull in quite advanced BPM maturity environments (which, unfortunately also is true for the notion of CEP and Decision Services - I am not seeing them yet in client implementations - which will in the future lead to additional maintenance...)
For me a decision platform definitely contains BRM technology. What I am still thinking of is - what if a certain decision "service" actually needs some human knowledge? Does it mean that from the presentation layer we will get some possibility of opening a "decision to-be-taken inbox"? Or will decision services actually lead to a normal task (decide on XYZ and let the process know?".
Another thing I am still wondering is the balance between putting decision logic in components, or centralizing it all in a decision services hub. Sure, centralizing it sounds like a estethic architecture thing to do. But the amount of overhead is large. For every IF statement, a component would have to go centrally. And why? So that we can maintain it easier? A trade-off is needed.
A presentation I saw from the Business Rules Platform in the Netherlands talked about this as well. They proposed a mixed model where a decision service hub/business rules hub could either:
- Work as a central service
- Distribute the logic towards the components.
- Do both, but synchronize the logic
Interesting to see which model will emerge.
Phil proposed "James suggests some enhancements to the architecture, largely to ensure that the Decision Platform can interact with the Complex Event Processing (CEP) and Business Activity Monitoring (BAM) components. I agree with his rationale, and would even take it a step further - every component in an architecture (that isn't pure technology) should be able to interact with every other, ensuring that really advanced business requirements can be considered, offering more and more business value."
Hm. Not sure that I agree. Sure, interoperatability is a great thing. And yes, in my picture, although not shown, most components will be able to contact other services.
However, what we should not forget is, again, the notion of coupling. Providing a integration platform does allow for flexible communication between components, or call it services. However, this does not take away the notion of coupling. When I, as a service A, am using service B, I do know about it! I have knowledge of the provided functionality and data, so that I can use it and add value to it, for a certain requirement. Knowing about it is nice, because then I can make use of it. But it also leads to dependencies - changing B will need carefull consideration.
The more connections, the more "services governance" you need.
In advance one cannot predict the business requirements, so true: you would want an architecture that supports easy connectivity. But the objective of my architecture drawing was also maintainability and manageability. Based on an (okay, not disclosed :-) set of requirements, you would want to find the architecture coupling that is on one hand supporting the requirements and provide flexibility towards change cases, but on the other hand limits the coupling/knowledge. I wanted to create an architecture where most components know eachother on a lean, need-to-know basis.
Gentlemen, thanks for your feedback and reactions welcome!
Regards,
Roeland
Sunday, August 26, 2007
BPM Suite as a component in a logical architecture
In a number of assignments in the past, I was involved in defining a high level logical component architecture for organizations handling administrative processes. My thinking around this continuously develops (learning from mistakes ;-), so I decided to take a snapshot.
I am always interested in additional views....and still puzzling with roles of each component (for instance: who will have the knowledge to show a certain taskscreen, interfacing with the core admin platform, based on a certain task from a certain process? And what will trigger the output management platform - an event or a service call from the BPM engine)
Here is a diagram of the logical architecture:
Three notes:
- Components are rectangulars
- Arrows are showing dependence (e.g. knowledge of), not information flow.
- All arrows would use the integration component (call it ESB or SOA grid or message broker, or someone picking up the phone...)
A breakdown, by component:
Input management - component (consisting of people, technology, processes) responsible for receiving messages from outside the scope of this system: external parties, such as clients, other service providers etc. This would be multi-channel, e.g. able to serve incoming documents by regular mail, phone, email, internet, etc.
CEP - Complex Event Processing platform - component that is able to detect and interprete an incoming message (or other internal event), and, using business rules, translate it to an action. Most times this either means triggering a BPM engine to start a new process, or, in the event of a correlation, to restart/continue an existing process. Of course, the CEP Platform has its own datastore for events (logging) and provide services to ask what happened when, and why did it result in a certain action.
BAM Solution - used for process monitoring. Is able to analyse data in the BPM engine, and present it. Could have all types of additional services, around KPI calculation, SLA monitoring and alerts.
I am always interested in additional views....and still puzzling with roles of each component (for instance: who will have the knowledge to show a certain taskscreen, interfacing with the core admin platform, based on a certain task from a certain process? And what will trigger the output management platform - an event or a service call from the BPM engine)
Here is a diagram of the logical architecture:
Three notes:- Components are rectangulars
- Arrows are showing dependence (e.g. knowledge of), not information flow.
- All arrows would use the integration component (call it ESB or SOA grid or message broker, or someone picking up the phone...)
A breakdown, by component:
Input management - component (consisting of people, technology, processes) responsible for receiving messages from outside the scope of this system: external parties, such as clients, other service providers etc. This would be multi-channel, e.g. able to serve incoming documents by regular mail, phone, email, internet, etc.
CEP - Complex Event Processing platform - component that is able to detect and interprete an incoming message (or other internal event), and, using business rules, translate it to an action. Most times this either means triggering a BPM engine to start a new process, or, in the event of a correlation, to restart/continue an existing process. Of course, the CEP Platform has its own datastore for events (logging) and provide services to ask what happened when, and why did it result in a certain action.
ECM Solution - used for storing all non-structured data, such as emails, documents, faxes, etc. It might also be used to store structured data (incoming XML), to have a complete overview of all incoming messages (incoming records management). It of course contains all index/meta data, and provide services to lookup, view, alter content.
BPM Engine - used for process coordination. Based on process templates, it can execute steps in a proces, including logistic decisions (if's, supported by decision or BRE platform). It can handle automated tasks (e.g. calling services), and it can handle manual tasks (assigning a task to a certain user/usergroup), using information from the workforce management system. It records various process execution statistics for business activity monitoring.
BAM Solution - used for process monitoring. Is able to analyse data in the BPM engine, and present it. Could have all types of additional services, around KPI calculation, SLA monitoring and alerts.
CRM system - used for customer information. Gives a complete oversight of the customer, in terms of profile, current products, processes/services running and done. Decoupled of core administration system (since you could have multiple core admin systems, but only one customer!).
Workforce system - used to supply BPM engine with appropriate information to be able to send tasks to the right user or usergroup (via pull or push). This could range from simple static user/usergroup assignment, to complex rule-driven skill/availability/workload driven assignments.
Core administration platform - this would be the core system for supporting the business of this organization. For instance: account administration for banking, insurance handling, claim handling, stock market transaction system, etc. The system allows functionality and data to be accessed through services, or through context driven screens (triggered by a certain task in a process).
PLM Platform (product life cycle management, or product configurator). A platform which allows for quick adaption of products, supported by the core platform. It supports definition of products, product components, data definitions, rules and calculations.
The Decision Platform has the responsibility to make decisions (or advice them), based on data and rules. This could be fully automated, or a more manual supporting service (think traffic light signaling, for instance, frequently done in credit scoring, where a employee can still decide to accept a loan-request, even though the signal is orange or red).
Security platform. Users, roles, etc. Should be implemented centrally, otherwise you end up with various platforms (usually as part of the other components), with less strength and much more synchronisation effort.
Presentation technology - this component is able to enable various users to interact with the components, and view data. Think of the regular inbox and task windows, document viewing, but also event monitoring, BAM and system admin.
Outputmanagement - this component is responsible for communication outward. Based on events (or a called service from the BPM engine, not sure yet!), it knows, using output rules, what to communicate, to whom, through what channel (paper, email, website, etc). It is able to format a message (email, pdf, Word file, etc) and send it directly or to a printing facility. Of course, output is also send to the ECM solution for storage.
Sunday, August 05, 2007
A process model is not enough...
A number of years ago, I worked for a system integrator, with a a manager that was, well, a bit limited in his overview of the IT best practices.
One day he came up with a great idea. Recently visiting a seminar, he had learned that UML was the answer for everything. And he had a solution to the workfloor's problem: unclear requirements and an expensive process to define them. His solution: use case diagrams. His expectations: we could draw these in 1 hour, and voila - we had our requirements - quickly, clear, and cheap!
It took a while to make him understand that a picture, with some actors, circles and words, where a great idea to structure requirements, but by now means, a use case diagram would cover the requirements in enough detail. So reluctantly he allowed us to continue to analyse and write good software requirement specs.... :-)
It seems that we seem to make the same mistake now with process models. In a number of situations now, a business person delivered a set of process models, and said "well, here are our processes, have fun analyzing them and supporting them with good IT solutions".
Right....
So for once and for all:
When people need to communicate about process - a lot more information is needed.
My checklist:
- The goal of the process, how does the goal relate to the strategic intent of the company
- The criticality of this process
- Compliance, legal and other regulation requirements for this process
- The involved stakeholders (internal, external)
- The power structure/organization structure of these stakeholders
- The trigger that starts the process + through what channels
- The process owner
- The place of this process in the process architecture
- Input to the process
- Output of the process (including exceptions)
- The central object of the process (document, case, transaction, claim, client request, etc) that flows through the steps
- Events during the process, and typical + exceptional sequences and timings
- Data involved through the process
> Logistics (who does what when)
> Execution (data needed to perform task, data produced by task)
> Management (measures)
- Business rules for process:
> Decisions on flow (what task now.... if XXX then do YYY)
> Decisions on assignment (task XXX will be done by YYYYexcept if AAAA then performed by BBBB)
- The critical decisions, rules or patterns associated with it, and required authority to take them
- Succes - when is execution of this process considered a success? What performance criteria?
- CSF's - what is needed to reach this performance? and KPI's
- History of the process - when was it created, how has it performed, when and why has it broken down, what are typical issues, when and why performed it exceptionally well
Well, that's a lot more than a process model. It's the process context that BPM specialists need.
One day he came up with a great idea. Recently visiting a seminar, he had learned that UML was the answer for everything. And he had a solution to the workfloor's problem: unclear requirements and an expensive process to define them. His solution: use case diagrams. His expectations: we could draw these in 1 hour, and voila - we had our requirements - quickly, clear, and cheap!
It took a while to make him understand that a picture, with some actors, circles and words, where a great idea to structure requirements, but by now means, a use case diagram would cover the requirements in enough detail. So reluctantly he allowed us to continue to analyse and write good software requirement specs.... :-)
It seems that we seem to make the same mistake now with process models. In a number of situations now, a business person delivered a set of process models, and said "well, here are our processes, have fun analyzing them and supporting them with good IT solutions".
Right....
So for once and for all:
When people need to communicate about process - a lot more information is needed.
My checklist:
- The goal of the process, how does the goal relate to the strategic intent of the company
- The criticality of this process
- Compliance, legal and other regulation requirements for this process
- The involved stakeholders (internal, external)
- The power structure/organization structure of these stakeholders
- The trigger that starts the process + through what channels
- The process owner
- The place of this process in the process architecture
- Input to the process
- Output of the process (including exceptions)
- The central object of the process (document, case, transaction, claim, client request, etc) that flows through the steps
- Events during the process, and typical + exceptional sequences and timings
- Data involved through the process
> Logistics (who does what when)
> Execution (data needed to perform task, data produced by task)
> Management (measures)
- Business rules for process:
> Decisions on flow (what task now.... if XXX then do YYY)
> Decisions on assignment (task XXX will be done by YYYYexcept if AAAA then performed by BBBB)
- The critical decisions, rules or patterns associated with it, and required authority to take them
- Succes - when is execution of this process considered a success? What performance criteria?
- CSF's - what is needed to reach this performance? and KPI's
- History of the process - when was it created, how has it performed, when and why has it broken down, what are typical issues, when and why performed it exceptionally well
Well, that's a lot more than a process model. It's the process context that BPM specialists need.
Subscribe to:
Posts (Atom)