The trap of IT focused change

Sometimes we forget what the essence is of business change.

Picture an example situation: a business understands that they want to improve their customer relation processes. Several painpoints were found - complaints were lost, addresses of customers incorrect, wrong shipments, whatever. The company understands something must be done, and since data plays a big role, a project is started with a large IT component (and sometimes even a "communication plan" is included).
During the project a lot of effort is spent on the IT part. But, during the project the projectmanager of the IT part starts encountering various issues that he of she can not solve, and does not see as part of the scope. It's the " fluffy" stuff around business change - responsibilities in the to-be organisation, training, the full process, process management of the new CRM department, hiring people, etc etc. The issues get signaled (if we are lucky) to the steering group, but most of them are not picked up and acted upon. In the end the IT project delivers, stuff is installed, and hell appears: confusion, frustrated users, boycots, desperate management, and the easy one to blame: the PM.....

Hm....

In the Netherlands we have a nice Dutch abbreviation that businesses are more and more using:
It's called COPAFITHJ. In essence it's a list of factors that you need to cover as a project/program, when dealing with change. It tries to make us clear that when changing a business IT is only one factor, and that time/effort/money/people will need to be directed to the other factors as well, to make a change succesful.

COPFITHJ stands for:
- Commercial: will our change effect our commercial position (products/services/markets, customer relations)?
- Organisation: will our change effect the structure in which people work, in terms of home base, power, communitity, management?
- People: will the change impact people, in terms of their role, communication, relations, work, responsibilities, required level of expertise/required training & coaching
- Administration, will the change impact required handling and storage of information, including reporting, etc
- Finance, will the change impact current and future budgets, allocation of budgets etc?
- Information, will the change impact the information we require and produce, to perform our work, answer questions and allow the management to steer our work?
- Technology, will the change impact the technology we have in our business? Software, hardware/infrastructure
- "Huisvesting" - Worklocation, will the change effect the psysical location we work at?
- "Juridisch" - "Legal", will the change impact our legal situation.

Is it a great abbreviation? I don't know. But I am glad I can use it to make people aware that change is something bigger than just IT. Too often I still see projects where on the basis of some vague improvement needed by management, some SME's and IT people start running the IT game (requirements, design, build, test), and sometime along the project the "Fluff" start hitting in, gets lost or given too little attention, and we end up with a mess...

The key lesson: approach your business challanges in a holistic way, and do this right from the start until the end. Do not think that IT change will magically result in change in the other dimensions. Focus on the complete business context.

As a sidenote: I am glad that BPM is getting more and more attention. It means that at least the P in the above abbrevation is getting more attention!

My rule of projectmanagement: smaller and deeper

December is for me a time to look back on the year and see what I've encountered and learned.

One observation keeps coming back every year, based on the IT projects I have done so far:

I call it the "It always gets smaller, but deeper" rule on projectmanagement.

This is the essence:



In a typical start of a project, your planned scope will look something like this:

But, after working hard and delivering results, slowly the team starts to realize that finally the scope will turn out like this:

The patterns in play:
- At the start we are optimistic, want to please the customer, maximize the business case value, but are unaware of the complexity of some of the requested features/requirements

- Optimistic planning does not cover all kinds of time-loss on inefficiencies and unplanned ubut needed activities

- During the course of the project, more and more insight develops (the "ouch, does this feature mean this/require this...." moments).

- Time pressure starts growing.... discussions on scope grow. The projectmanager will, depending on sr. level try to hide this or create an early and open discussion with the client

- Negotiations are done. Functionality is prioritized (again) - often succesful, because the client was not aware of the complexity either!

- And stuff is delivered (or another smaller but deeper iteration is started ;-))

Be aware....

Sanity in project processes - back to reality pleassseee

Sorry, cynical mode today....

Ah, I succeeded again... Finding myself suddenly being pulled into a, well, project from Hell.
We all know them. The ingredients are simple:
Mix:
- Not enough time
- No formal proces for requirements, design and build
- No test tools
- No issue tracking procedure and supporting tool
- A strict deathline
- No recording of decisions and knowledge
- Changes in project staffing

And within no time, everybody is running around, trying franticly to discuss, rediscuss, implement, change, discuss, test, discuss, report delays, etc. Working software? I think not.

Key ingredient is, I think, a recurring thing. Let's do a short parabel:

Company A wants a new building as headquarters. They approach a architect and a builder and tells them: well, we want a building, we don't know what we want yet, but why don't you start architecting and building already in parallel, we will test in parallel, so we can move on magicaly date XX-YY-ZZZZ, which we have promised to everyone in the world (and our heads will roll!) without knowing the scope of our demands or any knowledge of timelines for buildings anyway. Make sure it's on time, on budget, and to the right quality!
And what would the architect and builder tell them? Simply: "interesting, but no thanks. Good luck trying to find someone crazy enough to go nuts together with you. And when (or if) you come to your senses, don't hesitate to contact us, so we can tell you how you architect and build buildings for real, with lasting succes and sanity"...

But what do IT companies do? They say - sure. (because O my, it might go to the competitor).

It reminds me of a funny story (happened really!) I heard when working in Sweden.
The king wanted to build this great warship. So, he invited a shipbuilder (dutch, like me :-)), who promised to deliver a 3 deckship. After some years the king visited the king in Danmark, and oh my, saw that his ship had one deck in addition. King back to shipbuilder: I want an extra deck! Shopbuilder, knowing his stuff, tries to scope down, but the king insists, it's the extra deck or the shipbuilder's neck... Easy choice. The ship is delivered, and within about 15minutes of her maidenvoyage it sinks.

My lessons:
Projects should have:
- Realism in scope, cost and time
- Clear processes

Back to work, cleaning up :-)

BPM technology - what happened to two-phase commit?

Between Mid-80's and beginning 90's, when distributed databases and client/server applications began to become mainstream a great technology was introduced: two-phase commit. It basically meant that when a client (or middle tier) wanted to perform a data transaction (create, update, delete) over 2 or more separate databases, it was able to do this in a safe way: either the transation succeeded in all the involved back-end databases, or the complete transaction was rolled back. Various databases started to support this, using the (if I recall correctly) XA standard.

Now I find myself wondering what happened to that great transaction model.
In the current world that we call SOA and BPM technology, suddenly we talk about calling services, hoping everything goes well. And if not, well, there is Compensation. Which, by the way, might fail too. Ouch!

It's amazing - I've seen client/server development tools mature, until they provided a quite stable development and run-time environment, including transactions, debugging, etc.
Then, boom, came the webmodel, and we started all over, with buggy development, debugging ("println("I am at point X and variable Y has value:", y).
And now we have the BPM technology development and run-time environment, and I find myself once again missing all these things that used to be normal....

Anyone knows if there are BPM technology suites that support a two-phase commit like transaction model? Has any of those WS-* standards reinvented this yet? I hope so.
At least than we can think up a new technology to start all over again :-) Mobile?

SOA & BPM? Beware - two-tier solutions might be back...

Lately, I have been seeing a lot of wonderfull features of a BPM technology suite.
Point, click, ah, building applications is so easy. Point to existing web services, pull them in a BPMN flow, create Xforms that talk to web services, no pains!
Really?
Sure, it's easy to create the application, given the current requirements and services.
But what if a service change?

Suddenly I found myself back in early 90's, with 2 tier client server tooling. Oh, what a delight after the green screen software. Widgets, windows, and click click link it to a database table, an SQL query, and tada, the software worked!
Until a requirement changed, we needed to make a small change to the database, and realized 40% of our windows would not work and needed fixing (even though their functionality did not change...).

It's called coupling....

I am assuming that many of these BPM tools have this risk: the risk that we build BPMN and presentation layer stuff, directly connecting to services. Leading to heavy coupling.

The alternative is supported by most tools too, I hope: an intermediate layer, that shields presentation to back-end services. Call it controller or business objects or whatever -
but please use it...

Towards a new TLA: ITPM

Short post, it's late :-)

Interesting article at BPM enterprise about BPM for software development
(link: http://www.bpmenterprise.com/content/c071022a.asp)

What's missing is a key metric: the likelyhood/trackrecord that we deliver on scheduled date.

It's funny - all the effort in various software development methodologies.
In a way it's all (new TLA:) ITPM. IT Process management....

As a BA/BPM specialist, and 15 years of IT development/projectmanagement experience, it often happens that when starting on a new or running project, 20 - 40% of my time is actually spent on process improvement... in the project. E.g. implementing, structuring, refining and improving requirement management process, issue management, testing approach etc. :-).

Is your job also often ITPM?

Towards a new requirements framework

James triggered me again, around Requirements in a BPM context. See http://www.ebizq.net/blogs/decision_management/2007/10/whats_the_difference_between_r.php
I notice in discussions with clients that there is a great deal of confusion about the notion of requirements. I get quite careful when clients start asking "are the requirements done?", "do we have clear requirements"?
Same with terms as "Software requirements" "Specifications" "Business Requirements". When I ask a bit further, the definition, content and context of these terms differ per person, even in the same organisation. If Standish sees 'Clear Requirements" as one of the top-3 CSF's for projects, then it would be quite nice if we all know what we mean by that. How I love clear concepts :-).

Working with a client recently, we discovered together that their change processes were, well, somewhat messy. As change as a process is clearly an element in your BPM strategy: e.g you want to have control over this process as well!, we researched the painpoints in more detail.

One central thema - you guessed it - no shared language around the concept of requirements.

If we don't share the same language on the meta-concepts and the quality criteria around requirements, how would we even start to hope that we understand the requirements in itself?

What we came up with was a high level Requirements Framework, which defines requirement area's. The following diagram is useful:

It defines a framework of types/areas of requirements.

The area's:

1. Business Requirements. As in: requirements you put upon your business.

Strategic Goals & Drivers

These are statements about the intent and plans of the organization. 'Be number 1 in market XYZ, region ABC". "We sell to <target audience> through channels X, Y and Z". "We want to have an operating income of at least XXXX".

Product requirements

All the properties of your products (or services). And the rules of engagement - when are you willing to sell a product? What terms and conditions?

Information requirements

What information do you need when performing your processes? Per process? Per activity? What information do you produce?

How is this information structured? What rules apply (correctness, etc)?

What information is needed to steer your organisation and it's processes?

What information is needed from external stakeholders?

Process requirements

What processes do you need to exist? To serve your customers, deal with suppliers, produce products, steer processes and change processes? What supporting processes?

Which steps in a process? Inputs, outputs, link to goals, etc.

Organisation requirements?

How do you want the organisation to be structured? Which roles? Which authorities? What skill levels?

How many people?

2. Software requirements. As in: requirements you put upon your IT systems?

Functional requirements

WHAT does the application need to be able to do.

Non-functional requirements

Think URPS. Number of users to support, performance, etc.

Architecture requirements

Often put in the same area as non-functionals, but in fact very different. These are requirements from another stakeholder group, and need another area: what (mostly longterm) structures, practices, and technology standards do you require the application to comply with.

A number of notes:

- Each area has it's own value and it's own techniques & best practices to understand, formulate and document the requirements in the area.

- Areas are interrelated. Actually, most interesting requirements you will find on the borders: when performing process A to sell product B with information do we need when in step C. When performing process B, what functionality is required from the IT system?

- In each area you will find hierarchy in terms of detail. In process, for instance, a high level services overview, then drill down in process architecture, process activity flows, etc.

- Functional Requirements is the area that gets most attention. Typical techniques to document requirements in this area are "System Should .....", use cases, business rules documents, etc.

- Note that business rules are types of requirements you will find in more area's. In information: an customer can order multiple products in one order... In process: if insured amount > XYZ manual approval by role ABC is needed. Etc.

- You could call this complete framework "business architecture". I am always a bit carefull with "architecture", since that's an even vaguer concept. My objectives are to have a framework that evryone in a company can understand (and not some ivory tower group).

- The pyramid is also a great tool for knowledge management. If you work somewhere, these are the area's you should understand, to know your own role and added value!

Observation:

I think that methodologies that play a part in changing a business, including software development life cycle methods such as RUP, SDM, etc. should be innovated to methods that understand that requirements can be defined on multiple levels (e.g. system - as in organization and in software system) and that are able to integrate into these broader frameworks. This to prevent the earlier blogged about "task oriented applications" (see http://process-transformation.blogspot.com/2007/10/need-for-new-requirements-approach-and.html).

Requirements vs Process and Rules

James Taylor reacted to a posting about the importance of good Requirements. He sees process as something else than requirements. And he advocates a clear seperation of the concept of Rule and Requirement.
See http://www.ebizq.net/blogs/decision_management/2007/10/requirements_business_process.php

First on the process part vs requirements.
I see a process as a specific concept in requirements. It can be requirements towards a business:
- This is the way I want our people to perform this process (a business requirement)
Or it can form part of the requirements to a system (a software requirement)
- This is the process I want the system to follow when processing claims.
Or a mix of course.
So, is a process definition a software requirement? Sometimes.

Then the part of the rules: again, i don't fully agree.

First of all - YES. When we are trying to understand how an AS-IS situation is working or a TO-BE situation should be working, we can use various concepts in analysis. What processes do you have? What information flow, inputs, output? What do you do in an activity? And, what rules apply for decisions - why do you refuse this order, how do you calculate that price, why do you decide to escalate this?
Rules analysis are a great tool for understanding. And during requirements analysis, I typically seperate them out of the other requirements, simply because it's a separate set of things/concepts.
But are rules requirements or not? I think rules are, for a start, always a business requirement: "when we are doing business, I want our people to follow this rule".
And, in my opinion, when a rule will get automated, we will need a requirement defining the rule. The rule becomes a software requirement: we want to IT system to perform CHECK XYZ, and based on the results do A or B or C.

Now - before I continue, my requirement constraint: a requirement is part of the WHAT definition of a desired system, and does not define the solution.

I have the suspicion that people are separating rules out of the requirements, because they have the solution already in the back of their minds: a rules engine or BRMS.
This links back to my earlier posting on "pick the right technology for your functionality".
Rules can be put in a BRMS. Or programmed in Java. Or in Cobol. Or asked to a user (continue Y/N?). It depends on the requirements...mainly the non-functional ones.

Are the requirements that define this rules highly violatile? Or should they be maintainable by non-programmers? Well, don't put them deeeeep in code in that case.
Are rules quite fixed? Do you only have Java programmers and no money for a BRMS? Does the business delegate all IT rule keeping to the IT people? Go ahead, bury them in code (but do make sure you defined them as a seperate part in your requirements, and make them traceabile in your code...!)

/**=== Check business Rule XYZ , traceability: BR23.22 =======
if (bla.blo(var1, var2, var3)
{.....}
else
{....}

/** end check BR23.22

Conclusion: each technology has it's business case. BPMS and BRMS as well.

The right technology for the right functionality

A number of posts reminded me about a discussion I had some years ago.

Situation: we were introducing advanced output management technology (e.g. a tool + programming language to create applications that were able to receive XML and generate output (PDF's, prints, etc)).
This technology had a number of advantages:
- Quicker time to market
- Powerfull language
- Easier testing (what you see is what you get programming environment)
- More possibilities in layout

The objective was to reduce the code base in the legacy area considerably. In those huge Cobol programs, many were programmed to generate output (line print, so very '70-ies output). Of course, whenever output needed to change - a new product, new rules, a new contract, quote, etc, Cobol programmers would need to start hacking. They did that for years and saw no issue with that. But the business did - waiting for 2 - 3 months of implementation time, a lot of testing, and that 70-ies layout.
The Cobol group had a lot of resistance towards the new technology. One argument was: well, now we can programm the WHOLE application in one language, but with this technology, we will need multiple skills in multiple languages.
Yep. And I explained that we don't use a hammer to crush coffeebeans to make coffee. We use a grinder. And the hammer we use for nails.

The lesson: when requirements ask for a certain solution, pick the right technology for it.

The same discussion I recognize in a number of postings on BPM (Why Java developers hate BPM tools):
http://weblogs.java.net/blog/johnreynolds/archive/2007/10/why_do_java_dev_1.html
http://www.oreillynet.com/onjava/blog/2007/10/do_i_hate_bpm_no_i_hate_bpm_pr.html?CMP=OTC-FP2116136014&ATT=Do+I+hate+BPM+No+I+hate+BPM+Products

The key question is again: what are the requirements, and what is the best technology to solve something. I am sure most BPM featurs you can easily develop in Java. Hell - you can develop a database with Java, but big chance is you don't....

The key advantages with using BPM technology:
- You can change a process quickly
- The process diagram reads (most of the time) quicker than 10 pages of a programming languages, heck - maybe some people on the business side even understand!
- The whole SOA calling is hidden - it helps you to focus on the process and information flow instead of Java and XML
- Process and activities are measured for you, in a standard way.

Case management - The empowered information worker...

Recently I was at the Cordys Cordial (http://www.cordys.com/) day, a day were they presented the new version of their platform.

A funny statement by (I think John Pyke) was: processes are created when something failed. :-) - now, that's a nice start of BPM... And John also nagged towards SAP, which as an abbreviation would stand for Stone Age Processing or Spent All Profit. Hm.

A good presentation was on the new case management features of the Cordys platform.
These features were added to the platform when they discovered (while doing a project with my company) that certain processes are not that straight forward as hoped for. Aka - forget modelling it in BPMN, the diagram would be too complex and add no value. I sometimes wonder how many processes actually are really that structured, but that's another post...

A small summary on the case management features and thoughts behind Cordys:
- Where in production workflow, each process instance has a certain flow, each case will have a unique sequence of activities
- When a case is started (based on some event), certain activities are automatically opened, based on the type of the case and certain event criteria.
- A user is presented with activities, can open them (role bases), execute them, delegate them or create new activities (select or freely define).
- Status of cases and activities can be tracked, measured (monitoring)
- Three main components were presented:
+ Document matching - used to trigger creation of a case, or to be able to link an incoming document to a running case. Remark: strange - I would not have called this a document matching system. Incoming documents (and other information flows) basically define EVENTS. So - it's a CEP or Event Correlation Engine....
+ Case management - the actual presentation towards the user of all case activities, history, data, etc.
+ activity management - the ability to execute, define, delegate, etc activities.
- Another component is the maintenance area - in which case rules and activities can be defined, including roles etc.

They had a nice slide which defined a sliding scale between pure workflow and pure case management.
- 100% production workflow
- production workflow with small cases
- Flow + case
- Case management, with small flows
- Pure case management
I liked it a lot - since I do think you will encounter many processes where things are mixed. For instance a straight through processing unit, which asks for case management when exceptions occur.

Some remarks:
- Unfortunately, the "activities" are NOT integrated in the current BPM engine. E.g. when I create flows in the BPM engine, I can not mark them as optional or mandatory sets of activities in the case management component. Ideally, in the case environment, as a user (or automatic triggers by certain rules) I want to be able to pick: a listed activity, a free activity, a process fragment or a transaction. This also meant that in the current Cordys setup - the BPM inbox and the case management module are not integrated. Ouch!
- Cordys talked a lot about "the ability to measure status of all cases, allowing you to manage the workforce". Well - since Case flow can not be predicted, in my opinion, workforce management will be difficult. Sure I can see what activities are open NOW, but I have no clarity on the expected activities for these cases, and the required resources...
- What I see as a challenge is case data. When you are doing production workflow, most process instance data that you will need, is clear on design time. Each activity and decision needs certain data - and you will need to make sure it's available in the process instance. But what if you are working with cases, where you do not exactly know what information is needed to supply the knowledge worker with enough information to decide on activities and case closure.
- Another item is where to store this data? In the Cordys solution, all data is stored in unstructured documents. But what to do with all data that is stored in backend systems, which users maybe need to decide? Store in the case management system, or call/get every time the case is opened and viewed?

The demo showed a nice presentation layer which showed the case in full context. It allowed a user to understand the trigger, all done activities, notes, etc. A true process workbench.
The main lesson I started realizing, seeing the demo, was that whatever the type of process support a knowledge worker gets with his or her BPM application - the best would be to provide him or her with this full context. Don't make the mistake of underestimating your users and only give them the microscopic task view (aka inbox with single task - do this, no questions asked), without a clue about the full process!
Empowerment and process ownership are things that should be the basis for process workers..

BPM technology consolidation...

Another one bites the dust...
http://www.ebizq.net/news/8554.html

Oracle to buy BEA.

Hm, interesting. Definitely a step towards further consilidation.
How many process engines will Oracle have now???

The need for a new requirements approach - and BPM's role

Ok, it has happened again. Fortunately, it doesn't happen often, but from time to time, I find myself in the unlucky projectworld of "Type 2" , described by the Standish Chaos research as "Over time, Over budget and Less Functionality". Ouch.

Remember the Standish Chaos Report? Also see for instance http://www.projectsmart.co.uk/docs/chaos-report.pdf

It lists succesrates for projects, based on research. And it lists succes and failure factors.
It's findings (So, I am part of the majority with my project :-):
- 16.2% of all projects are Resolution Type 1, or project success: The project is completed on-time and on-budget, with all features and functions as initially specified.
- 52.7% - Resolution Type 2, or project challenged: The project is completed and operational but
over-budget, over the time estimate, and offers fewer features and functions than
originally specified.
- 31.1% - Resolution Type 3, or project impaired: The project is cancelled at some point during the development cycle.

The failure factors for projects ending up as a 2 or 3:

The survey participants were also asked about the factors that cause projects to be challenged.
1. Lack of User Input 12.8%
2. Incomplete Requirements & Specifications 12.3%
3. Changing Requirements & Specifications 11.8%

And yes - in this project, for all kinds of reaons, these items are true.

It wasn't me, though, I was not working on the requirement side ;-). But a repeating lesson I learned: good requirements are essential. And in my opinion that needs process-context awareness.

Remember the old days of applicatie analysis and design? I call it feature oriented requirements engineering: talk to users, find out what's difficult in their job, and think up a set of features to cover their issues.
Example: we do sales, but miss information on our clients and status of orders.
Ah! We prototype a nice screen, build you a nice database with clients, and the order status we get through EAI from your order entry system. Problem solved!
Really? No.
You end up with a so-called data & isolated task oriented application. It's not aware of the process.
To be honest, as IT requirements person, I used to like this. Process was difficult, all those exceptions, terminology, discussions, yucky. Features where clear - a screen here, a database there, a business rule there, data and task oriented.

But in these days of evry growing complexity, task and data oriented requirements analysis will not do anymore. Why? Because it leads to suboptimal solutions, which force people to work AROUND an application, to perform the process. (And ask a random user how many systems he or she typically needs to use to execute a front to back process - usually a lot...)

Premises: EVERY application we use, we use in the context of a certain process. So, by carefully analysing the process, we can get to better requirements, more project succes (standish) and much better solutions.

There is a problem though. Most of the current Requirements frameworks are still in the Data/Task oriented mode. Think of...
- Feature driven development
- XP's User stories
- RUP's Use cases
All these items are focused on isolated functionality, that brings value to a user, in a logical transaction. Process is much more than that! So, how do we capture this.
Sure, some methods call for "business modelling", but most of these elements are, to say the least, not very developed.

BPM would be a great vehicle to improve this.
It asks us from an analysis perspective to:
- SCOPE: Define the scope for a to-be process area and supporting IT solution
- GET CONTEXT: Analyse the processes, front to back, including trigger, input, output, steps, information required in a step, data produced in a step, data flow, actors, KPI's, etc
- DEFINE HIGH LEVEL REQUIREMENTS And then dive deeper: per step, we can start thinking about the required IT support.

From this step, we can revert back to for instance use cases - to define the functionality a user needs to complete a logical process step. (So: a use case is not the same thing as a business process!)

My lesson: BPM is there for many reasons. And one of them is to help you improve your understanding of the business context and get to better requirements. Add it to the more IT driven methods, and you have a better chance of getting to project success.

Came across the new BPM institute magazine, http://www.bpminstitute.org/magazine.html, and found the following diagram:

Some observations:

1. It's first of all a nice overview of the improvement frameworks existing today!
2. For some reason I always get a bit tired of the arrogant tone of voice of the Omega-6 and CEM/CEMM supplier (now Bennugroup). Well, I guess the above diagram tells enough - no trace....
3. I am glad to see Lean high on the list. In my opinion a great framework.
4. It would be nice to see this broken down by industry - for some reasons, in Finance, Lean is only slowly cathing on.
5. It would be nice to see this broken down by "reason/goals for improvement". My suspicion is that we can link goals and methods still quite a bit (visibility - balanced score card, EFQM - Quality, ISO - Compliance, etc)

From organisation chart to process chart

When business analysts come into a new business, they need to quickly understand the structure, people, products and processes in the company.

One of the key things I usually ask for is the organizational chart (aka organigram, organogram).
As a side note: it's amazing how often I either get incomplete or outdated diagrams. I wonder in that case how the heck people in their own organization understand the structure, and can place themselves + link themselves to the goals of the organization. So much for knowledge management, in such a simple form!

In these process-orientation days, I have found organization charts more and more disfunctional. What do you get?

- Powerstructure/reportingstructure
- Functional area's and departments (but often functional departments support multiple processes, and only parts of it)
- Names of people

The thing is - the basic question: "What does this company do" is not easy to answer. For a process-oriented person (like me :-)), that is often difficult - I want to see the chain of actions that lead to results for customers, and their interrelation + the people responsible for the different elements in the chain. Stuff that not easily can extracted from the organization chart.


So - can we come up with a new standard? The processchart, procesogram? Some high-level, I understand this in 1 minute, easy to draw and maintain diagram?

That quickly answers:
- External stakeholders
- Key transactions
- Chains of actions/process areas that support and fullfill transactions
- Key people responsible for parts in the chain, chain accountability

I challenge the reader - do you know something like this??

Some of my thoughts:
- The dutch "Demo" method (http://www.demo.nl/) offers a very concise diagram (organisation construction diagram) - however, this misses the accountability structure.
- In the method of Cordys - there is a business model

Curious if other people have a better diagram technique!

Five areas you need in your process architecture

Whether you are trying to understand processes enterprise wide, or just for a small department, in my opinion, there are always five process areas that you should consider.


Here in diagram:

1. Operational Processes

Processes in this area are considered core to the organization''s goals. They make up the business model of the company and focus on making customers happy, by procuring, producing, selling, marketing and servicing products/services. The value chain of your company.

2. Supporting Processes

Although not core, these processes facilitate all the other process areas and are often of crucial importance. Think Admin, HR, IT support, Finance, Legal, QC/QA.

3. Supplier and Customer processes

From an outside-in perspective, it is very important to understand (and help improve!) the processes of your main outside stakeholders: your suppliers and customers. By understanding their processes, you are able to help improve them, which in the end, will lead to competitive advantage. Model these processes from a supply chain perspective - optimize the chain in total.

4. Management processes

These processes steer the other processes (and cover the plan, check and act of the Demming cycle). They form the process control layer in your business.

5. Change processes
Yes, change is a process to. In modern days, change will occur more and more often. A repeatable framework for change, aka a process, will help you to become more agile: when a change is needed, all people will know their role and responsibility, to enable the business to respond fast. The change process includes detecting the need for change, understanding it, designing it, and implementing it, including all impact on the people and technology in all layers.

The 5 competence-areas for real process transformation

I often use this model to explain but also check if a company is well positioned to achieve succesful process transformations - the 5 competence areas:

1. Process orientation
2. BPM technology
3. Change management
4. Process improvement
5. Subject matter expertise

1. Process orientation

This competence area is what I would call the typical "BPM as a discipline". It asks for people, supported by methods and techniques to see the processes that are embedded in the organization, see the interrelation between them and the alignment with the busines goals, and see opportunities and risks in the as-is process situation. It asks for people that understand how interventions in process, people and technology can help the company to perform better. Concepts, methods and tools that people should be able to understand/use are for instance:

• Business model - linking goals, business services and processes
• Business architecture
• Process architecture
• Business case - TCO of processes, sourcing
• Process models for operational processes, supplier/client linked processes (supply chain understanding), management processes, supporting processes and change processes
• Information architecture - what information concepts are used, and in which step in what process is what information needed to perform the step and what information is delivered
• Performane measurement and process control models - what to measure when (based on CSF's and KPI's) for whom and to control what risk - and the possible steering that management can do when measures ask for action
• Business rules - what rules and knowledge drives what decisions in the processes
• Methods for analysis of processes, and problems-cause analysis and knowledge

2. BPM Technology

IT offers many powerful innovation possibilities. But, people are needed, with the ability to see the possibilities and potential benefits. This requires competences in the technology, including:

• Business process analysis technology, to support people to model / visualize processes
• BPM Suites - automated process coordination engines
• BAM - process intelligence
• WFM - Workflow management, often as part of the BPM suite, offering functionality to support people performing tasks in the context of a process
• Case management - again, functionality to support people performing tasks in processes, where the processes are more difficult to predict and model, and part of the process execution is based on people knowledge and decisions at run-time
• SOA - a way to architect automated processes, based on calling services
• ESB / EAI - Enterprise Service bus (or SOA grids) and other ways to deliver functionality and data needed to perform processes or tasks in a process
• STP - Straight through processing, where the process is coordinated and executed in a totally automated environment with no or very limited (exception handling, sample checks) human involvement
• BRE - Business rules engines (or in EDM terms: platforms for Decision Services)
  

Important to note it that people are needed that can think outside the "tool". E.g. we do need people that understand specific vendor technology (BEA, IBM, ARIS, Oracle, Cordys, etc), but we also need people that see beyond the limitations and design patterns of these tools, and understand the bigger "process model, execute, measure, improve" needs and features of your business.

3. Change management

Process transformation is a change. And change, asking people and their supporting systems to change, can be hard. It's an area often forgotten - many BPM projects are, unfortunately, run either from a technology perspective or an architecture/blue print perspectice. Change is not following a design-implement paradigma, and asks for people oriented interventions - two-way communication, training, coaching, steering, etc. It asks for people with competence in:

• Change management
• People management, development of people
• Project management
• Requirements management (as-is, to-be)
• Design, simulate, evaluate, test, learn

4. Process improvement

I am very happy that process improvement is slowing turning into a art and science on it's own. In fact, improvement can come in a number of impact/scope combinations:

• BPI - Business process improvement : small changes within the current processes, e.g. removing or reshuffling tasks, tuning business rules and tuning work/workforce-assignments
• BPT - Business process transformations: medium to large changes, were processes are analyzed and thoroughly adjusted, including BPM/Case management/STP automation
• BPR - Full scale process redesign, with often a total new look at product-market-channels orientation.

Beware - do not let a simple business analist or architect design process improvements - it will lead to nothing. Process improvement asks for people with skills in:

• Outside in thinking
• Measurement & benchmarking, statistical process control
• Lean
• Six-Sigma
• TOC
• Operations Research and modern Logistical concepts (JIT, Kaizen, Flow, Pull, Agility)
• Simulation

5. Subject Matter

Whenever you start a process transformation programme, make sure you involve the subject matters. The people you need, are people with extensive knowledge of

• Your company's internal structure, network, power grid and culture
• Your current process portfolio
• Your products
• Your customers
• Your suppliers
• Your legal and compliance environment

I hope this list can help you, as a checklist. Try to make sure you covers these skill area's. Whatever you miss - will mean risk to your transformation project. Risk is okay, as long as you are aware of it.

Being goal and result oriented

Today I saw an add, with a sentence that lingered for a while:
"Innovation is an illusion, until you actually do something about it".

I remember, while studying, that I read the excellent book by Goldratt, "The Goal", about the Theory of Constraints. The thing that stuck was actually a more basic concept: whatever you do, be aware how it relates to your overal goal. Something that in modern management sometimes is called Alignment.

I have been involved with many projects and transformations, and I have always tried this. When I was still more involved into IT/software development, my motto was: "In the end, it's about working software for a happy business".
To my surprise in many situations, I had to fight for this motto. The countless meetings, email conversations and phone conferences where people forgot what we were really striving for, it is amazing.

So, ask yourself: what is it that you REALLY try to achieve. What's your goal?
Are you in IT development? Working software, for a happy business!
Are you in IT support? Working software and empowered users, for a happy business
Are you in process improvement? Better processes!
Are you in change? Real change!

All the rest, the blabla, the politics, the countless reports, the discussions, sure - they are often needed. But they are not the goal. So make sure - know your goal, and, whatever you do, ask yourself - is it helping us forward to our goals?

What's you goal? And what are you doing? Aligned?

A personal transformation

It's been a while since I blogged, but that has to do with a large personal transformation. One that will definitely impact all my personal processes :-)

September 16th our first child Hannah Rosa was born. Mother and daughter are doing fine!

The challenge of being a BPM specialist

As a "process therapist" I encounter many processes in trouble.

The most difficult one is the "But we have always done it like this" or "still in the dark" processes.
The trouble with this is to make people understand that there is a whole new world out there, which will soon replace your business, because customers will simply switch. A world of new practices, new technology, and new demands.

I often encounter these processes in organizations with an almost monopolistic nature, where customers are almost forced to accept the service, simply because of the trouble to switch.
Think - utilities companies - water, electricity, national phone company, internet provider, mobile phone company with the 1 year subscription lock-in, your pension provider, the law, mortgage handler, insurance company, car repair dealer.
Sometimes arrogant, mostly naive. "We have always done it like this".

E.g. it's normal that....
- Customers need to wait weeks, sometimes months on requests
- Things get lost, and are found again
- No insight in metrics such as progress, workload, speed
- If customers call, they need to call back later, because it takes a lot of time to findout the current status (and we can't call them)
- Decisions are taken, but no one knows or tracks why, or even evaluates the business case behind the decision
- Processes and channels are there for the company's efficiency, not for the customer "openinghours: 9:00 - 17:00, closed in the weekend", "no, we can not react to email" "please press XYZ, for ..... The waiting time is YY. This call will cost you XX per minute. There are 40 people before you in queue" "yes, you will need to fill out this form, even though we have all your data already"

We all know these types of companies, and those poor processes in there. And the even more poor customers having to deal with this.
I'm often amazed - how can the people working there, have become used to this, and think it's "normal". The boiling frog metaphore comes to mind... (cook them slowly and they won't notice and get used....until it's too late)

What to do? Can BPM be an answer? The answer is a complex and yet simple one:
Yes, but you need leadership from local managers that understand that they have a growing issue and have the courage and committment to really do something about it.

Without this, forget it.

So if, as a BPM specialist, you encounter the "we have always done it like this" type of process, find the leadership, grow it, test it, and let go if it's not there...There are always other companies that do understand the need for process improvement, and your BPM expertise is very welcome there!

SIG Pam articles - open

I came accross this site through Sandy (column2.com).

A recent entree with some great research articles, freely published:
http://www.sigpam.org/2007/08/31/papers-presented-at-the-amcis-2007-minitrack-on-business-process-automation-and-management/

Package Vendors: Free your functionality, free your processes

ebizq published a good whitepaper (
http://www.ebizq.net/to/BIANWP082907W, direct link to http://www.ebizq.net/views/download_raw?metadata_id=8345&what=white_paper) written by Tom Dwyer.

It's key points conform to my point of view, which, in a nutshell, is "free your functionality, free your processes".

Sadly, Tom's view at this point only advice package vendors to select a BPM vendor and implement it's technology, and not look further for an open standard solution for process technology.

Back to basics.

In many current software packages (COTS) that you buy, you will get great functionality and in some cases even automated process support (in the form of workflow and possible straight through processing capability).

We have come a long way from packages which stored their data in closed proprietary persistence layers. Now, many vendors agree with the adagium "Set your data free!". Many vendors have taken the step to become database neutral - e.g. you can stick any major database brand underneath these packages - Oracle, SQLserver, DB2 to MySql, etc. Great news for technology consilidation - vendors finally started to understand that customers did not like to keep thousands of databases running, based on many different database technologies (why? think high cost of licenses, knowledge of staff, lack of centralized monitoring, difficulties interfacing between databases, etc).
The next step they have started is publishing their data store structure or (better) have implemented decoupled logical views, so that we can access data in the application, directly through database calls or predefined API's. A good step but not enough...

The next step should be that vendors set their functionality and processes free.

:-) I can also see those nice pieces of functionality, and processes, sitting there, in their jails of closed code. Very sad indeed.

So free them? How? Like this...

Functionality not locked up, only to be used from user interfaces or very limited API's, but free, in the form of services. We can see a number of vendors that have been doing this, or are working hard on it now. It's basically opening up the business logic (and data) in the form of services, the SOA-fication of your functionality stack.

And the same for processes. First from a implementation perspective: What I mean is being able to put in ANY (standard) BPM engine. Away from proprietary process technology, towards standard BPEL engines, from for instance IBM, BEA, Appian, etc.

Again the business case is clear here: one BPM technology. In addition, the process technology many package vendors are offering, is very limited. No process modelling, but XML script hacking. No freely defined BAM functionality, but only what they provide.

Second is from a logical perspective: either being able to startup a process through a standard interface, or being able to simple reuse the process definition from the package, but place it as part of the total process. I still have thinking to do here, because how much influence do I want to give to a package, to determine part of my process view?

Where we want to go to, is this: if packages only support parts in the end-to-end processes in our companies, we do not want to interface. We want to create the overall process, and link it to services in all the apps we need for that process.



In the following diagram I show a typical example, that you will often encounter:

(click to enlarge)

From a IT perspective, this leads to:
extra technology, extra maintenance burden.


From a business perspective, if we look at this process from a LEAN point of view (http://www.lean.org/), we see there is waste:
- Extra tasks, that do not add value (starting B, nightly batch, rekey activity)
- Time waste, waiting a night before we can continue to serve this customer



What we want is shown in the following diagram:



A key question that is hidden in Tom's article is this:
Can you, as a package vendor, still offer packages that:
- Are only task based (e.g. no process awareness)?
- Offer limited proprietary process technology?
- Which therefore result in difficult integration and suboptimal business and IT solutions?

I think not...

Sure, one last statement - we are not there yet. A recent IDC study also showed that BPM engines are mainly implemented on a project basis, not from an infrastructural basis. This worries me, because we will end up as customers with many many BPM pockets of technology, each requiring licences, knowledge to support, etc. A great role for Enterprise Architecture to help prevent this.... with some help from package vendors that is...

Additional thoughts on BPM architecture

James Taylor from EDM blog (http://www.edmblog.com/weblog/2007/08/process-transfo.html) and Phil Ayres of Improving NAO posted a reaction Improving New Account Opening: Component stack - a simplified architecture for applications - Solving complex business problems with financial services technology to my earlier post on an architecture around BPM.

A number of thoughts:

First on "liked his architecture except that I think the Decision Platform he identifies also needs to be able to support both the CEP and BAM components. After all, determining which process to trigger, which action to take or when to inform someone may be a non-trivial business decision and so require decision management. I also think that a decision platform needs both rules and predictive analytics, as you might expect, and that it needs to be available to all the various components that might need decisions (typically implemented through decision services perhaps as a decision service hub as Neil and I discussed in Smart (Enough) Systems). "

Yes - I agree on the coupling between CEP and Decision platform. Good point - knowing what to correlate when, and what process needs to be (re)started could be a complex task. I can see scenario's where for instance correlated events might (or might not) start up a fraud research process.
It's actually interesting to see that many BPM platforms currently have no to only limited inter-process synchronisation. This could also be a great area where the CEP & decision services could help out. For instance: deciding that a certain running process should wait, until a new event is handled by a new separate process. Think of for instance updating a customers address, while at the same time doing a claim process. You don't want to end up sending stuff to the old address (that just burned down!).

On the link between CEP and BAM I am still a bit puzzled. Possible one could see certain BAM data, that points to a certain event, which needs a decision? Not sure. And maybe only usefull in quite advanced BPM maturity environments (which, unfortunately also is true for the notion of CEP and Decision Services - I am not seeing them yet in client implementations - which will in the future lead to additional maintenance...)

For me a decision platform definitely contains BRM technology. What I am still thinking of is - what if a certain decision "service" actually needs some human knowledge? Does it mean that from the presentation layer we will get some possibility of opening a "decision to-be-taken inbox"? Or will decision services actually lead to a normal task (decide on XYZ and let the process know?".

Another thing I am still wondering is the balance between putting decision logic in components, or centralizing it all in a decision services hub. Sure, centralizing it sounds like a estethic architecture thing to do. But the amount of overhead is large. For every IF statement, a component would have to go centrally. And why? So that we can maintain it easier? A trade-off is needed.
A presentation I saw from the Business Rules Platform in the Netherlands talked about this as well. They proposed a mixed model where a decision service hub/business rules hub could either:
- Work as a central service
- Distribute the logic towards the components.
- Do both, but synchronize the logic
Interesting to see which model will emerge.

Phil proposed "James suggests some enhancements to the architecture, largely to ensure that the Decision Platform can interact with the Complex Event Processing (CEP) and Business Activity Monitoring (BAM) components. I agree with his rationale, and would even take it a step further - every component in an architecture (that isn't pure technology) should be able to interact with every other, ensuring that really advanced business requirements can be considered, offering more and more business value."

Hm. Not sure that I agree. Sure, interoperatability is a great thing. And yes, in my picture, although not shown, most components will be able to contact other services.
However, what we should not forget is, again, the notion of coupling. Providing a integration platform does allow for flexible communication between components, or call it services. However, this does not take away the notion of coupling. When I, as a service A, am using service B, I do know about it! I have knowledge of the provided functionality and data, so that I can use it and add value to it, for a certain requirement. Knowing about it is nice, because then I can make use of it. But it also leads to dependencies - changing B will need carefull consideration.
The more connections, the more "services governance" you need.
In advance one cannot predict the business requirements, so true: you would want an architecture that supports easy connectivity. But the objective of my architecture drawing was also maintainability and manageability. Based on an (okay, not disclosed :-) set of requirements, you would want to find the architecture coupling that is on one hand supporting the requirements and provide flexibility towards change cases, but on the other hand limits the coupling/knowledge. I wanted to create an architecture where most components know eachother on a lean, need-to-know basis.

Gentlemen, thanks for your feedback and reactions welcome!
Regards,
Roeland

BPM Suite as a component in a logical architecture

In a number of assignments in the past, I was involved in defining a high level logical component architecture for organizations handling administrative processes. My thinking around this continuously develops (learning from mistakes ;-), so I decided to take a snapshot.
I am always interested in additional views....and still puzzling with roles of each component (for instance: who will have the knowledge to show a certain taskscreen, interfacing with the core admin platform, based on a certain task from a certain process? And what will trigger the output management platform - an event or a service call from the BPM engine)

Here is a diagram of the logical architecture:
Three notes:
- Components are rectangulars
- Arrows are showing dependence (e.g. knowledge of), not information flow.
- All arrows would use the integration component (call it ESB or SOA grid or message broker, or someone picking up the phone...)

A breakdown, by component:

Input management - component (consisting of people, technology, processes) responsible for receiving messages from outside the scope of this system: external parties, such as clients, other service providers etc. This would be multi-channel, e.g. able to serve incoming documents by regular mail, phone, email, internet, etc.

CEP - Complex Event Processing platform - component that is able to detect and interprete an incoming message (or other internal event), and, using business rules, translate it to an action. Most times this either means triggering a BPM engine to start a new process, or, in the event of a correlation, to restart/continue an existing process. Of course, the CEP Platform has its own datastore for events (logging) and provide services to ask what happened when, and why did it result in a certain action.

ECM Solution - used for storing all non-structured data, such as emails, documents, faxes, etc. It might also be used to store structured data (incoming XML), to have a complete overview of all incoming messages (incoming records management). It of course contains all index/meta data, and provide services to lookup, view, alter content.

BPM Engine - used for process coordination. Based on process templates, it can execute steps in a proces, including logistic decisions (if's, supported by decision or BRE platform). It can handle automated tasks (e.g. calling services), and it can handle manual tasks (assigning a task to a certain user/usergroup), using information from the workforce management system. It records various process execution statistics for business activity monitoring.

BAM Solution - used for process monitoring. Is able to analyse data in the BPM engine, and present it. Could have all types of additional services, around KPI calculation, SLA monitoring and alerts.

CRM system - used for customer information. Gives a complete oversight of the customer, in terms of profile, current products, processes/services running and done. Decoupled of core administration system (since you could have multiple core admin systems, but only one customer!).

Workforce system - used to supply BPM engine with appropriate information to be able to send tasks to the right user or usergroup (via pull or push). This could range from simple static user/usergroup assignment, to complex rule-driven skill/availability/workload driven assignments.

Core administration platform - this would be the core system for supporting the business of this organization. For instance: account administration for banking, insurance handling, claim handling, stock market transaction system, etc. The system allows functionality and data to be accessed through services, or through context driven screens (triggered by a certain task in a process).

PLM Platform (product life cycle management, or product configurator). A platform which allows for quick adaption of products, supported by the core platform. It supports definition of products, product components, data definitions, rules and calculations.

The Decision Platform has the responsibility to make decisions (or advice them), based on data and rules. This could be fully automated, or a more manual supporting service (think traffic light signaling, for instance, frequently done in credit scoring, where a employee can still decide to accept a loan-request, even though the signal is orange or red).

Security platform. Users, roles, etc. Should be implemented centrally, otherwise you end up with various platforms (usually as part of the other components), with less strength and much more synchronisation effort.

Presentation technology - this component is able to enable various users to interact with the components, and view data. Think of the regular inbox and task windows, document viewing, but also event monitoring, BAM and system admin.

Outputmanagement - this component is responsible for communication outward. Based on events (or a called service from the BPM engine, not sure yet!), it knows, using output rules, what to communicate, to whom, through what channel (paper, email, website, etc). It is able to format a message (email, pdf, Word file, etc) and send it directly or to a printing facility. Of course, output is also send to the ECM solution for storage.

A process model is not enough...

A number of years ago, I worked for a system integrator, with a a manager that was, well, a bit limited in his overview of the IT best practices.
One day he came up with a great idea. Recently visiting a seminar, he had learned that UML was the answer for everything. And he had a solution to the workfloor's problem: unclear requirements and an expensive process to define them. His solution: use case diagrams. His expectations: we could draw these in 1 hour, and voila - we had our requirements - quickly, clear, and cheap!
It took a while to make him understand that a picture, with some actors, circles and words, where a great idea to structure requirements, but by now means, a use case diagram would cover the requirements in enough detail. So reluctantly he allowed us to continue to analyse and write good software requirement specs.... :-)

It seems that we seem to make the same mistake now with process models. In a number of situations now, a business person delivered a set of process models, and said "well, here are our processes, have fun analyzing them and supporting them with good IT solutions".

Right....

So for once and for all:
When people need to communicate about process - a lot more information is needed.
My checklist:
- The goal of the process, how does the goal relate to the strategic intent of the company
- The criticality of this process
- Compliance, legal and other regulation requirements for this process
- The involved stakeholders (internal, external)
- The power structure/organization structure of these stakeholders
- The trigger that starts the process + through what channels
- The process owner
- The place of this process in the process architecture
- Input to the process
- Output of the process (including exceptions)
- The central object of the process (document, case, transaction, claim, client request, etc) that flows through the steps
- Events during the process, and typical + exceptional sequences and timings
- Data involved through the process
> Logistics (who does what when)
> Execution (data needed to perform task, data produced by task)
> Management (measures)
- Business rules for process:
> Decisions on flow (what task now.... if XXX then do YYY)
> Decisions on assignment (task XXX will be done by YYYYexcept if AAAA then performed by BBBB)
- The critical decisions, rules or patterns associated with it, and required authority to take them
- Succes - when is execution of this process considered a success? What performance criteria?
- CSF's - what is needed to reach this performance? and KPI's
- History of the process - when was it created, how has it performed, when and why has it broken down, what are typical issues, when and why performed it exceptionally well

Well, that's a lot more than a process model. It's the process context that BPM specialists need.

Starting with BPM-Suites - but what to do with presentation layer

It was nice to see some of my remarks on my previous post (BPM and packages) confirmed in a good state of the BPMS market item on bpm.com:
http://www.bpm.com/BriefingRO.asp?BriefingId=31
"The true upside opportunity for BPM is to evolve into a platform that supports rapid application development, change, and integration with a visual model-centric paradigm that represents a clear advantage over previous application development approaches."

E.g. A BPM-Suite currently is nothing more that a new custom development tool....

I want to address another limitation, that causes confusion.
Many BPM-Suite vendors proudly say: well, use our BPM-Suite, model your processes, link to your services, and voila, you have your SOA enable, process aware, WFM portal..... And, even better: you re-use your legacy.

I doubt it.
On a recent project, a client wanted to use a BPM Solution, integrated with their current core application. BPM vendor comes in, and says: shield your application, by wrapping services around it.

What this means:
About 65 screens, that users are used to, and that work fine, should be replaced by 1. Services, en 2. Well, screens again, but now in a Portal component of the BPM-Suite.

Added value? Good question...
- Rebuilding screens
- Rebuilding screen based business rules/validations

I am a bit puzzled. How can we deliver the power of BPM technology for human centric workflow, in the reality of existing applications....

On my wish list: a BPM vendor that comes in, and says...
Oh sure, we have a tool that analyzes all screens in your application, analyzes all user interactions over the last X years, and then generate a portal with optimized screens. With some minimal work, if needed, you can optimize these screens where needed. Oh, and the generated screens use common techology X, conforming to standards Y.

Then... I see BPM as a new tool, that can take over and integrate various packages and custom built apps, and integrate them from a process perspective.

Key challenge for BPM-technology: integration with packages

I have now seen two major projects that decided not to go for a separate BPM technology layer.
Both had done a large package selection (both in the insurance market). And it turned out (surprise) that these packages had their own, simple but workable, workflow/orchestration technology component.

I participated in both projects during feasibility studies, aiming to analyse the possibility to integrate the package with the company's target BPM platform.

And in both situations, we had to conclude: yes it is possible, but it's complex, risky and leads to a lot of extra effort.

Hm. This is not the way that BPM Technology is going to be the next killer app....

Some facts
1. These package vendors build their own "BPM technology" type components, but with deep integration with their package. Make's sense from a vendor perspective (for the short run), but is a nightmare for integration and enterprise architecture goals for typical companies (that would like to limit different types of technology used for process coordination)
However, their defense is also: there is no consensus on what BPM technology should provide, and what standards it should follow. Result of course is a proprietary process language, and service integration platform.
As a result, it is very difficult to buy the package, but replace the BPM component with your own central platform.
So we end up with another process component, and extensive integration (back to EAI) with other software, that needs to play a role during the end-to-end processes (so we lose overall process views and coordination).

2. User interface issues around packages and human workflow components of your BPM platform is another pain. Typical requirements are:
- Inbox for tasks
- Ability to open, claim, pauze, intermediate save, finish task
- Context aware task windows (e.g. after claiming a task, open the right dialog and windows in the package
- Integrate with document management, showing for a task the related documents.

As there are no clear standards and answers in these area's, clients have the possibility to let the package do this (usually speedy, but limited and proprietary) or try to find a solution with BPM technology and some presentation layer, integrating BPM, package and ECM solution.
Unfortunately, this is for the most part hard coding, deep coupling web client, server and various other technology layers.
So we end with, again, package solution, proprietary presentation layer, including WFM user interface.

Question - Can we solve this?

Well - there will always be package vendors that provide their package with a proprietary BPM component. Just to be able to support client requirements, one stop shopping and to avoid being dependent on third party technology.

But I think from a customer perspective (mainly technology), customers and their IT architecture can require packages to be more open to integration with third-party BPM tools. Definitely a good advice.

And most vendors might not have an issue either - it's sometimes better to focus on the package functionality, and leave general functionality to others (see what happened to identity management/LDAP, database, app server, messaging). BPM technology is the next candidate for commodity infrastructural software.

However, the BPM vendors need to do something too - being able to standardize even more. Sure, the WFMC model is a start, and some standards are there. But solve the above two issues 9with minimal development required and customers can force package vendors to open up to process (as a next step, after opening up for data and services).

And then BPM technology has a chance to go more mainstream. Otherwise we will always view BPM technology as yet another 4GL, in which we can develop visually (but we still call it development...)

What's BPM and what's in your process portfolio?

I had a funny discussion with a collegue on BPM and the contents of your process portfolio.
He, being a bit new to the area of BPM, first asked me to define it. Well, I used an old definition and talked him through the following story line...

"Hm, BPM. Business process management. Can you define it?"
"Well, sure. You know financial management?".
"?? Uh, yeah, of course, that's been around for ages"
"Ok, well, what is it, financial management?"
"Duh. It's taking care of your money".
"Hm. How?"
"Well, all sorts of concepts, techniques, organization, people, sometimes IT, so in total a lot".
"Explain?"
"Well, cash flow, boring accountants, activity based costing, accounting, budget, budget review, balance sheet, general ledger, accounts, and of course people, some activities, and they usually have some of this stuff automated in a financial system. Oh, and they have al kinds of techniques for investments, business cases, ROI, EVA, etc".
"Wow, you really grasp that. So, it's a lot of things, aimed at taking care of your money, right?"
"yes!".
"Why?"
"Well, otherwise the company would go broke. That's why!"
"Ah! Excellent. So... money seems an important resource. What other stuff do you see as important to manage?"
"Uhm, let me think. Well, people (ah! human resource management"), risk (ah! Enterprise risk management), uhm, products! (ah, product life cycle management), oh and of course customers (crm).
"What about process?"
"Hm. yes, those too - if your processes are bad, no money will flow, people will leave, products will not develop, risk will run around, and customers.... well, you won't have them"
"So, there's your BPM, BPM is a whole lot of stuff to take care of your processes".

So, the discussion continued...

"So, can you tell me about this BPM. I understand Financial management, but I guess BPM has other concepts"
"Indeed. In BPM we have....
- A process portfolio
- Proces models (of all important processes in your portfolio)
- Proces owners
- KPI's for some or all processes
- A process life cycle approach (from define/discover, analyse, model, implement, measure to improve, innovate,and phase out)
- All types of practices around these life cycle steps
- People with specific skills that takes care of process modeling, consulting, and helps to improve them
- Automation for process modelling, measuring and sometimes executions"

"What's in this process portfolio?"
"All the processes you have identified, and you feel strong about"
"Can you name some?"
"Sure,
- Operational processes (procure, pay, build, market, sell, deliver, invoice)
- Product lifecycle processes (invent, develop, test, maintain, phase out)
- Management processes (operational, tactical, strategic)
- Supporting processes (IT service management, HR, Finance, Risk & Compliance, etc).
"
"That's it?"
"No, there is one more. It's called your Change process"
"Change process? That's not a process. That's a project!"
"Oh, is it? I bed that everytime you adapt your company to new markets, new regulations, new risks and new chances, you take simular steps".
"Well, yes, but..."
"And that around these so-called projects, you have processes to select yearly what changes to do, what projects to run, to measure, re-evaluate, etc?"
"Hm, yes, but..."
"And that, if you would really succeed at being great in change, all your people should be able to understand and take change steps really good, by repeating this proces?"
"Yes..."
"So, it's a process. A process that delivers change to your company. And you should know it, measure it, improve it, manage it. Like crazy. Because if you don't, all the other stuff doesn't matter".
"He. It's the same as an important concept in Financial management! The only way to really take care of money!"
"Which one?"
"To Invest!"

"you got it :-)!"






-

Requirements - need for new approach

Been doing some consultancy for a client, around the processes dealing with requirements. Yep, that's process management as well - making sure your innovation processes are able to capture requirements and initiate and guide change effectively and efficiently.

My client is struggling with two things:
- A large application, which misses a lot of baseline documentation (what else is new...), and no clear approach how to capture requirements for changes
- An unclear link between strategy/business drivers, business processes and the IT functionality to support the processes

Let's start with the first one - requirements methods for maintenance situations
It's amazing to see how little one can find on requirements management and engineering for existing applications. Sure, tons of stuff for "green-field, let's create a new application, with a great set of requirements". But honestly, how many times does that happen?
Right.
(One article I found was a good start, and I recommend it - www.processimpact.com/articles/reqs_not_green.pdf )

What is needed is a better, common way, of dealing with the requirements around an existing system. And how to define changes - so that business knows that it asked the correct things, designers/developers know what to build, testers know what to test (next to regression testing), and change management knows what to prepare the business with.
Something a bit light and agile (you know how many requirements an average system contains....many!) And.. With a clear process and clear artifacts.

For now, we came up with:
- Identify required change source, stakeholders and business drivers/strategy
- Understand requirements of change, in terms of changed/added/deleted
=> business requirements (e.g. what does the business want to reach)
=> process requirements
=> functional requirements (global and in the form of use cases or a functional decomposition)
=> non-functional requirements
=> information model
=> business rules
- Determine impact and possible solutions. Perform trade-off analysis
- Decide on scope and plan the change.
- Perform usual IT activities (refine, design, implement, test, etc)

Insights welcome!

And now the second one - the missing link between business intent, process, requirements and IT.
Most requirements methods are still focused on what I call "features". A business analyst will talk to users, understand processes and data, and will come up with a solution, expressed in a list of "the system should .... feature X". So, in the end we understand what functions it needs to do, maybe even understand the tasks they relate to, but we lost the link with process and with business drivers/intent.

What I hope, is that requirements methods will evolve, and will start linking to new developments in thinking around business architecture and process architecture.
In the end, we mainly build IT systems to support actors within businesses to perform tasks, that are executed in the context of processes!

So, what I would like is to see the convergence of:
- Business architecture concepts:
- What business intent/strategy do we want to implement?
=> Principles
=> Execution Objects (could be people, IT)
=> Data Objects
- Process architecture
=> What processes are needed to execute and deliver the strategic results?
- Requirements
=> Linked to process steps: this step, we want to automate, and it will require XYZ....
Hm, you might even call it: BPM driven requirements methods!

Again, comments welcome. I am puzzled by the naivitity of current requirements methods.

With BPM technology towards supportive, context-sensitive end-user applications

Picture a complex application, full of user-oriented functionality. Thousands of forms.
An ERP system. Or some custom application for pensions, complex trade, transport, etc.

Now image the time, the training, the effort people need to get used to the user interface. To find their way to the correct windows, menu's, dialogs. Quite a challenge. And imagine users in a situation where every X months new features are rolled out. For a new product, a new service. A new menu item, hidden under three layers of other menu items.

And so, think of the amount of process tasks people are performing daily. And then the amount of time they need to...
- Understand the context of a task
- Remember what functionality to use to perform the task
- Navigate to the correct functionality
- Remember how this all works and how to perform the task...
All this, eevery day, every time a new task needs to be done...

Now let's look at what BPM technology can deliver, together with more smarter/opener, let's say service based applications (and possibly some rulebased layer, or context engine)

A process is started in a BPM engine. A task is delivered to someone's inbox.
The person opens the task, and wants to perform it.
And the task has knowledge. It knows its context! So, why not let the task tell the application?
E.g.
- User opens inbox
- User selects task to execute
- Task informs application (through a context engine) what is needed from the application
- Application shows just the functionality/windows needed for the task
- And if needed, shows detailed instructions, based on the task context

Basically: the context-sensitive application.
No more navigation. The task will lead the way. And the user gets the right information and functionality at her/his fingertips.

Result:
- Less unneeded navigation, getting lost, mouse clicks, etc.
- Faster processing
- Reduced training cost

But, how do you train someone new? Or when a trained person gets a new task, or needs to use new functionality?
Well, the BPMS can tell (BAM) that a certain task has never been performed by a certain user. So the BPM-S or an trainingmodule can warn the user, provide computer based instructions, and guide the user through her/his first time. And if needed, again, and again (e.g. task inbox item should have a button saying "I need some help with this"). That's what I call supportive.

New RSS feed available (full articles)

As a result of Sandy's tip (http://www.column2.com/2007/07/links-for-2007-07-02/), I have now added an RSS feed with full articles. (And gosh, a quick search on Google showed me that there is a large discussion going on, on full or partial publishing. Hm, I vote full... ;-))

And: Other tips and comments always welcome!

Regards,
Roeland

(Just back from a lovely vacation in Italy - when in Umbrie, don't miss http://www.sanpietroinvalle.com/eng/, but don't tell others!)

From interface spaghetti to process coordination layer

Remember the old days of system integration?

Requirement: When we have done task X in application A, we need to send data to application B, so we can continue doing stuff.

Solution: we create a system to system interface.

Purely datadriven. As an example:

- We enter an order in our ordersystem

- During the night, a batchjob will wake up, and send the order data to the invoicing system

- The invoicing system will send an invoice and update general ledger

In diagram:

The results are known:

- Many interfaces, spaghetti

- Applications become coupled, having knowledge about eachother

- Many types of technologies for integration (file/FTP, messaging, API calls)

- A system is responsible for only part of a process (do + send).

- When exceptions arise - who will fix? Extra knowledge of system B in system A.

- Process understanding disappears from business to IT (and from IT to technical batch schedules, that no-one really oversees)

- Difficult to maintain, a change leads to changes in many systems

- Many different ways of reporting status and issues (leading to tons of paper output that no-one dares to read or check...)

And of course, we have tried. We invented EAI. Companies bought ESB's. The result: the same spaghetti, but now standards based.

The lesson: put the process back in the solution. E.g. don't let systems do hand-offs, but let a process coordination engine do this for us....

The solution now looks like this:

Advantages:

- Clear process

- Part of exceptions can be handled by process engine itself

- Other exceptions can be forwarded as manual tasks to business and IT without additional changes in systems

- Systems are decoupled and have no knowledge of each other

- Integrated, process based (in stead of integration based) reporting

- Information flow can be real time (or can still be done in batch)

Realize - this should change your thinking. Don't think interface, think process:

Requirement: we want to correctly sell and invoice our products (process). As a first step, we want to capture order data (step in process). After that, we want to generate and send an invoice (step) and update this correctly in our general ledger.

Solution: define the process, link the process and the solutions for the steps, and use a process coordination layer.

Take-aways from a Microsoft BPM Seminar

Recently I visited a seminar on BPM, organized by Microsoft and some of it's partners from the Microsoft BPM Alliance (in this case Ascentn, Ruleburst, IDS-Scheer and Amberpoint).

Ironic: the seminar was organized at the premises of the owner of Cordys, a direct competitor in the field of BPM-suites.
Another irony: while most Microsoft talks were about 30 - 50 minutes, each vendor was only allowed to speak for 10 minutes. How's that for powerplay. And that was unfortunate, because these partners seemed to understand BPM much better....
A bit scary is the strength of these parties in The Netherlands. When asked, most of these partners had only 1 or 2 implementations in the Netherlands.
Last.. I was surprised by the amount of people (about 100). Not bad for BPM.

On average, the BPM seminar was oriented at a fairly technical audience. Most talks were about SOA, services, engines, etc. E.g. not much attention to BPM as a business discipline. And as result, most talks were based on "if you built it, they will come...". Hm.

The typical SOA/BPM slides came along:
- Services from legacy and other apps
- A process layer to coordinate services and deliver outcome
- Business rules to drive decisions
- BI/BAM for process measurement/reporting
- Multichannel presentation layer for human interaction

And the typical promises on business case:
- Agility, nicely coordinated, loosely coupled...

It was nice that some anti-patterns were discussed:
- BDUF - Big design upfront , were architects design the grand future from an ivory tower
- "Buy an ESB and everything will go smoothly"
- Hacking it from the bottom-up

The suggestion was: Middle-Out, think big, act small, take incremental steps.

Two great presentations were given by David Chappel (http://www.davidchappell.com/blog/)
Some of his key statements:
- SOA has been around for years (DCE, Corba, COM). But now, with standards in place, we can really get it going (SOAP, WSDL).
- RPC standards are in place. However, a good standard for messaging is still missing.
- Reuse is unfortunately NOT the key business case driver for SOA. Did we ever leverage reuse that much? Remember OO? SOA's key driver is Agility. Reuse is limited to technical services, not business services.
- UDDI is a dead-end street. Nobody uses it.
- Business people developing and maintaining their own processes in a tool for execution - it is a myth.
- Business does not care about SOA.
- The future of Java is at risk. There is no clear process execution framework and infrastructure for Java, while Microsoft's Net.3.0 has this built in. In addition, there is no clear solution for business rules (repository and execution), where Microsoft has many.

He also went through the Microsoft technologies relating to BPM.
- Biztalk for system-2-system orchestration, with Workflow Foundation
- Sharepoint for Human processes
- Integration with browser and Outlook
- Ability to create process aware applications based on Office (OBA - Office Business Applications based on MOSS - Microsoft Office Services System)

An interesting thing was the Sharepoint Designer tool. Although only shown shortly, it was interesting to see that they had choosen not to use a traditional procesmodelling tool (activity, arrow, etc), but a rule-based (if XYZ happens, then create action ABC for person D), comparable with your "email rules".

I liked some of the collaborative options, with the integration of Outlook. For instance: define a proces, where a task will automatically schedule a meeting with certain stakeholders, to decide on a certain issue for this process.

But.... a key issue. If you want to define a business process (and rules) using Microsoft technology, and this process covers both System-2-System and Human workflow, you will need to work with 2 tools, and you will not have an integrated procesmodel.....
Microsoft and David did not see this as an issue "as there are not many other vendors that are supplying integrated process tools".

My soapbox...
I think they are wrong... In my opinion, this seriously limits the position of Microsoft in the BPM market. It leads to multiple technologies, competences, maintainance (2 repositories), and issues around BAM (where to measure what?).
They also said this themselves - MS will probably integrate the models.
Strangly, no word about Visio, which is I think a large player in the process modelling area.

Funny is that this lacking also explains partly the partner-strategy. But I wonder what will happen with the Alliance as soon as Microsoft has extended their capabilities.
In my view Microsoft is simply late, tries to compensates with partners, and will get back on track in some time. And some of the partners are so small in terms of ability to execute, lacking succesful implementations, I wonder who will go for these partners.
And as usual, no clear position on BPMN and BPEL (although BPEL seems to be supported end of this year...)

Take-aways from BPM-Forum session

Recently I visited a session of the Dutch BPM-Forum (http://www.bpm-forum.org/), where a dutch Insurance company talked about the implementation of a BPM tool (in this case http://www.cordys.com/) for a claims handling process.

Some key take-aways:

1. In the experience of the insurance company, in the end, the tools were NOT the issue. Most time was spent on understanding processes.
In my view, this is essential. Many vendors claim that their tools can do magic. Well, sure, but you still need a lot of work trying to understand and correctly model the processes. Business analysis is a key skill that you need in a BPM project, so be carefull with BPM projects run as an IT powerplay.

2. They understood that the processes they were trying to model, where too difficult to define in a sequential flow model. There were too many exceptions and possible events with unpredictable timing. They decide to take a Case management approach.


A great pictore from vd Aalst (professor on BPM at Technical University of Eindhoven, The Netherlands) that was used:


Also see: http://is.tm.tue.nl/staff/wvdaalst/workflowcourse/slides

My lesson: start with processes, and be very aware of the complexity. Don't use a modelling tool to understand the processes (because it will make you see reality through the limitations of the tool), but first seek to understand more freely. Then make the call if this is sequential/decision stuff or more complex event-based/collaboration based stuff, and pick the right tool for modelling and execution for that.

3. Essential for the business case, was the ability to use existing functionality. They found ways to abstract and use services provided by the legacy system. This allows them to use current investments, while opening the possibility (through the abstraction) to stepwise replace legacy with new technology, as long as it delivers the same service.

4. A good strategy was the focus on external partners. Part of the strategy, from the start, was the ability to integrate (from a process and BPM-suite) perspective with events and processes from and to other players in the service chain.

5. They kept Content solution and Process solution separate (but did integrate it).

In my opinion important. Currently some businesses invest heavily in ECM solutions and take a document-centric approach to process, implementing process in the ECM tool. In my view, documents are a temporary solution for information exchange. So, base your proces engine on proces, and let it integrate with information providers, including unstructured/ECM suppliers.