Wednesday, January 16, 2019

Systems thinking for process professionals

Systems thinking 
The last months I have been in the luxurious position that I had time to study and learn. One of the subjects that I have taken up is systems thinking. I followed a number of engaging MOOC's, read some great books and read many internet articles (more on this below).

The reason for this choice was that I realized that I recently encountered certain organizational issues and I with my toolbox was not able to really understand, less solve these issues. By accident I followed one MOOC and it felt as if previous notions and new knowledge suddenly fell in place, giving me a better overview of, well, reality and the things I can, and can not do in it.

As a process consultant, of course, I wondered that this systems thinking could bring to process analysis and improvement. I want to share some of my findings in this article.

My struggle with the current definition and views on the concept 'process'
To start - a confession: I have always struggled with the definition of process. It's usually something like "a series of activities, executed by people and machines that convert certain inputs to outputs and produce value". 
When i tried to make phenomenological sense of this, based on what I could observe, I was never able to see the direct truth and value of this view on reality.
The first thing is: activities in the context of a process are primarily a logical concept. Sure, you can observe people and machines doing things, but that's not necessarily an activity in a process. We only start calling people and machines doing things a process, when these activities can be seen repeatable: when there is a pattern.
In addition we only call this a process, if there are patterns that show that there are relations between these people/machines when doing work, in terms of some causal link that causes that when a certain event occurs (input arrives or a person/machine has finished an activity), there is some trigger that causes that the same or another/person or machine to start doing other things.

Towards a new notion of the concept 'Process'
From that previous observations I define a process as "the behavior of people and machines, in which clear patterns exist in the execution of activities and the relation between these executions and that converts certain inputs to certain outputs and produces value"
This definition directly leads to a number of better perspectives on process analysis, process change and process management/ownership.
1. If you want to understand a process, you need to  first find the patterns of behavior of people and machines - what do they do, and how are these patterns related, to understand how input is converted to output and how this produces value for whom
2. If you want to change a process, it means that you have to find a way to influence the behavior of people and machines. It means that change management is an essential skill set
3. We often talk about process as 'things'. 'This organization consists of the following processes'. 'I am the manager of the following processes, .....' I am the process owner'. But...Is an organization a set of behavior-patterns? Can you own patterns of behavior of people/machines? What do you own in that case? The people/machines themselves? The capability? The process descriptions (e.g. the descriptions of the patterns of behavior as is or as wanted)? The power (or illusion) that you can influence or even dictate the required behavior?
You see, when you use my proposed definition of process, things became tricky. You will suddenly realize that process as a concept in the classical sense is an abstraction of a fairly unclear aspect of a much more complex system of people, machines and interrelations. And that a new, system-view based definition of process can help people to deeper understand what processes really are, how they can be analysed and what it takes to improve the socio-technical systems that perform the behavior.

A very short introduction to systems thinking
A system is defined as: A set of elements or parts that is coherently organized and interconnected in a pattern or structure that produces a characteristic set of behaviors, often classified as its “function” or “purpose" (Donnela Meadows, page 188)

Some aspects:
- Components can be things, people, machines, ideas, policies/laws, etc)
- Interrelations are ways that parts influence each other - through certain flows, such as chemical, electricity, material, money, information, power-influence
- The behavior of each part has an effect on the behavior of the whole. The system has properties that are created by the parts and their interrelations. Properties of the whole system can often not be linked to properties of parts.
- Systems often behave in dynamic and non-linear ways, due to feedback loops. Systems as a whole can even develop unexpected behaviors (processes of emergence)

Systems thinking in essence is:
- Learning to zoom out and see the greater whole (holistic/synthesis and abstraction)
- Identifying useful boundaries (what is the system, what is outside the system)
- Understanding the function and relations that the system has with it's environment (what are inputs/outputs in terms of energy, money, material, power-influence, information)
- See the parts and interrelations of the system itself (what is there and how do these things influence each other) (analysis)

Some great insights I found:
- Every system as a deeper purpose that drives the behavior of the system. But this is not the same as the stated mission/vision/strategy by management. It can only be deducted out of the behavior of the system as a whole. Practical example: nobody wants people to be homeless, yet our society produces these outcomes...
- Every system simply works towards its purpose and produces certain outcomes. There is no right or wrong, there is nothing broken. Whether we like these outcomes, want them, is another question. If not, we will need to understand and intervene in the system.
- The reason that certain (unwanted) outcomes are produced is often not a linear chain of causal effects (e.g. a why-why tree) but a more complex interrelated set of causes that have loops in them. Most issues in our world are caused by deeper issues - wicked problems. Practical example: we build roads to reduce traffic congestion, but this leads to more congestion, and other 'side effects', such as polution.
- Our ability to create the perfect system is very limited. In many cases, due to unforeseen dynamics and our limited understanding of the system's inner working, our interventions may not lead to a system that produces the required outcomes, or might even make things worse! If we intervene and get unwanted side-effects, it's basically a sign that we didn't understand the system and perhaps even more, the limits of our mental model.
- Systems are always in flux, so our views on "as is situation" and "to be situation" is an illusion. Our "as is" are essentially vague moving pictures. And our "to be" is never really done - we are never done helping systems to keep producing the right outcomes and the intervention that worked yesterday might not work today. The lesson: unlock change, instead of imposing it, and learn & stay flexible.
- Systems don't exist. What we define as a certain system (e.g. what is in and out of the scope, what aspects/parts do we identify) is purely in the eye of the beholder/purpose of our research. In essence there is no system in the universe - all is connected and we are the ones that draw borders.
- We use models of systems to understand, but maybe even more to learn -together- other ways to see the system and help us challenge our mental models.

Systems Thinking has come with great analysis tools. Some techniques (and ways of views):
- Force field analysis, helping to define enablers and inhibitors for helping the system produce wanted outcomes
- Stocks and flow diagrams (for process specialists relatively easy cake)
Causal (loop) diagrams (think 5Why / Root Cause times 10: issues that reinforce other issues that create the first issues...)

Learning to look from a systems thinking perspective
If you learn to look through the systems lens to processes, I predict that....
- You will develop a deeper understanding of the system that 'performs the process'
      - You see the parts
      - You see the interconnections
      - You see the patterns of behavior and (hopefully) the purpose through that
- You will be able to intervene and influence a much more subtle and powerful ways, identifying leverage points where, with limited effort strong change results can be reached
There is even a maturity model for understanding your ability to see from a systems perspective! And you might even want to check out your systems intelligence with a test here.
Key lesson: the more people participate in systems analysis, the better understanding...

One very useful model for understanding (systemic) reality with a short story
I found the following model very helpful in looking at reality:

(A good explanation can be found here and here)

An example: suppose you are called by a manager. They had a major issue with a client (quality of a product was not good). You come in, and find out that somewhere an error was made. You see this as an unfortunate event and suggest to add a quality inspection.
Sometime later you get called again. Many clients are now angry - there are too many delays. You can not see these as single events any longer, and research the patterns. It turns out that the error is made often, that the added inspection increases lead time, and that correction of the error also increases lead time. You perform a root cause analysis (what deeper patterns cause the error) and find that new employees make the error the most. So you introduce training for new people. You monitor for a week the effects, and great - the errors disappear.
Some weeks later, the manager calls again. He sees costs growing and still delays. You analyse the cost situation and find out that many people are taking your proposed training, which lowers productivity and increases cost. In addition, you realize that the quality inspection lost its purpose. With great effort you find a way to fully improve the step that used to cause the errors. Now the error is prevented! Proudly you present the results and indeed - the error disappeared and delays seem to have vanished.
Two years later, the company goes broke. Clients have repeatedly complained on next upcoming issues. You wonder what happened - and find out quickly: the manager's mindset saw people as simple replaceable resources. Morale was low, employee turn over was high and the employees mindset was that they never saw process problems as their issue, someone else would fix it...
Conclusion: Don't focus on events and simple if-then's, but learn to see the patterns, structures and mindsets.

 I hope you enjoyed the article!

Some great resources on Systems Thinking
For me, my Systems Thinking journey has lead to many new insights and questions. I hope that with this mindset and toolbox I can address complex issues and intervene more successfully. What about you?

- Acumen's Systems Practice (highly recommended)
- Futurelearn's System Thinking and Complexity
- Open University's Systems Thinking and Practice

- Donella Meadow's Thinking in Systems
- Fritjof Capra's The Systems View of Life
- John D. Sterman's Business Dynamics - Systems Thinking and Modeling for a complex world

On the internet, including Youtube, there is ton's of material to be found!

Saturday, March 19, 2016

Towards personalized processes (for employees)

My assumption is that if we dive deeper in human centered process design (as described in my previous post), and research people that work in processes, we will find similar findings as user-centered designers do when they research customers:
> People are different in many ways, for instance in terms of needs, preferences, skills and behavior

Product and service delivery companies have struggled with this, and the big shift from industrialized homogeneous delivery to modern days have been personalisation and mass customization. You might argue on the extent that companies have been able to really reach this, but I think that this shift towards personalized services will continue. And also that the human centered approach to design of products and services will further grow - to meet personal individual needs of people in better ways.

Now, that's interesting. As BPM-consultants we typically define processes aimed at standardization. We want processes to be simple, consistent, the same for all (in the same role & swimlane of our nifty process models). Some process designers do not even have a clear picture of the employees, and will design jobs from a very generic perspective.

But if we want to help create work environments which engage, motivate and even help employees thrive, we need to take the individuality of employees into consideration, even put in central.
My assumption is that if we look deep into humans in the role of employee, we will find:
1. Generic human traits and needs that apply to all
2. Traits and needs for particular groups of people (persona's)
3. Pure individual traits and needs (the individual employees in your process)

As process designers, if we strive to fully engage each and everyone employee in process execution, we will need new methods and approaches to design processes that can address all these 3 levels.
In my opinion, this could be the dawn of the personalized process: processes that are able to tune in to each employee's unique traits and needs, and create a working environment that triggers that person's motivation, need for meaning, engagement and perhaps happiness.

Towards human centered process design

We spent a considerable part of our energy and life-time at work. That makes it painful to see the typical statistics on employee engagement. Some organizations even talk about a worldwide employee engagement crisis . And I think change is needed. As BPM/Lean consultants I think we should research this, and perhaps also discover that we are part of the problem.

A simple question you might want to ask yourself: when you design processes, and would need to pick the key 2 or 3 goals you or the organization wants to reach, which ones would win from the following list:
- Cost reduction
- Value for the customer
- Employee motivation
- Quality improvemen
- Improved Customer Experience
- Efficiency
- Employee engagement
- Agility
- Reach strategy alignment
- Increase Employee happiness
- Reduced cycle time

And? I bet that most people picked cost, efficiency, cycletime, perhaps quality, and perhaps customer experience. The reasons? I don't know, but my assumption is that we never really developed the vocabulary and methods on designing processes for employees. And that's maybe also because our BPM-assignments never included employee experience. Sure, as any Lean expert will tell you: the employees ARE the process, so you need to involve them, empower them and support them to create a improvement culture. But that's just one part of employee engagement.

We live in a time where using design thinking, human centered design we learn more and more how to engage people with products and services. We learn about behavioral economics to influence behavior. And we research how to help people thrive with findings from positive psychology.
And how much of these methods and findings have entered the field of BPM? Lean? My assumption, based on keeping on track on most literature and websites is: surprisingly little.

My key question is this: is it possible, and if yes, in what way can we, process designers, help create working contexts that are desirable for employees, that engage, perhaps motivate or even increase happiness?

I think this question is very relevant. Mainly because I think it is a waste of precious lifetime to work in disengaging work contexts. It kills spirit and creates waves of negativity in our societies. And of course there are business stakes as well: low retention, not to speak of the consequences on customer experience and thus, profit.

That's why I have decided to start a research project into 'Human centered process design', focused on finding ways to help design processes and work environments that help people thrive. Tips and help very welcome! I assume (as always standing on the shoulders of giants) that must be current and earlier researchers on this question. My hope: to contribute to BPM & Lean with new vocabulary, inspiration and methods to help our world forward.

Thursday, April 16, 2015

Six lessons on the art of change

Oh, irony, how many times I meet process consultants (including me) that often forget that changing processes is changing humans and their behavior. That think that creating a 'process design' document is a major step, forgetting that the influential power of a process diagram is about as strong as sending an LinkedIn invite to someone you don't know at all....

So, just for me, not to forget, six key lessons on the art of change (and no, I won't call it change management).

Be part of the change
So doing a number of interviews, and then writing a report is valuable? No. That's safe distance consulting. Step inside the system, influence and be influenced. Be prepared to not know AND strong enough to dare to take a stand. Get out there, organize workshops, discussions, and other interactions, and try to make them meaningful for the people involved: meaningful interactions and conversations that touch both you and other stakeholders. Out of comfort zone? Yes.

Get out with incomplete analysis and ideas
So you want to lock yourself up to create the perfect analysis, the perfect design, the perfect to-be? No. Think back when you received a 99% complete report, and someone asking you if you agree. Most of us either try to ignore it, or just give up at page 20/143 and say 'Sure, good ideas, ...I guess' and simply get on with our lives hoping that someone else will pick up the actions in the report. The key is: get out as consultant with your analysis and ideas when they are at 40%, 50%. It opens up a valuable feedback channel, because people get triggered by the openings, the vagueness, the incompleteness of your ideas. It stirs feedback, and most important: if people have the opportunity to add value to ideas, it becomes their ideas: ownership and support included.

Know the value of interventions
As a consultant, there are many ways to analyse and influence people and their behavior. First of all: try to fill your toolbox and experiment/learn what works when. If you only have a hammer and nail, your reflex will be to hammer on. Second: understand the power of an intervention. From interview, to workshop. From online survey to focus group. From newsletter to report. From think tank to management summit. What does it do with who.

Don't confuse means and goals
Yes, I used to think that a brown paper session was successful if I had enough stickies, with a reasonably clear process, and enough pain points and perhaps even some suggestions. The model was my goal. But to transform processes, to change behavior of people in groups, such workshop results are merely a side effect. The true value lies in bringing people together, having meaningful conversations, build trust and cohesion, influence each other, create awareness, etc.

Ask, listen, observe
I sometime meet great change specialists. Wow. These people see and feel in X dimensions. Are at meetings and read currents, moods, old pain, policies, stakes. And dare to ask, listen, reflect. The better you can be there, be mindful, really see, hear, feel, separate observations and interpretations, and build up intuition and cause-effect knowledge, the better you know what's going on, and what's needed.

Every change is a personal change - and goes through phases
A successful process transformation is done, if all process participants have changed behavior - one at a time. So 'process transformation' is a confusing word. It's behavior change of a group of people, of individuals (and often some IT-systems as well). It's essential to remember that change in time goes through phases. From awareness (is there a problem with my current behavior?) to desire (all right, perhaps it's good to change) to knowledge (so, what is it that I should do') to ability (ah, so this is what I should and now can do) to reinforcement ('and with these interventions I help me and others not to laps back to old behavior patterns when in stress). The secret: each phase requires different interventions. Don't send someone to training or give a 500 page manual of the To-Be process, if the person is still struggling with desire and motivation.

Monday, December 22, 2014

Design, UX, Flow, o dear.

I have gotten myself into a discussion @ twitter, and found myself with the following question, asked by Charles Webster MD: " what is the relationship between interaction design, user experience, workflow & process-aware technology"

It's a broad question, but I like it, because it forces me to create a coherent view on a number of items I have closely been working on in the last years.

A challenge is that by reading Charles' blog, I noticed that certain concepts we use might have different names and definitions. And, that we had another discussion going through this, dealing with the limitations of 'workflow' versus case management.
For starters, I encountered the definition 'Workflow': Workflow is what actually happens when work is done. It is a series of steps, or tasks, that consume resources (money, time, effort, attention), and achieve one or more goals (see here). In my eyes this is a perfect definition for 'Business Process'. Perhaps due to my BPM-technology background, I typically associate workflow by a process that has been implemented and can be run in a BPM-Suite.

And a later definition of workflow came along by a reaction to the discussion by InformIt:

First Sketches of an App: Planning the Design of a Mobile Application

It did not provide a clear definition, but deriving it from the text should give something like: a workflow is a series of steps a user follows when trying do a certain job, while using an IT-application. Hmmm.

Ouch. Let's get back to the basics of the question, and let's start with the end in mind: a situation in which we have found a way to support, let's say medical staff, with a great process aware information system. A system that:
1. Provides the right user experience: people using it can reach their goals (primarily diagnosing, monitoring and curing patients) in an effective, efficient and pleasant way, in the context and conditions they do their activities
2. The system is aware of pre-defined medical protocols and best practices, yet has the flexibility to deal with unforeseen events (symptoms), and can coordinate people's work, based on these protocols and practices. It means that the system can trigger people to perform certain activities 
3. While doing their activities, people can enter, and find/access appropriate information, that is related to all the work that is going on. Outside of the activity perspective, people can access the status of a patient (and the status of all ongoing and completed 'workflows').

(Note that I take a strictly medical staff perspective, just to limit the scope of this blog-item. If I would include a patient (and family/supporting network) perspective, I would also need to dive in to patient journeys, service design and patient experience. Another time. Also note that I have not included worker experience design and process redesign, something I wrote about here, for now, but limit it to the design of the IT-system)

Let's assume that the system is based on a modern BPM-Suite, with appropriate flexibility (e.g. advanced case management/dynamic process support). This is essential, as cure and care processes are often not completely predictable in terms of possible events, routes and outcomes. For more information, see this blog item or read the whitepaper on case management that I wrote together with a colleague when working at Capgemini (and that was referenced by Gartner), and where we use the 'navigator'  versus workflow metaphor.

Using a BPM-suite, this means that a large part of the software design is fixed (the building blocks, such as the portal, modelling facility, process execution engine, human activity coordination - inbox, form handlers for activity screens, document management module for documents, etc).

Now let's try to answer: how were we able to design the IT-system?
A number of actions needed to be taken. I am trying to take these apart, from two perspectives: the subject matter expertise, and the design expertise. Often, in reality, the actions are done in parallel, with the full team.

Action Deliverable Context Expertise Subject Matter Expertise
1. Understand the medical staff and their the working context Empathy maps persona's, contexts of use UX Medical Staff
2. Understand the activities that needs to be done, when, where, why. Understand the processes, triggers, exceptions. Process maps Process analysis Medical staff
3. Understand information usage - for each activity,
determine information needs, information produced. Understand information requirements
from compliancy and management needs.
Objects/attributes Information analysis Medical Staff
4. Design the 'workflows': the series of activities that the system should support.  Executable Process models Process designer Medical Staff
5. Design the user interaction in general: how do people work with the system in general, in terms of access to patient information, inbox of work-items, search, etc., using the contexts of use (mobile workers, administration behind desk, operating room, etc)  General UX concept UX Medical Staff
6. Design the activity forms: when performing a certain activity, how should information be presented, accessed, entered.   UX design of activity UX Medical Staff
5. Design the technical interactions (activities or process fragments that need to interface to technical equipment)  Technical interfaces Integration expertise Technical staff

The creation of these deliverables is often done using the BPM/ACM-technology (in combination with other tools - such as plain Office, but also Business Rules models, Data modelling tools, etc).
With the emergence of 'Business Technology' we see the trend of 'The specification/model IS the application' as I wrote about early here.

Key in these actions is to work iterative, and test test test. Testing from multiple perspectives:
- the process/workflow: is the set of activities correct in terms of flow, exceptions, decisions?

- the information perspective: can I work with the right information, record the right results?
- the user experience: can I as a user, in a certain context, perform the activity in an effective, efficient way, and does the system support me in a pleasant, perhaps even motivating way?
- the technical integration
Testing requires different perspectives and associated competencies/skills: a process/workflow perspective, a UX-perspective and a technical perspective. The people possessing these skills will typically organize / facilitate sessions with medical staff, guiding them, monitoring them and so finding ways to improve the system in terms of UX, workflow/data or technical integration. As BPM-Suites and Adaptive Case Management systems allow most parts to be configured refinement cycles can often be done quite quickly (hours to days).

Now, part of the discussion I have with Charles dealt with the notion ' Human Centric Design  is dead', pointing to sources such as here and here.  First of all, I don't disagree with the notion of activity centered design, as my actions above clearly show.
The activity centered design is for me a (welcome!) merging of BPM-expertise and UX-expertise.
The BPM-perspective has always placed activities as the center. What needs to be done when, why and by who. And.. what information comes in, is needed, and goes out. But what we have missed as BPM-specialists is often the UX: really understanding the people using it, the context of use (where? how?), e.g. taking a more psychological perspective, in terms of quality of use. As BPM-specialists we dropped BPM-suite driven applications to users, based on sound process analysis and information analysis, that were logically correct. But often not user-friendly. Too rigid. Too complex.
In my last program (a flexible BPM/Case management solution for a Ministry - with lots of adhoc processes, for 1200+ users) we involved a UX-specialist. The value of this UX-perspective (understanding users, their context, ability to prototype, test, and bringing in other ways of looking at forms-designs) has brought enormous advantages.

My notion of Human Centric Design has been formed by my work and various trainings of IDEO and other organizations (Service Design), see for instance here and here.
I think these HCD-approaches did not evolve from a strict user interface design perspective (as I would position D Norman), but much more from a holistic view on people using artifacts and services to collaborate and reach certain goals. These HCD-approaches have a rich toolkit, in which context-of-use, design for emotion and motivation, journeys/jobs to be done and process/workflow perspectives are all seen as essential elements). So from that notion I would be far from declaring HCD as harmful. Activity Centered design is essential part of HCD.
In the end, I think Charles and I are in agreement.

To conclude: a small dive into the limitations of many BPM-tools. In the whitepaper (with the model I designed, positioning various types of processes and associated forms of coordinations), I showed that there are types of processes that are more structured and predictable (doing your expenses) versus more adhoc/less predictable processes. Actually you could see a scale:
1 Structured process, where the full process model is known and all execution paths are known at design time
2 Less predictable, but the activities can be predicted/predefined, just not the sequence/possible paths
3 Even less predictable, the process might even require activity not predefined
The key issue with certain workflow technology or BPM-suites is that they can only support type 1 processes. I predict that in health care many type 2 and 3 processes exist. So to integrate a BPM-engine in an EHR, that supports type 1 only, will severely limit the EHR's value! And however great the process or UX design, it won't fix the key issue: users will be locked in ' hard'  workflows that will not support all the possible events and paths.

Tuesday, August 05, 2014

O dear, what will we do with the data?

In my previous post, I wrote about the possible scenario's to deal with integration with existing data-centric applications. In this blog-post, I want to cover situations, in which we want to digitize the process (with a BPMS/ACMS) and there is data, but no existing data-centric applications to store it.

Let's first investigate the typical data one encounters when analyzing a process that needs to be digitized...

Type Description
Customer data Names, addresses, contact persons, etc.
Product/transaction data Type of service or product, certain service / product specifications, pricing, other conditions and agreements
Decisions Specific decisions (what what the decision, who took it, when, and why was this decision taken/on what grounds)
Process and task performance data Durations of process and tasks (or even effort and costs associated with it)
Audit trails What was done, when, why and by who
Other process linked data Comments/communication between process participants, certain checklists/QC outcomes

All this data is crucial for the stakeholders around a process:
- The customer wants their service or product first time right, on time delivered
- The process participants need it to know what to do, and do it correctly
- Process management wants to manage the flow, work-distribution, work-load and compliance
- Responsible managers want to be able to report outcomes, performance and compliance
- Accountants / Audit / QA-people want to understand what happened, and see if it was compliant to regulations
- Future generations (Public sector mainly) want to historically research what happened

When we digitize a process, this data is, in the as-is situation, usually scattered in various forms:
- In unstructured forms, on paper, in folders, or in e-mails
- In semi-structured form, in (often) Excel files

And the user requirements are typically:
- We don't want Excel anymore, the new BPMS/ACMS solution should help us to digitize the process
- We want to store and report on all of this data somehow

Many times, I encounter organizations that are mostly paper-based, sometimes with low-volumes. So when implementing a BPMS/ACMS solution, we do not have the luxury of a CRM-system, and ERP-system (or other Product administration), a Datawarehouse. Sure, organizations should have these systems in place, but often, introducing these technologies are not in our scope. Then the inevitable question comes: O dear, what will we do with the data? Ouch.

From a solution architecture point of view, we have a number of paths.

1. Let parts of the data behind in existing office automation
- Simple

- Separate files that need to be updated manually (more effort)
- If the data is also stored in other places: risk of becoming out of sync
- Not addressing client needs
- Hard to query

2. Store parts of the data in office automation and integrate with it 
- Data is updated, no need for manually updates
- Less risk of becoming out of sync
- Low cost

- Usually quite risky, as business users can access the files and change structures
- Not suitable for large data volumes
- Technically unstable sometimes, and transaction integrity/locking issues
- Hard to query

3. Store parts of the data in structured form in the process instance
This scenario is tricky. My design rule is: keep the process instance very light, as:
- Performance might get bad, when too much data per instance
- Often hard to query on specific items (usually stored as XML/blobs)
- You never know when the data gets cleaned out (you need strict rules on instance archiving)

With very light I mean:
- Customer/transactional data is only temporarily stored in the process instance, but as quick as possible stored in a other structure
- Data that is really process instance specific (a contact person name for this order)
- Only keys to the data (customerid, productid) are stored
- Only data is stored that is needed to execute business rules (preferably retrieved from other structure just in time).

4. Store parts of the data in digital documents (unstructured)
- Simple (you will need a Document Management solution, and often a 'Case' concept)

- No reporting on structured data possible (Text mining too difficult)

5. Store parts of the data in a custom database-structure (linked to a case)
- Clear structured place to store data

- Depending on complexity of data, can grow large and complex in terms of effort and architecture
- If the BPMS needs to support multiple processes, each with their own set of data, complexity can become even more

What I sometimes encounter is that scope explodes, because customers would like access to this data through a non-BPMS user interface: a new data-centric app is born and included in the project. Usually not a nice thing.

6. Store parts of the data in structured form in a generic Case data - solution
In this scenario, we create (of buy) a component in which one can define meta-data schemes per process-class or case type. In each scheme, one can define possible data-elements, . When creating a process instance or case, the BPMS/ACMS allows users to maintain the defined data-elements. 
- Flexible, quick to configure and deploy

- Can be hard to query

7. Store parts of the data in a (small/custom) Datawarehouse
For some types of data, I would recommend to create a small custom set of Star-scheme tables with the relevant dimensions. This allows to query on performance of processes and tasks, by dimensions as process / case type, period, organization unit, etc.  In addition, it would be possible the extent the star-scheme with more transactional data (order volume, price, certain decisions) to allow more complex queries ('In period X  what was the total of orders in Euro, in which we refused to deliver')

Data governance can be a tricky question in BPMS/ACMS projects. It not properly handled it can require lots of effort, effort that we rather would focus on the process and BPMS-advantages. If a project is oversold in terms of BPMS/ACMS promises (and we all encounter this from time to time :-)), it can be hard to manage client's expectations and influence the client to select and implement additional components, such as a CRM-system.
As BPMS-specialist I think we need to be honest to the clients on the risks and limitations of the possible scenario's, and help them to make a sound decision.

Sunday, August 03, 2014

Need for better standard for integration between BPM/ACM and data-driven applications

In many of the projects that I worked in teams with BPM or ACM (Case Management) technology, we encountered the integration-issues with other data-driven systems.

Typically this would be the result of the following factors:
- The users wanted to perform work on a certain activity, based on a work-item trigger (from a BPM or Case Management solution). They liked the 'inbox' concept (of work to do), sometimes in combination with a Case view.
- As part of that activity, data needed to be retrieved, stored, updated in another existing data-driven application
- The existing application had a fine user-interface (a number of pages dealing with that information), and often exposed through that user-interface were many complex business rules.

The possible paths for the Solution Architecture are not optimal:

Path 1. Manual activity 
Just create a work-item that tells the user to go to the application to perform a certain transaction. Present the right data, if possible, that the user needs to enter/update. Ask the user to report the activity ready when done.

- Very simple, low investment/low risk from a project-scope perspective

- No contextual frame (see below)
- Risk that activity is declared 'done', while the transaction was not completed

Path 2. Duplicate the transaction in BPM/ACM, and call a service for the application
Design a user interface (1 or more screens, wizards, etc) for the activity, and implement this in the BPM/ACM solution. Create some type of service for the application, to support the transaction from the BPM/ACM solution (if needed, published on an ESB).

1. Contextual framing of data and functionality
> With a data-driven application, the user needed to assess an activity - what do I need to do - and then translate this to - how do I navigate to the right screens in the application. This takes time and learning, and is not value added (Lean) in the execution of a process. With a BPM/ACM solution, the system 'knows' the context and can present the right contextual frame
2. Uniform user interface
3. Usually better looking than the 'legacy' datadriven app

- Duplication of user interface and possibly business rules. See below.
- More complex (development work on two sides)
- Possible transaction faults (requiring roll-backs)

With path 2, a key question is: what are we going to do with the business rules around the data-driven transaction?
A number of scenario's:

2A: All relevant business rules are duplicated in the BPM/ACM layer
- Good user support in the user interface, while working on the transaction (data-checks, lookups, allowed values and value-combinations, etc), reducing or preventing errors or in Lean terms:  Poka Yoke.

- Maintaining business rules suddenly requires updates on two places, requiring good business rules governance and risk that business rules will start to differ over time.

2B: Only direct user interface rules duplicated
Only the direct User Interface driven business rules are duplicated, the harder more back end rules are kept in the application, and are used in the service (which may lead to a reject in the service, because certain rules are not met)

- Less business rules to duplicate

- Hard to sometimes decide what are 'user interface' rules and back end rues
- A work-item that is completed could lead to a failed transaction, requiring error handling functionality, that will lead to additional work-item loops to the user ('Your work-item lead to errors, please correct'). This is not value added.

2C: No business rules are duplicated
In that case, a rather 'dumb' workitem set of screens are presented. All business rules are checked as part of the service call, on the application side.

- Business rules on one place
- More simple

- Can create user confusion, and higher risks that people enter wrong data
- Requires error handling functionality, that will lead to additional work-items loops (not value added, not first-time-right)

Path 3: User interface inclusion
I have actually never seen this in real life (we always dropped it as a viable solution). In this scenario, in some kind of way, the user interface of the application is included in the BPM/ACM user interface. As soon as a user opens a work-item, the appropriate screen from the application is shown (if possible with populated data from the application and the BPM-work item & process instance data). As soon as the user performs a certain triggering activity, either the BPM-system closes the application screen, committing the transaction in the application, or the application detects the transaction-completion and triggers BPM so that the work-item can be reported ready.

- No duplication of screen and business rules
- Contextual framing
- User recognizes the well-known screens of the application

- Complex. Most presentation-layer technology has a hard job here
- Triggering to mark the completion (or errorstates) is very complex
- Risk that triggers are lost and application and BPM get out of sync (application has commited data, BPM is not informed)
- User interface variations in the BPM user interface
- Strong coupling

I have seen various implementations for path A and B. But I am never really happy. What would be a great way out, is that the technology vendors come up with a better integration standard: a standard to expose each user interface from their applications (ERP, CRM, etc) as a service, with some type of metadata exchange on functionality, presentation, and business logic/rules. This would allow BPM/ACM systems on design time or run time to present the right user interaction (but in the UI of the BPM/ACM system itself) ,while using all functionality, data and business rules on the application side.

Is JCA supporting this? I am not a Java-technie, but I don't think so. Not to this depth.
Of course, there are still complexities, for example what if the transaction granularity and focus in the back-end application is different from the required transactions from the BPM application?

But a standard in this area would lead to a path 4, that would have a lot of advantages:
- Contextual framing (Value Added)
- Direct checking of business rules (Poka Yoke)
- State consistency (in terms of data, transaction and work-item completion)
- No duplication of business rules and business logic
- Consistent user interface

Key question: are vendors of data-driven applications open for this? Especially vendors that have their own BPM-layers? I doubt it. Old-world thinking Application vendors might think: 'I don't want my application to become some black-box behind a BPM-layer! How will I compete in terms of brand recognition?'.
But perhaps I am too cynical here, and can Application vendors see the value of providing strong building blocks of data/rules centric functionality, that fit in 'plug & play' in modern BPM-based architectures.
We will see...

Sunday, July 27, 2014

Five approaches (and maybe six) to process design

As a BPM-specialist, one can encounter different situations for improving processes. These situations will require you to choose wisely from your BPM-toolbox, depending on various aspects such as client strategy, capabilities, trust and risk-appetite. One aspect is the approach you choose to improve a process. In my process improvement projects, I have encountered five different approaches (as shown in the diagram):

Let's dive in:

1. Common sense - incremental
This improvement approach is used quite frequently: gather process participants, analyse the current process and pain points (from outside in, and inside out) and come up with improvement ideas. Pick the ones that deliver most value, in regards to the investment of implementation, and implement. Iterate.
Key characteristics: it starts with was is, and with the people involved, and relatively small steps are taken.

2. Improvement practice based - incremental
In this approach, a specific improvement practice is selected. Typical examples include Lean, Six Sigma and ToC. Depending on scope and adoption of the improvement practice, various techniques are used on a certain process (such as Value Added Analysis, Root Cause Analysis, Leveling, Kanban, Bottleneck identification, etc). Key characteristics: it starts with what is, with the people involved (trained or guide by a improvement practice specialist), and relatively small steps are taken.

3. Best Practices / Reference model based - blueprint
This approach is often used, if, besides improvement goals, there are also harmonization goals (multiple locations running comparable processes). The organisation picks a certain reference process and replaces the current process with the reference process (often supported by specific IT solutions, such as ERP solutions). Examples of reference process models include eTom, SCOR, VSM, Scrum, ITIL, the dutch municipality GEMMA processes and SAP's reference processes for various industries and process domains.
Key characteristics: it starts with the to be, more top-down, often less influence of the process participants involved, a relatively large step is taken (although organizations can pilot a process in one location first).

4. Innovation driven (new business concept or IT) - blueprint
In this approach a new innovative concept has been adopted. This could be a new business concept (self-organizing teams, case management role, self-service supermarkets, mass customization), a specific technology (cloud, internet of things, BPM/Case Management solutions, digital documents/ECM, eCommerce, mobile, social) or a mix. Typically, as a team you would asses a certain set of processes, and come up with new designs, by asking what the innovation(s) could do for the processes and the involved stakeholders. 
Key characteristics: it starts with innovation,specialists assess and come up with the new to be process(es) blueprint, with top support, and implement.  Business cases may play a large role, if the innovation requires extensive investments.

5. Architectural driven Re-engineering - top down
In this approach, a thorough analysis takes place, from a new fresh perspective. All key questions around a business area is re-addressed, often in combination with a renewed strategic positioning. What is our strategy, who are our customers, what value do we want to provide, what capabilities does this require, what functions are needed, what competences, leading down to the to-be processes.
Key characteristics: is starts with the question why what, and through strategy, principles, and implementation decisions leads top-down to a process design, that is implemented.

So, here are 5 tools for your toolbox. Are there better or weaker approaches? No, it depends on the client situation and requirements. Can you manage all of these? That's a tough question. In my 20 years of process work, I think one can cover these approaches fairly well, but it takes time, effort and opportunity.

I actually think there is a sixth approach. It's called Design Thinking applied to process design. I am still investigating this approach. More to follow!

In addition, it's good to know that as part of the approach, as a BPM-specialist you will always need to figure out the type of role you need to take. This could vary from:
- Specialist/Key designer - you bring in the knowledge on improvement and design practices + the know-how on the specific process and possible improvements
- Facilitator, you help key process participants and stakeholder to discover the to-be process
Of course, typically elements of both roles can be needed. I most like the situations were clients want to learn the stuff themselves (as it's their process, their asset!), and I shift over the time from specialist to facilitator / trainer.

Last, it's also important to understand the change dynamics in the client organization. Is there a pragmatic go-try it and improve culture (promoting incremental - developmental approaches), or a more blueprint driven, think and align before trying (which asks more for a blueprint - approach)