Wow! Has it really been eight and a half months since my last post?

This summer I started working on an ERP implementation project that contains a fair amount of custom development. When I came to the project it was in iteration #2. This iteration was to focus on product definition. The project team was struggling to define the product requirements to obtain a design that could be implemented. They had been thrashing a little because so much of the product design was dependent on how the product would be used on the sales order.

When I arrived on-site, the sprint was already supposed to be complete. In fact, I was involved in a kickoff meeting intended to start capturing requirements for the Customer / Contact iteration (#3). However, the product design had been "completed" and data had been loaded but everyone knew it was not correct. They just didn't know how to make it correct yet because many of the design details would be discovered during the "Sales Order" sprint.

The project team was a little reluctant to define the customer requirements for the current sprint because they didn't want to end up in the same position with customer design that they were in with the product design. So a decision was made to start the "Sales Order" sprint right away and then use some of that discovery to "inform" the work in the "Customer Contact" and "Product" sprints so they can complete.

I'm sure you have guessed what happened. The attempt to capture "Sales Order" requirements dissolved into an attempt to understand "invoicing" requirements without starting that sprint also.

Did Agile Renege?

What happened to the "Agile" promise? You know, "early and continuous delivery of valuable software." One may argue that you can't expect an Agile methodology to deliver on a promise if core principles of the methodology are violated. For example, one of the principles is "Welcome changing requirements, even late in development." It is clear the team was not willing to live with unknown requirements, much less "changing" requirements.

Before you write off this organization as ignorant of agile methods and assume the solution is simply found in training let me tell you that this is a business is a mature development organization that has been successfully delivering software using agile methodologies for three years. There may some missteps involved but most of the issue revolves around trying to stick the "square peg" of development methodology into the "round hole" of integrated ERP system deployment and implementation.

Microsoft (the ERP vendor) provides project guidance for using an "Agile" implementation methodology. If you google "agile implementation methodology" the pages found will fall into one of two categories:

• information on how to implement an agile methodology for software development 
• Sales materials on how consultants and resellers will help you implement software using an "agile implementation methodology"

The Mythical Agile Implementation Methodology

What you won't find is any credible source that has actually defined an agile implementation methodology. There is a reason for this. Agile methodologies are applied to the domain of software development not general IT project management. The methodologies have one objective, build working software.

Software development and system implementation are two completely different animals. As a small example of where the disciplines diverge simply take on of the basic processes in agile methodologies - refactoring. "Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior." (Martin Fowler)

The purpose for refactoring is to preserve the current software functionality while changing the design to support the addition of new functionality. Since agile projects to not capture requirements and do "big design" at the beginning of a project, they need to continuously modify their design to support newly discovered requirements.

With modern development environments and tools it is often very easy to refactor code and with automated testing strategies it can be relatively risk free. This is not true with implementations. How do you refactor an implementation simply and with little risk? I have seen attempts but have not seen a cost effective solution.

Another aspect that does not transfer from software development to implementation is "deliver working software frequently." How does the business community define "working software?" I define working as "using the software for the business purpose for which it was purchased." This excludes a "conference room pilot" from "working software." Implementations often require so many concurrently "working" pieces that it becomes difficult to draw any line in the sand for a single sprint. My conclusion is that wide-breadth, integrated systems (like ERP) by their very nature do not allow for "early and continuous delivery of valuable software." The breadth and integration creates so many dependencies that you can not "deliver working software frequently." 

Let me say this a different way. If there is "go-live" date, you have a single point in time at which the software will be "valuable" and "working." That is not early and often. That is not continuous. That is "big bang."

Phased Implementation

My conclusion does not exclude iterative implementations. I simply contend that implementation iterations does not an agile methodology make. My recommendation is to iterate over the business processess defining requirements and possible solutions. These iterations can even produce useful (but not "valuable") conference room pilots. These pilots are often used to discover more requirements with changes in the current configuration. Much of this is messing and at times more closely resembles sausage making than software development.

Often these iterations may include actual development efforts that may be managed in an agile manner. Great! Scrum away, or use whatever agile methodology floats your boat. Just don't call your implementation agile.

Serious or Semantics

Is this really a serious topic or am I getting my shorts in a bunch over semantics. What difference does it make if we call it "agile" or "phased?"

My main concern is with successful implementations. To be successful an implementation has to meet business needs and it must do it in an affordable manner. A realistic project plan is mandatory to a successful implementation.

It is my opinion that the project approach that is being referred to as "agile" is neither workable nor agile. I want any project plan in which I have a stake to have a known path to success. If that path has a "go-live" date then I want to know the plan has a realistic management methodology that admits some sort of waterfall approach. 

To have a minimal level of comfort in an implementation plan I need to see an integrated approach to managing the dependencies. Managing dependencies often means that all the related requirements are discovered and all the related designs are defined so all the related configurations can be determined and entered and all the related reference data can be defined and entered and all the dependent master data can be defined and entered / imported etc.

My next blog (hopefully less than 8 1/2 months from now) will include more detail on what this implementation methodology may look like.

Stating what the User 'Really' Wants

In my previous post I stated my opinion that the "biggest problem" in software engineering is "bridging the 'gap' between understanding what the user truly wants to accomplish and how to best leverage technology to help accomplish that." For some reason, I seem to be wired to perform this particular task intuitively. What is missing for me is a way to communicate my discoveries and understandings. When I have successfully discovered what the user "truly wants" I have failed to accurately communicate that anyone else.


This article contains some of my thoughts on how I intend to address that failure moving forward. Let me forewarn the (unlikely) reader that I have not yet successfully validated this approach. It is simply my intent to test its validity.


User Stories


User stories are intended to be a tool used to capture "what the user wants to achieve." If you look at examples you will find they provide real promise in accomplishing that objective. One of the common structures recommended is in the form: "As a <role>, I want <goal/desire> so that <benefit>."  The benefit portion is sometimes considered optional.


There is an aspect of what the user wants that is missing in user stories. User stories don't talk about what the user doesn't want. I propos to add a special category of user story. I will call this a 'User Gripe'. In a user grip, we address what the user doesn't want to have happen, what the user hates about the current system or what activity the user wants to avoid. If we use the same format to describe a user gripe it may look like this: "As a <role>, I don't want < obstacle/undesired behavior> because <pain caused>." 


I like user stories. I think they have value, especially if we add the concept of a user gripe. There is something I don't like about user stories however. Who's story is it? If we use the recommended format we find ourselves having to record the same story for different roles with potentially different benefits. I could just say "As a <role> or a <role>, I want ..." but that can get really ugly.


I think I want to use a connection to associate actors to user stories. If I define the benefit on the connection then I can associate multiple actors to the same story and define the benefit the actor gets from the story. The definition of the benefit could still be optional but the connection is not.


Related User Stories


What should be done with two different stories that are simply two ways to accomplish the same thing? If each individual piece of functionality needs to be developed they both must be documented.


Some people would advocate making a generic user story and defining the details in the design phase of the sprint. In many cases this will work. However there are times when you want one method to be developed in one sprint and another method in another sprint. In these cases a single story will not allow you to plan your sprints appropriately.


Another issue is related to 'epics'. If you start with an epic and then refine the epic to multiple user stories, is there any value in keeping the epic? If so, how do you establish (track) the relationship between the epic and the story?


These and other scenarios drive me to the concept of modeling relationships between user stories. It appears to me there are two relationships:

• Aggregation or containment - One story includes others. This is the case with epics. 
• Specialization or inheritance - One story may be a specific means of accomplishing a more general story.

As an example, take the scenario with which I am currently working. Imagine a help desk system that helps manage support incidents for a machinery manufacturing company. A service technician and a service administrator are interviewed and they both want to create a customer interaction and relate it to a specific incident and a specific piece of machinery. Since they often interact with the customer via e-mail, they also want to create an interaction using e-mail.


With a traditional approach of creating user stories on a 3x5 card you would likely end up with about 8 cards. Four would start with "As a service technician" and four with "As a service administrator." All the stories would be about creating a customer interaction but two (two copies) would have the same content about a specific way to create an interaction, while four (two copies of two) would be about some information that may optionally be included during the creation of an interaction. With a modeling technique you could have four stories with their relationship to each other and their actors. Here is what it may look like.

By putting information on the connections I can describe the benefit portion of the user story from each actor's perspective. If I have the ability to generate document text from the diagram, I could assemble user stories for review with the user. Because of the connections I can focus on all stories for a specific user or all stories related to a specific story. If I can search text associated with the connections I can find all stories that provide a certain benefit.


I wanted to cover more ground but this post is long enough already. I will pick it up in another post with the relationship between user stories and use cases.


Tory

The Need for a Better Tool

I am currently in the process of implementing and ERP system (Microsoft Dynamics AX 2012) for my current employer  (The Aagard Group). One of our overarching implementation principles is Requirements First. What we mean by that is our business requirements will drive the implementation decisions. This requires us to understand our requirements (a topic for another day) and understand the out-of-the-box functionality of the system (another topic to be tackled) and document the gap between those. Once we have establish the gap we can design a solution to meet the requirement.

It has always been my view that requirements are part of the design process. If you start with a list of "musts" it is very likely you will find the software that needs those musts will not adequately meet the users needs. Sometimes the users needs are met but the software is still less than it could be.

During decades of software development and system implementations, it has been my experience that after the software engineer has designed the system to meet the requirements he understands what the user actually wants to accomplish and would do it differently if time allowed. I have had many experiences where we have developed to a spec, received high praise from the users and with my newly acquired knowledge of the problem domain, asked the user a few "what if"s only to find we could have provided a significantly better system because the user had never thought of doing the task in a different way. (Here is a great article by Mike Cohn about this phenomenon.)

In his book "Software Project Survival Guide", Steve McConnell identifies three intellectual phases of a software development project. He calls those phases Discovery, Invention and Implementation.McConnell lists "discovering the user's real requirements", technical investigation and building prototypes as activities in the Discovery phase.

In my experience the best solutions use the most skilled people in this phase. It is the most likely predictor of success or failure. Failure in discovery makes is very difficult to succeed in implementation.

Yet, when you think of the tools we use to aid in the discovery phase, what comes to mind? There are use cases / user stories, UI prototypes and requirements documents.How can one capture intent? This is my current struggle. I believe this to be one of the glaring holes in the tools available to aid with the software development lifecycle.

Of course, this is not just my opinion. Others have said this and even offered some ideas to address it. Sriram K. Rajamani states "the biggest problem in software engineering continues to be the bridging of the `gap' between the intent captured in requirements and expressed at a high level, and the detailed encoding of this intent in the code. There are no good tools, either mental or mechanical, that allow comprehension of large programs, and provide a mapping between how different parts of the code work together to satisfy the requirements.Thus, looking for any high-level requirements within a million line program is analogous to running around New York City looking for a lost cat, without a map, and without any street signs."

I would move the "biggest problem" up stream slightly to bridging the 'gap' between understanding what the user truly wants to accomplish and how to best leverage technology to help accomplish that. However, we agree on the issue. There are no tools to allow the business analyst, the software architect and the software engineer to capture what they understand. The only options we have is to capture what we have concluded is the solution.

For example, we may have a list of requirements that have been tediously recorded. What we won't have is the information that was discovered that lead to those requirements. Why is this important? It isn't important if you only want someone to design a system to meet the captured requirements. However, if you want to invent something that will actually add value to the job at hand it may be very important.

My proposal is to combine disciplines and tools. I propose that software projects should team business analysts and architects in the discovery phase. I propose we build a set of tools that combine a good narrative , use cases / story cards and technical diagrams into an interactive, connected modeling environment that allows an analyst to link multiple types of content and define the types of links. Text elements could be linked to other design artifacts like UI prototypes, UML static and dynamic diagrams. I can imagine a narrative explaining why a user currently performs a particular task being linked to a use case, a number of derived requirements and even an activity, sequence or state diagram.The derived requirements could also be linked to UI prototypes and UML classes that realize those requirements.

Here is a list of the attributes that a system of this sort may need to have.

• Objective: To provide a means of capturing the intent of a software development initiative including the problem domain and desired outcomes.
• The Target Audience: Outcome Stakeholders 
• Content Properties
• Multilevel - Captures the highest level of intent to the lowest level of detailed requirement or design - Captures highest level of abstraction to the lowest level of concrete implementation
• Connected - Associative / Relational - identify and document the associations and dependencies and use them for tractability - able to trace from the lowest level of detailed design to the highest level of intent
• Multifaceted - multiple perspectives or aspects
• Process view
• Requirements view
• Data view
• Workflow view
• User Role
• Security
• Accessible - Available and understandable to all stakeholders
• Maintainable
• discrete elements that can easily be modified without major impact to other information 
• Revisionable - able to fit any sane view of revision / change control
• Comparable - able to compare iterations
• Interactive - electronic interactive document with the ability to produce static, printable documents for all or portions.

Over the next several articles, I intend to flesh-out the essential components of such a system and give a few examples.

Tory