Evidence behind the deliverables


I am going to use TOGAF and the deliverables set out in the Architecture Development Method to highlight some examples of documents which should be evidence based. Within the Preliminary and Vision phases the Business Strategy is the principle design guide to the development of the core architectures.

I consider a baseline or current  architecture and the target architectures to be statements of fact and design and generally portrayed through conceptual models. How the architect arrived at theses models and on what basis is another issue and it is here that i see evidence based methods being most useful and appropriate. How does an architect decide which design is the right one to support a strategy? The architect’s decisions are based on the instructions within the business strategy and the goals and drivers within the blueprint. If the business strategy is weak then there is a likelihood that that the proposed architecture will not be able to fulfill the strategic requirements.

What makes a good business strategy document and where should evidence be used to back up findings and recommendations? Let’s consider the primary requirements of a typical business strategy document:

  • Strategic Focus
  • The Business AsIs
  • Market Analysis
  • Products
  • Marketing
  • Research and Development
  • Production and Delivery
  • Supply Chains
  • Business Systems and Processes
  • Stakeholder Relationships and Alliances
  • Organisational Development and Management
  • Environmental and Social Impacts
  • Risk Factors and Regulatory Compliance
  • Corporate Governance
  • Financials
  • Application of Investment Funds
  • Strategic Action Plan
  • Plan Improvement

Each of these topics represents key aspects of the business and as such the positions, findings and recommendations for change must be evidence based. I am going to assume that the business strategy has been written independently and is then presented to the Enterprise Architecture unit.  The first thing an architect must do is to review the evidence underpinning each of the topics and decide upon the validity and and impact of the decisions in relation to the size of change they propose.

Let’s consider an example where the strategy is recommending change. Market analysis has shown that sales for a particular product line have fallen over the last 12 months. A review of this based on interviews with retailers and questionnaires to customers has shown there to be a number of technical problems that customers are not happy with and thus they are returning products for repair against guarantees. This has led to costly updates and an extra burned on the service department. Further research has indicated problems with a supplier of key components and incompatibilities with technology within the product manufacturing and assembly division.

The strategy recommends a review of the current suppliers and a review of the manufacturing technology and has asked a team, which includes an Enterprise Architect, to begin a project to investigate a solution. The task for the Architect is to dig deeper in to the research and evidence to ensure that the root cause has been identified. Has the evidence located which technologies if any are failing. Has the original review discussed the problems with the service teams and considered what actions they have been taking? If the evidence has not identified specifics then there are still unanswered questions. It is too easy to take evidence at face value and begin activities without solid foundations.

The Architecture needs to consider the make up of the evidence, how detailled were the customer and retailer surveys, what questions did they ask and where they the right questions. Is the product line key to overall sales and what is the current relationship with the supplier. Is the product line merely reaching the end of its lifetime and perhaps it may be better to close the product line rather than commit further investment. This is where the Architect needs to consider a view from the business architecture and determine the proposed change in line with the wider enterprise architecture.

Enterprise Architecture value assurance


Why is value assurance important to Enterprise Architecture? Value assurance is a means to review plans, proposals, architectural designs and strategies to determine that value to the business will be delivered.

Like any assurance process it cannot guarantee value will be delivered but it can investigate and decide that the information is aligned to a framework or standard and that the evidence is sound and backs up the decisions, risk mitigation and actions. Value assurance should be independent in its approach and incorporate consultation and critical reviews of all proposals. The objectives of a value assurance review should be:

a) To provide external challenge to the architecture project team at each key decision stage; to help assess the validity and robustness of the work done and the key areas requiring focused attention; and to assist in achieving the value of the deliverable.

b) To assess the suitability of the plans and strategies to ensure a go ahead to operate within the context of the overall Enterprise Architecture.

c) To appraise the readiness and justification of the project to proceed into the next phase, including the project’s soundness for capital allocation.

d) To capture lessons learned for dissemination across EA teams and, where appropriate, facilitate best practice transfer into the maturation or project team.

Two important areas of value assurance are the compliance to the enterprise architecture framework and the validity of the evidence supplied. Checking compliance is relatively straight forward by ensuring the proposals are all based on the agreed template and process, checking evidence is a lot harder.

If an organisation has had a disciplined approach to information management (supported by an enterprise information management strategy ensuring all formal information is well classified, categorised and accessible) then an evidence relationship map should be straightforward. By evidence relationship map i mean the pedigree of references to past documentation pertaining to the current proposal.

For example, references to three similar projects implementing the same technologies or business decisions that have all proved value through their successes – delivered to schedule or cost, shown effective transformation and or growth through sales or customer satisfaction.

If an organisation has poorly managed their information assets then building up evidence to support decisions will be a lot harder. I suspect most organisations are a mix of these two scenarios and as Enterprise Architecture is relatively new, historic information assets aligned to the EA framework is unlikely. Value assurance and evidence management are two critical activities to support Enterprise Architecture maturity. If an organisation is not using evidence to support architectural decisions then it is running the risk of undermining the importance of its EA and its ability to deliver value in the future.