Sunday, August 4, 2013

SQL Data Management with MyBatis

Managing Persistence of Data

The majority of applications work against some sort of data store and in most cases that data store is a database. For many years now it has been standard practice to separate the Data Management (that which manages  persistence of data) from other business logic in our solutions. 
We do this because we do not want to have Business Logic with hard dependencies on the structure of our database, or with dependencies on how we query data from the database. 

Separating Data Management results in a lower impact on our application if we should we decide to make changes in the database or even migrate to another database.
One way to help with abstracting Data Management is to use an Entity Framework. There are a number of frameworks commonly in use such as Microsoft EF, NHibernate/Hibernate or Toplink.  Entity Frameworks hide much of the nitty gritty SQL from the developer and allowing us to work in an object oriented fashion with persistence of data. They often provide an SQL like query language for reading data. They generate SQL under the hood, provide advanced caching of data, can provide independence from the database vendor. Common statements such as INSERT, UPDATE and DELETE operations are managed by the framework. However there are some drawbacks with Entity Frameworks since they can add another layer of complexity. We also hand over control of the SQL to the framework and performance can suffer if we are running  complex select statements.
Another consideration is how we are designing our data persistence. If we are designing our data persistence from our object model (or Code First) this makes working with an Entity Framework easier. However if we are designing from the database first (using a relational model) there can be difficulties, particularly if there are a large amount of relations in the model. This is a likely indicator of complex SELECT statements.
A good alternative to an Entity Framework is the MyBatis framework which provides a different method of abstracting Data Management and addresses some of the drawbacks of an Entity Framework.

What is MyBatis?

MyBatis is a framework which allows us to bind objects to SQL statements and execute these statements. In this framework the developer defines the SQL statements which are to be executed and how they are mapped to objects. This gives the developer more control over the SQL which results in more flexibility than an Entity Framework. However MyBatis does not provide SQL generation so we have to define all SQL statements. Since we define the SQL we also have control over how statements are executed and therefore we can execute complex SQL select statements fairly simply and map the results to any given object. With control over the SQL we have more control over the execution and performance.

MyBatis is available in both .NET and Java flavors.


How does MyBatis Work?

MyBatis works with an SqlSession object which manages the SQL session. This object provides the context in which we execute the SQL statements. The SqlSession is created from an XML file containing the session properties. This XML file contains mainly information regarding how to connect to the database,  and contains references to all the "mapper" XML files (these are the files containing the SQL queries).
An example session file is shown below:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration
PUBLIC "-//mybatis.org//DTD Config 3.0//EN"
"http://mybatis.org/dtd/mybatis-3-config.dtd">
<configuration>
  <environments default="Development">
    <environment id="Development">
      <transactionManager type="JDBC" />
      <dataSource type="POOLED">
        <property name="driver" value="driver.MyDatabaseDriver" />
        <property name="url" value="jdbc:xxx://localhost;database=mydb;"/>
        <property name="username" value="myuser" />
        <property name="password" value="mypass" />
      </dataSource>
    </environment>
  </environments>
  <mappers>
    <mapper resource="myapp/CustomerQueryMapper.xml" />
  </mappers>         
</configuration>


An effective way to implement this is to create your own "session factory" which manages the reading of the Session Properties XML and provides configured SqlSessions.






The SQL statements are place in "mapper" XML files where we define the type of SQL statement (insert, update, delete, select) and write the actual statement. When writing the SQL statements we can inject parameters into the statement mapped from our Java/.NET objects. Then to make things even simpler we can define an Interface containing methods which execute the queries providing the correct parameters and return values. The methods in the interface have the same names as the SQL Query identities.
So for example, if we have some queries on a database for Customer Information we can place the queries in a CustomerQueries.xml. In the XML file we can define queries which take parameters from a Customer object or even return result sets of Customer objects. We can also use other objects for parameter values or return values, there are no restrictions here.





The queries are structured via mapper namespaces and a reference to the mapper file must exist in the SqlSession. Below is a sample code from a CustomerQueries.xml.

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="example.CustomerQueryMapper">
  <select id="findCustomerNames"  parameterType="example.Customer"     
          resultType="string">
                        
    SELECT CUSTOMER_NAME FROM CUSTOMER_TABLE
    WHERE
      LAND = #{land} AND
      SALES_CODE = #{salesCode}

 </select>
</mapper>

So we can see in this example that there is a single SQL query for finding customer names ("findCustomerNames"). We bind a parameter to this query in the form of a Customer object and indicate that the query will return a set of  "string" objects. We then inject the parameters to the query using the "#{}" notation. This will insert the "land" and "salesCode" properties from the Customer class into the above query. This provides a great deal of flexibility in mapping our queries to objects.
The example above shows only a mapping on a "select" statement, but there is also support for "insert", "update" and "delete". For full details regarding how to use MyBatis please refer to the MyBatis documentation (http://mybatis.github.io/mybatis-3/).


The Advantages and Disadvantages

Some of the advantages with MyBatis:
  • Flexibility - we can form the SQL exactly as we want and there is little restriction
  • Simple Configuration - the XML configuration is simple and easy to deploy
  • Complex Select Statement Performance - since select queries are executed directly there is no overhead for loading lots of entity objects before obtaining the result

Some of the disadvantages with MyBatis:
  • No SQL Generation - does not generate SQL like an entity framework so "inserts", "updates" and "deletes" have to be defined manually.
  • No database independence - the SQL is written by the developer specifically for the given database so migrating to another type of database is perhaps not as easy as with an Entity Framework
  • Less advanced caching - MyBatis perhaps lacks the advanced caching techniques that other Entity Frameworks possess.

Conclusion

MyBatis is a very useful framework which aids the persistence of data. The technology is mature and appears broadly used in industry, quoting references such as MySpace (https://code.google.com/p/mybatis/wiki/FeedbackTestimonial). However it still depends on what is required from the Data Management layer. When selecting a framework I would consider the following questions:
  • Do we have complex select statements, or table relationships?
  • Are our developers comfortable with coding SQL?
  • Is the data design driven from the relational database?

If the answer to these questions are "Yes", it would be well worth considering MyBatis as a viable framework for  Data Management. 


Sunday, May 5, 2013

NoUML or perhaps LowUML

Recently I attended ITARC 2013 where there was a very interesting seminar by Simon Brown regarding Software Architecture Sketches and NoUML (see his article on InfoQ). This seminar led me to further consider how we effectively communicate and visualize our architectures at both Enterprise and Solution level.

Traditionally when we were building software systems we often had a very homogeneous team of people working with the development. Often a team of technical developers and we may have worked on a basis where the architecture was driven from a technology perspective. In this respect the stakeholders in IT were often technical in nature and communication of the architecture between these stakeholders would be performed using models such as UML. This was suitable since the stakeholders often had experience of these models from their education and were familiar with the terms and concepts.

In recent years the landscape has changed. Architectures are built with the Business as a driver with a view to interlocking business and IT. The adoption of Agile also plays a part making the development process transparent and also employing skills in cross-functional teams.
The result is that we have a significant increase in stakeholders in the architecture many of whom may be non-technical and who all have different perspectives on the given Product or Enterprise. At the same time need an effective way to communicate and understand the architecture for all perspectives.

Difficulties with UML


Firstly we have to consider what we mean with "the architecture". When do we stop with an architecture and start with a design? We could spend a lot of time debating this but the following quote does a pretty good job of summarizing what "the architecture" represents.

"Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change."(Grady Booch)

This is important because UML still has a place in software development when we are working with lower level abstractions since we require significantly more details in our communication. So we are still likely to produce class diagrams, sequence diagrams and other UML diagrams to help communicate complex detail. We can still use these in "the architecture" but these feel more like "designs" than architecture. This is why I prefer the term LowUML rather than NoUML.

It is perhaps the complexity and more rigid notation which makes UML a difficult tool to use when communicating and architecture at a higher level of abstraction with many stakeholders. One problem is that many stakeholders, even in the development teams, simply don't know UML and have no ambition to learn it either. Such stakeholders may be non-technical or skilled in a field which does not require knowledge of the details but does require an understanding of the architecture. This is where simpler modeling methods such as "sketching" are more useful.

Separating the Levels of Abstraction


Part of making the architecture easier to understand for different stakeholders is the separation of the levels of abstraction. This will allow stakeholders to view the architecture from at a level of detail which is easy for them to understand. Simon Brown suggests using the following levels of abstraction.


  • Context - shows IT landscape over the enterprise or for a particular solution, mainly shows systems and actors
  • Container - the high level technology choices, where containers are often execution environments such as databases, application servers or file systems
  • Component - the relationships between components where a component can be thought of as a service or logical unit exposed through a known interface.
  • Class - a description of the classes in a component adhering to object oriented principles.


The "Class" level of abstraction is a low level and in this case I would recommend continued use of UML rather than creation your own notation.

Note that these diagrams are very focused on the "structure" of the architecture and are not intended to processes or detailed behavior.

Define Your Notation 


When using sketches in an architecture we can often use a number of symbols, line styles or colours in our diagram to improve understanding.

However we can often forget to explain our notation and while the diagram may be clear to the author it may not be so clear to the stakeholders. So it is important to have a clear explanation of the notation for the stakeholders. An example notation is shown in the diagram opposite, and this notation will be used in the following diagrams.
We should keep the notation simple so it can be easily understood by the different stakeholders. If we find ourselves adding more complex notation this is a warning sign. The purpose here is to create an easy to understand communication medium, not to create our own version of UML.
When we reach the lower levels of abstraction and require need a more powerful notation we can switch over to UML diagrams. We still want to use UML but at the lower levels of abstraction.


Context Level of Abstraction


The context level of abstraction is the highest level of abstraction and shows the system landscape. This is the level we want to use with Business and Non-Technical stakeholders. We can use a context diagram to describe both the Enterprise and the Solution landscape.

In the example below we can see an Enterprise Landscape containing a number of systems we may commonly find in any Enterprise. Most of the systems in the diagram are Internal (the Enterprise maintains control of the system) and only the Credit Check system is External (perhaps provided off-site by a vendor).
Notice that in this diagram we only indicate that there are relations between the systems and actors, we do not go into detail regarding what that relation is.



If we are working with the architecture of a specific solution we can also detail the Solution Landscape. This shows a specific solution and it's associated systems/actors. This basically focuses on a section of the Enterprise Landscape. In the example below we can see the landscape for the Web Booking system.

Keeping these diagrams simple will allow to communicate a level of the architecture to many stakeholders. At the same time these diagrams do not contain details so if we are looking for technical details or more information about the relations we have to lower the level of abstraction.

Container Level of Abstraction


Containers show the high level technology selections on a per System basis. In these diagrams we are moving away from the Business/Non-Tech stakeholders towards those who have an interest in the technology. So this is the domain of the Architects, Development  Teams and Operation Teams. The following is an example of containers for the "Web Booking" solution.




We can now see the technologies that will be used in the architecture and the relation between the containers, systems and actors. At this level we may also want to detail perhaps technology protocols used in the relation, such as SOAP or SQL, these are not shown in the above diagram.

Component Level of Abstraction


At the component level we are reaching the lower levels of abstraction in the solution architecture. It is here that we start to define our logical units or services on a per Container basis. There can also be more than one level of abstraction for components and this is perhaps where it gets more difficult in knowing where to draw the line between the simple notation and a more advanced notation such as UML. At this level we are communicating to technical stakeholders often in the development team. Below is an example for the "Booking Web Server" container.



As we can see this is where we start to hang more detail on the architecture. Not only are the components specified but the relations between the Components, Containers, Systems and Actors indicate direction and we give some idea of the data that is involved in the relation.
It is important not to over complicate the diagram. If we have a component which is complex we can create a new diagram specifically for that component and decide whether it is more suitable to use a UML notation for modelling or if the simple notation is sufficient.

Keeping Things Simple


One way to try and limit the complexity in diagrams is to put a boundary on the size of the diagram. This is not always possible but a good rule is to try and keep the size of each diagram limited to fitting on a single sheet of A3 paper.


Not Only Diagrams


The diagrams are probably not sufficient on their own to describe the architecture so textual description will also be required in formal documentation. However the diagrams as working documents can be used on their own. It can also be an idea to tag "responsibilities" to Systems, Containers and Components. This can help to build a better architecture since we can easily see if we have duplicating responsibilities over Systems, Containers or Components.

Make the Architecture Visible


Architectures are often documented, but lack a high level of visibility in the organisation or team. If we want to consider the architecture we often have to look it up in some document which is archived somewhere or on a web page.
 A better approach is to ensure that the architecture is constantly visible so that stakeholders can use it as a reference point when discussing proposals or changes. This simply involves pinning up the architecture diagrams in a team room so they are constantly visible and can actively be used in discussions.

Conclusion


There is a need today to communicate Enterprise and Solution architectures to a broad spectrum of stakeholders. Many of these stakeholders do not have an understanding of UML, and this makes UML a difficult tool to use at higher levels of abstraction.

Using a simpler notation makes it easier to communicate the architecture and using well defined levels of abstraction allows us to target different stakeholder perspectives. This also helps us easily visualise the architecture and use it as a daily reference point for development teams.

When we reach the lower levels of abstraction, closer to design, a simple notation may not be sufficient and it is at this point we can switch to using a notation such as UML which can help describe complex designs.

Monday, April 1, 2013

How Agile do you have to be to be Agile?


Agile methods for software development are widely used in the IT industry and are in many cases accepted as the "de facto" standard for effective product development. However employing Agile in an organisation is not clear cut. Many organisations are sold Agile methods as the answer to all their problems only to find that after a good deal of investment, in both time and money, the advantages that they were promised are not being realised. So why isn't working?

The truth is that Agile is not a solution in itself, in fact Agile is a set of principles (http://agilemanifesto.org/principles.html) which can be applied to facilitate a very effective way of working.  In some cases focus is placed so heavily on using what is considered Agile methods, that we forget what we are actually trying to achieve. This is where we have to remind ourselves that our aim is actually to deliver a product to our customer on time, on budget and at the right quality.
Changing the processes in an organisation is not easy regardless of which methods you intend to implement, and we can be sure that the hardest things to change are not the processes but people.  The capability and motivation of the people in the organisation are central to success.

So how Agile do you have to be before you can call your organisation Agile?

Waterfall and Agile


Often there is a lot written about Waterfall versus Agile methods of development, however this may not be a very good comparison. The original "pure" waterfall model (Winston W Royce 1970) may be considered a static model with a "plan everything up front" method. However since the late 90s the use of iterative and incrementive models have been widespread with each iteration resembling a mini-waterfall. Even Agile still adheres to the good engineering practice of Specification, Design, Implementation and Test in a sequential order.
The real difference with Agile is that it includes aspects of the  "project/product management" domain while traditional models are more focused on the technical development process and consider the "project/product management" as a separate domain.
It is also worth considering that there is no "one size fits all" for software development. Each product and organisation has its own characteristics, capabilities and constraints. In many cases a hybrid approach may be required with an organic growth towards Agile. Performing a "big bang" change to an organisation can be costly when migrating to any new process.

Know your Business Model


When migrating to an Agile model it is very important to consider the type of business model for the product or the organisation. This has a big impact on how the Agile process will be implemented. Failure to consider the business model can result in the Agile process simply not working or a long journey of trial and error to make processes fit.
We can consider the Iron Triangle (Dr. Martin Barnes) which shows the constraints of Scope, Schedule and Resources in project management. This shows that a change in one of the constraints has an effect on the other two. So for example, a reduction in Scope may result in a shorter deadline, or an increase in resources any allow for more features . But why is this important for Agile?


If we are running a business model which allows us a lot of freedom with Scope we can run a "line based" approach. This runs rather like a production line where we have a schedule determined by sprints and scope for delivery is negotiable. The host organisation has to have a good degree of influence over these constraints. Releases of the product are frequent and contain features which can fit into the defined budget and schedule. This allows us to quickly react to change since we can determine scope at the beginning of each sprint. This is a model which suits organisations providing products to a general audience where the product is purchased perhaps on licence or subscription. The decision regarding scope lies solely within the organisation.
On the other hand if we have a business model where the scope and deadline is often fixed we would probably want to run a "project based" approach. We can still use many of the Agile principles and tools but we are required to perform more planning up front. Deliveries from sprints may be more internal in nature rather than production quality since the final production delivery is dictated by the deadline. There is a reduced ability to embrace change since the constraints are not as flexible. This is a model which appears in organisations which often work to a contract. These constraints are given in a contract between two parties and must be adhered to legally.

If we are running a business model which allows us a lot of freedom with Scope we can run a "line based" approach. This runs rather like a production line where we have a schedule determined by sprints and the scope  is negotiable. Such a host organisation has to have a good degree of influence over these constraints. Releases of the product are frequent and contain features which can fit into the defined budget and schedule. This allows us to quickly react to change since we can determine scope at the beginning of each sprint. This is a model which suits organisations providing products to a general audience where the product is purchased perhaps on licence or subscription. The decision regarding scope lies solely within the organisation.On the other hand if we have a business model where the scope and deadline is often fixed we would probably want to run a "project based" approach. We can still use many of the Agile principles and tools but we are required to perform more planning up front. Deliveries from sprints may be more internal in nature rather than production quality since the final production delivery is dictated by the project deadline. We may expect stabilising sprints during the development. There is a reduced ability to embrace change since the scope may not be flexible. This is a model which appears in organisations which often work contract based, where the constraints are given in a contract between two parties and must be adhered to legally.
Good Engineering Practice still applies

One of the principles mentioned in the Agile manifesto that is often misinterpreted is "Working software over comprehensive documentation". In some cases this can often be wrongly interpreted as "no documentation required as long as the system works".  Documentation is an essential part of ensuring that a system can be maintained effectively. Without documentation we create a situation where we are dependent on "people with key knowledge" to maintain the product. This represents a serious risk for product maintenance whether we run Agile or not.    

Concrete Principles which help Agility


Regardless of whether you are running "line based" or "project based" approaches there are a number of concrete principles which can help our organisation become more Agile. Below are some principles which can help agility.

"encourage transparency"
With an open and transparent culture, risks and opportunities are quickly identified which also means that they can be quickly addressed. Removal of a "blame" culture will help teams to be self organised since individuals will be more likely to take on responsibilities. Tools such as Burndown charts are easy to understand and clearly communicate the status of development to all stakeholders.

"deliver frequently"
The use of sprints and frequent deliveries ensure that the team is constantly working towards a short term goal and this gives focus. This allows us to regularly monitor both quality and delivery precision. If we start missing our deliveries or the quality of the deliveries are not up to standard this becomes apparent very quickly. We can then make adjustments and monitor the effect in the next sprints.

"commitment to deliver"
In order to successfully make frequent deliveries the there has to be a commitment to deliver from everyone involved. To gain this commitment everyone has to be involved in the planning process. If the whole team are involved in defining the scope and time estimates, there will be more commitment to deliver. Even at company leading levels there have to be clear overall goals and realistic plans. If the requirement on the team is unrealistic the team will lose their commitment since they feel there is no realistic chance of success.

"do only a few tasks at one time"
Getting a working release (internal or production quality) completed is very important. It is at this point we can really test and evaluate the progress of development. In order to get working releases we need to make sure we complete the tasks we are working on and this is why it is much more advantageous to have a few tasks fully completed rather than a large amount of tasks only half completed. A half completed task adds no value and hinders the delivery of a release.

"cut waste, continually improve"
In order to be Agile we have to continually ask ourselves what is adding value to our product and how we can improve.  This is a continual process of fine tuning and keeps the development process aligned with the business. A good way to consider this is to think about which tasks add value to the product and which tasks do not add value to the product. Then consider if we can reduce the amount of time we spend on the "non-value added" tasks.

Practical implementation of an Agile Process


Agile can sound very easy to implement until you start to apply it to the organisation. Like any change in process there are likely to be problems and challenges since no single product or organisation is exactly the same. Each organisation has its own set of capabilities and legacy processes. The degree of change to be made has to be balanced with the required investment and long term gains.
It is important to give the change in process time to settle but at the same time if something is clearly not working take action. How well the Agile process fits is often related to the type of business model the organisation has. Below are some issues an organisation may face in a migration to Agile.

"owning the product"
The Product Owner is a key role in Agile and controls the starting point of the process. It is very important to ensure that product ownership works well. The Product Owner specifies the requirements and maintains the backlog. In order to gain a long term view of investment it is also wise to ensure that the Product Owner is responsible for maintaining "wide estimates" on the backlog. Just like any other development process if we get the requirements wrong we will be looking at expensive changes.

"up front planning"
The amount of planning we need to perform up front is related to the flexibility of our project constraints. The tighter the constraints the more planning will be required. Even where we are running a "project based" model and have a large project, we can attempt to reduce detailed planning by dividing the project into phases with a release per phase.

"time estimation"
To be Agile we want to be able to estimate time quickly and re-evaluate our accuracy after each sprint. We can use techniques such as planning poker and we can estimate time in story points or just hours. One difficulty for a team here is intrusions during a sprint which draw hours from the original estimate, this is in some organisations unavoidable . A simple approach to account for this is to estimate a productivity percentage for each sprint. So of we have a team with 500 hours available and a productivity of 50% we will be able to assign 250 hours of tasks to the sprint. At the end of each sprint we can check the outcome and adjust the productivity percentage accordingly.

"what is in and out of the sprints"
Another area which can be difficult to establish is what kind of tasks to we include in the sprints and what kind of tasks lie outside the sprints. We want tasks within our sprints to be determinable so we can assign accurate estimates. Requirements and analysis work in its nature can be difficult to determine, therefore these often lie outside the sprint. Design at a high level of abstraction may also lie outside the sprint with design regarding specific requirements being placed in the sprint. Implementation work and unit testing lies within the sprint. However "integration and regression testing" may lie in or outside the sprint depending on our business model and automated testing support.

"testing and stabilisation"
In a "line based" model we will be expecting to turn out production releases at a high frequency with the benefit being "time to market" for the product.  However for this to have a realistic chance of success a high degree of automated test is required since "integration and regression testing" has to be performed in a very short space of time. This may require investment in more advanced testing tools and frameworks.
In a "project based" model delivery to production may be more long term. In this case while automated testing is still favoured, it can be complimented with "stabilisation sprints" where "integration and regression testing" can be performed manually.

"don't be afraid to try"
Don't be afraid to try things that are perhaps not considered fully Agile. Remember that what we are aiming for is a process which helps our organisation get our product to market, on time, at the right quality and on budget. In many cases a "hybrid" Agile process allows the organisation to reduce the impact of change and can allow the process to grow organically with the organisation.

Conclusion


Agile is without a doubt a great platform upon which to base product development but in order to have an Agile process working well in practice the host organisation has to understand its own capabilities, and its business model. All organisations and products are different in nature and there is no "one size fits all", it is likely that the Agile process will require some degree of adaption if it is to be effective and avoid large initial investments. There are a number of concrete principles mentioned previously which can help get the Agile process moving. However the starting point for developing an Agile organisation is to buy into the principles and have your organisation buy into these principles. This provides the foundation for an organisation to be Agile.

Wednesday, January 30, 2013

Strategy is Critical to IT Success

Why do we need a Strategy?
Many business and IT initiatives are a long term investments where we have changing markets and trends. These initiatives often take years to develop and require a lot of effort, resources, time and money.
We may have a very effective organisation with all the right processes in place and the right capabilities with all the skills, but all this will be in vain if we do not know where we are going or have a clear direction. 
This is why Strategy is extremely important as it provides Vision and Objectives which give direction for what the organisation is trying to achieve. This allows us to align our IT initiatives with the strategy so that we all develop in the same direction.

Effects of poorly defined strategy
 An organisation which has a poorly defined or communicated strategy may well find that projects or products are going off at tangents, which can lead to fragmentation or disjointed products. Another effect of a poorly defined strategy is that goals in the organisation or projects keep changing, this means that processes and projects need to be consistently re-aligned with the new goals and this can be an expensive. Goals which are changed frequently can also have an effect on the capability of the organisation as personnel become confused as to what the organisation is trying to achieve and ultimately motivation can begin to dip. 
The end result is that deadlines are missed and costs rise as the organisation battles to produce the deliverables which are aligned with frequently changing goals.

From an IT perspective this can also be evident if we see fragmented technologies or architectures within the organisation.

Working with the Strategy

Working with a strategy is difficult as we are often dealing with the long term and there are many unknown factors. An IT Strategy has to fit with other strategic aspects in the organisation, for example the overall strategy or mission of the company.

A good strategy will provide a guiding light and a "Vision" but it will also force us to make choices in order to provide focus.This is a delicate balance since a vague strategy may lack meaning while a rigid strategy will reduce flexibility. A good deal of work is required here to assess if the the strategy is feasible and that we can clearly describe the benefits and risks with the given strategy. While a strategy should be challenging, communicating an unrealistic strategy can result in a loss of faith  and motivation, no-one will want to follow a strategy which appears doomed to fail. So we have to ensure that the strategy is feasible for the organisation.

Assessing the risks associated with the strategy is very important. Since the strategy penetrates the whole organisation, getting it wrong can be costly so a good risk analysis is important. There are a number of risk analysis techniques which can be used such as SWOT analysis to help identify and mitigate risks.

To get a drive behind the strategy it is a good idea to back up the strategy with facts and reasons behind why we want to employ the strategy. Simply pushing out a strategic statement might not be enough to get your organisation actively behind the strategy, and if the organisation does not believe in the strategy it will be difficult to achieve the goals. In this area we might want to show trends in industry or show white papers or reports which support the strategy. We can also try to create an open discussion around the strategy to gain involvement from the organisation, this will make acceptance of the strategy much easier.

Since strategy in it's nature is long term it can be difficult for an organisation to envisage reaching a goal which is perhaps five years in the future. For this purpose it is good to have a reasonable strategic plan which shows milestones (or sub-goals) for the short term which in turn provide a pathway to the strategic goals. This plan is an effective way to communicate and measure progress towards the strategic goals.

In some cases a change of strategy may be pushed upon us due to circumstances we cannot control. This can mean large changes in the organisation such as changing business sectors or even existing technologies. This can have an adverse effect on an organisation where certain skills may no longer be a focus of the new organisation goals. This can lead to turbulence in the organisation as personnel may feel that their skills are are no longer relevant, so it is important to have a plan as to how the strategy is to be launched and communicated to reduce this effect.

Actively using the Strategy

Once the strategy is defined it can be used at all levels of the organisation to guide and provide governance. When working with product management this can help in defining requirements. We may find ourselves debating over which features or technologies to use in a product. If we reflect on the strategy it can help us to decide and prioritise, we can ask ourselves "Do these features or technologies help us reach our strategic goals?". If the answer is "no", we then have to ask ourselves why we want to employ them.

We can use the strategy in our project teams to motivate and guide. A clear idea of how the project fits into the strategy can focus the team, give a greater sense of contribution and increase motivation.

We should also be open to feedback as high levels of activity which is not aligned with the strategy can indicate a flawed strategy or a lack of capability to carry out the strategy.

Conclusion

Having a clear and well communicated strategy is essential for an IT Enterprise. It provides direction and focus for the organisation and provides an foundation for governing the Enterprise. A poorly defined strategy can result in an ad-hoc organisation where projects go off at tangents and this can affect both the motivation of personnel in the organisation and even the marketing of services and products.
The strategy can be used at all levels of the organisation to pull everyone together and strive towards the same goals, providing synergy between leading groups, departments and teams.




Wednesday, January 2, 2013

Tracking Performance in Java with Perf4J

Where are the Bottlenecks?

Performance is a critical aspect of any solution and pretty difficult to measure since we have to account for all the different load scenarios a system might meet in production. There are a good number of performance profiling tools which can help find bottlenecks in development, but what happens when the system is in the test or production environment and you need to check performance? We can't use the profiler in the production environment so we probably have to fall back on good old logging. Even in development and test environment performance logging may be quicker and simpler especially if you know what you are looking for. This is where we might use the Date class and the "getTime()" can solve this problem as shown below.



However this can become messy and difficult to track and re-use in many parts of your code. Therefore we are probably faced with creating a framework of classes to manage this function and to manage logging of the data. This is such a common test requirement in development I thought there must be a framework out there which already does this, but at the same time is simple to use. I was right, there is and it's called Perf4J.

Simple Performance Logging with Perf4J

Perf4J is a framework for performance logging  which comes packaged in a single JAR. It is simple to use and provides support for timing code sections and exporting the result to standard logging tools such as Log4J.
However it doesn't stop there. Perf4J supplies a number of features for helping you analyse the results of your performance logging. Support is provided for generating real-time performance information using an appender in Log4J. There is also support for generating graphs using the Google Chart API.
The advantage of using a standard framework like Perf4J is that we don't need to maintain the code and that it has a much larger user base than a "build your own" solution which means help is available from the community. Perf4J is licensed as open source under the "Apache License Version 2.0".

Logging Execution Times

The logging of execution times is based around using a StopWatch class which allows you to "start" and "stop" a timer for a section of code. The StopWatch can be setup to send this data to the log of of your choice. A nice feature here is that when you start the StopWatch you can give a "tag" which identifies the performance measurement and a "message" which can identify a particular section of the performance measurement. This is really useful if you are taking many measurements and want to organise these measurements so they are easily readable. This allows us to group our performance measurements together making the resulting log much easier to understand.

Below is an example of a simple JUnit test for logging two sections of code. In order for this example to work the "rta.test" logger has been configured in a log4j.xml file.


As can be seen from the code, the StopWatch is set up in the Log4JStopWatch  for compatibility with Log4J. We are then free to log times using our "tags" and "messages". The result of the above code is shown below.


Conclusion

Perf4J is a simple framework for performance logging. It allows you to easily take performance measurements and organise these measurements. If you want more advanced visualisation of the statistics there are features provided for this as well. Using this framework reduces the maintenance associated with "build your own" solution and a broader user base provides better support. In short a nice framework which makes you life a little easier when performance testing.