Offline web application

Some time ago I was thinking about the issues that envolve the modern trend of having all the applications online somewhere on the "net". GoogleDocs, Picasa, e-banking, online calendars and more seem to be the right way to go ... unless you find yourself somewhere without a reliable connection.

One of the most powerfull buzzwords nowadays is "cloud computing" and the "everything-in-the-cloud" pattern. But I bet almost everyone has at least once been in a situation where he needed to work with his favourite web tool but he simply couldn't since he lacked a good quality connection. What then ? Wouldn't it be awesome if he could just use the tool localy and once connected synchronize the data without him even knowing ?

There are several basic ways how to achieve the required behaviour. The modern one would be to use HTML5, but due to low browser support and the fact that many users/institutions use old versions of browsers this is not usually a good road to take. Another solution if you already have a fully functional web application is to deploy it on a localy installed application/web server and implementing the tier responsible for data synchronization. Last but not least interesting is the somehow forgotten solution of creating a desktop application (thick client) that uses the business logic stored on the server through some middleware technology (with delayed evaluation untill online) or SOAP.

So I've started searching the web for some articles that cover the topic and here are some of the most interesting I've found:
Synchronizing a Web Client database (Google Calendar)
Operational transformation explained
HTML5: Client-side Storage
Making the Case for Local Database and Synchronization

One of the basic problems that we need to cover is the synchronization of databases and since "reinventing the wheel" is a commonly known anti-pattern I've searched for some tools that can do the job effectively:
SymetricDS
Daffodil Replicator

Also there are some interesting frameworks and technologies that might come handy:
SmartGWT and SmartClient Framework
JavaFX

Business intelligence - concepts

Since the ancient times people knew the importance of knowledge. It is a well-known fact that information if used properly means power. Today we are overwhelmed by information. It is everywhere and we now posses the technology required to gather it. But to gain the hidden power we have to use some techniques to help us manage, understand and benefit from all the data we have. To properly use the information is essential for banks, insurance companies, medical facilities and many many others. The importance of managing information lead to the new IT science field called "business intelligence". In short it can be described as the way of gathering, analyzing and using information for business benefits (statistics, estimations).

The first mention in technical literature about "business intelligence" comes from IBM in the 50's where it is described as the ability to perceive relations between presented data in a way that helps us achieve our goals through actions. The data is processed by a computer system, analyzed and the analysis results are passed to people that have the knowledge to use them for a specific benefit.

Today's understanding of BI is a little bit different but the main idea remains. It is more described as concepts and methods used to improve decision making based on data provided by systems managing and processing data. Since BI is a fast growing field there emerged numerous sub fields that although use different methods and algorithms have something in common. They all process data created at a different time, places, systems and for a different purpose. They all focus on "secondary data processing". A short list follows:

• data warehousing (gathering and storing semantically similar data from different resources)
• reporting (presenting analysis of data in an easy-2-understand manner)
• OLAP (analytical online data processing)
• EIS (executive information systems)
• MIS (management information systems)
• data mining (finding patterns and relations in potentially unrelated data)

Data is usually directly processed by the application that uses or generates it. The main purpose of data in these applications is usually only of supportive nature that matters only during the execution of a specific task but once the task is successfully done a big part of the data is of no use. This is called "primary data processing" and once it is done the data can be thrown away or stored in a sort of data archive where we have to ensure additional security. This was the main idea of using data in applications.

But once the amount of data started to grow some people realized that by looking at the archived data they might discover some useful information that was hidden before. The problem at that time was that the amount of data required to get relevant information out of it was bigger than the technology could handle and it kept growing. Luckily in the 90's the technology caught up and we finally got into our hands the resources to process and take advantage of all the gathered data. Investments came into the recycling of old data, purifying it and using it for business improvement.

At first the same methods that were used for both primary and secondary data processing but it soon became clear that both approaches differ. The primary processing relied on transactions with the typical CRUD operations and main aim on maintaining the data consistent, secured and accessible at any time by multiple users. This is called OLTP (on-line transaction processing) and is typical for working with small amounts of data. Systems for secondary data processing don't typically require that amount of security and concurrency since they mostly only read data but in much more bigger chunks thus they require specific optimizations for these operations. The structure of the data also varies since primary data is application oriented (sales, finance, investments, etc.) but secondary data is more "object-centric" (customer, product) gathering similar data from several applications.

In the next article we will look a little bit closer on the approaches used to process such data.

Software engineering 12 - Proposals and estimations

Introduction
We discussed about the importance of communicating with the client and developing a software product. But we missed an important part which is: "How to get a contract ?". This feature requires loads of soft-skills, good reputation and even then don't expect crowds standing at your door that want to hire your services. And event after acquiring a client we need to make him "an offer he won't be able to resist" but that would of course bring us profit for that's what business is about. Creating time/costs estimates that can be fulfilled is essential as they are the first thing that every businessman looks at.

Creating offer
When creating a offer to the client there are 2 situations that you can encounter. You're either creating an offer for a completely new product or your offer concerns the modification of an existing already developed one. Clients hire software developing companies for 2 basic reasons:

1. they need to make modifications to the system that are too large or expensive in relation to time/money that they can't be part of the standard "system modification request" policy
2. they have an unmaintained product or they ended a contract with the original contractor

When creating an offer there are several steps that need to be taken (in order):

• make a record about the new offer
• declare responsibilities
• create the offer
• analyze and review the offer
• describe the communication protocols that will be used
• create estimates of the time/labor requirements

Sometimes offers can seem misleading when they don't include the ideas, materials and processes that were used for estimates generation. Businessmen don't usually conform to time/cost estimates at first if they are defined in longer time intervals or bigger sums. They have the feeling that things can always be done faster and cheaper and having the methodology and data used included in the offer helps convincing.

The content of an offer should be understandable, clean and exact in such a way the client gets a first overview of what is it he'll be paying for. Everything that is not clearly defined might lead to potential future conflicts that are very bad for your reputation and if you don't want to loose the client also to an unpredicted increase in costs due to satisfying his need. As people tend to say: "The client is always right." The basic items that you should cover in your offer are:

• extent of work/project
• labor required (ideally in "man-days")
• deadlines/milestones
• quality description
• resource requirements
• potential risks
• marginal conditions
• everything else to make the "people in charge" make adequate decisions

While establishing the project extents it is common to use structured and . Defining the extent of things we will implement (positive limits) and we won't (negative limits) is good to have for future negotiations on the price of  additional features and changes. You should always avoid abstract limitations and rather use some countable way of doing it (instead of "fast" use "server's response under 5s"). Try to be defensive in terms of not promising things you can't do but don't be too restrictive or you might not get the contract. Part of the positive limitation is a set of all the feature we ply to implement. To know the right balance is the know-how that only experience can give you. Nevertheless it's good to include clauses saying that some items on will be specified later on as part of specification/analysis. Resource requirements for development, documentation and things related to the construction process should be included for they are part of the overall costs. One crucial part of every offer is to declare some conditions that would apply to the client as the requirements for cooperation which is essential for every phase of the development process (requirements gathering, specification, user acceptance testing).

Estimates

Everything around proposals, creating offers and attempting to convince the client you're the right one to get the contract is making correct assumptions about time, costs and resources required. That helps to intrigue the client and avoid problematic projects or those that are "out of our league". This quality is one of the basics of a good IT manager and having a pocket full of techniques and materials that help is key to success. There are some guidelines that can be found on the internet, also articles and discussions describing their experience is helpful but what helps the most is own experience and a history of previous project with all the project plans, analyses, resource management history, etc. Creating a summary after each project that would contain the system characteristics, time spend on various phases and activities, encountered problems and risks, time/cost summaries and everything you find useful will soon result in an essential knowledge base. This summary should be based mostly on measurements (time, size, effort, quality) and usually project management tools have mechanisms to support this which makes it really easy to do (BugZilla, Jira, SVNStat, etc). The estimates done in the initial phases of the project are very rough and they are modified during the various phases of the project or as reaction to an unpredicted event but keep in mind that for the client the estimations given in the offer are the ones that count the most so a common practice is to keep a time/cost reserve that can handle such situations.

Conclusion

Creating an offer for the client is a complex process that needs the involvement of management as well as IT staff to create a realistic plan capable of successful completion. Correct estimates are what can help projects success of fail and measures need to take place to help create them. Having summaries of previous projects is the logical place where to start an archiving them is essential. Always remember that it's business and the main goal is to gain "profit" (cash, reputation, knowledge).

Software engineering 11 - Project management

Introduction
Project management is not directly part of the essential parts of the software development process. Is is more of a supportive part for it helps coordinate and manage the rest of the software development phases which means you can live without it but it would take a huge amount of luck and self-discipline to succeed this way on larger projects.

Project management is understood slightly different by various world class software companies:

• NASA (First begin by start planning the whole project and understand the problems, requirements and work that is about to be done. Technical approach needs to be defined by choosing an adequate life-time model, methods, activities and products. Then the project plan should be finished defining the relations with management, organization, estimates, time-planning. After that it's time for implementing the project plan and project management's role is to monitor, control and maintain the plan of software development. The last responsibility of project management is the projects closure after everything is done or when the project is canceled.)
• SYBASE (By Sybase the project management tasks are 3 consecutive actions. The project "initiation" which defines the problems and analyzes potential solutions. It is also responsible for creating a project plan. Then it is "execution" which aims at implementing the plan while monitoring and managing the overall progress. Finally there is "closeout" of the project.

Software management program control
This is an overall look at the requirements of  project management as seen by some "best practices":

• Software management planning should include
• definitions of product goals (establish functionality, criteria of success)
• how will the product be structured
• project plan analysis and its testing
  
• time estimates for the project (schedules)
• estimating costs ("costing the plan")
• frequent plan revisions based on the environment changes
• Implementation of the plan
• Methods for including changes into the plan and project
• Methods for achieving project goals (resources, quality, etc.)
• Coordination of the effort of various groups and individuals
• Ensure adequate quality control mechanisms to assure the high quality of the product

Project management is in essence strongly connected with all the various phases of the software development for it's the central management point (requirements, architecture, configuration management, engineering process, quality assurance).

Basic "howto"
The basic point where to begin is establishing a project organization that should be absolutely clear. Absolutely essential is to understand "what is it we're going to do" to avoid problems with doing wrong things or the good things the wrong way. The key to successful project management is to decompose the whole work into small and compact units for it's way easier to make a time/cost estimate on a small problem than on the whole one. Also creating an analysis and resource estimation/planning is more easy on small pieces. After the decomposition we need to assign available resources to he various units or set an order in which resources will be assigned.  During the latter phases we need to have tools and mechanisms to monitor and measure progress, report and archive so we can react to situations when something goes not as planned and we have materials for further usage (future planning, etc.).

Conclusion (tips for manager)
There are several hints for IT managers that can represent the whole idea behind a successfull project management:

• you must create and then maintain a plan in advance for an adequate long time
• you need to be clear about deadlines and 3rd party companies obligations
• your employees need always to know what to do in the next few days/weeks so they can plan their time effectively
• you need to pay attention to the effectivity of your employees and create a good working environment
• you need to have everything well organized
• you need to keep benchmarking the progress and keep the "economic" aspect in mind at all times
• you need to understand the system, its complexity and be capable of making decisions when the client pushes you to a commitment of extent/deadlines
• you have to keep in touch with the client, attend all meetings, keep detailed records of them and various discussions, hand over protocols, etc.
• you need to keep an overview and index of all problems, potential risks and of course a proposal on their solution/elimination
• most of all you must pay attention to whatever happens in the project and keep track of the extent of the project

Articles
SAFE Project Handbook

Checklists and templates
CxCheck_MostCommonSchedRisks.txt
CxCheck_CompleteListOfSchedRisks.txt
PDD.doc
CxTemp_top10risks.doc
Risk_mgmt_plan_template.doc
Status_Report.doc
Status_Report_simple.doc

Software engineering 10 - Maintenance

Introduction

Every software system an IT development company delivers is delivered in a state that should resemble the extent given by the contract the client and the supplier made (usually based on the requirements or specification). The system is then accepted by the client and being run in production fulfilling it's job. It is possible to assume that a product in regular usage contains only a small number of additional defects that were not yet discovered or unrepaired. Nevertheless applications created to order don't last the same their entire lifetime and new features are being desired. Small changes, errors that show themselves only after several months in production or when situation changes (more users/data to handle, changes in enterprise business process, etc.) lead to the need of the softwares evolution. Most of the times the "guys" that perform such actions are the developers of the original system for they possess the required knowledge base.

New releases

It is common that the company developing a product keeps involved even after the delivery of the final product by offering "maintenance services" to the product. Both parties, the developing company and the customer, benefit from such an arrangement for the customer can have changes done quite fast and the IT company has jobs for a small percent of its staff and money for low effort since they have everything ready (the environment, all the documents, knowledge, etc.). The most common scenario is that the IT company keeps continuously improving the product making regular releases of new versions (new features, requested changes) and some minor releases for requests of critical importance (bug/defect fixes, security vulnerabilities).

ISO/IEC 14764 specifies 4 main types of maintenance:

• Corrective (repairing found defects and bugs)
• Adaptive (keep the software updated with the current environment/infrastructure)
• Perfective (improve maintainability or performance)
• Preventive (detection and repair of latent defects before they become serious defects)

Maintenance as system development life-cycle

Maintenance of a software product is very similar to the overall software development process with all its typical phases but on a minor scope. Every new request that is to be implemented can be looked at as an individual software project with the rest of the system being the platform where we want our new product to be integrated. It's simple as that and the same techniques and facts we mentioned in the previous articles apply. All the new requests need to be documented, analyzed, solutions designed, implemented and tested.

Maintaining 3rd party product

Sometimes an IT company is asked or obliged to maintain a product that is not of their making. Such situations usually come when you're hired instead of an another company that was a supplier for several years and your projects are tightly coupled with the current infrastructure (some companies are only specialized on maintenance but in some cases creating a whole new product might result more effective than maintaining the existing ones).

In such cases it is essential to have:

• a complete specification to estimate the extent of the work that is being asked from you
• architecture and design documents which you can follow to keep a unified state of code/functionality
• clearly defined practices and processes to follow

• Those are the minimum requirements without which you should really think about rebuilding everything from scratch otherwise you might spend horrendous amount of time and money by simply trying to understand the current system you're maintaining. Also be careful for the changes you're implementing as part of maintenance tend to silently break the architecture piece by piece.

Conclusion

Maintenance is a very effective process with quite exact estimates that client and developer can benefit from if the one providing maintenance is the same subject that created the maintained product. On the other hand for an external subject maintenance can become a nightmare if the product developers didn't follow the commonly used practices of software engineering (documentation, manageability of the whole project and all it's phases, etc.). While maintaining a product the focus is on quality and performance of all the various maintenance aspects and it's hard to keep it all organized. Also many people are not keen on maintenance since it doesn't involve too much creativity for the changes done to the product tend to be of more cosmetic nature. That might be the reasons why maintenance leads to a somehow slow destruction of all the nice software engineering practices that were done while the main development as many people to "delete what they don't understand", get fed up with the project or simply don't feel enthusiastic about it. Believe it or not but after several years of maintenance software usually looks completely different (even it's internals) that the original specified product. And worst of all the documentation of the changes rarely last more than a year.

Software engineering 09 - Development environment

Introduction
Every software project that runs through the various phases of development requires a environment in which the development takes place. A well setup environment is one of the greatest factors that helps the project to success. As we could see on the LCO/LCA/IOC diagram environment setup is done at the very beginning of the project for it's the logical place where to start.

The basic objective of the development environment is to provide a platform where analysis and development can take part and which models the client's infrastructure where our final product will live for the rest of its lifetime. One might think that having one environment for development purposes is more than enough but they would be mistaken.

Motivation

The base motivation behind spending costs on establishing an development environment is the fact that the software product we're about to create will be used on a specific client's platform and will be integrated into his infrastructure. Usually simply developing at client's place is not an option for various reasons so we need to create a similar environment of our own where we'll be developing and testing the product. Also we need a place where we can present the final product to the client and where acceptance tests will take part before it goes to production. Having a single environment is not ideal for it can usually be used for only one phase (either development or testing, etc.) so the solution is obvious - create several environments with tasks attached so they server their purpose only. Some simple environments we might want to setup are:

• development environment (for development purposes, coding, prototyping)
• integration environment (simulates client's infrastructure and helps in testing the integration of the product with other technologies/services)
• testing environment (benchmarking, automated tests, regular builds and continual integration)
• (pre)acceptance environment (acceptance testing, presenting to client/users)
• production environment (final stage where we test the product in real action)

There is one part to the various environments people tend to forget and that's the social aspect of it. Part of every environment is having a comfortable working environment (furniture, privacy, space, good collective, team-building, etc.) and quality tools (hardware, literature, libraries, sets of 3rd party tools, automatization, etc.).

Continuous integration
Continuous integration is a nice feature to have in your development environment for it saves quite a lot of time by automatically running builds of the product and test scenarios on them. Otherwise such actions need to be performed manually which costs your programmers/testers valuable time. The basic actions of continuous integration are building the product, running automated tests and scenarios and publishing the results. The results need to be of course revised by a human being but by means of automatic warnings on errors/bugs/non-standard situations the time required is minimal.

We should note that the environment should also have mechanisms for every possible/relevant configuration units, should support ways of creating backups and their restoring, maintain an index of all the software backups, should be able to provide smaller or bigger delivers of the product in development, define processes for installation, modification and accessing and ideally support techniques for auditing.

The importance of backuping can never be stressed enough for whenever shared data, repository, hardware or people get corrupt you can always rebuild the whole infrastructure from the backups in an economic way.

Build process

When establishing the build process for your software you should make sure he has the following capabilities for it's the regular builds you'll use to create a supply for your client and presentations (no non-IT guy will be pleased to know the installation requires several hours of shell configurations after which a BSOD appears):

• create a supply (choose the finished and stable parts with potentially not all features that can work together on the targets environment)
• install the supply (make sure it sets up all the necessary parts so that the system works)
• create a supply for user/customer's installation (prepare the product in such way the client is able to install it)
• deploy a complete product
• repair a small bug/defect in a fast and economic way (sometimes "be bulletproof" meaning you needn't redeploy everything due to a small defect but rather be able to "patch" the already running system)
• ability to work with different platforms and configurations (application servers, operating systems, database system, etc.)

Conclusion

When developing a product for a client you should try the best you can to create an environment identical to the one your application will be running on. That way it's easier to test the product and also avoid unexpected defects caused by synchronization, resources or simply human factors. A daily build is nice to have for it helps monitor progress and testing. The supply system should be quite simple and as automatic as possible for it's nice to be able get everything running by 1 simple click. Finally backuping, logging and automatic reporting will save you quite some time when totally unexpected situations happen. Usage of advanced tools (Ant, CruiseControl, Hudson) should be part of your daily developers/IT managements life.

Software engineering 08 - Configuration management

Introduction
The topic of "configuration management" is closely related to SCM ("supply chain management") and its main goal is to systematically manage the configuration of a software product. The attempt of "configuration management" is to provide order and some rules that can ease coping with additional changes that might be required during various periods of the project lifetime and were not foreseen. One typical case is when clients realize that the software should have additional features or that it doesn't satisfy their needs. In such cases when changes in specification, design and implementation need to happen it is essential for both time, money and customers satisfaction to have a clearly defined process and means to perform such changes. A simple set of problems "configuration management" needs to handle is:

• ensure audit of all part of the system or software product
• ensure identification of all individual parts as well as the whole system
• make sure that changes don't break the product itself
• ensure a possibility of determining the state (past/present) of the products configuration

As for almost every aspect of software engineering there is also an ISO (ISO 9000-3:1991) norm that describes the requirements in a more detailed and formal way:

• audit and identification of every single software item
• audit and identification of the software product
• provide data to generate and update every single version of the software product
• ensure correctness of making changes to the product
• provide report about the state of the configuration
• updates of a given software item by more than 1 person

Put in simple words what "configuration management" needs to provide is versioning of the product and a clean transparent process of making changes to it.

Versioning

The basic way of assuring that we keep track of various changes to the software product is by keeping all the versions by some sort of backup after every change with a documentation describing the changes made. Of course backuping the whole system every time a change is made would be a very naive way of doing things so we need to decide which items of the software product we'll be version controlled and which not. Also we need to decide with those unversioned for we always require a way of knowing how did they change during time. The identification of the parts and the system is simply done by a timestamp or version ID every time a change is made. To avoid too much versions which would prove completely unmanageable we create a new version after a set of changes takes part.

Having various versions of a software or its parts is time-saving for work can be done on different versions of the product at the same time without colliding. Also having the possibility of rollback to a concrete version has its benefits for test-driven programming has proven that sometimes it's way easier to restore a backuped version and re-implement from scratch the new functionality than having to debug it. Another great advantage of version control systems usage is that they were all created in such a way they allow better cooperation of people on the project with some techniques of minimizing or solving the typical conflicts of "2 people working on the same thing".

Today versioning is easy-to-do for there are many well-tested version control systems that offer great functionality and are easy to use. Their usage has become an essential part of the development process and they are well integrated with other project management tools to help increase manageability and performance. One of the older ones were RCS (Revision control system) and CVS (Concurrent version system) which are nowadays considered a little bit obsolete. Still used among various mostly open-source projects is the popular SVN (Subversion) and some modern ones are Mercurial and GIT.

Although version control systems have lots of common they can be separated into 2 basic categories based on the type of "repository" they work:

• centralized
• distributed

The centralized version control systems have a central repository where everything is stored. Each user has it's own local copy of the repository's content where he makes all the required changes and these are then uploaded to the centralized repository to the main branch where of course conflicts need to be resolved. These systems place emphasis on synchronization, tracking and backups. Whenever there is the need of trying something completely different the only way of doing sow without affecting the main branch is to create a separate single one but this is quite unpleasant for the longer the side-branch lives the more conflicts emerge and merging becomes very difficult to do.

On the other hand the distributed version control systems are composed of various repositories owned by each user and sharing is done based on the principle of "web of trust". Changes are made by user to his own repository and others decide whose changes they want to download (the download of a change and its application are 2 separate operations). Nevertheless there is usually a central repository that gathers changes from other repositories. The emphasis here is on changes sharing.

Management of SCR (system change requests)

Today there are no much project where the classic waterfall model of software development can be used for changes are an integral part of the process itself. It is clear then that we require some techniques and tools to handle the incoming changes in an elegant way. System change requests are one constant in the development and as such there should be absolutely clear what it's lifetime is. A simple illustration can be seen on the next picture.

At the very beginning there needs to be a need for change for making changes out of thin air makes no sense at all. The need for change needs to be described and of course identified so we can keep track of it. The request should contain an unique identifier, detailed description ideally with connection to specification and connection to the version in the version control system for every change is related to some. The request is analyzed and reviewed. Based on the evaluation it is either rejected in which case the requester should be informed or is approved. Approved requests are assigned to a software engineer, scheduled, implemented, tested and finally reviewed to make sure we satisfied the initial request.

For management of SCRs there are specialized ticket-based tools like Bugzilla, Trac, JIRA, etc. that are well established and commonly used which helps incorporate such feature into your software project management. Thanks to these systems you can not only benefit from easy management but it helps show statistics and reports of the amount of changes, their approval/rejection, etc. which can be used as one metric of determining the quality and overall management/development performance as well as it can be used for future analyses.

Conclusion

Never underestimate the power of a clean and understandable SCM process. Version control of various software items as well as of the whole product will help you not only while development but also when you need to deploy the software to varying environments. Having the possibility of branching your product at development time is crucial for saving time while trying new ideas and merging things back together or reverting makes it extremely easy to manage even big projects. But as with anything remember the tools and techniques need to be used properly, otherwise they may cause more harm than benefits.

Articles

Little Book of Configuration Management

NASA Software Configuration Management Guidebook

Guide to software configuration management

Templates

NASA_SwCMPlanTemplate.doc

Software engineering 07 - Quality assurance

What's QA ?

Quality assurance is a nice term in software engineering that many people use to impress but not all of them understand "what's under the hood" of it. Some confuse it with "quality", others suppose it's a synonym to "testing" or "validation and verification". Let's explore these terms a little.

"Quality" is a typical buzz-word you'll hear on almost every management meeting and nobody ever takes some time thinking about it's background. People usually tend to take this as granted without thinking about what quality really means for that specific project. If we have a look at an official definition from ISO 8402-1986 it is probable that we'll end up more confused than before. Here's the definition:

"QUALITY: The totality of features and characteristics of a product or service that bear on its ability to meet stated or implied needs."

Simply stated it says "to have a satisfied customer".

"Validation and verification" is a complex process or possibly a set of activities which's goal is to determine if a specific artifact or system component fulfills all the demands. Validation is usually related to specification as it's main purpose is to compare the specification and final product if they really match. Verification is a more detailed process that makes sure things have been done exactly the way they should have. Simply stated validation is "building the right thing" and verification "building it right". The relation of V&V to quality is simple for V&V is a method of determining and measuring quality.

"Testing" was discussed in one of the previous articles so we should already a quite idea about how it fits into this whole "what is quality assurance problem". It is simply one of many ways of doing V&V. The results of testing are a clean metric about the quality rate of the product.

So "quality assurance" is a set of activities which's goal is to ensure the quality of a product or service by systematic and trustworthy means. As stated by a known art critic John Rushkin: "Quality is never an accident; it's always the result of intelligent effort." Although it would be nice to be able to assure 100% functional and reliable product it is impossible to achieve. What we can do is by means of "quality assurance" make this somehow more probable.

Why bother with quality ?
Simple answer: "Because quality matters." Quality assurance brings several important benefits to every project:

• it's is economically effective (more than 50% of extra costs come from the fact of creating a bad-quality product and to repair defects found by customers)
• it retains customers and improves profit
• it's an advantage against your competitors
• it is essential for your business to last
• it improves your reputation even world-wide which helps establish yourself on international markets
• it's "classy"

A nice scheme that presents the benefits of quality assurance is the "Deming's chain reaction" scheme which puts into relation the quality, productivity, business success and new jobs providing.

Achieving quality

To achieve quality the first thing you need to do is create and initiate an overall "quality program" which should define the policy of quality, establish a department responsible for quality assurance and of course have the management's blessing by showing quality is what they support and require. Then it's essential to create a plan (sometimes called "quality framework") to help the quality achievements. This plan need to take into account both cultural and technical aspects of assuring quality. It should count with:

• a clean metric to measure achievements (of people, software, process, etc.)
• establishing what is good/wrong/acceptable
• what should we focus on
• how to proceed
• what should we require

There are many standardized "quality assurance frameworks" that will help you but if you're not a big company then the investments into purchasing the framework might not prove that useful and we're not mentioning how overcomplicated some of them are (as many standards are). As a easy-to-follow model you can simply use the classical already mentioned "V-model".

After every quality assurance plan and gaining its results there is the critically important part of analyzing all the results and behaving adequately. It is essential to be self-critic, regularly check and recheck the project, run quality audits and of course keep revising even the quality assurance plans for if we pass a bad plan it tells us nothing about the quality of the product or process.

Conclusion

Quality assurance is something that distinguishes amateur projects form real software engineering. It's main requirement is that it needs to be well planned in advance. It's nothing that will just miraculously appear out of thin air by simply testing your product. The whole process needs to be as pragmatic as possible and we need to think about quality on every level of the process from individuals to the organization as a whole. The most important part is to keep checking everything again and again which is usually the only viable way of assuring the quality of the product or process.

Articles

ESA Software Engineering Standards - Issue 2

NASA Software Assurance Guidebook

Software Quality Assurance Process

NASA Software Formal Inspections Guidebook

Overcoming the NAH Syndrome for Inspection Deployment

Templates

Software Quality Assurance Plan Template

Software engineering 06 - Documentation

Importance

Until now in the previous "software engineering" articles I tried to stress the relevance of many documents that should accompany every software project. We talked about specification, use cases, feature lists, design and system architecture documents, even the documentation of the source code. But why are these so important for the process of software creation ?

The basic answer is "People tend to forget, documents don't !". Whenever you write something down and keep it somewhere where it can be easily found when needed of course then it should never happen that you are uncertain about something you discussed earlier. Every solution, every decision that have been made are archived and no one can deny that. For some this might seem a problem for this fact applies to both sides, the developers as well as the clients and managers and to be honest sometimes people would like to leave some stuff hidden. Anyway in an ideal world that should never be the case.

The documentation is not only to be some sort of "memory" that is archived and mostly never looked at again but it's main advantage is that it helps in communication among various stakeholders. You needn't spend extent time explaining what it is you have in mind but you simply point your finger onto a paragraph in the document an everything is clear as sunshine. The problem is given, the context is known.

Of course the "archive" feature of documentation is handy for various reasons:

• it is the basic source of information for developers, system administrators and users
• it helps in maintaining the product (in 2 years there will be almost no-one remembering what reasons lead to the current solution, what communication protocols among objects were used, etc.)
• it is essential for future planning, helps people learn from mistakes done in the past, is useful for time estimations

Classification

The documentation of the software engineering process can be divided into 2 basic categories based on the topics and aspects of the development process they cover:

• Process documentation
• Product documentation

The "process documentation" describes the whole process of development and maintenance of the software product. It should include plans, time schedules, milestone and task estimates, quality assurance documents, standards and best practices, all the important meetings and decisions, reports and many many more. To illustrate why are such documents useful imagine you are creating a time schedule with estimates for a new software product. If you've done a similar one in the past it is very likely that the time estimates will be quite similar hence it gives you a nice place where to start from. Reports can be used to determine if the estimations made at the beginning of the project matched the reality and the deadlines were met, the tasks difficulty was guessed properly, etc. On the other hand some of the process documentation is relevant only during the development and can be discarded after the project is finished (for example working papers meant for communication support between the management and developers like presentations, specific prototypes, etc.).

Product documentation is more user/developer related for its main purpose is to server as a manual of the system. There should be several types of product documentation based on the knowledge base of the potential reader. Also the techniques used should vary based on that. For unexperienced users the documentation should be as simple as possible with lots of screenshots, detailed examples of usage and scenarios that help them understand the most basic features of the product. Experienced users on the other hand usually tend to have a feature/function list with all their capabilities and usage description. We should also distinguish various documents based on the users role for an end-user should have a different documentation (usage manual, feature list) than an administrator (installation and maintenance guide) or developer (API description, design description, communication protocols).

A typical user documentation is composed of:

• introductory manual
• functional description
• system reference manual
• system installation document
• system administrators' manual

A typical system documentation is composed of:

• system requirements and description of their reason
• specification
• architecture and design
• source code and comments
• validation documents (test plans, test scenarios, reports, etc.)
• maintenance documents (known bugs, infrastructure requirements and dependence, influence on rest of the system, etc)

Requirements and form

The basic requirement for documentation is keeping its consistency during the whole lifetime of a software product for it's one of the basic representations of the system. Although various documents describe the system from various points of view it is essential to keep them all in consistent state (design should reflect requirements, code should reflect design, tests should reflect requirements, etc.) otherwise they wont play their role right and on large software products this can have a significantly deteriorating effect.

The documentation tends to grow off limits so using a good management policies is key to success. To help keep the management under control consider the following:

• use an unified and ideally standardized template system
• unified language (same naming conventions, names for modules/objects, etc.)
• logical document structure
• a clearly defined place and means of storing documents
• use automatic generation tools (in-code documentation extractors (Doxygen), full-text indexing tools, etc.)
• create a "navigation central point" (possibly web page, CMS or  a wiki-like system)

As for the documents structure it doesn't really matter which technique or tool you'll use (MS Office, PDF, UML diagrams, emails, issue tracking systems, database tables, etc.). What matters is to create a basic set of rules at the beginning of the development process and them enforce their following. A typical document structure depends on the content but here are some basic stuff that you should include into your documents:

• project identification
• document identification
• author, certifier
• document type
• version, version history and changes description
• distribution list
• confidentiality level
• abstract and keywords
• copyright
• list of notions and abbreviations
• table of contents
• index

Conclusion

Having documentation costs time and money but it's benefits outweigh. Another thing is that it is not sufficient to have documentation but it's also essential to have it well organized so whenever you need to find something you know exactly where to look. Practice shows that in a lot of IT companies documentation tended to be something of very low importance but since we're living in times when products change quite often knowing what is it we've done months ago proves to be more than effective from both, time and money, points of view.

Articles

NASA Software Documentation Standard

Software engineering 05 - Testing

Introduction

Software testing is a complex set of actions with one main goal which is establishing a belief that the software does only things it is meant to do. This is not it's only function but the most important one. Testing is of course also used for evaluating the performance of the software, its functionality, helps in verification of the system if it achieves the requested or at least acceptable results, etc. It is also essential for code analysis with the attempt to find all weak or buggy spots, potential future problems of bottlenecks. Also helps sell the product for a software that is proven to be 99.9% tested is more likely to be sold than the one without testing.

The importance and reason for testing has changed dramatically during time for in 1960s tests we used mostly for demonstration purposes. Their goal was to show the client what the software is capable of. Soon developers realized (possibly because some of the demonstrations didn't end as expected - Bill Gates' BSOD)  that testing can provide them with a way of finding defects in the software that were later debugged. Since 1990s the usage of testing is mostly prevention for as mentioned several times the earlier we discover a defect in the software the less money it costs to repair it. Although a defect is not always that critical it needs to be resolved.

Testing is a complex process that is not related only to code but also to all the aspects of software creation. Design assumptions and interconnection of objects needs to be tested as well as the GUI design/prototypes, functionality and specification (by reviews to verify we are really doing what the client asks).

Testing

Every single test should have one single objective that should be as specific as possible. In case we need to test 2 or more similar objectives then it is usually better to create individual tests or a parametrized test. Complex tests can be composed of several simpler ones by creating advanced test plans and scenarios.

The testing of a software is somehow tricky because to test a software in a way to assure its quality we usually require loads of tests and some advanced management is required. For this purposes we should always have a "test plan" that describes which parts of the system will/won't be tested, how will the tests be conducted (what tools and methods will be used), who is going to be in charge for the tests (setting up responsibilities).  Each test consists of 3 basic parts:

• execution (prepare the software for testing, setup context, etc.)
• testing (run the test)
• evaluation (gather statistics and inform about the success/failure of the test)

Every test or a set of tests should be associated to a "test case" which describes the conditions that determine if the test passed or failed. This is quite critical for every test should attempt to cover all the possible contexts and act accordingly. This is a common place for mistakes for every assumption about an impossible case that certainly will never happen needs to be based on some invariants that usually come from the design and implementation of the overall system. So a tip is to "never make assumptions about anything and if we want to do them make damn sure they are right and wont change in time".

Basic principles

When testing a software we should try to keep in mind the following simple facts:

• there is no such thing as "complete testing"
• to create a test requires experience and creativity
• testing is driven by potential risks (more important parts should be more tested)
• tests are based on the analysis, design and specification
• tests require time, money and motivation
• tests should be prepared and run at the right time
• keeping track of statistics and code coverage is as important as testing itself

The tests and their creation can be divided into 2 simple groups based on the knowledge base of the subject in charge of the creation. On one hand we have tests created by people that understand the software and are quite familiar with the implementation and how thing really work inside. This is called "white-box testing" and is usually done by developers themselves at the time of the implementation as part of the debugging process just to test if the piece of code they just wrote does what is supposed to do. The other approach is called "black-box testing" and logically it's the exact opposite. The test designer knows nothing about the code internals and all he knows is the interface and what it's functionality ought to be based on the specifications or application design. This sort of testing is usually done by professional testers, users or by other developers and is better in a way that the tester can't make any assumptions based on the implementation. One great benefit over "white-box testing" is that implementation changes don't break the test if the interface remains the same which is a useful feature.

As mentioned in the facts about testing one crucial aspect is the exact timing of tests meaning good time planning is essential for the tests to play they part in the overall process so we can benefit from these. Simply said if we start testing late then there's quite a big chance we won't test the software properly mostly due to time pressure because deadlines need to be fulfilled. On the other hand if we start testing too early there is a possibility we'll be testing an unfinished and unstable piece of software which would result in a waste of time, money and testers work. To know when to test what part is some sort of magic and a sign of a good and experienced IT manager but a nice hint is the "V-model".

To avoid problems with test planning we need to remember that tests are created and performed by humans which means that no tests can be absolute, no good testing can be done without adequate resources. Good testing requires excellent cooperation between testers, developers and most of all management. A common mistake is the belief that testers are the ones responsible for the quality of the product. This is not really true for they are usually under pressure from the management that schedules test runs and deliveries without taking into account the real-life project limitations. Also lots of companies don't really invest into qualifying testers or they rate their performance by the number of "bugs" they found which leads to their focus on small and not-critical problems just to earn enough points. Another bad testing practice is limiting tests to a specific area of interest (specification based testing, function testing, etc.) or by exactly specifying which atomic parts to test and how leaving little space for testers creativity. Over-detailed tests cause lots of trouble too for they are hard to maintain and usually redundant which leads to too long test runs.

Another thing that should be taken into consideration are the mentioned resources that management needs to provide for the testing to have effect. To create an adequate estimate is also a great problem and sometimes people tend to under or over estimate these. As an example there have been quite a lot of studies with similar results that say that the amount of users testing an application is related to the quality of the testing and that to have more than 5 users test your product is unnecessary for it doesn't provide such extra benefits.

Basic testing strategies

It is not easy to create a universal guideline for creating test plans and making thus testing effective independent on the type or extent of a project. This is probably the reason why are experienced IT managers so important for this is some sort of magic that requires lots of experience on various projects. Anyway there are some tips that might be a nice to follow for starters:

• Begin with the basic tests. Test with values you know shouldn't break anything just to verify everything does what it should. Most software products work for quite some time without breaking anything and Windows users are used to restart the service from time to time. Anyway you can blame Windows if thats your softwares main platform.
• Test slightly everything first. You should always run some preliminary tests first to make sure the whole system works as expected most of the time. If it does there is just a slim chance there's something wrong with the design which are the worst problems to solve.
• Run complex and detailed tests as soon as possible. If the software passed all the basic tests it's the right time to start testing in a more aggressive fashion. Create detailed tests based on the simple ones, create complex testing scenarios with contexts that are not so likely to happen. This usually discovers small implementation mistakes, bugs and possible weak points of your software.
• Modify test boundaries. After some time the amount of tests is more than overwhelming and their execution takes quite a lot of time. Also there probably is quite a lot of redundance for many complex tests do the same job as the a set of simpler ones. The elimination of unnecessary tests will improve the situation significantly.
• Exploratory testing. The most useful testing advice is to keep creating new tests all the time. A nice practice is let every developer create a new test or a set of tests on weekly basis. That way we can discover forgotten untested places.

Reporting

Testing would be almost useless without reports that record the test results as well as all the defects software has. These reports should have a standardized form which can be easily achieved by the usage of a bug-tracking system (BugZilla, Trac). The basic and simple rule is that every issue should be reported and ideally traceable by connecting it to the version of code currently tested. The issue needs to be reported in such a way that enables others to reconstruct and simulate it and most of all repair it. This can be achieved by appending logs, scenarios, detailed results, etc. Also a note describing it's relevance and effect it has on the system is advisable.

Automating tests and regression testing

Ever since testing became an integral part of software development people though of ways how to make the testing process automatic for several reasons. First people needn't to run the tests by hand every time something changed, tests could be reused and of course test runs could be planned in advance for times when the system was not overused. Today regression tests (running old test scenarios on new versions of code) is one of the basic ways to determine new changes are consistent with the rest of the system. This leads to the principles of "continuous integration" (smoke testing), configuration testing on various software/hardware, stress and load testing, etc. Some specialized tools can be used (for example Hudson) to do the job.

Conclusion
Testing helps create a good and high-quality product if the resources are provided. Many people think of testing as an easy click-and-see job for they don't see the experience and creativity required. Also it is important to attempt to have a high "test code coverage" (some norms require a percentage) but we need to keep in ming there is no such thing as 100% test coverage for we may never know what can happen and we usually don't test the platform and tools we use. Anyway even the testing software we use might be buggy!

Articles

What Is Software Testing? And Why Is It So Hard?

Little Book of Testing - Volume I

Little Book of Testing - Volume II

Software Test Planning and Management Guide

Checklists and templates

CxCheck_TestPlan.pdf

CxTemp_TestPlan.doc

IEEE_TestPlanTemplate.pdf

MIL-STD-498_SwTestDescriptionTemplate.doc

MIL-STD-498_SwTestPlanTemplate.doc

MIL-STD-498_SwTestReportTemplate.doc

SPAWAR_SwTestPlanTemplate.doc

SPAWAR_SwTestReportTemplate.doc