Thing which seemed very Thingish inside you is quite different when it gets out into the open and has other people looking at it

Monday, February 9, 2009

Shortcomings in project management are more of a threat to the success of the software industry today than technical issues ?


IT project failure has become a very common factor during past 10 years, most  IT projects today face the problem of getting cancelled their project before completion or abandon their  project in the implementing stage itself. Researches continuously shows that most companies have the difficulty of completing IT projects successfully .
“For 2004 results show that 29% of all projects succeeded (delivered on time, on budget, with required features and functions); 53% are challenged (late, over budget and/or with less than the required features and functions); and 18% have failed (cancelled prior to completion or delivered and never used). A staggering 66% of IT projects prove unsuccessful in some measure, whether they fail completely, exceed their allotted budget, aren't completed according to schedule or are rolled out with fewer features and functions than promised.” (Chaos, 2004)
This leads to the question why do IT projects fail despite of the all the technology improvement and enhancement, therefore author would like to discuss this matter with the recent IT project  failures and disasters.

·         2004 Avis Europe PLC enterprise resource planning (ERP) project was terminated its development after causing £30m loss.
According to the company statement it state that  they have abounded the project  due to the high cost and delays. (Thomas, 2004)
·         An IT phone system was launched in April 2003, two years late and £56 million pounds over budget. The 1.2 million existing cases could not be transferred on to the new system because it failed to work. (Charette)
·         Sydney Water Corp, billing system to automate customer information and billing for Australia's largest water provider is canceled in 2002, due to inadequate planning, numerous change requests, and cost and schedule overruns, which leads to the loss of $33.2 million. (Robert N. Charette)
·         London Stock  an exchange Failure- Effort to design new stock settlement system is scrapped in 1993, after seven years, because of overly complex and cumbersome design and poor project management, , which leads to the loss of $600 million. (Robert N. Charette)

 All of the above projects have failed due to poor time management ,cost management and poor planning . Therefore most project which fails due to high cost and delays because of the inefficiency in project management. Delays and poor management may substantially reduce the effective benefits and quality of a project. A good project management should be able to analyze the cost and the benefits and include the cost of project planning and implementation as a component of total cost which could calculate the total cost beforehand.
·         In 2005 US Justice Department Inspector General report stated $170m FBI Virtual Case File project was a failure, after five years and $104m in expenditures . (Frieden, 2005)
After analyzing all the factors which were affected to fail the VCF project these factors had a major impact on failing the project.
o   Unrealistic or unarticulated project goals
o   Inaccurate estimates of needed resources
o   Badly defined system requirements
o   Poor reporting of the project's status
o   Unmanaged risks
o   Poor communication among customers, developers, and users
o   Use of immature technology
o   Inability to handle the project's complexity
o   Sloppy development practices
o   Poor project management
(Robert N. Charette)
All these above factors comes under poor project management, project manager should be able to set up  realistic project goals , and estimate the needed resources appropriately . And for VCF project, it has a major issue on requirement gathering and controlling requirements , this can be handled in the project management stage itself, project manager should be able to control the requirements and properly manage the requirement management team. Therefore project manager should be able to manage risks even If there are technical issues, and should be able to tackle the situation smoothly.
From the above research on failed IT projects author come to the conclusion of poor project management is a key factor of a project life cycle and  shortcomings in project management are more of a threat to the success of the software industry today than technical issues.
Technical issue that a project face which can lead a project to fail can be categorized as
  • Inappropriate architecture;
  • Insufficient reuse of existing technical objects;
  • Inappropriate testing tools;
  • Inappropriate coding language;
  • Inappropriate technical methodologies;
  • Lack of formal technical standards;
  • Lack of technical innovation (obsolescence);
  • Misstatement of technical risk;
  • Obsolescence of technology;
  • Poor quality code;
  • Poor systems testing;
  • Poor systems integration;
  • Poor configuration management;
  • Poor change management procedures;
  • Poor technical judgment (McManus, 2008).
All these above difficulties can overcome by the technical team if the project management is strong and capable of managing the risk appropriately. And most of the projects rely on the decisions that were made in all stages of software development life circle which is a responsible of project management, and it is project managers responsibility to allocated the limited resources appropriately.
“Most public firms have written off failed IT projects in the past year, typically due to poor project management, according to a new report from KPMG.” (ElectricNews.net, 2002)


 
Bull Survey,(1998),Failure Causes.[online] Available From < http://www.it-cortex.com/Stat_Failure_Cause.htm > [Accessed on  19th October 2008]
From the  above statistics shows that  major causes of project failure during the lifecycle of the project are a breakdown in communications (57%), a lack of planning (39%) and poor quality control (35%). (The Bull Survey, 1998).Proper communications , project planning and quality control can be done using proper project management skills, this  reveals that the basic factors which leads to a project failure is on project management’s hand.
From the above facts author come to the conclusion of proper project management is  the key fact for a project to be successful .There are many more projects which have failed due to bad performance in project management which had lack of managements skills, such as project  planning, monitoring of
project progress, risk management,  as a results  most assigned project managers lack the experience in handling a broad range of problems that arise during the course of a project and leads to unsuccessful projects. Therefore author suggest effective project management with senior management reviews / involvement and proper time, cost, resource management skills can decrease the number of project failur

Thursday, August 21, 2008

Introduction to Data Warehousing

Data warehousing is a very useful process to support business and organizational decision making activities to store organizational historical data which can be later presented using a portal.

Data warehousing is mainly done in three basic steps

Data warehouse mainly depends on the Extraction Transformation Load process

Extraction – Extract the data from the source systems and converts them into a format for transformation process

Resources used for extraction task.

DW can use source system to get data to the system. Such as employee allocation information, employee human resource information, and financial information ect. These resources can come in many forms such as excel/csv/rdbms ect.

These data can be extracted to the warehouses using DTS packages or SSIS packages.

Transformation – Apply set of rules and conditions to the source data and transform filtered data to the data warehouse.

Set of rules which can be applied in the transformation tasks.

    • Aggregation

    • Filtering

    • Joining

    • Sorting

    • Pivoting -Turning multiple columns into multiple rows or vice versa

Loading – Load the filtered data in to the tables arrange them accordingly to the data warehousing schema with the usage of facts and dimensions. Facts store measures calculated, usually business process is represented by a fact table. Dimension tables contain component attributes, which represents descriptive information about a fact.


Data warehouse schema is put to data cubes and they are access by the internal portal

Useful Technologies for Data Warehousing

Data can be extracted through DTS Packages and SSIS packages.

Schedule tasked can be assigned to extract data daily using scheduled job using SQL server 2000/2005.

Data transformation can be done using views and stored procedures designed accordingly and executed those using DTS packages as well as SSIS packages

Loading can be done using data cubes architecture . Table data is loaded to the cubes using SQL 2005 Business Intelligent Studio which are arrange accordingly to the relevant DW schema. These cubes can be access in three ways

  1. Access through the reports which uses SSRS 2005 MDX coding and VB.Net custom codes.

  2. Access data through the portal which uses share point technology and .Net framework.

  3. Generating pivots using Microsoft Excel


Friday, August 1, 2008

Adding A New Cube To The Warehouse

This post explains the tasks which were done/followed when adding a new data cube to an already existing data warehouse. This more like a guide line to follow when you are creating a new cube to the system.

To add a new cube to the system table structure should be created. Following steps are taken to add brand new set of tables from the sources to data warehouse

1 Extraction process

  • Create relevant tables in extraction

  • Create views for all those created tables

  • Create the DTS Package to transform data from source to extraction

2 Transformation process

  • Create the relevant fact table to store the measures

  • Create the needed dimensions to store new attributes

  • Create views for newly created tables

  • Create SPs to apply set of rules to filter information and create calculations for the fact tables

  • Create DTS packages to call the SPs.

All stored procedures and queries can be written using SQL using Microsoft SQL query analyzer.

3 Cube Structure Creations (Loading)

Create cube for the relevant fact table including the needed dimensions using Microsoft SQL business intelligent studio(2005)

After cube structure is created following task should be followed

  • Adjust the dimension property values in the cube structure

  • Under the calculation tab create an MDX query in order to create the calculation.

  • Under newly created dimensions create attribute hierarchy and adjust the properties accordingly.

  • Process the cube.

4 Unit Testing

Last but not least Unit testing. Unit testing is a very crucial task in data warehousing. Since we calculate sensitive information. You can do the unit testing process using SQL coding to check the transformation process is properly done

  • Source to extraction testing

    • Test all source records are extracted to Ext Database

  • Extraction to fact table testing

    • Test fact table calculations are properly calculated.

  • Fact tables to cubes

    • Create a pivot to the newly created cube compare data with the fact table, according to the cube calculation.


Tuesday, July 1, 2008

SSIS Vedio

This is a vedio on creating SSIS packages it might be useful for all da SSIS beginers