IV&V Facility Research Program Results and SARP Results  

Advanced Search
Criticality Analysis < Verification & Validation < Results Home

Click here to complete a short survey. The results of this survey will be used to help us improve the research program and this website.

Click here to view research projects that had new research results added in the last 90 days.
Measuring Confidence in IV&V Products
Point of Contact Bojan Cukic
bojan.cukic@mail.wvu.edu
Dates January 2007 - December 2009
Problem It is understood that high quality software artifacts, such as requirements documents, design specifications, readable code, etc., increase the likelihood of IV&V analysts finding faults and omissions. This project will attempt to formalize the relationship between the quality attributes of software artifacts and the resulting effectiveness of NASA IV&V process. By complementing the ongoing collection of IV&V effectiveness metrics, our research conjuncture is that the "quality" of the artifacts submitted for IV&V has a major impact on IV&V effectiveness. If proven, this conjecture will allow for better IV&V planning and scoping, support mid course correction decisions and, possibly, allow measuring the confidence in the outcomes of IV&V. The intention of this proposal is to design the framework that will enable proactive use of the lessons learned. The proposed methodology will contribute to establishing clear and realistic expectations related to the quality improvements, cost effectiveness and overall safety/reliability impacts of IV&V efforts. Proposed research is related to several Return-On-Investment SARP projects, but it couples new research ideas with increased availability of metrics, and likely collaboration with Mars - Phoenix project.
Objective We propose to investigate a comprehensive set of attributes related to software quality throughout the lifecycle and correlate them with the measures of IV&V Effectiveness. The following is the list of project objectives:
  1. By May 2007: Data sources needed for project's quality (so called testability) evaluation, but not collected through IV&V EM program, will be clearly described and their collection suggested.

  2. By December 2007: We will define the attributes of testability for software artifacts related to different stages of the development life-cycle. The relevant quality attributes which had been defined by other researchers, including past and current SARP projects, will be included. We plan to demonstrate that these testability attributes can be used to predict IV&V effectiveness metrics. The experiment will include the data describing at least one of the currently monitored projects, Mars Phoenix (most likely) and/or PC GOAL II. Alternatively, we can use the datasets available in one of the open source repositories.

  3. By August 2008: Based on the initial experiments, we plan to identify and collect additional data and propose a model that would enhance project IV&V planning and scoping. This step will require a careful consideration of current practices and their adjustment with the metrics and modeling based approach.

  4. By December 2008: Based on the initial model developed in Objective 3, we plan to develop a thorough understanding of the correlations between project's (product's) quality evaluations and the ability of IV&V to achieve its goals effectively and reliably. We plan to validate that testability and IV&V effectiveness metrics can be used as a software risk prediction and mitigation tools, thus providing the tool for mid-course correction in IV&V scoping and project management decisions.

  5. By October 2009: Finally, we plan to correlate selected testability and IV&V effectiveness attributes with the outcomes of IV&V activities. The outcome of IV&V activities will be measured in terms of effectiveness indicators specified in the IV&V EM program. We will evaluate the strength of correlation between testability and IV&V effectiveness metrics, as these become available, in form of a "confidence index".
Results SAS_07_ExecBrief_MeasuringConfidence_Cukic.ppt
SAS_07_Tech_Pres_Measuring_Confidence_Cukic.ppt
Keywords Independent Verification and Validation Software Testing Software Analysis Software Metrics
Categories Quality Assurance
Quality Control
Quality Engineering
Software Reliability
Criticality Analysis
Management & Planning of V&V


Curator: Josh Stonestreet
NASA Official: Lisa Montgomery
 NASA Insignia NASA Privacy, Security, Notices