[ISN] GAO: Security accreditation program a tough sell

[InfoSec News]

http://www.fcw.com/article92750-03-27-06-Web

By Michael Arnone
Mar. 27, 2006 

The federal government's program for testing and accrediting the
security of commercial technology has not been proven a success,
according to a report by the Government Accountability Office.

The National Information Assurance Partnership (NIAP), which is
sponsored by the National Security Agency and the National Institute
of Standards and Technology, was created to make it easier for
agencies to find products that meet basic industry standards for
security.

NIAP officials are responsible for implementing the Common Criteria
Evaluation and Validation Scheme, a rigorous set of security tests
that adhere to international standards. Officials provide technical
guidelines to commercial laboratories that conduct tests on the
products vendors submit. Once approved, a product is listed on the
NIAP Web site [1].

Unfortunately, agencies often find that the products they need are not
on the list or that only older versions have been accredited, GAO's
report states.

The program has other problems, auditors said. Nearly 10 years after
NIAP debuted, vendors still don't know much about the evaluation
process. And the number of qualified validating experts has dropped in
the past year, which could lead to delays in evaluations.

On a more fundamental level, NIAP program managers have not
established metrics by which to measure the program's effectiveness,
GAO's report states. For example, they have not collected data on the
findings, flaws and fixes that resulted from NIAP testing.

The NIAP accrediting process does provide some benefits to the
organizations that use it, the report states. It can improve agencies'
confidence that products will work as promised, and vendors can fix
flaws identified during the independent testing and evaluation.

The process can also make life easier for vendors and agencies because
it allows a broader range of international products, the report
states. It can also improve the processes vendors follow when
developing new products.

The report made two recommendations to help remedy existing problems.  
The first would have Defense Secretary Donald Rumsfeld order NSA and
NIST to develop workshops for agencies and vendors participating in
the NIAP program, the report states.

The Defense Department should also think about collecting, analyzing
and reporting metrics on how effective NIAP tests and evaluations are,
the report states. The metrics could include summaries of findings,
flaws and fixes.

Priscilla Guthrie, DOD's deputy chief information officer, agreed only
partially with the report's first recommendation. In a response letter
to GAO, she agreed that improving awareness and training is important.

However, she added that both NIST and DOD have cut support for NIAP to
fund other priorities, making it impossible to allot extra money to
such efforts.

DOD should instead direct partner vendors, evaluation laboratories and
industry associations to create workshops using existing resources,
Guthrie said. They should also bring in help from outside
organizations, she added.

She agreed fully with the report’s second recommendation. She said
NIAP has been collecting such metrics since 2004 and is developing a
template for an end-of-evaluation report that will review all changes
to products and vendor procedures throughout the evaluation process.

[1] http://niap.nist.gov/cc-scheme/vpl/vpl_type.html