[ISN] Microsoft Turns to External Patch Testers

[InfoSec News]

http://www.eweek.com/article2/0,1759,1750841,00.asp

By Ryan Naraine 
January 12, 2005 

Looking to improve—and possibly speed up - the creation and release of
software security patches, Microsoft Corp. is implementing a closed
beta program for external testing teams.

The formalization of Redmond's new Security Update Validation Program
clears the way for external patch testers to get "limited and
controlled access" to security updates ahead of public release.

The goal, according to company officials, is to provide a small number
of dedicated external evaluation teams with access to the patches to
test for application compatibility, stability and reliability in
simulated production environments.

Stephen Toulouse, program manager at the Microsoft Security Response
Center, told eWEEK.com that the external evaluation program was
implemented to add a new level of quality control to the engineering
process.

"We've always maintained that the most important thing is to make sure
the patches are of a high quality. A faulty patch is worse than no
patch at all. This program speaks to that commitment," Toulouse said.

He made it clear that the outside testers had no access to information
on the vulnerability addressed by the patch.

"They're evaluating the updates in a private, closed-lab environment.  
They are required to sign an NDA [nondisclosure agreement] and they
don't ever know what the patch is correcting. They're simply
simulating a real-world deployment in a lab environment and looking
for potential problems," Toulouse said.

"The end result of this program is higher quality updates for
customers to help ensure timely and effective deployment of patches."

According to Debby Fry Wilson, a director in the security research
center, the external testing teams were selected from trusted MVPs
(Most Valuable Players), ISVs and managed customers who were capable
of mimicking patch deployments in a lab environment.

"They had to make a heavy commitment to provide a dedicated evaluation
team and to restrict the use of the update to the test environment,"  
Wilson said.

"Based on customer feedback, one of the complaints we've dealt with
was that security updates had problems with application compatibility,
reliability and stability. We've done better in the last year, but we
can always improve the engineering process," she said.

It's a Catch-22 situation for the software giant as it struggles to
balance the need for glitch-free updates rolled out in a timely
manner.

Security experts have long criticized the company for being slow to
address critical software flaws. eEye Digital, a security research
outfit, maintains a section on its Web page that features Microsoft
patches that are long overdue.

On the other hand, Microsoft has also had to cope with the
embarrassment of having to recall faulty patches.

In 2003, a buggy patch from Microsoft even opened the door to the
widespread exploitation of a vulnerability in the Internet Explorer
browser.

"We're always looking at ways to create updates and get them rolled
out to customers quickly. But, the testing is an important part of
that process. With this program, the external evaluation teams aren't
looking to make sure the vulnerability is fixed. They're testing to
make sure that when the update is deployed across a network, it does
not break existing applications," Toulouse said.