[ISN] "Blue Hat" summit meant to reveal ways of the other side

[InfoSec News]

http://news.com.com/Microsoft+meets+the+hackers/2009-1002_3-5747813.html

By Ina Fried
Staff Writer, CNET News.com
June 15, 2005

REDMOND, Wash. -- The random chatter of several hundred Microsoft
engineers filled the cavernous executive briefing center recently at
the company's sprawling campus outside Seattle.

Within minutes after their meeting was convened, however, the hall
became hushed. Hackers had successfully lured a Windows laptop onto a
malicious wireless network.

"It was just silent," said Stephen Toulouse, a program manager in
Microsoft's security unit. "You couldn't hear anybody breathe."

The demo was part of an extraordinary two days in which outsiders were
invited into the heart of the Windows empire for the express purpose
of exploiting flaws in Microsoft computing systems. The event, which
Microsoft has not publicized, was dubbed "Blue Hat"--a reference to
the widely known "Black Hat" security conference, tweaked to reflect
Microsoft's corporate color.

The unusual March gathering, a summit of sorts between delegates of
the hacking community and their primary corporate target, illustrates
how important security has become to the world's most powerful
software company. Microsoft Chairman Bill Gates himself estimated
earlier this year that the company now spends $2 billion a year--more
than a third of its research budget--on security-related issues.  
Security has also become one of the main themes of the company's
developer conferences, including last week's TechEd event, where
Microsoft pitched security improvements in Windows to 11,000
attendees.

Blue Hat was significant for other, less tangible reasons as well. It
provided a rare glimpse inside the netherworld of computer security,
where the ethical lines are sometimes fuzzy in the technological arms
race between network engineers and the hackers who challenge them.  
During the course of the event, each side witnessed for the first time
the inner workings, culture and psychology of the other.

"I didn't know if we were going to end up with this massively
adversarial experience or if this was going to be something of a
collaborative mode between all of us," said Dan Kaminsky, one of the
outsiders who presented at the conference. Like others in the hacker
group--many of whom are known as "security researchers" in their
professions--he noted that the relationship ended up being the
collaborative sort.

Still, in such a charged atmosphere, it didn't take long for emotions
to show.

Matt Thomlinson, whose job it is to help make Microsoft engineers
create more secure code, noticed that some of the engineers were
turning red, becoming obviously angry at the demo hacking incident.  
Yet as painful as the lesson was, he was glad to see the crowd of
engineers taking things personally.

Thomlinson frequently makes similar entreaties to the engineers on the
need for secure code, but he said his own lectures don't have the same
effect. "It kind of hits people up here," Thomlinson said, pointing to
his head. "Things are different when a group of programmers watches
their actual code exploited. It kind of hits people in the gut."

For two days, Microsoft staffers took these body blows repeatedly as
they learned of various exploits. On day one, several dozen
executives, including some of the company's most senior ones, were
exposed to this simulated wrath in a makeshift boot camp. Among the
participants were Jim Allchin, Microsoft's Windows chief, and Brian
Valentine, head of core Windows operating system development. The
second day drew about 400 rank-and-file Windows engineers, including
people who don't necessarily focus on security features in their
day-to-day work.

Allchin is not just any high-ranking software executive: In the
technology industry, his name has become largely synonymous with the
Windows operating system he oversees. A strong supporter of Blue Hat,
Allchin wanted the Windows group not just to hear about security
issues, but to see them as well.

"I'd already been through lots of days of personal training on the
tools that are used to do this," Allchin said about the work of the
hackers. "I personally wanted to really do a deep dive and really
understand from their perspective."

It was a relatively safe way to get the experience. In a world where
"white hats" are the security do-gooders and "black hats" are the
hard-core villains, the hackers at Blue Hat were hardly representative
of the dark side; if they had any pigment at all, it was no more than
a tinge of gray.

This could well be a significant reason Microsoft held the event--to
woo an influential group that has the choice of reporting security
flaws discreetly or going public with them. The software maker
routinely preaches the benefits of what it calls "responsible
disclosure."

To the researchers, Microsoft's motivation was less important than the
opportunity to meet in person with those who hold the keys to the
kingdom and explain why they do the things they do.

"It is rare that I can present to the people who are both responsible
for and capable of fixing the issues that I cover," security
researcher HD Moore said,

adding that he doesn't plan to change his practice of giving companies
30 days before going public with issues. "I still have no desire to
play e-mail tag with the (security response team) for a year for every
bug that I find."

But Moore did gain a better understanding of why it takes Microsoft so
long to create patches and said his impression of the people who
create the products have changed. "I still may not agree with their
security policies and how they handle bug reports, but at least I know
they actually believe what they are saying," he said.

Others agreed. "They are taking this subject seriously. It was really
cool to see," said Kaminsky, a security researcher who does work for
telecommunications company Avaya. "At some point, there was a shift at
Microsoft."

That shift began in earnest with a well-publicized memo written by
Gates on the concept of "trustworthy computing" in 2002. Security had
long been a concern at Microsoft, but the issue became imperative
after several high-profile attacks exposed the degree of its
vulnerabilities.

"The security faults we are seeing could end up bringing an end to the
era of personal computing," Kaminsky said. "The ability to customize
our computers is under attack from those who are customizing it
against our will."

It was this kind of impassioned rhetoric that won respect even among
some of the more wary Microsoft participants.

Noel Anderson, a wireless networking engineer on Microsoft's Windows
team, became suspicious as soon as he walked into the hacking
demo--and saw the giant wireless antenna at the front of the
auditorium.

Anderson decided that he should leave his laptop turned off, an
instinct that saved him the embarrassment of falling into the hackers'
trap, even though the hackers focused on a demo laptop. But under
different circumstances, he thought to himself, "I might have even
fallen for that."

As a result, Anderson and his team walked away with some concrete
ideas on how to make sure future versions of Windows are more
resilient to wireless attacks. He also left the room with a new
respect for the hackers behind the demonstration.

"It's not just a bunch of disaffected teenagers sitting in their mom's
basement," he said. "These are professionals that are thinking about
these issues."

The hackers, for their part, seemed equally impressed with the
technical knowledge of the senior executives they encountered.

At one point, researcher Matt Conover was talking about a fairly
obscure type of problem called a "heap overflow." When he asked the
crowd, made up mostly of vice presidents, whether they knew about this
type of issue, 18 of 20 hands went up.

"I doubt that there is another large company on this planet that has
that level of technical competency in management roles," Moore said.

Yet regardless of the mutual admiration, some tense moments were
inevitable during the confrontation.

Microsoft developers, for instance, were visibly uncomfortable when
Moore demonstrated Metasploit--a tool that system administrators can
use to test the reliability of their systems to intrusion. But
Metasploit also includes a fair number of exploits, as well as tools
that can be used to develop new types of attacks.

"You had these developers saying, 'Why are you giving the world these
tools that make it so easy to do exploitation?'" Kaminsky said. They
calmed down, he said, once the researchers were able to state their
case.

"We do regression testing in the real world of software development,"  
Kaminsky said. "If we say, 'This thing isn't going to break,' then we
need to test that. What these tools give is the ability to do this
kind of testing, to be able to say not just, 'We did the best we
could,' but 'We tried stuff and nothing worked.'"

Nevertheless, he understands why not all Microsoft developers were
satisfied with the explanation.

"I'm also sure Ford wasn't too happy with (Ralph) Nader's reports in
the late '60s," he said. "What do you mean you are telling people our
cars can blow up?"

By the end of the two days, those on both sides felt they had just
scratched the surface and were more than willing to meet again.

And executives such as Toulouse and Anderson said they came to a
better understanding of what makes hackers tick.

"We have conversations where we say an attacker might do this or an
attacker might do that. Now there is a face to some of those guys,"  
Anderson said. "They were just as much geeks as we were."

The next time a Blue Hat event is held, as promised by Microsoft,
Kaminsky said he would jump at the chance to return--assuming
Microsoft lets him back.

"I'll be there next time, no matter what," he said. "I have some
really interesting and devious plans coming up."