[Dataloss] Brief analysis of "Analyzing Websites for User-Visible Security Design Flaws"

[Atul Prakash]

Thanks for your comments. You may want to see the copy of the
presentation and the videos from our presentation today at the
symposium we will be posting - plan is to do it tomorrow.

Irrespective of the quibbles one may have with the study (and we
disclose many limitations ourselves - that is the nature of research),
the key point we want to make is that there is substantial scope for
improvement in bank's web sites and we make specific recommendations.
What we are hoping is that bank sites will become both easier to use
and more secure for their customers as a result of this study.

We welcome other studies that look at more recent snapshots of bank
sites. It would be great if there is a finding by others that the
problems we observed have gone away.

We will post info on the presentation and videos at:

http://bankwebsecurity.blogspot.com



-- Atul


On Fri, Jul 25, 2008 at 11:05 AM, security curmudgeon
<jericho at attrition.org> wrote:
>
>
> After being provided a link to the original paper and reading additional
> comments, I wanted to follow-up to my original post [1] with more thoughts.
> If you want the slightly more technical review, search down to "methodology
> review". The paper in question is "Analyzing Websites for User-Visible
> Security Design Flaws" by Laura Falk, Atul Prakash and Kevin Borders [2]. I
> strongly encourage more security professionals to provide peer scrutiny to
> security research coming from universities.
>
> As was pointed out, the research was done in 2006 (testing in Nov/Dec) but
> the results are just now being published. Three people working on a study on
> 214 web sites should not take that long to publish. To wait so long in
> publishing research on a topic like this, one must question if it is
> responsible, or more to the point, relevant. In the world of high end custom
> banking applications, my experience consulting for such companies tells me
> that many will do periodic audits from third parties and that these sites
> get continuous improvements and changes every week. One of the web sites I
> use for personal banking has changed dramatically in the last 12 months,
> making huge changes to the functionality and presumably architecture,
> security and design. The results of a 2006 audit of that site are probably
> most irrelevant.
>
> As with most research papers, the lack of publish date in the header is
> annoying. The abstract does not mention the 2006 to 2008 time gap between
> research and publication either. This time difference is seen almost
> immediately in the citation of Schechter et al, regarding people
> "disregarding SSL indicators". The current releases of several browsers,
> most notably IE7 and Firefox 3 make pretty big shifts in how the browser
> handles and warns about SSL indicators. Each browser is considerably more
> paranoid and will throw a warning over more discrepencies that each would
> have ignored in previous versions.
>
> On page 1, Prakash et al list the criteria for the categories of "design
> flaws" they examined. As expected and mentioned in my previous post, the
> design flaws they examined are not necessarily a vulnerability, and often
> times do not put customer data at risk or they require additional
> requisites to be exploitable. To look at one of their design flaws as an
> example, consider the following:
>
>   Presenting secure login options on insecure pages: Some
>   sites present login forms that forward to a secure page but
>   do not come from a secure page. This is problematic because
>   an attacker could modify the insecure page to submit
>   login credentials to an insecure destination.
>
> This summary of a design flaw is problematic in that it makes several
> assumptions and/or does not fully qualify the attack vector. First, to
> "modify" an insecure page being served from the bank to the user's client
> (browser), the attacker would have to compromise the server (making this
> attack moot) or conduct a Man-in-The-Middle (MiTM) attack. I assume the
> latter is meant since the implication is that an attacker could not
> effectively MiTM attack a page wrapped in encryption (SSL).
>
> It is interesting that the lack of SSL encryption is chosen as a design
> flaw with the notion that manipulation of an insecure page is the
> preferred attack vector. Such an attack is considerably more difficult to
> conduct compared to other threats (e.g. SQL Injection, Privilege
> Escalation) and would essentially target a single customers. Many large
> applications serving hundreds of thousands of users makes this trade-off
> of mixed security pages for performance reasons, as the overhead of
> encrypting all traffic can be costly.
>
> Later in the paper when the team attempts to better define this design
> weakness, they say:
>
>   Consider the case where the customer service contact information for
>   resetting passwords is provided on an insecure page. To compromise the
>   system, an attacker only needs to spoof or modify the page, replacing
>   the customer service phone numbers with bogus numbers.
>
> Web pages can be spoofed regardless of the transport, so the presence of
> SSL encryption means little to nothing. If his team is implying an
> attacker "only need [..] modify the page", that would require compromising
> the server or performing a MiTM attack. Again, this is not a trivial
> attack by any means and in the latter, would affect one customer.
>
> While this is only one of five design flaws Prakash's team looked for,
> consider the third example which is the exact same design weakness:
>
>   Contact information/security advice on insecure pages: Some
>   sites host their security recommendations, contact information,
>   and various other sensitive information about their site
>   and company on insecure pages. This is dangerous because
>   an attacker could forge the insecure page and present different
>   recommendations and contact information.
>
> This is the exact same issue as #2 in the list but just makes the
> specification of the content on the page. Factor in that issue #1 will be
> more prevalent in large organizations but a non-issue in smaller ones and
> the criteria of five design weaknesses gets cut down from five to four,
> with one that is likely not to be seen on some of the sites tested at all.
>
> The paper quickly summarizes their findings before going in detail, before
> concluding "Overall, only 24% of the sites were completely free of these
> design flaws, indicating that some of the flaws we identified are not
> widely understood, even among institutions where security is critical."
> This assumption and conclusion is dangerous and irresponsible. The
> implication that the presence of one or more of these flaws is indicative
> of the site not understanding the threat is presumptious. With the example
> given above about the high overhead of encrypting all content, some of the
> "design flaws" may be business decisions and acceptable risk.
>
> Prakash et al begin to demonstrate their lack of understanding of
> client-server relationships and the transport mechanism for different
> protocols. The following paragraph from page 2 immediately calls his
> team's technical competence:
>
>   One of the most interesting design flaws we discovered is the
>   presentation of FAQs and contact information on insecure pages. In
>   the past, FAQs and contact information were usually sent through the
>   mail to the customer. It is not generally recognized that this
>   information should be protected. However, when this information is
>   presented online, the user becomes vulnerable to socialengineering and
>   offline attacks as a result of the information being displayed on an
>   insecure page.
>
> Prakash's contention that unencrypted content delivered from a web server
> to a browser is somehow different than when unencrypted content is
> delivered from a mail server to a mail reader is silly. If an attacker has
> the ability to MiTM attack a person, it isn't going to be limited to HTTP.
> Sending that contact information via mail will result in a user deleting
> it or maybe storing it in a folder. The first time the person needs to
> contact the bank, they would check the web page for the contact
> information. If said information is not available, it now further burdens
> the bank as they may call a generic number and get transferred around
> several times. This adds to customer frustration and causes bank employees
> to spend extra time dealing with a customer that could have called the
> correct number to begin with.
>
> Prakash's team goes on to make more assumptions or not fully understand the
> importance of how web clients behave. Without getting into a full discussion
> on the philosophy of e-commerce sites adding mechanisms to invalidate
> client-side vulnerabilities, the general notion that it should be done if
> feasible seems reasonable. In this context, feasible means that it doesn't
> overly burden the bank web site, does not impact performance and is
> generally transparent to the end user. One example of this is a Cross-frame
> spoofing issue that made it trivial for an attacker to use a phishing attack
> to MiTM attack MSIE 6 users [3]. Web sites can add a small bit of javascript
> to help ensure that browsers load their pages in a new frame and essentially
> mitigates this risk. This is a good example of how many banks were helping
> protect customers, even though the vulnerability was in the customer's
> software, not the bank web site. Prakash's team claims:
>
>   Our work is similar in that some of the flaws that we consider impair a
>   user's ability to make correct security decisions. However, our work
>   differs in that the cause is not poor or confusing client-side
>   interfaces. Instead, the flaws originate in poor design or policy
>   choices at the server that prevent or make it difficult for users to
>   make correct choices from the perspective of securing their
>   transactions.
>
> While a mismatched SSL certificate used to be virtually ignored in some
> cases, new versions of popular browsers now behave differently in how they
> alert users, giving them the ability to more easily make correct choices.
> Claiming that this research is not impacted by "poor or confusing
> client-side interfaces" is misleading. While the older browsers were not
> necessarily confusing, they handled some situations regarding establishing
> trust poorly.
>
> The next area of technology Prakash's team doesn't seem to fully
> understand is vulnerability scanners. In the paper his team says:
>
>   Network scanners, such as Nessus [11], and application-level
>   website scanners, such as AppScan [17], can be used to analyze for many
>   configuration and implementation bugs, such as use of unpatched
>   services and vulnerability to cross-side scripting or SQLinjection
>   attacks. As far as we are aware, the design flaws that we examine are
>   currently not identified by these scanners.
>
> Both Nessus and AppScan will identify several vulnerabilities that
> directly relate to the design flaws outlined. Both will give warning over
> invalid or expired SSL certificates, AppScan will warn about mixed-mode
> security pages and neither will perform tests for some of the design flaws
> listed (#3, #4, #5) because no scanner in the world can do it.
>
> Methodology Review:
>
> On page 8 (of 10), the team gives very brief descriptions of their testing
> methodology. The lack of description or their testing methodology undermines
> significant portions of the research. For "Break in the Chain of Trust", the
> paper says "Under no circumstance should an insecure page make a transition
> to a securitysensitive website hosted on another domain, regardless of
> whether the destination site uses SSL." This is an arbitrary 'rule' that is
> not widely accepted by anyone including the banking industry. Many web pages
> are designed to act as portals that link to additional features. The 'rule'
> as quotes from the paper would force large bank organizations to consolidate
> all web resources on a single domain. While that may be nice, it simply
> isn't feasible to many businesses, especially ones with a large organization
> that includes multiple companies. Linking from http://bigbank.com/ to
> https://regionalbank.com/ is perfectly acceptable and should use proper SSL
> certificates and technology controls to help ensure the user ends up on the
> correct page, loaded directly by the browser.
>
> The second design weakness studied was "Presenting Secure Login Options on
> Insecure Pages". The paper explains their methodology as ".. searched each
> web page for the string "login". If the string was found, we searched the
> same page for the strings "username" or "user id" or "password". If the
> string .login. and .username. or .user id. or .password. were found on the
> same page, we then verified whether the page was displayed using the HTTP
> protocol. If this was the case, we assumed this site contained the design
> flaw." The key word here being 'assumed'. There are scenarios where the
> above methodology could easily generate a false positive. Even back in 2006,
> there were trivial ways to more easily determine the use of HTTPS with
> certainty.
>
> The third design weakness studied was "Contact Information/Security Advice
> on Insecure Pages" and is perhaps the most technically lacking testing
> method one could perform:
>
>  We searched each web page for the string "contact", "information",
>  or "FAQ". If those strings where found, we checked whether
>  the page was protected with SSL. If not, then we considered it to
>  contain the design flaw.
>
> The mere presence of these words on a site do not mean they are in the
> context of listing bank contact information. While 'contact' will
> frequently link to a 'contact us' page, looking for 'information' or 'FAQ'
> is absurd.
>
> In the fourth weakness, "Inadequate Policies for User IDs and Passwords",
> the team openly admits that their methodology may produce "optimistic"
> results and that they had no way to verify their results "without
> generating an account on the website". Heaven forbid they find a couple
> hundred students at the university to participate by logging into their
> personal banks and checking this in more detail. That extra effort would
> have made this portion the only positive and accurate test. From the
> paper:
>
>   Our count could be optimistic; some sites may require strong
>   passwords without stating an explicit policy. We had no obvious
>   means of verifying this without generating an account on the website.
>   Our count could also be conservative for sites that have poor
>   policies resulting in weak passwords. Thus, our results for this design
>   flaw should only be taken as a rough estimate of the extent of
>   this particular problem.
>
> As before, the fifth design weakness was extrapolated using a glorified
> 'grep' of the web page, analyzing proximity of a few keywords and then
> verifying the hits above an 85% threshold. And as before, this testing
> methodology makes huge assumptions about the wording on the page, does not
> positively account for HTML formatting that would impact the 'distance'
> between words (especially in pages with frames) and does not begin to test
> the functionality (see page 9, section 4.5).
>
> Finally, the paper attempts to interpret the results of this poorly
> conceived and improperly tested study. Table 1 on page 10 says that 30% of
> the sites tested were affected by "Break in the chain of trust" but gets
> contradicted (clarified) in the first paragraph of the results:
>
>   With automated tools, such as the one used in our study, false
>   positives are possible. To the extent feasible, we manually examined
>   the results to eliminate false positives from the reported data.
>   Our break-in-chain-of-trust data had a significant number false
>   positives. Our automated tool reported about 30% of the websites to
>   potentially use third-party sites in an unsafe way, but only 17% were
>   found to do so without giving some sort of notification to the user
>   about that transition.
>
> Having such an admittedly large margin of error on the results based on
> their own methodology should be an eye-opener in regards to the integrity
> and accuracy of the results. Despite this revelation, the next paragraph
> immediately cites the table with the 30% number and begins to make
> conclusions based on the bogus numbers. They go on to further explain
> their primary tool for gathering information (the wget tool) may not have
> retrieved all of the information needed to properly assess the site.
>
> Despite the weak methodology, 44% error rate on at least one test and
> admitted errors, Prakash et al go on to say "We found that 76% of sites have
> at least one design flaw." Such statements are certainly not factual or even
> statistically correct based on the research presented.
>
> - jericho (security curmudgeon)
>
>
> [1] http://attrition.org/pipermail/dataloss/2008-July/002565.html
> [2] http://cups.cs.cmu.edu/soups/2008/proceedings/p117Falk.pdf
> [3] http://osvdb.org/4078
>