tssci security

What makes a solid security program?

In my most recent post, I identified the direction and state-of-the-art in application security. We all know of the importance of application security in today's environments. However, finding out where to fit application security policies and programs into an overall security program (or organizational security plan) is as difficult (or more difficult) than integrating mandatory regulations, compliance standards, secure enterprise architectures, and many other risk management activities.

Building a continually improving security program is an important and common topic. For many CISOs and other directors of security programs -- this has been their day job since they earned their titles. There still exists huge gaps between IT/Operations, Application Development, and Information Security Management organizations and how they work together. There are gaps in communication between departments, and even within departments. The challenges of finding and retaining talent are not unique only to appsec, as suggested in my last post.

I've only spoken about building a security plan once before on this blog, but it's a popular conversation making the rounds. securitymetrics.org (the blog, mailing-list, Metricon conferences, and book) resurfaced a lot of my interest, as well as the work that Mike Rothman did with the Pragmatic CSO, Michael Santarchangelo with his book and the SecurityCatalyst blog/podcast/forums, and numerous others.

Not all security programs and bloggers have picked up on these resources. Take Creating a Solid Security Program from Accuvant's new blog called Insight from Kirk Greene. He appears to be familiar with some of the above resources, but I think there is a lot more out there. After you read my comment (which never got "approved"), be sure to check out the new material I've been reading on the state-of-art in information security management, especially including the human element.

Comment gone wrong #2

I think what you wrote here is a great example of a vulnerability management program, but not a security program. Even then, it's actually more operational (like a compliance initiative) because it gives little strategic or tactical advice.

Starting with awareness is probably the worst way to build a vulnerability management or security program. Maybe we just disagree, but I'd like to see some evidence or metrics demonstrating that this technique has any value, if you can point me to the literature.

Capital planning based on current or mock Strategy Maps and Scorecards/Dashboards is really the first step for building a security program. It is often best to first work with risk management (an operational activity) that can feed metrics up to the strategy, although this should be done along with compliance, regulatory requirements, and potential liability factors. Risk assessments, especially ones done with data classifications, can be the tactical metrics to pull into a risk management report. Simple risk assessments can be done using business tools such as 5 Forces, PESTEL, and/or SWOT anlaysis -- although in security we have various others including FAIR, FMEA, and PRA.

I also like the concept of drilling down another strategic metric platform via Enterprise Architecture, in particular an Enterprise Architecture Blueprint (such as the one from Gunnar Peterson). Enterprise Architecture can bring metrics down to the operational level with security policy and certification standards. These can be turned into server and application hardening standards at the tactical level.

Finally, asset/inventory management is another strategic activity that can be conducted to build a proper security program. When combined with the risk analysis data, asset management will provide guidance on where to scan & patch, pen-test, and perform exploit development activities at the tactical level. These tactical procedures can then provide more metrics up to risk management, and back again up to more strategic activities.

On second or further iteration, a balanced scorecard can easily be created to include compliance metrics (operational) along with a strategic direction (suggested as a strategy map). The balanced scorecard could then include metrics from incident management, which in turn could feed back into risk management and liability factors. SABSA could be used to build a governance program to keep the capital planning and security program alive and running with the rest of the business. Additional qualitative metrics based on organizational development and organizational behavior could be included in a hybrid platform such as business scorecards very easily, including Six Sigma metrics such as Voice of the Customer, et al. Simple, isn't it?

Your notion of using Application Security Scanners in a vulnerability management program disturbs me -- especially in the way you have suggested it. Maybe you're not familiar with these tools or how an application assessment is best performed to today's standards.

First of all, the surface coverage for even the best app scanners is 94%, with many getting less than 1% surface coverage. Even IBM/Rational AppScan was only showing 74% surface coverage using modern link extraction application drivers.

Secondly, the false negative rate of app scanners is approaching 92%, often more. The false positive rate varies between tools, testers and apps, but I've seen figures as high as 40%. App scanners must be properly configured and utilized by an expert in order to be effective at all. Even then, black-box app scanners need to be combined with static analysis and manual expert review for a significant majority of applications falling under "most-risky" data classifications such as PII (PCI-DSS, HIPAA, state performance auditing, etc) or financial data (SOX, GLB, et al). Even middle-of-the-road risky data classifications (e.g. proprietary information that has yet to be patented) should probably have more done to them than a simple black-box app scanner.

When I say manual review + static analysis, I really mean it. The automated tools pay for themselves by the amount of time saved -- but can never be used alone. Security review tools that implement static analysis techniques, such as Fortify, Ounce, Checkmarx, Parasoft, Grammatech, DevInspect, AppScan DE, Coverity, Klocwork, and SciTools have better false negative rates than black-box scanners, but much worse false positive error rates. FN is usually between 65-85% (the tool FAILS to find vulnerabilities this often); FP is 85-99%, you'll often see more "vulnerabilities found" than lines of code averaged across apps. This is why manual expert review with full-knowledge remains the best application assessment technique.

I don't mean to harsh on you too hard, but it does appear that you need to do more homework before making prescriptions for building a security program -- let alone a vulnerability management program. You seem to be capable of providing this information accurately (based on your last blog post and the great blogroll you've setup so far), so I expect better out of future blog posts.

Aftermath and reasoning

The consulting companies that I work with (and other colleagues, often consultants from other consulting companies that have been on the same or similar engagements with me) have all taken a strong interest in building trusted advisory adjuncts to the "too busy IT manager" or Mascot CISO/CSO. We have to in order to remain relevant and respected. However, I've always viewed consultants as "the colostomy bag of a very ill organization". Fix the organization and the technology advancements (or whatever else is needed) become agile and sustainable.

Rafal Los recently had me on his 31337 Spotlight: Andre Gironda for his Digital Soapbox blog. BTW - Thanks Rafal -- hope you and nearly everyone else are having fun in Vegas right now! There are a few links which may have got lost in my nonsensical chatter, so I wanted to specifically point them out. I said:

I like the idea that I can use my hacking skills for good and cause organizational change through discovery of `organizational management and behavior <http://en.wikipedia.org/wiki/Category:Organizational_theory>`_. A real "hack" to me is to take a `disfunctional organization <http://blogs.bnet.com/ceo/?p=1462>`_ and turn it into something awesome.

There are very few state-of-the-art resources on organizational theory combined with information security management. Allow me to point you to the few that I'm familiar with and highly recommend. After you check them out, you may find yourself coming to similar or related conclusions as I did with the above comment.

I have at least one more of these "comments gone X" posts, but the next ones should both begin and end on more positive notes. If you have any suggestions of comments you've seen from me that you would like to see turned into a blog post, let me know!

Posted by Dre on Tuesday, July 28, 2009 in Security and Work.

Blackhat USA 2009 / Defcon 17

It's that time of year again, where we all come out of hiding and meet in Sin City to cause nothing but trouble. The brave venture out into the scorching hot sun during the day and some even dare tempt the waters at Rehab. The rest of us wait until dark, with the neon lights flickering in our eyes, with nothing on our mind but money and skin. As we wander like zombies from club to club, night becomes day and day becomes night, we keep going -- amazingly, ready, for the next round of mental exploitation. Now... where did these baseball cards come from?

See you all in Vegas!

Posted by Marcin on Monday, July 27, 2009 in Conferences and Security.

Appsec industry trends - looking forward

Recently, it has come to my attention that industry people I respect (and vice versa) have desired me to re-post some comments I've made on other blogs.

It's also high-time that we at TS-SCI/Security begin writing again. I can tell you that since March (our last post), Marcin and I have been involved heavily in our day-to-day work at client-sites and other community efforts/projects.

A lot of new research is going to begin to become available from BlackHat/Defcon. It's just that time of year where everyone starts to share their work with others. While we can't exactly reveal everything that we're working on quite yet, be sure to check in for updates. I have been begging Marcin to post something on an HTTP-related argument we got into about the Post/Redirect/Get pattern, as one example.

Comment gone wrong #1

There was some interesting discussion lately on the OWASP News Podcast, in particular, Podcast 32. This is the first News Podcast that I missed (I was on a plane at the time we recorded), and having just listened to it -- I certainly think it's worth your time to listen to.

This particular News Podcast set off a blog post from Jeremiah Grossman where he says OWASP Podcast #32 pulls no punches. I attempted to comment, but the comment eventually disappeared -- perhaps Jeremiah didn't appreciate my insights. Others did, so here it is:

on the appsec market maturity and potentiality prediction -- i rate [discount black-box appsec SaaS] as low-value, and in the future, will continue to be low-value.

selling discount app pen-tests hurts infosec management as a whole because you're trying to tell ciso's that they can buy some freedom for $25k/yr (or whatever it is). in reality, they need to spend millions of dollars over several years.

discount app pen-tests need to go out of style. here's why: because the middle-ground and potential high-value comes from partnering with a trusted adviser (i.e. an appsec consulting company), or attempting to retain this talent in-house (which most companies -- including Microsoft who built lists of individual talent to target -- have pretty much failed).

every BPO (business process outsourcing) expert knows that the ideal is to avoid "discount" shops and focus on real partnerships, but don't give any single one partnership everything.

for example, attempt to retain 20% of your appsec program internally ASAP (this does take time -- don't expect it to happen overnight), while outsourcing initially 20% to one minor company (e.g. Gotham Digital Science, Aspect Security, Denim Group, Matasano, Independent Security Evaluators), then adding a bigger company (e.g. Accenture/McAfee/Verisign) for another 20% to take over the smaller company's 20% if [expectations are not met -- or major changes occur, such as buyouts]. The next step is to figure out a balance of adding more consulting companies somewhere in the 40-80% range, while growing your internal talent.

investing in this model is extremely expensive and extremely difficult to manage. ciso's are having problems finding/retaining talent, drafting RFI's, reading RFP's, following up on references, and deciding who is really talented and how that talent applies to the applications in their appsec programs. most can't or won't even draft an appsec policy.

[low-bid/low-value app pen-test houses, especially SaaS-based ones] convolute and diminish the returns that are necessary to build or even start an efficient appsec program. that's EXACTLY what Andrew van der Stock was trying to say.

if you want software security ROI, go read Sadbury, Soo Hoo, and Jacquith's "Tangible ROI through secure software engineering" or follow any of the work that Steve McConnell has done, which this referenced paper was based on.

if you want to keep selling the idea that your McDonalds solution is the bread-and-butter of modern appsec innovation... best of luck to you. there's plenty of analysts, whole appsec consulting businesses, bloggers, and podcasters that are all saying that a) you're wrong, b) you sell a one-size-fits-all solution to companies that "don't get it" which almost forces them to stay in the "don't get it" mode near-permanently, and c) the jury is out and the case is closed: appsec consulting is the correct path and one-stop-shops that do one-off, cheap app pen-tests are so 2008.

Aftermath and reasoning

My comments were due in part to actual recent industry analyst research, so they were not unfounded or inappropriate. More to the point, they were factual and unbiased.

Chenxi Wang, Ph.D., Robert Whiteley, and Margaret Ryan of Forrester Research published a report entitled TechRadar For SRM Professionals: Application Security, Q3 2009 Application Security Comes Of Age Despite A Slowdown In Security Spend. The date on the report was July 18th, 2009.

In the report, several technologies were evaluated, including:

  1. Application scanning
  2. Application security consulting
  3. Application security SaaS
  4. Penetration testing
  5. Protocol testing
  6. Software protection
  7. Source code analysis
  8. Web application firewall

These topics and research are not new to our blog, where we have discussed many of them. Take these examples:

Posted by Dre on Saturday, July 25, 2009 in Security.

Virtual appliances for the security professional

Virtual Infrastructure Security Facts

Virtual Appliances (VAs) have several advantages over Live CD distributions. They are easier to enable persistence and customize (especially for real performance in a VM, instead of via a bootable ISO). It's easier to take snapshots that represent a "point-in-time" to rollback configurations -- or prevent security scanners from running into loop or crash conditions. Cloning and templating can have significant advantages in terms of agility for testing and scaling architectures, in addition to aiding changes and repair processes.

Microsoft (including the freeHyper-V Server) and VMware (including the free ESXi) are the major players for hardware-VMM server virtualization, with the FOSS project, Xen, being prominent in some other product implementations.

Both VMware and Microsoft have their own disk formats for importing VMs (aka "Guests") on to their Hosts (aka Hypervisor or Virtual Machine Monitor -- VMM). There is also a third, open format called OVF (or Open Virtualization Format).

  1. Microsoft: VHD (Virtual Hard Disk)
  2. VMware: vmdk (virtual machine disk)
  3. Open Virtualization Format: ovf

Sometimes, one-off scenarios will utilize tar, zip, or rar files to distribute VMs or encapsulated VMs, but this is becoming more and more rare.

Virtual Appliances

A Virtual Appliances is a pre-packaged VM. Normally, a VM is just like a new machine -- no OS, no nothing. Virtual Appliances come with stuff, and usually only require booting into a DHCP-enabled network, where they self-configure themselves and become available via a web interface for further interaction.

You can find VAs at the following sources:

For those of you still using the outdated OSI model (i.e. you stupid network security geeks, j/k ;> ), here is a general layout of what is available for you:

Certainly, if you haven't read or seen Chris Hoff's various recent presentations, then you're going to screw this up. However, anyone with even a few weeks of virtual infrastructure experience will understand the application of the above VAs in a virtual infrastructure environment.

VMware is very useful for fuzz testing (as seen with Sulley and other frameworks which include interfaces to VMware monitors), and full-state or kernel debugging (as seen with Syser, the replacement to the classic SoftICE), but this is more often for the VMware Server/Workstation products, not their Virtual Infrastructure products (i.e. ESX, ESXi, Virtual Center, vCenter Server, and vSphere).

Many ISOs are moving to VAs.

Many demo-ware and software evaluations are moving from standalone installs directly to VAs (i.e. demo the new app on the new OS at the same time!).

Take these examples outlined in the next sections for a test drive.

Pen-test VAs

Would it be nice if you could setup a perfect pen-test environment, save it, and then clone it a bunch of times in order to tweak one specific thing and then run all your tests in parallel (say, with different credentials). Well this is exactly what Pen-test VAs are going to allow you to do. One machine: 4 web application security scanners.

Or better -- run DRS (VMware's Distributed Resource Schedule), which will automatically move VMs around contended Host resources. Say you have four physical machines, all with a dual-core 2.2GHz proc and 3GB of memory. Now say that you're scanning some client machines in far away places (with constant ISP bandwidth churn on both ends -- and in between). Let's pretend you have this setup:

If any of you know what CloudAV is... think what CloudWASS would look like. I call it "WhiteRockSec", which is... "like WhiteHatSec, but on Crack".

Of course nobody has built these VAs yet. In the meantime, you can use these two VAs to accomplish something similar:

  1. OWASP Live CD VA
  2. InGuardians Samurai Web Testing Framework

WAF VAs or as I like to call them: VA+WAF

VA+WAF is a Virtual Appliance that includes a WAF. To those of you who don't love my humor, you're bound to definitely hate me for flipping the script on this marketing terminology.

Because network vendors (F5, Citrix, Breach, Cisco, Barracuda, Imperva, et al) really like to sell expensive appliances, it's likely that they aren't too keen on the idea of selling a software-based VA that is equivalent to their mind like an ISO (anyone remember the presentation on how to reverse-ISO a Netscreen IDP onto cheap PC hardware?). So you don't see too many of these around yet.

I did happen to find these two though:

  1. Microsoft IAG 2007 Virtual Machine Trial
  2. Security Enhanced Web Application Server with mod-security

AppDev/AppSec VAs

Again, there really isn't much here yet.

Microsoft has:

  1. Visual Studio Team System 2008 VSTS Hyper-V Image (Trial)
  2. Visual Studio Team System 2008 TFS Hyper-V Image (Trial)
  3. Microsoft Pre-release Software Visual Studio 2010 and .NET Framework 4.0 Community Technology Preview (CTP)

Note well that the last link above, for the VSTS 2010 pre-release, has the VA in "vmc" format. "vmc format" was from Microsoft's older product. Searching the Microsoft Download Center for vmc or vhd both have great results, but hopefully Microsoft will standardize on VHD or OVF. For now, you can convert in many ways -- including the latest tool from Microsoft, the VMC to Hyper-V Import Tool.

Integrating AppSec with the above VSTS and TFS tools is relatively easy. For those not familiar with FxCop, StyleCop, and CAT.NET -- you certainly should be. TFS has some great built-ins for Governance that apply equally well between quality and security. The TFS Team Blog has some decent postings on topic, not directly to security yet (but probably in the future). I'm working on additional ideas, heavily borrowed from the Microsoft Process Templates and Tools development center -- and from watching how Microsoft uses TFS with their new MPT toolkit.

Security folk such as myself might want to just load Source Insight (or the Microsoft Express Editions) along with using the command-line CAT.NET or possibly SharpDevelop until Ounce O2 is widely available.

For Java, you can search the VMware Appliance Directory, but I found nothing useful. Currently, the easiest and cheapest way to get JEE AppDev/AppSec going is to use EasyEclipse. There is a commercial equivalent called Yoxos that also sounds very promising. I think most of us would be flying blind without a few Eclipse plugins such as Classlocator, Jupiter, Flow4J, IvyDE, FindBugs, and PMD. Build server ISOs such as Buildix would be wonderful to turn into a VA.

Again, us security folk would probably stick to Source Insight and/or SciTE along with the command-line versions of FindBugs and PMD. Static analysis tools are slowly turning to be out of vogue these days... so YMMV.

Summary

Learning Virtual Infrastructure is going to take some time, but the payoff is worth it. In no time, you'll be turning your minimally-equipped Security Operations Center or appsec group into a real infrastructure to fear.

Download the hardware-VMMs to "whitebox supported" hardware (note: this doesn't always have to be on an "official list" from the vendor). Try both the evaluation versions (Microsoft Windows Server 2008 R2 Beta with Hyper-V Role enabled ; VMware ESX and vCenter Server VA) and the free ones (Microsoft Hyper-V Server 2008 R2 Beta ; VMware ESXi). Download a few VAs in various formats and learn how to import and start them. You're on your way!

Posted by Dre on Wednesday, March 18, 2009 in Defense, Hacking and Security.

Web application security incident handling

I thought I'd take a moment to post about some web security tools I use pretty often, which help as a security consultant when responding to various web hacking related incidents. These tools have helped me write my own scripts whenever I'm in a jam and need something good and quick to do the job.

Application Log File Forensics: The Hard Way

The first thing a security professional or administrator usually think of when handling an application security incident is to check the logs for the applications, databases, and other application-tiers involved. Often, these logs are either on the servers that run the applications themselves, or possibly in a central logging location. If a certain attacker tool can be identified from the log files (or other sources such as full packet-capture), then it may be of interest to run that exact same tool against your own application-under-target (preferably in a mocked-up lab or test environment, if it mirrors production well enough).

The most popular web servers, Apache httpd and Microsoft IIS, do create local log files by default. According to most compliance regulations and standards (e.g. COBIT, HIPAA, GLB, PCI-DSS, FISMA, EU Directive on Privacy and Electronic Communications, ISO 17799/27002, CA SB1386 and similar), logging must be centrally located, or may have other required provisions. This may include application-layer information, such as the log information from Apache and IIS. It may be very likely that your organization already has centralized logging where this information is available.

If centralized logging does not exist, it may be a good time to start up a project to enable it. The Apache Cookbook, 2E, is the best place to go in order to configure httpd to start sending syslog information. It's about as simple to add "ErrorLog syslog:user" into the httpd.conf file, but this only logs error messages, not authentication/access_log messages. The book gives two prescriptions, one using "AccessLog "|/usr/bin/logger" combined" if your OS supports the logger command properly. The other is to run a custom log message through a Perl script, as seen below:

CustomLog |/usr/local/apache/bin/apache_syslog combined

#!/usr/bin/perl
use Sys::Syslog
qw( :DEFAULT setlogsock );
setlogsock('unix');
openlog('apache', 'cons', 'pid', 'user');

while ($log = <STDIN>) {
    syslog('notice', $log);
}

closelog;

Microsoft IIS will need to go through the Event Log, which can be converted to syslog messages using a third-party software package such as Snare or MonitorWare Agent. If IIS logs can also be converted to w3c standard log format, then Apache log analyzer tools such as AWStats could also be used. W3C also has their own log analysis tool that also does HTML validation, called the Log Validator. These may be useful to run following your own scan of the application using the same or similar attacker tool, as they will not only point out where in your application the scan/tool covered, but also where you may have the most errors or lack of quality/security controls.

The book Practical Information Security Monitoring also makes some suggestions for log collections, including the use of Sawmill or Splunk to sort/search log messages and gain further information and detail. There may also be further adjustments you will want to do at the application (or other tier) layer, such as logging POST data. We discussed logging HTTP referrers on our old post: Using Google Analytics to Subvert Privacy. Practical Information Security Monitoring talks about Oracle audit logging, but there is also a detailed article on Pete Finnigan's blog on Oracle forensics and UKOUG. At the recent BlackHat DC conference, David Litchfield gave a talk on The Forensic Investigation of a Compromised Oracle Database Server, which may also be of interest (once the slides are available). There are also some new books coming out on the topic of Oracle Forensics in the next few months / year.

Web Application Incident Handling: The Easy Way

Most of the logfile "digging" takes time, even when consolidated and using expert tools and analysis. There are some very easy approaches that we've come up with, or seen others using and talking about. These tools integrate well at the HTML and Script layers.

Over a year ago, Mario Heiderich started the PHP-IDS project, as a way to build protection and monitoring capabilities into PHP applications. Several side projects spurred up as a direct result of the incredible work that was put into PHP-IDS, mainly its default_filter.xml regular expressions. This XML file of regular expressions provides capabilities to detect a vast range of attacks, including XSS, CSRF, SQL Injection, Directory Traversal, Local/Remote File Execution, DoS, and Information Disclosure. Part of the success behind the PHP-IDS project, was the constant testing and attacking of PHP-IDS regex filters, which can be reviewed extensively in this sla.ckers.org thread. More info on PHP-IDS can be found in the PHP-IDS FAQ.

Romain Gaucher, wrote Scalp, an Apache log analyzer in Python, which leverages PHP-IDS' default_filter.xml to detect attack strings in logs. I've used Scalp on numerous occasions, including a recent attack attempt on tssci-security.com. By nature, Scalp cannot examine POST content because Apache logs do not contain POST data. (See PHP-IDS or mod_security for those purposes)

Simply use Scalp by running it as follows (keep in mind there may be false positives with regards to the attack type, though it is very good at pulling attack queries from the log):

./scalp.py --log access_log --filters ./default_filter.xml \
--html --tough --exhaustive

Arshan Dabirsiaghi recently released OWASP Scrubbr. Scrubbr works by detecting input data in a specified database that does not match up with a specified AntiSamy policy file. Because Scrubbr uses an AntiSamy policy to validate data, does not mean it necessarily detects XSS in your database. Note, one does not require AntiSamy to be implemented in an application to use Scrubbr. Using Scrubbr, you have the capability of validating each and every column capable of holding strings of every row of every table in a database.

Together, Scalp and Scrubbr make for excellent web application security forensic tools. Scalp can help detect attacks in Apache logs, and Scrubbr can help you clean your database of content that does not match your site's policy.

Posted by Marcin on Monday, February 23, 2009 in Security.

« Newer entries — 2 — Older entries »

blog comments powered by Disqus