Qualities of good pen-testers
Taking care of business
Before I get into this post, I wanted to give you some updates on progress of other projects here at TS/SCI Security.
First off, I've been working on the OWASP Evaluation and Certification Criteria Project and hope to announce something very soon. Secondly, you'll want to take a look at today's post on The New School of Information Security book (Acidus also did a writeup). As a side-note, there was an interesting data breach recently announced, which makes information on breaches even that much more relevant.
Lastly, I wanted to announce the return of the "Day X: ITSM Vulnerability Assessment techniques". Expect a post everyday with new and relevant material.
Positive spin on pen-testers
Matasano posted last week on the Seven Deadly Pen-Test Sins, with follow-ups from two other blogs I'm aware of: Mike Andrews and BlogFranz. This should be a hot-topic!
At TS/SCI Security, we like to focus on the positive instead of the negative. Pen-testing, especially application pen-testing/review is an important topic to us. So here's our rundown of what we feel is important to know about pen-testing, as a response to what Matasano wrote.
Seven things you can do to improve your pen-test results
1. Time Management. If you're not already using GCalendar, RTM (with GCal), Twitter, Sandy, and Jott (thanks to Dennis Groves for some of these!) -- take a look at how these can help you manage your time better.
- Good pen-testing takes time. If you haven't gone through everything you need in #2, then you might need more time. What I've found is that pen-testers need to frame their pen-tests after good strategy consulting (1-2 days of a SWOT or similar analysis). If, like mentioned in The New School, the "basics" are not taken care of -- then help the client work on asset, configuration, and change management before doing a vulnerability assessment.
2. Utilize applicable MITRE resources. CAPEC (for runtime testing) and CWE (for static analysis testing, which may include reversing) should be utilized throughout. Be careful to only choose the relevant aspects of each (relevant to the application, not your skill or other criteria).
- My favorite MITRE resource is the Introduction to Vulnerability Theory, which Marcin and I spoke about at ShmooCon in our talk on Path X. It may take some extra time to spell out exactly how the vulnerabilities were found using this method, but it will help future assessment work, including repeat client business.
3. Work with developers. Assuming that the dev team is already doing Continuous Integration (or something like it), Fagan inspection, using the V-Model (in the requirements phase), continuous-prevention development, integrating concepts from the Microsoft SDL (or similar secure SDLC), and automated integration testing -- then get involved right away with a software walkthrough with all of the key players.
- Standard C and assembly code is difficult to reverse the design from, but they are becoming more easy to reverse. In the case of Java, C#, and C++ -- UML can be extracted to elicit the design of the application. Using other tools such as Klocwork K7, Fujaba, or IBM Rational Rose on the UML diagrams will possibly provide faster program understanding. Left with the source code, GrammaTech CodeSurfer also aids understanding, but open-source tools such as Doxygen can also be of use.
4. Automate development-testing. While web application security scanners (or fat application-based fuzzers) collectively only typically find at-most 30% of the bugs with a large amount of false positives, there are plenty of developer tools that don't have these problems.
- Developers should be made aware of integration testing with Canoo WebTest. Business analysts, customers, and test managers can submit table-driven test cases using FitNesse (a wiki that allows for collaborative test case design), along with additional tools such as HtmlFixture. I've spoke to the benefits of Dependency Injection before, as well as automating Continuous Integration both inside of the IDE as well as during the application builds.
- Automation is good for developers and the modern tools surrounding test automation are extreme improvements over last decade's technology. However, test cases and test-first strategies still find only as much as half (possibly up to 70% or more) of the bugs. It's good to combine exploratory testing with scripted (i.e. from a test case) testing. Allot some time with a buddy and explore the application as it wasn't meant to be run after learning as much as you can from the developers and their automated tests.
5. Peer review your work. This one is easy. Find a friend and have him/her check your work. The more eyes you get on your pen-test and assessment work, the less likely you're going to miss something.
- Consider a good workflow approach such as the open-source Review Board: Online Code Review Tool or the commercial Atlassian set of project management software, including JIRA, FishEye, and Crucicble. Many projects such as Adium do code review based on email notification of new source code revision control changes.
6. Stay up-to-date on both process and technology. Read forums/blogs/mailing-lists, trade journals/magazines, and books. Attend conferences. Keep it cheap and continuous if possible.
- I try to read every security-related book that pops up on Safari Library and ACM's access to Books24x7. Audible/Amazon are certainly other resources, especially for non-technical solution learning. Checking yourself at least once a year on a certification or re-certification program will make strides in your professional development.
- Using GReader and sharing daily using it's sharing features and OPML with your colleagues is important. Get involved in conversations on forums, IRC channels, blogs, and mailing-lists is likely the largest win here.
7. Go back to basics. Set a minimum bar of maturity with clients before you pen-test or do an assessment. The BITS Shared Assessments Standardized Information Gathering questionnaire is a nice start, especially combined with Visible Ops, ITIL, COBIT, ISO27K, PCI-DSS, PABP/PA-DSS, ISECOM, and OWASP information on process.
All of the above is going to go a long way towards improvements to what pen-testers and vulnerability assessors do. There's very few good certifications out there for what we do, and it's difficult to measure what we do. Put the cards on the table up front with your clients and teach them your methodology and approach.
blog comments powered by Disqus