From: Brewis, Mark (mark.brewis@eds.com)
Date: Thu Jan 09 2003 - 06:00:39 EST
Ron,
Thanks for your comments. You've raised some very interesting points, which
I will try to cover. I only have a superficial understanding of the ICSA
process, so please correct me on any incorrect assumptions.
>>We were discussing the merits of Common Criteria versus ICSA especially
for
>>firewalls. The discussion ended with a basic understanding that both were
>>needed.
I agree. This is a perfectly valid conclusion, and reflects the fundamental
differences in the rationale behind CC and ICSA evaluation. Without wanting
to get into detailed descriptions of CC Evaluation (because I'm not sure
this is the correct forum for that discussion, and I'm certainly not the
best qualified to do so,) CC is best understood as a detailed assessment of
a product at a milestone. To this end, detailed analysis of the development
environment, code, documentation and product can be undertaken.
>>The reason was that the Common Criteria evaluation materials didn't allow
>>for constant real world penetration tests against known vulnerabilities.
>>While the product was under "contract" for evaluation by ICSA labs, any
new
>>vulnerabilities would be also tested. Common Criteria evaluations don't
>>have a mechanism for this type of continuous testing.
This isn't quite correct, although it is substantively so. The scope always
existed for Ad Hoc testing, although the trigger for an ad hoc test had to
come out of the testing undertaken. The evaluation process comes out of a
requirement for stability - hacking boxes wasn't seen as meeting core
requirements on thoroughness or on the reproduction of results. Flexibility
and reactivity were sacrificed to meet these aims. There is far more to it
than that, but it is a good starting point.
In addition, it is perfectly acceptable to write a test script (by which I
mean a paper record that details what you do, how you do it, the expected
result and any deviation from the expected result. None of that perl stuff
here, thank you very much) that uses an automated tool - Nessus or ISS for
example. Load the most recent signatures on the day you test, and your
records detail the tool, version, signature date and time. At that point,
you should be fairly up to date. The problem lies in the testing window,
which may be a considerable time before the product is Certified.
>>This is important from a point of cost. Many of the tests for a product
to
>>get certified under CC and ICSA are the same. For most areas CC goes much
>>further than ICSA.
Without knowing ICSA very well, I can't comment too much here, but the CC
can be extremely detailed, and does, in general, go much further. The point
is that CC evaluations try to identify which vulnerabilities are relevant to
a particular product before testing, so many of the ICSA tests may not in
fact be carried out in CC if the in depth analysis that goes into CC shows
that the vulnerabilities being tested for under ICSA are not relevant to the
product. Also, the CC tests will include some that are as a direct result
of the in depth analysis of the design and source code and so will test for
vulnerabilities that are not known to anyone else not directly involved in
the CC work so there is an element of 'future proofing' in CC.
>>The problem is that a vendor has to put out real money
>>for both certifications. I would much rather see that the vendor only
need
>>to certify their products under one set of rules.
The schemes are complementary, but I think they serve different purposes.
CC is as much about the process of development as it is about the end
product. It looks to see if the product has been created in a methodical,
documented, detailed way. The important thing is to look at what was
evaluated in a product, to what level, and then to see what wasn't
evaluated. The Security Target is the critical element.
>>To that end Mark, is it possible that this draft method will cover the
>>missing testing from CC that ICSA delivers?
I'm not sure that it can close the gap between the testing phase of a CC
evaluation and the Certification of the product. Evaluation is a monolithic
process. Set it in motion and it trundles along in a straight line until it
stops. ICSA, I'm sure, also has a window between the end of testing and
certification, but I would assume it is much shorter. A more reactive
process.
Hope this is of some use to you,
Mark
Mark Brewis
Security Consultant
EDS
Information Assurance Group
Wavendon Tower
Milton Keynes
Buckinghamshire
MK17 8LX.
Tel: +44 (0)1908 28 4234/4013
Fax: +44 (0)1908 28 4393
E@: mark.brewis@eds.com
PGP Key ID: C36D 770F 49F7 CC91 2E5A A2BE FE6E CD43 E6CD 9184
----------------------------------------------------------------------------
This list is provided by the SecurityFocus Security Intelligence Alert (SIA)
Service. For more information on SecurityFocus' SIA service which
automatically alerts you to the latest security vulnerabilities please see:
https://alerts.securityfocus.com/
This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:53:26 EDT