We ask for forgiveness from our readers for the long silence but were unavailable due to some over-seas work. We're back now and going to start to follow up with reviews. Our next review by popular demand (22 requests!!) is going to be redspin. You can find them at http://www.redspin.com.
Friday, July 10, 2009
Saturday, April 5, 2008
Pivot Point Security ( A )
Comments from Pivot Point Security about our review process:
"The "SecReview" team demonstrated a high level of professionalism during the review process. We were given ample opportunity to express our point of view and found the process fair and objective.
Based on our experience we believe their efforts to be admirable and a benefit to consumers of Information Security services and the Information Security community as a whole.
John Verry, Principal Consultant @ Pivot Point Security"
Our review of Pivot Point Security:
Pivot Point Security, whose website can be found at http://www.pvtpt.com, is a provider of Information Security Auditing, Security Event Management, and Penetration Testing services. We found them by doing yet another search for “Penetration Testing” on Google. Unlike some other providers who are afraid to have us look under their hood, Pivot Point let us right in.
The first thing that we are going to say is that we would recommend Pivot Point over most of the other companies that we’ve reviewed to date. They are honest about their capabilities, they do not hide behind a colorful storm of pretty marketing fluff, and they will not lead you down the wrong path. They also properly differentiate their services and use the appropriate terminology in their reports, during telephone conversations, and on their website. While they do not have most technically advanced kung-fu, and are not comprised of a team of super hackers, they are able to deliver services that will help to increase their customers overall security posture.
During the telephone interview that we had with Pivot Point they told us that they do not have a Vulnerability Research and Development team. We feel very strongly that providers should perform Vulnerability Research and Development if they are going to be offering services like Penetration Testing and Vulnerability Assessments. This type of research can be used to enhance the quality of the services being delivered.
We can’t however say that Pivot Point performs no research. According to what we were told during our telephone call, Pivot Point performs very interesting and useful research that is focused on security events (firewalls, IDS, VPN, system logs, proxy logs). Their research is intended to improve the ability to detect “significant or anomalous” security events out of the large number of events that most enterprises generate While we know that most hackers worth their salt can bypass IDS and avoid detection, we appreciate anyone that is making an effort to further enhance it.
Ok, so we’ve been nice so far and we do like Pivot Point, but we’re going to be taking a jab at them soon. During our telephone call Pivot Point made it very clear to us that their primary line of business was not Penetration Testing or Vulnerability Assessments, but that it was auditing. Pivot Point views Penetration Testing as a substantiative form of controls auditing. . Pivot Point acknowledged that they are not “super hackers” and that there are a limited number of instances where they will refer a customer to a provider that can provide those types of services. They will not lie like some providers and offer an advanced service while delivering a standard service just for the buck.
With that in mind, we did review a sanitized penetration testing report that was given to us by Pivot Point. Don’t ask us for a copy of the report because we were asked to keep it confidential and that is what we plan on doing.
Based on a detailed analysis of the report, it appears that Pivot Point’s methodology for performing Penetration Testing is as follows. First, Pivot Point will run the Nessus automated vulnerability scanner against the network or computer being tested. They will then digest the results from the automated scan and produce a list of vetted vulnerabilities. Pivot Point makes use of a range of other reconnaissance/attack tools (e.g., Nikto, Paros, App Detective, Wire Shark, Cain & Abel, AirCrack) dependent on the project scope and customer objectives. Once they have those results, they use open source tools (e.g, Metasploit, pwdump, netcat, hydra) and/or custom scripting to target the vulnerabilities and attempt to penetrate the devices. The reports do contain screen shots, and some level of technical description per discovery. But like Pivot Point told us initially, the report certainly did not demonstrate an advanced capability with respect to penetration testing.
In addition to the reports we were given a series of case studies. We don’t particularly care about most case studies as we consider most of them to be marketing fluff. That is after all what they are used for, isn’t it?
So in closing, we would recommend Pivot Point to anyone that doesn’t require the level of assurance that can be provided by a vendor with super depth and advanced services. Pivot Point will help you to identify “known security issues”, and they will help you to make sure that you are locked down with respect to those known issues. It is important to note that they will not protect you from the unknown or 0-day type issues, as their services are standard level (but high quality and honesty). When it comes to performing research and locating 0-day type issues, they say that they will redirect you to a quality vendor that can deliver that level of service.
As usual we're open to suggestions about this review. If anything we've written is an untruth or does not accurately reflect Pivot Point Security let us know (the good and the bad).
Score Card (Click to Enlarge)
"The "SecReview" team demonstrated a high level of professionalism during the review process. We were given ample opportunity to express our point of view and found the process fair and objective.
Based on our experience we believe their efforts to be admirable and a benefit to consumers of Information Security services and the Information Security community as a whole.
John Verry, Principal Consultant @ Pivot Point Security"
Our review of Pivot Point Security:
Pivot Point Security, whose website can be found at http://www.pvtpt.com, is a provider of Information Security Auditing, Security Event Management, and Penetration Testing services. We found them by doing yet another search for “Penetration Testing” on Google. Unlike some other providers who are afraid to have us look under their hood, Pivot Point let us right in.
The first thing that we are going to say is that we would recommend Pivot Point over most of the other companies that we’ve reviewed to date. They are honest about their capabilities, they do not hide behind a colorful storm of pretty marketing fluff, and they will not lead you down the wrong path. They also properly differentiate their services and use the appropriate terminology in their reports, during telephone conversations, and on their website. While they do not have most technically advanced kung-fu, and are not comprised of a team of super hackers, they are able to deliver services that will help to increase their customers overall security posture.
During the telephone interview that we had with Pivot Point they told us that they do not have a Vulnerability Research and Development team. We feel very strongly that providers should perform Vulnerability Research and Development if they are going to be offering services like Penetration Testing and Vulnerability Assessments. This type of research can be used to enhance the quality of the services being delivered.
We can’t however say that Pivot Point performs no research. According to what we were told during our telephone call, Pivot Point performs very interesting and useful research that is focused on security events (firewalls, IDS, VPN, system logs, proxy logs). Their research is intended to improve the ability to detect “significant or anomalous” security events out of the large number of events that most enterprises generate While we know that most hackers worth their salt can bypass IDS and avoid detection, we appreciate anyone that is making an effort to further enhance it.
Ok, so we’ve been nice so far and we do like Pivot Point, but we’re going to be taking a jab at them soon. During our telephone call Pivot Point made it very clear to us that their primary line of business was not Penetration Testing or Vulnerability Assessments, but that it was auditing. Pivot Point views Penetration Testing as a substantiative form of controls auditing. . Pivot Point acknowledged that they are not “super hackers” and that there are a limited number of instances where they will refer a customer to a provider that can provide those types of services. They will not lie like some providers and offer an advanced service while delivering a standard service just for the buck.
With that in mind, we did review a sanitized penetration testing report that was given to us by Pivot Point. Don’t ask us for a copy of the report because we were asked to keep it confidential and that is what we plan on doing.
Based on a detailed analysis of the report, it appears that Pivot Point’s methodology for performing Penetration Testing is as follows. First, Pivot Point will run the Nessus automated vulnerability scanner against the network or computer being tested. They will then digest the results from the automated scan and produce a list of vetted vulnerabilities. Pivot Point makes use of a range of other reconnaissance/attack tools (e.g., Nikto, Paros, App Detective, Wire Shark, Cain & Abel, AirCrack) dependent on the project scope and customer objectives. Once they have those results, they use open source tools (e.g, Metasploit, pwdump, netcat, hydra) and/or custom scripting to target the vulnerabilities and attempt to penetrate the devices. The reports do contain screen shots, and some level of technical description per discovery. But like Pivot Point told us initially, the report certainly did not demonstrate an advanced capability with respect to penetration testing.
In addition to the reports we were given a series of case studies. We don’t particularly care about most case studies as we consider most of them to be marketing fluff. That is after all what they are used for, isn’t it?
So in closing, we would recommend Pivot Point to anyone that doesn’t require the level of assurance that can be provided by a vendor with super depth and advanced services. Pivot Point will help you to identify “known security issues”, and they will help you to make sure that you are locked down with respect to those known issues. It is important to note that they will not protect you from the unknown or 0-day type issues, as their services are standard level (but high quality and honesty). When it comes to performing research and locating 0-day type issues, they say that they will redirect you to a quality vendor that can deliver that level of service.
As usual we're open to suggestions about this review. If anything we've written is an untruth or does not accurately reflect Pivot Point Security let us know (the good and the bad).
Score Card (Click to Enlarge)
Wednesday, February 6, 2008
Layer 9 Corporation ( D )
This will be our shortest review yet. We've spent the past three weeks trying to get hold of the Layer 9 Corporation. We've placed several telephone calls (well over a dozen), and sent multiple emails all of which to no avail. As a result, this review is being done strictly on the information that we were able to collect from the Layer 9 Website located at http://www.layer9corp.com.
Short of the Layer 9 Website containing sounds that go "plink" and "swoosh" when the page loads, and short of the page being ugly and partially broken, its not all that bad. Each section of the website is well written. The services are clearly defined along with the service limitations. In fact, we were surprised as to how honest Layer 9 was on their website, we'll go so far as to call it refreshing.
Under the testing section of the Layer 9 website they clearly state that if they can not perform a specific service that they will find someone who can. Under the PCI section they say up front that they are not a PCI certified vendor, but that they can help companies to prepare for PCI compliance testing. This is the first time that we've encountered a business that tells customers what they can't do, instead of pretending that they can do everything.
We do take a few points away from Layer 9 because they resell third party hardware and software. We feel that companies who resell third party technologies become bias towards selling those technologies even if a better technology solution exists. This might not stand true for a business that makes such a strong effort to be honest like Layer 9, but it most certainly is true for most IT Security Providers.
We also noticed that Layer 9 seems to be more geared towards offering IT services than Professional IT Security Services. They sell PIX firewalls and discuss services that are designed to help their customers improve the performance of their IT Infrastructure. They do not offer the more advanced IT Security Services.
Based on the little bit of information that we were able to collect about Layer 9, it is our opinion that Layer 9 is a trustworthy company that will only offer services to their customers that they are capable of delivering. We can not comment on the talent or capabilities of Layer 9 as we couldn't find any information related
to that. Likewise, we can not comment on the quality of their services.
If anyone reads this review and knows how to get hold of people at Layer 9, please contact us at secreview@blogspot.com. We'd like to re-write this review after having interviewed Layer 9 people. Thanks for reading! (Yes we know that this is a light post).
Score Card (Click to Enlarge)
Short of the Layer 9 Website containing sounds that go "plink" and "swoosh" when the page loads, and short of the page being ugly and partially broken, its not all that bad. Each section of the website is well written. The services are clearly defined along with the service limitations. In fact, we were surprised as to how honest Layer 9 was on their website, we'll go so far as to call it refreshing.
Under the testing section of the Layer 9 website they clearly state that if they can not perform a specific service that they will find someone who can. Under the PCI section they say up front that they are not a PCI certified vendor, but that they can help companies to prepare for PCI compliance testing. This is the first time that we've encountered a business that tells customers what they can't do, instead of pretending that they can do everything.
We do take a few points away from Layer 9 because they resell third party hardware and software. We feel that companies who resell third party technologies become bias towards selling those technologies even if a better technology solution exists. This might not stand true for a business that makes such a strong effort to be honest like Layer 9, but it most certainly is true for most IT Security Providers.
We also noticed that Layer 9 seems to be more geared towards offering IT services than Professional IT Security Services. They sell PIX firewalls and discuss services that are designed to help their customers improve the performance of their IT Infrastructure. They do not offer the more advanced IT Security Services.
Based on the little bit of information that we were able to collect about Layer 9, it is our opinion that Layer 9 is a trustworthy company that will only offer services to their customers that they are capable of delivering. We can not comment on the talent or capabilities of Layer 9 as we couldn't find any information related
to that. Likewise, we can not comment on the quality of their services.If anyone reads this review and knows how to get hold of people at Layer 9, please contact us at secreview@blogspot.com. We'd like to re-write this review after having interviewed Layer 9 people. Thanks for reading! (Yes we know that this is a light post).
Score Card (Click to Enlarge)
Sunday, January 20, 2008
PlanNetGroup ( F )
One of our readers requested that we perform a review of the PlanNetGroup, so here it is. This is an updated version of our first review, as our readers suggested that the first review was too harsh.
The PlanNetGroup was founded by Jim Mazotas of Ohio USA according to the Affirmative Action Verification Form. We called Mr. Mazotas and spoke with him for about an hour about his company. Not only was he very polite and friendly, but we should thank him for spending so much time talking with us.
Before we truly dig into Jim, we should let everyone know that he did say that he was not the most technical person. With that in mind, we began to ask him questions that we thought he, as the founder of the PlanNetGroup should be able to answer. At the same time we also asked him for the contact information for his technical lead. He said that he would arrange to get that to us later, which he did.
Since Jim was not a technical specialist, we decided to keep the conversation at a high level. As such, we asked Jim to provide us with basic service descriptions. The first question to him was “How do you define a penetration test?”. In all fairness a penetration test can be performed a wide variety of ways, but all penetration tests have key points in common which is what make them a penetration test. The most important of which is that a Penetration Test will attempt to exploit any vulnerability discovered and penetrate into the targeted system.
Just as a reference, here is how wikipedia defines a penetration test:
“A penetration test is a method of evaluating the security of a computer system or network by simulating an attack by a malicious user, known as a cracker (though often incorrectly referred to as a hacker). The process involves an active analysis of the system for any potential vulnerabilities that may result from poor or improper system configuration, known and/or unknown hardware or software flaws, or operational weaknesses in process or technical countermeasures. This analysis is carried out from the position of a potential attacker, and can involve active exploitation of security vulnerabilities. Any security issues that are found will be presented to the system owner together with an assessment of their impact and often with a proposal for mitigation or a technical solution. The intent of a penetration test is to determine feasibility of an attack and the amount of business impact of a successful exploit, if discovered.”
Let us add that the difference between a Penetration Test and a Vulnerability Assessment is significant. With a Penetration Test the attacker will attempt to penetrate the targeted system, as denoted by the name. With a Vulnerability Assessment, the attacker will only assess the network and identify vulnerabilities, but will not penetrate or exploit the identified vulnerabilities.
When Jim answered our question “How do you define a penetration test?” his answer did not hit on any of the key points that define a penetration test. In fact, his answer, while still vague was better fit for the second question which was “How do you define a vulnerability assessment?”. Jim said, “We get to target object, where we go with that is based upon the client’s comfort level. We grab banner information, backend support information, and other kinds of information. During a penetration test we most will not penetrate. Most mid level companies will not want penetration.” – Sanitized Quote from Jim – Note that Jim said that they will most likely not penetrate.
Our second question to Jim was about his definition of Vulnerability Assessments. Specifically we asked him “How do you define a Vulnerability Assessment?”. Again, here is how wikipedia defines a “Vulnerability Assessment”. We’re not saying that Wikipedia is the best reference source, but in this case it is accurate.
“A vulnerability assessment is the process of identifying, quantifying, and prioritizing (or ranking) the vulnerabilities in a system. Examples of systems for which vulnerability assessments are performed for include, but are not limited to, nuclear power plants, information technology systems, energy supply systems, water supply systems, transportation systems, and communication systems. Vulnerability assessments can be conducted for small businesses to large regional infrastructures. Vulnerability assessment has many things in common with risk assessment. Assessments are typically performed according to the following steps:
1. Cataloging assets and capabilities (resources) in a system
2. Assigning quantifiable value (or at least rank order) and importance to those resources
3. Identifying the vulnerabilities or potential threats to each resource
4. Mitigating or eliminating the most serious vulnerabilities for the most valuable resources”
When we asked Jim “How do you define a Vulnerability Assessment?” his answer did not hit a single key point of a Vulnerability Assessment. Jim said “A Vulnerability Assessment is more a lab based environment type test. Analyze servers and all nodes that are a true vital asset to the company and assess the vulnerability in a very planned out manner. This is done in a lab based environment.” – Sanitized Quote from Jim
In all fairness we understand that Jim is not a technical security expert, but he is the CEO and founder of the PlanNetGroup. As such, we feel that he should at the very least understand how to define the services that his business is offering. It may be that his security services were never very well defined in the first place. If that is the case then we strongly suggest that he have his technical experts define his services clearly and that he learns those definitions.
Remember, our goal is not to bash Jim or any other company. Our goal is to expose Professional IT Security Providers for what they really are. It is important to remember that these reviews are also subjective and are the product of our cumulative opinions. If you have a different opinion we’d love to hear it.
Moving on. After we finished our interview with Jim we asked him if we could speak with one of his technical people. It took a while for us to get the technical contact information, but we did get in eventually. The next interview that we did was conduced over email. We sent Jim a series of questions and we received an answer from Kitty Mcmenemy but signed as “Jason Bourne”.
We’re not sure if Jim decided to sic a private investigator on us, but Kitty McMenemy is the name of a Private Investigator. In fact, that particular PI uses the irishgirl email address disclosed above. We also thought that it was odd that the email was signed by “Jason Bourne”, the movie the “Bourn Identity” comes to mind. (Realistically it doesn’t matter though, we post our reviews with tor and from open wireless connections that do not tie back to the secreview team.) On top of that, we’re not doing anything illegal, so why waste the money on a PI?
In any case, here are the answers that we received from Kitty McMenemy aka Jason Bourne. These answers are taken verbatim from the email that we received and have not been altered in any way. Our questions will be bolded, the answers will be in italics, and our comments will be in normal text.
-) How do you perform your vulnerability assessments?
“* Carefully! :) Typically, we will work with the customer to define the
scope of the assessment; limitations to OS, Network Equipment, Web
Server, etc. This could be a combination of components (depending on
scope), the real goal ultimately with this is to assess the patching
effort of a customer. Depending on time and availability, we will work
on finding any new vulnerability if we generate an anomaly of interest.
Currently, the focus is primarily on discovering new Oracle
vulnerabilities - as MS SQL 2K5 is more difficult to beat on, compared
to Oracle. Within vulnerability assessments, we disregard any attempts
to evade IDS, IPS, etc.”
The answer above does not give us a clear understanding of how the PlanNetGroup performs their Vulnerability Assessment. In their defense the author does tell us that they scope the project with the customer. Not only is this standard, but it is something that anyone must do during the delivery of any service. We run into a problem in the middle of the paragraph when the author writes “This could be a combination of components (depending on scope), the real goal ultimately with this is to assess the patching
effort of a customer.” The real goal of a Vulnerability Assessment is not to assess the patching effort of a customer, it is to identify the vulnerabilities in the targeted devices, and to provide methods for fixing those vulnerabilities.
Another issue that we have is with the second part of the answer when the author writes “Currently, the focus is primarily on discovering new Oracle vulnerabilities - as MS SQL 2K5 is more difficult to beat on, compared to Oracle.” Are we to understand that when the PlanNetGroup performs a Vulnerability Assessment that they only do it for databases? We also do not agree that Oracle is more vulnerable than MS SQL. Anything can be reasonably secure if you assess it and harden it properly. Unfortunately, the answer provided does not truly answer our question, but it does give us an understanding of the expertise of the PlanNetGroup expert that we were told to speak with by Jim.
-) How do you perform your penetration testing?
“* Again, carefully! The definition that I use with customers is -
Anything Goes! In addition to attempting to locate missing patches,
vulnerable IOS's, applications, etc - we will perform an assortment of
timed attacks, attempt to spoof trusted connections, or even perform
social engineering - like dropping a few pre-trojan'd usb data sticks
outside of a customer service area, a data center, etc. The only thing
that we do not perform, typically, is denial of service style or type of
attacks. We have had only one customer that we felt was in the position
to handle such a test and it was performed against their disaster
recovery infrastructure, not production.”
It concerns us that that the definition that the author uses with customers for a Penetration Test is “Anything Goes!”. If anything, a Penetration Test should be performed with much more caution than a Vulnerability Assessment as Penetration Tests actually exploit identified vulnerabilities. It also concerns us that the answer does not touch on any of the safety measures that should be required during the performance of a Penetration Test. We do agree that not performing Denial of Service attacks is a good idea as general practice. Again and unfortunately, the answer above does not truly answer our question.
-) How do you perform evasive IDS testing?
"* We use a series of proxy servers to attempt to perform basic hacking
techniques; port scans, blatant attacks, etc. We are typically going to
look for TCP resets as a means to evaluate if IDS is present and
possibly to find if IDS performs blocking activity. Often times, if a
system in a trusted DMZ can be compromised and used as a proxy
(exploiting a relationship or rule within a firewall) or an SSH, SSL,
encrypted tunnel can be established to a server behind the IDS sensor
than we can successfully pull off an attack without the customers
security staff even knowing."
IDS/IPS evasion testing means that an attacker is able to hide his or her attacks from the IDS or IPS technologies. IDS/IPS evasion does not mean that the attacker is hiding behind a proxy or a different IP address. If the attacker does hide behind a proxy and his or her attack is detected, then the evasion has failed. When performing evasive testing one of the worst things that anyone can do is to launch port scans as port scans will generate events in even the most generic of IDS/IPS technologies. One partially effective method of IDS/IPS evasion testing is to attack web applications over SSL so long as the IDS/IPS agent can not monitor the SSL’d connection. There are many much more effective methods for evading detection, but we won’t go into that here. Again, the answer provided by the PlanNetGroup security expert did not accurately answer our question. The answer was partial at best.
-) What tools do you favor?
“* We really do not favor any tools. The focus of our effort (Assuming we are performing a pen-test or assessment) is to analyze a situation and choose the best tool for the end result or compromise. I will use commercial applications, such as AppScan, WebInspect, even ISS. There are however plenty of freeware, low-cost tools that we use; nmap, nessus, metasploit - ultimately, I find that an internet browser and a telnet prompt will suffice for much of the testing. It ultimately gets back to interpreting the results and adjusting the testing accordingly. We make it a point to try out new freeware tools on every assignment. The more tools that we know of and can test with opens our options if in the future a situation best suited for a tool presents itself.”
We agree that there are many free tools available from the security community. In some cases those tools are more capable than the commercial tools, in other cases they are not. One of the benefits to purchasing commercial tools is support and maintenance, but the downside is the cost. We are also very fond of Web Inspect, Metasploit, Nessus, but also like other tools like CANVAS, Core Impact, etc.
-) Can you provide us with sample deliverables? (sanitized)
“* No, too much time. Even to sanitize creates an opportunity for a liability in the event that a customer name is exposed ... accidents do happen! I will say that we do not take dumps from applications and regurgitations the information on paper. We limit our executive summary to 6 pages at most and attempt to keep the entire report limited
to 25 pages in total. Our goal with a deliverable is to get the precise information to the key stake holders so that they can make a decision.”
This is reasonable but is probably something that the PlanNetGroup should invest time in creating. Sample reports do take time to create and they need to be done carefully. Disclosing any customer information would be a big no, no and could result in serious damages to the customer and the provider. At this point we do have a very good understanding of the PlanNetGroup’s service delivery capabilities, but this report would have been nice to have.
-) Do you offer the option of performing Distributed Metastasis?
“* No, not really. This is my decision as in a previous life I got walked out of Bell Atlantic Mobile (Verizon Wireless) using this technique when I compromised their Unix infrastructure by compromising the rlogin function (on all Unix servers, across all data centers). There is no substitute for experience, especially bad ones!”
We think that this is a bad decision, but that is just our opinion. Distributed Metastasis is often a very powerful testing module as it will enable a customer to truly determine the full scope of their vulnerability. It is very important that if Distributed Metastasis is used that it is built into the Scope of Work. If it is not built into the Scope of Work and is still performed then the tester could get in some hot water. Based on the answer to our question above it sounds like the author performed Distributed Metastasis without authorization.
-) What is your background with relation to information security?
“* Too long, too boring. Yeah got the CISSP (nice vocabulary test), but had to as I worked for DOD. Got a number of Certifications (I have a stack almost an inch thick and only get into them about once a year to throw another couple on top of the previous ones - too much alphabet soup for me, but bosses and customers like it. Spoke at a number of
European conferences, but found too many people did not understand a word I was talking about, so I got tired of that and quit that scene. My outlook on security has changed, to the point that I will advise customers of their risk, attempt to make it practical - but if they make a conscious choice not to listen - I do not fret over it.?”
When we asked the author about his or her history, the response that we got was very similar to the responses for the more technical questions. This question was particularly important as we wanted to assess the level of experience and expertise that the PlanNetGroup had to offer. Instead of getting an idea as to that level of expertise, we got “Too long too boring.” Unfortunately certifications are something that anyone can get with enough time and studying, so they don’t hold a lot of water in the real security world. We’re also a bit concerned about the last sentence in the answer that reads “My outlook on security has changed, to the point that I will advise customers of their risk, attempt to make it practical - but if they make a conscious choice not to listen - I do not fret over it.?” This makes us feel like the author does not care about his or her customers.
-) Do you resell third party technologies?
“* No, but kind of wished that we would. I think that it would help with sales.”
We actually like the fact that the PlanNetGroup does not sell third party technologies. Companies that do are often motivated to push those technologies as expensive solutions to problems that could be fixed by more cost effective solutions.
-) Can you tell me why the EIP is important?
“* The EIP controls an applications execution. If an attacker can modify the EIP while it is being pushed on the stack then the attacker *could* execute their own code and create a thread (aka. a buffer overflow condition exists). I had a good refresher this past year at Blackhat with a course run by Saumil Shah - he had an interesting buffer overflow
for the Linked-In client.”
This is mostly accurate. The EIP is the Instruction Pointer in i386 architecture computer systems. If an attacker can overwrite the EIP then the attacker can in most cases seize control over the target by forcing the system to execute arbitrary commands.
-) Can you define a format string exploit?
“* A format string exploit leverages what is considered a programming
bug. If input is not sanitized, an attacker can perform calls to the
stack; read, write, etc without knowing details about the EIP.”
This does not properly define what a format string exploit is. A format string exploit takes advantage bad input validation in a c based application and uses format tokens to overwrite key points in a systems memory. Examples of those format tokens are %x, %s, and %n.
-) Can you define an off-by-one exploit?
"* This exploit takes advantage of sloppy coding in which a developer
does not include a null byte terminator with their string. If a string
is not handled properly, it could allow the string to edge up against
another buffer in the stack, possibly treating the two as one - the end
result being a buffer overflow ... with a bit of luck."
An off-by-one exploit is an exploit that takes advantage of the logical off-by-one error. The off-by-one error is a common programming mistake that occurs when an iterative loop iterates one time too many or one time too few, and is usually the result of the programmer incorrectly counting iterations. This error can become an exploitable security vulnerability, especially in little endian architectures, if the attacker is able to rewrite the least significant byte of the frame pointer. The previous answer provided above does not accurately define an off-by-one exploit.
-) Can you define "logic flaw"?
"* Depending on who you talk to, there are a couple of different
definitions (especially for old people from the DOD). The popular
current definition is when you attempt to use business logic to
compromise an application (most often web-based in our situation).
Ultimately, through the use of security vulnerabilities and
understanding business logic, an attacker can exploit an application to
perform a function or expose sensitive data. An example in our case
would be an insurance company application that allowed us to turn on
debug mode remotely and then allowed us to dump the entire database of
high-net worth clients to the screen simply by guessing the output file
name of the application based on discussions about what the application
did."
A logic flaw is a computer programming error that causes a program to behave incorrectly but does not cause the program to crash or exit. Sometimes it is possible to use logic flaws to attack and exploit programs (off-by-one exploits are logic flaws), other times they can be used to divulge information. We asked this question in an effort to see if the PlanNetGroup would correlate off-by-one exploits with logic flaws, and they didn’t.
-) What is the most critical aspect of Computer Forensic Research?
(Jim mentioned encase when we spoke hence the question).
"* Depending on the wording of the question - if you are asking this
about 'research' I would say that knowledge of anti-forensic techniques
and encryption knowledge is pivotal. If you are asking this question
about a computer forensic 'investigation' it would be the integrity of
the evidence. If the integrity of the evidence is in question then it
is better to stop activities and not spend anymore time on the
investigation."
Forensic Science (also called Forensics) is the use of a range of sciences to answer questions of interest to a legal system. As such, the most critical aspect to performing Computer Forensic Research, or any form of Forensic Science is to maintain the chain of evidence. If that chain is broken then the resulting discoveries are useless in the court of law.
With that we bring our final and last interview to a conclusion. The next section will be a brief paragraph or two that outline our opinions about the PlanNetGroup’s website. As of right now things aren’t looking too good for the PlanNetGroup. The answer’s that we’ve received to not contain the depth, accuracy and detail that we expect to see from security experts.
The PlanNetGroup’s website leaves us with more questions that it does answers. We took the time to read each specific service offering in an attempt to understand what was being offered. At the end, we were left more confused than when we began.
For example, the PlanNetGroup’s Risk Assessment page does nothing to define what a Risk Assessment is or how they perform a Risk Assessment. At the bottom of the page is a description for a Vulnerability Assessment which seems very much like what is available at wikipedia. More over, the description that we were given by the PlanNetGroup expert about how they perform a vulnerability assessment, doesn’t match what is being offered on the website.
All in all we would not recommend the PlanNetGroup to anyone interested in protecting their infrastructure from malicious hackers. Not only is their website horribly confusing and non-informative, but their security experts were unable to answer most of our questions with solidity, clarity, depth and accuracy. Find a different provider if you are serious about your IT security. We’re sorry PlanNetGroup, but this is our opinion after this review.
Score Card (Click to Enlarge)
The PlanNetGroup was founded by Jim Mazotas of Ohio USA according to the Affirmative Action Verification Form. We called Mr. Mazotas and spoke with him for about an hour about his company. Not only was he very polite and friendly, but we should thank him for spending so much time talking with us.
Before we truly dig into Jim, we should let everyone know that he did say that he was not the most technical person. With that in mind, we began to ask him questions that we thought he, as the founder of the PlanNetGroup should be able to answer. At the same time we also asked him for the contact information for his technical lead. He said that he would arrange to get that to us later, which he did.
Since Jim was not a technical specialist, we decided to keep the conversation at a high level. As such, we asked Jim to provide us with basic service descriptions. The first question to him was “How do you define a penetration test?”. In all fairness a penetration test can be performed a wide variety of ways, but all penetration tests have key points in common which is what make them a penetration test. The most important of which is that a Penetration Test will attempt to exploit any vulnerability discovered and penetrate into the targeted system.
Just as a reference, here is how wikipedia defines a penetration test:
“A penetration test is a method of evaluating the security of a computer system or network by simulating an attack by a malicious user, known as a cracker (though often incorrectly referred to as a hacker). The process involves an active analysis of the system for any potential vulnerabilities that may result from poor or improper system configuration, known and/or unknown hardware or software flaws, or operational weaknesses in process or technical countermeasures. This analysis is carried out from the position of a potential attacker, and can involve active exploitation of security vulnerabilities. Any security issues that are found will be presented to the system owner together with an assessment of their impact and often with a proposal for mitigation or a technical solution. The intent of a penetration test is to determine feasibility of an attack and the amount of business impact of a successful exploit, if discovered.”
Let us add that the difference between a Penetration Test and a Vulnerability Assessment is significant. With a Penetration Test the attacker will attempt to penetrate the targeted system, as denoted by the name. With a Vulnerability Assessment, the attacker will only assess the network and identify vulnerabilities, but will not penetrate or exploit the identified vulnerabilities.
When Jim answered our question “How do you define a penetration test?” his answer did not hit on any of the key points that define a penetration test. In fact, his answer, while still vague was better fit for the second question which was “How do you define a vulnerability assessment?”. Jim said, “We get to target object, where we go with that is based upon the client’s comfort level. We grab banner information, backend support information, and other kinds of information. During a penetration test we most will not penetrate. Most mid level companies will not want penetration.” – Sanitized Quote from Jim – Note that Jim said that they will most likely not penetrate.
Our second question to Jim was about his definition of Vulnerability Assessments. Specifically we asked him “How do you define a Vulnerability Assessment?”. Again, here is how wikipedia defines a “Vulnerability Assessment”. We’re not saying that Wikipedia is the best reference source, but in this case it is accurate.
“A vulnerability assessment is the process of identifying, quantifying, and prioritizing (or ranking) the vulnerabilities in a system. Examples of systems for which vulnerability assessments are performed for include, but are not limited to, nuclear power plants, information technology systems, energy supply systems, water supply systems, transportation systems, and communication systems. Vulnerability assessments can be conducted for small businesses to large regional infrastructures. Vulnerability assessment has many things in common with risk assessment. Assessments are typically performed according to the following steps:
1. Cataloging assets and capabilities (resources) in a system
2. Assigning quantifiable value (or at least rank order) and importance to those resources
3. Identifying the vulnerabilities or potential threats to each resource
4. Mitigating or eliminating the most serious vulnerabilities for the most valuable resources”
When we asked Jim “How do you define a Vulnerability Assessment?” his answer did not hit a single key point of a Vulnerability Assessment. Jim said “A Vulnerability Assessment is more a lab based environment type test. Analyze servers and all nodes that are a true vital asset to the company and assess the vulnerability in a very planned out manner. This is done in a lab based environment.” – Sanitized Quote from Jim
In all fairness we understand that Jim is not a technical security expert, but he is the CEO and founder of the PlanNetGroup. As such, we feel that he should at the very least understand how to define the services that his business is offering. It may be that his security services were never very well defined in the first place. If that is the case then we strongly suggest that he have his technical experts define his services clearly and that he learns those definitions.
Remember, our goal is not to bash Jim or any other company. Our goal is to expose Professional IT Security Providers for what they really are. It is important to remember that these reviews are also subjective and are the product of our cumulative opinions. If you have a different opinion we’d love to hear it.
Moving on. After we finished our interview with Jim we asked him if we could speak with one of his technical people. It took a while for us to get the technical contact information, but we did get in eventually. The next interview that we did was conduced over email. We sent Jim a series of questions and we received an answer from Kitty Mcmenemy
We’re not sure if Jim decided to sic a private investigator on us, but Kitty McMenemy is the name of a Private Investigator. In fact, that particular PI uses the irishgirl email address disclosed above. We also thought that it was odd that the email was signed by “Jason Bourne”, the movie the “Bourn Identity” comes to mind. (Realistically it doesn’t matter though, we post our reviews with tor and from open wireless connections that do not tie back to the secreview team.) On top of that, we’re not doing anything illegal, so why waste the money on a PI?
In any case, here are the answers that we received from Kitty McMenemy aka Jason Bourne. These answers are taken verbatim from the email that we received and have not been altered in any way. Our questions will be bolded, the answers will be in italics, and our comments will be in normal text.
-) How do you perform your vulnerability assessments?
“* Carefully! :) Typically, we will work with the customer to define the
scope of the assessment; limitations to OS, Network Equipment, Web
Server, etc. This could be a combination of components (depending on
scope), the real goal ultimately with this is to assess the patching
effort of a customer. Depending on time and availability, we will work
on finding any new vulnerability if we generate an anomaly of interest.
Currently, the focus is primarily on discovering new Oracle
vulnerabilities - as MS SQL 2K5 is more difficult to beat on, compared
to Oracle. Within vulnerability assessments, we disregard any attempts
to evade IDS, IPS, etc.”
The answer above does not give us a clear understanding of how the PlanNetGroup performs their Vulnerability Assessment. In their defense the author does tell us that they scope the project with the customer. Not only is this standard, but it is something that anyone must do during the delivery of any service. We run into a problem in the middle of the paragraph when the author writes “This could be a combination of components (depending on scope), the real goal ultimately with this is to assess the patching
effort of a customer.” The real goal of a Vulnerability Assessment is not to assess the patching effort of a customer, it is to identify the vulnerabilities in the targeted devices, and to provide methods for fixing those vulnerabilities.
Another issue that we have is with the second part of the answer when the author writes “Currently, the focus is primarily on discovering new Oracle vulnerabilities - as MS SQL 2K5 is more difficult to beat on, compared to Oracle.” Are we to understand that when the PlanNetGroup performs a Vulnerability Assessment that they only do it for databases? We also do not agree that Oracle is more vulnerable than MS SQL. Anything can be reasonably secure if you assess it and harden it properly. Unfortunately, the answer provided does not truly answer our question, but it does give us an understanding of the expertise of the PlanNetGroup expert that we were told to speak with by Jim.
-) How do you perform your penetration testing?
“* Again, carefully! The definition that I use with customers is -
Anything Goes! In addition to attempting to locate missing patches,
vulnerable IOS's, applications, etc - we will perform an assortment of
timed attacks, attempt to spoof trusted connections, or even perform
social engineering - like dropping a few pre-trojan'd usb data sticks
outside of a customer service area, a data center, etc. The only thing
that we do not perform, typically, is denial of service style or type of
attacks. We have had only one customer that we felt was in the position
to handle such a test and it was performed against their disaster
recovery infrastructure, not production.”
It concerns us that that the definition that the author uses with customers for a Penetration Test is “Anything Goes!”. If anything, a Penetration Test should be performed with much more caution than a Vulnerability Assessment as Penetration Tests actually exploit identified vulnerabilities. It also concerns us that the answer does not touch on any of the safety measures that should be required during the performance of a Penetration Test. We do agree that not performing Denial of Service attacks is a good idea as general practice. Again and unfortunately, the answer above does not truly answer our question.
-) How do you perform evasive IDS testing?
"* We use a series of proxy servers to attempt to perform basic hacking
techniques; port scans, blatant attacks, etc. We are typically going to
look for TCP resets as a means to evaluate if IDS is present and
possibly to find if IDS performs blocking activity. Often times, if a
system in a trusted DMZ can be compromised and used as a proxy
(exploiting a relationship or rule within a firewall) or an SSH, SSL,
encrypted tunnel can be established to a server behind the IDS sensor
than we can successfully pull off an attack without the customers
security staff even knowing."
IDS/IPS evasion testing means that an attacker is able to hide his or her attacks from the IDS or IPS technologies. IDS/IPS evasion does not mean that the attacker is hiding behind a proxy or a different IP address. If the attacker does hide behind a proxy and his or her attack is detected, then the evasion has failed. When performing evasive testing one of the worst things that anyone can do is to launch port scans as port scans will generate events in even the most generic of IDS/IPS technologies. One partially effective method of IDS/IPS evasion testing is to attack web applications over SSL so long as the IDS/IPS agent can not monitor the SSL’d connection. There are many much more effective methods for evading detection, but we won’t go into that here. Again, the answer provided by the PlanNetGroup security expert did not accurately answer our question. The answer was partial at best.
-) What tools do you favor?
“* We really do not favor any tools. The focus of our effort (Assuming we are performing a pen-test or assessment) is to analyze a situation and choose the best tool for the end result or compromise. I will use commercial applications, such as AppScan, WebInspect, even ISS. There are however plenty of freeware, low-cost tools that we use; nmap, nessus, metasploit - ultimately, I find that an internet browser and a telnet prompt will suffice for much of the testing. It ultimately gets back to interpreting the results and adjusting the testing accordingly. We make it a point to try out new freeware tools on every assignment. The more tools that we know of and can test with opens our options if in the future a situation best suited for a tool presents itself.”
We agree that there are many free tools available from the security community. In some cases those tools are more capable than the commercial tools, in other cases they are not. One of the benefits to purchasing commercial tools is support and maintenance, but the downside is the cost. We are also very fond of Web Inspect, Metasploit, Nessus, but also like other tools like CANVAS, Core Impact, etc.
-) Can you provide us with sample deliverables? (sanitized)
“* No, too much time. Even to sanitize creates an opportunity for a liability in the event that a customer name is exposed ... accidents do happen! I will say that we do not take dumps from applications and regurgitations the information on paper. We limit our executive summary to 6 pages at most and attempt to keep the entire report limited
to 25 pages in total. Our goal with a deliverable is to get the precise information to the key stake holders so that they can make a decision.”
This is reasonable but is probably something that the PlanNetGroup should invest time in creating. Sample reports do take time to create and they need to be done carefully. Disclosing any customer information would be a big no, no and could result in serious damages to the customer and the provider. At this point we do have a very good understanding of the PlanNetGroup’s service delivery capabilities, but this report would have been nice to have.
-) Do you offer the option of performing Distributed Metastasis?
“* No, not really. This is my decision as in a previous life I got walked out of Bell Atlantic Mobile (Verizon Wireless) using this technique when I compromised their Unix infrastructure by compromising the rlogin function (on all Unix servers, across all data centers). There is no substitute for experience, especially bad ones!”
We think that this is a bad decision, but that is just our opinion. Distributed Metastasis is often a very powerful testing module as it will enable a customer to truly determine the full scope of their vulnerability. It is very important that if Distributed Metastasis is used that it is built into the Scope of Work. If it is not built into the Scope of Work and is still performed then the tester could get in some hot water. Based on the answer to our question above it sounds like the author performed Distributed Metastasis without authorization.
-) What is your background with relation to information security?
“* Too long, too boring. Yeah got the CISSP (nice vocabulary test), but had to as I worked for DOD. Got a number of Certifications (I have a stack almost an inch thick and only get into them about once a year to throw another couple on top of the previous ones - too much alphabet soup for me, but bosses and customers like it. Spoke at a number of
European conferences, but found too many people did not understand a word I was talking about, so I got tired of that and quit that scene. My outlook on security has changed, to the point that I will advise customers of their risk, attempt to make it practical - but if they make a conscious choice not to listen - I do not fret over it.?”
When we asked the author about his or her history, the response that we got was very similar to the responses for the more technical questions. This question was particularly important as we wanted to assess the level of experience and expertise that the PlanNetGroup had to offer. Instead of getting an idea as to that level of expertise, we got “Too long too boring.” Unfortunately certifications are something that anyone can get with enough time and studying, so they don’t hold a lot of water in the real security world. We’re also a bit concerned about the last sentence in the answer that reads “My outlook on security has changed, to the point that I will advise customers of their risk, attempt to make it practical - but if they make a conscious choice not to listen - I do not fret over it.?” This makes us feel like the author does not care about his or her customers.
-) Do you resell third party technologies?
“* No, but kind of wished that we would. I think that it would help with sales.”
We actually like the fact that the PlanNetGroup does not sell third party technologies. Companies that do are often motivated to push those technologies as expensive solutions to problems that could be fixed by more cost effective solutions.
-) Can you tell me why the EIP is important?
“* The EIP controls an applications execution. If an attacker can modify the EIP while it is being pushed on the stack then the attacker *could* execute their own code and create a thread (aka. a buffer overflow condition exists). I had a good refresher this past year at Blackhat with a course run by Saumil Shah - he had an interesting buffer overflow
for the Linked-In client.”
This is mostly accurate. The EIP is the Instruction Pointer in i386 architecture computer systems. If an attacker can overwrite the EIP then the attacker can in most cases seize control over the target by forcing the system to execute arbitrary commands.
-) Can you define a format string exploit?
“* A format string exploit leverages what is considered a programming
bug. If input is not sanitized, an attacker can perform calls to the
stack; read, write, etc without knowing details about the EIP.”
This does not properly define what a format string exploit is. A format string exploit takes advantage bad input validation in a c based application and uses format tokens to overwrite key points in a systems memory. Examples of those format tokens are %x, %s, and %n.
-) Can you define an off-by-one exploit?
"* This exploit takes advantage of sloppy coding in which a developer
does not include a null byte terminator with their string. If a string
is not handled properly, it could allow the string to edge up against
another buffer in the stack, possibly treating the two as one - the end
result being a buffer overflow ... with a bit of luck."
An off-by-one exploit is an exploit that takes advantage of the logical off-by-one error. The off-by-one error is a common programming mistake that occurs when an iterative loop iterates one time too many or one time too few, and is usually the result of the programmer incorrectly counting iterations. This error can become an exploitable security vulnerability, especially in little endian architectures, if the attacker is able to rewrite the least significant byte of the frame pointer. The previous answer provided above does not accurately define an off-by-one exploit.
-) Can you define "logic flaw"?
"* Depending on who you talk to, there are a couple of different
definitions (especially for old people from the DOD). The popular
current definition is when you attempt to use business logic to
compromise an application (most often web-based in our situation).
Ultimately, through the use of security vulnerabilities and
understanding business logic, an attacker can exploit an application to
perform a function or expose sensitive data. An example in our case
would be an insurance company application that allowed us to turn on
debug mode remotely and then allowed us to dump the entire database of
high-net worth clients to the screen simply by guessing the output file
name of the application based on discussions about what the application
did."
A logic flaw is a computer programming error that causes a program to behave incorrectly but does not cause the program to crash or exit. Sometimes it is possible to use logic flaws to attack and exploit programs (off-by-one exploits are logic flaws), other times they can be used to divulge information. We asked this question in an effort to see if the PlanNetGroup would correlate off-by-one exploits with logic flaws, and they didn’t.
-) What is the most critical aspect of Computer Forensic Research?
(Jim mentioned encase when we spoke hence the question).
"* Depending on the wording of the question - if you are asking this
about 'research' I would say that knowledge of anti-forensic techniques
and encryption knowledge is pivotal. If you are asking this question
about a computer forensic 'investigation' it would be the integrity of
the evidence. If the integrity of the evidence is in question then it
is better to stop activities and not spend anymore time on the
investigation."
Forensic Science (also called Forensics) is the use of a range of sciences to answer questions of interest to a legal system. As such, the most critical aspect to performing Computer Forensic Research, or any form of Forensic Science is to maintain the chain of evidence. If that chain is broken then the resulting discoveries are useless in the court of law.
With that we bring our final and last interview to a conclusion. The next section will be a brief paragraph or two that outline our opinions about the PlanNetGroup’s website. As of right now things aren’t looking too good for the PlanNetGroup. The answer’s that we’ve received to not contain the depth, accuracy and detail that we expect to see from security experts.
The PlanNetGroup’s website leaves us with more questions that it does answers. We took the time to read each specific service offering in an attempt to understand what was being offered. At the end, we were left more confused than when we began.
For example, the PlanNetGroup’s Risk Assessment page does nothing to define what a Risk Assessment is or how they perform a Risk Assessment. At the bottom of the page is a description for a Vulnerability Assessment which seems very much like what is available at wikipedia. More over, the description that we were given by the PlanNetGroup expert about how they perform a vulnerability assessment, doesn’t match what is being offered on the website.
All in all we would not recommend the PlanNetGroup to anyone interested in protecting their infrastructure from malicious hackers. Not only is their website horribly confusing and non-informative, but their security experts were unable to answer most of our questions with solidity, clarity, depth and accuracy. Find a different provider if you are serious about your IT security. We’re sorry PlanNetGroup, but this is our opinion after this review.
Score Card (Click to Enlarge)
Friday, January 4, 2008
Syrex ( B )
Syrex, located at http://www.syrex.com, is a quality Professional IT Security Services Provider that offers Risk Assessments, Risk Mitigation, Security Management, Security Training and Incident Response as well as advanced networking services. We found Syrex because they came to us and requested that we perform a review, so here are the results.
Looking at the Syrex website was refreshing in comparison to some of the other websites that we've reviewed. Not only was theirs written clearly, but the services were well defined and the content was complete. It is also clear that Syrex is ready to service a wide range of companies based on the structure of their service offerings. For example, under the Risk Assessment offering they have a specific "Snapshot offering" to help meet the requirements of smaller companies that can't afford a more intense service.
Syrex is not your average Professional IT Security Services Provider in that they do not offer Penetration Testing or ethical hacking type services. They also do not offer Web Application Security Assessments or source code reviews (at least not yet). Instead, Syrex helps their customers by performing complete or partial OSSTMM based security audits. The results of those audits enables Syrex to enhance the overall security of their customers IT Infrastructures by exposing weaknesses in policies, proceedures, technologies, etc. and proving remediation services. While these auditing services are not as technically deep as penetration testing services, or web application security assessment services, they do help to raise the proverbial security bar.
When speaking with the founder of Syrex, we learned that they do in fact have talent. The founder himself has a deep understanding of Intrusion Detection Systems ("IDS") and Intrusion Prevention Systems ("IPS"), Security Information Management Systems ("SIMS"), network and routing protocols, as well as key Cisco technologies like the ASA, Clean Access, ACS, MARS, and CSM. In conjunction with this, he also has experience as a programmer and understands quite a bit about malware, viruses, and other malicious technologies. This is more than we can say for a lot of the other companies that we've interviewed.
Another thing that we were impressed with during our telephone interview was the amount of effort that Syrex put into being honest and ethical. On multiple occasions they pointed out limitations in their service capabilities, and at no point did they try to flaunt anything that they were not certain about. This is the second company that we've interviewed that did not make an effort to sound like they are the best. Instead, they talk the talk and walk the walk.
In conjunction with the telephone interview and website review, we were given sample reports and materials. When reviewing the reports it became immediately clear that Syrex was focused on providing their customers with high quality services that were in fact human driven. The reports were very obviously not the product of automated tools, but instead were the product of human talent. Again, this is more than we can say for a lot of the companies that we review. Most companies these days seem to rely heavily on automation and have little to no real human talent.
All in all we would recommend using Syrex if you are looking to increase your levels of security. They will help you define methods for properly managing and maintaining your network, people and information, all the wile being honest and ethical. We almost feel bad giving Syrex a B instead of an A, but they are missing research and development capabilities, as well as advanced service delivery capabilities. Other than that, great company! Keep up the good work Syrex!
Score Card ( Click to Enlarge )
Looking at the Syrex website was refreshing in comparison to some of the other websites that we've reviewed. Not only was theirs written clearly, but the services were well defined and the content was complete. It is also clear that Syrex is ready to service a wide range of companies based on the structure of their service offerings. For example, under the Risk Assessment offering they have a specific "Snapshot offering" to help meet the requirements of smaller companies that can't afford a more intense service.
Syrex is not your average Professional IT Security Services Provider in that they do not offer Penetration Testing or ethical hacking type services. They also do not offer Web Application Security Assessments or source code reviews (at least not yet). Instead, Syrex helps their customers by performing complete or partial OSSTMM based security audits. The results of those audits enables Syrex to enhance the overall security of their customers IT Infrastructures by exposing weaknesses in policies, proceedures, technologies, etc. and proving remediation services. While these auditing services are not as technically deep as penetration testing services, or web application security assessment services, they do help to raise the proverbial security bar.
When speaking with the founder of Syrex, we learned that they do in fact have talent. The founder himself has a deep understanding of Intrusion Detection Systems ("IDS") and Intrusion Prevention Systems ("IPS"), Security Information Management Systems ("SIMS"), network and routing protocols, as well as key Cisco technologies like the ASA, Clean Access, ACS, MARS, and CSM. In conjunction with this, he also has experience as a programmer and understands quite a bit about malware, viruses, and other malicious technologies. This is more than we can say for a lot of the other companies that we've interviewed.
Another thing that we were impressed with during our telephone interview was the amount of effort that Syrex put into being honest and ethical. On multiple occasions they pointed out limitations in their service capabilities, and at no point did they try to flaunt anything that they were not certain about. This is the second company that we've interviewed that did not make an effort to sound like they are the best. Instead, they talk the talk and walk the walk.
In conjunction with the telephone interview and website review, we were given sample reports and materials. When reviewing the reports it became immediately clear that Syrex was focused on providing their customers with high quality services that were in fact human driven. The reports were very obviously not the product of automated tools, but instead were the product of human talent. Again, this is more than we can say for a lot of the companies that we review. Most companies these days seem to rely heavily on automation and have little to no real human talent.
All in all we would recommend using Syrex if you are looking to increase your levels of security. They will help you define methods for properly managing and maintaining your network, people and information, all the wile being honest and ethical. We almost feel bad giving Syrex a B instead of an A, but they are missing research and development capabilities, as well as advanced service delivery capabilities. Other than that, great company! Keep up the good work Syrex!
Score Card ( Click to Enlarge )
Tuesday, January 1, 2008
QuietMove ( Undergoing New Review)
The accuracy of the content of the existing QuietMove review was challenged by an employee at QuietMove. It is our policy to rebuild a review if the subject of the review feels that the initial review was untruthful or inaccurate in any way. Our goal is to be honest and to expose Professional IT Security Providers for what they really are.
Wednesday, December 19, 2007
Cybertrust ( C )
One of our readers made a request that we review Cybertrust ("http://www.cybertrust.com"). Cybertrust was recently acquired by Verizon and as a result this review was a bit more complicated and required a lot more digging to complete (In fact its now Cybertrust and Netsec). Never the less, we managed to dig information specific to Cybertrust out of Verizon representatives. We would tell you that we used the website for information collection, but in all reality the website was useless. Not only was it horribly written and full of marketing fluff, but the services were not clearly defined.
As an example, when you view the Cybertrust services in their drop down menu you are presented with the following service offerings: Application Security, Assessments, Certification, Compliance/Governance, Consulting, Enterprise Security, Identity Management Investigative Response /Forensics, Managed Security Services, Partner Security Program Security Management Program, and SSL Certificates. The first thing you think is "what the hell?" the second is "ok so they offer 12 services".
Well as you dig into each service you quickly find out that they do not offer 12 services, but instead they have 12 links to 12 different pages full of marketing fluff. As you read each of the pages in an attempt to wrap your mind around what they are offering as individually packaged services you're left with more questions than answers. So again, what the hell?
Here's an example. Their "Application Security" service page does not contain a description about a Web Application Security service. In fact, it doesn't even contain a description about a System Software/Application security service. Instead it contains a super high level, super vague and fluffy description that covers a really general idea of "Application" security services. When you really read into it you find out that their Application Security service should be broken down into multiple different defined service offerings.
Even more frustrating is that their Application Security service is a consulting service and that they have a separate service offering called Consulting. When you read the description for Consulting, it is also vague and mostly useless, but does cover the "potential" for Application Security.
So, trying to learn anything about Cybertrust from their web page is like trying to pull teeth out of a possessed chicken. We decided that we would move on and call Cybertrust to see what we could get out of them with a conversation. That proved to be a real pain in the ass too as their website doesn't list any telephone numbers. We ended up calling verizon and after talking to 4 people we finally found a Cybertrust representative.
At last, a human being that could provide us with useful information and answers to our questions about their services. We did receive about 2mb of materials from our contact at Cybertrust, but the materials were all marketing fluff, totally useless. That being said, our conversation with the representative gave us a very clear understanding of how Cybertrust delivers there services. In all honesty, we were not all that impressed.
Cybertrust does perform their own Vulnerability Research and Development (or so we were told) under the umbrella of ICSAlabs which they own. Usually we'd say that this is great because that research is often used to augment services and enhance overall service quality. With respect to Cybertrust, we couldn't find out what they were doing with their research. They just told us that they don't release advisories and then refused to tell us what they did with the research.
When we asked them about their services and testing methodologies, we were first told that they couldn't discuss that. We were told that their methodologies were confidential. But after a bit of Social Engineering and sweet talking we were able to get more information...
As it turns out, the majority of the Cybertrust services rely on what they say are proprietary automated scanners which were developed in-house. Their methodology is to run the automated scanners against a specific target or set of targets, and then to pass the results to a seasoned professional. That professional then verifies the results via manual testing and produces a report that contains the vetted results.
This methodology doesn't really offer any depth and doesn't do much to raise the proverbial security bar. In fact, it is only slightly better than running a Qualys scan, changing the wording of the report, and delivering that. Quality methodologies should contain no more than 20% automated testing and no less than 80% manual testing. Vulnerability discovery should be done via manual testing, not just via automated testing.
In defense of Cybertrust, they did say that they would test in accordance with the customers requirements. They also did say that if the customer wanted 100% manual testing that they would do it. If they want 100% automated "rubber stamp of approval" testing they would do that too. Saying it is a lot different than doing it though and we weren't impressed with their standard/default testing methodology as previously mentioned.
It is important to note that Cybertrust is also a full service security provider. They offer a wide range of services from supporting secure product development services, to security testing, and even forensic services. With that said, their services do not seem to be anything special. In fact, they seem to be just about average short of their horrible website and overwhelming marketing fluff.
It is our recommendation that you choose a different provider if you are looking for well defined, high quality services. Cybertrust is cloaked in a thick layer of marketing fluff and frankly doesn't seem to be very easy to work with. That being said, they were also not easy to review. If you disagree with this post or have worked with Cybertrust in the past, then please leave us a comment. We're going to give Cybertrust a "C" but if you can convince us that they deserve a different grade then we'll revise our opinion.
Score Card ( Click to Enlarge)
Thanks for reading.
As an example, when you view the Cybertrust services in their drop down menu you are presented with the following service offerings: Application Security, Assessments, Certification, Compliance/Governance, Consulting, Enterprise Security, Identity Management Investigative Response /Forensics, Managed Security Services, Partner Security Program Security Management Program, and SSL Certificates. The first thing you think is "what the hell?" the second is "ok so they offer 12 services".
Well as you dig into each service you quickly find out that they do not offer 12 services, but instead they have 12 links to 12 different pages full of marketing fluff. As you read each of the pages in an attempt to wrap your mind around what they are offering as individually packaged services you're left with more questions than answers. So again, what the hell?
Here's an example. Their "Application Security" service page does not contain a description about a Web Application Security service. In fact, it doesn't even contain a description about a System Software/Application security service. Instead it contains a super high level, super vague and fluffy description that covers a really general idea of "Application" security services. When you really read into it you find out that their Application Security service should be broken down into multiple different defined service offerings.
Even more frustrating is that their Application Security service is a consulting service and that they have a separate service offering called Consulting. When you read the description for Consulting, it is also vague and mostly useless, but does cover the "potential" for Application Security.
So, trying to learn anything about Cybertrust from their web page is like trying to pull teeth out of a possessed chicken. We decided that we would move on and call Cybertrust to see what we could get out of them with a conversation. That proved to be a real pain in the ass too as their website doesn't list any telephone numbers. We ended up calling verizon and after talking to 4 people we finally found a Cybertrust representative.
At last, a human being that could provide us with useful information and answers to our questions about their services. We did receive about 2mb of materials from our contact at Cybertrust, but the materials were all marketing fluff, totally useless. That being said, our conversation with the representative gave us a very clear understanding of how Cybertrust delivers there services. In all honesty, we were not all that impressed.
Cybertrust does perform their own Vulnerability Research and Development (or so we were told) under the umbrella of ICSAlabs which they own. Usually we'd say that this is great because that research is often used to augment services and enhance overall service quality. With respect to Cybertrust, we couldn't find out what they were doing with their research. They just told us that they don't release advisories and then refused to tell us what they did with the research.
When we asked them about their services and testing methodologies, we were first told that they couldn't discuss that. We were told that their methodologies were confidential. But after a bit of Social Engineering and sweet talking we were able to get more information...
As it turns out, the majority of the Cybertrust services rely on what they say are proprietary automated scanners which were developed in-house. Their methodology is to run the automated scanners against a specific target or set of targets, and then to pass the results to a seasoned professional. That professional then verifies the results via manual testing and produces a report that contains the vetted results.
This methodology doesn't really offer any depth and doesn't do much to raise the proverbial security bar. In fact, it is only slightly better than running a Qualys scan, changing the wording of the report, and delivering that. Quality methodologies should contain no more than 20% automated testing and no less than 80% manual testing. Vulnerability discovery should be done via manual testing, not just via automated testing.
In defense of Cybertrust, they did say that they would test in accordance with the customers requirements. They also did say that if the customer wanted 100% manual testing that they would do it. If they want 100% automated "rubber stamp of approval" testing they would do that too. Saying it is a lot different than doing it though and we weren't impressed with their standard/default testing methodology as previously mentioned.
It is important to note that Cybertrust is also a full service security provider. They offer a wide range of services from supporting secure product development services, to security testing, and even forensic services. With that said, their services do not seem to be anything special. In fact, they seem to be just about average short of their horrible website and overwhelming marketing fluff.
It is our recommendation that you choose a different provider if you are looking for well defined, high quality services. Cybertrust is cloaked in a thick layer of marketing fluff and frankly doesn't seem to be very easy to work with. That being said, they were also not easy to review. If you disagree with this post or have worked with Cybertrust in the past, then please leave us a comment. We're going to give Cybertrust a "C" but if you can convince us that they deserve a different grade then we'll revise our opinion.
Score Card ( Click to Enlarge)
Thanks for reading.
Subscribe to:
Posts (Atom)
