A quick post about an article on LinkedIn about ‘Pentester Syndrome‘ the link on linkedin is over http so apologies, it wasn’t me setting the tone or quality (http://www.cyberdefensemagazine.com/pentester-syndrome/)
As the thread expanded the position of the article was diluted to it’s best qualities, and I guess that’s the benefit of being able to challenge what is said, what is valuable and what is not. it’s worth looking at the thread on linkedIn to see the value and positions dilute, and or to re-enforce a view or introduce new perspectives
If you work in Enterprise security or in any technical domain you may have at one point had the pleasure of reading a report from a recent penetration test (otherwise known as ‘pentesting’). These tests are usually conducted against websites or exposed infrastructure to simulate a real-life attacker and are genuinely useful at evaluating your security posture at a particular point in time. That being said, there are also many misinterpretations of pentesting as an activity and pentesting results and reports that lead to a skewed perspective on risk in the Enterprise as a result of pentesting itself. I’ll cover off the main downsides to pentesting and how to compensate for these in an Enterprise environment so you don’t fall victim to ‘pentester syndrome’ yourself and get the best value out of your pentests.
Corrections: Pentesting is conducted against any asset in any position in a bid to demonstrate the presence of security or the lack of, and to what extent. engagements that simulate a real-life attacker are usually considered redteaming, as real-life attacks are heavy on phishing, harpooning and other methods to interactively obtain footholds. Pentesting is one facet of security assurance offerings, Pentesting is most common as it’s compliance driven. (if you’re doing pentests without a parental requirement, good for you!)
Pentests are frozen in time
One of the major downsides of pentesting today is that they don’t match the speed of development of modern applications. Most companies pentest annually but rarely will you now find an application or website that is only updated once a year. In today’s environments, applications and websites are updated frequently, sometimes once a day, and occasionally even more than that. Like I mentioned earlier, a pentest is a snapshot of your security posture at a particular point in time. That’s it. Once you’ve updated your website or application, those findings are potentially out of date, as you may have already introduced new configuration changes into your environment, which means potential new vulnerabilities.
Nope. The downside here resides within the organisation, If the organisation isn’t issuing security testing after code-changes, large reconfigurations or large environmental changes that might introduce new risks, that’s nothing to do with the annual pentest. – If an organisation doesn’t have an internal Pentesting capability, they should prioritise that, or continue to work with preferred suppliers to deliver work more frequently as per the speed of the changes.
Hear ye, my pentest begins today
As a pentest is supposed to be a simulation of a real-life attack, it’s always interesting to note the number of companies I’ve seen announce to all and sundry that they are beginning a pentest. This eliminates one of the primary advantages of the test – simulating a real-life attack. Telling everyone in advance of a test means everyone will discount logs, reports, and alerts generated by that test, effectively robbing you of a simulated incident response scenario. It’s a surprising discovery that once you stop announcing your pentests, no one detects them. If you cannot even detect a legitimate pentest with your current enterprise security setup, then it’s unlikely you’ll be able to detect a malicious attacker doing the same.
Nope. This person doesn’t know the difference in value of Penetration testing and Attack simulation (redteaming), or Purple teaming.
Pentesters don’t have the luxury of time in any engagement. A website pentest may go from 3-5 days, plus an extra day or two to write up the report. Because of this limitation, pentesters will gravitate towards low-hanging fruit – vulnerabilities that are easy to find, especially with automated tools. Vulnerabilities that required complex custom scripting or time investment won’t be looked at in depth simply because they don’t have time and this is completely normal. Attackers today don’t have any such time constraints so can spend as long as they like focusing on a single asset, dredging up obscure but nonetheless critical vulnerabilities that would require more time investment than a pentest can provide.
Pentesters dont gravitate towards low hanging fruit, they find it with comprehensive methodology with the goal of maximum coverage, maximum impact in a time boxed situation. … they don’t make low hanging fruit, they just find them and demonstrate the importance of configuration hardening and non-default configurations. internal pen testers with the right support and skills can eradicate the time-pressures, equally, some organisations game the fact that scope is so thin and time is so short, they know the report will be light, but that’s okay, because it means things can go live. *ahem* – if this is a subtle sell for Bugbounty style security work, I’ll say it again, IT’s NOT A PENTEST. (if you think it is, read this and get back to me)
Pentesting companies often have to prove their value to their customers – after all, they are paid lucrative rates to provide a niche service. Unfortunately for many, the ‘value’ derived from the point of view of the customer has been degraded to how many vulnerabilities were found. After all, if a company uses two different pentesting companies in a row, but one of them found more of them than the other company – are they not better?
This is where ‘pentester syndrome’ occurs – making things worse than they appear. I’ve had the pleasure (or misfortune – depending on your point of view) of having written or read hundreds of pentesting reports in my career. When a pentester doesn’t find any vulnerabilities of note, it becomes the norm to ‘talk up’ issues that can be classed as ‘informational’ to vulnerability status. This is damaging for many reasons, the first being, that you are being given a false assessment of your security posture. If you aren’t experienced or technical enough to translate these findings into quantifiable risk, you will then spend resource and time blindly chasing down remediations for these ‘vulnerabilities’ that have absolutely no bearing on your security posture and will never cause you any trouble. I’ve listed the kind of findings you don’t really need to spend too much resource on unless you need a very hardened security posture.
The recipient needs to apply context to the issues they’re looking at, if they aren’t capable of quantifying the issue in the context of their own application or infrastructure … they need to get some support, or find someone who can, dont blame the security consultant. if you lack the confidence to challenge it or explain why it isn’t that high perhaps find someone who can. or pay for some more time with the security consultant to break it down into crayon-talk so less experienced staff can comprehend/understand why it’s scored in such a way. after all, if your organisation is building, supporting or using a thing, your organisation is expected to command those things, accountability starts there.
Purple teaming is a great value point prior to implementing a Bugbounty program, the point with purple-teams being, the collaboration and clear goals of security improvement, education and attacker insight for internal security teams, again… Pentesting is a compliance driven requirement in most cases, you can hire ‘pentesters’ (security consultants) to do all manner of services, know this.
let’s have a look at Bugbounty syndrome for a moment (I’m 51% facetious ):
What kind of findings don’t I need to be worried about?
So a bit of a caveat: I don’t know what your risk profile is, so these might not all be applicable to you. What I can tell you, is that no one has suffered breaches due to the below by virtue of the fact that they only aid an attacker in very specific contrived situations, and then only for a specific kind of vulnerability or require a man-in-the-middle positioning which is, despite what you may hear, difficult to achieve.
Completely subjective. ignore that. fix your transport layer security, everyone will benefit from it, those that spend their time arguing round things like this tend to usually have a technical challenge they haven’t been able to overcome. be honest. google will hit your site with the shit stick if your TLS isn’t uptodate … do you want the UX/Marketing team up your arse ? isn’t it nice to have A+ on SSLLabs ? do your job.
No HSTS headers, any kind of SSL/TLS issues
A common one that finds itself onto pentest reports. If you have HSTS then client-side certificate errors will always be fatal (the user can’t click through the warnings) and that side can only be negotiated in HTTPS. What this means if someone tries to man-in-the-middle you, well they can’t. This also means it’s hardly likely to affect many users since a man-in-the-middle position for an attacker is hard to get. You can lump all SSL/TLS issues into this bucket, so things like weak ciphers and outdated versions of SSL are here. As the attack vector is identical (you need man-in-the-middle) these are never likely to affect a great number of individuals at once.
Completely subjective. ignore that. fix your transport layer security, everyone will benefit from it. (unless you’re dependent on crappy 3rd party code that doesn’t care about what you want since you’ve paid, remember that when renewals comes round).
Lack of Secure/HTTPOnly flags on cookies
This one’s another common culprit. The lack of the HTTPOnly flag on cookies will only ever help an attacker who’s found a cross-site scripting vulnerability and is only leveraging it to extract the cookie itself. As now we have things like the browser exploitation framework (BEEF) and many other vectors to exploit cross-site scripting apart from cookie theft, it is of limited use and is effective ‘hardening’ – it’s even less useful if they find no cross-site scripting vulnerabilities on your site. The ‘secure’ cookie only implies cookies can only be sent over HTTPS, so it also falls into ‘hardening’ like the HSTS issues above
Fix your web app you lazy shit, if XSS isn’t found on an engagement and you’re only doing them annually, who’s to say you won’t introduce XSS in the next code change? people who complain about the value of HTTPOnly usually have badly implemented analytics code. or if it’s acceptable, at the time of testing, it’s still worth reminding people of the pitfalls, if moving forward it’s never going to change. that’s the context the owner applies.
Use of a vulnerable library
I see this one a lot – usually affecting things like out of date third party libraries in products (things like jquery, reacts, etc.). A vulnerable library only indicates a method in a specific library is vulnerable if it is used and then only in a specific way. If it’s not used, no way you can be vulnerable. If you see this in a pentesting report, just ask for a proof of concept, and the finding usually gets taken off the report.
I have exploited many sites (XSS) due to vulnerable libraries’ (jQuery et al) – I dont think they understand why this is an issue, so they should be quiet.
Lack of x-frame-options header, user or account ID enumeration, lack of CAPTCHA, lack of password complexity requirements, inadequate SPF/DMARC/DKIM configs are all things which may appear but which will rarely cause you any trouble since they are not ‘vulnerabilities’.
X-frame prevents clickjacking, if your organisation doesn’t value the data its protecting or the threat of a successful clickjacking attack (usually user account theft) … as long as you can explain why you dont think you need those protections on your admin or your users … crack on. (and that usually takes more effort than implementing clickjacking defences)
similarly with anti-automation defences such as CAPTCHA if you’re happy resetting and locking your clients out and pretending you’re not being hacked all the time via password spray attacks … as long as you can explain that, crack on (dumbass).
Getting the most out of your pentests
So now that we’ve exposed the issues with pentesting today, how can we make them better. Some of them are down to pentesting being the incorrect tool for the job but a lot of it can be improved by engaging with your pentester provider a bit more.
This is called a Scope of Work, it’s done on every pentest. people are just getting lazy.
Even if you are not a technical individual or feel out of your depth when people start talking about XXE’s and Cross-origin resource sharing, you can always ask the pentester for proof of concept, especially if he’s written ‘X is vulnerable because of Y’. Pentesters will provide this by default for major vulnerabilities but may sometimes gloss over this part. Feel free to challenge the reports and have them modified. If they can’t provide proof of concept, then you can have it removed from the report.
Pentesters will always give you a PoC if they have time, if they dont have time, give them some. you’re paying for their time.
Don’t announce them
To get the ‘simulation’ get prior approval to stop announcing to the world when pentests are occurring. The data you gather will be more valuable and if your incident response procedures and operational teams detect them, that data can be fed back into a virtuous cycle of improvement. This is just as valuble if they aren’t detected. What failed? Why?
This is Attack Simulation/Redteam work, it’s not Pentesting.
Match them to major functionality improvements and updates
When your application or website has new functionality added or improved, try and cycle the pentest to coincide right after this. Now, this will largely be budget dependent. Alternatively, if you have frequent functionality updates, consider switching to crowdsourced security to replace some or all of your pentests, as this will mitigate the main weakness of pentests – that they occur at a single point in time.
If you have regular changes and deployments, you need a in-house security team, dont consider switching to crowdsourced security to replace some of all of your pentests, that’s stupid, and again, not the same deliverables. ffs.
One pentester will see vulnerabilities that a previous pentester missed, and this is entirely normal. To remedy this the best practice is to cycle your pentesters, be it by asking the vendor you use to send a different individual each time, or by just picking a different vendor each time. As offensive security skills are at a premium this will sometimes backfire and you’ll end up with the same pentester at a different company (this happens more often than you think). Switching to a crowdsourced security model remedies this entirely, as your pool of testers inflates so you have dozens if not hundreds of people testing your application.
I agree with this, having a preferred suppliers list of trusted organisations is good advice, for regular Pentesting, also keeps them on their toes.
Just another tool
Finally, pentesting is just another tool in your arsenal. You cannot get by with just a pentest, nor can you really get by without one. They are useful tools but are a complement to your arsenal, not the only thing. Whether you preach holistic security or ascribe to a defense in depth model then pentesting is just another aspect of that model. Just make sure you get the most out of it.
I agree with this too, but I would point out, Pentesting, attack simulation and crowd sourced security solve very different problems, know this first. and yes, defence in depth, like httponly,x-frame and TLS.
If I was to draw from the majority of this article it tells me either the Engineers aren’t confident in commanding the systems they support, or they’re too lazy or too busy to make these changes that would move their customers into higher states of security, maybe because the majority of concerns mentioned are client side, and clients aren’t as important as the servers. that’s a shame. it’s almost like people don’t want to be bothered with the technical elements of a very technical piece of work. Pentesting reports generally come in two parts, high-level summary and technical findings, if you’re a high-level kinda person, dont read the technical stuff.
that was exhausting. but does remind me of this https://www.youtube.com/watch?v=MtaUHMWQpiI