This is an article about honeypot awareness.
“Weird heroes and mould-breaking champions exist as living proof to those who need it that the tyranny of the 'rat race' is not yet final.”
~ Hunter S. Thompson
What is a honeypot? A honeypot is a security resource set up specifically to draw in unsuspecting visitors and thereby compromise their security. The classic honeypot example in the "VPN industry" is that run by CumbaJohnny. This "VPN service" was set up, operated, and designed solely to gather information on 'carders' using it, and thereby prosecute them. There are others, although none as well-documented publicly and as bright-line in their goals as that one (we have found a honeypot VPN service run by a... an entity, and have been tracking it for more than a year - that story will continue elsewhere).
But apart from pure-form honeypots there's a sliding scale down from there. How about the "VPN service" that uses such badly designed encryption that it is effectively useless against even the most minimal surveillance effort? Customers pay to use it, largely unaware that this "private network" is cryptographically useless. The classic case of this was iPredator in its early years. As a reseller of Relakks' PPTP-based "VPN service," iPredator was offering a security tool that was functionally useless in securing anything. Eventually, when confronted, Peter Sunde admitted the tool was useful only to make a "political statement" and not as a security technique. Today, iPredator offers more competent service - although at last check, they still supported PPTP, as well.
Is that old-form iPredator a honeypot? The usage of honeypot usually suggests some sense of intentional setup: a trap. In that sense, it's a bad match. All evidence suggests iPredator offered only PPTP because it was cheap, easy to deploy, built-in to most client OSes, and what Relakks had already used. That's poor security procedures, but not a honeypot.
How about a "VPN service" that advertises itself aggressively as "secure" and "private" with "no logs," but privately acknowledges that it hands over dockets on its customers to LEO dozens of times a month, on an "unofficial" basis - no warrants required? In this case, the company is actively marketing itself as "secure" and its marketing emphasizes the "no logging" element - which, of course, is total nonsense. This is much closer to a honeypot: there's an intentional misrepresentation, an effort to make visitors feel safe while knowing they're anything but.
So how do you know if a security service you are using is a honeypot?
We get asked this question alot. Often, it's from "trolls" or paid shills for other "VPN services" who are looking to limit competent competition. It's the nature of the business that such things happen; we take it in stride. Sometimes, smart members or prospective members ask about honeypot concerns, and we point them to that CumbaJohnny article and encourage them to do their own digging and research, and make their own decisions. That's all well and good, but there's not much out there worth reading on this subject, unfortunately. What is there usually has that kind of linear, overly-simplistic "don't get caught in honeypots!!" sort of useless advice that has nothing to say in terms of specifics.
So we've been kicking around our in-house views and advice on this topic. This article summarises what we know.
First off, talking about - and asking about honeypots and honeypotting and general trust in technology is good. Without discussion and questions being asked, this whole topic gets shrouded in FUD and whispered nonsense - that doesn't improve security and it doesn't keep people safe. On the flipside, talking about honeypotting and honeypot awareness will inevitably result in more accusations of being a honeypot - once folks realise this is an important topic with few cut-and-dried answers, they start to see honeypots everywhere. The pendulum swings. We are used to this, and on balance it works out ok,
But the real question is: how can someone determine whether a given service is a honeypot, or not? What's the punchlist to make that determination with confidence? And, in short, there is no such punchlist and no way to answer definitively. Sorry. That's reality.
So the first lesson is this: anyone who tells you they can prove they aren't a honeypot is, at best, not credible in their expertise and, at worst, has something overt they are trying to hide (like being a honeypot). There's things we might feel help gain confidence in a resource, but nothing to prove it's solid.
The flipside is that it is possible - on very rare occasions - to prove that something is a honeypot. Listen to those warnings, of they come! In the CumbaJohnny case, one researcher (Max Vision) noted that the server sometimes leaked IP addresses directly tied to the FBI. That's pretty much solid proof, by anyone's definition. He was largely ignored, under the assumption he was just a jealous admin of a competing site (which was true) and that his evidence was faked (which was not true - it was real). "I heard from this guy who heard from this guy" isn't hard evidence; nor, sadly, is the much-loved screenshot. It is very easy to fake most any screenshot, sorry but true. If you are experienced enough to understand the details of this sort of thing, you'll know if a service leaks a proof-positive instance of honeypotting. Watch for those, although they're quite rare.
That leaves us with a very large middle ground - not proved honeypots, but also no way to prove them 100% secure. Here's our rules of thumb, that we as a team use personally and have developed over decades of life out in the digital wilds...
- 1. Something looks too good to be true? That's a concern. The honeypot service we mentioned, that we've been tracking for a while, gives away their service. That makes you wonder, doesn't it? Note: this is not to say any free service is a honeypot, so just stop ok? We're saying that too good to be true is a possible red flag. Same goes for ridiculous claims of magical crypto or whatnot: if it sounds fishy, check deeper.
2. If it's too shiny and perfect, think twice. This one is really subjective, but we stand by it. Real life has bumps and scratches and bits of chuff sitting around. That's life. Real teams, working hard and under pressure and tight on cash, miss stuff like that sometimes - it happens. Broken link on a website, etc. In contrast, things that are so perfect they just sparkle make us nervous. There's a certain twitter account, and "he" seems to be available 24/7. Every link is perfect. Every page has Excellent Graphics(tm). Each post is without typo, every blog entry formatted to spec. This is totally, wildly unrealistic for anyone who actually lives in tech. For the general public, it seems an image they love: the Super Hacker Elite, no mistakes. Hoo-rah! But in reality, that's not how it is. What such perfection suggests is a great team: PR flack, a few interns, steely-eyed ops managers, etc. Think of those rooms of spooky-good workers in the Bourne movies. They don't have many typos, their blog posts go out on time, they don't drunk-tweet. Watch for a drunk-tweet now and again... a sign of mortal humans, not honeypots run by efficient LEO.
3. If the people associated with the project do strange and organic things, that's a good sign. Some projects have had coders who embodied nasty, ugly ideologies regarding racial topics. That's sad... but also not likely to happen in a well-run honeypot, is it? Not impossible of course - an existing project that gets "turned" could have all these little rough corners... but on balance, strangely discongruent stuff helps build confidence. No governmental agency or competent LEO operation is going to put a hard-drinking racist slob in charge of the servers... even if she's the best in the world at that particular job. Too risky, not their style.
4. Idiosyncratic tech choices can go either way. LEO and honeypots in general are going to go for low-risk, boring tools with licensing agreements and sales reps, on average. Wild-eyed, loopy, big-dreaming crypto tech teams usually have at least one erratic tech choice in the mix, and often more than one. We only communicate by... Pond! Or: our servers all run... whonix! You get the point. Real technologists develop fetish-like obsessions with weird areas of tech, often somewhat impractical and difficult to understand from the outside. This is a badge of authenticity, however, frustrating it may be otherwise. A tech team with no such fascinations? Again, just a bit too white-gloved and perhaps worth a second look.
5. Backstory. This is a big one, a very big one. Every tech team - every man or woman in the security tech world - has a backstory. Some really don't want to share those backstories, for any of a host of reasons. Fair enough. Some want to splash their PR pictures all over the website. Again: fair enough. Some are shy, some introverted, some loud-mouth braggarts. They're all, unquestionably, people: human people, with flaws and history and strange quirks and dark corners and, as often as not, more than a few scuffed spots somewhere in the past. Few will post all that on the project blog... not unheard-of, but rare. More common, folks will suggest that they're "known in the community." A bit of asking around, someone who knows someone who knows someone... and there's likely someone who got drunk with that person at some con a decade ago and ended up in a public restroom singing marching tunes. Or whatever. Point being: this is a very, very small world - the security tech world. Everyone knows everyone, everyone has history... and if people show up (or personas, really) that nobody knows firsthand? That's odd. No dirty stories of old days gone bad? A little odd, unless they're academics who tend to be more white-gloved (not always!). No fingerprints left anywhere in terms of past projects, failed startups, burned colleagues, jilted lovers, embarrassing rap sheets? Suspicious as fuck. Sorry, that's how we call it.
6. Shifty about discussing honeypots, snitching, LEO, and in general questions of trust? That's a concern, right there. Some folks get furious when accused of snitchy honeypotting. Some ignore it as beneath them, snooty and contemptuous. Some try to argue the trolls to a standstill when such questions come up, and some fume and vow revenge. All are, in a sense, valid replies - human replies. Honeypots seem to float above this fray, often as not. A shiny, teflon veneer. No response if questioned about such things: no emotion. This isn't a 100% rule, and indeed no honeypot rules are (see above). Some honeypots in other areas have been super-aggressive in attacking anyone who questioned them. That can be a red flag, too. Responding with indignation is one thing; going all red-hot-vengeance is sort of over the top. Mostly, look for a human, imperfect, varied, erratic, slightly ragged response... that's how reality often is. Good days, bad days... variation. And variation among the team. Some might be dismissive, some steely-eyed angry. Variation makes sense, for a real team.
7. Weird, unexplained absences that never really get folded into the narrative are a huuuuuge red flag. LEO seems to do these sorts of days-long absences - for training, for meetings, for whatever - far more than do real tech ops teams. Real teams are used to being paged 24/7, pinged by phones, tweeted at in the shower, called, jabbered, IM'd... the works. We might vanish for a few days due to exhaustion, personal crusades, whatever - but usually these vanishments fit in somehow, even if in only a jagged and weird way for folks watching from a distance. But the LEO vanishments, they seem to happen unannounced - and remain unexplained later, A drunk reply from an overworked sysadmin is really human and not totally uncommon, nor is a frazzled tech support staffer being needlessly crabby. Robotic drones that vanish for days, and then show back up as if nothing happened? That's a big read flag. Unless they were in jail, in which case... well, could go either way tbh.
8, Finally, and somewhat in summary, watch for gloves too white. This is security tech. It's not golf course management. That's not to say everyone in the infosec world is secretly a black hat rooting servers at night - obviously that's both silly and disrespectful and we make no such intimation. However, really.. if someone is so spooked by any rub-up with the seedier elements then that's a bit of a flag. Yes, the outfits selling 0days to spooky govs are less likely to be mixing with the hacker rabble... but not really, in fact. Where to the 0days come from? Where do they hire their analysts? Even those shops rarely have pure-white gloves. So if you see shiny-white gloves, what's that about?
Security tech and trust are intertwined. They always will be. Inside this little bubble of reality, many such decisions are made based on personal relationships and personal trust. We know someone, who knows someone, who has known someone for a very long time and trusts them - technically, personally, whatever. And we do make decisions about tech like this, often. Why use that OS, or that tool, or that parameter set? We know this gal, she's best-in-class. She is the uber-expert on that particular thing. And she says it's the bee's knees. She will talk your ear off for hours explaining why, and likely has. That - that matters. Listen to that, in our world.
Same goes for honeypot awareness. The deep tech people, the ones with roots and history and old feuds and scars and blurred memories and stories they'd rather not tell about mistakes they wish they didn't make? Ask them. They may have an ugly feud with a team or a person... but they'll likely know if that's a legit project or not. If nobody knows a team, nobody can say good or bad? That's a big flag.
To wrap it up, scars and rough edges prove a real existence. Nobody gets far in this space without racking up a good bit of both. Enemies, failures, embarrassing episodes. Broken tech. Also of course smashing victories, brilliant code, vibrant github porftolios... it's all part of being genuine, the good and the bad. If you winnow out anyone and any project with any "bad," what you're doing is ensuring that real teams are out of the running - for any real team has scars as well as plaudits. If you do that, what's left is basically the fake - and a good-sized chunk of those are honeypots.
That's our view of the terrain, take from it what you will.