Ξ welcome to cryptostorm's member forums ~ you don't have to be a cryptostorm member to post here Ξ
∞ take a peek at our legendary cryptostorm_is twitter feed if you're into that kind of thing ∞
Ξ we're rolling out voodoo network security across cryptostorm - big things happening, indeed! Ξ
Ξ any OpenVPN configs found on the forum are likely outdated. For the latest, visit GitHub Ξ

compressing encrypted packet streams? lzo 101

Freewheeling spot to chew the fat on anything cryptostorm-related that doesn't fit elsewhere (i.e. support, howto, &c.). Criticism & praise & brainstorming & requests for explanation... this is where it goes when it's hot & ready for action! :-)
User avatar

Topic Author
DesuStrike
ForumHelper
Posts: 346
Joined: Thu Oct 24, 2013 2:37 pm

compressing encrypted packet streams? lzo 101

Postby DesuStrike » Sat Dec 07, 2013 8:43 am

I'm interested in what is bad about lzo compression and thus why it is a goal to disable it.

I honestly have no idea what this does... :\
home is where the artillery hits

User avatar

Pattern_Juggled
Posts: 1492
Joined: Sun Dec 16, 2012 6:34 am
Contact:

Re: cryptostorm: config & parameter settings (client & serve

Postby Pattern_Juggled » Thu Dec 19, 2013 4:24 pm

DesuStrike wrote:I'm interested in what is bad about lzo compression and thus why it is a goal to disable it.

I honestly have no idea what this does... :\


Quick reply, for now: I'm the one that neutered lzo in the production conf's & my thinking on that is bare-bones simple...

Mixing compression with real crypto has proved to be a Bad Idea, repeatedly. In theoretical terms, this is no surprise: think about what compression is doing, in the context of packetized (and thus, definitionally, block-ciphered) data, and at least a few of your hairs will turn white as you see the open attack vectors this inevitably creates.

Basically, all manner of padding-based attacks become far more feasible. HMACs get less useful/reliable (in general, although not universally). Timing attacks become alot more juicy (unless you take clever steps to prevent them, each of which step is another potential attack surface). Traffic analysis might well become more fruitful - feed patterned data into the tunnel, watch what happens to the packet size differentials, and profit...

Or: if I feed a never-ending series of 1s into the tunnel, I don't want anything characteristic in the encrypted packet traffic to tip the hand that something like that is going on, do I? If that happens, it's a side channel leak. You can say it's not, but it really is - just a semantic issue.

Now, I'll be the first to say that I have confidence a clever cryptographer could deploy compression within heavily-encrypted channels with no loss in security - indeed, I know it's been done, and is done in the wild. But I don't assume we're clever - I assume we're mortals, and make mistakes. If we can remove the risk of a mistake from the model, ceteris paribus, we do. One less possible fuck-up.

(I'm also troubled, at an ontological level, by the functional congruence between real crypto and lossless compression - being two sides of one coin, fundamentally - and the convergence of these two functions just doesn't make me feel like I want to try to do both at once, from two directions... not something I could defend in a dissertation presentation, but nevertheless a horse-sense feeling I am not willing to abandon just yet)

The "benefit" of lzo, best-case, is a few percent uptick in theoretical maximum tunnel capacity... with all sorts of footnotes limning that benefit further (since so many OSI layers do some form of compression, nowadays - does anything really stream a bunch of 1s off a physical NIC, in this day and age?). Fuck that. I'd much rather throw exitnode capacity at things, than play games with security parameters to try to squeeze a few more micro-clicks out of the model. It's just got bad idea written all over it.

There's all sorts of interesting theoretical literature to bring to bear on this question - compressed encryption, etc. - and perhaps a quiet snowy weekend I'll troll my academic records and pull the more accessible papers for additoon to this thread. Or someone else is free to stick a few google'd results into the discussion, here. Start with SSL vulns exposed via assumptive compression of HTTPS session streams - that's as obvious a place as any. And a damned good object lesson in the risks.

~ pj
...just a scatterbrained network topologist & crypto systems architect……… ҉҉҉

    ✨ ✨ ✨
pj@ðëëþ.bekeybase pgpmit pgpðørkßöt-on-consolegit 'er github
bitmessage:
BM-NBBqTcefbdgjCyQpAKFGKw9udBZzDr7f

User avatar

Topic Author
DesuStrike
ForumHelper
Posts: 346
Joined: Thu Oct 24, 2013 2:37 pm

Re: compressing encrypted packet streams? lzo 101

Postby DesuStrike » Tue Dec 24, 2013 5:22 pm

I didn't answer to this thread because I slowly start to feel spammy with all my "thank you" towards you and the team. On the other hand I don't want to create the impression I would not appreciate detailed answers to my question. So have my thanks!

I now understand the underlying concept why compression works towards predictable values but it is a complicated topic indeed. So I fully support your decision to "play it safe" instead of buying a neglectable advantage for the price of an attack vector.
home is where the artillery hits

User avatar

parityboy
Site Admin
Posts: 1092
Joined: Wed Feb 05, 2014 3:47 am

Re: compressing encrypted packet streams? lzo 101

Postby parityboy » Mon Feb 17, 2014 4:56 am

Noob question: if compression exposes predictable bit patterns, would it not be sensible/preferable/choose a "*ble" to compress first and then encrypt so that the "outside layer" of the data stream is the encryption? Or does encryption work better on larger data sets? Or am I completely off track?

User avatar

Pattern_Juggled
Posts: 1492
Joined: Sun Dec 16, 2012 6:34 am
Contact:

Re: compressing encrypted packet streams? lzo 101

Postby Pattern_Juggled » Mon Feb 17, 2014 4:42 pm

parityboy wrote:Noob question: if compression exposes predictable bit patterns, would it not be sensible/preferable/choose a "*ble" to compress first and then encrypt so that the "outside layer" of the data stream is the encryption? Or does encryption work better on larger data sets? Or am I completely off track?


In a sense, compression (lossless) and encryption are siblings in mathematical terms.

Historically, there have been some cases in which combining the two can result in unexpected, almost emergent, behaviours that expose subtle chinks in the armour of the crypto. I'm thinking, of course, of TLS when I say this.

This lzo issue being discussed above is both an ongoing research topic for the team and a seriously sore point for me, personally. It does indeed turn out that some implementations of OpenVPN client-side essentially mandate the existence of lzo in the packet stream - without it, a desynch of exactly four bytes per packet (payload) results... as anyone who has parsed a few hundred error logs will quickly realise and recognise. It's a bug, basically, but since everyone just enables lzo by default, few have noticed it.

I've noticed it, and burned many many hours wrestling with it in deployment and at the source code level. It's an ongoing... project. We'll call it a project, rather than a "bloody, merciless war" - just to be nice.

I haven't found much existing cryptoanalytic research on lzo yet, and haven't opened up time to put the question to the academic world as the next step in validating assumptions. It's on the to-do list, and will get done in due course.
...just a scatterbrained network topologist & crypto systems architect……… ҉҉҉

    ✨ ✨ ✨
pj@ðëëþ.bekeybase pgpmit pgpðørkßöt-on-consolegit 'er github
bitmessage:
BM-NBBqTcefbdgjCyQpAKFGKw9udBZzDr7f

User avatar

parityboy
Site Admin
Posts: 1092
Joined: Wed Feb 05, 2014 3:47 am

Re: compressing encrypted packet streams? lzo 101

Postby parityboy » Tue Feb 18, 2014 5:12 am

@PJ

Thanks for the reply. I'm no mathematician, but I kind of see where you're going with this. Another question: do your concerns apply to all lossless compression methods, or is there some characteristic of LZO in particular? Why was LZO chosen by the OpenVPN people, as opposed to the myriad of more widely used compression algorithms (bzip2, gzip etc)? (I've only ever seen LZO used with OpenVPN)

User avatar

parityboy
Site Admin
Posts: 1092
Joined: Wed Feb 05, 2014 3:47 am

Re: compressing encrypted packet streams? lzo 101

Postby parityboy » Mon Feb 24, 2014 7:17 pm

@PJ

Here's another question. If (lossless) compression and encryption are two sides of the same mathematical coin, and combining them can result in unpredictable behaviour, can the same not apply to multiple layers of encryption such as SSL (application level e.g HTTPS) over Tor (4 layers in total, but reduced in transit towards the exit node) or SSL over VPN (two layers), or even combinations of all three?


Return to “general chat, suggestions, industry news”

Who is online

Users browsing this forum: No registered users and 14 guests

Login