OpenSSL::Cipher decrypt returns 'wrong final block length'

rubyonrails-talk
1

I just want to encrypt a string submitted through a form before saving it to the DB. And then decrypt it again when I need to retrieve and use it.

Im trying to use the OpenSSL::Cipher library. I have the following module for encryption/decryption [code] require 'openssl'

module AESCrypt   # Decrypts a block of data (encrypted_data) given an encryption key   # and an initialization vector (iv). Keys, iv's, and the data   # returned are all binary strings. Cipher_type should be   # "AES-256-CBC", "AES-256-ECB", or any of the cipher types   # supported by OpenSSL. Pass nil for the iv if the encryption type   # doesn't use iv's (like ECB).   #:return: => String   #:arg: encrypted_data => String   #:arg: key => String   #:arg: iv => String   #:arg: cipher_type => String   def AESCrypt.decrypt(encrypted_data, key, iv, cipher_type="aes-256- cbc")     aes = OpenSSL::Cipher::Cipher.new(cipher_type)     aes.decrypt     aes.key = key     aes.iv = iv if iv != nil     aes.update(encrypted_data) + aes.final   end

  # Encrypts a block of data given an encryption key and an   # initialization vector (iv). Keys, iv's, and the data returned   # are all binary strings. Cipher_type should be "AES-256-CBC",   # "AES-256-ECB", or any of the cipher types supported by OpenSSL.   # Pass nil for the iv if the encryption type doesn't use iv's (like   # ECB).   #:return: => String   #:arg: data => String   #:arg: key => String   #:arg: iv => String   #:arg: cipher_type => String   def AESCrypt.encrypt(data, key, iv, cipher_type="aes-256-cbc")     aes = OpenSSL::Cipher::Cipher.new(cipher_type)     aes.encrypt     aes.key = key     aes.iv = iv if iv != nil     aes.update(data) + aes.final   end end [/code] And here is the model code where I encrypt and decrypt the string. [code]   def encrypt_cc_pass     return if cc_pass.blank?     self.cc_pass_key = Time.now.to_s     self.cc_pass_iv = Date.today.to_s     self.encrypted_cc_pass = AESCrypt.encrypt(cc_pass, cc_pass_key, cc_pass_iv)   end

  def decrypted_cc_pass     AESCrypt.decrypt(encrypted_cc_pass, cc_pass_key, cc_pass_iv)   end [/code]

And this is the error I get

wrong final block length in:config/initializers/aes_crypt.rb:20:in `final'

Anyone know what im doing wrong here? Also, Im not able to understand the inners of encryption here. Is there any gem with good documentation or simpler usage?

2

I'm going to seem rude probably, but it's still true.

You are dabbling in crypto and you don't know what you're doing. Stop, and find someone who does.

From what I read, you're trying to encrypt credit card data. Before you leak all the data from your database by using the _current time_ -- one of the weakest possible keys and IV values out there -- talk to someone, hire a consultant, or at least read 'applied cryptography.'

AESCrypt.encrypt("this", "X" * 32, "I" * 32)

=> "\223\335@\233\323d\254u9]\241\351\031M\301\352"

AESCrypt.decrypt("\223\335@\233\323d\254u9]\241\351\031M\301\352", "X" * 32, "I" * 32)

=> "this"

Your code works. Your lack of understanding about the algorithm you are using, the meaning of a key in that context, how to securely generate one, and how important a secure IV is, has bitten you.

Storing the key, IV, and encrypted data in a database is exactly the same as storing the unencrypted data. It will stop casual browsing. If that is what you are trying to do, there are far easier methods, such as converting the whole thing into base64 or hex or something. If you believe your code actually adds security...

--Michael

3

Michael,

I do agree with you in that I really have little to no idea of cryptography. Think I mentioned that in the OP itself. And I simply left out the complex IV and key for the sake of making this post a little more readable. Also, I have been reading up a bit and I hear the same of what you say in that its safer to store the iv and key in a database of their own. Thanks for that..

Well, the point though is that im getting an error that I dont have a solution for. And I think im giving bad input to the encrypt and decrypt methods and thats exactly what im trying to get someone to bring me up to speed with. Even in your example, im guessing you're generating a 32 bit something....? Well the real key and iv in my code are generated like this

Digest::SHA1.hexdigest("whatever").unpack('a2'*32).map{|x| x.hex}.pack ('c'*32)

Now I DID pick this up from the net so im not completely sure what this does either.

I would appreciate it if you could establish the basics that I need to know here. Ive been trying to find relevant ROR specific documentation for the OpenSSL gem but none that are understandable so far. Will look for 'applied cryptography'. Thanks for the response.

4

(There is a bit of hand-waving in this reply, but it'll get you started.)

AES-256-CBC means this: AES algorithm. Block size 256-bits. Cipher-block-chaining.

What this describes is a system where your data is encrypted in a 256-bit block, which GENERALLY means you need a 256-bit key. The documentation for the Ruby OpenSSL system is, as near as I can tell, non-existent. However, looking up the openssl library for C will tell you a lot -- Ruby just calls the C library your system has installed.

In this case, AES-256-CBC requires a 256-bit key and IV. (IV is "initialization vector" and is generally a random value. Its purpose is to make the same plaintext encrypt to different cyphertext.)

In my example, I generated 32-character long strings using the format: "X" * 32. This will make a string with 32-X's in it. Probably not all that secure, but works for an example. :slight_smile:

If you are running on a Unix system with /dev/random, and it has sufficient "real" randomness, you can use it to generate your IVs, and in your implementation, keys.

  File.open("/dev/random") do |rnd|     key = rnd.read(32)     iv = rnd.read(32)   end

If you are unable to use /dev/random, you could use /dev/urandom, but understand what the difference is. /dev/random is probably strong (check with your system docs) and unless random data is available, will "block" -- not return until there is. /dev/urandom is pseudo-random, and will return always. However, it will use what little randomness exists and stretch it out.

Note that your system is a lot like the method the restful_authentication plugin does. restful_authentication uses a global, per-application "secret" that is used much like a key, and a per-user "salt" which is used as an IV. Its only purpose is to make the password "mypassword" look different across users. The main difference is that you can get the data back, while restful_authentication is a one-way thing -- you can't recover a password from what it stores.

For your purposes, you can probably generate a one-time "secure" secret key, much like restful_authentication does, and store it in a global. You would then generate per-item IVs of length 256-bits (32 characters) and store those, along with the encrypted CC info.

This is much more important to do if you have your application on one server and the database on another. The database itself does not have all the bits needed to open the data. Unfortunately, most people will break into your web server, not your database engine.

As for what your digest thing does, it appears to "stretch" the passphrase out from an ASCII format to something much longer. This is commonly referred to as "key crunching" and does not make a more secure key than just padding the value out by duplicating the passphrase until it is long enough. That is, the digest thing is more or less the same as using the string: "whateverwhateverwhateverwhatever". The difference is that you can look at the readable one and know it is a very bad key to use, while 9755571c6b7df2cdb1dddaaae2b399b7 almost looks random. (It's not.)

--Michael

5

Phew! That was a LOT of new information all at once! Thanks a lot for the crash course! :slight_smile: The restful_authentication analogy helped and I get what you mean by thats one-way. Now about your suggestion for generating a one-time key and storing it in a global variable, not having decided on the hosting yet, I was wondering if its still secure if both the database and the application were on the SAME server. Or is it best to keep the IV/key on a separate server? Oh and yes, the Digest thing.. what you said helped me understand better. :slight_smile: By the way, I tried your test code above on IRB and I dont get the same encrypted string. Here're the results..

AESCrypt.encrypt("this","X" * 32,"I" * 32)

=> "\223?@\233?d\254u9]\241?M??"

AESCrypt.decrypt("\223?@\233?d\254u9]\241?M??","X" * 32,"I" * 32)

OpenSSL::CipherError: wrong final block length   from /Users/fire/Sites/Vinay/ROR/RealApps/fi_rest_auth/config/ initializers/aes_crypt.rb:20:in `final'   from /Users/fire/Sites/Vinay/ROR/RealApps/fi_rest_auth/config/ initializers/aes_crypt.rb:20:in `decrypt'   from (irb):2

Im guessing the ? marks suggest something? Thanks again for the guidance Michael. Appreciate it :).

6

Storing the key at all where someone can get to is insecure. You really need to question if you should store CC info at all. What purpose do you need to store it for? If recurring charges, most processing centers will allow you to schedule that sort of thing and not have to store the CC info locally. If logging, can you perhaps store only the last few digits of the card?

If you are storing it "just in case" but don't expect to use it, I would use a public key system, such as RSA. In RSA, you keep one of the keys (the public one) on the server, and it encrypts data. To get it out, you need the private key. People can hack away but so long as they don't have the private key, they cannot read your data. (more hand waving here, it IS possible, but we're entering into the world of theoretical quantum computing here.)

The numerous times I've had to handle CC data I was explicitly forbidden to store the numbers (card, expiry date, or CCV) other than the last 6 digits of the card. I could store anything returned from the processor though, like auth numbers, status codes, etc. This was in all our cases sufficient to reverse charges (if needed) or to bill in a recurring manner.

As for the examples, I think your "irb" is hiding data from you. Try something like:

x = AESCrypt.encrypt(...) AESCrypt.decrypt(x, ...)

I think your output is the same as mine, you just have ? marks where I have some \301 escaped characters.

--Michael

7

Michael Graff wrote:

Storing the key at all where someone can get to is insecure. You really need to question if you should store CC info at all. What purpose do you need to store it for? If recurring charges, most processing centers will allow you to schedule that sort of thing and not have to store the CC info locally. If logging, can you perhaps store only the last few digits of the card?

If you are storing it "just in case" but don't expect to use it, I would use a public key system, such as RSA. In RSA, you keep one of the keys (the public one) on the server, and it encrypts data. To get it out, you need the private key. People can hack away but so long as they don't have the private key, they cannot read your data. (more hand waving here, it IS possible, but we're entering into the world of theoretical quantum computing here.)

The numerous times I've had to handle CC data I was explicitly forbidden to store the numbers (card, expiry date, or CCV) other than the last 6 digits of the card. I could store anything returned from the processor though, like auth numbers, status codes, etc. This was in all our cases sufficient to reverse charges (if needed) or to bill in a recurring manner.

As for the examples, I think your "irb" is hiding data from you. Try something like:

x = AESCrypt.encrypt(...) AESCrypt.decrypt(x, ...)

I think your output is the same as mine, you just have ? marks where I have some \301 escaped characters.

--Michael

Hi,

I wonder whether this was resolved.

I'm having a similar issue.

I encrypt using a Flash AES CTS cipher from the library (hurlant). I'm decrypting in Ruby. When used in Flash for both encrypt and decrypt, all is joyous. However, when I try to use the openSSL package within Ruby - the one mentioned above, I am able to decrypt most of the cipher text. However, the last 17 bytes are garbled.

Lots of Google searching...

This is a test system, and in it, I set the IV to 16 bytes of 0. (I really don't want to be yelled at here about the lack of security in such a scheme)...

I get the same 'OpenSSL::CipherError: wrong final block length' error when calling the 'final' method. 'update' works like a charm. I see all of the clear test, in 32 byte chunks, up to the last partial block. Then, I see garbled data in the last partial.

So, I'm guessing either the hurlant package handles the last partial differently than Ruby's openSSL, or (much much much more likely), I am doing something obviously wrong.

Could a kind soul please let me know if they have experience and overcome such a problem?

Thanks, Mike

8

Don't worry, you know what an IV is and why using 0 is bad, so I won't 'yell' at you :slight_smile:

You are probably running into padding issues. Since AES is a block cypher, each block must be exactly the correct length I suspect.

How are you calling the decrypt code? You're just feeding data into the encryption system and after you are all done with the data, calling final? That should work, but I do not have any way to write code in Flash to test this. If you could have Flash encrypt something, perhaps 101 bytes or so long (odd to ensure padding occurs), and send the plaintext, the key, and the encrypted string (uuencode, hex dump, whatever) I can experiment.

--Michael

9

Mike Cook wrote:

Michael Graff wrote:

Don't worry, you know what an IV is and why using 0 is bad, so I won't 'yell' at you :slight_smile:

You are probably running into padding issues. Since AES is a block cypher, each block must be exactly the correct length I suspect.

How are you calling the decrypt code? You're just feeding data into the encryption system and after you are all done with the data, calling final? That should work, but I do not have any way to write code in Flash to test this. If you could have Flash encrypt something, perhaps 101 bytes or so long (odd to ensure padding occurs), and send the plaintext, the key, and the encrypted string (uuencode, hex dump, whatever) I can experiment.

--Michael

Hi,

Thanks... Definitely appears to be padding issues. We are using Bouncy Castle CTS AES encrypt. The openSSL appears to be using CBC. I'm no expert... In looking at the RFC, it appears there is a difference between how CBC and CBC/CTS handles padding. There is a reference in the rfc to swapping within the last partial block as part of the process... not sure how/whether this is implemented.

Below is the output from my irb session. The clear text is supposed to be: AAAAAAAABBBBBBBBAAAAAAAABBBBBBBBCCCCCCCCDDDDDDDDCCCCCCCCDDDDDDDDAAAAAAAABBBBBBBBAAAAAAAABBBBBBBBCCCCCCCCDDDDDDDDCCCCCCCCDDDDDDDDXXXYYY

key: 4b1114cc73fed8b5428c3dee60d7773a

Please notice that the first part of the message is decrypted correctly.... the partial at the end is not.

irb(main):821:0> a= OpenSSL::Cipher::Cipher.new('aes-256-cbc') => #<OpenSSL::Cipher::Cipher:0xb79b9378>

irb(main):822:0> a.key=d => "4b1114cc73fed8b5428c3dee60d7773a"

irb(main):823:0> a.iv='00000000000000000000000000000000'.unpack('a2'*32).map{|x| x.hex}.pack('c'*32)=> "\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"

irb(main):824:0> data='TmI9HrNrsMBxSfwApvSaQrLIDsLboNhIW/FawPjNUB0x/G0ZDf+gfk4JaTc/tGxDg1s4mrIRFOoBJemK+txUF0+aPw8bxIgzxmB3gq18aJRoSo5PWqbzS8FCCHrb3leKf4UUNFaIAaVVY1a5ymZ/HMPhwAKbii8x9Uk/S0MxaDofHTluc1E='.unpack('m')[0] => "Nb=\036\263k\260\300qI\374\000\246\364\232B\262\310\016\302\333\240\330H[\361Z\300\370\315P\0351\374m\031\r\377\240~N\ti7?\264lC\203[8\232\262\021\024\352\001%\351\212\372\334T\027O\232?\017\e\304\2103\306`w\202\255|h\224hJ\216OZ\246\363K\301B\bz\333\336W\212\177\205\0244V\210\001\245UcV\271\312f\177\034\303\341\300\002\233\212/1\365I?KC1h:\037\0359nsQ"

irb(main):825:0> s=a.update(data) => "AAAAAAAABBBBBBBBAAAAAAAABBBBBBBBCCCCCCCCDDDDDDDDCCCCCCCCDDDDDDDDAAAAAAAABBBBBBBBAAAAAAAABBBBBBBBCCCCCCCCDDDDDDDD8\300u\003|\200\243jf\bkf\005\251\327\242"

irb(main):826:0> s<<a.final OpenSSL::Cipher::CipherError: wrong final block length   from (irb):826:in `final'   from (irb):826   from :0 irb(main):827:0>

As a follow on, I was just reading an interesting article on Wikipedia.

Here's a snippet from it:

CBC ciphertext stealing decryption using a standard CBC interface

   1. Dn = Decrypt (K, Cn−1). Decrypt the second to the last ciphertext block.    2. Cn = Cn || Tail (Dn, B−M). Pad the ciphertext to the nearest multiple of the block size using the last B−M bits of block cipher decryption of the second-to-last ciphertext block.    3. Swap the last two ciphertext blocks.    4. Decrypt the ciphertext using the standard CBC mode.    5. Truncate the plaintext to the length of the original ciphertext.

Looks like the above is the mechanism to translate the last couple of blocks of ciphertext to a form understandable by the CBC interface.

I think I understand points 1, 3, 4, and 5. I'm having a tougher time with number 2, but I get the basic idea.

Let's say the the message is 134 bytes long. Then, I'd do 134 % 16 and I get 6. If my block size is 128 bits (16 bytes*8 bits), then I'd subtract 48 bits (6 bytes * 8 bits). That means 80 bits would be used as a repeated pattern to pad the end of the message to make it evenly divisible by 16. So, I'd add in one iteration (10 bytes) in this case, to bring the message to 144 bytes (evenly divisible by the block size - 16).

Anyway, that's what I'm going to try. Please wish me luck :wink:

Mike

10

Hey there... well... took a while but I got it to work by going through some code with a colleague and translating it to Ruby. The CTS cipher is now being deciphered using CBC.

I hope this helps some other folks in Ruby land. Please suggest some ways to better document the steps.

Take care, Mike

#key d='4b1114cc73fed8b5428c3dee60d7773a'

tmpArray=nil aes=nil alpha=nil omega=nil

#cipher text, unpacked from Base64 a='TmI9HrNrsMBxSfwApvSaQrLIDsLboNhIW/FawPjNUB0x/G0ZDf+gfk4JaTc/tGxDg1s4mrIRFOoBJemK+txUF0+aPw8bxIgzxmB3gq18aJRoSo5PWqbzS8FCCHrb3leKf4UUNFaIAaVVY1a5ymZ/HMPhwAKbii8x9Uk/S0MxaDofHTluc1E='.unpack('m')[0]

#compute the extra cipher beyond a 16 byte boundary partial = a.length % 16

#set up the beginning and end of the CTS blocks we need to work on alpha = a.length-16-partial omega = a.length-1

#grab the end of the cipher c=a.slice(alpha..omega)

#start + partial..end-partial c<<a.slice(alpha+partial..omega-partial)

#first decrypt the beginning

aes = OpenSSL::Cipher::Cipher.new('aes-256-cbc') aes.decrypt aes.key=d aes.iv='00000000000000000000000000000000'.unpack('a2'*16).map{|x| x.hex}.pack('c'*16) firstBlock=aes.update a.slice(0..((a.length/32)*32)-1)

#next decrypt the end aes = OpenSSL::Cipher::Cipher.new('aes-256-cbc') aes.decrypt aes.key=d aes.iv='00000000000000000000000000000000'.unpack('a2'*16).map{|x| x.hex}.pack('c'*16) decrypted3=aes.update c

lastBlock="" #here we need to Xor the result against the contents of end of the undecrypted cipher text (0..partial-1).each {|index| tmpA='';tmpA= (decrypted3[index]^c[16+index]).chr; lastBlock<<tmpA}

#put a number of bytes of undecrypted c then followed with the last number of the decrypted c... the number of bytes sliced depends upon the number of bytes in the partial block at the end of the cipher...

tmpArray = c.slice(16..16+partial-1)<<decrypted3.slice(partial..decrypted3.length-1)

#now handle the middle aes = OpenSSL::Cipher::Cipher.new('aes-256-cbc') aes.decrypt aes.key= d aes.iv='00000000000000000000000000000000'.unpack('a2'*16).map{|x| x.hex}.pack('c'*16) tmpArray=tmpArray<<tmpArray middle=aes.update tmpArray

#now take the last block of the decipherable original block last = a.slice(alpha-16..alpha-1)

midBlock=""

#and ^or it against the middle component (0..middle.length-1).each {|index| tmpB='';tmpB<<(middle[index]^last[index]).chr;midBlock<<tmpB}

finalBlock = firstBlock+midBlock+lastBlock