Asp Forum - Mimic AES_ENCRYPT and AES_DECRYPT functions in Ruby

Felipe Coury

3/24/2009 3:22:00 AM

Hello there!

I need to mimic what MySQL does when encrypting and decrypting strings
using built-in functions AES_ENCRYPT() and AES_DECRYPT().

Even though the application I am writing is a Rails application, I think
this question suited here better because, the encryption will take place
in Ruby and not necessarily depends on Rails.

I have read a couple of blog posts and apparently MySQL uses AES 128-bit
encryption for those functions. On top of that, since this encryption
requires a 16-bit key, MySQL pads the string with x0 chars (\0s) until
it's 16-bit in size.

The algorithm in C from MySQL source code is spotted here:
http://gtowey.blogspot.com/2009/01/mysql-aes-encryption-compatab...

I have even tried to examine MySQL's C source code, but that didn't help
me much, since I can't really program in C. Maybe someone with a little
more experience can have some insights.

The source code that implements the encryption (rijndaelKeySetupEnc) and
decryption (rijndaelKeySetupDec) functions is here:

http://pastie....

And the actual AES_ENCRYPT (function my_aes_encrypt) and AES_DECRYPT
(my_aes_decrypt) source code is here:

http://pastie....

Please note that the necessity of using MySQL's compliancy was not my
call and is not a choice. I need that in order to communicate properly
with a legacy application, and I don't "own" this database. Please take
into consideration that security is definitely not the goal, talking to
that system properly is. The key length was not chosen by me and I know
it's a little peculiar, as you'll see below, on my replication "script".

Now I need to replicate what MySQL does in a Rails application, but
every single thing I tried, doesn't work.

Here's a way to replicate the behavior I am getting (in this case, using
Rails):

1) Create a new Rails app

rails encryption-test
cd encryption-test

2) Create a new scaffolding

script/generate scaffold user name:string password:binary

3) Edit your config/database.yml and add a test MySQL database

development:
adapter: mysql
host: localhost
database: test
user: <<user>>
password: <<password>>

4) Run the migration

rake db:migrate

5) Enter console, create an user and update its password from MySQL
query

script/console
Loading development environment (Rails 2.2.2)
>> User.create(:name => "John Doe")
>> key = "82pjd12398JKBSDIGUSisahdoahOUASDHsdapdjqwjeASIduAsdh078asdASD087asdADSsdjhA7809asdajhADSs"
>> ActiveRecord::Base.connection.execute("UPDATE users SET password = AES_ENCRYPT('password', '#{key}') WHERE name='John Doe'")

That's where I got stuck. If I attempt to decrypt it, using MySQL it
works:

>> loaded_user = User.find_by_sql("SELECT AES_DECRYPT(password, '#{key}') AS password FROM users WHERE id=1").first
>> loaded_user['password']
=> "password"

However if I attempt to use OpenSSL library, there's no way I can make
it work:

cipher = OpenSSL::Cipher::Cipher.new("AES-128-ECB")
cipher.padding = 0
cipher.key = key
cipher.decrypt

user = User.find(1)
cipher.update(user.password) << cipher.final #=>
"########gf####\027\227"

I have tried padding the key:

desired_length = 16 * ((key.length / 16) + 1)
padded_key = key + "\0" * (desired_length - key.length)

cipher = OpenSSL::Cipher::Cipher.new("AES-128-ECB")
cipher.key = key
cipher.decrypt

user = User.find(1)
cipher.update(user.password) << cipher.final #=>
""|\e\261\205:\032s\273\242\030\261\272P##"

But it really doesn't work.

Does anyone have a clue on how can I properly mimic whatever MySQL is
doing in Ruby?

Thanks a lot for your help.

Cheers,

-- Felipe.
--
Posted via http://www.ruby-....

8 Answers

Felipe Coury

3/24/2009 3:25:00 AM

One additional information:

You may find weird that I tried padding the key, and even using \000
char to pad it. The reasoning behind it is the way MySQL documents those
two functions:

"AES_ENCRYPT() and AES_DECRYPT() allow encryption and decryption of data
using the official AES (Advanced Encryption Standard) algorithm,
previously known as â??Rijndael.â? Encoding with a 128-bit key length is
used, but you can extend it up to 256 bits by modifying the source. We
chose 128 bits because it is much faster and it is secure enough for
most purposes.

AES_ENCRYPT() encrypts a string and returns a binary string.
AES_DECRYPT() decrypts the encrypted string and returns the original
string. The input arguments may be any length. If either argument is
NULL, the result of this function is also NULL.

Because AES is a block-level algorithm, padding is used to encode uneven
length strings and so the result string length may be calculated using
this formula:

16 Ã? (trunc(string_length / 16) + 1)

If AES_DECRYPT() detects invalid data or incorrect padding, it returns
NULL. However, it is possible for AES_DECRYPT() to return a non-NULL
value (possibly garbage) if the input data or the key is invalid."

Hope that also helps.

Thanks again.

-- Felipe
--
Posted via http://www.ruby-....

Rob Biedenharn

3/24/2009 3:44:00 AM

On Mar 23, 2009, at 11:25 PM, Felipe Coury wrote:

> Because AES is a block-level algorithm, padding is used to encode =20
> uneven
> length strings and so the result string length may be calculated using
> this formula:
>
> 16 =D7 (trunc(string_length / 16) + 1)

Do you mean to have:

16 * (string_length + 15)/16

If the string length is 32, what do you expect the result to be? Your =20=

formula gives 48, (16*(trunc(32/16)+1))=3D=3D(16*(2+1)), while mine =
gives =20
32, (16*(32+15)/16)=3D=3D(16*(47/16))=3D=3D(16*2) [integer division].

Here's a bit of code that I've lifted out of another project:

# Encrypt the content of the document, block by block, in a manner
# compatible with the original Python (so we can decrypt it and
# remain backwardly compatible)
rijndael =3D Crypt::Rijndael.new(self.key, 256, 256)
encryptedData =3D ""
data << 'X' # a marker added to cope with partial block
blocks, bytes =3D data.length.divmod(32)
unless bytes.zero?
data << "\0" * (32 - bytes)
blocks +=3D 1
end
(0...blocks).each do |block|
encryptedData << rijndael.encrypt_block(data[block * 32, 32])
end

The decrypting side was Java and I don't know why the 'X' was chosen =20
(seems that I recall something about there being a byte with the =20
number of extra bytes of padding... or your formula might hold a clue).

Anyway, you'd have to adjust it for 128-bit/16-byte keys (and blocks).

-Rob

Rob Biedenharn http://agileconsult...
Rob@AgileConsultingLLC.com

gcristelli

3/24/2009 4:33:00 AM

On 24 Mar, 04:21, Felipe Coury <felipe.co...@gmail.com> wrote:
> Hello there!
>
> I need to mimic what MySQL does when encrypting and decrypting strings
> using built-in functions AES_ENCRYPT() and AES_DECRYPT().
> ...
> However if I attempt to use OpenSSL library, there's no way I can make
> it work:
>
> cipher = OpenSSL::Cipher::Cipher.new("AES-128-ECB")
> cipher.padding = 0
> cipher.key = key
> cipher.decrypt
>
> user = User.find(1)
> cipher.update(user.password) << cipher.final #=>
> "########gf####\027\227"
>
I use the following code for encrypt/decrypt:

@cipherAES256=OpenSSL::Cipher::AES256.new("CBC") if @cipherAES256.nil?
@cipherAES256.encrypt
@cipherAES256.key=key
ct = @cipherAES256.update(plainPassword) + @cipherAES256.final
password=ct.unpack("H*")[0]

@cipherAES256=OpenSSL::Cipher::AES256.new("CBC") if @cipherAES256.nil?
@cipherAES256.decrypt
@cipherAES256.key=key
ct = @cipherAES256.update([password].pack("H*")) + @cipherAES256.final

I don't know if it can helps you (it uses 256 and CBC), try changing
your code from

cipher.update(user.password) << cipher.final
to
cipher.update([user.password].pack("H*")) << cipher.final

Giovanni

Felipe Coury

3/24/2009 5:32:00 AM

Giovanni / Rob,

Thanks a lot for your responses, really.

Unfortunately, it it still doesn't work. I tried what you both
suggested, take a look at this console transcript:

>> def aes(m,k,t)
>> (aes = OpenSSL::Cipher::AES128.new("ECB").send(m)).key = k
>> aes.update(t) << aes.final
>> end
=> nil
>>
?> def encrypt(key, text)
>> aes(:encrypt, key, text)
>> end
=> nil
>>
?> def decrypt(key, text)
>> aes(:decrypt, key, text)
>> end
=> nil

>> key = "82pjd12398JKBSDIGUSisahdoahOUASDHsdapdjqwjeASIduAsdh078asdASD087asdADSsdjhA7809asdajhADSs"
=>
"82pjd12398JKBSDIGUSisahdoahOUASDHsdapdjqwjeASIduAsdh078asdASD087asdADSsdjhA7809asdajhADSs"

>> u = User.find(1)
=> #<User id: 1, name: "John Doe", password:
"###\270##\206Î®5\202?\003\021###", created_at: "2009-03-23 20:31:43",
updated_at: "2009-03-23 20:31:43">
>> u.password.length
=> 16

>> decrypt(key, u.password)
OpenSSL::CipherError: bad decrypt
from (irb):8:in `final'
from (irb):8:in `aes'
from (irb):16:in `decrypt'
from (irb):19

>> decrypt(key, [u.password].pack("H*"))
OpenSSL::CipherError: wrong final block length
from (irb):8:in `final'
from (irb):8:in `aes'
from (irb):16:in `decrypt'
from (irb):32

>> [u.password].pack("H*").length
=> 8
>> card = ([u.password].pack("H*") + ("\0" * 8))
=> "9##n###\005\000\000\000\000\000\000\000\000"
>> decrypt(key, card)
OpenSSL::CipherError: bad decrypt
from (irb):8:in `final'
from (irb):8:in `aes'
from (irb):16:in `decrypt'
from (irb):43

Any other ideas?

Thanks,

-- Felipe
--
Posted via http://www.ruby-....

Felipe Coury

3/24/2009 6:27:00 AM

Some more discoveries...

According to the blog post I sent before, here's how MySQL works with
the key you provide AES_ENCRYPT / DECRYPT:

"The algorithm just creates a 16 byte buffer set to all zero, then loops
through all the characters of the string you provide and does an
assignment with bitwise OR between the two values. If we iterate until
we hit the end of the 16 byte buffer, we just start over from the
beginning doing ^=. For strings shorter than 16 characters, we stop at
the end of the string."

I don't know if you can read C, but here's the mentioned snippet:

http://pastie....

Specially this part:

bzero((char*) rkey,AES_KEY_LENGTH/8); /* Set initial key */

for (ptr= rkey, sptr= key; sptr < key_end; ptr++,sptr++)
{
if (ptr == rkey_end)
ptr= rkey; /* Just loop over tmp_key until we used all key */
*ptr^= (uint8) *sptr;
}

So I came up with this method:

def mysql_key(key)
# The algorithm just creates a 16 byte buffer set to all zero,
final_key = "\0" * 16

# Number of string "blocks"
t = key.length / 16

t.times do |i|
# For each block
key_block = key[i*16, 16]

# Runs bitwise XOR for each char on string
# and the same char on the block
16.times do |j|
final_key[j] ^= key_block[j]
end
end

final_key
end

But it still fails:

>> key = "82pjd12398JKBSDIGUSisahdoahOUASDHsdapdjqwjeASIduAsdh078asdASD087asdADSsdjhA7809asdajhADSs"
=>
"82pjd12398JKBSDIGUSisahdoahOUASDHsdapdjqwjeASIduAsdh078asdASD087asdADSsdjhA7809asdajhADSs"
>> mkey = mysql_key(key)
=> "\027\024GK\023P{#8?G!8[r."
>> mkey.length
=> 16

>> decrypt(mkey, User.find(1).password)
User Load (11.3ms) SELECT * FROM `users` WHERE (`users`.`id` = 1)
OpenSSL::CipherError: bad decrypt
from (irb):4:in `final'
from (irb):4:in `aes'
from (irb):12:in `decrypt'
from (irb):42

>> decrypt(mkey, [User.find(1).password].pack("H*"))
User Load (2.8ms) SELECT * FROM `users` WHERE (`users`.`id` = 1)
OpenSSL::CipherError: wrong final block length
from (irb):4:in `final'
from (irb):4:in `aes'
from (irb):12:in `decrypt'
from (irb):43

Question is: did I miss something :) ?

I have a feeling I am *almost* there...

Thanks again!

-- Felipe
--
Posted via http://www.ruby-....

Felipe Coury

3/24/2009 6:35:00 AM

Just as a FYI, it works!!!

I forgot about the remains... Take a look at the final incarnation:

def mysql_key(key)
# The algorithm just creates a 16 byte buffer set to all zero,
final_key = "\0" * 16

# Number of string "blocks"
blocks, remain = key.length.divmod(16)

blocks.times do |i|
# For each block
key_block = key[i*16, 16]

# Runs bitwise XOR for each char on string
# and the same char on the block
16.times do |j|
final_key[j] ^= key_block[j]
end
end

if remain
remain.times do |i|
final_key[i] ^= key[(blocks * 16) + i]
end
end

final_key
end

And:

>> mkey = mysql_key(key)
=> "dp&!{\021?pK?G!8[r."
>> decrypt(mkey, User.find(1).password)
User Load (2.9ms) SELECT * FROM `users` WHERE (`users`.`id` = 1)
=> "password"

Just BEATIFUL!

Thanks a lot everyone!
--
Posted via http://www.ruby-....

Rob Biedenharn

3/24/2009 7:52:00 AM

On Mar 24, 2009, at 2:35 AM, Felipe Coury wrote:
> Just as a FYI, it works!!!
>
> I forgot about the remains... Take a look at the final incarnation:
>
> def mysql_key(key)
> # The algorithm just creates a 16 byte buffer set to all zero,
> final_key = "\0" * 16
>
> # Number of string "blocks"
> blocks, remain = key.length.divmod(16)
>
> blocks.times do |i|
> # For each block
> key_block = key[i*16, 16]
>
> # Runs bitwise XOR for each char on string
> # and the same char on the block
> 16.times do |j|
> final_key[j] ^= key_block[j]
> end
> end
>
> if remain
> remain.times do |i|
> final_key[i] ^= key[(blocks * 16) + i]
> end
> end
>
> final_key
> end
>
> And:
>
>>> mkey = mysql_key(key)
> => "dp&!{\021?pK?G!8[r."
>>> decrypt(mkey, User.find(1).password)
> User Load (2.9ms) SELECT * FROM `users` WHERE (`users`.`id` = 1)
> => "password"
>
> Just BEATIFUL!
>
> Thanks a lot everyone!
> --
> Posted via http://www.ruby-....
>

I'm glad you got it. Your key-building function doesn't need to be
quite so complex:

def mysql_key2(key)
final_key = "\0" * 16
key.length.times do |i|
final_key[i%16] ^= key[i]
end
final_key
end

Hardly needs any comments now ;-) Just a pointer to the MySQL doc
perhaps.

irb> mkey2 = mysql_key2(key)
=> "dp&!{\021?pK?G!8[r."
irb> mkey == mkey2
=> true

-Rob

Rob Biedenharn http://agileconsult...
Rob@AgileConsultingLLC.com

Felipe Coury

3/24/2009 1:33:00 PM

Rob Biedenharn wrote:
> I'm glad you got it. Your key-building function doesn't need to be
> quite so complex:

Rob, you nailed it. Thanks a lot!

Best regards,

-- Felipe
--
Posted via http://www.ruby-....

comp.lang.ruby

Mimic AES_ENCRYPT and AES_DECRYPT functions in Ruby

Felipe Coury

Felipe Coury

Rob Biedenharn

gcristelli

Felipe Coury

Felipe Coury

Felipe Coury

Rob Biedenharn

Felipe Coury

x Login to ForumsZone