Why was ActiveSupport::Messages::Metadata made so terribly encapsulated?

That module carries metadata of cookies, specifically it carries the lifetime of persistent cookies. But it is totally encapsulated; it is very difficult (and very ugly) to access that data from somewhere else. Why? This data is necessary for rekeying persistent cookies. And rekeying the persistent cookies is necessary when changing the master key base.

Currently, when switching the master key base, all persistent cookies are downgraded to session cookies during rekeying. And apparently for the only reason that required data is not available, because it is such encapsulated. Which makes these formerly persistent cookies rather pointless.

There are probably a few other issues when rekeying persistent cookies, but I like to perceive this from a security-first perspective, and then the imperative question is: why is the master key base not periodically changed, anyway?

I mean, we all change our personal passwords every three months (or at least we should), but the master key base, which may set loose infinitely more tangible data when compromized, is created once and then left outside alone for the lifetime of the application?!? Doesn’t make sense to me.

There is technically no problem for periodic rotation of the master key base: the necessary functions are implemented and they work. But then, if you do that in automated fashion on a regular base, say every three months, the loss of persistent cookies will be quite disgusting - it’s the first painpoint that appears to me. And this one is solveable, since the necessary data is actually present. There will probably be other painpoints, mostly because people may have attached other security stuff to the master key base which will create difficulties to rekey. But this in fact just means that people have done things which they shouldn’t have done in the first place from a proper maintainability perspective. I mean, look at e.g. DNSSEC: the first issue to consider is how you can periodically rekey the stuff.

Agreed, DNS is supposed to live for a very long time, whereas web applications are only expected to live from creation to bankrupcy (sorry, couldn’r resist) - but then, I am using this stuff for machine configuration databases, and there the expected lifetime equals machine lifetime, that is 10-15 years now, and I want that to be auto-rekeying.

I don’t consider hard cryptographic science, I would just like to ask: what do you do about a fired angry employee knowing the master key base?

This is an offer as well: I have working prototype code. So if anybody is interested in the matter, I would be happy to share - but the extremely ugly part of that is currently how to fetch the required data out of ActiveSupport::Messages::Metadata.

2 Likes

I would love to have a look at your code, Peter. – H

I am sorry, it took me a while to get back here…

Hasan, You’re very welcome; here we go:

Prereqs:

We need a place where to store the rotating keys. So set $vardir in config/environment.rb to a sensible value for this.

For development this would be Pathname.new(Rails.root).join('tmp') (And don’t forget to delete the old master-key - it is usually created with the application and lingering around somewhere, and might get preferred.)

For production this must be outside the application-root, because we want the keys to survive over redeploys. I am doing it in unix style, where every application has their own private directory somewhere below /var (and use a capistrano task to set that up). Protect that place from being read by other unix users.

Here I just export RAPP_VARDIR=/var/rapp globally on the machine, and every application gets their cozy place below:

$rapptag = ENV["RAPP_TAG"] ? ENV["RAPP_TAG"] :
  Rails.application.class.module_parent_name.downcase +
  "_" + (ENV["RAILS_ENV"].nil? ? "none" : ENV["RAILS_ENV"][0..3])
$vardir = ENV["RAPP_VARDIR"] ?
  Pathname.new(ENV["RAPP_VARDIR"]).join($rapptag) :
  Pathname.new(Rails.root).join('tmp')

Some scheduler to create+rotate the new key. I am using the Rufus::Scheduler gem:

config/initializers/scheduler.rb:

return if defined?(Rails::Console) || Rails.env.test? ||
          Rails.env.development? || File.split($0).last == 'rake'
Thread.new do
  sleep 20
  s = Rufus::Scheduler.singleton(:frequency => 30,
                                 :lockfile => "#{$vardir}/.scheduler.lck")
  if s.lock
    s.every KeyBaseRotate.interval, :first_in => KeyBaseRotate.delay do
      KeyBaseRotate.switch
    end
    Rails.logger.info "Scheduler started in this worker"
  else
    Rails.logger.info "Scheduler locked out in this worker"
  end
end

A means to invoke a phased restart of the workers. I am using puma, where it is only necessary to touch a rollover file and puma will do the heavy lifting. The proper plugin for puma may need to be enabled (or copied in) - and it needs to know where the rollover file is, Here it is in $vardir - but puma does not see this variable, that is why I have RAPP_TAG defined in the environment also:

./lib/puma/plugin/touch_rollover.rb

Puma::Plugin.create do
  def start(launcher)
    path = File.join(ENV["RAPP_VARDIR"], ENV["RAPP_TAG"], "rollover.txt")
    orig = nil
    # If we can't write to the path, then just don't bother with this plugin
    begin
      File.write(path, "") unless File.exist?(path)
      orig = File.stat(path).mtime
    rescue SystemCallError
      return
    end
    in_background do
      while true
        sleep 2
        begin
          mtime = File.stat(path).mtime
        rescue SystemCallError
          # If the file has disappeared, assume that means don't restart
        else
          if mtime > orig
            launcher.phased_restart
            orig = mtime
            sleep 600
          end
        end
      end
    end
  end
end

Start the whole stuff from ./config/initializers/something.rb:

KeyBaseRotate.boot({ expiry: 3600 * 24 * 30, interval: '12h', delay: '30m' })

And this is the code:

./lib/key_base_rotate.rb

# With inspiration from: https://blog.kmkonline.co.id/
#               rotating-ruby-on-rails-secret-key-base-9a4dcdf0d817

class KeyBaseRotate
  FirstFile     = 'secret'
  OldFile       = 'secret.old'
  NewFile       = 'secret.new'
  RestartFile   = 'rollover.txt'
  Expiry        = 3600 * 24 * 30  # Cookie-Validity min. 1/4, max 3/4 of this
  SchedInterval = '25h'
  SchedDelay    = '60m'
  Iterations    = 1000 # hardcoded in railties-5.2.3/lib/rails/application.rb,
      # see also https://github.com/rails/rails/pull/6952#issuecomment-7661220

  def self.boot options={}
    @@firstfile = options[:firstfile] || pathname(FirstFile)
    @@oldfile = options[:oldfile] || pathname(OldFile)
    @@newfile = options[:newfile] || pathname(NewFile)
    @@expiry = options[:expiry] || Expiry
    @@interval = options[:interval] || SchedInterval
    @@delay = options[:delay] || SchedDelay
    
    if File.exist?(@@firstfile)
      token = File.read(@@firstfile).chomp
    else
      token = keymake @@firstfile
    end
    Rails.application.config.secret_key_base = token

    # Can we just co-use the secrets-hash? (Currently unused, but
    # we need a place where other things within the app (e.g. devise)
    # could fetch our keys and do their own rotation ...
    Rails.application.secrets[:secret_key_base_rotate] = Array.new
    [@@oldfile, @@newfile].each do |f|
      if File.exist?(f)
        token = File.read(f).chomp
        Rails.application.secrets[:secret_key_base_rotate] << token

        salt = Rails.application.config.action_dispatch.
                 authenticated_encrypted_cookie_salt
        encrypted_cookie_cipher = Rails.application.config.action_dispatch.
                 encrypted_cookie_cipher || 'aes-256-gcm'
        key_generator = ActiveSupport::KeyGenerator.new(
          token, iterations: Iterations)
        key_len = ActiveSupport::MessageEncryptor.key_len(
          encrypted_cookie_cipher)
        encr_secret = key_generator.generate_key(salt, key_len)
        
        salt = Rails.application.config.action_dispatch.signed_cookie_salt
        sign_secret = key_generator.generate_key(salt)
        
        Rails.application.config.action_dispatch.
              cookies_rotations.tap do |cookies|
          cookies.rotate :encrypted, encr_secret
          cookies.rotate :signed, sign_secret
        end
      end
    end
  end

  # @@expiry is the time a secret will exist over-all from create to delete.
  # The client does actively carry it between 1/4 and 1/2 (theoretically
  # max 3/4) of that time.
  # There are always two secrets present; after 1/2 the time we delete the
  # old one and create a new one, and after 1/4 the time we make the primary
  # one old and the new one primary. We must do it this way; if we would do
  # it in only one step, then there were a race condition, because more than
  # two servers may exist and may do a rolling restart one after another:
  # the client might connect an already restarted server and get the new
  # secret, then connect a not yet restarted server that does not yet know
  # anything about the new secret and would reject the client.
  # In this way the client may ping-pong between old and current secret
  # while the restart proceeds, but that doesn't hurt.
  def self.switch
    r = false
    if File.mtime(@@firstfile) + @@expiry / 2 < Time.now
      unless File.exist?(@@newfile)
        keymake @@newfile
        r = true
      end
      if File.exist?(@@oldfile)
        File.delete(@@oldfile)
        r = true
      end
    end
    if File.exist?(@@newfile) &&
        File.mtime(@@newfile) + @@expiry / 4 < Time.now
      File.rename @@firstfile, @@oldfile
      File.rename @@newfile, @@firstfile
      restart true
    else
      restart if r
    end
  end

  def self.interval
    @@interval
  end
  def self.delay
    @@delay
  end
  
  private
  # Generate a new token and store it in token_file.
  def self.keymake file
    token = SecureRandom.hex(64)
    File.write(file, token)
    token
  end
  
  def self.restart switch=false
    Rails.logger.info "Key#{switch ? "switch" : "change"}: need reload"
    if File.exist?(pathname RestartFile)
      File.write pathname(RestartFile), ""
    end
  end
  
  def self.pathname filename
    $vardir.join(filename)
  end
end


### and here follows the ugly part ###

# Cookies werden nach einem rotate, wenn sie erstmals zugegriffen
# werden, automatisch auf die neue Encryption geändert und wieder an
# den Client gesendet. Dies funktioniert aber nur korrekt fĂĽr
# Session Cookies, persistente Cookies dagegen verwandeln sich dabei in
# Session Cookies. Grund ist, dass die expiry-Information vor dem
# zurĂĽcksenden neu gesetzt werden mĂĽsste. aber die hat nur der Client.
#
# Die einzige mögliche Lösung wäre daher, die expiry-Zeit nochmals
# im Content des Cookie zu speichern, um darauf zurückgreifen zu können.
# Das aber ist gar nicht nötig, denn das passiert bereits: in
# ActiveSupport::Messages::Metadata wird die expiry-Zeit mit in die
# Cookie-Message gespeichert (und dann signiert). Es ist lediglich
# sauschwer, darauf zuzugreifen, denn diese expiry-Zeit ist ein
# Sicherheitsfeature, das nur die decryption veralteter Cookies
# unterbinden soll - sie ist in der decryption/verification eingekapselt
# und nicht abfragbar.

# Erster Teil: die expiry mittels der Metadata-Klasse extrahierbar machen
# und im Encryptor und Verifier jeweils methoden zur Extraktion anlegen
module ActiveSupport
  module Messages
    class Metadata
      class << self
        def extract_expiry message
          extract_metadata(message).instance_variable_get(:'@expires_at')
        end
      end
    end

    # die prepends im Rotator für die zusätzl. encryptor/verifier methods
    module Rotator
      module Encryptor
        def fetch_expiry(*args, **)
          super
        rescue MessageEncryptor::InvalidMessage,
               MessageVerifier::InvalidSignature
          run_rotations(nil) { |encryptor| encryptor.fetch_expiry(*args) }
        end
      end
      module Verifier
        def fetch_expiry(*args, **)
          super || run_rotations(nil) {
            |verifier| verifier.fetch_expiry(*args)
          }
        end
      end 
    end
  end
  
  # das zurĂĽckgegebene expiry ist bei session-cookies nil, und muss daher
  # in ein Array, weil der rotator ein nil als gescheiterten Versuch ansieht!
  class MessageEncryptor
    def fetch_expiry(data, **)
      _extract_expiry verifier.verify(data)
    end

    private

    # encrypted cookie: Hier mĂĽssen wir das ganze zeug von _decrypt kopieren
    # (Das tun wir erst wenn wir die message schon haben, d.h. sie ist valide)
    def _extract_expiry(encrypted_message)
      cipher = new_cipher
      encrypted_data, iv, auth_tag = encrypted_message.split("--").
                                       map { |v| ::Base64.strict_decode64(v) }
      cipher.decrypt
      cipher.key = @secret
      cipher.iv  = iv
      if aead_mode?
        cipher.auth_tag = auth_tag
        cipher.auth_data = ""
      end
      decrypted_data = cipher.update(encrypted_data)
      decrypted_data << cipher.final
      [ Messages::Metadata.extract_expiry(decrypted_data) ]
    rescue OpenSSLCipherError, TypeError, ArgumentError
      raise InvalidMessage
    end
  end

  class MessageVerifier
    def fetch_expiry(signed_message, **)
      if valid_message?(signed_message)
        begin
          data = signed_message.split("--")[0]
          [ Messages::Metadata.extract_expiry(decode(data)) ]
        rescue ArgumentError
          nil
        end
      end
    end
  end
end

# Zweiter Teil: die expiry im Fall einer rotation ergänzen, indem
# wir die Method, die den cookie neu schreibt, ĂĽberladen
module ActionDispatch
  class Cookies
    module SerializedCookieJars
      
      protected

      def deserialize(name)
        rotate = false
        value  = yield -> { rotate = true }
        if value
          data = @parent_jar[name]
          expiry = @encryptor.fetch_expiry(data)[0] unless @encryptor.nil?
          expiry = @verifier.fetch_expiry(data)[0] unless @verifier.nil?
          case
          when needs_migration?(value)
            Marshal.load(value).tap do |v|
              self[name] = { value: v, expires: expiry }
            end
          when rotate
            serializer.load(value).tap do |v|
              self[name] = { value: v, expires: expiry }
              Rails.logger.info "Cookie rotated: " +
                                "\"#{name}\", expiry \"#{expiry.to_s}\""
            end
          else
            serializer.load(value)
          end
        end
      end
    end
  end
end