Setup lograge, logstash-logger

rubyonrails-talk
1

Can anybody provide some links to how to set up correctly Lograge, logstash-logger with a Rails API app ? I followed their READMEs but still nothing sent to Kibana :(.

Here is how these gems are declared in Gemfile:

group :development do

gem ‘lograge’, ‘~> 0.10.0’ gem ‘logstash-event’, ‘~> 1.2’ gem ‘logstash-logger’, ‘~> 0.26.1’ end

``

I added the following lines in config/environments/development.rb:

config.lograge.enabled = true config.lograge.base_controller_class = ‘ActionController::API’ config.lograge.formatter = Lograge::Formatters::Logstash.new #config.lograge.logger = LogStashLogger.new(uri: ENV[‘logstash_uri’], verify_hostname: false) config.lograge.logger = LogStashLogger.new(host: ENV[‘logstash_host’], port: ENV[‘logstash_port’], verify_hostname: false) config.lograge.custom_options = lambda do |event| { name: “decastore-development” } end

``

As you see, I tried 2 different ways to initialize LogStashLogger, but still without success.

What’s wrong with that ?

I’m using Rails 5.2.0 (API only), ruby 2.5.0.

Thank you

2

Even moving the above declaration to config/application.rb:

module MyAppApi

class Application < Rails::Application

… config.api_only = true

config.lograge.enabled = true
config.lograge.base_controller_class = 'ActionController::API'
config.lograge.formatter = Lograge::Formatters::Logstash.new
config.lograge.logger = LogStashLogger.new(type: :tcp, host: ENV['logstash_host'], port: ENV['logstash_port'])
config.lograge.custom_options = lambda do |event|
  { name: "myapp-development" }
end

``

didn’t send anything to Kibana.

3

Never used either of these but had a little play.

My impression is that LogStashLogger defaults to a UDP connection, so you might try with e.g.

LogStashLogger.new(type: :tcp, host: ENV['logstash_host'], ...)

HTH,

4

Can anybody provide some links to how to set up correctly Lograge,

logstash-logger with a Rails API app ? I followed their READMEs but still

nothing sent to Kibana :(.

config.lograge.logger = LogStashLogger.new(host: ENV[‘logstash_host’],

port: ENV[‘logstash_port’], verify_hostname: false)

Never used either of these but had a little play.

My impression is that LogStashLogger defaults to a UDP connection,

so you might try with e.g.

LogStashLogger.new(type: :tcp, host: ENV[‘logstash_host’], …)

Yeas, I knew that. I tried both tcp and udp, none of them worked. The only one that works is when I specify the type as file:

config.lograge.logger = LogStashLogger.new(type: :file, path: ‘log/development.log’, sync: true)

``

This way the logs shall be written in Logstash format in development.log.

The question I have not found the answer to is what host and port values should correspond to ? Are they the values of Logstash server ?

5

Yeas, I knew that. I tried both tcp and udp, none of them worked. The only one that works is when I specify the type as file:

config.lograge.logger = LogStashLogger.new(type: :file, path: 'log/development.log', sync: true)

Taking "lograge" out of the picture for the moment, using this config:

  config.logger = LogStashLogger.new(type: :udp, host: '127.0.0.1', port: 5228, verify_hostname: false)

This way the logs shall be written in Logstash format in development.log. The question I have not found the answer to is what host and port values should correspond to ? Are they the values of Logstash server ?

I do this below in a Rails console:

Loading development environment (Rails 5.2.1) 2.5.1 (main):0 > Rails.logger.info("test message from a rails app") => true 2.5.1 (main):0 > Rails.logger.flush() => true 2.5.1 (main):0 >

with this test endpoint (Erlang, but that's not important)

Eshell V9.2 (abort with ^G) 1> {ok, Logstash} = gen_udp:open(5228, [binary, {active,false}]). {ok,#Port<0.490>} 2> gen_udp:recv(Logstash, 0). {ok,{{127,0,0,1},      49369,      <<"{\"message\":\"test message from a rails app\",\"@timestamp\":\"2018-08-16T13:25:48.113-07:00\",\"@versio"...>>}} 3>

Note that nothing showed up on receiving end until I flushed the logger on the Rails side, so that might have something to do with your testing not apparently doing anything. But for sure, the `host` and `port` values are for the remote logstash endpoint.

HTH!

6

Yeas, I knew that. I tried both tcp and udp, none of them worked. The only

one that works is when I specify the type as file:

config.lograge.logger = LogStashLogger.new(type: :file, path:

‘log/development.log’, sync: true)

Taking “lograge” out of the picture for the moment, using this config:

config.logger = LogStashLogger.new(type: :udp, host: ‘127.0.0.1’,

port: 5228, verify_hostname: false)

This way the logs shall be written in Logstash format in development.log.

The question I have not found the answer to is what host and port values

should correspond to ? Are they the values of Logstash server ?

I do this below in a Rails console:

Loading development environment (Rails 5.2.1)

2.5.1 (main):0 > Rails.logger.info(“test message from a rails app”)

=> true

2.5.1 (main):0 > Rails.logger.flush()

=> true

2.5.1 (main):0 >

with this test endpoint (Erlang, but that’s not important)

Eshell V9.2 (abort with ^G)

1> {ok, Logstash} = gen_udp:open(5228, [binary, {active,false}]).

{ok,#Port<0.490>}

2> gen_udp:recv(Logstash, 0).

{ok,{{127,0,0,1},

 49369,

 <<"{\"message\":\"test message from a rails

app","@timestamp":"2018-08-16T13:25:48.113-07:00","@versio"…>>}}

3>

Note that nothing showed up on receiving end until I flushed the logger

on the Rails side, so that might have something to do with your testing

not apparently doing anything. But for sure, the host and port values

are for the remote logstash endpoint.

HTH!

Thank you for your time. Yeah, I also tested Lograge/Logstash from inside the rails console in the deployed docker container:

rails c

logger = LogStashLogger.new(type: :tcp, host: ‘logstash_host’, port: logstash_port, verify_hostname: false)

logger.info ‘logstash-draft => test port XXX’

and I could see the produced log message in Kibana dashboard.

The problem is now on Logstash server configuration side :slight_smile: (devops will take a look at that).