kinesis_producer can't refresh AssumeRole Credential.

[tatsu-yam]

The kinesis_producer plugin got into the state which can't refresh AssumeRole Credential in our environment. (#58)
Even though, we use fluent-plugin-kinesis 1.0.1 (or later version).

Our setting is below

• version
  • td-agent 2.3
  • fluent-plugin-kinesis 1.0.1 (or later version)
• td-agent.conf

<match hoge>
  @type kinesis_producer
  stream_name  xxx
  region ap-northeast-1
  <assume_role_credentials>
    role_arn    xxxxxxxxxx
    external_id xxxxxxxxxx
  </assume_role_credentials>
  time_format %s
  include_time_key true
  buffer_type file
  buffer_path /var/log/td-agent/buffer/kinesis
  buffer_chunk_limit 500k
  buffer_queue_limit 16384
  flush_interval 1s
  try_flush_interval 0.1s
  queued_chunk_flush_interval 0.01s
  num_threads 4
</match>

When an exeception occurs in the KinesisProducer::Daemon#updaet_credentials, the thread which calls #update_credentials seems to die. (

aws-fluent-plugin-kinesis/lib/kinesis_producer/daemon.rb

Line 159 in 75e8a34

add(make_set_credentials_message)

)

I tried deleting a policy from an IAM Role which the EC2 Instance assumes.
As a result, KinesisProducer::Daemon#update_credentials was not called every credentials_refresh_delay second.

When I added the code which rescues a Exception add(make_set_credentials_message) throws, even if its policy was changed, #update_credentials was called regularly.

But I don't know why that method is failed in our environment.

[riywo]

Hi @tatsu-yam ,

Thank you for reporting. I'll look into this.

[riywo]

I'm sorry for no response for a long time. We're are going to release v2.0.0 which supports record aggregation by a different way without KPL. So, this problem never happen and I'm closing.

Please try v2.0.0 first. If you need more help, feel free to add comment.