Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

unexpected error error_class=LoadError error="cannot load such file -- aws-sdk-core/plugins/client_metrics_plugin.rb" #161

Closed
kvitali opened this issue Sep 8, 2018 · 5 comments · Fixed by #171
Closed

unexpected error error_class=LoadError error="cannot load such file -- aws-sdk-core/plugins/client_metrics_plugin.rb" #161

kvitali opened this issue Sep 8, 2018 · 5 comments · Fixed by #171
Milestone
v3.0

Comments

@kvitali
Copy link

kvitali commented Sep 8, 2018 •
edited
Loading

Hey,

We are using td-agent with kinesis plugin to send events to AWS Kinesis.
Starting from yesterday we start to get below error:

2018-09-07 10:28:03 +0000 [error]: #0 unexpected error error_class=LoadError error="cannot load such file -- aws-sdk-core/plugins/client_metrics_plugin.rb" 2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in require'
2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in require' 2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/aws-sdk-kinesis-1.5.0/lib/aws-sdk-kinesis/client.rb:22:in <top (required)>'
2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/aws-sdk-kinesis-1.5.0/lib/aws-sdk-kinesis.rb:13:in require_relative' 2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/aws-sdk-kinesis-1.5.0/lib/aws-sdk-kinesis.rb:13:in <top (required)>'
2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in require' 2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in require'
2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-kinesis-2.1.1/lib/fluent/plugin/kinesis_helper/client.rb:92:in client_class' 2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-kinesis-2.1.1/lib/fluent/plugin/kinesis_helper/client.rb:77:in client'
2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-kinesis-2.1.1/lib/fluent/plugin/out_kinesis_streams.rb:47:in block in write' 2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-kinesis-2.1.1/lib/fluent/plugin/kinesis_helper/api.rb:90:in batch_request_with_retry'
2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-kinesis-2.1.1/lib/fluent/plugin/kinesis.rb:149:in block in write_records_batch' 2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-kinesis-2.1.1/lib/fluent/plugin/kinesis_helper/api.rb:85:in split_to_batches'
2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-kinesis-2.1.1/lib/fluent/plugin/kinesis.rb:147:in write_records_batch' 2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-kinesis-2.1.1/lib/fluent/plugin/out_kinesis_streams.rb:43:in write'
2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/compat/output.rb:131:in write' 2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/plugin/output.rb:1099:in try_flush'
2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/plugin/output.rb:1378:in flush_thread_run' 2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/plugin/output.rb:440:in block (2 levels) in start'
2018-09-07 10:28:03 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/plugin_helper/thread.rb:78:in block in thread_create' 2018-09-07 10:28:03 +0000 [error]: #0 cannot load such file -- aws-sdk-core/plugins/client_metrics_plugin.rb path="aws-sdk-core/plugins/client_metrics_plugin.rb" error_class=LoadError error="cannot load such file -- aws-sdk-core/plugins/client_metrics_plugin.rb"

The versions are below:
2018-09-07 10:28:05 +0000 [info]: starting fluentd-1.2.2 pid=1171 ruby="2.4.4"
2018-09-07 10:28:05 +0000 [info]: spawn command to main: cmdline=["/opt/td-agent/embedded/bin/ruby", "-Eascii-8bit:ascii-8bit", "/opt/td-agent/embedded/bin/fluentd", "--log", "/var/log/td-a
gent/td-agent.log", "--daemon", "/var/run/td-agent/td-agent.pid", "--under-supervisor"]
2018-09-07 10:28:06 +0000 [info]: gem 'fluent-plugin-concat' version '2.3.0'
2018-09-07 10:28:06 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '2.10.3'
2018-09-07 10:28:06 +0000 [info]: gem 'fluent-plugin-kafka' version '0.7.3'
2018-09-07 10:28:06 +0000 [info]: gem 'fluent-plugin-kinesis' version '2.1.1'
2018-09-07 10:28:06 +0000 [info]: gem 'fluent-plugin-loggly' version '0.0.9'
2018-09-07 10:28:06 +0000 [info]: gem 'fluent-plugin-record-modifier' version '1.1.0'
2018-09-07 10:28:06 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.1.0'
2018-09-07 10:28:06 +0000 [info]: gem 'fluent-plugin-s3' version '1.1.3'
2018-09-07 10:28:06 +0000 [info]: gem 'fluent-plugin-td' version '1.0.0'
2018-09-07 10:28:06 +0000 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.4'
2018-09-07 10:28:06 +0000 [info]: gem 'fluent-plugin-webhdfs' version '1.2.3'
2018-09-07 10:28:06 +0000 [info]: gem 'fluentd' version '1.2.2'

Thank you,
Vitali
@kvitali
Copy link
Author

kvitali commented Sep 8, 2018

I also think that the actual version running on my node is v3 and not 2.1.1 as printed.

@riywo
Copy link
Contributor

riywo commented Sep 9, 2018

Could you provide the version list of whole gems? We noticed some combinations are failed and want to confirm you are in one of them.

@riywo
Copy link
Contributor

riywo commented Sep 9, 2018

Also, could you try this alpha version? We think this version will solve such dependency issue. https://rubygems.org/gems/fluent-plugin-kinesis/versions/3.0.0.alpha.2.0

@simukappu
Copy link
Contributor

Adding reference to aws/aws-sdk-ruby#1872

simukappu added a commit to simukappu/aws-fluent-plugin-kinesis that referenced this issue Jan 7, 2019
@simukappu
Fix dependency problem on AWS SDK - awslabs#161
4aa2300
simukappu added a commit to simukappu/aws-fluent-plugin-kinesis that referenced this issue Jan 19, 2019
@simukappu
Fix dependency problem on AWS SDK - awslabs#161
5e6957b
simukappu added a commit to simukappu/aws-fluent-plugin-kinesis that referenced this issue Jan 19, 2019
@simukappu
Fix dependency problem on AWS SDK - awslabs#161
70a3ea3
@riywo riywo added this to the v3.0 milestone Jan 20, 2019
@riywo riywo mentioned this issue Jan 20, 2019
Merged
simukappu added a commit that referenced this issue Jan 20, 2019
@simukappu
Fix dependency problem on AWS SDK - #161
e02072e
@riywo
Copy link
Contributor

riywo commented Jan 21, 2019

We've released v3.0.0.rc.1.0 https://rubygems.org/gems/fluent-plugin-kinesis/versions/3.0.0.rc.1.0

If there is no negative feedback, we'll release v3.0.0 soon.

@simukappu simukappu mentioned this issue Mar 18, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v3.0
Development

Successfully merging a pull request may close this issue.

Update to v3 RC awslabs/aws-fluent-plugin-kinesis
3 participants
@riywo @simukappu @kvitali