feat: add cosmos RPC failure / success metrics and alerts

[kamiyaa]

Description

add cosmos RPC failure / success metrics and alerts)

RPC

For RPC clients, I added a wrapper function around each call.
This works out better because the HttpClient from tendermint's sdk is not very extensible, and most of the methods require us to manually provide the method metric.

GRPC

For GRPC, I added a wrapper around Tonic's Channel, inspired by code from https://docs.rs/tonic_prometheus_layer/0.1.11/tonic_prometheus_layer/

Drive-by changes

Related issues

#5252

Backward compatibility

Testing

• add check to e2e to make sure metrics are being exported

[changeset-bot]

⚠️ No Changeset found

Latest commit: c049c4e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.53%. Comparing base (6dcc30e) to head (c049c4e).
Report is 12 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #5523   +/-   ##
=======================================
  Coverage   77.53%   77.53%           
=======================================
  Files         103      103           
  Lines        2110     2110           
  Branches      190      190           
=======================================
  Hits         1636     1636           
  Misses        453      453           
  Partials       21       21           

Components	Coverage Δ
core	87.80% <ø> (ø)
hooks	79.39% <ø> (ø)
isms	83.68% <ø> (ø)
token	91.27% <ø> (ø)
middlewares	79.80% <ø> (ø)