Multiple channel interceptors

[Ayanda-D]

Proposed Changes

Hi folks 👋

The following changes seek to add support for multiple channel interceptors within the broker. Currently, when developing channel interceptor type plugins, development is restricted to single interceptor per AMQP operation, per installation. When use-cases grow and need for multiple interceptors for the same AMQP operation arises, this becomes a problem (and combining multiple use-cases within the same channel interceptor module makes things even more complex).

In these changes, we propose supporting multiple channel interceptors for the same AMQP operation, based on configurable priority. Interceptors with the same priority level for the same AMQP operation will not be allowed, as per current behaviour in channels. For example, if 3 interceptors need to intercept queue.declare, these would be configured as following in rabbitmq.conf:

channel_interceptor.priority.example_interceptor_1 = 1
channel_interceptor.priority.example_interceptor_2 = 2
channel_interceptor.priority.example_interceptor_3 = 3

These are executed in priority order (starting with the lowest) in the channel. If 2 interceptors for same AMQP operation have the same priority level, registry will fails as current RabbitMQ channel interceptor behaviour. The following wouldnt work if both intercept the same AMQP operation (also priorities default to 0 if not configured, same restriction applies if multiple interceptors are enabled without any config in rabbitmq.conf):

channel_interceptor.priority.example_interceptor_1  = 0
channel_interceptor.priority.example_interceptor_2 = 0

In addition, these changes also introduce a new rabbitmqctl list_channel_interceptors command to get visibility on active interceptors (as number of interceptors grow, we'll need this to keep track of whats active and execution order from the priority levels)

$> rabbitmqctl list_channel_interceptors
Listing channel interceptors ...
  name                    applies_to                    priority
  dummy_interceptor_1     basic.publish, queue.declare  0
  dummy_interceptor_2     basic.publish, queue.declare  1

NOTE: I've moved dummy_interceptor module from rabbit/test/ to rabbit/src/ to make available for the cli (elixir) tests - we could optionally keep it in the test DIR and do dynamic code loading over RPC of compiled beam/object code - but it feels like too much complexity for running simple cli tests. Placing in rabbit/src/ much simpler.

Please take a look, we are currently stuck and restricted on only having a single channel-interceptors, whilst uses-cases grow. This change would significantly help and improve extensibility of the broker for plugin developers (and also mean less need to propose changes to the broker core, which is always the goal 😇) - we'd appreciate this a lot. Thanks!

Types of Changes

What types of changes does your code introduce to this project?
Put an x in the boxes that apply

• Bug fix (non-breaking change which fixes issue #NNNN)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause an observable behavior change in existing systems)
• Documentation improvements (corrections, new content, etc)
• Cosmetic change (whitespace, formatting, etc)
• Build system and/or CI

Checklist

Put an x in the boxes that apply.
You can also fill these out after creating the PR.
This is simply a reminder of what we are going to look for before merging your code.

• Mandatory: I (or my employer/client) have have signed the CA (see https://github.com/rabbitmq/cla)
• I have read the CONTRIBUTING.md document
• I have added tests that prove my fix is effective or that my feature works
• All tests pass locally with my changes
• If relevant, I have added necessary documentation to https://github.com/rabbitmq/rabbitmq-website
• If relevant, I have added this change to the first version(s) in release-notes that I expect to introduce it

[michaelklishin]

@Ayanda-D thanks but for the umpteenth time, the core team suggests that you start a discussion first to gauge the interest in a certain change instead of showing up straight with a PR.

[michaelklishin]

This change would significantly help and improve extensibility of the broker for plugin developers

Sorry but I'm a little bit less enthusiastic about the impact of this change.

This is a lot of complexity for a change that 99% of RabbitMQ users will not need, and it was not previously discussed.

[Ayanda-D]

@michaelklishin sorry for not starting off with discussion - the change seemed straightforward and not too complex, as it's mainly altering the restriction rabbit_channel_interceptor:check_no_overlap/1 currently imposes on conflicting channel interceptors and making it priority based. The actual AMQP interception code rabbit_channel_interceptor:intercept_in/3 remains unchanged (this already supports looping through all registered interceptor in rabbit_registry). The rest of the changes are more config management, cli tools for maintenance, and tests. Its only initialization checks that we're relaxing a bit. (having this in the core also saves us the continual custom patching per release in future to enable these features - which can be a chicken & egg situation on whats merged upstream and what we backport on our end).