doc: 阅读redis xinfo文档
This commit is contained in:
@@ -109,6 +109,11 @@
|
|||||||
- [XCLAIM](#xclaim)
|
- [XCLAIM](#xclaim)
|
||||||
- [JUSTID option](#justid-option)
|
- [JUSTID option](#justid-option)
|
||||||
- [Automatic claiming](#automatic-claiming)
|
- [Automatic claiming](#automatic-claiming)
|
||||||
|
- [Claiming and the delivery counter](#claiming-and-the-delivery-counter)
|
||||||
|
- [working with multiple consumer groups](#working-with-multiple-consumer-groups)
|
||||||
|
- [Enhanced deletion control in Redis 8.2](#enhanced-deletion-control-in-redis-82)
|
||||||
|
- [Stream Observability](#stream-observability)
|
||||||
|
- [Difference with kafka partitions](#difference-with-kafka-partitions)
|
||||||
|
|
||||||
|
|
||||||
# redis
|
# redis
|
||||||
@@ -1952,4 +1957,111 @@ XAUTOCLAIM <key> <group> <consumer> <min-idle-time> <start> [COUNT count] [JUSTI
|
|||||||
|
|
||||||
当`XAUTOCLAIM`返回“0-0”作为cursor时,代表其到达了`end of the consumer group pending entries list`。其并不代表没有新的idle pending messages,可以重新从begining of the stream来调用`XAUTOCLAIM`。
|
当`XAUTOCLAIM`返回“0-0”作为cursor时,代表其到达了`end of the consumer group pending entries list`。其并不代表没有新的idle pending messages,可以重新从begining of the stream来调用`XAUTOCLAIM`。
|
||||||
|
|
||||||
|
##### Claiming and the delivery counter
|
||||||
|
通过`XPENDING`命令输出的counter代表该message的被传递次数,该counter在两种场景下会增加:
|
||||||
|
- when a message is successfully claimed via `XCLAIM`
|
||||||
|
- when a `XREADGROUP` call is used in order to access the history of pending messages
|
||||||
|
|
||||||
|
当存在failure时,message可能会被传递多次,但是message最终会被处理并ack。但是,在处理特定的消息时,可能会在处理逻辑中抛出异常,在该类场景下consumer会持续的在处理该消息时抛出异常。故而,可以通过delivery counter来探知那些不可处理的message。`一旦delivery counter到达给定的值时,可以将该消息发送给另一个stream,并且向系统的管理员发送notification。`这就是redis stream实现`dead letter`的基础。
|
||||||
|
|
||||||
|
##### working with multiple consumer groups
|
||||||
|
redis stream可以关联多个consumer groups,每个entries都会被传递给每个consumer group。在consumer group内,每个consumer instance处理一部分entries。
|
||||||
|
|
||||||
|
当consumer对message进行处理时,其会使用`XACK`命令来对message进行确认,`并且从consumer group的Pending Entries List(PEL)中移除该entry reference`。但是,`被ack的message仍然保存在stream中`,且consumer group A中对message的ack并不会影响consumer group B的PEL,group A在对message进行ack后,message仍然位于group B的PEL中,直到group B中的consumer对message进行ack,此时message才从group B的PEL中被移除。
|
||||||
|
|
||||||
|
通常来说,如果想要从stream中删除entries,必须要等到所有的consumer groups都对entries进行了ack,应用需要实现复杂的逻辑。
|
||||||
|
|
||||||
|
###### Enhanced deletion control in Redis 8.2
|
||||||
|
从redis 8.2开始,一些命令为`entries在多个consumer groups间的处理`提供了增强控制:
|
||||||
|
- `XADD`支持KEEPREF, DELREF, ACKED模式
|
||||||
|
- `XTRIM`同样支持KEEPREF, DELREF, ACKED选项
|
||||||
|
|
||||||
|
如下选项控制consumer group references是如何被处理的:
|
||||||
|
- `KEEPREF`(默认): Preserves existing references to entries in all consumer groups' PELs
|
||||||
|
- `DELREF`: Removes all references to entries from consumer groups' PELs, effectively cleaning up all traces of the messages
|
||||||
|
- `ACKED`: Only processes entries that have been acked by all consumer groups
|
||||||
|
|
||||||
|
`ACKED` mode对于`coordinating deletion across multiple consumer groups`的复杂逻辑十分有用,确认entires在所有consumer groups在完成对其的处理后才移除。
|
||||||
|
|
||||||
|
##### Stream Observability
|
||||||
|
缺乏可观测性的消息系统将十分难以使用,一个透明的消息系统需要令如下信息可观测:
|
||||||
|
- who is consuming messages
|
||||||
|
- what messages are pending
|
||||||
|
- the set of consumer groups active in a given stream
|
||||||
|
|
||||||
|
在前面章节中,已经介绍了`XPENDINGS`命令,通过其可以观测处于`处理中`状态的消息,并且能够获取消息的idle time和number of deliveries。
|
||||||
|
|
||||||
|
`XINFO`命令和sub-commands一起使用,可以用于获取stream和consumer group相关的信息。
|
||||||
|
|
||||||
|
`XINFO`命令的使用示例如下:
|
||||||
|
```redis-cli
|
||||||
|
> XINFO STREAM race:italy
|
||||||
|
1) "length"
|
||||||
|
2) (integer) 5
|
||||||
|
3) "radix-tree-keys"
|
||||||
|
4) (integer) 1
|
||||||
|
5) "radix-tree-nodes"
|
||||||
|
6) (integer) 2
|
||||||
|
7) "last-generated-id"
|
||||||
|
8) "1692632678249-0"
|
||||||
|
9) "groups"
|
||||||
|
10) (integer) 1
|
||||||
|
11) "first-entry"
|
||||||
|
12) 1) "1692632639151-0"
|
||||||
|
2) 1) "rider"
|
||||||
|
2) "Castilla"
|
||||||
|
13) "last-entry"
|
||||||
|
14) 1) "1692632678249-0"
|
||||||
|
2) 1) "rider"
|
||||||
|
2) "Norem"
|
||||||
|
```
|
||||||
|
上述示例中,通过`XINFO`命令获取了stream本身的信息,输出展示了stream内部的编码方式,并且记录了stream中的第一条消息和最后一条消息。
|
||||||
|
|
||||||
|
如果想要获取和stream相关的consumer group消息,参照如下示例:
|
||||||
|
```redis-cli
|
||||||
|
> XINFO GROUPS race:italy
|
||||||
|
1) 1) "name"
|
||||||
|
2) "italy_riders"
|
||||||
|
3) "consumers"
|
||||||
|
4) (integer) 3
|
||||||
|
5) "pending"
|
||||||
|
6) (integer) 2
|
||||||
|
7) "last-delivered-id"
|
||||||
|
8) "1692632662819-0"
|
||||||
|
```
|
||||||
|
如果想要查看consumer group中注册的consumer实例,可以通过`XINFO CONSUMERS`命令来进行查看:
|
||||||
|
```redis-cli
|
||||||
|
> XINFO CONSUMERS race:italy italy_riders
|
||||||
|
1) 1) "name"
|
||||||
|
2) "Alice"
|
||||||
|
3) "pending"
|
||||||
|
4) (integer) 1
|
||||||
|
5) "idle"
|
||||||
|
6) (integer) 177546
|
||||||
|
2) 1) "name"
|
||||||
|
2) "Bob"
|
||||||
|
3) "pending"
|
||||||
|
4) (integer) 0
|
||||||
|
5) "idle"
|
||||||
|
6) (integer) 424686
|
||||||
|
3) 1) "name"
|
||||||
|
2) "Lora"
|
||||||
|
3) "pending"
|
||||||
|
4) (integer) 1
|
||||||
|
5) "idle"
|
||||||
|
6) (integer) 72241
|
||||||
|
```
|
||||||
|
##### Difference with kafka partitions
|
||||||
|
Redis stream中的consumer group可能在某些方面类似于kafka中基于分区的consumer groups,但是仍然存在较大差别。
|
||||||
|
|
||||||
|
在redis strema中,`partition`这一概念是逻辑的,实际上stream所有的messages都存在相同的key中。故而,redis stream中consumer instance并不从实际的partition中读取信息,也不涉及partition在consumer间的分配。
|
||||||
|
|
||||||
|
> 例如,如果consumer C3在某个时刻fail permanently,redis在所有新消息到达时都会传递给C1和C2,将好像redis stream只存在2个逻辑分区。
|
||||||
|
|
||||||
|
类似的,如果某个consumer处理消息比其他consumers都快,那么该consumer将在单位时间内按比例收到更多的消息。Redis会追踪所有尚未被ack的消息,并且记录哪条消息被哪个consumer接收,`the ID of the first message never delivered to any consumer`也会被redis记录。
|
||||||
|
|
||||||
|
在redis Stream中:
|
||||||
|
- 如果redis stream的数量和consumer的数量都为1,那么消息将是按照顺序被处理的
|
||||||
|
- 如果stream的数量为1,consumer的数量为n,那么可以将负载均衡给n个consumers,但是,在这种情况下消息的消费可能是无序的
|
||||||
|
- 当使用n个stream和n个consumers时,一个consumer只用于处理所有streams中的一部分,可以将`1 stream->1 consumer`拓展到`n stream->n consuimer`
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user