For example a consumer can reset to an older offset to reprocess. This offset is controlled by the consumer: normally a consumer will advance its offset linearly as it reads messages, but in fact the position is controlled by the consumer and it can consume messages in any order it likes. In fact the only metadata retained on a per-consumer basis is the position of the consumer in the log, called the "offset".
Kafka's performance is effectively constant with respect to data size so retaining lots of data is not a problem.
#Manual for norton partition magic 8.0 quick start guide free#
For example if the log retention is set to two days, then for the two days after a message is published it is available for consumption, after which it will be discarded to free up space. The Kafka cluster retains all published messages-whether or not they have been consumed-for a configurable period of time. The messages in the partitions are each assigned a sequential id number called the offset that uniquely identifies each message within the partition. So, at a high level, producers send messages over the network to the Kafka cluster which in turn serves them up to consumers like this:Įach partition is an ordered, immutable sequence of messages that is continually appended to-a commit log.
You're viewing documentation for an older version of Kafka - check out our current documentation here.ĭocumentation Kafka 0.8.0 Documentation for the 0.7 release is here.
0 kommentar(er)
