In modern binary protocols like Apache Kafka, Compact Arrays are used to save bandwidth. Instead of using a fixed 4-byte integer to store the length of a list, Kafka uses a Varint (Variable-length Integer) and an offset of 1.
In standard programming, an array is like a row of identical lockers. Even if you only put a tiny pebble in a locker, the locker stays the same huge size. This wastes space.
A Compact Array is "shrink-wrapped" data. It uses two main tricks to save space:
Varints: The length of the array isn't a fixed 4-byte block; it's a "stretchy" number that only uses the bytes it needs.