We use spring cloud stream to produce the event to kafka and it was all working fine until we started to introduce redis cache.
I think for some reason the combination of Redis and Kafka on the serialization fails.
At this stage, we seem to have the redis cache working fine but emitting an event to kafka broken and getting the below Redis exception while producing the event onto Kafka.
Error while producing SampleEvent: org.springframework.messaging.MessageDeliveryException: failed to send Message to channel 'sampleOutput'; nested exception is org.springframework.data.redis.serializer.SerializationException: Could not write JSON: Not an array: {"type":"record","name":"SampleEvent","namespace":"com.sample.redisandkafka","fields":[{"name":"field","type":"string","default":"null"}]} (through reference chain: org.springframework.cloud.schema.registry.ParsedSchema["schema"]->org.apache.avro.Schema$RecordSchema["elementType"]); nested exception is com.fasterxml.jackson.databind.JsonMappingException: Not an array: {"type":"record","name":"SampleEvent","namespace":"com.sample.redisandkafka","fields":[{"name":"field","type":"string","default":"null"}]} (through reference chain: org.springframework.cloud.schema.registry.ParsedSchema["schema"]->org.apache.avro.Schema$RecordSchema["elementType"]), failedMessage=GenericMessage [payload={"field": "value"}, headers={id=16108acf-6ca1-f1b2-2ed3-44eb857daa6b, contentType=application/*+avro, timestamp=1712741537549}]Spring boot: 2.4.5
Spring cloud stream: 2020.0.2
And this github repo contains the reproducible example.