This project is a PoC (Proof of Concept). The main idea is insert data into Redis from a Kafka topic, using the Landoop Redis Sink Connect.
This project uses Docker, and create a Docker image configured with kafka-connect-redis-1.0.0.jar. You can check the Dockerfile here.
This PoC uses docker-compose to set up all necessary environment to proof this concept.
You can just build and run everything running docker-compose up command.
What does the docker-compose will start?
- Redis
- Zookeeper
- Kafka
- Schema-registry
- Kafka Connect
To run this PoC, you can just run the script ./poc.sh. What does the poc.sh do?
- Build and up all docker services from docker-compose.
- Create topic
redis-sink. - Create Redis sink connect
connector_redis_sink. - Produce a test message to a specific Kafka topic.
You can them access Redis, using redis-cli for example, to check the keys generate from the topic:
redis-cli->get test:gabriel