Administration of Confluent Apache Kafka Training Course

Module 1: Introduction to the Architecture and Configuration of the Confluent Apache Kafka Cluster

• The role of Kafka in modern data pipelines
• Distinguishing between Apache Kafka and Confluent Kafka
• Essential components: producers, consumers, brokers, topics, and partitions
• Kafka cluster deployment models and strategies for scaling

Module 2: Configuring Zookeeper Quorums

• An introduction to Zookeeper
• The function of Zookeeper within a Kafka cluster
• Determining the appropriate Zookeeper Quorum size
• Setting up Zookeeper configurations
• Establishing SSH connections on servers
• Practical exercise: Configuring Zookeeper (both as a team and as a service)
• Utilising the Zookeeper Command Line Interface (CLI)
• Practical exercise: Setting up the Zookeeper Quorum
• The Zookeeper internal file system
• Key performance factors influencing Zookeeper
• Demonstration of management tools for Zookeeper and Zoonavigator

Module 3: Configuring the Kafka Cluster

• Fundamental Kafka concepts
• Core Kafka configuration parameters
• Practical exercise: Configuring a Kafka broker
• Practical exercise: Running Kafka commands
• Practical exercise: Configuring a multi-broker Kafka cluster
• Practical exercise: Testing Kafka cluster integrity
• Verifying connectivity to the Kafka cluster
• Configuring Advertised.listeners: a critical setting
• Topic configuration best practices
• Settings for downloading and ingesting messages into topics
• Practical exercise: Demonstrating Kafka resilience
• Kafka performance: I/O considerations
• Kafka performance: Network (RED) metrics
• Kafka performance: RAM utilisation
• Kafka performance: CPU usage
• Kafka performance: Operating System (OS) optimisations
• Kafka performance: Other relevant factors
• Practical exercise: Modifying Kafka broker configurations

Module 4: Advanced Kafka Configuration

• Configuring the Landoop Kafka topic user interface, Confluent REST Proxy, and Confluent Schema Registry
• Handling message sending and receiving (via CLI, Java, and the Spring framework)
• Monitoring metrics and leveraging tools (such as Confluent Control Center and Elasticsearch)
• Managing log files and offsets
• Ensuring high availability and disaster recovery
• Achieving high availability through replication
• Optimising producer and consumer performance
• Implementing robust disaster recovery strategies
• Controlling failover and managing data recovery
• Configuring connectors
• Implementing Kafka Connect
• Exploring Kafka security features

Course Summary and Next Steps