Jay's Cookbook
Menu
  • Tags
  • Categories
  • Projects
Computer Science
OS
Network
Data Structure
Algorithm
Language
Code Architecture
Python
Javascript
Typescript
Java
Backend
Backend Theory
TypeORM
Node.js
NestJS
FastAPI
Frontend
HTML/CSS
React
Next.js
Data Engineering
DE Theory
MySQL
MongoDB
Elastic
Redis
Kafka
Spark
Airflow
AI
Basic
Pytorch
NLP
Computer Vision
Data Analytics
Statistics
Pandas
Matplotlib
DevOps
Git
Docker
Kubernetes
AWS
Kafka Series [Part10]: MySQL Connector
data_engineering
kafka

Kafka Series [Part10]: MySQL Connector

Jay Kim
Jay Kim 11 Apr 2022
Kafka Series [Part8]: Client Won’t Connect to Apache Kafka Cluster in Docker/AWS/My Laptop.[번역] Kafka Series [Part9]: Kafka Connector

Table of Contents


./kafka/bin/zookeeper-server-start.sh [-daemon] ./kafka/config/zookeeper.properties
./kafka/bin/kafka-server-start.sh ./kafka/config/server.properties
./kafka/bin/connect-distributed.sh ./kafka/config/connect-distributed.properties

# 등록 가능한 커넥터 플러그인 목록
curl -X GET http://localhost:8083/connector-plugins
# connect-distributed.properties

# 이렇게 추가하면 FileStreamSinkConnector, FileStreamSourceConnector 커넥터 클래스가 추가됨 (아래와 같은 경로가 없음에도 불구하고.. 그래서 이해가 안됨)
plugin.path=/usr/local/share/java,/usr/local/share/kafka/plugins,/opt/connectors
# 현재 등록된 커넥터 목록
curl -X GET http://localhost:8083/connectors
# FileStreamSinkConnector 커넥터 등록 (토픽 메세지를 파일에 쓴다)
curl -X POST http://localhost:8083/connectors -H 'Content-Type: application/json' -d '{ "name": "file-sink-test", "config": { "topics": "test", "connector.class": "org.apache.kafka.connect.file.FileStreamSinkConnector", "tasks.max": 1, "file": "./connect-test.txt"}}'
# 커넥터 상태 조회
curl -X GET http://localhost:8083/connectors/file-sink-test/status
요청 메서드 경로 설명
GET /connector-plugins 사용 가능한 커넥터 플러그인 목록
GET / 커넥트 정보
GET /connectors 커넥터 목록
POST /connectors 커넥터 등록
GET /connectors/<커넥터 이름=""> 커넥터 정보 확인
GET /connectors/<커넥터 이름="">/config 커넥터 설정값 확인
PUT /connectors/<커넥터 이름="">/config 커넥터 설정값 변경
GET /connectors/<커넥터 이름="">/status 커넥터 상태 확인
POST /connectors/<커넥터 이름="">/restart 커넥터 재실행
DELETE /connectors/<커넥터 이름=""> 커넥터 삭제
Kafka Series [Part8]: Client Won’t Connect to Apache Kafka Cluster in Docker/AWS/My Laptop.[번역] Kafka Series [Part9]: Kafka Connector

You may also like

See all kafka
11 May 2022 Kafka Series [Part11]: Python API for Kafka
data_engineering
kafka

Kafka Series [Part11]: Python API for Kafka

11 Apr 2022 Kafka Series [Part9]: Kafka Connector
data_engineering
kafka

Kafka Series [Part9]: Kafka Connector

31 Jan 2022 Kafka Series [Part8]: Client Won’t Connect to Apache Kafka Cluster in Docker/AWS/My Laptop.[번역]
data_engineering
kafka

Kafka Series [Part8]: Client Won’t Connect to Apache Kafka Cluster in Docker/AWS/My Laptop.[번역]

클라이언트와 카프카간의 메세지를 주고받기 위해서는 두 가지의 연결이 반드시 선행되어야 한다.

Jay Kim

Jay Kim

Web development, data engineering for human for the Earth. I share posts, free resources and inspiration.

Rest
Lifestyle
Hobby
Hobby
Hobby
Hobby
2025 © Jay's Cookbook. Crafted & Designed by Artem Sheludko.