Quantcast
Channel: Active questions tagged mongodb-atlas - Stack Overflow
Viewing all articles
Browse latest Browse all 271

How to setup and configure mongodb kafka soruce connector using aws msk connect?

$
0
0

I have setup a MSK cluster using public subnets. I also have turned on public access for this MSK cluster. The cluster is up and running. I am trying to setup MongoDB Kafka Source Connector using MSK connect. I have selected IAM authentication and created IAM Role and corresponding policy for authentication. Below is my mongodb kafka source connector configuration

connector.class=com.mongodb.kafka.connect.MongoSourceConnectortasks.max=1connection.uri=mongodb+srv://XXXXX:XXXXXXX@XXXXXXX.qricz.mongodb.net/user?authSource=admin&replicaSet=atlas-bftscy-shard-0&readPreference=primarydatabase=usertopic.prefix=mongopoll.max.batch.size=1000poll.await.time.ms=5000pipeline=[]batch.size=0publish.full.document.only=truemongo.errors.tolerance=allmongo.errors.deadletterqueue.topic.name=data-ingestion-dlq-betaoffset.partition.name=mongodb-source-connector-beta.1startup.mode=copy_existingoutput.format.value=jsonoutput.format.key=jsonkey.converter.schemas.enable=falsevalue.converter.schemas.enable=falsekey.converter=org.apache.kafka.connect.storage.StringConvertervalue.converter=org.apache.kafka.connect.storage.StringConverterchange.stream.full.document=updateLookup

But when I am creating the connector it's getting failed due to the following error

[Worker-0aef29e0af9181e21] [2024-01-31 20:47:37,864] INFO Exception in monitor thread while connecting to server XXXXXXX.qricz.mongodb.net:27017 (org.mongodb.driver.cluster:76)[Worker-0aef29e0af9181e21] com.mongodb.MongoSocketOpenException: Exception opening socket[Worker-0aef29e0af9181e21]  at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:70)[Worker-0aef29e0af9181e21]  at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:180)[Worker-0aef29e0af9181e21]  at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:193)[Worker-0aef29e0af9181e21]  at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:157)[Worker-0aef29e0af9181e21]  at java.base/java.lang.Thread.run(Thread.java:829)[Worker-0aef29e0af9181e21] Caused by: java.net.SocketTimeoutException: connect timed out[Worker-0aef29e0af9181e21]  at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)[Worker-0aef29e0af9181e21]  at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)[Worker-0aef29e0af9181e21]  at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)[Worker-0aef29e0af9181e21]  at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)[Worker-0aef29e0af9181e21]  at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)[Worker-0aef29e0af9181e21]  at java.base/java.net.Socket.connect(Socket.java:609)[Worker-0aef29e0af9181e21]  at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:305)[Worker-0aef29e0af9181e21]  at com.mongodb.internal.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:107)[Worker-0aef29e0af9181e21]  at com.mongodb.internal.connection.SocketStream.initializeSocket(SocketStream.java:79)[Worker-0aef29e0af9181e21]  at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:65)[Worker-0aef29e0af9181e21]  ... 4 more

What am I missing? Please help


Viewing all articles
Browse latest Browse all 271

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>