-
Bug
-
Resolution: Incomplete
-
P4
-
None
-
17
-
x86_64
-
linux
ADDITIONAL SYSTEM INFORMATION :
Java SDK 17
Java Application Deployed on Azure Kubernetes Services. OS LINUX
Backend data Base ADX (Kusto) , OS : Windows server 2019. Enabled to Support TLS1.3
A DESCRIPTION OF THE PROBLEM :
Recently we upgraded Java form 11 to Java 17 .
At the same time ADX (Kusto) was also upgraded by Microsoft to windows server2019 and TLS1.3.
to connect to ADX we are using Kusto SDK
<groupId>com.microsoft.azure.kusto</groupId>
<artifactId>kusto-data</artifactId>
<version>5.1.0</version>
Since then we started getting TLS SSL handshake error.
To fix this issue we forced to TLS1.2 and it worked.
The SSL handshake error is popping randomly we are not able to reproduce it every time.
But 10% of the request failure is happening due to this error.
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Enable TLS1.3 and try to pull large dataset form Backend server (ADX)
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Must get List of records form ADX
ACTUAL -
ServiceException: IOException in post request:Incorrect inner plaintext: no content type
com.XXXXXXXXXXXX.reports.consumer.exception.QueryExecutionException: Exception while executing Query: com.microsoft.azure.kusto.data.exceptions.DataServiceException: IOException in post request:Incorrect inner plaintext: no content type
at com.XXXXXXXXXXXX.reports.consumer.service.impl.ShipmentQueryGeneratorClient.executeQuery(ShipmentQueryGeneratorClient.java:71)
at com.XXXXXXXXXXXX.reports.consumer.processor.ReportDetailProcessor.process(ReportDetailProcessor.java:126)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.EventConsumerServiceImpl.processMessage(EventConsumerServiceImpl.java:182)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.EventConsumerServiceImpl.lambda$new$1(EventConsumerServiceImpl.java:150)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.EventConsumerServiceImpl.lambda$new$2(EventConsumerServiceImpl.java:150)
at com.azure.messaging.eventhubs.EventProcessorClientBuilder$1.processEventBatch(EventProcessorClientBuilder.java:888)
at com.azure.messaging.eventhubs.PartitionPumpManager.processEvents(PartitionPumpManager.java:303)
at com.azure.messaging.eventhubs.PartitionPumpManager.lambda$startPartitionPump$2(PartitionPumpManager.java:214)
at reactor.core.publisher.LambdaSubscriber.onNext(LambdaSubscriber.java:160)
at io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
at io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:100)
at io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
at io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.runAsync(FluxPublishOn.java:446)
at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.run(FluxPublishOn.java:533)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.util.concurrent.ExecutionException: com.microsoft.azure.kusto.data.exceptions.DataServiceException: IOException in post request:Incorrect inner plaintext: no content type
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.ShipmentQueryGeneratorClientHelper.executeKQLQuery(ShipmentQueryGeneratorClientHelper.java:95)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.ShipmentQueryGeneratorClient.executeQuery(ShipmentQueryGeneratorClient.java:57)
... 22 more
Caused by: com.microsoft.azure.kusto.data.exceptions.DataServiceException: IOException in post request:Incorrect inner plaintext: no content type
at com.microsoft.azure.kusto.data.http.HttpPostUtils.post(HttpPostUtils.java:77)
at com.microsoft.azure.kusto.data.ClientImpl.lambda$executeToJsonResult$1(ClientImpl.java:224)
at com.microsoft.azure.kusto.data.instrumentation.MonitoredActivity.invoke(MonitoredActivity.java:33)
at com.microsoft.azure.kusto.data.instrumentation.MonitoredActivity.invoke(MonitoredActivity.java:26)
at com.microsoft.azure.kusto.data.ClientImpl.executeToJsonResult(ClientImpl.java:223)
at com.microsoft.azure.kusto.data.ClientImpl.executeImpl(ClientImpl.java:173)
at com.microsoft.azure.kusto.data.ClientImpl.lambda$execute$0(ClientImpl.java:122)
at com.microsoft.azure.kusto.data.instrumentation.MonitoredActivity.invoke(MonitoredActivity.java:33)
at com.microsoft.azure.kusto.data.ClientImpl.execute(ClientImpl.java:121)
at com.microsoft.azure.kusto.data.ClientImpl.execute(ClientImpl.java:116)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.ShipmentQueryGeneratorClientHelper.getKustoResultSet(ShipmentQueryGeneratorClientHelper.java:55)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.ShipmentQueryGeneratorClientHelper$ShipmentTaskCallable.call(ShipmentQueryGeneratorClientHelper.java:157)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.ShipmentQueryGeneratorClientHelper$ShipmentTaskCallable.call(ShipmentQueryGeneratorClientHelper.java:143)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
... 3 more
CUSTOMER SUBMITTED WORKAROUND :
Downgrade to TLS1.2
Java SDK 17
Java Application Deployed on Azure Kubernetes Services. OS LINUX
Backend data Base ADX (Kusto) , OS : Windows server 2019. Enabled to Support TLS1.3
A DESCRIPTION OF THE PROBLEM :
Recently we upgraded Java form 11 to Java 17 .
At the same time ADX (Kusto) was also upgraded by Microsoft to windows server2019 and TLS1.3.
to connect to ADX we are using Kusto SDK
<groupId>com.microsoft.azure.kusto</groupId>
<artifactId>kusto-data</artifactId>
<version>5.1.0</version>
Since then we started getting TLS SSL handshake error.
To fix this issue we forced to TLS1.2 and it worked.
The SSL handshake error is popping randomly we are not able to reproduce it every time.
But 10% of the request failure is happening due to this error.
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Enable TLS1.3 and try to pull large dataset form Backend server (ADX)
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Must get List of records form ADX
ACTUAL -
ServiceException: IOException in post request:Incorrect inner plaintext: no content type
com.XXXXXXXXXXXX.reports.consumer.exception.QueryExecutionException: Exception while executing Query: com.microsoft.azure.kusto.data.exceptions.DataServiceException: IOException in post request:Incorrect inner plaintext: no content type
at com.XXXXXXXXXXXX.reports.consumer.service.impl.ShipmentQueryGeneratorClient.executeQuery(ShipmentQueryGeneratorClient.java:71)
at com.XXXXXXXXXXXX.reports.consumer.processor.ReportDetailProcessor.process(ReportDetailProcessor.java:126)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.EventConsumerServiceImpl.processMessage(EventConsumerServiceImpl.java:182)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.EventConsumerServiceImpl.lambda$new$1(EventConsumerServiceImpl.java:150)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.EventConsumerServiceImpl.lambda$new$2(EventConsumerServiceImpl.java:150)
at com.azure.messaging.eventhubs.EventProcessorClientBuilder$1.processEventBatch(EventProcessorClientBuilder.java:888)
at com.azure.messaging.eventhubs.PartitionPumpManager.processEvents(PartitionPumpManager.java:303)
at com.azure.messaging.eventhubs.PartitionPumpManager.lambda$startPartitionPump$2(PartitionPumpManager.java:214)
at reactor.core.publisher.LambdaSubscriber.onNext(LambdaSubscriber.java:160)
at io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.lambda$onNext$1(TracingSubscriber.java:64)
at io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:100)
at io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.withActiveSpan(TracingSubscriber.java:91)
at io.opentelemetry.javaagent.shaded.instrumentation.reactor.v3_1.TracingSubscriber.onNext(TracingSubscriber.java:64)
at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.runAsync(FluxPublishOn.java:446)
at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.run(FluxPublishOn.java:533)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.util.concurrent.ExecutionException: com.microsoft.azure.kusto.data.exceptions.DataServiceException: IOException in post request:Incorrect inner plaintext: no content type
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.ShipmentQueryGeneratorClientHelper.executeKQLQuery(ShipmentQueryGeneratorClientHelper.java:95)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.ShipmentQueryGeneratorClient.executeQuery(ShipmentQueryGeneratorClient.java:57)
... 22 more
Caused by: com.microsoft.azure.kusto.data.exceptions.DataServiceException: IOException in post request:Incorrect inner plaintext: no content type
at com.microsoft.azure.kusto.data.http.HttpPostUtils.post(HttpPostUtils.java:77)
at com.microsoft.azure.kusto.data.ClientImpl.lambda$executeToJsonResult$1(ClientImpl.java:224)
at com.microsoft.azure.kusto.data.instrumentation.MonitoredActivity.invoke(MonitoredActivity.java:33)
at com.microsoft.azure.kusto.data.instrumentation.MonitoredActivity.invoke(MonitoredActivity.java:26)
at com.microsoft.azure.kusto.data.ClientImpl.executeToJsonResult(ClientImpl.java:223)
at com.microsoft.azure.kusto.data.ClientImpl.executeImpl(ClientImpl.java:173)
at com.microsoft.azure.kusto.data.ClientImpl.lambda$execute$0(ClientImpl.java:122)
at com.microsoft.azure.kusto.data.instrumentation.MonitoredActivity.invoke(MonitoredActivity.java:33)
at com.microsoft.azure.kusto.data.ClientImpl.execute(ClientImpl.java:121)
at com.microsoft.azure.kusto.data.ClientImpl.execute(ClientImpl.java:116)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.ShipmentQueryGeneratorClientHelper.getKustoResultSet(ShipmentQueryGeneratorClientHelper.java:55)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.ShipmentQueryGeneratorClientHelper$ShipmentTaskCallable.call(ShipmentQueryGeneratorClientHelper.java:157)
at com.XXXXXXXXXXXX.reports.consumer.service.impl.ShipmentQueryGeneratorClientHelper$ShipmentTaskCallable.call(ShipmentQueryGeneratorClientHelper.java:143)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
... 3 more
CUSTOMER SUBMITTED WORKAROUND :
Downgrade to TLS1.2