-
Bug
-
Resolution: Incomplete
-
P3
-
None
-
20.0.1, 21
-
x86
-
linux
ADDITIONAL SYSTEM INFORMATION :
Happens only on x86/linux with both JDK 20 (latest) and JDK 21-ea (build 30 but also on previous).
Cannot be reproduced on ARM/Mac with same JDK versions. Works with both JDK 17 and 19.
A DESCRIPTION OF THE PROBLEM :
In Trino project we are running vast amounts of tests that are highly concurrent. When we started testing forward-compatibility with JDK20 and JDK21 we've noticed spurious failures that rendered one specific string broken. Instead of expected: `another_query.0.0.0` we were getting assertions error because String value was `another_query��.0.0` (which failed to parse correctly because 4, dot separated parts were expected)
We've encoded those broken Strings .toCharArray() representations and base64-encoded them to see a difference:
Expected fullId is:
```
echo -n "another_query.0.0.0" | od -A n -t x1
61 6e 6f 74 68 65 72 5f 71 75 65 72 79 2e 30 2e
30 2e 30
```
But we've got:
```
echo YW5vdGhlcl9xdWVyeQAALjAuMA== | base64 -D | od -A n -t x1
61 6e 6f 74 68 65 72 5f 71 75 65 72 79 00 00 2e
30 2e 30
```
As you can see instead of a dot and zero number we've got 00 00 sequence.
This value is created in a class called TaskId, in the constructor: https://github.com/trinodb/trino/blob/e8bde4f94478d8107823020139f0e76fdef8fee6/core/trino-main/src/main/java/io/trino/execution/TaskId.java#L47
as the result of the concatenation of:
```
this.fullId = stageId + "." + partitionId + "." + attemptId;
```
or
```
this.fullId = stageId.toString() + '.' + partitionId + '.' + attemptId;
```
Where stageId is a StageId object with explicit toString() (returning String + "." + stageId), partitionId and attemptId are integers.
Basically fullId is a String + "." + integer + "." + integer + "." + integer.
We've also noticed that when the explicit concat is used:
```
this.fullId = stageId.toString().concat(".").concat(String.valueOf(partitionId)).concat(".").concat(String.valueOf(attemptId));`
```
or explicit join:
```
this.fullId = join(".", stageId.toString(), String.valueOf(partitionId), String.valueOf(attemptId));
```
The error no longer occurs. Also when we've disabled -XX:-CompactStrings we can no longer reproduce this error as well.
The error itself is not deterministic (whether it will occur) but when it does, the corrupted String is always the same.
I've also tried different values (single digit integers):
Expected:
```
echo another_query.1.2.3 | od -A n -t x1
61 6e 6f 74 68 65 72 5f 71 75 65 72 79 2e 31 2e
32 2e 33 0a
```
Actual:
```
echo YW5vdGhlcl9xdWVyeQAALjIuMw== | base64 -D | od -A n -t x1
61 6e 6f 74 68 65 72 5f 71 75 65 72 79 00 00 2e
32 2e 33
```
REGRESSION : Last worked in version 19
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
concatenated string being: another_query.0.0.0
echo -n "another_query.0.0.0" | od -A n -t x1
61 6e 6f 74 68 65 72 5f 71 75 65 72 79 2e 30 2e
30 2e 30
ACTUAL -
concatenated string being broken: another_query��.0.0
echo YW5vdGhlcl9xdWVyeQAALjAuMA== | base64 -D | od -A n -t x1
61 6e 6f 74 68 65 72 5f 71 75 65 72 79 00 00 2e
30 2e 30
---------- BEGIN SOURCE ----------
We cannot distill it down further
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
use explicit String.concat:
```
this.fullId = stageId.toString().concat(".").concat(String.valueOf(partitionId)).concat(".").concat(String.valueOf(attemptId));`
```
or explicit join:
```
this.fullId = join(".", stageId.toString(), String.valueOf(partitionId), String.valueOf(attemptId));
```
Using shorter test string also works:
```
TaskId anotherTaskId = new TaskId(new StageId("another", 1), 2, 3);
```
FREQUENCY : occasionally
Happens only on x86/linux with both JDK 20 (latest) and JDK 21-ea (build 30 but also on previous).
Cannot be reproduced on ARM/Mac with same JDK versions. Works with both JDK 17 and 19.
A DESCRIPTION OF THE PROBLEM :
In Trino project we are running vast amounts of tests that are highly concurrent. When we started testing forward-compatibility with JDK20 and JDK21 we've noticed spurious failures that rendered one specific string broken. Instead of expected: `another_query.0.0.0` we were getting assertions error because String value was `another_query��.0.0` (which failed to parse correctly because 4, dot separated parts were expected)
We've encoded those broken Strings .toCharArray() representations and base64-encoded them to see a difference:
Expected fullId is:
```
echo -n "another_query.0.0.0" | od -A n -t x1
61 6e 6f 74 68 65 72 5f 71 75 65 72 79 2e 30 2e
30 2e 30
```
But we've got:
```
echo YW5vdGhlcl9xdWVyeQAALjAuMA== | base64 -D | od -A n -t x1
61 6e 6f 74 68 65 72 5f 71 75 65 72 79 00 00 2e
30 2e 30
```
As you can see instead of a dot and zero number we've got 00 00 sequence.
This value is created in a class called TaskId, in the constructor: https://github.com/trinodb/trino/blob/e8bde4f94478d8107823020139f0e76fdef8fee6/core/trino-main/src/main/java/io/trino/execution/TaskId.java#L47
as the result of the concatenation of:
```
this.fullId = stageId + "." + partitionId + "." + attemptId;
```
or
```
this.fullId = stageId.toString() + '.' + partitionId + '.' + attemptId;
```
Where stageId is a StageId object with explicit toString() (returning String + "." + stageId), partitionId and attemptId are integers.
Basically fullId is a String + "." + integer + "." + integer + "." + integer.
We've also noticed that when the explicit concat is used:
```
this.fullId = stageId.toString().concat(".").concat(String.valueOf(partitionId)).concat(".").concat(String.valueOf(attemptId));`
```
or explicit join:
```
this.fullId = join(".", stageId.toString(), String.valueOf(partitionId), String.valueOf(attemptId));
```
The error no longer occurs. Also when we've disabled -XX:-CompactStrings we can no longer reproduce this error as well.
The error itself is not deterministic (whether it will occur) but when it does, the corrupted String is always the same.
I've also tried different values (single digit integers):
Expected:
```
echo another_query.1.2.3 | od -A n -t x1
61 6e 6f 74 68 65 72 5f 71 75 65 72 79 2e 31 2e
32 2e 33 0a
```
Actual:
```
echo YW5vdGhlcl9xdWVyeQAALjIuMw== | base64 -D | od -A n -t x1
61 6e 6f 74 68 65 72 5f 71 75 65 72 79 00 00 2e
32 2e 33
```
REGRESSION : Last worked in version 19
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
concatenated string being: another_query.0.0.0
echo -n "another_query.0.0.0" | od -A n -t x1
61 6e 6f 74 68 65 72 5f 71 75 65 72 79 2e 30 2e
30 2e 30
ACTUAL -
concatenated string being broken: another_query��.0.0
echo YW5vdGhlcl9xdWVyeQAALjAuMA== | base64 -D | od -A n -t x1
61 6e 6f 74 68 65 72 5f 71 75 65 72 79 00 00 2e
30 2e 30
---------- BEGIN SOURCE ----------
We cannot distill it down further
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
use explicit String.concat:
```
this.fullId = stageId.toString().concat(".").concat(String.valueOf(partitionId)).concat(".").concat(String.valueOf(attemptId));`
```
or explicit join:
```
this.fullId = join(".", stageId.toString(), String.valueOf(partitionId), String.valueOf(attemptId));
```
Using shorter test string also works:
```
TaskId anotherTaskId = new TaskId(new StageId("another", 1), 2, 3);
```
FREQUENCY : occasionally