-
Bug
-
Resolution: Fixed
-
P1
-
None
-
None
-
None
There have been various recent reports of disappearing messages that turned out to have been miscategorized as spam by gmail. I continue to see openjdk messages in my spam folder from non-spammers at Google and elsewhere.
A dump of information follows from some local investigation at Google:
spam analyst:
The issue is the list configuration, they modify the email body which breaks the DKIM signature, which results in the message no longer being authenticated.
Authentication-Results: mx.google.com;
dkim=fail header.i=@google.com header.s=20161025 header.b=v3HuYX4A;
spf=neutral (google.com: 141.146.126.229 is neither permitted nor denied by best guess record for domain of compiler-dev-bounces@openjdk.java.net) smtp.mailfrom=compiler-dev-bounces@openjdk.java.net;
dmarc=fail (p=REJECT sp=REJECT dis=QUARANTINE) header.from=google.com
For the domain "google.com", we've set a DMARC reject policy. This means that we tell all email services (including ourselves) that for email to say it comes from google.com, it has to either come directly or not be modified. This email purports to be from ronshapiro@google.com, but it doesn't come directly from Google, and the content has been modified. So we can't confirm that it truly came from Google without modification, and as such we end up spam foldering the message. We don't actually reject the message here even though we're supposed to according to the DMARC policy, because we can tell that it was likely forwarded, so we take the gentler step of accepting it and spam foldering it.
This is an unfortunate case where DMARC doesn't work very well.
How can they fix this? They can pursue any of:
a) Not modifying the message so the DKIM signature won't break
b) Using the ARC standard, which Googlers have developed for exactly this type of problem, though for some reason even the ARC header is broken in this case. Perhaps multiple hops, or a misconfiguration.
c) Having the mailing list re-write the From header when they modify messages
me:
Is this the exact problem discussed here:
https://wiki.list.org/DEV/DMARC
Is there a google-blessed way to continue to run a mailman mailing list? E.g. "you should upgrade to Mailman 3"
I see the header
X-Mailman-Version: 2.1.17
but https://wiki.list.org/DEV/DMARC refers to features added in Mailman 2.1.18
so maybe "upgrade to 2.1.18"?
spam analyst:
I'm not aware of any officially Google-sanctioned way to run a mailing list; plus the issues in this case aren't Google-specific: the mailing list will have the same issues with any email service provider that enforces DMARC. The closest thing we have are forwarding best practices, which can be found here: https://support.google.com/mail/answer/175365?hl=en
As to Mailman, I'm not familiar with it, but the link you shared is a pretty good one with regards to dealing with DMARC from mailing lists. The suggestions in the Mailman 3 section would fix the issue.
mail headers from message referenced above:
Delivered-To: martinrb@google.com
Received: by 2002:a25:c791:0:0:0:0:0 with SMTP id w139-v6csp35860ybe;
Wed, 3 Oct 2018 04:21:59 -0700 (PDT)
X-Google-Smtp-Source: ACcGV616UmG2NafO4Z7S5WeZ03NTOQNkF6dLCxPh5XIHUH1UPuqKz3Y4/S7D3sI2cmzwsKslRksP
X-Received: by 2002:a63:9d01:: with SMTP id i1-v6mr922759pgd.98.1538565718960;
Wed, 03 Oct 2018 04:21:58 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1538565718; cv=none;
d=google.com; s=arc-20160816;
b=0XMlm35jtMStgbCmGPnnELDdBOizEhLR4zyRojBGsd9g7TeFjYDIpIrA99fxGNT5h8
lh4tBe4/qCw0vzkRvxpQ6AraNGBOEH5dGjCJeKGETr8r877oUljYQWq1ZthMY8V2A/Nf
kZY3a1sPZHPG0OtJJAgCIslI1mBKtCFuM0A786EjZZA3eiytQbwbI/dHbPNBhfZ4a72U
fdR7iQGdTUN7gXFIxCBqSsDWqy6KiyF5uZnkPfnNbmqfAz/zest+DZDsW2VW6l/h2INW
eTIvNgDeAS+jCUlqmsL0gmhU6ZFzq40OjoGB5FEOzbbKNSXQIy59gtec83pItuwdi7Wp
qVTw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
h=sender:errors-to:list-subscribe:list-help:list-post:list-archive
:list-unsubscribe:list-id:precedence:to:subject:message-id:date:from
:mime-version:dkim-signature:delivered-to;
bh=X01jHIYgqHE9R0MHnbgV5r3ICMISwUt97OJfQofiTig=;
b=yGl5Q8aLC09dwQv/DiPTNAjNxuJy7wIb40iKrdTguXdUmBIORqoBs0YFFSANWuiuqf
0lZ+Kk28Rtsb/GmilQJFaTmhICRYaoqKvNoudRaN6vYhF22zUVlux84T1L9LXxIIW5Sx
rhiD500qpR0JvlvvaJL6ZQehpfXZCeYxw4fCvtfp8pM5qOii63nD4XjFgoYEUQQNsKXO
lu8zRbU8BJDvfTn+7jqSEnATLHawDBlT1HLUymvBt9S3DbiJGlUHCPTJAsgFogEVCpys
iB7AtpIim8+jqdrlUIJ7aOi04ppDzp4oJWYGXY5cz5M12AnZrpxWMtvGyTf1BTrX9iFD
dkPw==
ARC-Authentication-Results: i=1; mx.google.com;
dkim=fail header.i=@google.com header.s=20161025 header.b=v3HuYX4A;
spf=neutral (google.com: 141.146.126.229 is neither permitted nor denied by best guess record for domain of compiler-dev-bounces@openjdk.java.net) smtp.mailfrom=compiler-dev-bounces@openjdk.java.net;
dmarc=fail (p=REJECT sp=REJECT dis=QUARANTINE) header.from=google.com
Return-Path: <compiler-dev-bounces@openjdk.java.net>
Received: from acsinet41.oracle.com (acsinet41.oracle.com. [141.146.126.229])
by mx.google.com with ESMTPS id f9-v6si1158678pgh.325.2018.10.03.04.21.27
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Wed, 03 Oct 2018 04:21:58 -0700 (PDT)
Received-SPF: neutral (google.com: 141.146.126.229 is neither permitted nor denied by best guess record for domain of compiler-dev-bounces@openjdk.java.net) client-ip=141.146.126.229;
Authentication-Results: mx.google.com;
dkim=fail header.i=@google.com header.s=20161025 header.b=v3HuYX4A;
spf=neutral (google.com: 141.146.126.229 is neither permitted nor denied by best guess record for domain of compiler-dev-bounces@openjdk.java.net) smtp.mailfrom=compiler-dev-bounces@openjdk.java.net;
dmarc=fail (p=REJECT sp=REJECT dis=QUARANTINE) header.from=google.com
Received: from aojmv0009 (unknown [137.254.59.6]) by acsinet41.oracle.com with smtp
id 5bfe_0653_90869e51_ac08_468f_8c6f_4a1ff1145675; Wed, 03 Oct 2018 11:21:09 +0000
Received: from aojmv0009.oracle.com (localhost [127.0.0.1]) by aojmv0009 (Postfix) with ESMTP id 8D4A329001C; Wed,
3 Oct 2018 11:21:05 +0000 (UTC)
X-Original-To: compiler-dev@openjdk.java.net
Delivered-To: compiler-dev@openjdk.java.net
Received: from ucsinet40.oracle.com (ucsinet40.oracle.com [156.151.31.68]) by aojmv0009 (Postfix) with ESMTP id 7773F290012 for <compiler-dev@openjdk.java.net>; Wed,
3 Oct 2018 11:21:03 +0000 (UTC)
Received: from mail-qt1-f178.google.com (unknown [209.85.160.178]) by ucsinet40.oracle.com with smtp (TLS: TLSv1/SSLv3,128bits,ECDHE-RSA-AES128-GCM-SHA256) id 048d_18d4_cb52d178_2344_4963_b050_5071bc8fb074; Wed, 03 Oct 2018 11:21:02 +0000
Received: by mail-qt1-f178.google.com with SMTP id q40-v6so5434753qte.0 for <compiler-dev@openjdk.java.net>; Wed, 03 Oct 2018 04:21:02 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=X01jHIYgqHE9R0MHnbgV5r3ICMISwUt97OJfQofiTig=; b=v3HuYX4APEny94kHJMQq0O2nDoSTs8vQXHbhrW6+rVFOtqn7tZfL5GD7zAh6/rRIVm TJcGl+WEzVn5sTmA15jopECRSfNbQ2XgCZkH7dvgAn6pIZTvqE5+392BP4KBCqmmR5/i qLWoGjq4bRxl7jiGBRzV/nKa2TKrMrvfBgsTaq0tKJmv0EEAhxw6rzee+IWQaWK18sdH 9ydNUsxss4Okg2b7HjxpDioYhzNC+Z9TRPLf1/CWbovvi6IUWLZMGevxOh/LORf2IGMB DM1uieHf4TX/9zu4/0AQw8JR3GnHnKc8b340aU0vgW+xNoyBvK4qCUZe0dIRG3GvO5ex kanA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=X01jHIYgqHE9R0MHnbgV5r3ICMISwUt97OJfQofiTig=; b=p/SnBL3ydnY+th6aE0K4ecMlunnAvHKXJnJfFry1g4CL4BcJNsl5NSxb76Zjc7Q/Ye vrW7PO2pTl88zHT0IPQKO6vvqRUpPzJKtpDX/Aw6DGAZV2B3BO7vK8dKSRaVDot71NZg qIHMeUQ00M6tGPF994tbwg946iGf4ho4XUik/lBgrHAlhbBO7ckmUz3lRwychq+ATL1Q +tXdg3NddtnlKQHkVXwmfp9lfCAPZYfjxSxQGBbeUt8KjATVFJJ0Ot1D4tEqv4hXnNPS tt5Y5SjcvOHeAHfw6NG2Tcsl+F5cJORchFCVwV11B0PyB784DzRESyaeT9wj5vLDJ1a1 6h1A==
X-Gm-Message-State: ABuFfogUHZlw56wb54U+PkNPikDgZjc6uv9AdHozvwPQrNDz8cqMip3A uUyO0d0I9eLSyrmxjtbzDaJE67P/IPyckgIheIVxvn3RWro2nQ==
X-Received: by 2002:a05:6214:1091:: with SMTP id o17mr724665qvr.97.1538565661022;
Wed, 03 Oct 2018 04:21:01 -0700 (PDT)
MIME-Version: 1.0
From: Ron Shapiro <ronshapiro@google.com>
Date: Wed, 3 Oct 2018 14:20:48 +0300
Message-ID: <CACA6h9to7szM0dSpOjUhLWFWTUEHCvRNFhS_x+TgeesS6qYC_w@mail.gmail.com>
Subject: Name.contentEquals() performance
To: compiler-dev <compiler-dev@openjdk.java.net>
Content-Type: multipart/alternative; boundary="000000000000b0d7b30577513b6e"
X-BeenThere: compiler-dev@openjdk.java.net
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: "Technical discussion about the development of the Java compiler \(javac\)" <compiler-dev.openjdk.java.net>
List-Unsubscribe: <http://mail.openjdk.java.net/mailman/options/compiler-dev>,
<mailto:compiler-dev-request@openjdk.java.net?subject=unsubscribe>
List-Archive: <http://mail.openjdk.java.net/pipermail/compiler-dev/>
List-Post: <mailto:compiler-dev@openjdk.java.net>
List-Help: <mailto:compiler-dev-request@openjdk.java.net?subject=help>
List-Subscribe: <http://mail.openjdk.java.net/mailman/listinfo/compiler-dev>,
<mailto:compiler-dev-request@openjdk.java.net?subject=subscribe>
Errors-To: compiler-dev-bounces@openjdk.java.net
Sender: compiler-dev <compiler-dev-bounces@openjdk.java.net>
A dump of information follows from some local investigation at Google:
spam analyst:
The issue is the list configuration, they modify the email body which breaks the DKIM signature, which results in the message no longer being authenticated.
Authentication-Results: mx.google.com;
dkim=fail header.i=@google.com header.s=20161025 header.b=v3HuYX4A;
spf=neutral (google.com: 141.146.126.229 is neither permitted nor denied by best guess record for domain of compiler-dev-bounces@openjdk.java.net) smtp.mailfrom=compiler-dev-bounces@openjdk.java.net;
dmarc=fail (p=REJECT sp=REJECT dis=QUARANTINE) header.from=google.com
For the domain "google.com", we've set a DMARC reject policy. This means that we tell all email services (including ourselves) that for email to say it comes from google.com, it has to either come directly or not be modified. This email purports to be from ronshapiro@google.com, but it doesn't come directly from Google, and the content has been modified. So we can't confirm that it truly came from Google without modification, and as such we end up spam foldering the message. We don't actually reject the message here even though we're supposed to according to the DMARC policy, because we can tell that it was likely forwarded, so we take the gentler step of accepting it and spam foldering it.
This is an unfortunate case where DMARC doesn't work very well.
How can they fix this? They can pursue any of:
a) Not modifying the message so the DKIM signature won't break
b) Using the ARC standard, which Googlers have developed for exactly this type of problem, though for some reason even the ARC header is broken in this case. Perhaps multiple hops, or a misconfiguration.
c) Having the mailing list re-write the From header when they modify messages
me:
Is this the exact problem discussed here:
https://wiki.list.org/DEV/DMARC
Is there a google-blessed way to continue to run a mailman mailing list? E.g. "you should upgrade to Mailman 3"
I see the header
X-Mailman-Version: 2.1.17
but https://wiki.list.org/DEV/DMARC refers to features added in Mailman 2.1.18
so maybe "upgrade to 2.1.18"?
spam analyst:
I'm not aware of any officially Google-sanctioned way to run a mailing list; plus the issues in this case aren't Google-specific: the mailing list will have the same issues with any email service provider that enforces DMARC. The closest thing we have are forwarding best practices, which can be found here: https://support.google.com/mail/answer/175365?hl=en
As to Mailman, I'm not familiar with it, but the link you shared is a pretty good one with regards to dealing with DMARC from mailing lists. The suggestions in the Mailman 3 section would fix the issue.
mail headers from message referenced above:
Delivered-To: martinrb@google.com
Received: by 2002:a25:c791:0:0:0:0:0 with SMTP id w139-v6csp35860ybe;
Wed, 3 Oct 2018 04:21:59 -0700 (PDT)
X-Google-Smtp-Source: ACcGV616UmG2NafO4Z7S5WeZ03NTOQNkF6dLCxPh5XIHUH1UPuqKz3Y4/S7D3sI2cmzwsKslRksP
X-Received: by 2002:a63:9d01:: with SMTP id i1-v6mr922759pgd.98.1538565718960;
Wed, 03 Oct 2018 04:21:58 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1538565718; cv=none;
d=google.com; s=arc-20160816;
b=0XMlm35jtMStgbCmGPnnELDdBOizEhLR4zyRojBGsd9g7TeFjYDIpIrA99fxGNT5h8
lh4tBe4/qCw0vzkRvxpQ6AraNGBOEH5dGjCJeKGETr8r877oUljYQWq1ZthMY8V2A/Nf
kZY3a1sPZHPG0OtJJAgCIslI1mBKtCFuM0A786EjZZA3eiytQbwbI/dHbPNBhfZ4a72U
fdR7iQGdTUN7gXFIxCBqSsDWqy6KiyF5uZnkPfnNbmqfAz/zest+DZDsW2VW6l/h2INW
eTIvNgDeAS+jCUlqmsL0gmhU6ZFzq40OjoGB5FEOzbbKNSXQIy59gtec83pItuwdi7Wp
qVTw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
h=sender:errors-to:list-subscribe:list-help:list-post:list-archive
:list-unsubscribe:list-id:precedence:to:subject:message-id:date:from
:mime-version:dkim-signature:delivered-to;
bh=X01jHIYgqHE9R0MHnbgV5r3ICMISwUt97OJfQofiTig=;
b=yGl5Q8aLC09dwQv/DiPTNAjNxuJy7wIb40iKrdTguXdUmBIORqoBs0YFFSANWuiuqf
0lZ+Kk28Rtsb/GmilQJFaTmhICRYaoqKvNoudRaN6vYhF22zUVlux84T1L9LXxIIW5Sx
rhiD500qpR0JvlvvaJL6ZQehpfXZCeYxw4fCvtfp8pM5qOii63nD4XjFgoYEUQQNsKXO
lu8zRbU8BJDvfTn+7jqSEnATLHawDBlT1HLUymvBt9S3DbiJGlUHCPTJAsgFogEVCpys
iB7AtpIim8+jqdrlUIJ7aOi04ppDzp4oJWYGXY5cz5M12AnZrpxWMtvGyTf1BTrX9iFD
dkPw==
ARC-Authentication-Results: i=1; mx.google.com;
dkim=fail header.i=@google.com header.s=20161025 header.b=v3HuYX4A;
spf=neutral (google.com: 141.146.126.229 is neither permitted nor denied by best guess record for domain of compiler-dev-bounces@openjdk.java.net) smtp.mailfrom=compiler-dev-bounces@openjdk.java.net;
dmarc=fail (p=REJECT sp=REJECT dis=QUARANTINE) header.from=google.com
Return-Path: <compiler-dev-bounces@openjdk.java.net>
Received: from acsinet41.oracle.com (acsinet41.oracle.com. [141.146.126.229])
by mx.google.com with ESMTPS id f9-v6si1158678pgh.325.2018.10.03.04.21.27
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Wed, 03 Oct 2018 04:21:58 -0700 (PDT)
Received-SPF: neutral (google.com: 141.146.126.229 is neither permitted nor denied by best guess record for domain of compiler-dev-bounces@openjdk.java.net) client-ip=141.146.126.229;
Authentication-Results: mx.google.com;
dkim=fail header.i=@google.com header.s=20161025 header.b=v3HuYX4A;
spf=neutral (google.com: 141.146.126.229 is neither permitted nor denied by best guess record for domain of compiler-dev-bounces@openjdk.java.net) smtp.mailfrom=compiler-dev-bounces@openjdk.java.net;
dmarc=fail (p=REJECT sp=REJECT dis=QUARANTINE) header.from=google.com
Received: from aojmv0009 (unknown [137.254.59.6]) by acsinet41.oracle.com with smtp
id 5bfe_0653_90869e51_ac08_468f_8c6f_4a1ff1145675; Wed, 03 Oct 2018 11:21:09 +0000
Received: from aojmv0009.oracle.com (localhost [127.0.0.1]) by aojmv0009 (Postfix) with ESMTP id 8D4A329001C; Wed,
3 Oct 2018 11:21:05 +0000 (UTC)
X-Original-To: compiler-dev@openjdk.java.net
Delivered-To: compiler-dev@openjdk.java.net
Received: from ucsinet40.oracle.com (ucsinet40.oracle.com [156.151.31.68]) by aojmv0009 (Postfix) with ESMTP id 7773F290012 for <compiler-dev@openjdk.java.net>; Wed,
3 Oct 2018 11:21:03 +0000 (UTC)
Received: from mail-qt1-f178.google.com (unknown [209.85.160.178]) by ucsinet40.oracle.com with smtp (TLS: TLSv1/SSLv3,128bits,ECDHE-RSA-AES128-GCM-SHA256) id 048d_18d4_cb52d178_2344_4963_b050_5071bc8fb074; Wed, 03 Oct 2018 11:21:02 +0000
Received: by mail-qt1-f178.google.com with SMTP id q40-v6so5434753qte.0 for <compiler-dev@openjdk.java.net>; Wed, 03 Oct 2018 04:21:02 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=X01jHIYgqHE9R0MHnbgV5r3ICMISwUt97OJfQofiTig=; b=v3HuYX4APEny94kHJMQq0O2nDoSTs8vQXHbhrW6+rVFOtqn7tZfL5GD7zAh6/rRIVm TJcGl+WEzVn5sTmA15jopECRSfNbQ2XgCZkH7dvgAn6pIZTvqE5+392BP4KBCqmmR5/i qLWoGjq4bRxl7jiGBRzV/nKa2TKrMrvfBgsTaq0tKJmv0EEAhxw6rzee+IWQaWK18sdH 9ydNUsxss4Okg2b7HjxpDioYhzNC+Z9TRPLf1/CWbovvi6IUWLZMGevxOh/LORf2IGMB DM1uieHf4TX/9zu4/0AQw8JR3GnHnKc8b340aU0vgW+xNoyBvK4qCUZe0dIRG3GvO5ex kanA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=X01jHIYgqHE9R0MHnbgV5r3ICMISwUt97OJfQofiTig=; b=p/SnBL3ydnY+th6aE0K4ecMlunnAvHKXJnJfFry1g4CL4BcJNsl5NSxb76Zjc7Q/Ye vrW7PO2pTl88zHT0IPQKO6vvqRUpPzJKtpDX/Aw6DGAZV2B3BO7vK8dKSRaVDot71NZg qIHMeUQ00M6tGPF994tbwg946iGf4ho4XUik/lBgrHAlhbBO7ckmUz3lRwychq+ATL1Q +tXdg3NddtnlKQHkVXwmfp9lfCAPZYfjxSxQGBbeUt8KjATVFJJ0Ot1D4tEqv4hXnNPS tt5Y5SjcvOHeAHfw6NG2Tcsl+F5cJORchFCVwV11B0PyB784DzRESyaeT9wj5vLDJ1a1 6h1A==
X-Gm-Message-State: ABuFfogUHZlw56wb54U+PkNPikDgZjc6uv9AdHozvwPQrNDz8cqMip3A uUyO0d0I9eLSyrmxjtbzDaJE67P/IPyckgIheIVxvn3RWro2nQ==
X-Received: by 2002:a05:6214:1091:: with SMTP id o17mr724665qvr.97.1538565661022;
Wed, 03 Oct 2018 04:21:01 -0700 (PDT)
MIME-Version: 1.0
From: Ron Shapiro <ronshapiro@google.com>
Date: Wed, 3 Oct 2018 14:20:48 +0300
Message-ID: <CACA6h9to7szM0dSpOjUhLWFWTUEHCvRNFhS_x+TgeesS6qYC_w@mail.gmail.com>
Subject: Name.contentEquals() performance
To: compiler-dev <compiler-dev@openjdk.java.net>
Content-Type: multipart/alternative; boundary="000000000000b0d7b30577513b6e"
X-BeenThere: compiler-dev@openjdk.java.net
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: "Technical discussion about the development of the Java compiler \(javac\)" <compiler-dev.openjdk.java.net>
List-Unsubscribe: <http://mail.openjdk.java.net/mailman/options/compiler-dev>,
<mailto:compiler-dev-request@openjdk.java.net?subject=unsubscribe>
List-Archive: <http://mail.openjdk.java.net/pipermail/compiler-dev/>
List-Post: <mailto:compiler-dev@openjdk.java.net>
List-Help: <mailto:compiler-dev-request@openjdk.java.net?subject=help>
List-Subscribe: <http://mail.openjdk.java.net/mailman/listinfo/compiler-dev>,
<mailto:compiler-dev-request@openjdk.java.net?subject=subscribe>
Errors-To: compiler-dev-bounces@openjdk.java.net
Sender: compiler-dev <compiler-dev-bounces@openjdk.java.net>