1. Forum
  2. Usenet
  3. COMP.MAIL.SENDMAIL

  • Google "multiple destination domains" issue

    From Vincent Fox@21:1/5 to All on Thu Jun 10 07:58:56 2021
    Hi,

    It's been 9 years since Google did whatever backend change causes this: 451-4.3.0 Multiple destination domains per transaction is unsupported.
    Please
    451 4.3.0 try again. fa9si788651wic.42 - gsmtp

    MTA like sendmail efficiently look up the MX for the recipient list and batch them up, but Google doesn't allow it, leading to a logjam. I have a mailing list server and also a bulkmail server on which I both notice stuff queueing up for retries quite
    often.

    Is there any workaround for this in Sendmail yet?

    It has.become a VEXING problem as more and more people MX Google for their domains. Exim I could set "multi_domains = false" to workaround it. I am however using LDAP mail-routing and certificate authentication, I have to figure out how to do those in
    Exim. I'd rather not have to toss out decades of Sendmail experience over one feature.

    Thanks!

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From bje@ripco.com@21:1/5 to Vincent Fox on Thu Jun 10 16:13:25 2021
    Vincent Fox <vincent.fox@gmail.com> wrote:

    MTA like sendmail efficiently look up the MX for the recipient list and
    batch them up, but Google doesn't allow it, leading to a logjam. I have a mailing list server and also a bulkmail server on which I both notice
    stuff queueing up for retries quite often.

    Is there any workaround for this in Sendmail yet?


    Not sure about this, Claus will probably know off the top of his head, but
    we ran into a similar problem and there was sort of a fix.

    Check out the doc/op/op.me and specifically section 3 "Queue Groups and
    Queue Directories".

    From memory there was a way to change delivery based on MX, size, priority
    and other things. Again not sure but I think you can group the queue by
    domain, so that anything to bill@dom1.com and sally@dom1.com would be
    delivered as a group even if fred@dom2.com and betty@dom2.com are also on
    a gmail MX. Point is, email to dom1.com and dom2.com are not mixed together
    on one delivery even though they end up at the same place.

    This I think was with yahoo or hotmail (google wasn't around yet) but keep
    in mind they sort of count IP addresses these days. No matter who you are emailing, if they see the same IP coming in over a period of time, they'll still block you.

    It's a different error message, something to do with detection of an
    abnormal rate so you might be trading off one problem for another.

    -bruce
    bje@ripco.com

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Grant Taylor@21:1/5 to Vincent Fox on Thu Jun 10 11:14:27 2021
    On 6/10/21 8:58 AM, Vincent Fox wrote:
    Hi,

    Hi,

    It's been 9 years since Google did whatever backend change causes this: 451-4.3.0 Multiple destination domains per transaction is unsupported.
    Please
    451 4.3.0 try again. fa9si788651wic.42 - gsmtp

    I don't recall running into that. Maybe I've been lucky.

    MTA like sendmail efficiently look up the MX for the recipient list
    and batch them up, but Google doesn't allow it, leading to a logjam.
    I have a mailing list server and also a bulkmail server on which I
    both notice stuff queueing up for retries quite often.

    Is there any workaround for this in Sendmail yet?

    I don't know if there is a setting to prevent such batching or not. I
    feel like there probably is.

    The other thing that you might consider - which I use - that might help
    is to try "Interactive" delivery instead of "Queued" delivery. The idea
    being that Sendmail will try to deliver the individual message when it's
    quite likely only the single recipient domain at original inbound SMTP
    time as opposed to queuing and getting a batch that may be multi-domain.



    --
    Grant. . . .
    unix || die

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Claus =?iso-8859-1?Q?A=DFmann?= @21:1/5 to Vincent Fox on Thu Jun 10 17:42:22 2021
    Vincent Fox wrote:

    451-4.3.0 Multiple destination domains per transaction is unsupported.

    It's fantastic that these big companies can make up whatever
    rules they like...

    Is there any workaround for this in Sendmail yet?

    Maybe turn off the m flag for the mailer?

    m This mailer can send to multiple users on the same
    host in one transaction.

    There are other options, but this should be the easiest.

    --
    Note: please read the netiquette before posting. I will almost never
    reply to top-postings which include a full copy of the previous
    article(s) at the end because it's annoying, shows that the poster
    is too lazy to trim his article, and it's wasting the time of all readers.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Vincent Fox@21:1/5 to All on Thu Jun 10 11:24:12 2021
    On Thursday, June 10, 2021 at 10:50:01 AM UTC-7, Claus Aßmann wrote:
    Vincent Fox wrote:

    451-4.3.0 Multiple destination domains per transaction is unsupported.
    It's fantastic that these big companies can make up whatever
    rules they like...

    I agree, but they are a 900 lb gorilla and will not change. Most infuriating is how Google doesn't even respond to questions about why they do it. Non-Google apologists claim it is "anti-spam" which makes no sense at all.

    But evidently there is a way to work around this which works in Exim.
    I was hoping something similar had come along for Sendmail.

    Is there any workaround for this in Sendmail yet?
    Maybe turn off the m flag for the mailer?

    m This mailer can send to multiple users on the same
    host in one transaction.

    There are other options, but this should be the easiest.

    Is this the same as max_rcpts 1?

    Simple case I'm submitting an email:
    67 *@gmail.com
    1 *@foo.com. (MX is google)
    12 *@bar.edu. ( ditto)
    3 *@baz.org. (ditto)

    Google accepts 67 recipients and tempfail the others. So they go for another few spins in the queue. The 4th spin finishes off the last recipents after the set retry interval, which I have shortened but it's still noticeable.

    If I move to splitting per.single recipient.... well I can do that but it's more to the inefficient end of things than I'd like.

    Unfortunately my recipient list is too varied and frankly there are
    too many domains now MX thru google, to use queue groups. I'd be
    chasing my own tail forever manually managing queue groups.

    Thanks!

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Claus =?iso-8859-1?Q?A=DFmann?= @21:1/5 to Vincent Fox on Thu Jun 10 18:39:55 2021
    Vincent Fox wrote:

    I was hoping something similar had come along for Sendmail.

    Not yet - maybe you can write a patch? Unfortunately I don't have
    much time to loook into this, hopefully someone does!

    m This mailer can send to multiple users on the same
    host in one transaction.

    Is this the same as max_rcpts 1?

    Not sure which option you mean here.

    If you mean this:

    5.10. Q -- Queue Group Declaration

    recipients
    The maximum number of recipients per enve-
    lope. Envelopes with more than this number
    of recipients will be split into multiple
    envelopes in the same queue directory. The
    default value 0 means no limit.

    then the answer is: "No" (don't use that).

    Simple case I'm submitting an email:
    67 *@gmail.com
    1 *@foo.com. (MX is google)
    12 *@bar.edu. ( ditto)
    3 *@baz.org. (ditto)

    If I move to splitting per.single recipient.... well I can do that but it's more to the inefficient end of things than I'd like.

    Not really, it's all done in one session - of course it depends on
    the size of the mail - so you don't have to wait for multiple queue
    runs.


    --
    Note: please read the netiquette before posting. I will almost never
    reply to top-postings which include a full copy of the previous
    article(s) at the end because it's annoying, shows that the poster
    is too lazy to trim his article, and it's wasting the time of all readers.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andrzej Adam Filip@21:1/5 to Vincent Fox on Thu Jun 10 21:09:39 2021
    Vincent Fox <vincent.fox@gmail.com> wrote:
    On Thursday, June 10, 2021 at 10:50:01 AM UTC-7, Claus Aßmann wrote:
    Vincent Fox wrote:

    451-4.3.0 Multiple destination domains per transaction is unsupported.
    It's fantastic that these big companies can make up whatever
    rules they like...

    I agree, but they are a 900 lb gorilla and will not change. Most infuriating is how Google doesn't even respond to questions about why they do it. Non-Google apologists claim it is "anti-spam" which makes no sense at all. […]

    IMHO it simplifies a lot "inbound SMTP proxy"
    *without* "store now, forward later".

    --
    [Andrew] Andrzej A. Filip

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kelsey Cummings@21:1/5 to All on Tue Nov 1 20:29:30 2022
    On Thursday, June 10, 2021 at 11:45:01 AM UTC-7, Claus Aßmann wrote:
    Not yet - maybe you can write a patch? Unfortunately I don't have
    much time to loook into this, hopefully someone does!

    Claus, pulling up an old thread here but would having a sponsor provide you the opportunity to implement a solution? This has finally reached my attention too and the ability to split envelopes by domain (something which Brandon Long thinks sendmail can
    do, but I can't figure out how) or perhaps being able to assign a queue group by wildcard matching on an MX record which would allow the cheesy fix of just setting a single recipient per transaction on a queue assigned to gmail's mx servers.

    In any case, savvy users have started to notice and complain about delays into gmail hosted domains because of this.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From bje@ripco.com@21:1/5 to Kelsey Cummings on Wed Nov 2 13:18:29 2022
    Kelsey Cummings <kgc@corp.sonic.net> wrote:

    In any case, savvy users have started to notice and complain about delays into gmail hosted domains because of this.

    I kind of lost the original thread but I think you are looking for this solution.

    In your sendmail.mc stick these in:

    define(`SMTP_MAILER_MAXRCPTS', `1')dnl
    define(`RELAY_MAILER_MAXRCPTS', `1')dnl

    Those will make sendmail queue them out one at a time rather than bundle them together and get that multi-domain isn't allowed error from gmail.

    Been using that for a while and haven't seen a single rejection or error dealing with them. Also works well for the idiots at protection.outlook.com because they randomly set max nrcpts to 2.

    -bruce
    bje@ripco.com

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kelsey Cummings@21:1/5 to b...@ripco.com on Wed Nov 2 11:36:41 2022
    On Wednesday, November 2, 2022 at 6:18:32 AM UTC-7, b...@ripco.com wrote:
    In your sendmail.mc stick these in:

    define(`SMTP_MAILER_MAXRCPTS', `1')dnl
    define(`RELAY_MAILER_MAXRCPTS', `1')dnl

    I'm reluctant to break a fundamental optimization in email delivery to
    work around google's behavior. Although perhaps on today's
    infrastructure it wouldn't really make all that much of difference.

    -K

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Claus =?iso-8859-1?Q?A=DFmann?= @21:1/5 to Kelsey Cummings on Thu Nov 3 03:01:42 2022
    Kelsey Cummings wrote:

    attention too and the ability to split envelopes by domain (something which Brandon Long thinks sendmail can do, but I can't figure out how) or perhaps

    Try the following:
    Add this to your mc file in the proper places:

    QUEUE_GROUP(`one', `Path=/var/spool/mqueue/one, r=1')
    LOCAL_CONFIG
    Kbestmx bestmx

    LOCAL_RULESETS
    Squeuegroup
    R< $+ > $1
    R $+ @ $+ $: $(bestmx $2 $)
    R $+ $: $>SearchList <! qgrpmx> $| <D:$1> <>
    R<?> $@
    R<$+> $# $1

    Add something like this to access_db:
    qgrpmx:google.com. one

    Rebuild the cf file and the map.
    mkdir /var/spool/mqueue/one
    with proper owner/group/permissions

    Now domains which have a "best MX" pointing to google.com
    should end up in queuegroup one which should have this
    option:

    recipients
    The maximum number of recipients per envelope.

    This might be a good start - but I haven't tested it!

    It would be nice if someone gives it a try and actually
    reports back (not like with the IPv4 patch where people
    ask for it but don't give feedback...)


    Note: an enhanced version to send all recipients with the
    same domain to google instead of just 1 per transaction
    probably requires some code changes -- which would take
    much more time and effort.


    --
    Note: please read the netiquette before posting. I will almost never
    reply to top-postings which include a full copy of the previous
    article(s) at the end because it's annoying, shows that the poster
    is too lazy to trim his article, and it's wasting the time of all readers.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kelsey Cummings@21:1/5 to All on Thu Nov 3 19:05:56 2022
    On Thursday, November 3, 2022 at 12:01:44 AM UTC-7, Claus Aßmann wrote:
    Try the following:
    Add this to your mc file in the proper places:
    ...

    Claus, this seems to work as expected provided you don't get stuck thinking that existing FEATURE(`queuegroup') configs are still going to work as well. Perhaps it's possible to do both the bestmx lookup and then override the result with a non-empty
    qgrp lookup. But, that's not an issue for our use case and arguably the other groups we have defined would benefit from being MX based and not domain based in the first place. With two qgrpmx's defined, I was able to split envelopes across both of them
    and the default mqueue with them all respecting the queues' recipient limit. I'd call this a success!

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)