1. Forum
  2. Usenet
  3. NEWS.SOFTWARE.NNTP

  • Diablo IFILTER/InternalFilter Issue

    From Jesse Rehmer@21:1/5 to All on Thu Feb 2 04:43:12 2023
    I’m taking on partial binary feeds and am in the process of migrating to Diablo, but am having an issue with the IFILTER/InternalFilter. I had used Diablo with a similar setup many years ago, and thought this filter worked
    well back then, but am finding it is detecting many binary articles posted in binary groups and rejecting them.

    Looking at the articles it is rejecting, I cannot determine why. Diablo isn’t rejecting all binary articles, but a fair amount. The log lines look like:

    2023-02-01 22:01:42.411 chomp-out - <IdZuKgIdGdSnNsJzQaCpWqIt-1675310467589@nyuu> 42789 050200 IncomingFilter

    When I look at the headers and beginning of the body, I don’t see any reason why this wouldn’t pass through the InternalFilter:

    Message-Id: <IdZuKgIdGdSnNsJzQaCpWqIt-1675310467589@nyuu>
    Subject: [022/290] - "15de68c2ff06204845b63c707ea123e7.part03.rar.par2" yEnc (1/1) 40452
    From: 55c6e45471b4aef7@.com
    Newsgroups: alt.binaries.games.encrypted, alt.binaries.boneless
    Date: Thu, 02 Feb 2023 04:01:07 GMT
    Path: chomp-out!feed1.usenet.blueworldhosting.com!npeer.as286.net!npeer-ng0.as286.n et!proxad.net!feeder1-1.proxad.net!94.232.112.246.MISMATCH!feed.abavia.com!ab e006.abavia.com!abe003.abavia.com!feeder1.feed.usenet.farm!feed.usenet.farm!t r1.eu1.usenetexpress.com!feeder.usenetexpress.com!tr1.iad1.usenetexpress.com! news.newsgroupdirect.com!not-for-mail
    User-Agent: Nyuu/0.4.0
    Lines: 324
    Nntp-Posting-Date: Thu, 02 Feb 2023 04:01:07 +0000
    X-Received-Bytes: 42305
    Organization: NewsgroupDirect
    X-Complaints-To: abuse@newsgroupdirect.com
    Xref: feed1.usenet.blueworldhosting.com junk:25332712

    =ybegin part=1 total=1 line=128 size=40452 name=15de68c2ff06204845b63c707ea123e7.part03.rar.par2
    =ypart begin=1 end=40452

    The article matches the group filter *.bina* and the arttype should be
    detected as binary, but seems not to be. Any ideas?

    Regards,

    Jesse Rehmer

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jesse Rehmer@21:1/5 to jesse.rehmer@blueworldhosting.com on Thu Feb 2 05:01:59 2023
    On Feb 1, 2023 at 10:43:12 PM CST, "Jesse Rehmer" <jesse.rehmer@blueworldhosting.com> wrote:

    I’m taking on partial binary feeds and am in the process of migrating to Diablo, but am having an issue with the IFILTER/InternalFilter. I had used Diablo with a similar setup many years ago, and thought this filter worked well back then, but am finding it is detecting many binary articles posted in binary groups and rejecting them.

    Looking at the articles it is rejecting, I cannot determine why. Diablo isn’t
    rejecting all binary articles, but a fair amount. The log lines look like:

    2023-02-01 22:01:42.411 chomp-out - <IdZuKgIdGdSnNsJzQaCpWqIt-1675310467589@nyuu> 42789 050200 IncomingFilter

    When I look at the headers and beginning of the body, I don’t see any reason
    why this wouldn’t pass through the InternalFilter:

    Message-Id: <IdZuKgIdGdSnNsJzQaCpWqIt-1675310467589@nyuu>
    Subject: [022/290] - "15de68c2ff06204845b63c707ea123e7.part03.rar.par2" yEnc >> (1/1) 40452
    From: 55c6e45471b4aef7@.com
    Newsgroups: alt.binaries.games.encrypted, alt.binaries.boneless
    Date: Thu, 02 Feb 2023 04:01:07 GMT
    Path:
    chomp-out!feed1.usenet.blueworldhosting.com!npeer.as286.net!npeer-ng0.as286.n
    et!proxad.net!feeder1-1.proxad.net!94.232.112.246.MISMATCH!feed.abavia.com!ab
    e006.abavia.com!abe003.abavia.com!feeder1.feed.usenet.farm!feed.usenet.farm!t
    r1.eu1.usenetexpress.com!feeder.usenetexpress.com!tr1.iad1.usenetexpress.com!
    news.newsgroupdirect.com!not-for-mail
    User-Agent: Nyuu/0.4.0
    Lines: 324
    Nntp-Posting-Date: Thu, 02 Feb 2023 04:01:07 +0000
    X-Received-Bytes: 42305
    Organization: NewsgroupDirect
    X-Complaints-To: abuse@newsgroupdirect.com
    Xref: feed1.usenet.blueworldhosting.com junk:25332712

    =ybegin part=1 total=1 line=128 size=40452
    name=15de68c2ff06204845b63c707ea123e7.part03.rar.par2
    =ypart begin=1 end=40452

    The article matches the group filter *.bina* and the arttype should be detected as binary, but seems not to be. Any ideas?

    Regards,

    Jesse Rehmer


    I think I figured out why, but looking at the code am not sure how to fix it.
    After looking at a larger sampling of articles, the one characteristic that stands out is the filtered articles contain a Newsgroup: header with multiple newsgroups, but contains a space after the comma separating the group names.

    (Corrected my subject line, after staring at the code, I think I'm officially toast for the night.)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)