Here's another one from the DBRIDGE echo that's come through after I switched my feed from Tommi back on:

On 22 Oct 2021, Rudi Timmermans said the following...

Message 1:

@MSGID: 2:292/140 092a1b30
@REPLY: 1:229/426 1AC2B28E
@SEEN-BY: 1/123 14/0 90/1 105/81 114/709 120/340 123/131 129/305 203/0 @SEEN-BY: 221/1 226/30 227/114 702 229/424 426 428 450 550 664 700
@SEEN-BY: 230/0 240/5832 249/1 206 317 400 280/464 282/1038 292/140 @SEEN-BY: 292/789 854 8125 301/1 317/3 322/757 335/364 342/200 410/9 @SEEN-BY: 633/280
@PATH: 292/140 854 229/426

Message 2:

@MSGID: 2:292/140 092a1b30
@REPLY: 1:229/426 1AC2B28E
@SEEN-BY: 30/0 80/1 103/705 153/757 154/10 203/0 218/700 840 221/1 6 6000 @SEEN-BY: 229/426 664 230/0 240/1120 261/38 280/464 282/464 292/140 789 854 @SEEN-BY: 292/8125 301/0 1 101 113 335/364 341/66 410/9 712/848 920/1 > 4500/1
@SEEN-BY: 5020/1042 5058/104
@PATH: 292/140 854 301/1 221/6

The only difference between the two messages is the blank line between "Rudi" and the tear line:

Message 1:

Greetings,
Rudi

--- D'Bridge 4
* Origin: AfterShock Beta Tester (2:292/140)

Message 2:

Greetings,
Rudi
--- D'Bridge 4
* Origin: AfterShock Beta Tester (2:292/140)

So I think I'm going to leave my extra feeds switched off for a bit while I experiment with another tosser. Any suggestions for something that can run on a Pi? I'm only aware of Mystic, SBBS & Husky.


Jay

... Only dull people are brilliant at breakfast.

--- Mystic BBS v1.12 A47 2021/09/29 (Raspberry Pi/32)
* Origin: Northern Realms (1:229/664)

The only difference between the two messages is the blank line between "Rudi" and the tear line:

As both versions originate from my system and then go separate ways it's obvious there is a problem at 2:301/1 ...

I have the blank in my message base.

\%/@rd
--- DB4 - Oct 12 2021
* Origin: Hou het veilig, hou vol. Het komt allemaal weer goed (2:292/854)

22 Oct 21 10:06, Jay Harris wrote to All:

Here's another one from the DBRIDGE echo that's come through after I switched my feed from Tommi back on:

...

The only difference between the two messages is the blank line between "Rudi" and the tear line:

Mystic thinks they are not dupes even if they have identical MSGID. Would it be possible to adjust Mystic to check only MSGID when it exists?

Message 1:

Greetings,
Rudi

--- D'Bridge 4
* Origin: AfterShock Beta Tester (2:292/140)

Message 2:

Greetings,
Rudi
--- D'Bridge 4
* Origin: AfterShock Beta Tester (2:292/140)

So I think I'm going to leave my extra feeds switched off for a bit while I experiment with another tosser. Any
suggestions for something that can run on a Pi? I'm only aware of Mystic, SBBS & Husky.

Husky runs fine in my Raspberry PI / Ubuntu 20.04.3 LTS. :)

'Tommi

---
* Origin: 2a01:4f9:c011:1ec5:f1d0:2:221:1 (2:221/1)

22 Oct 21 16:33, Ward Dossche wrote to Jay Harris:

The only difference between the two messages is the blank line between
"Rudi" and the tear line:

As both versions originate from my system and then go separate ways it's obvious there is a problem at 2:301/1 ...

2:301/1 is stripping empty lines from the end of a message. 2:203/0 and 2:240/5832 are stripping empty lines from the beginning.

Three RC's are altering in-transit messages. Isn't that cool. :)

'Tommi




---
* Origin: (2:221/1)

Mystic thinks they are not dupes even if they have identical MSGID.
Would it be possible to adjust Mystic to check only MSGID when it exists?

Do you know if these extra lines are CR characters (ASCII 13)?

I think a better solution for Mystic would be that it does not take the ASCII 13 into consideration when it scans for message content duplication. If they are represented as CRs (ASCII 13) and that is excluded then Mystic will report these messages as duplicates.

The downside is that people would lose their existing dupe database and have to start over in the next version of Mystic but I'm down to do it if people are okay with that.

I'd be hestitant to have Mystic rely only on MSGID since there are network types compatible with Mystic that do not use MSGID at all (that can be gated), there are softwares that don't even use MSGID, and there are softwares that spit out duplicate MSGIDs for completely different messages. Content dupe checking is important to make sure that those situations do not allow dupe floods or report false positives dupes. I'd like to keep that feature in while also addressing this specific issue if possible.

--- Mystic BBS v1.12 A47 2021/09/23 (Windows/64)
* Origin: Sector 7 * Mystic WHQ (1:129/215)

22 Oct 21 13:09, James Coyle wrote to Tommi Koivula:

Mystic thinks they are not dupes even if they have identical MSGID.
Would it be possible to adjust Mystic to check only MSGID when it exists?

Do you know if these extra lines are CR characters (ASCII 13)?

A hexdump of my test message in fidotest echo:

=== Cut ===
Hexdump of message text:

0000 01 4D 53 47 49 44 3A 20 32 3A 32 32 31 2F 36 20 .MSGID: 2:221/6
0010 36 31 36 63 36 32 31 32 0D 01 50 49 44 3A 20 74 616c6212..PID: t 0020 78 74 32 70 6B 74 2F 6C 6E 78 20 31 2E 39 2E 30 xt2pkt/lnx 1.9.0 0030 2D 63 75 72 20 32 30 32 31 2D 31 30 2D 30 38 0D -cur 2021-10-08. 0040 01 54 49 44 3A 20 74 78 74 32 70 6B 74 2F 6C 6E .TID: txt2pkt/ln 0050 78 20 31 2E 39 2E 30 2D 63 75 72 20 32 30 32 31 x 1.9.0-cur 2021 0060 2D 31 30 2D 30 38 0D 01 43 48 52 53 3A 20 55 54 -10-08..CHRS: UT 0070 46 2D 38 20 34 0D 0D 0D 0D 20 2D 0D 0D 0D 0D 2D F-8 4.... -....- 0080 2D 2D 20 74 65 73 74 0D 20 2A 20 4F 72 69 67 69 -- test. * Origi 0090 6E 3A 20 74 65 73 74 20 28 32 3A 32 32 31 2F 36 n: test (2:221/6 00A0 2E 30 29 0D 53 45 45 4E 2D 42 59 3A 20 32 32 31 .0).SEEN-BY: 221 00B0 2F 31 20 36 20 33 36 30 20 32 38 30 2F 35 35 35 /1 6 360 280/555 00C0 35 20 33 30 31 2F 31 20 33 33 35 2F 33 36 34 20 5 301/1 335/364
00D0 33 34 31 2F 36 36 20 34 36 30 2F 35 38 20 34 35 341/66 460/58 45 00E0 30 30 2F 31 20 35 30 32 30 2F 31 30 34 32 0D 01 00/1 5020/1042.. 00F0 50 41 54 48 3A 20 32 32 31 2F 36 0D 00 00 00 00 PATH: 221/6.....
=== Cut ===

There you can see 0D's in line of "0070".

I think a better solution for Mystic would be that it does not take the ASCII 13 into consideration when it scans
for message content duplication. If they are represented as CRs (ASCII 13) and that is excluded then Mystic will
report these messages as duplicates.

Not a bad idea. :)

The downside is that people would lose their existing dupe database and have to start over in the next version of
Mystic but I'm down to do it if people are okay with that.

I'd be hestitant to have Mystic rely only on MSGID since there are network types compatible with Mystic that do
not use MSGID at all (that can be gated), there are softwares that don't even use MSGID, and there are softwares
that spit out duplicate MSGIDs for completely different messages. Content dupe checking is important to make sure
that those situations do not allow dupe floods or report false positives dupes. I'd like to keep that feature in
while also addressing this specific issue if possible.

It's not easy, I know..

'Tommi

---
* Origin: rbb.fidonet.fi (2:221/1)

A hexdump of my test message in fidotest echo:

There you can see 0D's in line of "0070".

Thank you for the confirmation. Its kind of odd that Mystic wasn't already ignoring them for text comparison considering it does ignore the 0A
linefeed.

The only thing to worry about now is that if I add this in, it will force SysOps to have to reset their dupe database. It'll be worth it in the long run that's for sure, but it might annoy some people now.

... There will be a rain dance Friday night, weather permitting!

--- Mystic BBS v1.12 A47 2021/09/23 (Windows/64)
* Origin: Sector 7 * Mystic WHQ (1:129/215)

Three RC's are altering in-transit messages. Isn't that cool. :)

Maybe the FTSC could define it as a standard ...

\%/@rd

--- DB4 - Oct 12 2021
* Origin: Hou het veilig, hou vol. Het komt allemaal weer goed (2:292/854)

Hello Jay!
22 Oct 21 10:06, you wrote to all:

So I think I'm going to leave my extra feeds switched off for a bit
while I experiment with another tosser. Any suggestions for something that can run on a Pi? I'm only aware of Mystic, SBBS & Husky.

Husky Fido project, its a pain, but it runs on Linux just about any distro; You build from source so you can tune it for your machine.

Scott


---
* Origin: -={ The Digital Post }=- (1:266/420)

22 Oct 21 22:00, Wilfred van Velzen wrote to Tommi Koivula:

2:301/1 is stripping empty lines from the end of a message. 2:203/0
and 2:240/5832 are stripping empty lines from the beginning.

Three RC's are altering in-transit messages. Isn't that cool. :)

And they (and their software) probably have been doing so for decades, yet it wasn't a problem in dupe detecting
for that same period. So it seems the tossers that have been arround for that same time, have been adapted to this
behaviour, and are coping quite well... ;-)

Sure thing. And now Mystic needs to be changed because of flaws in other software.

2:301/1 will probably be fixed, Matthias only needs to wake up first.

'Tommi

---
* Origin: rbb.fidonet.fi (2:221/1)

Hello Tommi!

22 Oct 21 17:59, you wrote to Ward Dossche:

2:301/1 is stripping empty lines from the end of a message. 2:203/0
and 2:240/5832 are stripping empty lines from the beginning.

2:203/0 and 2:240/5832 both use Squish. I've never noticed them stripping empty lines, but I would have to test further to confirm. I can confirm that your message arrived here with 3 blank lines at the top following the kludge lines. Since we're linked directly, there wasn't much opportunity for it to get messed up.

Andrew

--- GoldED+/LNX 1.1.5-b20180707
* Origin: Phoenix BBS * phoenix.bnbbbs.net (1:320/219)

Hi Andrew.

23 Oct 21 13:19, you wrote to me:

2:301/1 is stripping empty lines from the end of a message. 2:203/0
and 2:240/5832 are stripping empty lines from the beginning.

2:203/0 and 2:240/5832 both use Squish. I've never noticed them stripping empty lines, but I would have to test further to confirm.

I sent few messages to fidotest echo and there I found out that blank lines in the beginning of the message were stripped.

Please do some more tests.

'Tommi

--- GoldED+/LNX 1.1.5-b20210627
* Origin: nntps://news.fidonet.fi (2:221/6)

Hello Tommi,

@CHRS: UTF-8 4
@TZUTC: 0300
@TID: hpt/lnx 1.9.0-cur 2021-10-08
Hi Andrew.

23 Oct 21 13:19, you wrote to me:

2:301/1 is stripping empty lines from the end of a message.
2:203/0 and 2:240/5832 are stripping empty lines from the
beginning.

2:203/0 and 2:240/5832 both use Squish. I've never noticed them
stripping empty lines, but I would have to test further to
confirm.

I sent few messages to fidotest echo and there I found out that blank lines in the beginning of the message were stripped.

Please do some more tests.

'Tommi

--- GoldED+/LNX 1.1.5-b20210627
* Origin: nntps://news.fidonet.fi (2:221/6)
SEEN-BY: 153/757 154/10 218/840 221/1 6 6000 280/5555 301/1 335/364
4500/1
SEEN-BY: 5058/104
@PATH: 221/6

I received two copies of this message. This one directly from your node. I'll quote tho other next.

Ttyl :-),
Al

... Diplomacy gets you out of what tact would have prevented
--- GoldED+/LNX 1.1.5-b20180707
* Origin: The Rusty MailBox - Penticton, BC Canada (1:153/757)

Hello Tommi,

@CHRS: UTF-8 4
@TZUTC: 0300
@TID: hpt/lnx 1.9.0-cur 2021-10-08
Hi Andrew.

23 Oct 21 13:19, you wrote to me:

2:301/1 is stripping empty lines from the end of a message.
2:203/0 and 2:240/5832 are stripping empty lines from the
beginning.

2:203/0 and 2:240/5832 both use Squish. I've never noticed them
stripping empty lines, but I would have to test further to
confirm.

I sent few messages to fidotest echo and there I found out that blank lines in the beginning of the message were stripped.

Please do some more tests.

'Tommi
--- GoldED+/LNX 1.1.5-b20210627
* Origin: nntps://news.fidonet.fi (2:221/6)
SEEN-BY: 15/0 16/101 19/36 20/4609 30/0 80/1 103/705 106/201 116/18 120/302 331
SEEN-BY: 128/2 153/757 7715 154/10 218/700 221/1 6 6000 222/2 229/426 240/1120
SEEN-BY: 250/1 261/38 100 1466 266/512 267/155 275/100 280/464 5555 282/464
SEEN-BY: 282/1056 1060 291/100 111 292/854 301/0 1 101 113 123 812
320/219
SEEN-BY: 335/364 340/400 341/66 396/45 460/58 640/1321 712/848 801/161
189
SEEN-BY: 920/1 2320/105 3634/12 4500/1 5020/1042 5058/104
@PATH: 221/6 301/1 261/38

I received this copy also with the blank line between your sig and tear line missing.

I sometimes see these dupes and hpt doesn't catch them as dupes and sends them on to my links.

I have a point running MBSE and MBSE's tosser does catch these dupes and puts them in the dupe area.

Ttyl :-),
Al

... I am a Klingon, sir. I do NOT whistle while I work!
--- GoldED+/LNX 1.1.5-b20180707
* Origin: The Rusty MailBox - Penticton, BC Canada (1:153/757)