• ANNOUNCE: mqtt 3.0 and broker 2.0 released

    From Schelte@21:1/5 to All on Thu Oct 28 00:24:14 2021
    MQTT (Message Queuing Telemetry Transport) is a simple and lightweight publish-subscribe-based messaging protocol.

    The mqtt package is a Tcl library for adding the client side of the MQTT protocol to an application. The broker package provides resources for implementing an MQTT server.

    Version 3.0 of the mqtt library and version 2.0 of the broker library
    add support for version 5.0 of the MQTT specification. Versions 3.1 and
    3.1.1 of the MQTT specification are still supported and can be selected
    using a configuration option.

    For more information, see https://chiselapp.com/user/schelte/repository/mqtt


    Enjoy,
    Schelte.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Arjen Markus@21:1/5 to Schelte on Tue Nov 2 01:49:10 2021
    On Thursday, October 28, 2021 at 12:28:04 AM UTC+2, Schelte wrote:
    MQTT (Message Queuing Telemetry Transport) is a simple and lightweight publish-subscribe-based messaging protocol.

    The mqtt package is a Tcl library for adding the client side of the MQTT protocol to an application. The broker package provides resources for implementing an MQTT server.

    Version 3.0 of the mqtt library and version 2.0 of the broker library
    add support for version 5.0 of the MQTT specification. Versions 3.1 and
    3.1.1 of the MQTT specification are still supported and can be selected
    using a configuration option.

    For more information, see https://chiselapp.com/user/schelte/repository/mqtt


    Enjoy,
    Schelte.
    Intriguing. Apparently, this is a Tcl-only package, so it requires nothing even remotely exotic

    Regards,

    Arjen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paul Obermeier@21:1/5 to All on Tue Nov 2 11:25:03 2021
    Am 28.10.2021 um 00:24 schrieb Schelte:
    MQTT (Message Queuing Telemetry Transport) is a simple and lightweight publish-subscribe-based messaging protocol.

    The mqtt package is a Tcl library for adding the client side of the MQTT protocol to an application. The broker package provides resources for implementing an MQTT server.

    Version 3.0 of the mqtt library and version 2.0 of the broker library add support for version 5.0 of the MQTT specification. Versions 3.1 and 3.1.1 of the MQTT specification are still supported and can be selected using a configuration option.

    For more information, see https://chiselapp.com/user/schelte/repository/mqtt


    Enjoy,
    Schelte.

    Hi Schelte,

    is it be possible to add a tag or create a release file for the new version? This would make it easier for people generating Tcl distributions (ex. BAWT).

    Thanks,
    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Arjen Markus@21:1/5 to Arjen Markus on Tue Nov 2 12:27:38 2021
    On Tuesday, November 2, 2021 at 9:49:13 AM UTC+1, Arjen Markus wrote:
    On Thursday, October 28, 2021 at 12:28:04 AM UTC+2, Schelte wrote:
    MQTT (Message Queuing Telemetry Transport) is a simple and lightweight publish-subscribe-based messaging protocol.

    The mqtt package is a Tcl library for adding the client side of the MQTT protocol to an application. The broker package provides resources for implementing an MQTT server.

    Version 3.0 of the mqtt library and version 2.0 of the broker library
    add support for version 5.0 of the MQTT specification. Versions 3.1 and 3.1.1 of the MQTT specification are still supported and can be selected using a configuration option.

    For more information, see https://chiselapp.com/user/schelte/repository/mqtt


    Enjoy,
    Schelte.
    Intriguing. Apparently, this is a Tcl-only package, so it requires nothing even remotely exotic

    Regards,

    Arjen

    I tried to get a simple application running:
    - One client sending a small set of messages
    - The other receiving these messages and printing them on stdout
    Unfortunately I am doing something wrong as the receiver did not receive anything. I am using the package on Windows 10.

    I also tried running the tests but that fails on not being able to find the packages (whereas I set the tm path explicitly ...)

    Regards,

    Arjen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Schelte@21:1/5 to Arjen Markus on Tue Nov 2 22:23:31 2021
    On 02/11/2021 20:27, Arjen Markus wrote:
    I tried to get a simple application running:
    - One client sending a small set of messages
    - The other receiving these messages and printing them on stdout Unfortunately I am doing something wrong as the receiver did not receive anything. I am using the package on Windows 10.

    Did you also have a broker running?

    I have just added an example of a minimal setup on the mqtt wiki: https://chiselapp.com/user/schelte/repository/mqtt/wiki?name=Minimal+example

    I also tried running the tests but that fails on not being able to find the packages (whereas I set the tm path explicitly ...)

    See the Makefile for how I run the tests on linux. Basically I'm setting
    the TCL8_6_TM_PATH environment variable before running tests/all.tcl. I
    assume something similar should work on Windows.

    Note that the tests depend on mosquitto and its clients being available.
    I thought it makes more sense to test the packages against an external reference. If I tested them against each other, a misunderstanding of
    the specifications would go unnoticed because the same mistake would be
    in both parts.


    Schelte.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Schelte@21:1/5 to Paul Obermeier on Tue Nov 2 22:26:03 2021
    On 02/11/2021 11:25, Paul Obermeier wrote:
    is it be possible to add a tag or create a release file for the new
    version?
    This would make it easier for people generating Tcl distributions (ex.
    BAWT).

    I have added the mqtt-3.0 and broker-2.0 tags, which both refer to the
    same checkin. Does that work for you?


    Schelte.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Arjen Markus@21:1/5 to Schelte on Wed Nov 3 00:49:46 2021
    On Tuesday, November 2, 2021 at 10:26:05 PM UTC+1, Schelte wrote:
    On 02/11/2021 20:27, Arjen Markus wrote:
    I tried to get a simple application running:
    - One client sending a small set of messages
    - The other receiving these messages and printing them on stdout Unfortunately I am doing something wrong as the receiver did not receive anything. I am using the package on Windows 10.

    Did you also have a broker running?

    I have just added an example of a minimal setup on the mqtt wiki: https://chiselapp.com/user/schelte/repository/mqtt/wiki?name=Minimal+example
    I also tried running the tests but that fails on not being able to find the packages (whereas I set the tm path explicitly ...)

    See the Makefile for how I run the tests on linux. Basically I'm setting
    the TCL8_6_TM_PATH environment variable before running tests/all.tcl. I assume something similar should work on Windows.

    Note that the tests depend on mosquitto and its clients being available.
    I thought it makes more sense to test the packages against an external reference. If I tested them against each other, a misunderstanding of
    the specifications would go unnoticed because the same mistake would be
    in both parts.


    Schelte.
    Hi Schelte,

    yes, my example looks a lot like yours. I turned on logging (via "broker log puts") and saw that both my clients connect. I do filter on a particular topic. Leaving out the filter does not solve the problem.
    I have put the source code of my example and the log from the broker in a ticket for mqtt.

    Regards,

    Arjen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Schelte@21:1/5 to Arjen Markus on Wed Nov 3 10:04:57 2021
    On 03/11/2021 08:49, Arjen Markus wrote:
    yes, my example looks a lot like yours.

    The crucial difference is the last part of my publisher:

    # Allow the client to connect and send the message before
    terminating
    after 100 exit
    vwait briefly

    Because your runclient1.tcl doesn't enter the event loop, it never
    processes the CONNACK from the broker. Published messages are queued
    until the connection is completely established. Your script terminates
    before it gets to that point.

    See also the update in the ticket: https://chiselapp.com/user/schelte/repository/mqtt/tktview?name=7f9280a6ac


    Schelte.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Arjen Markus@21:1/5 to Schelte on Wed Nov 3 02:46:56 2021
    On Wednesday, November 3, 2021 at 10:06:05 AM UTC+1, Schelte wrote:
    On 03/11/2021 08:49, Arjen Markus wrote:
    yes, my example looks a lot like yours.
    The crucial difference is the last part of my publisher:

    # Allow the client to connect and send the message before
    terminating
    after 100 exit
    vwait briefly

    Because your runclient1.tcl doesn't enter the event loop, it never
    processes the CONNACK from the broker. Published messages are queued
    until the connection is completely established. Your script terminates
    before it gets to that point.

    See also the update in the ticket: https://chiselapp.com/user/schelte/repository/mqtt/tktview?name=7f9280a6ac


    Schelte.
    Hi Schelte,

    the quick and dirty solution did not seem to work - or I did something wrong, but your other one did do the trick!

    Regards.

    Arjen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paul Obermeier@21:1/5 to All on Wed Nov 3 11:38:07 2021
    Am 02.11.2021 um 22:26 schrieb Schelte:
    On 02/11/2021 11:25, Paul Obermeier wrote:
    is it be possible to add a tag or create a release file for the new version? >> This would make it easier for people generating Tcl distributions (ex. BAWT).

    I have added the mqtt-3.0 and broker-2.0 tags, which both refer to the same checkin. Does that work for you?


    Schelte.

    That's fine.

    Thanks Schelte.

    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Schelte@21:1/5 to Arjen Markus on Wed Nov 3 12:41:49 2021
    On 03/11/2021 10:46, Arjen Markus wrote:
    the quick and dirty solution did not seem to work - or I did something wrong, but your other one did do the trick!

    This is the full quick and dirty version of the runclient1 script:

    # runclient1.tcl --
    # Start an MQTT client
    #
    ::tcl::tm::path add .
    package require mqtt

    mqtt create Send
    Send connect Sender

    after 100 {set connection 1}
    vwait connection

    for { set i 0} {$i < 10} {incr i} {
    Send publish ECHO [list "Echo $i"]
    puts $i
    after 1000
    }

    I have successfully tested it. If that doesn't work for you, your
    computer may be slow and it needs more than 100 ms to process the
    connection handshake.

    In any case, the ticket can be closed?


    Schelte.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Arjen Markus@21:1/5 to Schelte on Wed Nov 3 06:29:33 2021
    On Wednesday, November 3, 2021 at 12:44:05 PM UTC+1, Schelte wrote:
    On 03/11/2021 10:46, Arjen Markus wrote:
    the quick and dirty solution did not seem to work - or I did something wrong, but your other one did do the trick!

    This is the full quick and dirty version of the runclient1 script:

    # runclient1.tcl --
    # Start an MQTT client
    #
    ::tcl::tm::path add .
    package require mqtt

    mqtt create Send
    Send connect Sender

    after 100 {set connection 1}
    vwait connection

    for { set i 0} {$i < 10} {incr i} {
    Send publish ECHO [list "Echo $i"]
    puts $i
    after 1000
    }

    I have successfully tested it. If that doesn't work for you, your
    computer may be slow and it needs more than 100 ms to process the
    connection handshake.

    In any case, the ticket can be closed?


    Schelte.
    Hi Schelte,

    yes, please close it.

    I have an other question, though, more a usage question: is it possible to publish or receive messages so that only the first client to receives it actually gets it. The idea I have is to have several workers that simply take a message from the queue and
    each one is getting a unique message: first come first serve. In my naïve experiment with two receivers, they both got the same messages.

    Regards,

    Arjen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Arjen Markus@21:1/5 to Schelte on Wed Nov 3 07:47:29 2021
    On Wednesday, November 3, 2021 at 3:30:05 PM UTC+1, Schelte wrote:
    On 03/11/2021 14:29, Arjen Markus wrote:
    I have an other question, though, more a usage question: is it possible to publish or receive messages so that only the first client to receives it actually gets it. The idea I have is to have several workers that simply take a message from the queue
    and each one is getting a unique message: first come first serve. In my naïve experiment with two receivers, they both got the same messages.

    There is a new feature in MQTT v5 called shared subscriptions, which
    seems like it should do exactly what you are describing. To use a shared subscription, you prefix the topic filter with $share/name/. So, in your example you would use something like:

    Receive subscribe {$share/echopool/ECHO} echoString

    Here "echopool" is a name to group the workers for this subscription.
    You can use a different name to create a different set of workers for another topic filter.

    Having said that; it doesn't yet work with my libraries. When you
    brought it up, I already knew I hadn't implemented it in the broker. But
    a quick test against the mosquitto broker showed that it also doesn't
    work on the client side, probably due to the mismatch between the
    subscribed and actual topic ($share/echopool/ECHO vs ECHO). When I tried
    to work around that, I ran into a bug. I will look into it and let you know.


    Schelte.
    Thanks - I suppose a workaround might be to let the client that is distributing the tasks know that another client is ready, so that a message with a specific topic can be sent. A bit roundabout, but that should work.
    And now that I am at it: what forms do the patterns take? Is that a glob pattern or is it something completely different? I did not read it in the manual.

    Regards,

    Arjen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Schelte@21:1/5 to Arjen Markus on Wed Nov 3 15:27:47 2021
    On 03/11/2021 14:29, Arjen Markus wrote:
    I have an other question, though, more a usage question: is it possible to publish or receive messages so that only the first client to receives it actually gets it. The idea I have is to have several workers that simply take a message from the queue
    and each one is getting a unique message: first come first serve. In my naïve experiment with two receivers, they both got the same messages.

    There is a new feature in MQTT v5 called shared subscriptions, which
    seems like it should do exactly what you are describing. To use a shared subscription, you prefix the topic filter with $share/name/. So, in your example you would use something like:

    Receive subscribe {$share/echopool/ECHO} echoString

    Here "echopool" is a name to group the workers for this subscription.
    You can use a different name to create a different set of workers for
    another topic filter.

    Having said that; it doesn't yet work with my libraries. When you
    brought it up, I already knew I hadn't implemented it in the broker. But
    a quick test against the mosquitto broker showed that it also doesn't
    work on the client side, probably due to the mismatch between the
    subscribed and actual topic ($share/echopool/ECHO vs ECHO). When I tried
    to work around that, I ran into a bug. I will look into it and let you know.


    Schelte.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Schelte@21:1/5 to Arjen Markus on Wed Nov 3 17:34:11 2021
    On 03/11/2021 15:47, Arjen Markus wrote:
    And now that I am at it: what forms do the patterns take? Is that a glob pattern or is it something completely different? I did not read it in the manual.

    I expect that people using the mqtt libraries have some basic knowledge
    of the protocol ;-p

    Of course MQTT had to invent their own syntax, which differs from well
    known match patterns such as used by glob, regexp, or even the SQL LIKE operator. It's all explained very well here: https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/


    Schelte.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Arjen Markus@21:1/5 to Schelte on Thu Nov 4 04:05:20 2021
    On Wednesday, November 3, 2021 at 5:34:22 PM UTC+1, Schelte wrote:
    On 03/11/2021 15:47, Arjen Markus wrote:
    And now that I am at it: what forms do the patterns take? Is that a glob pattern or is it something completely different? I did not read it in the manual.
    I expect that people using the mqtt libraries have some basic knowledge
    of the protocol ;-p

    Of course MQTT had to invent their own syntax, which differs from well
    known match patterns such as used by glob, regexp, or even the SQL LIKE operator. It's all explained very well here: https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/


    Schelte.
    Thanks, got the e-book that should enlighten me about it all :).

    Regards,

    Arjen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Uwe Klein@21:1/5 to All on Thu Nov 4 12:09:26 2021
    Am 03.11.21 um 17:34 schrieb Schelte:
    On 03/11/2021 15:47, Arjen Markus wrote:
    And now that I am at it: what forms do the patterns take? Is that a
    glob pattern or is it something completely different? I did not read
    it in the manual.

    I expect that people using the mqtt libraries have some basic knowledge
    of the protocol ;-p

    Of course MQTT had to invent their own syntax, which differs from well
    known match patterns such as used by glob, regexp, or even the SQL LIKE operator. It's all explained very well here: https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/



    Schelte.


    similar:
    years ago i wrote my package "GIN" that used udp multicast
    and allowed to push namespace structured information.

    subscription selection was via mcast addr and port

    ref: https://wiki.tcl-lang.org/page/GIN

    Uwe

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Arjen Markus@21:1/5 to Arjen Markus on Sun Nov 7 05:20:34 2021
    On Thursday, November 4, 2021 at 12:05:23 PM UTC+1, Arjen Markus wrote:
    On Wednesday, November 3, 2021 at 5:34:22 PM UTC+1, Schelte wrote:
    On 03/11/2021 15:47, Arjen Markus wrote:
    And now that I am at it: what forms do the patterns take? Is that a glob pattern or is it something completely different? I did not read it in the manual.
    I expect that people using the mqtt libraries have some basic knowledge
    of the protocol ;-p

    Of course MQTT had to invent their own syntax, which differs from well known match patterns such as used by glob, regexp, or even the SQL LIKE operator. It's all explained very well here: https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/


    Schelte.
    Thanks, got the e-book that should enlighten me about it all :).

    Regards,

    Arjen
    I tried to run one client on one machine and a broker and a client on another, but that failed. No connection was set up. That is probably due to the way the networking is configured [*]. It was merely an experiment. But I noticed that there was no
    timeout message from the client script. When I tried a simple client/server application, I did get messages like "unknown host" or "connection timed out".

    Regards,

    Arjen

    [*] This is all a trifle beyond my knowledge - the machine I tried to connect to is a virtual machine accessible via a VPN connection. I have no idea what the restrictions are or how to deal with them.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Schelte@21:1/5 to Arjen Markus on Mon Nov 8 21:22:36 2021
    On 07/11/2021 14:20, Arjen Markus wrote:
    I tried to run one client on one machine and a broker and a client on another, but that failed. No connection was set up. That is probably due to the way the networking is configured [*]. It was merely an experiment. But I noticed that there was no
    timeout message from the client script. When I tried a simple client/server application, I did get messages like "unknown host" or "connection timed out".

    That is correct. For MQTT clients it is usually important to keep trying
    to connect, rather than give up at the first sign of trouble.

    If you want to make sure the client connection gets established within a certain time limit, subscribe to the $SYS/local/connection topic. If no
    message is reported to the callback saying "state connected" within the
    allowed time, you can report that in whatever way is appropriate.

    For example:

    tcl::tm::path add [file dirname [file normalize [info script]]]
    package require mqtt

    mqtt create mq
    mq connect
    mq subscribe {$SYS/local/connection} connection
    # Receive all messages
    mq subscribe # message

    set timeoutid [after 30000 {puts "Connection timeout";exit 2}]

    proc connection {topic message args} {
    if {[dict get $message state] eq "connected"} {
    puts "Connected"
    after cancel $::timeoutid
    }
    }

    proc message {topic message args} {
    puts "$topic: $message"
    }

    vwait forever


    Schelte.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Arjen Markus@21:1/5 to Schelte on Mon Nov 8 21:59:49 2021
    On Monday, November 8, 2021 at 9:24:05 PM UTC+1, Schelte wrote:
    On 07/11/2021 14:20, Arjen Markus wrote:
    I tried to run one client on one machine and a broker and a client on another, but that failed. No connection was set up. That is probably due to the way the networking is configured [*]. It was merely an experiment. But I noticed that there was no
    timeout message from the client script. When I tried a simple client/server application, I did get messages like "unknown host" or "connection timed out".

    That is correct. For MQTT clients it is usually important to keep trying
    to connect, rather than give up at the first sign of trouble.

    If you want to make sure the client connection gets established within a certain time limit, subscribe to the $SYS/local/connection topic. If no message is reported to the callback saying "state connected" within the allowed time, you can report that in whatever way is appropriate.

    For example:

    tcl::tm::path add [file dirname [file normalize [info script]]]
    package require mqtt

    mqtt create mq
    mq connect
    mq subscribe {$SYS/local/connection} connection
    # Receive all messages
    mq subscribe # message

    set timeoutid [after 30000 {puts "Connection timeout";exit 2}]

    proc connection {topic message args} {
    if {[dict get $message state] eq "connected"} {
    puts "Connected"
    after cancel $::timeoutid
    }
    }

    proc message {topic message args} {
    puts "$topic: $message"
    }

    vwait forever


    Schelte.
    Ah, that makes sense. Thanks! That reduces the problem to the question why the connection fails ;).

    Regards,

    Arjen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)