MqTT in Wiser for KNX

[JRP]

Hello
I have been trying to use wiser for KNX to connect to an MQTT broker (mosquitto) but am getting nothing.
The broker is running on my PC, I check the topics and they work, in the script I put the ip address of the pc where the broker runs and I have the addresses defined in the objects tab, but without result.
I don't know if wiser have what it takes to connect or if it is necessary to install libmosquitto.ipk, luamosquitto.ipk and luasockets.ipk as it comes in the Gbridge.io guide that I found in the forum.

Would someone be so kind as to tell me how to proceed?
Thank you.
a greeting

[Daniel]

First we have to see your script. What fw do you use?
Have you seen this thread?
https://forum.logicmachine.net/showthread.php?tid=2628

[JRP]

First we have to see your script. What fw do you use?
Have you seen this thread?

Hi Daniel thanks for replying.

Wiser has the fw 2.4.0 as you can see in the following image.


I use the script that I publish Admin in this thread
https://forum.logicmachine.net/showthrea...light=mqtt
I have created the resident script at interval 0 and changed the IP address to the address of the pc where the mosquitto broker runs.
The object mapping I have used the same as a test.

Code:

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154

if not broker then   broker = '192.168.0.12'   function multiply(mult)     return function(value)       local num = tonumber(value)       if num then         return num * mult       else         return value       end     end   end   -- topic to object map   mqtt_to_object = {     ['in/topic1'] = '32/1/1',     ['in/topic2'] = '32/1/2',   }    -- object to topic map   object_to_mqtt = {     ['1/1/1'] = 'out/topic1',     ['1/1/2'] = 'out/topic2',   }   datatypes = {}   grp.sender = 'mq'   require('socket')   for addr, _ in pairs(object_to_mqtt) do     local obj = grp.find(addr)     if obj then       datatypes[ addr ] = obj.datatype     end   end   mclient = require('mosquitto').new()   mclient.ON_CONNECT = function(res, ...)     log('mqtt connect status', res, ...)     if res then       for topic, _ in pairs(mqtt_to_object) do         mclient:subscribe(topic)       end     else       mclient:disconnect()     end   end   mclient.ON_MESSAGE = function(mid, topic, payload)     local addr = mqtt_to_object[ topic ]     if addr then       local fn = mqtt_to_object_conv[ topic ]       if fn then         payload = fn(payload)       end       grp.write(addr, payload)     end   end   mclient.ON_DISCONNECT = function(...)     log('mqtt disconnect', ...)     mclientfd = nil   end   function mconnect()     local fd     mclient:connect(broker)     fd = mclient:socket()     -- fd ref is valid     if fd then       mclientfd = fd     end   end   mconnect()   function publishvalue(event)     -- message from us or client is not connected     if event.sender == 'mq' or not mclientfd then       return     end     local addr = event.dst     local dpt = datatypes[ addr ]     local topic = object_to_mqtt[ addr ]     -- unknown object     if not dpt or not topic then       return     end     local value = busdatatype.decode(event.datahex, dpt)     if value ~= nil then       if type(value) == 'boolean' then         value = value and 1 or 0       end       mclient:publish(topic, tostring(value))     end   end   lbclient = require('localbus').new(1)   lbclient:sethandler('groupwrite', publishvalue)   lbclientfd = socket.fdmaskset(lbclient:getfd(), 'r')   -- run timer every 5 seconds   timer = require('timerfd').new(5)   timerfd = socket.fdmaskset(timer:getfd(), 'r') end -- mqtt connected if mclientfd then   mclientfdset = socket.fdmaskset(mclientfd, mclient:want_write() and 'rw' or 'r')   res, lbclientstat, timerstat, mclientstat =       socket.selectfds(10, lbclientfd, timerfd, mclientfdset) -- mqtt not connected else   res, lbclientstat, timerstat =     socket.selectfds(10, lbclientfd, timerfd) end if mclientstat then   if socket.fdmaskread(mclientstat) then     mclient:loop_read()   end   if socket.fdmaskwrite(mclientstat) then     mclient:loop_write()   end end if lbclientstat then   lbclient:step() end if timerstat then   -- clear armed timer   timer:read()   if mclientfd then     mclient:loop_misc()   else     mconnect()   end end

Putting the log lines suggested in that same post, this is the result.
MQTT 20.05.2020 16:03:19
Resident script:130: bad argument #2 to 'selectfds' (number expected, got nil)
stack traceback:
[C]: in function 'selectfds'
Is it necessary to install a library or package?
The topics are published and the broker subscribed to them.
Thank you.
a greeting

[Daniel]

In the link which I originally forgot to paste you can find how to use encrypted communication
https://forum.logicmachine.net/showthread.php?tid=2628

[JRP]

In the link which I originally forgot to paste you can find how to use encrypted communication
https://forum.logicmachine.net/showthread.php?tid=2628

Hello
Thank you for responding.
In the broker I have not activated any user or password, as it is only a test for now.
I'm still wondering if it's necessary to install some library in wiser.
Greetings and thanks for helping

[Erwin van der Zwart]

Hi,

There is already a lib in the Wiser 2.4 firmware however i don't think it's the most recent version as 2.4 is out for quite a while.

I'm currently testing on FW 2.5 that will be released (don't pinpoint me on it) somewhere around this summer, and all features that are in Daniels latest script work on FW 2.5 including the broker app.

So you need to wait a couple of weeks (:

BR,

Erwin

[Daniel]

I just tried this script
https://forum.logicmachine.net/showthrea...6#pid16896
on SL with fw 2.4.0 and it works just fine. I used broker on LM.

[JRP]

Hi,

There is already a lib in the Wiser 2.4 firmware however i don't think it's the most recent version as 2.4 is out for quite a while.

I'm currently testing on FW 2.5 that will be released (don't pinpoint me on it) somewhere around this summer, and all features that are in Daniels latest script work on FW 2.5 including the broker app.

So you need to wait a couple of weeks (:

BR,

Erwin

I just tried this script
https://forum.logicmachine.net/showthrea...6#pid16896
on SL with fw 2.4.0 and it works just fine. I used broker on LM.

Hello
Thanks Erwin for the information on the fw update. While the update arrives I will do some more tests.

Hellol Daniel.
Thanks for answering.
I have activated user and password in the mosquito broker, I have copied the script that you indicate and changed ip, user, password and port (1883 is where mosquitto runs on my PC).
I use the MQTT.fx client and post both topic in / topic1 and out / topic1.
I subscribe with the same MQTT.fx and with mosquitto_sub and it works perfectly, but I don't see any changes in the wiser.
In the register tab I see that it does not connect to the broker, put this.

* string: mqtt connect failed Operation timed out
This is the script (I remove the password).

Code:

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167

if not broker then   socket = require('socket')   json = require('json')   broker = '192.168.0.12'   port = 1883   username = 'wiser'   password = 'mypassword'   -- topic to object map   mqtt_to_object = {     ['in/topic1'] = {       addr = '1/0/0',       convert = json.pdecode,     },     ['in/topic2'] = {       addr = '1/0/19',     }   }   -- object to topic map   object_to_mqtt = {     ['1/0/0'] = {       topic = 'out/topic1',       qos = 1,       retain = true,       convert = json.encode,     },     ['1/0/19'] = {       topic = 'out/topic2',     }   }   datatypes = {}   grp.sender = 'mq'   for addr, _ in pairs(object_to_mqtt) do     local obj = grp.find(addr)     if obj then       datatypes[ addr ] = obj.datatype     end   end   mclient = require('mosquitto').new()   mclient.ON_CONNECT = function(res, ...)     log('mqtt connect status', res, ...)     if res then       for topic, _ in pairs(mqtt_to_object) do         mclient:subscribe(topic)       end     else       mclient:disconnect()     end   end   mclient.ON_MESSAGE = function(mid, topic, payload)     local map = mqtt_to_object[ topic ]     if map then       if map.convert then         payload = map.convert(payload)       end       grp.write(map.addr, payload)     end   end   mclient.ON_DISCONNECT = function(...)     log('mqtt disconnect', ...)     mclientfd = nil   end   function mconnect()     local status, rc, msg, fd     status, rc, msg = mclient:connect(broker, port)     if not status then       log('mqtt connect failed ' .. tostring(msg))     end     fd = mclient:socket()     if fd then       mclientfd = fd     end   end   mclient:tls_insecure_set(true)   mclient:login_set(username, password or '')   mconnect()   function publishvalue(event)     -- message from us or client is not connected     if event.sender == 'mq' or not mclientfd then       return     end     local addr = event.dst     local dpt = datatypes[ addr ]     local map = object_to_mqtt[ addr ]     -- unknown object     if not dpt or not map then       return     end     local value = busdatatype.decode(event.datahex, dpt)     if value ~= nil then       if map.convert then         value = map.convert(value)       elseif type(value) == 'boolean' then         value = value and 1 or 0       end       mclient:publish(map.topic, tostring(value), map.qos or 0, map.retain)     end   end   lbclient = require('localbus').new(1)   lbclient:sethandler('groupwrite', publishvalue)   lbclientfd = socket.fdmaskset(lbclient:getfd(), 'r')   -- run timer every 5 seconds   timer = require('timerfd').new(5)   timerfd = socket.fdmaskset(timer:getfd(), 'r') end -- mqtt connected if mclientfd then   mclientfdset = socket.fdmaskset(mclientfd, mclient:want_write() and 'rw' or 'r')   res, lbclientstat, timerstat, mclientstat =       socket.selectfds(10, lbclientfd, timerfd, mclientfdset) -- mqtt not connected else   res, lbclientstat, timerstat =     socket.selectfds(10, lbclientfd, timerfd) end if mclientstat then   if socket.fdmaskread(mclientstat) then     mclient:loop_read()   end   if socket.fdmaskwrite(mclientstat) then     mclient:loop_write()   end end if lbclientstat then   lbclient:step() end if timerstat then   -- clear armed timer   timer:read()   if mclientfd then     mclient:loop_misc()   else     mconnect()   end end

The port number should not be between ''?
He does not put them in the thread from which he copies it.
Thank you very much for the help.
Greetings

[Daniel]

If you don't use encryption (port 1883)
Then change this

mclient:tls_insecure_set(true)

to

mclient:tls_insecure_set(false)

[JRP]

If you don't use encryption (port 1883)
Then change this

mclient:tls_insecure_set(true)

to

mclient:tls_insecure_set(false)

Hello
I have changed it and the same error continues.
Thanks for answering.
a greeting

[Daniel]

This work with my broker so your issue is somewhere there.  Can you connect to your broker by using  MQTT Explorer  client but from another machine?

[JRP]

This work with my broker so your issue is somewhere there.  Can you connect to your broker by using  MQTT Explorer  client but from another machine?

Hello
I have tried with Mqtt explorer and MQtt.fx and neither of them connects from another pc. I put the IP of the pc where I have mosquitto installed and the username and password and nothing.
Surely the problem comes from here.
I have all the computers on the same network, I ping and they all respond.
If in MQtt.fx installed on the same pc as the broker, I put instead of address 127 ......., the ip address of the same pc 192.168.012. It connects and on netstat I see the connection to port 1883.
Will I be the router that blocks traffic? It is what occurs to me.
Thanks for the help.
a greeting

[Daniel]

It is most likely your PC firewall.

[JRP]

It is most likely your PC firewall.

Hello
Eureka!
Effectively it is the w10 firewall that was blocking traffic, I have opened port 1883 and it has connected and worked the first time.
After writing the last message it occurred to me that it could be the firewall, but until you told me to try from another PC, the light bulb did not light up.
Thank you very much Daniel for your interest and your reconmedances.
a greeting

[manos@dynamitec]

Hello Erwin,

I have a SpaceLynk (V1.3 FW 2.7.0) that I would like to use as MQTT client and publish data to a cloud platform here in Belgium "Eniris". This device will serve only the purpose of MQTT client.
Could you or anyone assist me with the MQTT client script to be able to publish data in a certain json structured format?

The format should be like mentioned in the link below:

https://wiki.eniris.be/wiki/publicinform...3A%20MQTT/

In short something like this:

Topic = 'standart1/rp_one_m/T/userid/' and then in the data packet the following
 

Code:

123456789

{   "time" :  1661506491 , --In Unix format   "extraTags" : {     "subId" :  "T19"   },   "fields" : {     "temperature_degC" :  16.76 --this must be the value of the KNX object as a float datatype towards the broker)   } }

I have tested the MQTT client script below and the connection with the broker is ok. And data is being published but not in the certain format sctructure their platform is expecting it.

Code:

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173

if not broker then   broker = 'mqtt.eniris.be'     username = 'xxx'   password = 'xxx'     socket = require('socket')   port = 1883   function multiply(mult)     return function(value)       local num = tonumber(value)       if num then         return num * mult       else         return value       end     end   end   -- topic to object map   mqtt_to_object = {     ['Actual Temp/Hal A/S2'] = '4/0/1',     ['Actual Temp/Hal B/S1'] = '4/0/2',   }   -- optional topic value conversion function   --mqtt_to_object_conv = {   --  ['in/topic1'] = multiply(100),   --  ['in/topic2'] = multiply(0.01),   --}   -- object to topic map   object_to_mqtt = {     ['4/0/1'] = 'Actual Temp/Hal A/S2',     ['4/0/2'] = 'Actual Temp/Hal B/S1',   }   datatypes = {}   grp.sender = 'mq'   require('socket')   for addr, _ in pairs(object_to_mqtt) do     local obj = grp.find(addr)     if obj then       datatypes[ addr ] = obj.datatype     end   end   mclient = require('mosquitto').new()   mclient.ON_CONNECT = function(res, ...)     log('mqtt connect status', res, ...)     if res then       for topic, _ in pairs(mqtt_to_object) do         mclient:subscribe(topic)       end     else       mclient:disconnect()     end   end   mclient.ON_MESSAGE = function(mid, topic, payload)     local addr = mqtt_to_object[ topic ]     if addr then       local fn = mqtt_to_object_conv[ topic ]       if fn then         payload = fn(payload)       end       grp.write(addr, payload)     end   end   mclient.ON_DISCONNECT = function(...)     log('mqtt disconnect', ...)     mclientfd = nil   end   function mconnect()     local status, rc, msg, fd     status, rc, msg = mclient:connect(broker, port)     if not status then       log('mqtt connect failed ' .. tostring(msg))     end     fd = mclient:socket()     if fd then       mclientfd = fd     end   end   --mclient:tls_insecure_set(true)   mclient:login_set(username, password or '')     mconnect()   function publishvalue(event)     -- message from us or client is not connected     if event.sender == 'mq' or not mclientfd then       return     end     local addr = event.dst     local dpt = datatypes[ addr ]     local topic = object_to_mqtt[ addr ]     -- unknown object     if not dpt or not topic then       return     end     local value = busdatatype.decode(event.datahex, dpt)     if value ~= nil then       if type(value) == 'boolean' then         value = value and 1 or 0       end       mclient:publish(topic, tostring(value))     end   end   lbclient = require('localbus').new(1)   lbclient:sethandler('groupwrite', publishvalue)   lbclientfd = socket.fdmaskset(lbclient:getfd(), 'r')   -- run timer every 5 seconds   timer = require('timerfd').new(5)   timerfd = socket.fdmaskset(timer:getfd(), 'r') end -- mqtt connected if mclientfd then   mclientfdset = socket.fdmaskset(mclientfd, mclient:want_write() and 'rw' or 'r')   res, lbclientstat, timerstat, mclientstat =       socket.selectfds(10, lbclientfd, timerfd, mclientfdset) -- mqtt not connected else   res, lbclientstat, timerstat =     socket.selectfds(10, lbclientfd, timerfd) end if mclientstat then   if socket.fdmaskread(mclientstat) then     mclient:loop_read()   end   if socket.fdmaskwrite(mclientstat) then     mclient:loop_write()   end end if lbclientstat then   lbclient:step() end if timerstat then   -- clear armed timer   timer:read()   if mclientfd then     mclient:loop_misc()   else     mconnect()   end end

Thank you in advance for your responce.

Regards

[admin]

If you don't have a lot of frequent updates you can use event script to publish new values then disconnect.

Add to Common functions and change username/password:

Code:

123456789101112131415161718192021222324252627282930313233343536373839

function mqttpublish(topic, data)   local broker = 'mqtt.eniris.be'   local username = 'xxx'   local password = 'xxx'   local port = 1883   local mqtt = require('mosquitto')   local client = mqtt.new()   if type(data) == 'table' then     data = require('json').encode(data)   end   client.ON_CONNECT = function(status, rc, msg)     if status then       log('mqtt cloud connected')       client:publish(topic, data)     else       log('mqtt cloud connect failed ' .. tostring(msg))       client:disconnect()     end   end   client.ON_PUBLISH = function()     log('disconnecting')     client:disconnect()   end   client:login_set(username, password)   local status, rc, msg = client:connect(broker, port)   if status then     client:loop_forever()   else     log('cloud connect failed: ' .. tostring(msg))   end end

Event script example:

Code:

123456789101112

topic = 'Actual Temp/Hal A/S2' data = {   time = os.time(),   extraTags = {     subId = 'T19'   },   fields = {     temperature_degC = event.getvalue()   } } mqttpublish(topic, data)

[manos@dynamitec]

If you don't have a lot of frequent updates you can use event script to publish new values then disconnect.

Add to Common functions and change username/password:

Code:

123456789101112131415161718192021222324252627282930313233343536373839

function mqttpublish(topic, data)   local broker = 'mqtt.eniris.be'   local username = 'xxx'   local password = 'xxx'   local port = 1883   local mqtt = require('mosquitto')   local client = mqtt.new()   if type(data) == 'table' then     data = require('json').encode(data)   end   client.ON_CONNECT = function(status, rc, msg)     if status then       log('mqtt cloud connected')       client:publish(topic, data)     else       log('mqtt cloud connect failed ' .. tostring(msg))       client:disconnect()     end   end   client.ON_PUBLISH = function()     log('disconnecting')     client:disconnect()   end   client:login_set(username, password)   local status, rc, msg = client:connect(broker, port)   if status then     client:loop_forever()   else     log('cloud connect failed: ' .. tostring(msg))   end end

Event script example:

Code:

123456789101112

topic = 'Actual Temp/Hal A/S2' data = {   time = os.time(),   extraTags = {     subId = 'T19'   },   fields = {     temperature_degC = event.getvalue()   } } mqttpublish(topic, data)

Hello Admin,

Thank you for the quick reply.
Unfortunately I will have to publish around 200 objects and its not that they come in the bus very frequently.
Is there and option to publish data every one minute? Like adding a resident script with sleep internal of 1min?

Regards,

[admin]

Are all 200 objects temperature sensors or are there different values that affect the JSON message format? Is extraTags = { subId = 'T19' } common for all messages or not?

[manos@dynamitec]

Are all 200 objects temperature sensors or are there different values that affect the JSON message format? Is extraTags = { subId = 'T19' } common for all messages or not?

Hello Admin,
 
The following things are to be published. 


WaterMeter
Data:
standard1/rp_one_m/waterMetrics/<username>
{
    "time": <UNIX timestamp>,
    "extraTags": {
        "subId": <string>
    }
    "fields": {
        "absVolume_m3": <float>,
        "flow_m3ph": <float>
    }
}

Weather Station
Data:
standard1/rp_one_m/weatherMetrics/<username>
{
    "time": <UNIX timestamp>,
    "extraTags": {
        "subId": <string>
    }
    "fields": {
        "outdoorTemperature_degC": <float>,
        "outdoorBrightness_lux": <float>,
        "windSpeed_kmph": <float>
    }
}

GasMeter
Data:
standard1/rp_one_m/gasMetrics/<username>
{
    "time": <UNIX timestamp>,
    "extraTags": {
        "subId": <string>
    }
    "fields": {
        "absVolume_Nm3": <float>
    }
}

Errors & Messages:
standard1/autogen/alarms/<username>
{
    "time": <UNIX timestamp>,
    "extraTags": {
        "subId": <string>
    }
    "fields": {
        "alarmActive": <boolean>,
        "msg": <srtring>,
        "mbusStatus": <int>    //zero = OK otherwise alarm
    }
}


Thank you again for the great support.

[admin]

You can create a scheduled script that runs every minute and publishes all values. Each message has to specified manually due to a variable format. Add as many addmessage calls as needed and change grp.getvalue addresses. The script will automatically disconnect once all messages are published.

Code:

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798

broker = 'mqtt.eniris.be' username = 'xxx' password = 'xxx' port = 1883 messages = {} time = os.time() function addmessage(topic, data)   messages[ #messages + 1 ] = {     topic = topic,     data = require('json').encode(data),   } end addmessage('standard1/rp_one_m/waterMetrics/' .. username, {   time = time,   extraTags = {     subId = 'T19',   },   fields = {     absVolume_m3 = grp.getvalue('1/1/1'),     flow_m3ph = grp.getvalue('1/1/2'),   } }) addmessage('standard1/rp_one_m/weatherMetrics/' .. username, {   time = time,   extraTags = {     subId = 'T19',   },   fields = {     outdoorTemperature_degC = grp.getvalue('1/1/3'),     outdoorBrightness_lux = grp.getvalue('1/1/4'),     windSpeed_kmph = grp.getvalue('1/1/5'),   } }) addmessage('standard1/rp_one_m/gasMetrics/' .. username, {   time = time,   extraTags = {     subId = 'T19',   },   fields = {     absVolume_Nm3 = grp.getvalue('1/1/6'),   } }) mqtt = require('mosquitto') client = mqtt.new() mids = {} pubcount = 0 client.ON_CONNECT = function(status, rc, msg)   if status then     log('mqtt cloud connected')     for _, message in ipairs(messages) do       local mid = client:publish(message.topic, message.data)       if mid then         pubcount = pubcount + 1         mids[ mid ] = true       end     end     if pubcount == 0 then       client:disconnect()     end   else     log('mqtt cloud connect failed ' .. tostring(msg))     client:disconnect()   end end client.ON_PUBLISH = function(mid)   if mids[ mid ] then     mids[ mid ] = nil     pubcount = pubcount - 1     if pubcount == 0 then       log('disconnecting')       client:disconnect()     end   end end client:login_set(username, password) status, rc, msg = client:connect(broker, port) if status then   client:loop_forever() else   log('cloud connect failed: ' .. tostring(msg)) end