Storage is pushed on localbus

[Smeagol]

Hello,

I have a resident script with interval 0 that is reading out a 'smart' electrical meter (dutch). The meter pushes a new telegram out every second. Which gets parsed and processed by the script. First question, what does the interval 0 mean? Is it run continiously? Does this have negative effects? It seems to work fine... 

To save data between script iterations I use the storage. I save quite a lot of parameters to the storage since the data requires some processing to be useable in trends. I have noticed that this in turn results in a 'heavy' load of the websocket (localbus) since every set storage is pushed out the socket. Is there a way to prevent this? What is the best way to save data between iterations? I can think of one improvement and this is to try and save data in a table instead of separate parameters but this still will be pushed trough the localbus...

The script below:

Code:

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204

lastchar = '!' require('serial') port = serial.open('/dev/ttyUSB0', { -- Let op type van de slimme meter voor de onderstaande instellingen.   baudrate = 115200,             --DSMR 2/3 = 9600        DSMR 4 = 115200   databits = 8,                    --DSMR 2/3 = 7            DSMR 4 = 8   stopbits = 1,                    --DSMR 2/3 = 1            DSMR 4 = 1   parity = 'none',            --DSMR 2/3 = even        DSMR 4 = none   duplex = 'full' }) function readdata(port)   local char, buf   buf = {}   while true do     char = port:read(1)     -- error (timeout) or newline, stop     if char == nil or char == lastchar then       break     -- ignore cr char     elseif char ~= '\r' then       table.insert(buf, char)     end   end   return table.concat(buf) end function splitString(s, delimiter)     local result = {};     for match in string.gmatch(s, "([^"..delimiter.."]+)") do                     table.insert(result, match);     end     return result; end function parseTelegram(data)          local parsed = {}    for line in data:gmatch("([^\n]*)\n?") do     local pristineLine = string.gsub(string.sub(line, 5), "%)", "")         local keyValue = splitString(pristineLine, "%(")     if (keyValue[1]) then         parsed[keyValue[1]] = keyValue[2]     end     end    return parsed end data = readdata(port) port:flush() --------------------------------------------------------------- dailyConsumption = storage.get('dailyConsumption') dailyInjection = storage.get('dailyInjection') intervalConsumption = storage.get('intervalConsumption') intervalInjection = storage.get('intervalInjection') intervalPoints = storage.get('intervalPoints') baseDayConsumption = storage.get('baseDayConsumption') baseDayInjection = storage.get('baseDayInjection') baseNightConsumption = storage.get('baseNightConsumption') baseNightInjection = storage.get('baseNightInjection') parsed = parseTelegram(data) totalDayConsumption = tonumber(splitString(parsed["1.8.1"], "%*")[1]) totalDayInjection = tonumber(splitString(parsed["2.8.1"], "%*")[1]) totalNightConsumption = tonumber(splitString(parsed["1.8.2"], "%*")[1]) totalNightInjection = tonumber(splitString(parsed["2.8.2"], "%*")[1]) currentConsumption = tonumber(splitString(parsed["1.7.0"], "%*")[1]) currentInjection = tonumber(splitString(parsed["2.7.0"], "%*")[1]) now = os.date('*t') if ((not dailyConsumption)) then   dailyConsumption = 0 end if ((not dailyInjection)) then   dailyInjection = 0 end if (not intervalConsumption) then   intervalConsumption = currentConsumption end if (not intervalInjection) then   intervalInjection = currentInjection end if (not intervalPoints) then   intervalPoints = 0 end intervalPoints = intervalPoints + 1 if ((not baseDayConsumption)) then    baseDayConsumption = totalDayConsumption   storage.set('baseDayConsumption', baseDayConsumption) end if ((not baseDayInjection)) then   baseDayInjection = totalDayInjection   storage.set('baseDayInjection', baseDayInjection) end if ((not baseNightConsumption)) then   baseNightConsumption = totalNightConsumption   storage.set('baseNightConsumption', baseNightConsumption) end if ((not baseNightInjection)) then   baseNightInjection = totalNightInjection   storage.set('baseNightInjection', baseNightInjection) end currentConsumtionPhase1 = tonumber(splitString(parsed["21.7.0"], "%*")[1]) currentConsumtionPhase2 = tonumber(splitString(parsed["41.7.0"], "%*")[1]) currentConsumtionPhase3 = tonumber(splitString(parsed["61.7.0"], "%*")[1]) currentInjectionPhase1 = tonumber(splitString(parsed["22.7.0"], "%*")[1]) currentInjectionPhase2 = tonumber(splitString(parsed["42.7.0"], "%*")[1]) currentInjectionPhase3 = tonumber(splitString(parsed["62.7.0"], "%*")[1]) intervalConsumption = intervalConsumption + currentConsumption intervalInjection = intervalInjection + currentInjection dailyConsumption = dailyConsumption + (currentConsumption * (1/3600)) storage.set('dailyConsumption', dailyConsumption) dailyInjection = dailyInjection + (currentInjection * (1/3600)) storage.set('dailyInjection', dailyInjection) if (math.fmod(now["min"], 5) == 0 and now["sec"] == 0) then     grp.write('x/x/x', dailyConsumption)   grp.write('x/x/x', dailyInjection)   grp.write('x/x/x', intervalConsumption / intervalPoints)   grp.write('x/x/x', intervalInjection / intervalPoints)   grp.write('x/x/x', currentConsumtionPhase1)   grp.write('x/x/x', currentConsumtionPhase2)   grp.write('x/x/x', currentConsumtionPhase3)     grp.write('x/x/x', currentInjectionPhase1)   grp.write('x/x/x', currentInjectionPhase2)   grp.write('x/x/x', currentInjectionPhase3)   storage.set('intervalConsumption', nil)   storage.set('intervalInjection', nil)   storage.set('intervalPoints', nil) else     storage.set('intervalConsumption', intervalConsumption)   storage.set('intervalInjection', intervalInjection)   storage.set('intervalPoints', intervalPoints) end if (now["min"] == 0 and now["sec"] == 0) then     grp.write('32/6/8', totalDayConsumption - baseDayConsumption)   grp.write('x/x/x', totalDayInjection - baseDayInjection)   grp.write('x/x/x', totalNightConsumption - baseNightConsumption)   grp.write('x/x/x', totalNightInjection - baseNightInjection) end -- At the end of the day after all was set, reset the values for the next day if ((now["hour"] == 0 and now["min"] == 0 and now["sec"] == 0)) then   dailyConsumption = 0   grp.write('x/x/x', dailyConsumption)   storage.set('dailyConsumption', dailyConsumption) end if ((now["hour"] == 0 and now["min"] == 0 and now["sec"] == 0)) then   dailyInjection = 0   grp.write('x/x/x', dailyInjection)   storage.set('dailyInjection', dailyInjection) end if ((now["hour"] == 0 and now["min"] == 0 and now["sec"] == 0)) then   log('reset consumption for day')   baseDayConsumption = totalDayConsumption   grp.write('x/x/x', totalDayConsumption - baseDayConsumption)   storage.set('baseDayConsumption', baseDayConsumption) end if ((now["hour"] == 0 and now["min"] == 0 and now["sec"] == 0)) then   baseDayInjection = totalDayInjection   grp.write('x/x/x', totalDayInjection - baseDayInjection)   storage.set('baseDayInjection', baseDayInjection) end if ((now["hour"] == 0 and now["min"] == 0 and now["sec"] == 0)) then   baseNightConsumption = totalNightConsumption   grp.write('x/x/x', totalNightConsumption - baseNightConsumption)   storage.set('baseNightConsumption', baseNightConsumption) end if ((now["hour"] == 0 and now["min"] == 0 and now["sec"] == 0)) then   baseNightInjection = totalNightInjection   grp.write('x/x/x', totalNightInjection - baseNightInjection)   storage.set('baseNightInjection', baseNightInjection) end

Any advise would be appreciated!

Kind regards

[Erwin van der Zwart]

Hi,

The resident to 0 is no problem, the socket waits for new chars and the smart meter is pushing out data every 10 seconds so the socket delays the execution of the script until new chars are received.

Some questions: 

1) Without having to "understand" what you have build can you eleborate on what your final goal is?
2) What do you mean with "Every set storage is send out of the socket" ? The meter is read only, and localbus is not in your script so i don't understand that question.

BR,

Erwin

PS: Ben je een Nederlandse gebruiker van de Wiser for KNX?

[Smeagol]

Hi,

The resident to 0 is no problem, the socket waits for new chars and the smart meter is pushing out data every 10 seconds so the socket delays the execution of the script until new chars are received.

Some questions: 

1) Without having to "understand" what you have build can you eleborate on what your final goal is?
2) What do you mean with "Every set storage is send out of the socket" ? The meter is read only, and localbus is not in your script so i don't understand that question.

BR,

Erwin

PS: Ben je een Nederlandse gebruiker van de Wiser for KNX?

Hello,

Thank you for answering! The smart meter is pushing data out every second instead of every 10 seconds but i suppose given your explanation that that does not matter... As long as there is some delay it won't eat up the CPU.

The script i shared works fine and does what i require it to do:

• Segment the data and extract the usefull parts
  
• At the start of a new day (00:00:00) store the meter values (day consumtion, night consumption, day injection and night injection) in memory and for the rest of the day substract these value from the new meter values. This gives you your daily consumption and injection. I wirte this to a virtual address every hour so i can use it in a trend.
  
• For more fine grained trends i use the actual power consumption and injection values of the meter and calculate a 'running' discrete integral with them (so at the end of the day you once again have your daily consumption) and push this on a virtual address every 5 minuts for use in detail trends. I also do this for the power consumtion of each phase.
  

Since the script runs every ~1 seconds and i need data between iteration of the script, the storing of values is done with 'storage.set'. Again this works fine. My main problem, however, is that i noticed that when you use an app like Mosaic (or a custom app) that relies on the localbus.js.gz library, the websocket the library esthablishes with the LM comes under 'heavy' load since all the storage.set method calls trigger an update from the LM to the app via the websocket. 

So in short storage.set triggers a websocket message with it's new value to apps. Since i use storage.set quite heavily and frequently (~20 x every second) i fear for the load on the websocket. The data that i require to store for this script is also only important for this script and should not be on the websocket. I assume however that it is normal for storage to be shared with the apps. I am wondering however if there is a better way of storing data between script iteration or at the very least to find a way to limit it?

Thanks for the advise!

Kind regards

P.S. Ik ben een Belgische gebruiker van een Logic Machine (LM5 Lite) en heb geen Wiser 

[admin]

All global variables in resident scripts are retained after each loop. Only script restart clears these variables. So I'm not sure whether storage is needed at all. Alternatively you can add app: prefix to the storage key. Such keys are not sent over localbus. For example:

Code:

1

storage.set('app:dailyInjection', dailyInjection)

[Bitver]

All global variables in resident scripts are retained after each loop. Only script restart clears these variables. So I'm not sure whether storage is needed at all. Alternatively you can add app: prefix to the storage key. Such keys are sent over localbus. For example:

Code:

1

storage.set('app:dailyInjection', dailyInjection)

You mean "Such keys are not sent over localbus"? because they're not 

[admin]

Yes, they are not sent. There was a typo in my post

[Smeagol]

Yes, they are not sent. There was a typo in my post

Thank you! If global variable state is retained between iteration, then indeed i do not even need storage.