[vz-dev] skew detector

[Justin Otherguy]

Hello *,

there has been a lot of discussion on the list on the "which is the best method for determining the timestamp?". So far there are 3 different methods:
a) controller time in full seconds (using the standard resolution ethersex' main git repo currently provides)
b) controller time in sub second ("hires") resolution (using Jakob's beautiful, minimalistic patches [1] and [2])
c) server time in sub seconds (using the php function "microtime")

(well - actually Jakob has improved his hires method in the mean time [3] - I haven't tried that one yet; will surely do so shortly)

So, I created a php script that does the following:
- expect a parameter that contains the (controller timestamp) in ms since 1970-01-01
- call the backend with UUID_A and the (unmodified) controller timestamp
- call the backend with UUID_B and the controller timestamp - after rounding that to full seconds
- call the backend with UUID_C and the server timestamp

-> so, performing one call from the controller we end up having 3 entries in our db with different UUIDs

Using this I wanted to investigate a bit closer the difference between the methods to help you with your decision on which is most suitable for you (resp. find a common recommendation to vz users). Comparing the three generated graphs we can visualize the skew between the data series and that's why the script is called "skew detector".

I copied 24h of my own data to new uuids on the volkszaehler.org db. You can access the whole bunch using this url [4]. (oh, how I like that grouping feature!)

Side note: on my way I found a bug in ethersex that made the clock run at double speed when using the external clock crystal - they're after fixing it now :-)
So - just in case you are using the clock crystal (which I think you should): make sure to "git pull" and check the clock.

My findings:
- the servertime gives the smoothest reading (whether it is the most accurate one is a different story, though)
- there are (false) peaks that appear to be caused by ntp time correction (pulse with time x; ntp time setting; pulse with time y with false delta)
  I expect Jakob's latest patches to improve that
- the average values are pretty much identical - except for the "controller time full seconds" reading; that's due to the fact that the db scheme on vz.org still has a "unique" property for timestamps - that's been changed in our current git release, so nothing you should worry about if you run our current code and db structure
- for the same reason the readings differ for "controller time full seconds" for phases of high power (1.000 Watts or more): duplicate timestamp entries are not allowed -> reading is below actual value

Conlusion:
all in all I think it doesn't make much of a difference, which time you use - provided:
- your network connection between the controller and the backend is reliable
- the availability of your backend is good
- your network latency is neglectable
If that's the case, I'd go for the server time. Otherwise you're on the safer side using Jakob's hires controller time.

Looking forward to your comments :-)

And: let me know if you're interested in running your controller against the script on vz.org or having the script to run it on your own backend (...lousy prototyping...).


Regards, J.

[1] https://github.com/jahir/ethersex/commit/6b7d46ffb58d5a321f3898f1141125a7684fd303
[2] https://github.com/jahir/ethersex/commit/00c044ef2cb341694900d7608d20451dbe7e30d7
[3] https://github.com/jahir/ethersex
[4] http://volkszaehler.org/demo/frontend/?from=1296082800000&to=1296169200000&uuid=f3b50245-dc6e-d586-23d9-ffd291a58b72