[vz-dev] API

[Bart Van Der Meerssche]

Hello Klaus,

On 07/18/2010 09:11 PM, Justin Otherguy wrote:
> Hello Bart,
>
> thanks for sharing your experience with us!
>
> I'd like to sort out some general questions that will allow us to get to the interesting parts.
>
> Am 13.07.2010 um 17:33 schrieb Bart Van Der Meerssche:
>
>> icarus75 at nimbus:~$ curl -k -v -X GET -H "Accept: application/json" -H "X-Version: 1.0" -H "X-Token: d8a8ab8893ea73f768b66b45234b5c3a" "https://api.flukso.net/sensor/c1411c6b4f9910bbbab09f145f8533b9?interval=hour&unit=watt"
>
> so, basically we have two parts:
> - header info: mime-type + protocol version 1) + access token
> - request paramaters:
>    - identification of the sensor and controller:
>      - Flukso:
>        - "sensor" (references the ucid, correct?)
>        - there seems to be no keyword to reference the controller / flukso meter, yet - correct?
>      - VZ:
>        - "ucid" identifies the sensor ->  I suggest switching to "sensor" here; like flukso and more readable than "ucid"
>        - "uuid" identifies the controller ->  I suggest switching to "meter" here; this is more readable than "uuid" and more precise than "user id"

What's in a name? :-)

We use sensor id's and device id's. The latter are the equivalent of 
your controller/uuid's. You should indeed not call them user id's as 
users might have multiple devices in use. So we associate sensors to 
devices and devices to users. We plan to offer an anonymous service with 
device/sensor auto-registration. If you would then like to 
visualize/share your data within Flukso, you would need to create an 
account and associate your devices with your account.

The device id is not required for fetching data from the server. A 
sensor id suffices.

>    - reading/count: we've discussed that one already; should not be the number of pulses since the last request, but the total number of requests; Steffen asked a very good question here: what happens after a reboot of the controller or any other event that would cause the counter to restart at 0: I don't think it's a big issue to detect this and take care of it while reading the values from the db. I can only think of one tricky situation: the interval between two data points in the diagram is greater than the interval between two reboots/counter resets; I don't think that should be a real world issue, should it?
>
>>> And how do we want to save this value (EEPROM vs. RAM?).
> speaking of AVR net io: we'll have to find out - I don't know yet; EEPROM would be the place, I suppose (RAM: doesn't survive a reboot; SD: in most cases not available);
> Flukso: shouldn't be a big deal :-)

Flukso uses a power-loss detection circuit which uses the AVR's analog 
comparator. The latter triggers an interrupt the moment the rail voltage 
drops below a certain threshold. The interrupt then writes all meter 
values to EEPROM. We use an elco and a diode to store energy and prevent 
leakage to the router board in the event of a power loss. By being 
parsimonious with the AVR's energy consumption, we can still write +16 
bytes to EEPROM the moment a power loss occurs.

>
>    - "unit":
>      I just understood the different approaches we had; so far we (VZ) are transmitting pulse counts that are calculated into consumptions later on; that's why we need to know the meter's resolution on the server side; I can't see any disadvantage of switching to the flukso approach here: transmitting the consumption together with the unit. Can someone else? Otherwise: let's get rid of ours!

Euh, actually we use a quite similar approach for logging. We just 
normalize the pulses on the sensor board. So for analog (clamps) or 
digital (pulses) inputs, a pulse = 1Wh. From this point onwards we work 
with the meter value.

For the local JSON/REST interface on the Fluksometer, we calculate the 
power every second in the AVR and update the JSON array accordingly: pop 
the oldest value and push the newest one.

The 'unit' I was referring to was for fetching data from the server. You 
can have it in watt, kWh/year, eur/year, etc.

>    - "interval":
>      we don't have that at all so far; what's the idea behind transmitting that as well?

This is again for fetching data from the server. It makes for a 
straightforward call: when iterval=day, then the Flukso server will 
provide you with a JSON array containing a day's worth of data in 15 min 
slots, so 96 data points. If you make the same call again an hour later, 
you don't need to specify the time. It's again the data between 
'now-24h' and 'now'. Pretty much the way RRDtool allows you to fetch 
data from an RRD.

>
> Bart: do you have more parameters planned so far? If not, I suggest we try to agree on the ones present so far and then move on to the additional ones we find necessary, alright?

I'm thinking of having a sort of 'detail=y' option which would give you 
all parameters related to the sensor, like:
- type (eg electricity)
- function (eg main, solar, ...)
- last metervalue
- etc.

>
> remarks:
> 1) in our api reference [1] we currently have the version information as one of the regular request parameters; I don't mind switching to that variant; we'll be using http headers anyway

Actually, we allow for the version and token to be sent either as a 
header field, or as part of the query parameters.

> 2) I never liked the idea of using uuids to identify sensor as I always supposed this would mean that you would have to include all sensor uuids when you wanted to retrieve all sensor data for one controller; in that case I would have preferred to only transmit one uuid (the controller one); having uuids for the controller _and_ the sensors seem to give us the best of both worlds ->  I like that

Enter Avahi/Bonjour. We'll just use this mDNS service (thanks gonium!) 
on the Fluksometer to advertise the available sensors to computers, 
smartphones, Chumbies in the LAN. No need for writing down/copying all 
those sensor id's. We use the Bonjourfoxy FF plugin in our demo setup to 
auto-detect the sensors and present them to the JS code. The JS code can 
then form its JSON/REST call based on the information deliverd by Bonjour.

Just think about this: What's the difference between auto-detecting a 
printer or a sensor in your LAN?

With HTML5 you've got this local storage option. So your smartphone 
would only have to associate to your wireless LAN once to discover all 
the sensors and store them. It could then fetch data from the web server 
when on the road. All of this without having to copy/paste a single <uuid>.

Note: There still remains some fair amount of coding and testing to get 
this feature working properly. But I think it's an awesome one.

>
> I'm not sure about the syntax; to me it seems best practice would be to have all parameters/values separated by slashes and not to use the "?" and "&" at all - am I wrong? Is this defined at all?

I made the same 'mistake' in my first API attempt. Use clean url's for 
people. Machines are much better off with the explicit queries. They are 
much more flexible since they are not fixed to a specific location 
within your URL. What will happen when you have e.g. two optional 
parameters in your API request? The slashes won't cut it.

>> We have already implemented a JSONP query option, which allows you to fetch this data from JS within any web page, process the array and visualize it in your JS graphing library of choice.
> yummi - nice :-)
>
>> On 13 Jul 2010, at 12:21, Steffen Vogel wrote:
>>
>>> At the moment, i'm soldering my 10 fnordlicht-minis ;) I'm curious to
>>> see them in action with volkszaehler.org/ethersex :)
> oh, BTW, our friends in K'lautern (Fraunhofer ITWM, mysmartgrid) are making progress, too:
> http://developer.mysmartgrid.de/doku.php?do=show&id=start ->  Code running on the Chumby ->  Code running on the Chumby
>
> not bad at all :-)
>
> BTW: did I mention the barcamp Mathias is planning for October 29th/30th in K'lautern?
> http://developer.mysmartgrid.de/doku.php

open:elektro.camp(<<2010>>).

Quiz: Guess which language the barcamp title refers to. First one to 
guess correctly will get ein Helles from me during the camp. Barcamp 
organizers cannot participate in this contest.

>
> I'll be there; it would be nice if more vz members could arrange to go there as well
>
>
> Regards, J.
>
> [1] http://de.volkszaehler.wikia.com/wiki/Entwicklung/API/Referenz
>
> _______________________________________________
> volkszaehler-dev mailing list
> volkszaehler-dev at lists.volkszaehler.org
> https://volkszaehler.org/mailman/listinfo/volkszaehler-dev
>

Cheers,
Bart.