[vz-dev] API

[Steffen Vogel]

Am Mittwoch, den 21.07.2010, 10:27 +0200 schrieb Justin Otherguy:
> > What's in a name? :-)
> ...just hollow words... ;-)
> 
> Seriously: I'm German, so we'll need to sort out thing, right?! ;-)

Its could be quite hard to discuss something when the participants have
diffrent understandings of the meaning of these basic components.
So i think this definition is not nessesary but quite helpful for us.

> I'd say both are sensors. The meter type will integrate it's readings 
> while the 'other one' (I can't seem to find a suitable name) will make
> use of the instantaneous readings.

Exactly. That's my understanding to. "Sensors" are a kind of simplified
"Meters" because they directly send differentiated readings.
As you said by yourself, you have no suitable name vor them yet.
So I would suggest to call them as follow:

Channel		abstract superclass for all kinds of sensors, meters..
 \
  \ Meter	anything that counts a consumption (with pulses)
		and could be differentiated
		Examples: power, gas, water, radiation? 
 \
  \ Sensor	anything that return direct values
		Examples: temperature, humidity etc..
	
That's the same structure as we have it implemented as objects in our
backend.

> > We use sensor id's and device id's. The latter are the equivalent of 
> > your controller/uuid's. You should indeed not call them user id's as 
> > users might have multiple devices in use.
> well - the data type is called "uuid" (there is a psql data type with that name and there even is an RFC for the format: http://tools.ietf.org/html/rfc4122); we're not going to associate that to users -> so that's fine, too.
> for the moment we'll go with uuids for the sensors - every sensor will have it's own uuid.
> Steffen came up with a pretty brilliant concept of "virtual groups". On the server side these sensor ids can be associated to groups. These groups are being referenced by uuids again. So, to request a graph showing the data of 3 or 10 sensors you associate these sensors to a group and use the group's uuid to request all associated data.
> Every sensor can be in several groups at the same time. Say, we have 2 controllers and both controllers have 3 power sensors and one temperature sensor. We could have one group with all 6 power sensors; we could have a second group with the 2 temperature sensors and we could have a third group with all 8 sensors. Referring to the group you can always retrieve the desired data using a single uuid in the request.
> The concept would also allow for comparison of different consumers or ... or ...
> I'm quite impressed by the idea.
> 
> > So we associate sensors to 
> > devices and devices to users. We plan to offer an anonymous service with 
> > device/sensor auto-registration. If you would then like to 
> > visualize/share your data within Flukso, you would need to create an 
> > account and associate your devices with your account.
> maybe this is the same idea and only just now I understood ;-)

It sounds quite similiar to my concept of "virtual" groups.

To correct my last mail:
We've abandoned the user entity. On closer attention, I realized that
groups and users are quite the same. So I kicked them out of our
database scheme.

Groups only contain a uuid, a name and a short description to keep alive
our anonymity base concept.

> > The device id is not required for fetching data from the server. A 
> > sensor id suffices.
> -> perfect.
I agree.

> I had a loooong phone conversation with Steffen on monday (hello Steffen, how's the ear?).
Still a bit red ;)

> I keep loosing track, so mainly for myself:
> - we're (mainly) discussing the frontend/backend API here
> - the controller/backend API (currently XMLRPC for flukso, will be JSON/REST as well at some stage) will be a second step - not our focus at the moment

Let's get surprised by the controller->backend api ;)

I planned an logging interface for our backend which is capable to
"parse" diffrent APIs (or diffrent API versions, as stated by the
&version parameter).

So we will be able to also use meters compatible with 3rd party APIs as
the Google Power Meter API
(http://code.google.com/intl/de/apis/powermeter/).
Or directly log from the backend server with PHP-CLI calls to eliminate
the overhead of HTTP requests.

Our first ambition is to use the new Flukso JSON-REST API as our default
API.
Which should, in a next step, also be implemented on the ethersex
firmware (Pollin AVR NET-IO, etherrape, Hutschiene etc..)

> >> 2) I never liked the idea of using uuids to identify sensor as I always supposed this would mean that you would have to include all sensor uuids when you wanted to retrieve all sensor data for one controller; in that case I would have preferred to only transmit one uuid (the controller one); having uuids for the controller _and_ the sensors seem to give us the best of both worlds ->  I like that
> > 
> > Enter Avahi/Bonjour. We'll just use this mDNS service (thanks gonium!) 
> > on the Fluksometer to advertise the available sensors to computers, 
> > smartphones, Chumbies in the LAN. No need for writing down/copying all 
> > those sensor id's. We use the Bonjourfoxy FF plugin in our demo setup to 
> > auto-detect the sensors and present them to the JS code. The JS code can 
> > then form its JSON/REST call based on the information deliverd by Bonjour.
> > 
> > Just think about this: What's the difference between auto-detecting a 
> > printer or a sensor in your LAN?
> > 
> > With HTML5 you've got this local storage option. So your smartphone 
> > would only have to associate to your wireless LAN once to discover all 
> > the sensors and store them. It could then fetch data from the web server 
> > when on the road. All of this without having to copy/paste a single <uuid>.
> > 
> > Note: There still remains some fair amount of coding and testing to get 
> > this feature working properly. But I think it's an awesome one.
> yep, I remember you got me on this one a few weeks back, already. That sounds nice.

Really nice :) I really like these new HTML5 features. I can't wait for
it :p

> >> I'm not sure about the syntax; to me it seems best practice would be to have all parameters/values separated by slashes and not to use the "?" and "&" at all - am I wrong? Is this defined at all?
> > 
> > I made the same 'mistake' in my first API attempt. Use clean url's for 
> > people. Machines are much better off with the explicit queries. They are 
> > much more flexible since they are not fixed to a specific location 
> > within your URL. What will happen when you have e.g. two optional 
> > parameters in your API request? The slashes won't cut it.
> just to be sure, I got you right on this one:
> you prefer using only a few fixed parameters as part of the path and having the rest as part of the "query string" (i.e. separated by the "?"), right?

That also would be my approach. Every entity should be uniquely
addressable by a fixed path of the url.
Dynamic parameters as the time interval should be passed as a part of
the "query string".

> that takes me to my next question: 
> how do you take care of the url paths on the server side?
> If I'm not mistaken you actually do create dirs/files for each sensor on the flukso, right?
> Is that the way you do it on flukso.net as well?

? I don't think so ;) At the moment the flukso only sends requests. So
there are no "url paths" on the controller. I'm right?

On the server side, there is the erlang webmachine.
At first glance, it has its own dispatcher which maps the sensor path of
the url to the framework.
So there is no real path of a sensor on the filesystem of the controller
neither the backend server.

I realy like this pretty looking urls.
PHP doesn't offers such a dispatcher as webmachine does because its just
an programming language and not an complete REST framework.

Buts that's not a problem. We will use mod_rewrite, mod_alias or
multiviews to solve this issue.
Unfortunatly that needs some experience with the webserver
configuration. That's contrary to our "Volks" concept..
But im sure we will find a goog compromise here..

> A word to Mathias' concerns:
> no one intends to break anything ;-)
ack ;)
We have the API versioning for this ;)

> Seriously:
> I'd like to use the flukso frontend API without modification for volkszaehler; that's why I try to understand it. So far it seems to me that there's no need to change anything. We might find some additional parameters to extend the API - but that wouldn't break compatibility, then. 

In parts, i agree with you.

The Flukso frontend API is quite simple. Its only an API to retrieve
data from the flukso backend. The management of channels, users, etc is
done with drupal modules.. Im right?

We've a more common approch here. Our API is the only way to interact
with the backend. So its more complex but offers us the ability to use
other frontends as well. For example: chumby, fnordlicht-controllers,
plugin-integration with other sites and so on..

We make use of same technologies (HTTP requests, REST, JSON). So
compability should be no problem.

But im really missing some important parameters in the flukso api which
apparently just havn't been documented (start, end, resolution,
jsonp_callback...)
http://github.com/icarus75/flukso/blob/develop/server/api/flukso/src/flukso_resource.erl (line 25-31).

> To my understanding this is even clearer for the flukso/backend API: a change (xmlrpc -> json/rest) is coming up; the current API has to be supported for quite a long while after the new API is available. We're not interfering with that at all as we (volkszaehler) will not be implementing the xmlrpc interface but json/rest here as well from the start.

Regards Steffen

-- 
Steffen Vogel
Roonstraße 106
Köln

Cell: +49 (176) 96978528
Web: http://www.steffenvogel.de
Mail & MSN: info at steffenvogel.de
ICQ: 236033