10 a.m.
Hi Frederik!
I suppose you wanted to keep this on the list, so I'm copying the list
again.
On Tue, 2009-07-28 at 20:39 +0100, Frederik Elwert wrote:
Am Montag, den 27.07.2009, 09:13 +0200 schrieb Patrick Ohly:
> Agreed so far, except that I think the logic should be inside the D-Bus
> server.
Ok, I see. Then we should probably think about an API for automated
syncs. Just one question: Automated syncs require also some kind of
autostart. Once a user sets up, let’s say, a sync interval of 60
minutes, the syncs should run in the background right after booting the
computer (or at least after logging in).
The user has to be logged in. The user's session bus must exist for
reliable access to the EDS. There are workarounds for headless setups
(server), like running dbus-launch manually, but if we do that on a
machine where the user might log in later there's no guarantee that we
won't end up with two EDS instances.
To make it even more obvious, in some setups (laptop) the home directory
might encrypted and only become available when the user logs in.
Unfortunately, there is no "per user session" cron. For permanently
monitoring the local state I think we could get a D-Bus service started
as part of a user session - but I'm not entirely sure. If anyone knows
more, please chime in...
Does the D-Bus server get started on login? Or is it just started
the
first time a client requests it?
Currently it is only started on demand. This is good for startup time
(power on to usable desktop), an important feature in Moblin, so we have
to be careful not to slow down the startup.
> > * The information about local changes could be used
not only for
> > automated syncs. For example, Genesis could show the information
> > about changes to the data in the notification area, leaving the
> > start of the sync to the user.
>
> Yes, this might be useful, but we should design it so that the server
> does not needlessly monitor for changes. Monitoring would require that
> the server runs permanently, and monitoring itself could be costly - not
> so much for EDS, but for other backends (file, for example).
What about inotify?
Yes, that works. But I have heard mixed opinions about it, so I'm not
entirely sure how well it really works.
If the server doesn’t permanently monitor for changes, how would one
get
this information (regardless if for event-based automated syncs or for
user notification)?
There are no good alternatives, so the server really should do it as
good as it can.
> > The problem with that is that Ubuntu removed actions from
the
> > notifications.
>
> I suppose that applies to libnotify as well?
Ubuntu introduced a new implementation of the notification demon called
notify-osd. It conforms to the freedesktop spec, but doesn’t implement
everything the old demon did (which isn’t required by the spec). So one
can still use libnotify, but not everything will work with notify-osd,
especially actions.
I see.
But to come back to the D-Bus API: I am currently trying to port
Genesis
over to that API. Could somebody help me with understanding the API?
Jussi would be the prime candidate for that, but he is on vacation at
least this and next week. Let me try to help out.
What I think I understand so far is:
1. The client starts a Sync with StartSync(). (I didn’t manage to
call it with an empty array for a full sync, so I read the modes
per source from the config and pushed this to the method call.)
2. The clients registers a handler for the Progress signal. I think
I don’t need all the details here, so I simply wait for
success/failure notices? Could someone please tell me, what the
type values mean and what extra values are passed?
The values are a direct copy of the Synthesis progress events, using the
same integer codes and parameters as inside the Synthesis C/C++ SDK. The
definitions are in libsynthesis src/sysync_SDK/Sources/engine_defs.h.
Clearly this is not useful for a Python D-Bus client, but we weren't
sure about how this could be abstracted in D-Bus. We might have to
duplicate the whole set of progress events and document them as part of
our D-Bus API.
Regarding end of sync, sync_progress_cb() checks for PEV_SYNCEND.
3. Once the sync completed, I get the details by calling
GetSyncReports(). But what do I actually get? What do the values
mean?
This is intended to change. My proposal in another email in this thread
was to use a flat key/value hash with special encoding of the keys
instead.
Given that you would end up depending on an under-documented, almost
obsolete D-Bus API in 0.9, would it make more sense to take your
feedback and work on a better API for 1.0 and then release that as soon
as possible, without releasing Genesis for 0.9?
I read through src/dbus/interfaces/syncevo-full.xml, but not
everything
is documented in detail. And I really have a hard time reading the C
sources. ;-)
Yeah, I can imagine that it'll look nicer in Python. One more reason to
get Genesis ported ;-)
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
5:36 p.m.
New subject: [SyncEvolution] D-Bus API + server mode
Hi,
Am Mittwoch, den 29.07.2009, 11:00 +0200 schrieb Patrick Ohly:
I suppose you wanted to keep this on the list, so I'm copying the
list
again.
Oops, of course. Thank you!
The user has to be logged in. The user's session bus must exist
for
reliable access to the EDS. There are workarounds for headless setups
(server), like running dbus-launch manually, but if we do that on a
machine where the user might log in later there's no guarantee that we
won't end up with two EDS instances.
To make it even more obvious, in some setups (laptop) the home directory
might encrypted and only become available when the user logs in.
Unfortunately, there is no "per user session" cron. For permanently
monitoring the local state I think we could get a D-Bus service started
as part of a user session - but I'm not entirely sure. If anyone knows
more, please chime in...
Yes, I see, a running user session makes sense.
> Does the D-Bus server get started on login? Or is it just
started the
> first time a client requests it?
Currently it is only started on demand. This is good for startup time
(power on to usable desktop), an important feature in Moblin, so we have
to be careful not to slow down the startup.
That is a good point. But for automated syncs, I think some kind of
running process is required. Maybe one should at least start the D-Bus
server on login only if automated syncs are on, and on request
otherwise.
Are delayed processes possible in the Moblin session? I think older
versions of GNOME had the possibility to start some processes after the
more important desktop related thinks were loaded. But I don‘t know if
this is possible today (if not via a hackish "sleep 30;
syncevo-dbus-server" wrapper).
> What about inotify?
Yes, that works. But I have heard mixed opinions about it, so I'm not
entirely sure how well it really works.
I think Rhythmbox does use it to keep the library up to date, and
nautilus also makes use of it, as does Tracker. So I think it should
work, but I’m no expert on this field.
> But to come back to the D-Bus API: I am currently trying to port
Genesis
> over to that API. Could somebody help me with understanding the API?
Jussi would be the prime candidate for that, but he is on vacation at
least this and next week. Let me try to help out.
I’m on vacation next week, too. So maybe I’ll pause working on it until
I’m back.
> What I think I understand so far is:
>
> 1. The client starts a Sync with StartSync(). (I didn’t manage to
> call it with an empty array for a full sync, so I read the modes
> per source from the config and pushed this to the method call.)
> 2. The clients registers a handler for the Progress signal. I think
> I don’t need all the details here, so I simply wait for
> success/failure notices? Could someone please tell me, what the
> type values mean and what extra values are passed?
The values are a direct copy of the Synthesis progress events, using the
same integer codes and parameters as inside the Synthesis C/C++ SDK. The
definitions are in libsynthesis src/sysync_SDK/Sources/engine_defs.h.
Clearly this is not useful for a Python D-Bus client, but we weren't
sure about how this could be abstracted in D-Bus. We might have to
duplicate the whole set of progress events and document them as part of
our D-Bus API.
I think the only issue is documentation. Once you know where to find it,
one can well work with it. NetworkManager also sends it status as enum
values, which one has to copy to any python program as well.
Regarding end of sync, sync_progress_cb() checks for PEV_SYNCEND.
That should be sufficient for the moment. :-)
> 3. Once the sync completed, I get the details by calling
> GetSyncReports(). But what do I actually get? What do the values
> mean?
This is intended to change. My proposal in another email in this thread
was to use a flat key/value hash with special encoding of the keys
instead.
Ok, then I’d wait until this has finalised.
Given that you would end up depending on an under-documented, almost
obsolete D-Bus API in 0.9, would it make more sense to take your
feedback and work on a better API for 1.0 and then release that as soon
as possible, without releasing Genesis for 0.9?
Yes, that sounds reasonable. I think the syncevolution output did’t
change all that much, so maybe I’ll try updating the regexes for an
intermediary release that works with 0.9, but doesn’t use the D-Bus API.
Cheers,
Frederik
8:36 a.m.
New subject: [SyncEvolution] D-Bus API + server mode
[My ISP seems to be alive again, I'll try resending all the messages I
wrote yesterday. My apologies if you've already received them.]
Frederik Elwert wrote:
Hi,
Hello Frederik, sorry for taking my time...
>> Does the D-Bus server get started on login? Or is it just
started the
>> first time a client requests it?
> Currently it is only started on demand. This is good for startup time
> (power on to usable desktop), an important feature in Moblin, so we have
> to be careful not to slow down the startup.
That is a good point. But for automated syncs, I think some kind of
running process is required. Maybe one should at least start the D-Bus
server on login only if automated syncs are on, and on request
otherwise.
I don't like the idea of running the server continuously... It may be
unavoidable, but I'd like to try other methods first. The needs, AFAICT,
are:
* Do things on user demand (from UI or notifications etc.)
This is easy and works already.
* Do things when remote data changes
This we can't really do, so syncing at regular intervals sounds best.
There has to be a better way to do this than running all the time (if
the other requirements don't force us to do it anyway)
* Do things when local data changes
This isn't a must-have feature (syncing at regular intervals is
acceptable), but would be nice. I assume we'd solve this for EDS
sources first:
The EDS client could be syncevo-dbus-server, a tiny
syncevo-eds-watcher, or another app: at least in Moblin we have other
EDS users (the people panel) running already, maybe we could have the
people panel start syncs?
Are delayed processes possible in the Moblin session? I think older
versions of GNOME had the possibility to start some processes after the
more important desktop related thinks were loaded. But I don‘t know if
this is possible today (if not via a hackish "sleep 30;
syncevo-dbus-server" wrapper).
This is an implementation detail that is almost certainly already solved
(considering my netbook boots in ~10 seconds)...
>> What about inotify?
> Yes, that works. But I have heard mixed opinions about it, so I'm not
> entirely sure how well it really works.
I think Rhythmbox does use it to keep the library up to date, and
nautilus also makes use of it, as does Tracker. So I think it should
work, but I’m no expert on this field.
inotify does work nowadays but isn't EDS the correct way to handle EDS
sources?
>> But to come back to the D-Bus API: I am currently trying to
port Genesis
>> over to that API. Could somebody help me with understanding the API?
> Jussi would be the prime candidate for that, but he is on vacation at
> least this and next week. Let me try to help out.
I’m on vacation next week, too. So maybe I’ll pause working on it until
I’m back.
>> What I think I understand so far is:
>>
>> 1. The client starts a Sync with StartSync(). (I didn’t manage to
>> call it with an empty array for a full sync, so I read the modes
>> per source from the config and pushed this to the method call.)
If you can you file a bug on that, that would be great.
>> 2. The clients registers a handler for the Progress
signal. I think
>> I don’t need all the details here, so I simply wait for
>> success/failure notices? Could someone please tell me, what the
>> type values mean and what extra values are passed?
> The values are a direct copy of the Synthesis progress events, using the
> same integer codes and parameters as inside the Synthesis C/C++ SDK. The
> definitions are in libsynthesis src/sysync_SDK/Sources/engine_defs.h.
> Clearly this is not useful for a Python D-Bus client, but we weren't
> sure about how this could be abstracted in D-Bus. We might have to
> duplicate the whole set of progress events and document them as part of
> our D-Bus API.
I think the only issue is documentation. Once you know where to find it,
one can well work with it. NetworkManager also sends it status as enum
values, which one has to copy to any python program as well.
Using raw synthesis values isn't a very clean solution, but I don't see
us getting totally rid of it in the near future.
> Regarding end of sync, sync_progress_cb() checks for
PEV_SYNCEND.
That should be sufficient for the moment. :-)
24 (PEV_SYNCEND): sync for a source ended
16 (PEV_SESSIONEND): whole sync session ended
-1: EvolutionSyncClient.sync() returned
There are some problems with the current progress implementation wrt to
future requirements... Most importantly the events should be session
specific (possibly with a new session being a new dbus object), and some
of the progress logic should be in the server side.
So expect this to change as well.
>> 3. Once the sync completed, I get the details by
calling
>> GetSyncReports(). But what do I actually get? What do the values
>> mean?
> This is intended to change. My proposal in another email in this thread
> was to use a flat key/value hash with special encoding of the keys
> instead.
Ok, then I’d wait until this has finalised.
Agree with Patrick, plus the hashtable should probably be returned on a
SyncSessionEnd signal because most who subscribe to that signal want it
anyway.
-Jussi
5:44 p.m.
New subject: [SyncEvolution] D-Bus API + server mode
Hello,
Am Freitag, den 14.08.2009, 10:36 +0300 schrieb Jussi Kukkonen:
>>> Does the D-Bus server get started on login? Or is it
just started the
>>> first time a client requests it?
>> Currently it is only started on demand. This is good for startup time
>> (power on to usable desktop), an important feature in Moblin, so we have
>> to be careful not to slow down the startup.
>
> That is a good point. But for automated syncs, I think some kind of
> running process is required. Maybe one should at least start the D-Bus
> server on login only if automated syncs are on, and on request
> otherwise.
I don't like the idea of running the server continuously... It may be
unavoidable, but I'd like to try other methods first.
Yes, if it is possible to achieve the goals without having the server
always running, that would be nice.
The needs, AFAICT,
are:
* Do things on user demand (from UI or notifications etc.)
This is easy and works already.
* Do things when remote data changes
This we can't really do, so syncing at regular intervals sounds best.
There has to be a better way to do this than running all the time (if
the other requirements don't force us to do it anyway)
* Do things when local data changes
This isn't a must-have feature (syncing at regular intervals is
acceptable), but would be nice. I assume we'd solve this for EDS
sources first:
The EDS client could be syncevo-dbus-server, a tiny
syncevo-eds-watcher, or another app: at least in Moblin we have other
EDS users (the people panel) running already, maybe we could have the
people panel start syncs?
>>> What about inotify?
>> Yes, that works. But I have heard mixed opinions about it, so I'm not
>> entirely sure how well it really works.
>
> I think Rhythmbox does use it to keep the library up to date, and
> nautilus also makes use of it, as does Tracker. So I think it should
> work, but I’m no expert on this field.
inotify does work nowadays but isn't EDS the correct way to handle EDS
sources?
I’m not very familiar with EDS. But if it sends a signal on data
changes, this is of course the best option for EDS sources.
>>> But to come back to the D-Bus API: I am currently trying
to port Genesis
>>> over to that API. Could somebody help me with understanding the API?
>> Jussi would be the prime candidate for that, but he is on vacation at
>> least this and next week. Let me try to help out.
>
> I’m on vacation next week, too. So maybe I’ll pause working on it until
> I’m back.
>
>>> What I think I understand so far is:
>>>
>>> 1. The client starts a Sync with StartSync(). (I didn’t manage to
>>> call it with an empty array for a full sync, so I read the modes
>>> per source from the config and pushed this to the method call.)
If you can you file a bug on that, that would be great.
It wasn’t a bug, it was be being unfamiliar with python-dbus... :-( It
does work with an empty array when I specify the signature explicitly.
It fails with an empty list, because python-dbus can’t guess the
signature from that.
>>> 2. The clients registers a handler for the Progress signal. I think
>>> I don’t need all the details here, so I simply wait for
>>> success/failure notices? Could someone please tell me, what the
>>> type values mean and what extra values are passed?
>> The values are a direct copy of the Synthesis progress events, using the
>> same integer codes and parameters as inside the Synthesis C/C++ SDK. The
>> definitions are in libsynthesis src/sysync_SDK/Sources/engine_defs.h.
>> Clearly this is not useful for a Python D-Bus client, but we weren't
>> sure about how this could be abstracted in D-Bus. We might have to
>> duplicate the whole set of progress events and document them as part of
>> our D-Bus API.
>
> I think the only issue is documentation. Once you know where to find it,
> one can well work with it. NetworkManager also sends it status as enum
> values, which one has to copy to any python program as well.
Using raw synthesis values isn't a very clean solution, but I don't see
us getting totally rid of it in the near future.
>> Regarding end of sync, sync_progress_cb() checks for PEV_SYNCEND.
>
> That should be sufficient for the moment. :-)
24 (PEV_SYNCEND): sync for a source ended
16 (PEV_SESSIONEND): whole sync session ended
-1: EvolutionSyncClient.sync() returned
Ah, nice, thanks. That brings me to the idea of reporting the results
per source, not for the whole session. I think I’ll play around with
that.
There are some problems with the current progress implementation wrt
to
future requirements... Most importantly the events should be session
specific (possibly with a new session being a new dbus object), and some
of the progress logic should be in the server side.
Yes, I read the ideas about that. I think I’ll just wait for the API to
stabilize and then implement whatever you come up with :-)
So expect this to change as well.
>>> 3. Once the sync completed, I get the details by calling
>>> GetSyncReports(). But what do I actually get? What do the values
>>> mean?
>> This is intended to change. My proposal in another email in this thread
>> was to use a flat key/value hash with special encoding of the keys
>> instead.
>
> Ok, then I’d wait until this has finalised.
Agree with Patrick, plus the hashtable should probably be returned on a
SyncSessionEnd signal because most who subscribe to that signal want it
anyway.
So one could connect to the Progress signal, if one needs the detailed
information, or to the SyncSessionEnd signal if one is only interested
in the results? Sounds reasonable.
By the way, where do I get information about an error? Is that sent
through the ServerMessage signal? That currently lacks a description in
the API doc, so I didn’t know what it is good for.
Cheers,
Frederik
4658
days inactive
4674
days old
syncevolution@synevolution.org
3 comments
3 participants
participants (3)
-
Frederik Elwert -
Jussi Kukkonen -
Patrick Ohly