[SyncEvolution] schedule SyncEvolution 0.9.1 + 1.0
by Patrick Ohly
Hello everyone!
Here's some background information about when we might release updates
for SyncEvolution. This isn't set in stone; think of it more like the
grand master plan that we strive for... and then change after impact
with the real world ;-)
Scheduling is driven by "release when ready" and "we want regular
updates at a deterministic point in time", so let's look at the feature
list first:
The high-level goals for 0.9.1 are:
* bug fixing, including string changes that we had to postpone in
0.9
* include message resend (merged, found problems in testing)
* revised backend API (merged, no regressions so far)
* mobical.net interoperability
* Memotoo interoperability (because of user demand)?
* stretch goal: revised D-Bus API
Message resending was merged, but then a thorough testing against
Funambol and ScheduleWorld showed that there is one particular case
where the sync hangs when a certain message reply gets lost. This looks
like a server bug. With the Synthesis server it works.
The high-level goals for 1.0 are:
* implement revised D-Bus API and architecture (support Genesis
GUI; syncevolution as D-Bus client)
* performance and reliability improvements (EDS API)
* minimal support in sync-ui for SyncML server mode
* stretch goal: implement the intended SyncML client design in
sync-ui (current one is lacking recovery features, progress spinner,
etc.)
* stretch goal: OBEX support for SyncML client and server
* SyncML server mode (peers to be decided)
In particular the last item has a huge tail of detailed testing work
associated with each peer. For 1.0, our goal has to be to get the
infrastructure in place and prove that it works with a few phones. We
probably could work this forever. At the same time we want to get
another stable release out the door so that we can ship the other
improvements.
The biggest unknown factor for 1.0 are the necessary changes in
libsynthesis. Lukas has reassured me that he has made good progress
towards those, but they have a business to run and therefore might have
more important work to deal with first. September 4th in the schedule
below is way too early - more realistic proposals welcome ;-}
With that in mind, here are some milestones that could get us towards 1.0,
with names attached to the major tasks and until when they need to be done
to stay on track:
* Jussi, Patrick, August 26: decide about D-Bus server-side
technology - done, let's use libdbus + gdbus + custom C++ binding
* Patrick, August 28: core server engine and complete C++ binding -
much of that is done.
* Jussi, August 28: D-Bus GUI API redefined, with feedback by others -
still a bit of work needed
* Patrick, Yongsheng, September 11: new D-Bus API implemented (must
work incrementally, so that Jussi can adapt the GUI in parallel)
* Synthesis, September 4 (?): SyncML server API discussed and usable
* Jussi + Patrick + Nick, September 9+10: face-to-face in London,
discuss design issues
* Congwu, September 4: EDS API enhancements + usage of those in
SyncEvolution backends - patches ready for review, Ross will do
that for EDS, Patrick for SyncEvolution
* Congwu, September 11: outgoing OBEX connections with transport
inside SyncEvolution
* Patrick, September 18: Synthesis server feature integrated into
SyncEvolution
* all, end of September: first alpha
* all, end of October: final release
We have to be aware that we have to leave enough gaps in the schedule
for unexpected problems or requests and the "optional" open issues (see
attachment). I consider some of these important that I don't want to
push them out forever:
* code cleanup
* revive SQLite demo backend and perhaps Mac OS X and Maemo
support (although I really hope that the later two will be
covered by external developers)
* split data transformation part out of libsynthesis
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
12 years, 7 months
[SyncEvolution] SyncML + OBEX
by Patrick Ohly
Hello Forrest!
I've just had a chat with Marcel about OBEX/SyncML integration. I'd like
to continue the discussion by email. Note that the public SyncEvolution
list is on CC, please group reply, I'll approve posts.
What we (= SyncEvolution) have to implement is the SyncML OBEX binding:
http://www.openmobilealliance.org/Technical/release_program/docs/copyrigh...
As explained in the design document for the future SyncML server, we
want to establish a central D-Bus service as the single component in the
system which runs SyncML sessions:
http://moblin.org/documentation/syncevolution/direct-synchronization-aka-...
This service has to interact with peers over OBEX in two ways:
1. actively contact them (SyncML server = OBEX client)
2. wait for connection requests (SyncML client = OBEX server)
These are the common use cases. The spec allows a SyncML client (phone)
to contact the server (desktop), but typically the UI of a phone only
allows triggering a sync via HTTP, not via OBEX. Instead, the software
on the desktop is expected to contact the phone.
The first point could be implemented via libopenobex. libsyncml could be
used as example for this. We might even copy the code, the license is
compatible. However, we don't have SyncML server support yet. Whether
that is really simpler than using obexd remains to be seen, which is why
I'd like to look at the second point first.
The second point is best solved via an obexd plugin. This is were help
would be very welcome. Forrest, do you think you could point us in the
right direction? My hope that this is trivial for someone who knows the
obexd source code. We could use help:
1. addding a new SyncML plugin to obexd
2. adding the SyncML service description (OBEX binding document,
5.2.1)
3. writing stubs for Connect/Disconnect/Put/Get/Abort operations:
* CONNECT: use a dummy connection ID
* PUT: discard incoming data
* GET: send dummy data
4. how the data could be exchanged with a D-Bus service as part of
the obexd event loop
These stubs then need to be extended so that they actually talk to the
sync D-Bus service via its (still to be defined) API.
The OBEXD binding allows splitting of one SyncML message into smaller
PUT/GET chunks. I'm undecided whether the D-Bus API should also support
this, because the maximum size of a SyncML message could already be
chosen so that we don't overload the system.
Speaking of D-Bus communication: Marcel pointed out that this additional
traffic and the amount of data can be problematic and mentioned that
D-Bus 0.4 will support fd exchange to hook up two peers directly. We
should keep that in mind, but start with data exchange via plain D-Bus
before tuning it.
We are not the only ones who transmit the data over D-Bus anyway, EDS
also does it. So our use of D-Bus is not a qualitative difference
(completely different way of using it), only a quantitative one (twice
the traffic).
To complete the OBEX support and server-initiated syncs, we (= the
SyncEvolution team and/or Synthesis) also needs to add encoding and
decoding of the SyncML Notification message (OMA DS 1.2.2, chapter 12).
This is non-trivial because the URIs of the device must be known in
advance. To get started we should make this configurable and later
decide how to simplify that for users (global device database?).
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
12 years, 8 months
[SyncEvolution] next generation syncevo-dbus-server
by Patrick Ohly
Hello!
Let me do a little "thinking aloud" here... please chime in, otherwise
I'll start to feel very sad and lonely. Cheers from the sideline are
also welcome ;-)
One D-Bus API which has been discussed fairly well is the Connect()
method for incoming connection requests. It starts a new session under
the hood, which is something that the GUI should be able to notice and
visualize.
As mentioned before, only one session at a time is currently practical
(avoids issues with concurrent access to data, internal
EvolutionSyncClient API limitations). Should we allow concurrent
operations, like listing past sessions, config modifications or data
restore?
I don't think so. Some of these operations might be safe to execute in
parallel to a running session, but this can be tricky to determine.
Clearly modifying the configuration of an active session isn't possible,
but even just reading old session logs suffers from a race condition:
the number and content of old sessions can change as the current session
expires old logs.
I therefore suggest that we model the new API and implementation around
"locking" of a server configuration. Each lock request creates a
org.syncevolution.sync interface instance on D-Bus. That instance
remains active as long as it has clients or a sync is running. Clients
can disconnect and reconnect to such an active instance. In that sense
they are not exclusive.
Communication between the instance and clients is 1:n. The instance
sends out information to all clients as signals. Some of these signals
might provide progress or status information. Others might request
feedback, like for example a confirmation question ("Do you really want
to erase all local data?").
The server needs to know how long it should wait for a reply before
proceeding with some default operation. This depends on the kind of
clients which are connected: with no client connected, it should wait
for a certain amount of time, because a client might show up. With just
the command line client connected, it should proceed immediately because
that client is not going to ask questions (at the moment...). With the
sync-ui connected, it should wait for the sync-ui forever.
Clearly the clients need to declare how they are to be treated. The
server needs to arbitrate this decision process, because clients have no
knowledge of each other. For example, the command line client must not
simply ack all questions immediately, because that would prevent the
sync-ui from doing any useful user interaction.
Locking a config can be rejected immediately or queued, depending on
what the client wants. An incoming connection request would be queued.
Locking requests can be removed by clients, time out or become obsolete
when the client disconnects unexpectedly.
Lock requests and running syncs are really the same object, just in
different phases, so they should have the same interface and same D-Bus
path. I suggest org/syncevolution/sync/<ID number>, with <ID number>
being unique (start time plus running counter). Phase changes of these
instances need to be signaled so that a client can notice when some
other client has started a sync.
This sounds rather abstract at the moment. I could move forward with
specifying the API, but first I'd like to give a chance for feedback.
There are also some other open questions:
* Is it okay to transfer sync reports as string->string
dictionaries? Example below.
* The current GUI API exposes some parts of the configuration, but
not all of it. I'd like to switch to a "read key/value dict",
"update specific key", "write dict back" model. The value here
could be either a single value or a pair of comment+value. The
later is more complex and I'm not sure whether the (unlocalized)
comments would be useful for any client at all.
Here's the current status.ini style format for sync reports. Using a
dict removes the need to parse the file. New information can be added
without breaking the API.
start = 1250597114, 2009-08-18 14:08:14 +0200
end = 1250597121, 2009-08-18 14:08:21 +0200
status = 200
source-addressbook-mode = two-way
source-addressbook-first = false
source-addressbook-resume = false
source-addressbook-status = 0
source-addressbook-backup-before = 623
source-addressbook-backup-after = 623
source-addressbook-stat-local-updated-total = 2
source-addressbook-stat-local-any-received = 562
Most of this is almost self-explanatory, expect for the (ab)use of
stat-local-any-sent/received for number of bytes. A full documentation
must become part of the API anyway.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
12 years, 8 months
[SyncEvolution] why not a forum
by Thierry Chen
why do you still use mailing lists for that kind of service? Is there one advantage against forums ??? I see only disavantages. First you can't make search on pevious messages to see if your problem has been discussed sooner and need to disturb anybody even if your case has been treated.
Or could someone tell me how to use mailing lists correctly ?
Thierry Chen
Gérant POSTGATE
09 54 95 13 95
12 years, 8 months
[SyncEvolution] inscription mailing-list
by Thierry Chen
I subscribed to the mailing list. Obtained message that says I'll receive confirmation by mail. No confirmation 3 hours later !
Why do u use theese archaïc and stupid mailing-lists ?!
Thierry Chen
12 years, 8 months
[SyncEvolution] sync evolution/horde
by Thierry Chen
Hi
Your page http://syncevolution.org/documentation/compatibility says:
"... Copy one of the prepackaged configuration directories (like etc/scheduleworld_1) to ~/.sync4j/evolution/horde as a starting point. It doesn't matter much which template directory you use. They're not very different, and there's not much to configure. You need to modify (or create) the following items. In ~/.sync4j/evolution/horde/spds/syncml/config.txt: "
I installed syncevolution 0.8 then 0.9 on Debian 5.0 and found no any etc/schedulworld or other ~/.sync4j/evolution/horde to start server configuration with.
Have read all documentation and found nothing about how to define a new horde server in syncevolution.
Thierry Chen
12 years, 8 months
[SyncEvolution] atomic updates of local data (EDS)
by Patrick Ohly
Hello!
Congwu has made great progress in implementing the Evolution Data Server
API changes for safe atomic updates of items in the server (see my posts
on the Evolution hackers list).
The API adds a revision check to the EDS backend that (if requested by
the client) aborts an update or item delete if the check fails because
some other EDS client modified the item in the meantime.
That raises the question: how do we handle a conflict in SyncEvolution
once it was detected?
For "server delete/client update" conflicts I think we should refuse the
delete and in the next sync, send the item as updated.
For "server update/client update" conflicts it becomes more difficult.
If we send a temporary error code to the server and then an update in
the next sync ourselves, will the server understand? Should we do the
merging ourselves, even on clients?
The Synthesis engine already supports that (receive item from server,
parse server item into field list, retrieve item from DB, parse DB item
into field list, merge, store back into DB). In this scenario we could
redo the merging. But I am not sure right now whether we use this, nor
what the API could look like for redoing the merge. We should use this
feature (it allows preserving local properties), but we don't have tests
for it.
I'd like to see tests for atomic updates be added first:
* In Client::Source, open two sources and simulate the two kinds
of conflicts. The source which runs into a conflict should
return a suitable error (TBD) instead of overwriting more recent
data.
* In Client::Sync, devise an injection mechanism (possibly hooked
into the source progress events) which injects conflicting
changes when the sync is already running.
These tests should fail for simple SyncSource's which don't support
atomic updates (yet) and later pass once we updated the Evolution
backends; thus they have to be enabled separately in the
ClientTestConfig with "off" being the default.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
12 years, 8 months
[SyncEvolution] sync-ui and Genesis compatibility
by Frederik Elwert
Hello,
since SyncEvolution 0.9 ships with the GTK ui, I just tried it out. But
after startup, it tells me that no sync service is configured. When I go
into the configuration dialogue, it lists my account data for
ScheduleWorld, except for the password.
This surely is not top priority, but I think it would be nice if one
could run sync-ui and Genesis next to each other. So services set up in
Genesis should work in sync-ui and the other way around. So could anyone
help me find out why sync-ui doesn’t recognise the services I set up
with Genesis? Then I could adjust Genesis’ configuration mechanisms to
be compatible with sync-ui.
Cheers,
Frederik
12 years, 8 months
[SyncEvolution] D-Bus bindings: C++ or C?
by Patrick Ohly
Hello!
I'm having second thoughts about the decision that Jussi and I made back
in January about using dbus-glib as the main interface to D-Bus in the
syncevo-dbus-server. The main rationale at that time was that it is
readily available and a known quantity.
However, after looking at some of the code that Jussi had to write to
make our C++ classes work as part of a D-Bus server I'm wondering
whether this advantage of dbus-glib is really worth it. Much of it is
manually written glue code between dbus-glib/GObject and C++.
I bet it would be a lot easier to solve this problem with dbus-c++.
Except that it isn't part of most (all?) distros and much of the
development takes place outside of the main project, with no recent
official release...
Another problem is lack or insufficient support for asynchronous method
calls. Some branches of dbus-c++ might have it. This would be a
deal-breaker, because we need that feature.
So anyway, here's my question to Jussi: how much work is it really to
implement the D-Bus API?
Question to packagers: would you mind if we bundled a version of
dbus-c++ with our source until some official release makes it into the
distros?
Any other alternatives?
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
12 years, 8 months
