[SyncEvolution] internal SyncSource API redesign
by Patrick Ohly
Hello!
Bob's questions made it clear that some of the internal API changes
should be done rather sooner than later. There are several other
enhancement ideas which depend on similar changes:
#5047 ODBC sync source
#5049 field list sync source (the feature needed by Bob)
#5046 raw file sync source
#3472 SyncEvolution code cleanup (EvolutionSyncClient and
EvolutionSyncSource class names, namespaces)
Let me summarize my current thinking. In order to do something useful in
SyncEvolution, a backend
* must implement the part which integrates it into the framework
(meta information like name, Synthesis XML config fragments) =>
SyncEvolutionBackend, a subset of the current
EvolutionSyncSource
* must implement the Synthesis DB Interface, a varying (depending
on the backends capabilities) number of C-style function
callbacks => provide a mechanism how SyncEvolution backends can
bind these callbacks to methods in their own object, but without
prescribing a specific interface as EvolutionSyncSource does
right now
* if automatic testing is desired, it must implement a standard
interface for iterating over changes and another interface to
import/export data (in contrast to the current solution, were
iterating over items includes item data exchange - this was
inherited from the Funambol C++ client library and should be
changed also for other reasons)
* if backup/restore is desired, another interface for that must be
implemented
The interfaces must be pure virtual to allow multiple inheritance.
Default implementations of them provide the current functionality (for
example, EvolutionSyncSource: iterate over fixed lists of changes,
import/export items as blobs) and hook that up with the Synthesis DB
Interface. An implementor can put all of his code in one class which is
derived from the appropriate base classes.
The current EvolutionSyncSource and TrackingSyncSource would do that to
preserve the current, rather simple to implement API, and to avoid
rewriting existing backends. I would rename EvolutionSyncSource though
and move the remaining Evolution specific code into a common class in
src/backend/evolution.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
12 years, 10 months
[SyncEvolution] [Fwd: [Bug 5632] New: Memotoo and refresh-from-server]
by Patrick Ohly
Hello Thomas!
Not sure whether you have seen the news (okay, by now not so new
anymore). SyncEvolution switched the SyncML engine when going from 0.8
to 0.9 (final version released last week). Instead of the Funambol C++
client library, the Synthesis SyncML engine is now used.
During the 0.9 development cycle we didn't have time to test with
Memotoo. It wasn't an easy choice, considering that you supported
Evolution quite well last year, but we had to start somewhere. Several
users reported issues in combination with Memotoo. See attachement and
http://bugzilla.moblin.org/show_bug.cgi?id=5633
#5633 is understood and will be fixed in SyncEvolution, scheduled for
0.9.1. Do you have an idea why "refresh-from-server (#5632) fails to
send any items to the client?
Given that there is obviously a demand for it, we should add Memotoo to
the list of tested and supported servers:
http://bugzilla.moblin.org/show_bug.cgi?id=5635
Do you think you can help us once we start that testing? We'll need
fully enabled test accounts for several different developers/testers and
help when we have questions.
--
Best Regards
Patrick Ohly
Senior Software Engineer
Intel GmbH
Open Source Technology Center
Hermuelheimer Strasse 8a Phone: +49-2232-2090-30
50321 Bruehl Fax: +49-2232-2090-29
Germany
12 years, 10 months
[SyncEvolution] "Mobiltelefon" entry is empty after sync
by Dietmar Gottke
Hello,
first I have to thank very much the programmer of this very very useful tool. I
just switched to Deb Ubuntu 9.04 over from Windows and I'm very impressed what I
have now on my desktop. In the moment I think I'll never switch back.
But back to syncevolution which serves me a small bug which results in
disappearing of the "Mobiltelefon" entry in the Evolution contacts view.
Evolution is version 2.26.1, syncevolution is version 0.9-2 and I use Genesis
0.4.1 too but the bug shows up also without Genesis. My Syncserver is Memotoo
and the configurated vcard format is vcard 2.1 I also tried vcard 3, but with
the same bug. What happens: I set up a completely empty contacts list. Then I
put in a test contact with all fields filled with records. I started
"syncevolution memotoo" and found all records in my Evolution contacts view,
except the "Mobiltelefon" entry. The field in the listview and cardview is
empty. But when I open the contact in the contacteditor then the "Mobiltelefon"
entry is filled correctly. Only switching back to the listview changes nothing,
but when I change the cellphone number in the contacteditor then the cellphone
number shows up afterwards in the listview. Doing a new sync will again make the
"Mobiltelefon" entry empty.
Cause I'm a real novice concerning Linux and all this pretty good stuff I hope
that this list can help me solve the problem.
Best regards and again all thumbs up for this very good syncing tool.
Godie
12 years, 10 months
[SyncEvolution] Problem with timezone/libecal?
by Rui Lopes
Hi all!
I've been trying syncevolution with mixed feelings: sync addressbook
works flawlessly and sync calendar fails always. I attach the
syncevolution log and strace of the execution. I believe it is something
related to getting the timezone, but haven't been able to figure it out.
Does someone have a clue?
Cheers,
/rp
12 years, 10 months
Re: [SyncEvolution] SyncML Tests 2009-07-27: head-evolution-testing
by Patrick Ohly
Hello!
Analysis of latest nightly run below.
On Tue, 2009-07-28 at 04:47 +0100, Ohly, Patrick wrote:
> http://runtests.syncevolution.org/2009-07-27-22-00/head-evolution-testing
>
> libsynthesis successful
> syncevolution successful
> compile successful
> dist skipped: disabled in configuration
> evolution successful
This worked on all of the other platforms, too. The problem with not
finding the default memo in earlier runs was a local install problem:
I'm pretty sure I had started Evolution once in each platform and opened
addressbook, calendar, tasks, memos, but despite that, the system memo
lists didn't exist on all platforms. It seems it is necessary to create
an entry in Evolution before it creates the database.
> scheduleworld: failed
Fails because of the new testSlowSyncSemantic test (#4703). I have added
it to the list of know failures and will document it in a
test/README.scheduleworld if it doesn't get resolved.
> memotoo skipped: disabled in configuration
> egroupware skipped: disabled in configuration
> synthesis: failed
Only one failure:
ClientTest.cpp:3851:Assertion
Test name: Client::Sync::text_vcard21::Suspend::testUserSuspendClientRemove
equality assertion failed
- Expected: 0
- Actual : 500
This was because of a timeout, according to the log at the start of the
sync session. Because the "timestampall" option is not set in the
Synthesis log, it is hard to tell how long that session ran. I have
enabled that option now.
There's also not enough debug information to identify where it got stuck
- communication with EDS perhaps?
> funambol:
One failure:
ClientTest.cpp:3851:Assertion
Test name: Client::Sync::itodo20::testCopy
equality assertion failed
- Expected: 0
- Actual : 20043
[ERROR] http://my.funambol.com/sync via libsoup: Bad Gateway
Message resending might have avoided this failure...
> zyb: failed
Still no word from ZYB. David, is anyone looking at the anchor problem?
http://bugzilla.moblin.org/show_bug.cgi?id=2424#c28
> google successful
Yeah! Congwu's script for cleaning the server data works.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
12 years, 10 months
Re: [SyncEvolution] SyncML-Klient KDE
by Patrick Ohly
Hello Sascha!
Let's switch to English as you suggested, with the mailing lists on CC.
I've summarized the main points below. For those joining the discussion,
I mailed Sascha after I read about his KDE SyncML work and introduced
SyncEvolution+libsynthesis. Some relevant links:
http://syncevolution.org/
http://moblin.org/documentation/syncevolution/direct-synchronization-aka-...
As you said, libsynthesis wasn't available when you started your work on
the KDE SyncML client/server. It's a real pity that we (Synthesis.ch for
libsynthesis, Intel for the revampled SyncEvolution) couldn't get the
open sourcing done sooner. On the other hand, this way you and the other
SyncML projects which seem to happen this summer had a chance to
experiment with the technology. That might make it easier to understand
what libsynthesis and SyncEvolution are about.
Regarding "libfunambol is good enough for a client": yes, after all I
used it for over three years. But it has its limitations. In addition to
the obvious lack of WBXML support (required for Google), you'll also
find that the "pass KDE data to SyncML server directly" approach is too
limited once you start doing interoperability testing with SyncML
servers. I already mentioned vCalendar 1.0 support (mobical.net) and
vCard extensions (X-SPOUSE); this is trivial to solve with libsynthesis
and harder with libfunambol.
But I'd like to go one step further: instead of switching to
libsynthesis, I suggest that you consider integrating your work with
SyncEvolution. libsynthesis is really just the raw engine. The app
framework that you know from libfunambol is provided by SyncEvolution
for libsynthesis. The "syncevolution" command line tool should work with
a KDE backend right away.
We are currently in the process of revising the D-Bus API for GUIs. The
sync engine is already decoupled from the GTK GUI via D-Bus, but this
was done in a rather ad-hoc fashion and we think this can be improved.
It would be great to get your feedback, but as you are going on vacation
(lucky you!), we'll have to give it a shot and then revise it when you
come back end of September - assuming that you are interested, of
course.
--
Bye, Patrick Ohly
--
Patrick.Ohly(a)gmx.de
http://www.estamos.de/
12 years, 10 months
[SyncEvolution] status 2009-08-18
by Patrick Ohly
Hello!
I didn't send around a summary last week, so this one here covers two
weeks. The big topic of course were the SyncEvolution 0.9 release and
the on-going API discussions, covering backends (SyncSource), peers
(OBEX) and GUIs. Expect more information about the SyncEvolution 1.0
schedule soon.
Development one week ago:
* Yongsheng: almost completed mobical.net interoperability testing
and fixing, pending review; improvements to nightly testing
output
* Congwu: simplified client-test setup (new config now mirrors the
corresponding server template, merged); started looking at EDS
API extensions and handling outgoing OBEX connections
* Patrick: internal backend API rewrite; several release-related
changes (support packaging on Fedora 11, analysis of bugs
reported by latest QA testing) and planning for >0.9
Development last week:
* Yongsheng: worked on ‘autoenddate’ issue in mobical and finally
verified it as a bug by Lucas. Refined patches in
‘origin/mobical’ branch according to Patrick’s comments.
Mobical.net interoperability test should be done. Waiting for
merge from synthesis and ‘origin/mobical’ branch (merged today).
* Congwu: evolution-data-server atomic modification
implementation: addressbook part is finished, passed unit test.
* Patrick: finished internal SyncSource API rewrite, now needs
testing.
* Jussi: fixed several GUI issues for 0.9 (startup slowness #5021,
error callbacks #4919, link tooltip formatting #5017, startup
notification for Moblin chooser #4752)
New issues:
* performance improvements necessary for key/value store
in .other.ini file (#5377, not assigned)
* GUI still needlessly affected by this performance problem when
switching between configs (#5388, assigned to Jussi)
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
12 years, 10 months
[SyncEvolution] compiling 0.9 on F-11 and rawhide
by Peter Robinson
Hi All,
I'm having a few issues compiling the syncevolution 0.9 release on
either F-11 or rawhide.
I'm not sure if its a missing dep that isn't picked up by ./configure
or something else. The last beta compiled OK so I suspect its
something quite minor that I've missed. I get the following undefined
reference errors below.
Cheers,
Peter
g++ -DHAVE_CONFIG_H -I. -I.. -DHAVE_CONFIG_H -Idbus/interfaces
-I/usr/include/dbus-1.0 -I/usr/lib64/dbus-1.0/include
-I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include
-I/usr/include/gnome-keyring-1 -I/usr/include/glib-2.0
-I/usr/lib64/glib-2.0/include
-I.//home/perobinson/rpmbuild/BUILD/syncevolution-0.9/src/build-synthesis
-I./dbus -I./core -I./backends/addressbook -I./backends/evolution
-I./backends/file -I./backends/sqlite -I./gtk-ui -I./../test -I..
-I/home/perobinson/rpmbuild/BUILD/syncevolution-0.9/src/build-synthesis/src
-DORBIT2=1 -pthread -I/usr/include/evolution-data-server-2.26
-I/usr/include/libxml2 -I/usr/include/glib-2.0
-I/usr/lib64/glib-2.0/include -I/usr/include/gconf/2
-I/usr/include/libsoup-2.4 -I/usr/include/libbonobo-2.0
-I/usr/include/orbit-2.0 -I/usr/include/dbus-1.0
-I/usr/lib64/dbus-1.0/include -I/usr/include/bonobo-activation-2.0
-DORBIT2=1 -pthread -I/usr/include/evolution-data-server-2.26
-I/usr/include/libbonobo-2.0 -I/usr/include/glib-2.0
-I/usr/lib64/glib-2.0/include -I/usr/include/orbit-2.0
-I/usr/include/bonobo-activation-2.0 -I/usr/include/libxml2
-I/usr/include/gconf/2 -I/usr/include/libsoup-2.4
-I/usr/include/dbus-1.0 -I/usr/lib64/dbus-1.0/include -DORBIT2=1
-pthread -I/usr/include/evolution-data-server-2.26
-I/usr/include/libbonobo-2.0 -I/usr/include/glib-2.0
-I/usr/lib64/glib-2.0/include -I/usr/include/orbit-2.0
-I/usr/include/bonobo-activation-2.0 -I/usr/include/libxml2
-I/usr/include/gconf/2 -I/usr/include/libsoup-2.4
-I/usr/include/dbus-1.0 -I/usr/lib64/dbus-1.0/include
-I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include
-I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include
-I/home/perobinson/rpmbuild/BUILD/syncevolution-0.9/src/build-synthesis/src
-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -MT
syncevo_dbus_server-test.o -MD -MP -MF
.deps/syncevo_dbus_server-test.Tpo -c -o syncevo_dbus_server-test.o
`test -f '../test/test.cpp' || echo './'`../test/test.cpp
mv -f .deps/syncevo_dbus_server-syncevo-dbus-server.Tpo
.deps/syncevo_dbus_server-syncevo-dbus-server.Po
/bin/sh ../libtool --tag=CXX --mode=link g++
-I/home/perobinson/rpmbuild/BUILD/syncevolution-0.9/src/build-synthesis/src
-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic
-uSyncEvolution_Module_Version -Wl,--export-dynamic -o syncevolution
syncevolution-syncevolution.o syncevolution-test.o
core/libsyncevolution.la -lglib-2.0
mkdir .libs
DIE_RPATH_DIE="/usr/lib64/syncevolution:$DIE_RPATH_DIE" g++
-I/home/perobinson/rpmbuild/BUILD/syncevolution-0.9/src/build-synthesis/src
-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic
-uSyncEvolution_Module_Version -Wl,--export-dynamic -o
.libs/syncevolution syncevolution-syncevolution.o syncevolution-test.o
core/.libs/libsyncevolution.so -lglib-2.0
/usr/bin/ld: warning: libsynthesis.so.0, needed by
core/.libs/libsyncevolution.so, not found (try using -rpath or
-rpath-link)
core/.libs/libsyncevolution.so: undefined reference to
`sysync::SySyncDebugPuts(void*, char const*, int, char const*, int,
char const*, char const*)(a)VER_1.0'
core/.libs/libsyncevolution.so: undefined reference to
`SySync_ConnectEngine(a)VER_1.0'
core/.libs/libsyncevolution.so: undefined reference to
`SySync_DisconnectEngine(a)VER_1.0'
collect2: ld returned 1 exit status
make[4]: *** [syncevolution] Error 1
make[4]: *** Waiting for unfinished jobs....
mv -f .deps/syncevo_dbus_server-test.Tpo .deps/syncevo_dbus_server-test.Po
mv -f .deps/client_test-ClientTest.Tpo .deps/client_test-ClientTest.Po
make[4]: Leaving directory
`/home/perobinson/rpmbuild/BUILD/syncevolution-0.9/src'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory
`/home/perobinson/rpmbuild/BUILD/syncevolution-0.9/src'
make[2]: *** [all] Error 2
make[2]: Leaving directory
`/home/perobinson/rpmbuild/BUILD/syncevolution-0.9/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/perobinson/rpmbuild/BUILD/syncevolution-0.9'
make: *** [all] Error 2
12 years, 10 months
[SyncEvolution] binfile: beware when moving home directory between machines
by Patrick Ohly
Hello!
I just had a very weird problem: SyncEvolution consistently sent all
items to the server in a normal sync, although they were not recognized
as updated or new. The DB CreateContext method was called twice for the
same datastore. Consider myself totally confused at that point.
To cut a long story (several hours) short, the underlying reason was
that I had copied my home directory from an older 32 bit Linux
installation to a newer installation in 64 bit. The binfile
implementation does not seem to cope with that well. I'm not sure how it
caused the effects above, but after removing the entire .synthesis
directory (the place where SyncEvolution keeps the Synthesis binfiles)
syncs started to work normally again.
It is not guaranteed that migrating from 32 bit to 64 bit by simply
copying files works. However, my expectation was (and still is) that
software should detect this, warn the user and the abort. Thoughts?
Of course this happened while testing the final 0.9 release binaries,
which is why I was close to despair and they are not announced yet ;-)
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
12 years, 10 months
Re: [SyncEvolution] D-Bus API + server mode
by Patrick Ohly
Hi Frederik!
I suppose you wanted to keep this on the list, so I'm copying the list
again.
On Tue, 2009-07-28 at 20:39 +0100, Frederik Elwert wrote:
> Am Montag, den 27.07.2009, 09:13 +0200 schrieb Patrick Ohly:
> > Agreed so far, except that I think the logic should be inside the D-Bus
> > server.
>
> Ok, I see. Then we should probably think about an API for automated
> syncs. Just one question: Automated syncs require also some kind of
> autostart. Once a user sets up, let’s say, a sync interval of 60
> minutes, the syncs should run in the background right after booting the
> computer (or at least after logging in).
The user has to be logged in. The user's session bus must exist for
reliable access to the EDS. There are workarounds for headless setups
(server), like running dbus-launch manually, but if we do that on a
machine where the user might log in later there's no guarantee that we
won't end up with two EDS instances.
To make it even more obvious, in some setups (laptop) the home directory
might encrypted and only become available when the user logs in.
Unfortunately, there is no "per user session" cron. For permanently
monitoring the local state I think we could get a D-Bus service started
as part of a user session - but I'm not entirely sure. If anyone knows
more, please chime in...
> Does the D-Bus server get started on login? Or is it just started the
> first time a client requests it?
Currently it is only started on demand. This is good for startup time
(power on to usable desktop), an important feature in Moblin, so we have
to be careful not to slow down the startup.
> > > * The information about local changes could be used not only for
> > > automated syncs. For example, Genesis could show the information
> > > about changes to the data in the notification area, leaving the
> > > start of the sync to the user.
> >
> > Yes, this might be useful, but we should design it so that the server
> > does not needlessly monitor for changes. Monitoring would require that
> > the server runs permanently, and monitoring itself could be costly - not
> > so much for EDS, but for other backends (file, for example).
>
> What about inotify?
Yes, that works. But I have heard mixed opinions about it, so I'm not
entirely sure how well it really works.
> If the server doesn’t permanently monitor for changes, how would one get
> this information (regardless if for event-based automated syncs or for
> user notification)?
There are no good alternatives, so the server really should do it as
good as it can.
> > > The problem with that is that Ubuntu removed actions from the
> > > notifications.
> >
> > I suppose that applies to libnotify as well?
>
> Ubuntu introduced a new implementation of the notification demon called
> notify-osd. It conforms to the freedesktop spec, but doesn’t implement
> everything the old demon did (which isn’t required by the spec). So one
> can still use libnotify, but not everything will work with notify-osd,
> especially actions.
I see.
> But to come back to the D-Bus API: I am currently trying to port Genesis
> over to that API. Could somebody help me with understanding the API?
Jussi would be the prime candidate for that, but he is on vacation at
least this and next week. Let me try to help out.
> What I think I understand so far is:
>
> 1. The client starts a Sync with StartSync(). (I didn’t manage to
> call it with an empty array for a full sync, so I read the modes
> per source from the config and pushed this to the method call.)
> 2. The clients registers a handler for the Progress signal. I think
> I don’t need all the details here, so I simply wait for
> success/failure notices? Could someone please tell me, what the
> type values mean and what extra values are passed?
The values are a direct copy of the Synthesis progress events, using the
same integer codes and parameters as inside the Synthesis C/C++ SDK. The
definitions are in libsynthesis src/sysync_SDK/Sources/engine_defs.h.
Clearly this is not useful for a Python D-Bus client, but we weren't
sure about how this could be abstracted in D-Bus. We might have to
duplicate the whole set of progress events and document them as part of
our D-Bus API.
Regarding end of sync, sync_progress_cb() checks for PEV_SYNCEND.
> 3. Once the sync completed, I get the details by calling
> GetSyncReports(). But what do I actually get? What do the values
> mean?
This is intended to change. My proposal in another email in this thread
was to use a flat key/value hash with special encoding of the keys
instead.
Given that you would end up depending on an under-documented, almost
obsolete D-Bus API in 0.9, would it make more sense to take your
feedback and work on a better API for 1.0 and then release that as soon
as possible, without releasing Genesis for 0.9?
> I read through src/dbus/interfaces/syncevo-full.xml, but not everything
> is documented in detail. And I really have a hard time reading the C
> sources. ;-)
Yeah, I can imagine that it'll look nicer in Python. One more reason to
get Genesis ported ;-)
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
12 years, 10 months
