Am Thu, 02 Aug 2012 11:24:54 +0200 schrieb Patrick Ohly: The calendar backend on the N9 only supports events. If I remember correctly, the same storage (KCal) is also used for tasks; not so sure about notes. Either way, it's primarily lack of developer time for the N9 port which prevents adding support. -- Best Regards, Patrick Ohly Patrick, Thank you very much for your reply. Indeed, there seems to be a need for more developer time. I understand, that this cannot be done by one person. And you do a lot of important coding for syncevolution. Do you believe that we could have a better chance if we find some interested developer at talk.maemo.org? I could ask there. I would be really glad if I could sync my N9 with all the funambol categories, especially with calendar, contacts and task as well as notes. Thanks again and have a nice weekend. Michael

Den 02. aug. 2012 11:24, skrev Patrick Ohly: Well, the Notes app does not allow you to choose a calendar or anything, so it might appear independent, but I've just taken a look to see where notes are stored. Unfortunately, Aegis prevented me from simply poking around the filesystem for PIM data, but after a bit of digging, it appears to me that notes actually do exist in mkcal, just in a different database with an apparently hardcoded name, or something. So if, on my N950, I run syncevolution to list the databases, I get something like mkcal = KCalExtended = MeeGo Calendar: Birthdays (/home/user/.calendar/db) Notes (/home/user/.calendar/db) Personal (/home/user/.calendar/db) <default> I guess it's just necessary to use different default databases for the events/tasks and the notes, then... Well, it's just a matter of modifying the KCalExtended backend, I suppose. I've been thinking about it. If people encourage/pester me enough, I might get around to it...

michael.kummer(a)gmx.net wrote on Fri, 3 Aug 2012 21:29:27 +0000: Same here! You have my maximum encouragement! I'm sure there are others out there, since this would close the most glaring gap in the N9's PIM usability. Justus

Ove, Patrick, I posted an request at talk.meamo.org to find more interested developers for the full PIM support of the N9 on the basis of syncevolution. You can find my post at http://talk.maemo.org/showthread.php?p=1247281#post1247281 . If I made any mistake please correct me. Please do not hesitate to contact me, I know that I'am responsible for the additional work for you. Thank you very much and please keep in touch. Michael -----Ursprüngliche Nachricht----- Von: Ove Kåven [mailto:ovek@arcticnet.no] Gesendet: Freitag, 3. August 2012 21:43 An: Patrick Ohly Cc: Michael Kummer; 'SyncEvolution' Betreff: Re: [SyncEvolution] Nokia N9 sync with task and notes support Den 02. aug. 2012 11:24, skrev Patrick Ohly: Well, the Notes app does not allow you to choose a calendar or anything, so it might appear independent, but I've just taken a look to see where notes are stored. Unfortunately, Aegis prevented me from simply poking around the filesystem for PIM data, but after a bit of digging, it appears to me that notes actually do exist in mkcal, just in a different database with an apparently hardcoded name, or something. So if, on my N950, I run syncevolution to list the databases, I get something like mkcal = KCalExtended = MeeGo Calendar: Birthdays (/home/user/.calendar/db) Notes (/home/user/.calendar/db) Personal (/home/user/.calendar/db) <default> I guess it's just necessary to use different default databases for the events/tasks and the notes, then... Well, it's just a matter of modifying the KCalExtended backend, I suppose. I've been thinking about it. If people encourage/pester me enough, I might get around to it...

Hi - If/when :-) these changes are implemented, I suppose the sync will work with all sorts of peers, not only with funambol and the likes? I sync my N9 with a syncevo-http-server running on my laptop. Cheers, Justus

On 06. aug. 2012 12:16, Patrick Ohly wrote: Maybe, but I was more concerned about getting periodical reminders if I work on it for a while, take a break, start doing something else of importance, and then something else, and so on, and forget all about it. Yes, I was going to anyway, of course. (Though it turned out that the new code and build system took a heck of a lot of work to manage to build on Maemo. A whole weekend spent just on that, patching .cpp files that don't build with g++ 4.2, patching configure to check for missing headers and other stuff, debugging linking failures, and whatnot.) And just in case you want to know, I'll change the Maemo 5 backend a bit as well, to no longer use TrackingSyncSource, but do custom change tracking like the KCalExtended backend does, asking the database only for stuff modified since the previous sync, instead of always asking the database for everything. (I was always concerned about it always unnecessarily loading the whole calendar into RAM in listAllItems, so it was about time I fixed this. My personal calendar is probably not going to get smaller anytime soon, since I rarely delete stuff from it, so...) There's one question I have about the change tracking in KCalExtended. In beginSync, you ask the database for all modifications done since the previous anchor. In endSync, you generate an anchor based on the current time when endSync is called. Wouldn't this risk losing any local changes done by the user *during* the sync, between beginSync and endSync? Would it be safer to generate the anchor in beginSync, so if the user does anything during the sync, the next sync can catch it? Yeah, I thought about something like that already. I didn't want to assume text/plain, as the N9 shows the creation time of the notes, which might be hard to sync with the simple plaintext format.

On 07. aug. 2012 10:15, Patrick Ohly wrote: Is that really a big issue? I figured we're selecting on last modification times here. By causality, assuming all clocks are mostly in sync, typical events transmitted in a synchronization would have a last-modification-time *before* the sync began, and that same last-modification time would be replicated on the other side, and thus also be before the beginSync time, and therefore, not picked up by the next beginSync. And if something slipped through the cracks anyway (clock drift?), hopefully some duplication check would catch it in most cases (at least if UIDs are used). Or maybe last modification times from the future could be "corrected" to avoid surprises. Anyway, to me, this seems less risky than maybe not noticing a change at all?

On 07. aug. 2012 18:10, Patrick Ohly wrote: As far as I can tell from the sources, at least KCalExtended's Tracker storage stores and selects on the stamp of the time stored in the received entry, not the local time of reception. It makes sense that the created-time and modified-time in the entry should be the time the *user* modified the data, otherwise sane conflict resolution becomes impossible. Thus, I'd say an implementation that does what you say is broken. (I did see one instance of possible brokenness in KCalExtended's SQLite backend regarding the created-time, but not in the Tracker backend, which is probably what's actually used here.) If the local storage time is of interest, it should probably be kept separate. And KCalExtended doesn't bother with keeping track of that, I think. Why would that be? If it cares, all the app needs to know is that the last-modification-time changed, it shouldn't need to be later than when the app last checked it. I don't think a non-syncing app should be able to assume too much about these things. Of course, this might make it inconvenient for the app to get a list of local changes, but most apps should have no need for that. When it's launched, it should simply load the entries it needs (probably based on start time), no matter when they were last modified, and then recheck its caches as needed (if the calendar storage isn't sophisticated enough to notify it or something).

On Wed, 2012-08-08 at 06:57 +0200, Ove Kåven wrote: Then it would be the responsibility of the app to update the LAST-MODIFIED - see my other mail about the semantic of that field. I kind of hoped that the storage does it automatically to avoid the need for remembering that semantic aspect when writing apps, but I see how it might be useful to let the app decide whether data really changed. On the other hand, if nothing changed, why does the app write the item into storage? That might be an option. But I suspect that users would forget to set that flag when configuring multiple peers because they don't know about it. Perhaps the GUI can ensure that it is set correctly... which might become a problem when it is not set initially and later gets set when adding that second peer. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On Wed, 2012-08-08 at 10:17 +0200, Ove Kåven wrote: The backend started as a skunk work project to work with and verify the calendar API in MeeGo at a time when SyncEvolution wasn't the official sync solution for MeeGo. Because the native sync solution was using time-based change tracking, it made sense to test that. I never got around to changing it later. It wasn't relevant until you picked it up for the N9 port of SyncEvolution. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On Wed, 2012-08-08 at 10:54 +0200, Ove Kåven wrote: Can KCalExtended efficiently report UID, RECURRENCE-ID and LAST-MODIFIED? In that case, moving to a TrackingSyncSource based approach with UID+RECURRENCE-ID as key and LAST-MODIFIED as revision string is the better solution because it handles (some) concurrent modifications by the user. If not, then sticking with the time-based change tracking might be better. Its performance advantage is probably large enough that users prefer that over better handling of concurrent changes. Hard to be sure. My own stance is usually "correctness always trumps performance". When you switch to TrackingSyncSource because it handles concurrent changes, then perhaps you might also want to start detecting conflicts during the sync (item was not modified when sync started, user modifies item, sync needs to store an updated of that item that it received during the sync) - this is something that I never got around to implement with EDS or WebDAV (*). The right reaction would be to reject the update and let the engine deal with the conflict in another sync session (or iteration, something that SyncEvolution 1.2.99.x can do when talking to itself). Totally untested, though - https://bugs.freedesktop.org/show_bug.cgi?id=52681 (*) EDS doesn't have a good API for it, one would have to emulate it with checking the current state again shortly before writing. WebDAV could do it via ETag checks in the PUT header. Unfortunately Google Calendar has so many bugs (and workarounds for them) that this atomic update would not work for it. Might work with other servers. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On Wed, 2012-08-08 at 02:42 +0200, Ove Kåven wrote: There's three different UTC time stamps in iCalendar 2.0: * CREATED - set when created in the local storage (once per storage) * LAST-MODIFIED - set each time the item is updated in the local storage (possibly multiple times per storage); the standard just says that it gets updated when "information associated with the calendar component was last revised in the calendar store" (which implies some kind of semantic check), but also compares it with "modification date and time for a file in the file system", which is closer to a "updated by storage on each write". * DTSTAMP - generated each time the calendar item is turned into iCalendar 2.0 text format (each time when reading from local storage) I was talking about change tracking based on LAST-MODIFIED and assumed that the storage enforces updating it on each write. CREATED can be used to find new items without comparing lists of items. The Synthesis engine keeps track of all items and needs a complete list of current items from the storage backend, so CREATED doesn't help much. You seem to assume that LAST-MODIFIED is taken from the incoming data and thus marks the time when the data was last modified somewhere. But that's not how it works. According to the spec, the incoming LAST-MODIFIED needs to be overwritten with the time when the modified item actually hits the local storage. The spec makes sense because relying on all peers having a synchronized time base is risky. It's hard enough to ensure locally. Depending on synchronized time cannot be avoided completely: LAST-MODIFIED values from different machines still get compared when resolving conflicts. That happens a lot less than change detection, which must work reliably each time a sync is done. There were bugs regarding these time stamps, but I reported them and as far as I remember, they were fixed to conform with the spec. Here's an example. Assume there is an item X with LAST-MODIFIED time x1 = 150 and CREATED = 0 in the local storage. At time a1 = 200, sync app A checks for "updated items" since time a0 = 100, the time of its last check. The app sees the item update and remembers a1 for the next check. At time b1 = 250, app B stores an updated version of X that it received from somewhere with x2 = 190. If the LAST-MODIFIED value is stored as it is, then another check by app A for items "modified >= a1 = 200" will not report the modified item, because x2 = 190 < 200. The app therefore fails to react to the change. If the LAST-MODIFIED value is updated to b1 = 250, app A will see the change, as it should be. There are ways to avoid this scenario (only sync app stores LAST-MODIFIED of the peer, all other apps update it when writing), but that's not how LAST-MODIFIED and the KCalExtended API are defined at the moment. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.

On Wed, 2012-08-08 at 10:48 +0200, Ove Kåven wrote: I wasn't there when iCalendar 2.0 was defined, so I can only speculate about the reasons for doing it. To me, it kind of makes sense, but I also agree that more explicit tracking of when a user changes something could be useful. There are a lot of things in PIM which can be surprising ;-) Unfortunately there are a lot more common problems with sync implementations which have that effect. Don't get me started. Agreed. However, it is a simplification that may be hard to enforce. It's mute anyway, because as you found, KCalExtended enforces the updating of LAST-MODIFIED, and thus even a single sync app cannot store a foreign LAST-MODIFIED even if it wanted to. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.