10:49 p.m.
Hi all,
Quick update on the N85 sync -- or generic S60 sync if we're lucky:
Patrick made some changes on monday and I can currently do some sort of
syncs on all sources. Patricks work is in pohly branch ATM.
So status by source type:
1. Memo seems to work fully.
2. Addressbook sync itself seems to work but there is a major problem:
the vcards from the phone contain very little data: no phone number or
email for example. This is bug #8868. PC Suite snooping may be required
before fixing.
3. calendar+todo syncs fine otherwise, but the client will return 500 to
any "Replace" commands, so server side modifications do not work at the
moment. As far as I can see this only happens to Replace and only on
calendar+todo. Here is an example (timestamps removed for readability):
* Started processing Command 'Status' (incoming MsgID=3, CmdID=3)
* WARNING: RECEIVED NON-OK STATUS 500 for command 'Replace' (outgoing
MsgID=2, CmdID=5)
* - TargetRef (remoteID) = '145'
* Found matching command 'Replace' for Status
* dsConfirmItemOp completed, syncop=replace, localID='', remoteID='145',
FAILURE, errorstatus=500
* Status: General error 500 (original op was wants-replace) -> marking
item for resend in next session
* localID='w64241AY4EIva653k52Yn2-rid' marked for resending by setting
mapflag_resend (AND mark for eventual resume!), flags now=0x20
* Status: processed, removed command 'Replace' from status wait queue
* Status: command 'Replace' has handled status and allows to be deleted
* Deleted command 'Replace' (outgoing MsgID=2, CmdID=5)
* Deleted command 'Status' (incoming MsgID=3, CmdID=3)
I thought maybe there is something wrong with the outgoing syncml (a web
search indicated the Nokia client responds with 500 if e.g.
<NumberOfItems> is in the wrong place in <Sync> but I can't see anything
wrong here:
<Sync>
<CmdID>4</CmdID>
<Target><LocURI>./C:Calendar</LocURI></Target>
<Source><LocURI>./Calendar</LocURI></Source>
<NumberOfChanges>1</NumberOfChanges>
<Replace>
<CmdID>5</CmdID>
<Meta><Type
xmlns='syncml:metinf'>text/x-vcalendar</Type></Meta>
<Item>
<Target><LocURI>145</LocURI></Target>
<Data>
<![CDATA[BEGIN:VCALENDAR
VERSION:1.0
TZ:+02:00
DAYLIGHT;ENCODING=QUOTED-PRINTABLE:TRUE;+03;20100328T010000Z;20101=
031T010000Z;;freeassociation.sourceforge.net;Tzfile;Europe;Helsink=
i
BEGIN:VEVENT
LAST-MODIFIED:20100215T191439Z
UID:w64241AY4EIva653k52Yn2
SEQUENCE:1
CLASS:PRIVATE
PRIORITY:2
SUMMARY:Test
DESCRIPTION:Test . . . edit on server
DTSTART:20100217T060000Z
DTEND:20100217T060000Z
END:VEVENT
END:VCALENDAR
]]>
</Data>
</Item>
</Replace>
</Sync>
Should I upload the whole logs somewhere or do we need to take a
wireshark look at what PC Suite sends the phone in a similar situation?
Maybe it is expecting something not specified in the OMA spec?
-Jussi
7:17 a.m.
On Mo, 2010-02-15 at 22:49 +0000, Jussi Kukkonen wrote:
3. calendar+todo syncs fine otherwise, but the client will return 500
to
any "Replace" commands, so server side modifications do not work at the
moment. As far as I can see this only happens to Replace and only on
calendar+todo.
Does adding of events to the phone work?
Should I upload the whole logs somewhere or do we need to take a
wireshark look at what PC Suite sends the phone in a similar situation?
Maybe it is expecting something not specified in the OMA spec?
I'm afraid at some point we'll have to figure out how to do this kind of
tracing.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
7:28 a.m.
Patrick Ohly wrote:
On Mo, 2010-02-15 at 22:49 +0000, Jussi Kukkonen wrote:
> 3. calendar+todo syncs fine otherwise, but the client will return 500 to
> any "Replace" commands, so server side modifications do not work at the
> moment. As far as I can see this only happens to Replace and only on
> calendar+todo.
Does adding of events to the phone work?
Yes it does.
> Should I upload the whole logs somewhere or do we need to take a
> wireshark look at what PC Suite sends the phone in a similar situation?
> Maybe it is expecting something not specified in the OMA spec?
I'm afraid at some point we'll have to figure out how to do this kind of
tracing.
I'm not particularly looking forward to it either...
-Jussi
7:55 a.m.
On Mo, 2010-02-15 at 22:49 +0000, Jussi Kukkonen wrote:
<Data>
<
8:52 a.m.
Hi Jussi,
On Feb 15, 2010, at 23:49 , Jussi Kukkonen wrote:
2. Addressbook sync itself seems to work but there is a major
problem: the vcards from the phone contain very little data: no phone number or email for
example. This is bug #8868. PC Suite snooping may be required before fixing.
This might be due to the device not understanding the CTCap correctly. We had that with
other Nokias some time ago. I added the (debug aid only)
<showctcapproperties>no</showctcapproperties> config option (place it within
<server>) then, which globally suppresses sending any CTCap. With this, the Nokias
did send all fields, along with their own devInf, which then can be compared with the
devInf we would send.
If I remember correctly, Nokia did (does?) expect a strange way of representing the
possible TYPE enum values (as <property> rather than <valenum>, depending on
SyncML version. See comment in mimedirprofile.cpp:4637:
// For some older 1.1 devices (in particular Nokia
7610), enum values of default params
// in pre-MIME-DIR must be shown as param NAMES (not enums).
// But newer 1.2 Nokias like E90 need proper TYPE param with valEnums (when
run in 1.2 mode. E90 is fine with 7610 style for 1.1)
// So: normally (fEnumDefaultPropParams==undefined==-1), we show 7610 style
for 1.1 and E90 style for 1.2.
// <enumdefaultpropparams> and ENUMDEFAULTPROPPARAMS() can be used to
control this behaviour when needed
So, depending on the time the device queries the server devinf (before or after it gets
the device's devInf and can possibly apply remote rules), the
<enumdefaultpropparams> remote rule option might help here.
Best Regards,
Lukas Zeller (luz(a)synthesis.ch)
-
Synthesis AG, SyncML Solutions & Sustainable Software Concepts
info(a)synthesis.ch, http://www.synthesis.ch
10:29 a.m.
Lukas Zeller wrote:
On Feb 15, 2010, at 23:49 , Jussi Kukkonen wrote:
> 2. Addressbook sync itself seems to work but there is a major
> problem: the vcards from the phone contain very little data: no
> phone number or email for example. This is bug #8868. PC Suite
> snooping may be required before fixing.
This might be due to the device not understanding the CTCap
correctly. We had that with other Nokias some time ago. I added the
(debug aid only) <showctcapproperties>no</showctcapproperties> config
option (place it within <server>) then, which globally suppresses
sending any CTCap. With this, the Nokias did send all fields, along
with their own devInf, which then can be compared with the devInf we
would send.
Hi Lukas,
Quick test shows your hunch is correct. At least phone numbers show up
nicely after adding
"<showctcapproperties>no</showctcapproperties>".
I'll file a bug on this as I don't know how to really solve this (I just
hard-coded).
If I remember correctly, Nokia did (does?) expect a strange way of
representing the possible TYPE enum values (as <property> rather than
<valenum>, depending on SyncML version. See comment in
mimedirprofile.cpp:4637:
I'm not sure how how to test this... What is an example problem I should
be looking for?
-Jussi
11:16 a.m.
Hi Jussi,
On Feb 16, 2010, at 11:29 , Jussi Kukkonen wrote:
Quick test shows your hunch is correct. At least phone numbers show
up nicely after adding
"<showctcapproperties>no</showctcapproperties>".
I'll file a bug on this as I don't know how to really solve this (I just
hard-coded).
> If I remember correctly, Nokia did (does?) expect a strange way of
> representing the possible TYPE enum values (as <property> rather than
> <valenum>, depending on SyncML version. See comment in
> mimedirprofile.cpp:4637:
I'm not sure how how to test this... What is an example problem I should be looking
for?
I'd try to compare the CTCap that libsynthesis is sending to the Nokia (with
showctcapproperties=yes) with the devInf the Nokia sends back. That will probably show
what's different. If you have traces (best would be 1:1 WBXML message dumps) of these
two cases, please just send them to me so I can have a look. As there are already some
mechanisms to make Nokia devices happy, it might need only a little tweak in the engine to
catch the N85 et. al. as well.
Best Regards,
Lukas Zeller (luz(a)synthesis.ch)
-
Synthesis AG, SyncML Solutions & Sustainable Software Concepts
info(a)synthesis.ch, http://www.synthesis.ch
2:06 p.m.
Lukas Zeller wrote:
>> If I remember correctly, Nokia did (does?) expect a strange
way
>> of representing the possible TYPE enum values (as <property>
>> rather than <valenum>, depending on SyncML version. See comment
>> in mimedirprofile.cpp:4637:
> I'm not sure how how to test this... What is an example problem I
> should be looking for?
I'd try to compare the CTCap that libsynthesis is sending to the
Nokia (with showctcapproperties=yes) with the devInf the Nokia sends
back. That will probably show what's different. If you have traces
(best would be 1:1 WBXML message dumps) of these two cases, please
just send them to me so I can have a look. As there are already some
mechanisms to make Nokia devices happy, it might need only a little
tweak in the engine to catch the N85 et. al. as well.
Great, thanks.
I've never needed wbxml dumps so far so I don't really know how to do
that but here are the <CTCap>-portions of the xml files.
-Jussi
11:35 a.m.
Jussi Kukkonen wrote:
Hi all,
Quick update on the N85 sync -- or generic S60 sync if we're lucky:
Patrick made some changes on monday and I can currently do some sort of
syncs on all sources. Patricks work is in pohly branch ATM.
So status by source type:
[clip]
3. calendar+todo syncs fine otherwise, but the client will return 500
to
any "Replace" commands, so server side modifications do not work at the
moment. As far as I can see this only happens to Replace and only on
calendar+todo.
This turned out to be another symptom of the <CTCap> problem mentioned
in this same thread. The <showctcapproperties>-trick makes Replaces work
as well.
Lukas, do you have a vision on how and where this should be fixed properly?
- Jussi
12:45 p.m.
On Mi, 2010-02-17 at 11:35 +0000, Jussi Kukkonen wrote:
Jussi Kukkonen wrote:
> 3. calendar+todo syncs fine otherwise, but the client will return 500 to
> any "Replace" commands, so server side modifications do not work at the
> moment. As far as I can see this only happens to Replace and only on
> calendar+todo.
This turned out to be another symptom of the <CTCap> problem mentioned
in this same thread. The <showctcapproperties>-trick makes Replaces work
as well.
At least it has something to do with the capabilities. I don't think we
have any X- properties in the vcalendar profile; we'd have to check more
carefully which property it is that the device stumbles over.
Lukas, do you have a vision on how and where this should be fixed
properly?
One solution would be to never send any capabilities for the device,
based on a remote rule. This is the sledgehammer approach.
The other solution would be to disable certain properties and/or
parameters, again based on a remote rule. This is a lot more work and
would require changes to the profile mechanism/configuration: first, we
need the ability to activate multiple rules, not just one as supported
now. This is something we already discussed and found useful elsewhere.
Second, we need that for parameters (X-EVOLUTION-SLOT). Right now, only
properties can be controlled like that.
I'm a bit afraid about taking the second path. Lukas, comments?
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
5:29 p.m.
Hello Jussi,
thanks for the update.
On Feb 17, 2010, at 12:35 , Jussi Kukkonen wrote:
Jussi Kukkonen wrote:
[clip]
> 3. calendar+todo syncs fine otherwise, but the client will return 500 to any
"Replace" commands, so server side modifications do not work at the moment. As
far as I can see this only happens to Replace and only on calendar+todo.
This turned out to be another symptom of the <CTCap> problem mentioned in this same
thread. The <showctcapproperties>-trick makes Replaces work as well.
Lukas, do you have a vision on how and where this should be fixed properly?
Not really, yet.
The big problem is that the device (Nokia phone) usually asks for the server devInf BEFORE
it sends it's own devInf, but the server can't detect what device it is talking to
before receiving devInf. So the the changes we make to the devInf to make Nokias happy
would be global for all devices. While hiding a problematic parameter like
X-EVOLUTION-UI-SLOT with show="no" is acceptable, <showctcapproperties>=no
is certainly not; we don't want to globally disable all field negotiating capabilities
just because one class of devices.
For local sync which is triggered by the server, SyncEvolution probably could tell the
engine beforehand which remoteRule to activate, such that outgoing devInf could be made
conformant. However, for a HTTP server accepting incoming connections, that's not
possible.
Best Regards,
Lukas Zeller (luz(a)synthesis.ch)
-
Synthesis AG, SyncML Solutions & Sustainable Software Concepts
info(a)synthesis.ch, http://www.synthesis.ch
4:53 p.m.
On Mi, 2010-02-17 at 17:29 +0000, Lukas Zeller wrote:
The big problem is that the device (Nokia phone) usually asks for
the
server devInf BEFORE it sends it's own devInf, but the server can't
detect what device it is talking to before receiving devInf.
Asking for DevInf before sending its own would probably confuse even
more devices, right?
Speaking of confused, I'm a bit confused about CTCap inside DevInf right
now. In the SyncEvolution client <-> SyncEvolution server scenario, the
client sends DevInf including CTCap to the server in its first message.
The server sends DevInf in its reply, but without any CTCap. I don't
have <showctcapproperties> in my config.
I would expect to see the server's CTCap in the session. Any idea why it
is not sent?
So the the changes we make to the devInf to make Nokias happy would
be global for all devices. While hiding a problematic parameter like
X-EVOLUTION-UI-SLOT with show="no" is acceptable,
<showctcapproperties>=no is certainly not; we don't want to globally
disable all field negotiating capabilities just because one class of
devices.
I agree.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
10:52 p.m.
Hello Patrick,
On Feb 18, 2010, at 17:53 , Patrick Ohly wrote:
On Mi, 2010-02-17 at 17:29 +0000, Lukas Zeller wrote:
> The big problem is that the device (Nokia phone) usually asks for the
> server devInf BEFORE it sends it's own devInf, but the server can't
> detect what device it is talking to before receiving devInf.
Asking for DevInf before sending its own would probably confuse even
more devices, right?
Probably. Without checking details, IMHO the standard would allow to delay the
<Results> for a <Get>, but that would imply that the init phase would be
extended to have two message exchanges. Given all the troubles we've seen with
<final/> statemachine handling, it's likely to cause problems.
Speaking of confused, I'm a bit confused about CTCap inside
DevInf right
now. In the SyncEvolution client <-> SyncEvolution server scenario, the
client sends DevInf including CTCap to the server in its first message.
The server sends DevInf in its reply, but without any CTCap. I don't
have <showctcapproperties> in my config.
I would expect to see the server's CTCap in the session. Any idea why it
is not sent?
The only reason I can think of right now would be a too small message size, which the
devInf exceeds. If that happens, CTCap is left out to make the devInf sendable, because
<results> can't be split across messages.
Lukas Zeller (luz(a)synthesis.ch)
-
Synthesis AG, SyncML Solutions & Sustainable Software Concepts
info(a)synthesis.ch, http://www.synthesis.ch
12:47 p.m.
On Do, 2010-02-18 at 22:52 +0000, Lukas Zeller wrote:
Hello Patrick,
On Feb 18, 2010, at 17:53 , Patrick Ohly wrote:
> Speaking of confused, I'm a bit confused about CTCap inside DevInf right
> now. In the SyncEvolution client <-> SyncEvolution server scenario, the
> client sends DevInf including CTCap to the server in its first message.
> The server sends DevInf in its reply, but without any CTCap. I don't
> have <showctcapproperties> in my config.
>
> I would expect to see the server's CTCap in the session. Any idea why it
> is not sent?
The only reason I can think of right now would be a too small message
size, which the devInf exceeds. If that happens, CTCap is left out to
make the devInf sendable, because <results> can't be split across
messages.
That was exactly the reason. The WBXML message turned out to be only
slightly smaller than 64KB. SyncEvolution had a very small maximum
message size of 20000 bytes - I don't remember why we made it so small.
I've increased it to 150000 bytes. What are the maximum message sizes
used by the Synthesis clients?
The CTCap turned out to be that large because it includes all
datastores, even the sub-datastore that are only configured because they
are part of the superdatastore. Is there a possibility to prevent
inclusion of these stores in the DevInf?
I understand that a general-purpose HTTP server leaves the choice of the
store to the client, but in our case (phone over Bluetooth) we expect
the device to use exactly the stores that we have prepared for it.
Coming back to the N85, the DevInf sent to the device includes three
datastore entries:
<DataStore>
<SourceRef>./Calendar</SourceRef>
<DisplayName>calendar+todo@todo</DisplayName>
...
<DataStore>
<SourceRef>./Calendar</SourceRef>
<DisplayName>calendar+todo@calendar</DisplayName>
...
<DataStore>
<SourceRef>./Calendar</SourceRef>
<DisplayName>calendar+todo</DisplayName>
While stepping through the code I noticed that there is code which uses
the URI from the client's Alert - is that perhaps why the two
sub-datastores and the superdatastore use the same <SourceRef>?
At least it wastes some bandwidth. If we are unlucky, in confuses the
client.
I can imagine that including stores which have not been alerted yet
might be useful, in case that the client sends an <Alert> for one store
first and later for another one, but sending all stores with the same
URI as above doesn't make sense to me.
In the SyncEvolution as client case this doesn't happen because the
server's DevInf is created before the client's first Alert.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
2:33 p.m.
On Fr, 2010-02-19 at 12:47 +0000, Patrick Ohly wrote:
I can imagine that including stores which have not been alerted yet
might be useful, in case that the client sends an <Alert> for one store
first and later for another one, but sending all stores with the same
URI as above doesn't make sense to me.
Attached is a patch which might do the job of suppressing sub-datastores
in the DevInf. Untested, because I cannot reproduce the situation here.
Jussi, can you apply it and check whether the "right thing" (= one
datastore for "calendar+todo") is sent?
Does it have any effect on the remaining problem with calendar updates
on the device?
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
3:03 p.m.
Hello,
maybe I misread the patch (on the way home, iPhone...) but for the
server subdatastores should not be hidden. For the client, I agree, in
a session using the superdatastore the subdatastores should not be
included in the defInf.
luz(a)synthesis.ch
On 19.02.2010, at 15:33, Patrick Ohly <patrick.ohly(a)intel.com> wrote:
On Fr, 2010-02-19 at 12:47 +0000, Patrick Ohly wrote:
> I can imagine that including stores which have not been alerted yet
> might be useful, in case that the client sends an <Alert> for one
> store
> first and later for another one, but sending all stores with the same
> URI as above doesn't make sense to me.
Attached is a patch which might do the job of suppressing sub-
datastores
in the DevInf. Untested, because I cannot reproduce the situation
here.
Jussi, can you apply it and check whether the "right thing" (= one
datastore for "calendar+todo") is sent?
Does it have any effect on the remaining problem with calendar updates
on the device?
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
<no-substores.patch>
3:30 p.m.
On Fr, 2010-02-19 at 15:03 +0000, Lukas Zeller wrote:
maybe I misread the patch (on the way home, iPhone...) but for the
server subdatastores should not be hidden.
Perhaps not in all cases, but I can only think of one case where it
makes sense to show it: when the client has not yet sent its Alert and
we don't know which source it is going to use.
The patch hides them only when the client has already alerted us and
thus the sub-datastores have been identified as such - at least that's
the idea.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
11:12 p.m.
Hello Patrick,
On Feb 18, 2010, at 23:52, I wrote:
[...]
So I'm trying to move your additions for client-side superdatastore up to the
abstract level (i.e. localengineDS and syncagent) before I merge with master. I'll
work on that right now, so maybe I can commit it today (if not, it might take longer as
I'm in vacation next week).
committed to luz/synthesis-indefero :-)
Best Regards,
Lukas Zeller (luz(a)synthesis.ch)
-
Synthesis AG, SyncML Solutions & Sustainable Software Concepts
info(a)synthesis.ch, http://www.synthesis.ch
12:32 p.m.
On Do, 2010-02-18 at 23:12 +0000, Lukas Zeller wrote:
Hello Patrick,
On Feb 18, 2010, at 23:52, I wrote:
> [...]
> So I'm trying to move your additions for client-side superdatastore
up to the abstract level (i.e. localengineDS and syncagent) before I
merge with master. I'll work on that right now, so maybe I can commit
it today (if not, it might take longer as I'm in vacation next week).
committed to luz/synthesis-indefero :-)
Thanks! I've adapted SyncEvolution to the <source>uri format and ran
some minimal tests successful. I have updated our master branch to your
"luz" branch.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
4564
days inactive
4568
days old
syncevolution@synevolution.org
19 comments
4 participants
participants (4)
-
Jussi Kukkonen -
Jussi Kukkonen -
Lukas Zeller -
Patrick Ohly