I seem to experience a similar problem.
I have set up a new sync endpoint instance. (I had to use a sync-endpoint docker image that I built a couple of months ago, because there seems to be a problem with the the current code base? I will try to reproduce that particular problem and report it in a separate issue, but in relation to this issue, it suffice to say, I finally got it working by using the older sync-endpoint image. The rest of the docker images are built from the current code base).
I have several devices (phones and tablets of various brands) running different versions of Android (version 6 and 9). For the most part it works fine when I connect ODK Services to the sync endpoint, but a couple of the tablets have some issues. One of them a relatively high-end Samsung tablet running android version 9, that fails to connect/authenticate as well as a budget Lenovo tablet running Android version 6 that has the same problem… In other words - it doesn’t necessarily has to do with the android version.
I have inspected the nginx-logs, to see what was going on.
When it works, the logs say:
10.255.0.2 - - [26/Jul/2019:11:58:06 +0000] “GET /odktables/ HTTP/1.1” 401 1111 “-” “Sync 232 (gzip)” “18.104.22.168”
10.255.0.2 - myusername [26/Jul/2019:11:58:06 +0000] “GET /odktables/ HTTP/1.1” 200 31 “-” “Sync 232 (gzip)” “22.214.171.124”
10.255.0.2 - myusername [26/Jul/2019:11:58:06 +0000] “GET /odktables/default/privilegesInfo HTTP/1.1” 200 193 “-” “Sync 232 (gzip)” “126.96.36.199”
When it doesn’t work there is only one entry in nginx log - with the difference of the last segment containing just a “-”, where it otherwise contains an IP address when it works (is that “$http_x_forwarded_for” or something else, idk?):
10.255.0.2 - - [26/Jul/2019:11:58:29 +0000] “GET /odktables/ HTTP/1.1” 401 1111 “-” “Sync 232 (gzip)” “-”
… and the android tablet shows an error toast: “Authentication Error. Please verify your username/password or your account access.”.
The logfile generated on the device by ODK Services contains the following:
---- flushing ----
I/IOdkSyncServiceInterfaceImpl: 2019-07-26 11:58:21.773 SERVICE INTERFACE: getSyncResult WITH appName:default
I/SyncProgressTracker: 2019-07-26 11:58:26.732 Update SYNC Notification -default TEXT:Starting sync… PROG:0
I/AppSynchronizer: 2019-07-26 11:58:26.733 APPNAME IN SERVICE: default
I/AppSynchronizer: 2019-07-26 11:58:26.733 [SyncThread] begin VERIFYING timestamp: 1564142306733
I/PropertiesSingleton: 2019-07-26 11:58:26.769 readProperties(false)
E/HttpRestProtocolWrapper: 2019-07-26 11:58:26.801 AggregateUri:http://myhostname.westeurope.cloudapp.azure.com
D/HttpRestProtocolWrapper: 2019-07-26 11:58:26.801 normalizeUri: http://myhostname.westeurope.cloudapp.azure.com/
E/HttpRestProtocolWrapper: 2019-07-26 11:58:26.802 baseUri:http://myhostname.westeurope.cloudapp.azure.com/
D/HttpRestProtocolWrapper: 2019-07-26 11:58:26.806 normalizeUri: http://myhostname.westeurope.cloudapp.azure.com/
I/HttpRestProtocolWrapper: 2019-07-26 11:58:26.807 adding credential for host: myhostname.westeurope.cloudapp.azure.com username:myusername
I/ProcessAppAndTableLevelChanges: 2019-07-26 11:58:26.811 entered verifyServerConfiguration()
I/SyncProgressTracker: 2019-07-26 11:58:26.831 Update SYNC Notification -default TEXT:Verifying server supports this user application name. PROG:0
D/HttpRestProtocolWrapper: 2019-07-26 11:58:26.831 normalizeUri: http://myhostname.westeurope.cloudapp.azure.com/odktables/
I/HttpRestProtocolWrapper: 2019-07-26 11:58:26.832 buildBasicRequest: agg_uri is http://myhostname.westeurope.cloudapp.azure.com/odktables/
E/HttpRestProtocolWrapper: 2019-07-26 11:58:27.087 Unexpected server response statusCode: 401
E/ProcessAppAndTableLevelChanges: 2019-07-26 11:58:27.089 [verifyServerConfiguration] exception verifying support of appName exception: org.opendatakit.services.sync.service.exceptions.AccessDeniedException: Unexpected server response statusCode: 401
I/AppSynchronizer: 2019-07-26 11:58:27.090 [SyncThread] work completed (begin SyncStatus determination) timestamp: 1564142307090
E/SyncProgressTracker: 2019-07-26 11:58:27.104 FINAL SYNC Notification -default TEXT:Authentication Error.
Please verify your username/password or your account access.
Then a very unsatisfying and strange thing occurred; simply entering the server configuration in ODK Services again (by scanning the same QR code as before) made it work.
My suspicion is that there is something preventing the device from sending a correct http request? Or that it somehow doesn’t answer the auth. challenge that nginx answers with? Could ODK Services (or the underlying android subsystems) somehow have cached something so that it doesn’t send a fresh http request? (ODK Services seem to have some issues regarding state, where it “remembers” some things from previous connection attempts, but I have tried resetting it, clearing it’s data, deleting the opendatakit-folder and even re-installing it, but the problem seems to persist (until it magically disappeared on one of the tablets, that is ;-)))
On the couple of phones that I have connected, I have not seen this issue.
I am by no means an expert of this, and I seem to have exhausted my ideas for resolving this problem on my own, so any help would be greatly appreciated.
Notice that the hostnames in the logs have been replaced. If anyone wants access to inspect the sync endpoint please don’t hesitate to send me a pm.
Further exploration returned this error (when trying to connect to the server using a cloudflare proxy and https://)