ODK-X Service Sync Failed

Hi,

I got ODK-X Endpoint 2.1.6 installed on AWS EC2 with some tables. ODK-X Tables/Services 2.1.6 installed on android.

  • In server settings of ODK-X Services:
    * Server URL: http://ec2-xxx.com (http prefix is added)
  • Verify user permission and Sync Now => It worked
  • Sync Now again => Internal error with NullPointerException in the log
  • Visit Server Settings again => Server Url became ec2-xxx.com (http has gone) => Hence the NPE as there is no scheme in the URL
  • Add http:// to the url again and Sync Now => It worked.
  • Sync Now again (may need some times) => NPE occurred.

The issue occurred on both Android 5 and 10.

Services prevents users from using http without an admin specifically adding a setting to allow this so people do not accidentally expose data on the Internet.

You need to set an administrator setting to allow “http” instead of “https”.
See: https://docs.odk-x.org/services-managing/#administrator-settings

Here are the steps:
You need to change/establish an admin password as this will allow you to get more settings. This will will allow the enabling of http connections.

  1. ​"enable user restrictions"
  2. enter a password you want
  3. more options will appear
  4. ​manage ability to change Server Settings
  5. ​select http allowed to specifically the “allow unsafe/unsecure authentication” checkbox
1 Like

Hi @W_Brunette, the issue still persists with allowed http.

Can you attach the log with the NullPointerException?

I/SQLiteConnection: 2020-06-09 22:37:29.593 A SQLiteConnection object for database ‘default’ sessionQualifier ‘uuid:f3216896-8618-49f8-b747-86919253c466’ successfully closed.
---- flushing ----
I/OdkSyncService: 2020-06-09 22:37:24.070 Sync Service onStartCommand startId: 1
I/IOdkSyncServiceInterfaceImpl: 2020-06-09 22:37:24.076 SERVICE INTERFACE: getSyncResult WITH appName:default
I/IOdkSyncServiceInterfaceImpl: 2020-06-09 22:37:24.937 SERVICE INTERFACE: synchronizeWithServer WITH appName:default
I/SyncProgressTracker: 2020-06-09 22:37:24.965 Update SYNC Notification -default TEXT:Starting sync鈥� PROG:0
I/AppSynchronizer: 2020-06-09 22:37:24.966 APPNAME IN SERVICE: default
I/AppSynchronizer: 2020-06-09 22:37:24.966 [SyncThread] begin SYNCING timestamp: 1591717044966
I/PropertiesSingleton: 2020-06-09 22:37:25.000 readProperties(true)
E/HttpRestProtocolWrapper: 2020-06-09 22:37:25.011 AggregateUri:ec2-54-206-98-134.ap-southeast-2.compute.amazonaws.com
D/HttpRestProtocolWrapper: 2020-06-09 22:37:25.012 normalizeUri: ec2-54-206-98-134.ap-southeast-2.compute.amazonaws.com/
E/HttpRestProtocolWrapper: 2020-06-09 22:37:25.012 baseUri:ec2-54-206-98-134.ap-southeast-2.compute.amazonaws.com/
D/HttpRestProtocolWrapper: 2020-06-09 22:37:25.019 normalizeUri: ec2-54-206-98-134.ap-southeast-2.compute.amazonaws.com/
java.lang.NullPointerException: Attempt to invoke virtual method ‘boolean java.lang.String.equals(java.lang.Object)’ on a null object reference
at org.opendatakit.services.sync.service.logic.HttpRestProtocolWrapper.(HttpRestProtocolWrapper.java:720)
at org.opendatakit.services.sync.service.logic.AggregateSynchronizer.(AggregateSynchronizer.java:118)
at org.opendatakit.services.sync.service.logic.SynchronizerFactory.create(SynchronizerFactory.java:23)
at org.opendatakit.services.sync.service.AppSynchronizer$SyncTask.sync(AppSynchronizer.java:293)
at org.opendatakit.services.sync.service.AppSynchronizer$SyncTask.run(AppSynchronizer.java:193)
at java.lang.Thread.run(Thread.java:919)

E/SyncProgressTracker: 2020-06-09 22:37:25.021 FINAL SYNC Notification -default TEXT:Internal Error.
Please submit a bug report
I/IOdkSyncServiceInterfaceImpl: 2020-06-09 22:37:25.296 SERVICE INTERFACE: getSyncResult WITH appName:default
I/IOdkSyncServiceInterfaceImpl: 2020-06-09 22:37:26.682 SERVICE INTERFACE: getSyncResult WITH appName:default
I/OdkSyncService: 2020-06-09 22:37:27.404 Sync Service onUnbind – no more bound interfaces
I/OdkSyncService: 2020-06-09 22:38:24.068 Sync Service shutdownActor – calling stopSelf
---- flushing ----

1 Like

It may have something to do with have an extra “/”. To test this do not have any extra “/” at the end of the URL. If you simply use the ‘sync’ button it will both verify the user and sync. You might want to try that multiple times in a row to see if the verify credentials has a code path problem with “http”. The majority of our users, use “https” so the “http” paths do not get tested very often. Also Android updates of the underlying transport system have been known to try issues.

Thanks you for pointing this out to us, I am interested to see what you find so we can hopefully find the cause and fix it.

Adding / at the end does not help. I also found that the issue does not occur when I sync ODK-X service with Aggregate 1.7.1.

Hi @Thien_Mai. I will try to reproduce this locally. Just to confirm my understanding, this issue does not occur every time? The first sync will succeed, and then the second sync might fail, but it might be the third or the fourth or it might take many tries?

Is there new data to be synced each time? Or will it occur just pressing sync again and again?

You are not seeing this issue on Aggregate? Even with many tries?

1 Like

Hi @Jeff_Beorse,

  • The issue occurs every time
  • When a sync failed, all subsequent sync failed with the same NPE issue. Sometimes sync failed after 2 or 3 tries, and then subsequent sync always failed.
  • Sync failed even with no new data synced
  • I do not see the issue on Aggregate
  • I also tested on my local. My Mac and phone were on the same wifi network. ODK-X Service Server url is set as http://192.168.x.x. It succeeded after first or second sync, then the next sync failed with NPE (no new data). Interestingly, the server url somehow was changed to localhost behind the scene.

I did not test with https://

2 Likes

Images added, FYI.

  • 1st sync

  • 2nd sync (no new data)

  • Visit server settings page - url became localhost

1 Like

Interesting. Perhaps because “anonymous access” is turned on Aggregate and not in Sync Endpoint? @linl33 thoughts?

1 Like

Hi @Thien_Mai. I don’t have a solution yet but wanted to update this thread and say I haven’t forgotten about you. Still trying to debug this. Thanks for all the info.

1 Like

Hi @Jeff_Beorse, thanks for the update. I forgot to mention that I uploaded my custom application and tables to ODK-X server using ODK-X Suitecase. The upload script generated /config/assets/app.properties file containing server url, like this (no http prefix):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
<entry key="common.sync_server_url">ec2-x-x-x.ap-southeast-2.compute.amazonaws.com</entry>
</properties>

Once the app uploaded, I sync with ODK-X Tables to download the app (include app.properties file). After that, the issue occurred as I mentioned above.

When I modify the script to include http:// in the app.properties file:

<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
<entry key="common.sync_server_url">http://ec2-x-x-x.ap-southeast-2.compute.amazonaws.com</entry>
</properties>

and upload it, sync with ODK-X Tables, the issue gone: Once I update server url setting in the ODK-X Tables, the url is never reverted back to ec2-x-x-x.com (no http://)

So now that seems not an issue to me as I can update upload script and make it work. If you think that is issue, hopefully that would provide you better context for analysis.

3 Likes

Thanks for this. I am glad it is working for you now.

This is very strange. I will file a ticket and we will try to fix this.

1 Like