Problem installing ODKv2 Sync on Ubuntu with Custom LDAP (AD) - yaml: line 98: did not find expected key

Support
1

Hi,

I am trying to install ODKv2 Sync on Ubuntu Server v18.09, Docker and Swarm Mode has already been enabled.

I got up to Step 10 in the PDF without any issues, and from that point on the document is extremely vague almost to expect you are some super saavy developer that has in depth knowledge of the source code, which I don’t, so it is very intimidating.

It says to security.properties but I found 3 versions:
./config/sync-endpoint/security.properties
./sync-endpoint/postgres-test/resources/security.properties
./sync-endpoint/mysql-test/resources/security.properties

I assume it is the first one and added (yes added because none of the following 3 were actually in the first file):
security.server.hostname=odk.mydomain.com
security.server.port=880
security.server.securePort=7443

I did not touch the other 2 files.

Next it says to edit the ports on docker-compose.yml under sync, so the only thing that came close was under nginx: so I put the following:

nginx:
image: nginx:1.13
networks:
- sync-network
ports:
- “880:80”:~/sync-endpoint-default-setup#
- “7443:443”

For Custom LDAP I exported by Active Directory Root CA Certificate as a base64 and placed it into a cer file and ran

docker config create org.opendatakit.sync.ldapcert ~/sync-endpoint-default-setup/ca_cert.cer

and it returned a hex value.

It also shows up in “docker config ls”
root@vc-forms2:~/sync-endpoint-default-setup# docker config ls
ID NAME CREATED UPDATED
60u4j8x0tiqbz39ouvc68gopu org.opendatakit.sync.ldapcert About an hour ago About an hour ago

Then I uncomment the “relevant” lines in docker-compose.yml which I assume should look like this

env_file:
- sync.env
# uncomment these after uncommenting the ones below in the root level configs
# configs:
- org.opendatakit.sync.ldapcert
# - org.opendatakit.aggregate.logging.properties


configs:

uncomment this to import a CA certificate into the Sync service

org.opendatakit.sync.ldapcert:
external: true

Next, I removed the ldap-service and phpldapadmin under the sevices section, and then modified ./config/sync-endpoint/security.properties to enter my LDAP AD Properties while leaving the other security.properties alone.

security.server.ldapQueryUsername=svcs.domain@mydomain.com
security.server.ldapQueryPassword=MyTestSampleP@ssw0rd1!TestPW
security.server.ldapDomainDClevel2=domain-controller1.mydomain.com
security.server.ldapDomainDClevel1=domain-controller2.mydomain.com
security.server.ldapUrl=ldaps:/mydomain.com:636/
security.server.ldapBaseDn=dc=domain,dc=com
security.server.ldapPooled=false
security.server.userSearchBase=ou=MyUsers
security.server.groupSearchBase=ou=${security.server.groupPrefix},ou=
security.server.groupRoleAttribute=cn
security.server.userFullnameAttribute=givenName
security.server.usernameAttribute=uid

Finally I try to start the server with:

docker stack deploy -c docker-compose.yml syncldap

And get this error:

yaml: line 98: did not find expected key

Which refers to these lines under docker-compose.yml:

configs:

uncomment this to import a CA certificate into the Sync service

org.opendatakit.sync.ldapcert:
external: true <<======= Line 98

What did I do wrong here? The documentation is too vague and provides no example so that I end up guessing a lot of things which is not good.

Please advise.

Thanks

2

Found the problem, turns out that docker is very picky about extra spaces when it parses docker-compose.yml

However even with the services running, the system fails to contact the external LDAP Server. Wireshark captures to the ODKv2 system keeps on showing it is trying to query a DNS A Record named “sync”, and “docker logs ” keeps showing it cannot connect to my external LDAP Server. AFAIK “sync” is a system available on the docker internal overlay network so no idea why it is trying to query externally, and the system cannot access the external LDAP probably because the internal container has issues accessing the external network i.e. it has no routes. The sync DNS Queries start happening the moment I switch the config file to “activeDirectory”.

This looks like a major bug in the system, and I doubt that any one has ever gotten to work with an external AD or LDAP.

Even the install at “mvn clean install” has issues even with all the prerequisites met. It got stuck testing mysql at the end and kept saying it cannot login using ‘odk_unit’@‘172.20.0.1’ just after it gets stuck at “writing large dataset” and I had to manually using another terminal do a “docker exec -it mysql -uroot -pmysqlPassword” and manually grant access! There is a script that does this in the mysql-test folder, but it looks like it is another HUGE bug in the code!! Even with all this effor the mysql test fails at the end >.<‘’ and this is following exactly step by step the instructions.

Docker just makes everything THAT much more complex!

For now I got it to run under POSTGRES Sql using the integrated OpenLDAP and im using a 3rd party tool to sync my Active Directory Users to OpenLDAP and exposing ports 389 and 686 by adding this do docker-compose.yml:

ldap-service:
image: odk/openldap
deploy:
replicas: 1
ports: <<—This
- “389:389” <<—This
- “636:636” <<—And This Line

3

The reason why AD didn’t work is because these 2 lines were not set properly,

security.server.ldapDomainDClevel2=
security.server.ldapDomainDClevel1=

As an example, if my AD domain is example.org

security.server.ldapDomainDClevel2=example
security.server.ldapDomainDClevel1=org

If your domain has more levels, you will need to modify these instead

security.server.ldapDomain=
security.server.ldapDomainControllerUrl=
security.server.ldapDomainDC=

The default value for these 3 fields are configured to infer from ldapDomainDClevel1 and ldapDomainDClevel2.

On the LDAP/AD CA,
The certificate needs to be importable using Java keytool

Edit
The Active Directory Domain Controller must have Domain Services enabled. This is required for Sync Endpoint to query the AD using LDAP.