You can run the client with the -v option to have some logs in the console.
There is nothing special to configure if you can already connect with the web client.
But you have to use a client from the same series. Also if you activated SSL you must use a certificate signed by a known authority on the OS.
If you switched from an SSL connection to a clear text connection, you may need to edit the known_hosts file.
I used the command : C:\Program Files (x86)\Tryton-5.0>tryton -v
==> Start of W10 Client is OK
==> …but where is the log generated ? … nothing either under mmc