CRM 2013 on premise IFD : 404 error

I have set up IFD.

When I log in from outside, I get a 404 error. I can place a test.html file in the default web site folder and it is rendered.

I have org.domain.com set up in my DNS. The only other quirk I can see is that the auth.domain.com entry in the IFD wizard requires me to append :443 to it or it will fail.

So the issue seems to be the CRM web site not processing the request somehow.

I can log in through forms authentication and am taken to https://org.domain.com/default.aspx.  Not sure id this is DNS, IIS or Firewall issue.

How do I go about troubleshooting this?