CRM 2013 IFD via ADFS 3.0 Proxy - External Access failing with Event ID's 276 and 422

Hi All, Hope this is the right area. Not sure if this should sit somewhere more dedicated to AD FS.

I am attempting to Setup CRM 2013 for IFD, i have the below setup in place and the internal CBA works fine (I have not detailed the network side as I can see connectivity is working because i get the below errors each time i hit the external URL).

I believe the External part is almost working but failing with the two errors below, one logged on the ADFS servers and one logged on the WAP server. The WAP configuration is set to use a domain account which is a local admin on the ADFS servers (Call it domain\ADFSProxy). The wildcard cert we use is on the WAP server with the private key as well (also tried granting the service account full permissions on the Cert). I have tried changing the WAP service to use the domain account domain\ADFSProxy still get the same error.

Has anyone had this issue, its obviously a permissions issue. To add to the confusion the cert thumbprint displayed in the error does not appear to be assocaitated to any certs on either the AD FS or WAP servers.

Setup:

- CRM 2013 on Server 2012 R2 (this includes front end servers and application servers in a farm).

- AD FS 3.0 on Server 2012 R2 (2 servers in a farm) (configured to use SQL DB)

- Web Application Proxy on Server 2012 R2. (2 servers running WAP role)

WAP Error:

Log Name:      AD FS/Admin
Source:        AD FS
Date:          10/03/2014 14:00:50
Event ID:      422
Task Category: None
Level:         Error
Keywords:      AD FS
User:          NETWORK SERVICE
Computer:      proxy01.domain.mycompany.com
Description:
Unable to retrieve proxy configuration data from the Federation Service. 

Additional Data 

Trust Certificate Thumbprint: 
18A4F4D38117A9B39074C6FB74CEAD545938098E 

Status Code: 
Unauthorized 

Exception details: 
System.Net.WebException: The remote server returned an error: (401) Unauthorized.
   at System.Net.HttpWebRequest.GetResponse()
   at Microsoft.IdentityServer.Management.Proxy.StsConfigurationProvider.GetStsProxyConfiguration()
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="AD FS" Guid="{2FFB687A-1571-4ACE-8550-47AB5CCAE2BC}" />
    <EventID>422</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8000000000000001</Keywords>
    <TimeCreated SystemTime="2014-03-10T14:00:50.282499700Z" />
    <EventRecordID>16041</EventRecordID>
    <Correlation />
    <Execution ProcessID="3792" ThreadID="9336" />
    <Channel>AD FS/Admin</Channel>
    <Computer>proxy01.domain.mycompany.com</Computer>
    <Security UserID="S-1-5-20" />
  </System>
  <UserData>
    <Event xmlns="http://schemas.microsoft.com/ActiveDirectoryFederationServices/2.0/Events">
      <EventData>
        <Data>18A4F4D38117A9B39074C6FB74CEAD545938098E</Data>
        <Data>Unauthorized</Data>
        <Data>System.Net.WebException: The remote server returned an error: (401) Unauthorized.
   at System.Net.HttpWebRequest.GetResponse()
   at Microsoft.IdentityServer.Management.Proxy.StsConfigurationProvider.GetStsProxyConfiguration()</Data>
      </EventData>
    </Event>
  </UserData>
</Event>

AD FS Error:

Log Name:      AD FS/Admin
Source:        AD FS
Date:          10/03/2014 15:07:52
Event ID:      276
Task Category: None
Level:         Error
Keywords:      AD FS
User:          Domain\ADFS_Service
Computer:      ADFS01.domain.mydomain.com
Description:
The federation server proxy was not able to authenticate to the Federation Service. 

User Action 
Ensure that the proxy is trusted by the Federation Service. To do this, log on to the proxy computer with the host name that is identified in the certificate subject name and re-establish trust between the proxy and the Federation Service using the Install-WebApplicationProxy cmdlet. 

Additional Data 

Certificate details: 

Subject Name: 
<null> 

Thumbprint: 
<null> 

NotBefore Time: 
<null> 

NotAfter Time: 
<null>
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="AD FS" Guid="{2FFB687A-1571-4ACE-8550-47AB5CCAE2BC}" />
    <EventID>276</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8000000000000001</Keywords>
    <TimeCreated SystemTime="2014-03-10T15:07:52.286882500Z" />
    <EventRecordID>39182</EventRecordID>
    <Correlation ActivityID="{00000000-0000-0000-7002-0080020000EE}" />
    <Execution ProcessID="8368" ThreadID="2724" />
    <Channel>AD FS/Admin</Channel>
    <Computer>ADFS01.domain.mydomain.com</Computer>
    <Security UserID="S-1-5-21-2675044235-485420783-3068902212-18162" />
  </System>
  <UserData>
    <Event xmlns="http://schemas.microsoft.com/ActiveDirectoryFederationServices/2.0/Events">
      <EventData>
        <Data>&lt;null&gt;</Data>
        <Data>&lt;null&gt;</Data>
        <Data>&lt;null&gt;</Data>
        <Data>&lt;null&gt;</Data>
      </EventData>
    </Event>
  </UserData>
</Event>