Posts Tagged ‘Adapters’

First of all, Happy birthday BizTalk Server for your 16th birthday! For does you don’ remember, the first version of BizTalk Server was release 12/12/2000, Congratulations!!

Continuing with the topic of my last posts “Errors and Warnings, Causes and Solutions”, we will talk about an error that I face today using the BizTalk Server WCF adapter while trying to communicate with an external WCF service:

System.ServiceModel.EndpointNotFoundException: There was no endpoint listening at http://<ip/host name>/<ServiceName>.svc that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details. —> System.Net.WebException: Unable to connect to the remote server —> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 127.0.0.1:8888
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)

— End of inner exception stack trace —

at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context)
at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStreamAsyncResult.CompleteGetRequestStrea

BizTalk-WCF-Adapter-Unable-to-connect-to-the-remote-server

If this problem happens, it normally means that the IP address or host name specify in the URL exists but: it has no services listening on the specified port or there is a firewall stopping you.

However, I try to open the URL using the browser in the BizTalk machine and I was able to access without any problem, which means, it wasn’t a firewall problem and the service exist on that specific port.

CAUSES

Again, this type of problem normally means that the IP address or host name specify in the URL exists but:

  • It has no services listening on the specified port;
  • Or there is a firewall stopping you.

But… it also can be a proxy problem that may be blocking the access to the service! In fact, this was my problem.

SOLUTION

I will not address the first two possible causes here, instead I will focus in what was my problem, proxies, and how can you solve it.

On the HTTP Transport bindings of the WCF adapter there are several properties to control the proxy, like:

  • proxyAddress: A URI that specifies the address of the HTTP proxy. If useSystemWebProxy is true, this setting must be null. The default is null.
  • proxyAuthenticationSchema: Specifies the protocol used for authenticating client requests being processed by an HTTP proxy. The default is Anonymous.
  • bypassProxyOnLocal: A Boolean value that indicates whether to bypass the proxy server for local addresses. The default is false.

and others properties. But also, contains a very important property: useDefaultWebProxy – this is a Boolean value that specifies whether the machine-wide proxy settings are used rather than the user specific settings. The default is true.

BizTalk-WCF-Adapter-useDefaultWebProxy-true

But how can you know there is a proxy set on the server?

You can check this using one of this two option:

  • You can open Command Prompt (CMD) and type the following command:

reg query "HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings" | find /i "proxyserver"

BizTalk-WCF-Adapter-check-if-proxy-is-set

  • Or Open the Internet Explorer and click the Tools button.
    • Click on Internet Options and then click on the Connections tab.
    • Click “LAN settings”.

BizTalk-WCF-Adapter-check-if-proxy-is-set-in-internet-explorer

    • You can also click in “Advanced” to check more details

If you notice in the pictures there is a default proxy set in the server and there are some exceptions defined in the Internet Explorer, a few names and some IPs. One of this IP’s was in fact the IP of the machine that was hosting the service that I was trying to communicate.

In my case to solve the problem I just need to set the useDefaultWebProxy property in the HTTP transport bindings of my WCF port to false.

BizTalk-WCF-Adapter-useDefaultWebProxy-false

In the sequence of my last two post (here and here) and following the same topic, in order to finalize it, let’s talk about the last option and see how we can accomplish the same goal but this type configuring the Send Handler for existing static Send Ports in your environment.

Well, the two previous post are, in my opinion, more critical than this one for se simple reason that the default BizTalk installation will had ports in your environment that needs to be configure if you want to configure BizTalk Host in a High Availability way, i.e., dedicate logical hosts to run specific areas of functionality such as receiving messages, sending messages, processing orchestrations or tracking data.

However, and although less critical, static send ports can also be a problem in some scenarios, for example:

  • If you install ESB Toolkit before configuring your hosts for each functionality, because during the ESB Toolkit installation it will also be created a static send port call “ALL.Exceptions” that connects with SQL “EsbExceptionDb” database. And this port will be configured with the default send handler (at that time) configured in your environment, that is by default the “BizTalkServerApplication”
  • but also if you reach to an existing environment, already containing several BizTalk Applications running, that is not configured according to best practices in terms of host and host instances (dedicate logical hosts for each functionality).

Once again, we need to do basically do the same steps described in my last two posts to accomplish the task of deleting “BizTalkServerApplication”, this time as a send handler of each adapter:

  • Manually reconfigure the Send handler for each the existing Static Send Port first, configuring them with the new or correct Send Handler;
  • and then manually delete the “BizTalkServerApplication” as a Send handler for each adapter;

You may have heard this before, but it never hurts, all of these tasks are time consuming, and a little boring to do after a while or if we need to do it several times;

So how can we automate tasks? and reuse them whenever necessary and at the same time saving significant time for other tasks?

Using PowerShell is a good option J. Windows PowerShell is a Windows command-line shell designed especially for system administrators and can be used by BizTalk administrators to help them in automating repetitive tasks or tasks that are time consuming to perform manually.

This is a simple script that allows you to configure the Send handler associated with all the existing Static Send Ports in your environment independently of the adapter that is configured:

$catalog = New-Object Microsoft.BizTalk.ExplorerOM.BtsCatalogExplorer
$catalog.ConnectionString = "SERVER=$bizTalkDbServer;DATABASE=$bizTalkDbName;Integrated Security=SSPI"

foreach($SendPort in $catalog.SendPorts)
{
    # For each receive location in your environment
    if($sendPort.IsDynamic -eq $False)
    {
        # Let's look for send handlers associated with Adapter configured in the send port
        foreach ($handler in $catalog.SendHandlers)
        {
            # if the Send Handler is associated with the Adapter configured in the send port
            if ($handler.TransportType.Name -eq $sendPort.PrimaryTransport.TransportType.Name)
            {
                # We will configured the port with the default send handler associated in each adapter in you system  
                # independently if it is "BizTalkServerApplication" or not.
                # Note: it's is recomended that you first configure the default send handlers for each adapter 
                #       and also not to use the "BizTalkServerApplication" (my personal recomendation)
                if($handler.IsDefault)
                { 
                    $sendPort.PrimaryTransport.SendHandler = $handler
                    break
                }
            }
        }
    }
}
$catalog.SaveChanges()

Prerequisites for this script: The host, host instances and Receive handlers needs to be already configured in your environment before you run the script;

THIS POWERSHELL IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

The full script can be found and download on Microsoft TechNet Gallery:
PowerShell to Configure the Default Send Handler for each static Send Port (3.0 KB)
Microsoft TechNet Gallery

Following the topic from my previous post, when we are configuring/optimizing a new environment, if some features are installed like EDI features or RosettaNet Accelerator, if we create different or dedicated host and host instances for each functionality, for example a host instance for receive, send, process (orchestrations), tracking and so on; of course then we need to associate them as a specific handler of each adapter (receive or send handler) and if we want to delete the “BizTalkServerApplication” as a receive handler from each adapter… we can’t!

This happens because, at least, both EDI and RosettaNet will create during the installation some Receive Ports:

  • BatchControlMessageRecvPort with a receive location BatchControlMessageRecvLoc using SQL Adapter
  • ResendReceivePort with a receive location ResendReceiveLocation using SQL Adapter
  • LOB_To_PrivateInitiator with a receive location LOB_To_PrivateInitiator using SQL Adapter
  • LOB_To_PrivateResponder with a receive location LOB_To_PrivateResponder using SQL Adapter

And all these ports, by default during the installation, are configured with the only existing Receive handler available at the time: “BizTalkServerApplication”.

To accomplish the task of deleting “BizTalkServerApplication” as a receive handler of each adapter, we need to do basically the same steps described in my last post:

  • Manually reconfigure the Receive handler for each the existing receive location first, configuring them with the new Receive Handler;
  • and then manually delete the “BizTalkServerApplication” as a Receive handler for each adapter;

This task can be more normal to happen on the first time we configure/optimize the environment – day one of the BizTalk Server – but can be more critical if you reach to an existing environment, already containing several BizTalk Applications running, that is not configured according to best practices in terms of host and host instances.

Once again, all of these tasks are time consuming, and to be fair… again…, they are a little boring to do after we know how to do it manually;

So how can we automate tasks? and reuse them whenever necessary and at the same time saving significant time for other tasks?

Using PowerShell is a good option J. Windows PowerShell is a Windows command-line shell designed especially for system administrators and can be used by BizTalk administrators to help them in automating repetitive tasks or tasks that are time consuming to perform manually.

This is a simple script that allows you to configure the receive handlers associated with all the existing Receive Ports in your environment that are using the SQL Adapter, but this can be easily changed to cover all the Receive Locations independently of the configured adapter:

$catalog = New-Object Microsoft.BizTalk.ExplorerOM.BtsCatalogExplorer
$catalog.ConnectionString = "SERVER=$bizTalkDbServer;DATABASE=$bizTalkDbName;Integrated Security=SSPI"

foreach($receivePort in $catalog.ReceivePorts)
{
    # For each receive location in your environment
    foreach($recLocation in $receivePort.ReceiveLocations)
    {
        # In this case I want only Receive location that are using SQL Adapter
        if($recLocation.ReceiveHandler.TransportType.Name -eq 'SQL')
        {
            # Let's look for receive handlers associated with SQL Adapter
            foreach ($handler in $catalog.ReceiveHandlers)
            {
                # if is a SQL Adapter Receive Handler
                if ($handler.TransportType.Name -eq "SQL")
                {
                    # And is not BizTalkServerApplication, then configure that as the handler of the receive location
                    # Note: that in this case we will configure as a Receive Handler of the Receive location, the first 
                    #       receive handler that we find that is not the "BizTalkServerApplication"
                    #       because the goal is to delete this handler
                    if($handler.Name -ne 'BizTalkServerApplication')
                    { 
                        $recLocation.ReceiveHandler = $handler
                        break
                    }
                }
            }
        }
    }
}
$catalog.SaveChanges()

Prerequisites for this script: The host, host instances and Receive handlers needs to be already configured in your environment before you run the script;

THIS POWERSHELL IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

The full script can be found and download on Microsoft TechNet Gallery:
PowerShell to Configure Receive Handlers from existing Receive locations (3.0 KB)
Microsoft TechNet Gallery

As happens almost every year on this day (my birthday), I always try to write a post for the BizTalk community… something like a gift/present to the community… today will be automate the task of configuring default Dynamic Send ports handlers using PowerShell.

Since BizTalk Server 2013 we have available a new feature: Configurable Dynamic Send Port Handler. Which means that, when we are creating a Dynamic Send Port, an adapter Send Handler can be configurable for every installed adapter, by default it uses the default send handler associated in each adapter. This Includes both One-Way and Solicit-Response Dynamic Ports.

Note that in previous BizTalk versions, a dynamic send port uses the adapter’s default host, and we cannot change this behavior.

However, this new features also brings us some setbacks, special for BizTalk Administrators, for example:

  • When we are installing a new environment, if we install the EDI features, this will add a dynamic port call “ResendPort” (don’t need to describe the propose of this port) and the default send handler for each adapter will be the “BizTalkServerApplication”;
    • If we create different or dedicated host and host instances for each functionality, for example a host instance for receive, send, process (orchestrations), tracking and so on; of course then we need to associate them as a specific handler of each adapter (receive or send handler) and if we want to delete the “BizTalkServerApplication” as a send handler for each adapter… we can’t, we first need to:
      • Manually reconfigure the default Dynamic Send port handlers for each dynamic port first, configuring them with the new default send handler;
      • and then manually delete the “BizTalkServerApplication” as a send handler for each adapter;
  • The same happens when we install a new adapter. By default, it assumes the default host in the group as the default send handler of the adapter and in consequence the default send handler associated with this adapter in the existing dynamic send ports. Which means that once again we need to manually reconfigure the send handler in all the existing dynamic send ports for this new adapter;
  • And so on;

All of these tasks are time consuming, and to be fair, they are a little boring to do after we know how to do it manually;

So how can we automate tasks? and reuse them whenever necessary and at the same time saving significant time for other tasks?

Using PowerShell is a good option. Windows PowerShell is a Windows command-line shell designed especially for system administrators and can be used by BizTalk administrators to help them in automating repetitive tasks or tasks that are time consuming to perform manually.

This is a simple script that allows you to configure the default send handlers associated with all the existing Dynamic Send ports in your environment:

[string] $sendHost32bits = "BizTalkServerSend32Host"
[string] $sendHost64bits = "BizTalkServerSendHost"

$catalog = New-Object Microsoft.BizTalk.ExplorerOM.BtsCatalogExplorer
$catalog.ConnectionString = "SERVER=$bizTalkDbServer;DATABASE=$bizTalkDbName;Integrated Security=SSPI"

foreach($sendPort in $catalog.SendPorts)
{
    if($sendPort.IsDynamic -eq' True')
    {
        # A Dynamic send port was found so now we need to configure the send handler as desired
        # 64 bits adapters
        # Changing the default send handlers of the dynamic port
        $sendPort.SetSendHandler("FILE", $sendHost64bits)
        $sendPort.SetSendHandler("HTTP", $sendHost64bits)
        $sendPort.SetSendHandler("MQSeries", $sendHost64bits)
        $sendPort.SetSendHandler("MSMQ", $sendHost64bits)
        $sendPort.SetSendHandler("SB-Messaging", $sendHost64bits)
        $sendPort.SetSendHandler("SFTP", $sendHost64bits)
        $sendPort.SetSendHandler("SOAP", $sendHost64bits)
        $sendPort.SetSendHandler("WCF-BasicHttp", $sendHost64bits)
        $sendPort.SetSendHandler("WCF-BasicHttpRelay", $sendHost64bits)
        $sendPort.SetSendHandler("WCF-Custom", $sendHost64bits)
        $sendPort.SetSendHandler("WCF-NetMsmq", $sendHost64bits)
        $sendPort.SetSendHandler("WCF-NetNamedPipe", $sendHost64bits)
        $sendPort.SetSendHandler("WCF-NetTcp", $sendHost64bits)
        $sendPort.SetSendHandler("WCF-NetTcpRelay", $sendHost64bits)
        $sendPort.SetSendHandler("WCF-SQL", $sendHost64bits)
        $sendPort.SetSendHandler("WCF-WebHttp", $sendHost64bits)
        $sendPort.SetSendHandler("WCF-WSHttp", $sendHost64bits)
        $sendPort.SetSendHandler("Windows SharePoint Services", $sendHost64bits)       

        # 32 bits adapters
        # SMTP Supports 64 bits but I want to run in 32 because of the MIME/SMIME Encoder
        $sendPort.SetSendHandler("FTP", $sendHost32bits)
        $sendPort.SetSendHandler("SMTP", $sendHost32bits)
        $sendPort.SetSendHandler("SQL", $sendHost32bits)
    }
}

$catalog.SaveChanges() 

Prerequisites for this script: The host, host instances and send handlers needs to be already configured in your environment before you run the script;

THIS POWERSHELL IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND.

The full script can be found and download on Microsoft TechNet Gallery:
PowerShell to configure Default Dynamic Send Port Handlers for each Adapter (3.0 KB)
Microsoft TechNet Gallery

I just released yesterday, but I’ve decided to do a small update to this tool bringing new functionalities in order to solve some limitations, especially Character Encoding problem.

Portuguese language have many letters with accents and while importing/reading the files in the previous version, all accented letters are printed as a question mark. It was obvious that I had a problem with Encoding.

Again, “Microsoft Message Queuing Testing Tool” is a simple tool that you can use to test sending files to Microsoft Queue in order to evaluate whether other applications are reading correctly the messages.

I’m using it to send messages to a queue that is being consumed by BizTalk Server

RELEASE NOTES

Version 2.0 brings:

  • Improvements in the “Message Label” auto fill
  • Improvements in the “Import Message File” functionality with better control of the encoder associated with the file
    • Add the ability to choose the code page identifier

Microsoft-Message-Queuing-Testing-Tool-v2

Hope you enjoy it.

You can download this tool here:

Microsoft Message Queuing Testing Tool (79,9 KB)
Microsoft | Code Gallery

In the last couple of days I been working (migrating a BizTalk Solution from BizTalk Server 2004 to BizTalk Server 2013 R2) in a solution that works deeply with Microsoft Message Queuing (MSMQ). It receives messages from private queue with the use of the MSMQ Adapter and send messages to other Queues (among other channels) that are consumed by other applications/servers.

And one of the things that I intended to do was partial tests, in a very simple way, to validate if all my receive ports were well configured in order to consume the expected messages.

Microsoft Message Queuing Testing Tool” is a simple tool that you can use to test sending files to Microsoft Queue.

Microsoft Message Queuing or MSMQ is a message queue implementation developed by Microsoft and deployed in its Windows Server operating systems and enables applications running at different times to communicate across heterogeneous networks and systems that may be temporarily offline. It provides guaranteed message delivery, efficient routing, security, and priority-based messaging.

With this tool you can easily send messages to a queue in order to evaluate whether other applications are reading correctly the messages.

Microsoft-Message-Queuing-Testing-Tool

This tool allows you to set:

  • The queue where you want to send the files/messages
  • A Label associated to the file/message
  • The Queue transaction type
  • Import an existing file or manually add a message

Microsoft-Message-Queuing-Testing-Tool-result

And it will provide:

  • success of delivery information (with transaction id)
  • or error detail information

I like simple tool that allow me to test my integration scenarios/projects, so I decided today to take some of my time to create this very simple and basic tools, but at least for me, that will be very useful.

Hope you enjoy also.

You can download this tool here:

Microsoft Message Queuing Testing Tool (79,9 KB)
Microsoft | Code Gallery

In the sequence of my last post, and following the result obtained while trying to solve the problems that had been originated by the SAP system migration to a newer version, I end up receiving another error. This time the connectivity problem with SAP was overcome, however without any reason I started to receive an error saying that the schema was not known

I got the following error:

“There was a failure executing the receive pipeline: "Microsoft.BizTalk.DefaultPipelines.XMLReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" Source: "XML disassembler" Receive Port: "IN_IDOC_PORT" URI: "sap://CLIENT=[SAPClientID];LANG=[LANG];@A/[SAPServer]/[SystemID]?ListenerGwServ=[GWServer]&ListenerGwHost=[GwHost]&ListenerProgramId=[ProgID]&RfcSdkTrace=True&AbapDebug=False" Reason: Finding the document specification by message type "http://Microsoft.LobServices.Sap/2007/03/Idoc/3/IDOC_NAME//740/Receive#Receive&quot; failed. Verify the schema deployed properly.”

It was strange to receive this kind of error because the process was in testing stage, for about a month, and working properly.

CAUSE

After the SAP System migration to a newer version the process started also to use a newer version of the IDOC Schema. I really don’t know if the SAP team change anything in the SAP side or it had to do with the migration itself, however when I checked the source code I noticed that we were using the 711 version of the schema:

And now the SAP system process was using and sending the 740 version of the schema:

SOLUTION

The solution is very simple, you just need to generate and import the newest version of IDOC schema (740) to your BizTalk Solution, make all the necessary changes and redeploy all the required artifacts (schemas, orchestrations, maps, and so on)

Recently one of my clients migrated one of their SAP systems and after that intervention the BizTalk receive location that was listening in a specific Program ID started to failed with the following error message present in the event viewer: “RFC IDOC_INBOUND_ASYNCHRONOUS could not be resolved against SAP system because its metadata could not be obtained

Full error message:

The adapter "WCF-Custom" raised an error message. Details "Microsoft.ServiceModel.Channels.Common.MetadataException: RFC IDOC_INBOUND_ASYNCHRONOUS イ楸 could not be resolved against SAP system because its metadata could not be obtained. —> Microsoft.Adapters.SAP.RFCException: Details: ErrorCode=RFC_EXCEPTION. ErrorGroup=RFC_ERROR_APPLICATION_EXCEPTION. SapErrorMessage=EXCEPTION FU_NOT_FOUND RAISED. AdapterErrorMessage=An error occurred while determining the function interface of the RFC IDOC_INBOUND_ASYNCHRONOUS イ楸.
at Microsoft.Adapters.SAP.RFCException.HelperThrow(Int32 retCode, String additionalErrorMessage)
at Microsoft.Adapters.SAP.RfcClientConnection.GetRfcFunctionInterface(String rfcName)
at Microsoft.Adapters.SAP.InternalRfcMetadata..ctor(String originalRfcName, SAPConnection sapConnection)
at Microsoft.Adapters.SAP.SAPMetadataContract.ResolveOperationMetadata(String operationId, TimeSpan timeout, TypeMetadataCollection& extraTypeMetadataResolved)
at Microsoft.ServiceModel.Channels.Common.Design.MetadataCache.GetOperationMetadata(String uniqueId, Guid clientId, TimeSpan timeout)
at Microsoft.Adapters.SAP.SapFunctionMetadata.ResolveOperationMetadataUsingSdk(String absoluteName, String displayName, String funcName, String operationNamespace, SAPConnection sapConnection, Boolean isTrfc, MetadataLookup metadataLookup, TimeoutHelper timeoutHelper)
— End of inner exception stack trace —
at Microsoft.ServiceModel.Channels.Common.Design.AdapterAsyncResult.End()
at Microsoft.ServiceModel.Channels.Common.Channels.AdapterReplyChannel.EndTryReceiveRequest(IAsyncResult result, RequestContext& requestContext)
at System.ServiceModel.Dispatcher.ReplyChannelBinder.EndTryReceive(IAsyncResult result, RequestContext& requestContext)
at System.ServiceModel.Dispatcher.ErrorHandlingReceiver.EndTryReceive(IAsyncResult result, RequestContext& requestContext)".

Of course followed by the generic warning messages:

The adapter "WCF-Custom" raised an error message. Details "Microsoft.Adapters.SAP.RFCException: Details: ErrorCode=RFC_INVALID_HANDLE. AdapterErrorMessage=An exception has occurred on the listener while executing RfcWaitForRequest..".

The adapter "WCF-Custom" raised an error message. Details "System.ServiceModel.CommunicationObjectFaultedException: The communication object, Microsoft.ServiceModel.Channels.Common.Channels.AdapterReplyChannel, cannot be used for communication because it is in the Faulted state.".

The adapter "WCF-SAP" raised an error message. Details "The WCF service host at address "sap://CLIENT=[SAPClientID];LANG=[LANG];@A/[SAPServer]/[SystemID]?ListenerGwServ=[GWServer]&ListenerGwHost=[GwHost]&ListenerProgramId=[ProgID]&RfcSdkTrace=True&AbapDebug=False" has faulted and as a result no more messages can be received on the corresponding receive location. To fix the issue, BizTalk Server will automatically attempt to restart the service host."

However this time Wireshark didn’t provide me additional information to help me diagnose and solve the problem.

CAUSE

Well I’m not an “SAP expert” neither I have the intension to become, I only want to know and understand the basic, so I can properly diagnose and resolve the different problems that may arise.

In this case, and sorry if I’m not giving all the precise technical details, the root cause of this problem is related to the Encoding type of the RFC Connection – Non-Unicode or Unicode (you may see the differences between them here)

Additional you can found some useful information in this two forum threads:

SOLUTION

You should keep this SAP transaction names in mind: SALE, SM59, WE20, WE21 and WE02 As my dear friend Nino Crudele mention in one of his posts, “in SAP exist thousands of transactions, the most important for us are, SALE, SM59, WE20, WE21, WE02.”

In this particular problem you should use SM59 transaction to check if all the configuration are set correctly like the: message type, program id, the channel and so on… but in special the Unicode settings

For that you should the following macro steps

  • Launch SAPGUI and login in the SAP System.
  • Execute the SM59 (Display and maintain RFC destinations) transaction
  • Select TCP/IP Connections option and pick RFC destination that we are using
    • Go to special options and check the "Character Width in Target System" in the Unicode tab.

SAPGUI-SM59-RFC-destination-TCP-IP-Connections-Unicode

Make sure the Unicode option is selected, otherwise to contact your SAP team and they must ensure that the RFC Connection is properly configured to use Unicode.

In my case, by changing the RFC Connection from non-unicode to unicode solved the problem that we were facing.

This week while migrating a demo, that I will present in BizTalk Summit 2015 – London event, from BizTalk Server 2013/Visual Studio 2012 to BizTalk Server 2013 R2/Visual Studio 2013 I found a strange behavior.

This is a simple demo where I have canonical schemas that will be transformed into different Schemas in order to perform 3 types of operations: Insert, Delete and Select in a custom SQL database using the WCF-SQL adapter and receive back the response.

However when I tried the demo I got the following exception in the Select operation:

“Microsoft.ServiceModel.Channels.Common.XmlReaderGenerationException: The columns FullName and Address are either duplicated or not in a sequence. Each column can only be selected one time, and columns must be selected in sequence.
at Microsoft.Adapters.Sql.SelectBodyWriter.OnWriteBodyContents(XmlDictionaryWriter writer)”

WCF-SQL-columns-are-either-duplicated-or-not-in-a-sequence

Nevertheless, everything was working fine in the old environment (BizTalk Server 2013). The select statement transformation is what we normally use daily in our projects, by just putting a "*" in the Columns element of the SQL Table Operation Select Schema

WCF-SQL-Select-Statement-transformation

CAUSE

The error message is clear and we can easily identify the problem but the reason why this strange behavior happens is not, and I really don’t know why or how it happened, but when I checked and compare the SQL Table Operation SelectResponse Schemas in both projects I realize that for some reason during the project migration Visual Studio changed the order of elements inside SelectResult as you can see in the picture bellow:

WCF-SQL-SelectResponse-Schemas-Comparation

And, of course, the SELECT * statement is respecting the correct order of the columns in the database witch is the order that we have in the left side in the picture above.

WCF-SQL-columns-are-either-duplicated-or-not-in-a-sequence-SQL-Table-Sctructure

SOLUTION

You have two options to solve this problem:

  • You can either rectify the order of the elements in your schema to respect the order that exist in the database
  • Or you need to change the value that you are passing to the Columns element inside your map to return the result in the right order of your schema.
    • In this case you need to change the value “*” with the following string value:
      • Email, Address, CitizenCard, ZipCode, PhoneNumber, FullName

WCF-SQL-Select-Statement-transformation-fixed

There is a lot of inconsistent and incorrect information about the BizTalk SAP Adapter (or in this case WCF-SAP Adapter) installation process and how it works – It is normal to see comments that the adapter only runs on 32-bit, or to see indications that we have to copy DLLs to two different places (System32 and SysWOW64) but which ones?

I hope with this step-by-step installation guide to clarify a bit and document a little better this task.

So, one of the key question that we need to clarify is: Does the WCF-SAP adapter runs under 64-bit Host Instance?

And the correct response to this question is: Yes, it does! The SAP adapter supports both the 32-bit and the 64-bit versions of the SAP RFC SDK, so it can run under 32 or 64-bit Host Instances.

The second important question that we need to clarify is that: Documentation specifies that we need to add the 32-bit version of the client DLLs for the SAP adapter to C:\Windows\SysWow64 folder and the 64-bit version of the DLLs must be added to C:\Windows\System32 folder, but what this means? Do need to add the same DLL in two different folders?

Indeed, the documentation is correct and we need to add both 32 and 64-bit to different folder in our system… but this are different DLL and no the same ones – so if you only have one SDK resource, for example the SAP RFC SDK 32-bit, you just need to add this into one of the folders, copy for both places doesn’t make any difference because the adapter will only run in this case under a 32-bit Host Instance.

For each SAP Client version (6.4, 7.0 and so on) there are two SAP RFC SDK one for 32-bit and one for 64-bit and you need to download and install both… just like the BizTalk Server Adapter Pack.

There are SAP UNICODE and NON-UNICODE, so witch version of the SAP RFC SDK should I download?

The SAP adapter requires Unicode version of the RFC SDK irrespective of whether the SAP system is Unicode or non-Unicode.

Official documentation states that only the following SAP server versions are supported:

  • SAP 7.2, SAP 7.0, SAP ERP 6.0 with EHP 4.0, SAP ECC 6.0 Unicode

However earlier version, which were mentioned in previous versions of the documentation, are also supported:

  • SAP ECC 5.0 Non-Unicode, SAP ECC 5.0 Unicode, SAP R/3 4.7 Non-Unicode, SAP R/3 4.7 Unicode, SAP R/3 4.6c Non-Unicode, SAP ERP 6.0 with EHP 4.0

Also official documentation states that the only supported client version is the SAP RFC SDK 7.2 UNICODE – I always recommend to use the latest version… however, once again, as previous versions of the documentation mentioned, these client versions are also supported:

  • SAP client 6.4 version: SAP RFC SDK 6.40 UNICODE
  • SAP client 7.0 version: SAP RFC SDK 7.00 UNICODE
  • SAP client 7.1 version: SAP RFC SDK 7.10 UNICODE
  • SAP client 7.11 version: SAP RFC SDK 7.11 UNICODE

I’m not a SAP expert but I think that all depends if the SAP client version is supported in your SAP System version. For example, in one BizTalk Server 2013 R2 environment I have the SAP client 7.2 version installed and in another one, a BizTalk Server 2013, I have the 7.0 and both are working fine.

And finally: Can I install only the 64-bit version?

No, you can’t and for the exact same reason that we need to install both version of the BizTalk Server Adapter Pack: Visual Studio and BizTalk Administration console are 32-bit applications, so in order to configure the adapter in runtime (Receive Locations or Send Ports that will be using the SAP Adapter) or to use it to generate the schemas in the development phase you always need to have installed the 32-bit version of the SAP RFC SDK.

The 64-bit version of the SAP RFC SDK is optional, but, if you have a 64-bit BizTalk environment and you want to run it under 64-bit Host Instance, then you need to also install both versions of the SAP RFC SDK.

Download Prerequisites

As happen with other LOB adapters (like Oracle for example), installing the BizTalk Adapter pack is not enough and you will need the addition resources that will be specify bellow.

Download SAP Resources from SAP Service Marketplace

You need to obtain a few resources from the SAP Service Marketplace in order to properly install WCF-SAP adapter:

  • SAP RFC SDK <version> UNICODE 32-bit
  • SAP RFC SDK <version> UNICODE 64-bit
  • R3DLLINST.zip containing Microsoft run-time DLLs
  • SAPCAR – SAPCAR is a compress utility (similar to WinZip, tar and so on), that is used by SAP to compress and decompress nearly all delivered files

Unfortunately, SAP Service Marketplace is restricted, so if you like to benefit from the content and services offered in the different portals of the SAP Service Marketplace, you need to request your personal User ID otherwise you need to ask your client or your SAP team to give you this resources.

In additional to the links above you may also find useful to give these two SAP notes to your client or SAP team:

Unlike R3DLLINST.zip, that is a unique resource, on the SAP Service Marketplace portal SAP RFC SDK page you will find several resources and we must understand exactly what we need, in this case for example purposes, I’m using the SAP RFC SDK 7.20 page:

SAP-Service-Marketplace-portal-SAP-RFC-SDK-7.20

Depending in the SAP client version you want to install this may change a little, but it is practically the same for the other versions, as you can see in the case of SAP RFC SDK 7.00 page

SAP-Service-Marketplace-portal-SAP-RFC-SDK

What you need to download is the:

  • Windows Server on IA32 32bit” –> this is the SAP RFC SDK 7.00 UNICODE 32-bit
  • And the “Windows on x64bit” –> SAP RFC SDK 7.00 UNICODE 64-bit

You also need to download SAPCAR to extract (unzip) the SAP resources, I think you may found this resource at service.sap.com/patches under:

  • Download, Support Packages and Patches, Entry by Application Group, Additional Components and then SAPCAR

SAP-Service-Marketplace-portal-SAPCAR

Extract SAP Resources

Once we have downloaded all this SAP resources, we need to use the SAPCAR to extract the SAP resources, in this case (SAP RFC SDK 7.20):

  • “RFC_12-10009744.SAR” is the 64-bit SDK that we want to extract to “rfcsdk 64” folder
  • “RFC_12-100097446.SAR” is the 32-bit SDK that we want to extract to “rfcsdk 32” folder

BizTalk-Server-SAP-Software-resources

To accomplished that we need to:

  • Open a command line prompt
  • Change our directory to the location where we saved the “SAPCAR.exe” and execute the following command to extract the the SAR archive:
    • SAPCAR.exe -xfv “RFC_12-10009744.SAR”
      • This will create a folder call “rfcsdk”, rename that to “rfcsdk 64”

Extract-SAR-archive-with-SAPCAR

    • SAPCAR.exe -xfv “RFC_12-10009746.SAR”
      • This will create a folder call “rfcsdk”, rename that to “rfcsdk 32”

The final step regarding to SAP Resource is to unzip the “r3dllinst.zip” file using for example an open source Windows utility for manipulating archives like 7-Zip.

Download Microsoft Visual C++ 2005 SP1 Redistributable Package

Microsoft Visual C++ run-time DLLs, again both 32 and 64-bit are required for SAP 7.11 client or higher. This resources are available from download in the following links:

download-Microsoft-Visual-C  _2005_SP1_Redistributable_Package

download-Microsoft-Visual-C  _2005_SP1_Redistributable_Package-files

Step-by-Step WCF-SAP Adapter installation guide

The first thing you need to make sure that you have installed already is both 32 and 64-bit versions of the Microsoft BizTalk Adapter Pack.

  • The Microsoft BizTalk Adapter Pack contains adapters that enable enterprise applications and databases to interface with each other by implementing a common adapter framework. Similar to programming to Web services, adapters enable clients to program to different enterprise applications. Technically, adapters are a binding to Windows Communication Framework (WCF). The BizTalk Adapter Pack consists of the following adapters:
    • Microsoft BizTalk Adapter for Oracle Database (Oracle Database adapter).
    • Microsoft BizTalk Adapter for Oracle E-Business Suite (Oracle E-Business adapter).
    • Microsoft BizTalk Adapter for mySAP Business Suite (SAP adapter). This also includes the .NET Framework Data Provider for mySAP Business Suite (Data Provider for SAP).
    • Microsoft BizTalk Adapter for Siebel eBusiness Applications (Siebel adapter). This also includes the .NET Framework Data Provider for Siebel eBusiness Applications (Data Provider for Siebel).
    • Microsoft BizTalk Adapter for SQL Server (SQL adapter).
  • Find more about the BizTalk Adapter Pack installation process in the following articles:

Notice that, at this point, if you try to create a receive or send port using the WCF-SAP adapter you will get the following error:

Exception has been thrown by the target of an invocation. (mscorlib)
Could not load file or assembly ‘Microsoft.Adapters.SAP.SAPGInvoker.dll’ or one of its dependencies. The specified module could not be found. (Microsoft.Adapters.SAP)

Could-not-load-file-assembly-Microsoft.Adapters.SAP.SAPGInvoker.dll

This is because we need the SAP resources required to run the adapter are not install (or are i installed incorrectly).

So, now that we have install the Microsoft BizTalk Adapter Pack, we need to install the SAP Resources that we previous download in our BizTalk Server machine:

  • Open the “rfcsdk 64” folder containing the SAP RFC SDK 7.20 UNICODE 64-bit and access to the lib folder
    • “C:\SAP Resources\rfcsdk 64\lib”

SAP-rfcsdk-7.2-64-bit-lib-Resources

  • Install the SAP RFC SDK 64-bit DLLs by coping (or drag) them to the “System32” folder in your system “Windows” folder
    • “C:\Windows\System32”:

SAP-rfcsdk-7.2-64-bit-lib-Resources-Installation

  • And now we need to do the a similar process to the Open the SAP RFC SDK 7.20 UNICODE 32-bit
  • For that you need to open the “rfcsdk 32” folder containing the SAP RFC SDK 7.20 UNICODE 32-bit and access to the lib folder
    • “C:\SAP Resources\rfcsdk 32\lib”

SAP-rfcsdk-7.2-32-bit-lib-Resources

  • Install the SAP RFC SDK 32-bit DLLs by coping (or drag) them to the “SysWOW64” folder in your system “Windows” folder
    • “C:\Windows\SysWOW64”:

SAP-rfcsdk-7.2-32-bit-lib-Resources-Installation

  • Open the “r3dllinst\ ntpatch” folder containing the content of “r3dllinst.zip” file, ,and we need to execute the “R3DLLINS.EXE” tool, in order to install the SAP R/3 DLLs (Microsoft run-time DLLs):
    • msvcr71.dll
    • msvcp71.dll
    • mfc71.dll
    • mfc71u.dll

install-SAP-r3dllinst-R3DLLINS

  • The tool specifies that the DLLs were installed under “C:\Windows\System32” folder, however because I’m using a 64-bit environment instead they were installed in “c:\Windows\SysWOW64” folder

install-SAP-r3dllinst-R3DLLINS-SysWOW64

    • You should check if DLL are present in “C:\Windows\System32”, if they are not I advise you to:
      • Leave them in the ” c:\Windows\SysWOW64” folder
      • And, just to be sure, copy the same four DLLs to “C:\Windows\System32” folder

install-SAP-r3dllinst-R3DLLINS-windows-folder

  • And finally we also need to installed Microsoft Visual C++ 2005 SP1 Redistributable Package – 32-bit and 64-bit – containing the Visual C++ run-time DLLs required for SAP 7.20 (or 7.11) client:
    • Install the Microsoft Visual C++ 2005 SP1 Redistributable Package (x86) – 32-bit – by executing the “Vcredist_x86.exe” file that we previous download
      • On the Microsoft Visual C++ 2005 SP1 Redistributable Package window, click “Yes”

Microsoft-Visual-C  -2005-SP1-Redistributable-Package-x86

    • Do the same steps to install the Microsoft Visual C++ 2005 SP1 Redistributable Package (x64) – 64-bit – by executing the “Vcredist_x64.exe” file that we previous download
      • On the Microsoft Visual C++ 2005 SP1 Redistributable Package window, click “Yes”
Add adapters to BizTalk Administration Console

As it happens with all adapters that we installed on our BizTalk Servers environment, before we can begin to use them we need to register or add the adapters. So in the next step it will be describing how can we add the enterprise adapters, or any other custom adapter, on the BizTalk Administration Console. To accomplish that we need to:

  • Open BizTalk Administration Console by pressing the “Windows key” to switch to the Start screen, type “BizTalk Server Administration” or “BizTalk”, click “BizTalk Server Administration” option from the Search menu
  • In the console left tree, expand “BizTalk Server Administration –> BizTalk Group –> Platform Settings” and then “Adapters”
  • Right-click on “Adapters” and add a new adapter by selecting the option “New –> Adapter”

BizTalk-Administration-Console-Register-WCF-SAP-adapter-add-new

  • In the “Adapter Properties” window
    • In the Name box, type a descriptive name for this adapter.
      • WCF-SAP
    • In the Adapter combo box, select the adapter from the drop-down that you want to add.
      • WCF-SAP
    • In the Description box, type a description for the adapter (this is optional).
      • The WCF-SAP adapter provides a rich metadata layer on top of SAP that enables the consumption of RFCs and enables seamless BAPI and IDOC exchange in an interoperable manner. The SAP adapter exposes the SAP system as a WCF service to client applications.

BizTalk-Administration-Console-Register-WCF-SAP-adapter

    • Click “OK” to complete the process of adding the adapter.
      • We will receive a warning message saying that we need to restart the Host Instances
      • Click “Ok”, we will deal with it in a moment

BizTalk-Administration-Console-Register-WCF-SAP-adapter-warning-message

  • Now we need to create other send and receive adapter handlers for the WCF-SAP Adapter, to accomplished that we need to:
  • Expanded the adapter list, right-click the “WCF-SAP” adapter, point to New, and then:
    • click “Send Handler to create a send handler”
    • Or click “Receive Handler” to create a receive handler.

BizTalk-Administration-Console-Register-WCF-SAP-adapter-add-receive-send-handlers

Notice: that by default the adapter already have one Receive and one Send handler that are associated to the default Host Instance – normally the “BizTalkServerApplication” Host Instance

  • Add two send handler using Dedicated Send Hosts
    • One running under 32-bit: BizTalkServerSend32Host
    • One running under 64-bit: BizTalkServerSendHost
  • Add two receive handler using Dedicated Receive Hosts
    • One running under 32-bit: BizTalkServerSend32Host
    • One running under 64-bit: BizTalkServerSendHost
  • Delete the default handlers that use the “BizTalkServerApplication” Host (in my case it was “BizTalkServerApplication64Host”)

BizTalk-Administration-Console-Register-WCF-SAP-adapter-handler-configuration

Note: Now more about Create and Configure BizTalk Server Host and Host Instances in my TechNet Wiki article: BizTalk Server Best Practices: Create and Configure BizTalk Server Host and Host Instances

  • Finally we just need to restart our BizTalk Server Host Instances, for that you need to:
    • Expand the BizTalk group, click “Platform Settings”, and then click “Host Instances”.
    • In the details pane, right-click the host instance you want to start, and then click “Start”.

BizTalk-Administration-Console-restart-host-instances-final-step

Validate if the WCF-SAP adapter is properly installed

To do a preliminary test you to your WCF-SAP Adapter installation you can create a new receive port:

  • Right-click “Receive Ports”, point to “New”, and click “One-way Receive Port”

SAP-validation-add-new-receive-port

  • Just leave the default name and click on the “Receive Locations” tab and then click “New”.

SAP-validation-add-new-receive-location

  • In the Receive Location Properties dialog box, do the following:
    • From the Type drop-down list, select the “WCF-SAP” adapter you added earlier, and then click “Configure”.

SAP-validation-add-new-receive-port-configure-receive-location

If you were able to access the WCF-SAP Transport Properties window and the URI SAP configuration windows, this is half way thru and a good indication that the adapter is properly configured in your environment

WCF-SAP-Transport Properties-adn-URI-Windows

Of course now you need to actually test it against your SAP environment to see if you actually can receive or send messages from or to SAP System.