This week while migrating a demo, that I will present in BizTalk Summit 2015 – London event, from BizTalk Server 2013/Visual Studio 2012 to BizTalk Server 2013 R2/Visual Studio 2013 I found a strange behavior.

This is a simple demo where I have canonical schemas that will be transformed into different Schemas in order to perform 3 types of operations: Insert, Delete and Select in a custom SQL database using the WCF-SQL adapter and receive back the response.

However when I tried the demo I got the following exception in the Select operation:

“Microsoft.ServiceModel.Channels.Common.XmlReaderGenerationException: The columns FullName and Address are either duplicated or not in a sequence. Each column can only be selected one time, and columns must be selected in sequence.
at Microsoft.Adapters.Sql.SelectBodyWriter.OnWriteBodyContents(XmlDictionaryWriter writer)”

WCF-SQL-columns-are-either-duplicated-or-not-in-a-sequence

Nevertheless, everything was working fine in the old environment (BizTalk Server 2013). The select statement transformation is what we normally use daily in our projects, by just putting a "*" in the Columns element of the SQL Table Operation Select Schema

WCF-SQL-Select-Statement-transformation

CAUSE

The error message is clear and we can easily identify the problem but the reason why this strange behavior happens is not, and I really don’t know why or how it happened, but when I checked and compare the SQL Table Operation SelectResponse Schemas in both projects I realize that for some reason during the project migration Visual Studio changed the order of elements inside SelectResult as you can see in the picture bellow:

WCF-SQL-SelectResponse-Schemas-Comparation

And, of course, the SELECT * statement is respecting the correct order of the columns in the database witch is the order that we have in the left side in the picture above.

WCF-SQL-columns-are-either-duplicated-or-not-in-a-sequence-SQL-Table-Sctructure

SOLUTION

You have two options to solve this problem:

  • You can either rectify the order of the elements in your schema to respect the order that exist in the database
  • Or you need to change the value that you are passing to the Columns element inside your map to return the result in the right order of your schema.
    • In this case you need to change the value “*” with the following string value:
      • Email, Address, CitizenCard, ZipCode, PhoneNumber, FullName

WCF-SQL-Select-Statement-transformation-fixed

BizTalk Health Monitor (BHM) is a BizTalk snap-in that can be added to the existing BizTalk Administration Console or can be run individually, that helps you monitor the health of your BizTalk Server environment. Basically it’s similar to the “old” BizTalk MsgBoxViewer tool that we used monitor a BizTalk environment by gathering all information of a BizTalk group and detecting any issues, non-critical or critical warnings to detect any potential problems in advance.

BHM is now in is third version, released in March 6, 2015 that you can download here and know more about it here.

The goal of this post is not to explain what it is the BizTalk Health Monitor, but instead if you already have a previous version – V1 or V2 – how can you update to the latest one – V3. Do you need to uninstall the previous one?

Note: If you don’t have a previous version of BizTalk Health Monitor installed then check the post to describe the installation steps required.

How to upgrade BizTalk Health Monitor to the latest version

Basically you don’t need to uninstall V2 (or V1) to upgrade to the latest version, you just need to register the snap-in of the new one but there are some consideration that you need to be aware.

  • First, if you have BizTalk Health Monitor integrated with the BizTalk Administration Console you need to close all the BizTalk Administration Console in all sessions, if not you need to close BizTalk Health Monitor in all sessions again, otherwise you cannot replace or delete the existing files.

Error-While-replacing-BHM-files

  • After closing all consoles, delete all the existing files under the BizTalk Health Monitor path in with the previous version was registered, for example:
    • “C:\Program Files (x86)\Microsoft BizTalk Server 2013 R2\SDK\Utilities\BizTalkHealthMonitor”
  • And unzip the version of the BHM zip file to this folder
    • You could also replace the existing files by the new ones, however I already encountered some issues while registering the new version of the snap-in after doing that
    • Or you can create a new folder with this new resources.

Note: Once you register the BizTalk Health Monitor Snap-In you cannot delete or change the folder path or BHM will stop working.

  • Open a command prompt as an administrator
  • Navigate to the directory folder where you have the BHM resources
    • For example: C:\Program Files (x86)\Microsoft BizTalk Server 2013 R2\SDK\Utilities\BizTalkHealthMonitor
  • And from the command prompt type:
    • InstallUtil.exe MBVSnapIn.dll
  • And hit enter. This step will register the new version of the Snap-In to be used

Because you already add the BizTalk Health Monitor integrated in the BizTalk Administration Console you don’t need to do nothing more. Just open the BizTalk Administration Console and the last version of the BHM will be available.

BizTalk-Administration-Console-BHM-V3

Notes:

  • If you replace the existing BHM files by the new ones and then try open the BizTalk Administration Console without registering the new BHM version you will receive the following error:
    • “MMC has detected an error in a snap-in and will unload it”

BHM-MMC-detected-error-snap-in-unload-it

  • If you replace the existing BHM files by the new ones and you find some kind of error while trying to register the snap-in then:
    • Delete all files from the directory and then copy again the files to the new BHM version to the folder
    • And register again the BHM span-in

Exciting news… Version 1.8.0.0 of BizTalk Mapper Extensions UtilityPack for BizTalk Server 2013 is now available!

But before talking about the change-log of this version there is another subject that I want to talk first… This project as two new members that was responsible for all the new functionalities available in this version:

  • Martijn Schiedon, Integration architect at ValueBlue (http://www.valueblue.nl) and a Dutch BizTalk community member.
  • And Salvatore Pellitteri. He is Developer Team Manager in Microsys Srl. He works on integration application projects using BizTalk Server and on business intelligence projects using SQL Server platform. He works with BizTalk Server since the first version and has worked in many big integration project with several Italian enterprise organizations using many technologies like SAP, Oracle, SQL Server, AS400, J.D. Edwards and HL7. Salvatore has his own blog https://pellitterisbiztalkblog.wordpress.com/ on which he shares his experiences.

Here’s the change-log for this release:

  • Seven new functoids:
    • MSCRM Map Helper Base Types Functoid
    • MSCRM Map Helper Guid Functoid
    • MSCRM Map Helper Money Functoid
    • MSCRM Map Helper Option Value Functoid
    • MSCRM Map Helper References Functoid
    • XPath Functoid
    • Add SharePoint 2013 Document Set Functoid

What’s new in this version?

Although BizTalk Server provides many functoids to support a range of diverse operations when working with conditional mapping, we are limited to the number of existing operations or otherwise have to use custom XSLT.

MSCRM Map Helper Base Types Functoid

Use this functoid when you want map CRM base data types such as “xs:string”, “xs:int” and so on.

Parameters

The functoid takes three mandatory input parameters:

  • Source Field: Link from source schema field value.
  • Target Field Name: CRM target field name.
  • Target Field Type: CRM target field type (Example: “xs:dateTime”).

The output of this functoid should be linked to:

  • “/<Schema>/<CRM_Operation>/entity/Attributes/KeyValuePairOfstringanyType/key”.
MSCRM Map Helper Guid Functoid

Use this functoid when you want map CRM guid type.

Parameters

The functoid takes two mandatory input parameters:

  • Source Field: Link from source schema field value.
  • Target Field Name: CRM target field name

The output of this functoid should be linked to:

  • “/<Schema>/<CRM_Operation>/entity/Attributes/KeyValuePairOfstringanyType/key”.

You may use this functoid in combination with CRM Lookup functoid, where first find for entity guid and then use the result to map to the destination schema.

MSCRM Map Helper Money Functoid

Use this functoid when you want map CRM money type.

Parameters

The functoid takes two mandatory input parameters:

  • Source Field: Link from source schema field value.
  • Target Field Name: CRM target field name.

The output of this functoid should be linked to:

  • “/<Schema>/<CRM_Operation>/entity/Attributes/KeyValuePairOfstringanyType/key”.
MSCRM Map Helper Option Value Functoid

Use this functoid when you want map CRM option value type.

Parameters

The functoid takes two mandatory input parameters:

  • Source Field: Link from source schema field value.
  • Target Field Name: CRM target field name.

The output of this functoid should be linked to:

  • “/<Schema>/<CRM_Operation>/entity/Attributes/KeyValuePairOfstringanyType/key”.
MSCRM Map Helper References Functoid

Use this functoid when you want map CRM reference type.

Parameters

The functoid takes three mandatory input parameters:

  • Source Field: Link from source schema field value.
  • Target Field Name: CRM target field name.
  • Entity Name: CRM referenced entity.

The output of this functoid should be linked to:

  • “/<Schema>/<CRM_Operation>/entity/Attributes/KeyValuePairOfstringanyType/key”.
MSCRM Map Helper Functoids

These functoids allow you to Integrate Directly with the Dynamics CRM SOAP API, avoiding the use of inline XSLT Call template script functoid.

MSCRM requires that different data types are specified in a different way. For this reason, we provide a set of functoid each for specific data type..

Use this functoids in two steps:

First Step

MSCRM-Map-Helper-Functoids-1

Second Step

MSCRM-Map-Helper-Functoids-2

XPath Functoid

This functoid natively integrates custom XPath queries in the BizTalk mapper. This unique functoid which is a powerful competitor to the ScripterFunctoid or a custom XSLT stylesheet and mixes exceptionally well with other functoids.

Parameters

The functoid takes one mandatory input parameter and one optional parameter:

  • The first parameter specifies an absolute or relative XPath expression, which becomes a native part of the generated XSLT.
  • The (optional) second parameter is a link to the source tree node that becomes the (looping) context for the XPath expression.

This defines the current context node for the XPath expression and allows the use of relative XPath expressions. Additionally, the mapper will generate a looping construct for that linked node where appropriate.

The functoid output can be linked to a destination schema node or serve as input to other functoids.

XPath-Custom-Functoids

See more information at The case of the forgotten Functoid – By Martijn Schiedon

Add SharePoint 2013 Document Set Functoid

Creates a Document Set in an existing SharePoint 2013 List. It is a requirement that the SharePoint list contains a custom content-type of type document set.

Parameters

The following parameters are required:

  • string siteUrl = SharePoint Site Url
  • string listName = SharePoint List Name containing the custom content-type Document Set
  • string docSetContentTypeName = Document Set Content Type name
  • string newDocSetName=Name of the to document set to created

Comment: An custom content-type of type documentset needs to be available in SharePoint.

You can found and download Source Code, Application Binaries and Documentation in CodePlex BizTalk Mapper Extensions UtilityPack home page:

BizTalk Mapper Extensions UtilityPack
CodePlex

 

or from MSDN Code Gallery:

BizTalk Mapper Extensions UtilityPack for BizTalk Server 2013 (628.8 KB)
Microsoft | MSDN Code Gallery

One of the principal needs for BizTalk Administrators is the ability to monitor the health of BizTalk environments on a regular basis and react promptly to solve any possible issues that may appear in order to keep your BizTalk Server applications accessible to your users/organization.

You can accomplished this by using certain tools such as: BizTalk Administration Console; BizTalk360; SCOM and many more… However, unfortunately many times, some of these tools are not available for us but we still need to accomplish this task.

However sometimes we don’t have access to some of these tools and in this case PowerShell is a good way to address and implement some of the basic monetization tasks or even complement and extend the existing tools.

Now I’m playing a little with PowerShell… following my last post and the series “Monitor your BizTalk environment using PowerShell”.

So how can PowerShell help us?

BizTalk Server writes all errors that occur in the environment in the Event Viewer Application Log.

So is important that you maintain this log clean of noise (custom application errors, warnings of information that you can use inside your code)

With this script you can be able to monitoring the Event Viewer for BizTalk Server related errors

This script allows you to set:

  • The timeframe that you want to monitor, the Entry Type and the Sources
$getEventLog = Get-Eventlog -log application -after ((get-date).AddHours(-4)) -EntryType Error | Where-Object {($_.Source -eq 'BizTalk Server')} 
  • And configure your email notification settings
#Set mail variables
[STRING]$PSEmailServer = "smtp"
[STRING]$Subject = "BizTalk Job Monitor"
[STRING]$From = "biztalk@monitor.com"
[array]$EmailTo = ("support@biztalk.com")

if($count -gt 0)
{
    #Send mail
    foreach ($to in $EmailTo)
    {
        $Body = $HTMLmessage
        $SMTPClient = New-Object Net.Mail.SmtpClient($PSEmailServer)
        $message = New-Object Net.Mail.MailMessage($from, $to, $Subject, $Body)
        $message.IsBodyHtml = $true;
        $SMTPClient.Send($message)
    }
}

Report sample:

BizTalk-Server-Event-Viewer-monitor

Note: This type of script must be viewed as a complement to the tools mentioned above or used in the absence of them.

The script can be found and download on Microsoft TechNet Gallery:
Monitoring Event Viewer Errors in your BizTalk environment with PowerShell (2.4 KB)
Microsoft TechNet Gallery

One of the principal needs for BizTalk Administrators is the ability to monitor the health of BizTalk environments on a regular basis and react promptly to solve any possible issues that may appear in order to keep your BizTalk Server applications accessible to your users/organization.

When monitoring BizTalk Server, keep these points in mind:

  • Your infrastructure could be healthy, but your applications might not be (for example, they are receiving invalid messages and are unable to process them).
  • Your infrastructure could be unhealthy, but your applications might be running fine (for example, if a server is down, but there are enough servers assigned to the host to take over the load).
  • An infrastructure problem could surface as an application problem (for example, messages are not being processed fast enough because a server is down).

Source: Monitoring BizTalk Server

You can accomplished this by using certain tools such as: BizTalk Administration Console; BizTalk360; SCOM and many more… However, unfortunately many times, some of these tools are not available for us but we still need to accomplish this task.

Since BizTalk Server 2010 the product brings a job (Monitor BizTalk Server) that monitors the health of your environment identifying any known issues:

  • Messages without any references
  • Messages without reference counts
  • Messages with reference count less than 0
  • Message references without spool rows
  • Message references without instances
  • Instance state without instances
  • Instance subscriptions without corresponding instances
  • Orphaned DTA service instances
  • Orphaned DTA service instance exceptions
  • TDDS is not running on any host instance when global tracking option is enabled.

However this is not enough…

So how can PowerShell help us?

In some of my previous post:

I demonstrated how we could use PowerShell to monitor some aspects or features of your BizTalk environment including a script to monitor BizTalk Jobs, however this script will only monitor if the last execution of the job was successfully or not and if not send a notification.

Although this is a useful, at least in my opinion, in the last two month I found three additional situations that are important to monitor:

  • Some of the jobs were disable
  • MessageBox_Message_Cleanup_BizTalkMsgBoxDb was enable
  • SQL Server Agent was running, BizTalk Jobs were correctly enable… but despite all that no job was running (the last execution has been 7 days back – coinciding with an intervention in the system)

So I decided to create another PowerShell script. With this script you can be able to monitoring SQL Agent Jobs in your BizTalk environment using PowerShell, checking:

  • If the jobs are being executed according to the schedulers that are configured
  • If all jobs (with the exception of MessageBox_Message_Cleanup_BizTalkMsgBoxDb) are enable
  • If MessageBox_Message_Cleanup_BizTalkMsgBoxDb is disable

This script allows you to set:

  • The Jobs execution timeframe and configurations
WITH MostRecentSched AS
 (
 -- For each job get the most recent scheduled run date (this will be the one where Rnk=1)
 SELECT job_id,
		last_executed_step_date,
		RANK() OVER (PARTITION BY job_id ORDER BY last_executed_step_date DESC) AS Rnk
 FROM sysjobactivity
 )
 select name [Job Name],
	last_executed_step_date [Last Scheduled Run Date],
        DATEDIFF(minute, last_executed_step_date, SYSDATETIME()) AS [Minutes Delayed],
		CASE WHEN enabled=1 THEN 'Enabled'
          ELSE 'Disabled'
        END [Job Status]
from MostRecentSched MRS
JOIN   sysjobs SJ
ON     MRS.job_id=SJ.job_id
where Rnk=1
and ((
		((name = 'Backup BizTalk Server (BizTalkMgmtDb)' and DATEDIFF(minute, last_executed_step_date, SYSDATETIME()) > 17))
		OR (name = 'CleanupBTFExpiredEntriesJob_BizTalkMgmtDb' and DATEDIFF(minute, last_executed_step_date, SYSDATETIME()) > 722)
		OR (name = 'Monitor BizTalk Server (BizTalkMgmtDb)' and DATEDIFF(minute, last_executed_step_date, SYSDATETIME()) > 11520)
		OR (name = 'Rules_Database_Cleanup_BizTalkRuleEngineDb' and DATEDIFF(minute, last_executed_step_date, SYSDATETIME()) > 62)
		OR (name = 'MessageBox_UpdateStats_BizTalkMsgBoxDb' and DATEDIFF(minute, last_executed_step_date, SYSDATETIME()) > 7)
		OR (name IN ('DTA Purge and Archive (BizTalkDTADb)','MessageBox_DeadProcesses_Cleanup_BizTalkMsgBoxDb', 'MessageBox_Parts_Cleanup_BizTalkMsgBoxDb', 'Operations_OperateOnInstances_OnMaster_BizTalkMsgBoxDb', 'PurgeSubscriptionsJob_BizTalkMsgBoxDb', 'TrackedMessages_Copy_BizTalkMsgBoxDb') and DATEDIFF(minute, last_executed_step_date, SYSDATETIME()) > 1)
	) OR (name IN ('Backup BizTalk Server (BizTalkMgmtDb)','CleanupBTFExpiredEntriesJob_BizTalkMgmtDb',
	               'DTA Purge and Archive (BizTalkDTADb)', 'MessageBox_DeadProcesses_Cleanup_BizTalkMsgBoxDb',
				   'MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb', 'MessageBox_Parts_Cleanup_BizTalkMsgBoxDb',
				   'MessageBox_UpdateStats_BizTalkMsgBoxDb', 'Monitor BizTalk Server (BizTalkMgmtDb)',
				   'Operations_OperateOnInstances_OnMaster_BizTalkMsgBoxDb',
				   'PurgeSubscriptionsJob_BizTalkMsgBoxDb', 'Rules_Database_Cleanup_BizTalkRuleEngineDb',
				   'TrackedMessages_Copy_BizTalkMsgBoxDb') AND SJ.enabled = 0)
	  OR (name = 'MessageBox_Message_Cleanup_BizTalkMsgBoxDb' AND SJ.enabled = 1)
	)
order by name, last_executed_step_date desc;
  • And configure your email notification settings
#Set mail variables
[STRING]$PSEmailServer = "smtp"
[STRING]$Subject = "BizTalk Job Monitor"
[STRING]$From = "biztalk@monitor.com"
[array]$EmailTo = ("support@biztalk.com")

if($count -gt 0)
{
    #Send mail
    foreach ($to in $EmailTo)
    {
        $Body = $HTMLmessage
        $SMTPClient = New-Object Net.Mail.SmtpClient($PSEmailServer)
        $message = New-Object Net.Mail.MailMessage($from, $to, $Subject, $Body)
        $message.IsBodyHtml = $true;
        $SMTPClient.Send($message)
    }
}

Report sample:

BizTalk-SQL-Jobs-another-way-monitorNote: This type of script must be viewed as a complement to the tools mentioned above or used in the absence of them.

Credits: Special thanks to José Dias who helped me developing this SQL Script.

The script can be found and download on Microsoft TechNet Gallery:
Another way to monitor BizTalk SQL Agent Jobs with PowerShell (2.8 KB)
Microsoft TechNet Gallery

Almost one month after I have released, BizTalk Scheduled Task Adapter suffered its first update in order to fix one important bug.

One of the important bug that was fixed in the previous version (5.0) was the support for timespan with more than 60 seconds, but while doing that I unintentionally create another bug.

  • The adapter was not respecting Start time property defined in the Schedule property page after restarting the Host Instance, instead it was using the Host Instance date time.

The time interval defined in the Schedule property page of the receive location must respect the Start time property defined there (and is now fixed and restored the correct behavior in this minor version)

BTS-Scheduled-tasl-adapter-5.0.4-properties

The expected behavior of the BizTalk Scheduled Task Adapter in this case is:

  • The first message will be triggered at 16:55;
  • The second at 17:00;
  • Third at 17:05… 17:10, 17:15 and so on.

However if I stop the host instance, let’s say at 17:18, and start the host instance again at 17:21, the expected behavior of the Scheduled Task Adapter is to:

  • The first message will be triggered at 17:25, because is the next 5 min interval based in start date defined in the properties, then 17:30 and so on.

 

Once again, this bug is now fixed and restored the correct behavior in this new minor version.

You can download this new version of the adapter in BizTalk Scheduled Task Adapter CodePlex project page:

V5.0.4 Change log

  • Bug fixes
    • Bug: TimeSchedule didn’t respect the start date defined in the receive location, instead it was using the host instance starting date after a restart – Solved

Release History

This adapter is available since BizTalk Server 2004.

  • Release 5.0.4: minor release for BizTalk Server 2013 R2 in March 16 2015 by Sandro Pereira: bug fixes.
  • Release 5.0: release in February 18 2015 by Sandro Pereira, this adapter was tested to work on BizTalk Server 2013 R2. Compiled in .NET Framework 4.5
  • Release 4.0: release in June 12 2012 by Sandro Pereira, this adapter was tested to work on BizTalk Server 2010. Compiled in .NET Framework 4.0
  • Release 3.0: release in Aug 10 2010 by Greg Forsythe, this adapter was tested to work on BizTalk Server 2009. Compiled in .NET Framework 2.0
  • Release 2.0: last release in Apr 20 2008 by Greg Forsythe, this adapter works with BizTalk Server 2006 and BizTalk Server 2006 R2. Compiled in .NET Framework 2.0
  • Release 1.02: last release in Apr 20 2008 by Greg Forsythe, this adapter works with BizTalk Server 2004, BizTalk Server 2006 and BizTalk Server 2006 R2. Compiled in .NET Framework 1.1

Note: The version change from V5.0 to V5.0.4 because some internal versions where made

In my last post I described Top 7 Reasons to attend BizTalk Summit London 2015 event and as I mention there this year this annual event will be even bigger and better! There are already 250 registrations (based on what I heard from the speaker update email) and we will have a capacity for 500 attendees, so if you are working in Microsoft Integration (Microsoft’s evolving (cloud) application platform, BizTalk Server on-premise and as IaaS, and many more), BizTalk Summit London 2015 is an event that you must attend!

The event will be held in April 13-14 at ExCeL London Exhibition and Convention Centre and once again is being organized by BizTalk360 in conjunction with Microsoft and the BizTalk Product group. Please visit this link for registration information.

sandro-speakers-badges

My topic is about “BizTalk Server tips and tricks for developers and admins”.  You can read the details below.

Session Abstract

It’s critical to use good tools and techniques to produce working solutions as quickly as possible and at the same time, given the increase the requirements and number of applications organizations develop today. But at the same time, it’s also critical to maintain the health of the entire platform.

In this session, which I’ll try to be a very interactive session (so be prepare to participate), I’ll address and share some useful BizTalk Server Tips and Tricks (and Workarounds) both for developers and administrators. Covering some topics like RosettaNet, SAP, database maintenance, debatching, out-of-the-box pipelines vs custom pipelines and many more.

I hope that, like me, you find it an interesting topic! See you there!

Hard to imagine that on 16th January, 2013 the “BizTalk Crew” (Steef-Jan Wiggers, Nino Crudele, Saravana Kumar, Tord Glad Nordahl and me) were starting one of the most important Integration event in the world with bit of help from the BizTalk product group (Guru Venkataraman and Ravi Krishnaswamy) The event was a huge success that time with 128 attendees from 72 companies across 15 countries.

1thEdition

Then Saravana Kumar and is his team (BizTalk360) took care of it and was able to take it to the next level! And what an amazing job they have done (doing..)

Last year BizTalk360 team took the next step by converting the one day event into two days event with 12 Microsoft Integration MVP’s from across the world and 2 Microsoft Product group members. 200 attendees and more than 100 different companies across 20 countries were present in this event. These are some of the numbers that reflect the increase and the huge success of this event! Everyone I spoke to during the event was extremely excited and appreciated the work put together by BizTalk360 team in organizing such a focused event for Microsoft Integration professionals, which was missing for a while.

So what to expect this year?

Well the answer is easy… Even bigger and better! The event will be held in ExCeL London Exhibition and Convention Centre (used in the 2012 Summer Olympics in London) and we will have a capacity for… do you want to guess?… 500 people! From what I heard from the speaker update email, there are already 250 registrations, and still 5 more weeks to go.

Let’s make the goal a reality! And create a platform for ourselves who are working in Integration space. From my experience I wanted to highlight why you should attend the event, I took top 7 reasons I can think, of course you know there are more, to attend this event:

reason1

Learn from the experts

The #1 reason people should come to this event, is to make themselves more valuable to their companies by learning new ideas and techniques from the experts. 18 speakers (combination of Microsoft product group members and Microsoft Integration MVP’s) with real world implementation experience from around the world will provide valuable information on integration topics, new trends and technologies that will benefit your firm and your clients.

360 coverage! There is something for everyone regardless of job role: developers, administrators and architects, or interest: Cloud an on premise

Get insight and answers to your questions from these real world experts.

reason2

Imagine the future of Integration

Understand the future of Microsoft integration and its role in the "Internet of Things". Microsoft public annouce BizTalk MicroService in Integrate 2014 – Global BizTalk Summit which was held last December in Redmond so it is expected to be announced new updates, we never know what to expect from Product Group they maybe have some surprises for us.

reason3

Network, Connect, and Reconnect with Colleagues

Take this amazing opportunity to get to meet the people you have been following on Twitter and blogs and network with others who are interested in the same things you are. For return attendees, another chance to hang out with the smartest people you know – and I’m not talking about the speakers! J – and meet new ones.

Don’t be afraid or shy, don’t wait for them and take your chances, engage the people you want to meet by easily saying "Hi my name is… ". This experience can be a great morale booster for you, lifelong friendships and connections have evolved from such conferences – I’m speaking for personal experience!

reason4

Face to Face, learn from one another and Develop Professionally

The knowledge and experience of all the attendees, speakers and product group members at these events is unreal!!! You will not find an opportunity like this every day.

At these events speakers are usually available to chat and answer questions. And if you are bold, remember to pull out your camera and ask if you can take a photo with them.

Take to this opportunity to connect face to face with experts and peers who will give you valuable insight to industry trends; meet influential colleagues and generate new solutions to common problems, like increasing profitability and solving staffing challenges.

I love being around great thinkers…it can really get your own mind flowing. We always have something new to learn from one another and take back with us.

reason5

The next big thing…

Last year Nino introduce us the best BizTalk Developer tool ever made: BTSG NoS addin. A tool intended to improve the developer experience and why not reduce the development time in new or existent BizTalk projects. BizTalk360 team has some surprise announcements in this area.

So what to expect this year? Will Nino surprise us once again, or is going to be another speaker with the next big “Yehhhh” by the attendees. Or you may be inspired to produce yourself, the best ideas come when you least expect it….

reason6

Business Networking and Opportunity for Partnerships

Socializing is fun but this audience is made up by experts and minded people, meeting them not only inspires but it can also create an opportunity to build partnerships. There will be people from all over the world, this can be a huge opportunity for you and your business.

Bring business cards. Lots of them.

reason7

Refresh and recharge and have some fun

Take these two days off busy work schedules to learn, refresh and recharge. And most important: Have some fun!

Once you’ve soaked in all the knowledge from the speakers and breakout sessions your day will be far from over. Use the evenings to connect with colleagues and experts in a relaxed fun environment like a Bar or speaker dinner or restaurants – sometimes the most interesting conversations happen at the very end of each day in the Lobby Bar.

And then return to work reinvigorated.

This is one conference you can’t afford to miss! See you in London. What are you waiting for, go ahead and book it.

BSL2015-register

There is a lot of inconsistent and incorrect information about the BizTalk SAP Adapter (or in this case WCF-SAP Adapter) installation process and how it works – It is normal to see comments that the adapter only runs on 32-bit, or to see indications that we have to copy DLLs to two different places (System32 and SysWOW64) but which ones?

I hope with this step-by-step installation guide to clarify a bit and document a little better this task.

So, one of the key question that we need to clarify is: Does the WCF-SAP adapter runs under 64-bit Host Instance?

And the correct response to this question is: Yes, it does! The SAP adapter supports both the 32-bit and the 64-bit versions of the SAP RFC SDK, so it can run under 32 or 64-bit Host Instances.

The second important question that we need to clarify is that: Documentation specifies that we need to add the 32-bit version of the client DLLs for the SAP adapter to C:\Windows\SysWow64 folder and the 64-bit version of the DLLs must be added to C:\Windows\System32 folder, but what this means? Do need to add the same DLL in two different folders?

Indeed, the documentation is correct and we need to add both 32 and 64-bit to different folder in our system… but this are different DLL and no the same ones – so if you only have one SDK resource, for example the SAP RFC SDK 32-bit, you just need to add this into one of the folders, copy for both places doesn’t make any difference because the adapter will only run in this case under a 32-bit Host Instance.

For each SAP Client version (6.4, 7.0 and so on) there are two SAP RFC SDK one for 32-bit and one for 64-bit and you need to download and install both… just like the BizTalk Server Adapter Pack.

There are SAP UNICODE and NON-UNICODE, so witch version of the SAP RFC SDK should I download?

The SAP adapter requires Unicode version of the RFC SDK irrespective of whether the SAP system is Unicode or non-Unicode.

Official documentation states that only the following SAP server versions are supported:

  • SAP 7.2, SAP 7.0, SAP ERP 6.0 with EHP 4.0, SAP ECC 6.0 Unicode

However earlier version, which were mentioned in previous versions of the documentation, are also supported:

  • SAP ECC 5.0 Non-Unicode, SAP ECC 5.0 Unicode, SAP R/3 4.7 Non-Unicode, SAP R/3 4.7 Unicode, SAP R/3 4.6c Non-Unicode, SAP ERP 6.0 with EHP 4.0

Also official documentation states that the only supported client version is the SAP RFC SDK 7.2 UNICODE – I always recommend to use the latest version… however, once again, as previous versions of the documentation mentioned, these client versions are also supported:

  • SAP client 6.4 version: SAP RFC SDK 6.40 UNICODE
  • SAP client 7.0 version: SAP RFC SDK 7.00 UNICODE
  • SAP client 7.1 version: SAP RFC SDK 7.10 UNICODE
  • SAP client 7.11 version: SAP RFC SDK 7.11 UNICODE

I’m not a SAP expert but I think that all depends if the SAP client version is supported in your SAP System version. For example, in one BizTalk Server 2013 R2 environment I have the SAP client 7.2 version installed and in another one, a BizTalk Server 2013, I have the 7.0 and both are working fine.

And finally: Can I install only the 64-bit version?

No, you can’t and for the exact same reason that we need to install both version of the BizTalk Server Adapter Pack: Visual Studio and BizTalk Administration console are 32-bit applications, so in order to configure the adapter in runtime (Receive Locations or Send Ports that will be using the SAP Adapter) or to use it to generate the schemas in the development phase you always need to have installed the 32-bit version of the SAP RFC SDK.

The 64-bit version of the SAP RFC SDK is optional, but, if you have a 64-bit BizTalk environment and you want to run it under 64-bit Host Instance, then you need to also install both versions of the SAP RFC SDK.

Download Prerequisites

As happen with other LOB adapters (like Oracle for example), installing the BizTalk Adapter pack is not enough and you will need the addition resources that will be specify bellow.

Download SAP Resources from SAP Service Marketplace

You need to obtain a few resources from the SAP Service Marketplace in order to properly install WCF-SAP adapter:

  • SAP RFC SDK <version> UNICODE 32-bit
  • SAP RFC SDK <version> UNICODE 64-bit
  • R3DLLINST.zip containing Microsoft run-time DLLs
  • SAPCAR – SAPCAR is a compress utility (similar to WinZip, tar and so on), that is used by SAP to compress and decompress nearly all delivered files

Unfortunately, SAP Service Marketplace is restricted, so if you like to benefit from the content and services offered in the different portals of the SAP Service Marketplace, you need to request your personal User ID otherwise you need to ask your client or your SAP team to give you this resources.

In additional to the links above you may also find useful to give these two SAP notes to your client or SAP team:

Unlike R3DLLINST.zip, that is a unique resource, on the SAP Service Marketplace portal SAP RFC SDK page you will find several resources and we must understand exactly what we need, in this case for example purposes, I’m using the SAP RFC SDK 7.20 page:

SAP-Service-Marketplace-portal-SAP-RFC-SDK-7.20

Depending in the SAP client version you want to install this may change a little, but it is practically the same for the other versions, as you can see in the case of SAP RFC SDK 7.00 page

SAP-Service-Marketplace-portal-SAP-RFC-SDK

What you need to download is the:

  • Windows Server on IA32 32bit” –> this is the SAP RFC SDK 7.00 UNICODE 32-bit
  • And the “Windows on x64bit” –> SAP RFC SDK 7.00 UNICODE 64-bit

You also need to download SAPCAR to extract (unzip) the SAP resources, I think you may found this resource at service.sap.com/patches under:

  • Download, Support Packages and Patches, Entry by Application Group, Additional Components and then SAPCAR

SAP-Service-Marketplace-portal-SAPCAR

Extract SAP Resources

Once we have downloaded all this SAP resources, we need to use the SAPCAR to extract the SAP resources, in this case (SAP RFC SDK 7.20):

  • “RFC_12-10009744.SAR” is the 64-bit SDK that we want to extract to “rfcsdk 64” folder
  • “RFC_12-100097446.SAR” is the 32-bit SDK that we want to extract to “rfcsdk 32” folder

BizTalk-Server-SAP-Software-resources

To accomplished that we need to:

  • Open a command line prompt
  • Change our directory to the location where we saved the “SAPCAR.exe” and execute the following command to extract the the SAR archive:
    • SAPCAR.exe -xfv “RFC_12-10009744.SAR”
      • This will create a folder call “rfcsdk”, rename that to “rfcsdk 64”

Extract-SAR-archive-with-SAPCAR

    • SAPCAR.exe -xfv “RFC_12-10009746.SAR”
      • This will create a folder call “rfcsdk”, rename that to “rfcsdk 32”

The final step regarding to SAP Resource is to unzip the “r3dllinst.zip” file using for example an open source Windows utility for manipulating archives like 7-Zip.

Download Microsoft Visual C++ 2005 SP1 Redistributable Package

Microsoft Visual C++ run-time DLLs, again both 32 and 64-bit are required for SAP 7.11 client or higher. This resources are available from download in the following links:

download-Microsoft-Visual-C  _2005_SP1_Redistributable_Package

download-Microsoft-Visual-C  _2005_SP1_Redistributable_Package-files

Step-by-Step WCF-SAP Adapter installation guide

The first thing you need to make sure that you have installed already is both 32 and 64-bit versions of the Microsoft BizTalk Adapter Pack.

  • The Microsoft BizTalk Adapter Pack contains adapters that enable enterprise applications and databases to interface with each other by implementing a common adapter framework. Similar to programming to Web services, adapters enable clients to program to different enterprise applications. Technically, adapters are a binding to Windows Communication Framework (WCF). The BizTalk Adapter Pack consists of the following adapters:
    • Microsoft BizTalk Adapter for Oracle Database (Oracle Database adapter).
    • Microsoft BizTalk Adapter for Oracle E-Business Suite (Oracle E-Business adapter).
    • Microsoft BizTalk Adapter for mySAP Business Suite (SAP adapter). This also includes the .NET Framework Data Provider for mySAP Business Suite (Data Provider for SAP).
    • Microsoft BizTalk Adapter for Siebel eBusiness Applications (Siebel adapter). This also includes the .NET Framework Data Provider for Siebel eBusiness Applications (Data Provider for Siebel).
    • Microsoft BizTalk Adapter for SQL Server (SQL adapter).
  • Find more about the BizTalk Adapter Pack installation process in the following articles:

Notice that, at this point, if you try to create a receive or send port using the WCF-SAP adapter you will get the following error:

Exception has been thrown by the target of an invocation. (mscorlib)
Could not load file or assembly ‘Microsoft.Adapters.SAP.SAPGInvoker.dll’ or one of its dependencies. The specified module could not be found. (Microsoft.Adapters.SAP)

Could-not-load-file-assembly-Microsoft.Adapters.SAP.SAPGInvoker.dll

This is because we need the SAP resources required to run the adapter are not install (or are i installed incorrectly).

So, now that we have install the Microsoft BizTalk Adapter Pack, we need to install the SAP Resources that we previous download in our BizTalk Server machine:

  • Open the “rfcsdk 64” folder containing the SAP RFC SDK 7.20 UNICODE 64-bit and access to the lib folder
    • “C:\SAP Resources\rfcsdk 64\lib”

SAP-rfcsdk-7.2-64-bit-lib-Resources

  • Install the SAP RFC SDK 64-bit DLLs by coping (or drag) them to the “System32” folder in your system “Windows” folder
    • “C:\Windows\System32”:

SAP-rfcsdk-7.2-64-bit-lib-Resources-Installation

  • And now we need to do the a similar process to the Open the SAP RFC SDK 7.20 UNICODE 32-bit
  • For that you need to open the “rfcsdk 32” folder containing the SAP RFC SDK 7.20 UNICODE 32-bit and access to the lib folder
    • “C:\SAP Resources\rfcsdk 32\lib”

SAP-rfcsdk-7.2-32-bit-lib-Resources

  • Install the SAP RFC SDK 32-bit DLLs by coping (or drag) them to the “SysWOW64” folder in your system “Windows” folder
    • “C:\Windows\SysWOW64”:

SAP-rfcsdk-7.2-32-bit-lib-Resources-Installation

  • Open the “r3dllinst\ ntpatch” folder containing the content of “r3dllinst.zip” file, ,and we need to execute the “R3DLLINS.EXE” tool, in order to install the SAP R/3 DLLs (Microsoft run-time DLLs):
    • msvcr71.dll
    • msvcp71.dll
    • mfc71.dll
    • mfc71u.dll

install-SAP-r3dllinst-R3DLLINS

  • The tool specifies that the DLLs were installed under “C:\Windows\System32” folder, however because I’m using a 64-bit environment instead they were installed in “c:\Windows\SysWOW64” folder

install-SAP-r3dllinst-R3DLLINS-SysWOW64

    • You should check if DLL are present in “C:\Windows\System32”, if they are not I advise you to:
      • Leave them in the ” c:\Windows\SysWOW64” folder
      • And, just to be sure, copy the same four DLLs to “C:\Windows\System32” folder

install-SAP-r3dllinst-R3DLLINS-windows-folder

  • And finally we also need to installed Microsoft Visual C++ 2005 SP1 Redistributable Package – 32-bit and 64-bit – containing the Visual C++ run-time DLLs required for SAP 7.20 (or 7.11) client:
    • Install the Microsoft Visual C++ 2005 SP1 Redistributable Package (x86) – 32-bit – by executing the “Vcredist_x86.exe” file that we previous download
      • On the Microsoft Visual C++ 2005 SP1 Redistributable Package window, click “Yes”

Microsoft-Visual-C  -2005-SP1-Redistributable-Package-x86

    • Do the same steps to install the Microsoft Visual C++ 2005 SP1 Redistributable Package (x64) – 64-bit – by executing the “Vcredist_x64.exe” file that we previous download
      • On the Microsoft Visual C++ 2005 SP1 Redistributable Package window, click “Yes”
Add adapters to BizTalk Administration Console

As it happens with all adapters that we installed on our BizTalk Servers environment, before we can begin to use them we need to register or add the adapters. So in the next step it will be describing how can we add the enterprise adapters, or any other custom adapter, on the BizTalk Administration Console. To accomplish that we need to:

  • Open BizTalk Administration Console by pressing the “Windows key” to switch to the Start screen, type “BizTalk Server Administration” or “BizTalk”, click “BizTalk Server Administration” option from the Search menu
  • In the console left tree, expand “BizTalk Server Administration –> BizTalk Group –> Platform Settings” and then “Adapters”
  • Right-click on “Adapters” and add a new adapter by selecting the option “New –> Adapter”

BizTalk-Administration-Console-Register-WCF-SAP-adapter-add-new

  • In the “Adapter Properties” window
    • In the Name box, type a descriptive name for this adapter.
      • WCF-SAP
    • In the Adapter combo box, select the adapter from the drop-down that you want to add.
      • WCF-SAP
    • In the Description box, type a description for the adapter (this is optional).
      • The WCF-SAP adapter provides a rich metadata layer on top of SAP that enables the consumption of RFCs and enables seamless BAPI and IDOC exchange in an interoperable manner. The SAP adapter exposes the SAP system as a WCF service to client applications.

BizTalk-Administration-Console-Register-WCF-SAP-adapter

    • Click “OK” to complete the process of adding the adapter.
      • We will receive a warning message saying that we need to restart the Host Instances
      • Click “Ok”, we will deal with it in a moment

BizTalk-Administration-Console-Register-WCF-SAP-adapter-warning-message

  • Now we need to create other send and receive adapter handlers for the WCF-SAP Adapter, to accomplished that we need to:
  • Expanded the adapter list, right-click the “WCF-SAP” adapter, point to New, and then:
    • click “Send Handler to create a send handler”
    • Or click “Receive Handler” to create a receive handler.

BizTalk-Administration-Console-Register-WCF-SAP-adapter-add-receive-send-handlers

Notice: that by default the adapter already have one Receive and one Send handler that are associated to the default Host Instance – normally the “BizTalkServerApplication” Host Instance

  • Add two send handler using Dedicated Send Hosts
    • One running under 32-bit: BizTalkServerSend32Host
    • One running under 64-bit: BizTalkServerSendHost
  • Add two receive handler using Dedicated Receive Hosts
    • One running under 32-bit: BizTalkServerSend32Host
    • One running under 64-bit: BizTalkServerSendHost
  • Delete the default handlers that use the “BizTalkServerApplication” Host (in my case it was “BizTalkServerApplication64Host”)

BizTalk-Administration-Console-Register-WCF-SAP-adapter-handler-configuration

Note: Now more about Create and Configure BizTalk Server Host and Host Instances in my TechNet Wiki article: BizTalk Server Best Practices: Create and Configure BizTalk Server Host and Host Instances

  • Finally we just need to restart our BizTalk Server Host Instances, for that you need to:
    • Expand the BizTalk group, click “Platform Settings”, and then click “Host Instances”.
    • In the details pane, right-click the host instance you want to start, and then click “Start”.

BizTalk-Administration-Console-restart-host-instances-final-step

Validate if the WCF-SAP adapter is properly installed

To do a preliminary test you to your WCF-SAP Adapter installation you can create a new receive port:

  • Right-click “Receive Ports”, point to “New”, and click “One-way Receive Port”

SAP-validation-add-new-receive-port

  • Just leave the default name and click on the “Receive Locations” tab and then click “New”.

SAP-validation-add-new-receive-location

  • In the Receive Location Properties dialog box, do the following:
    • From the Type drop-down list, select the “WCF-SAP” adapter you added earlier, and then click “Configure”.

SAP-validation-add-new-receive-port-configure-receive-location

If you were able to access the WCF-SAP Transport Properties window and the URI SAP configuration windows, this is half way thru and a good indication that the adapter is properly configured in your environment

WCF-SAP-Transport Properties-adn-URI-Windows

Of course now you need to actually test it against your SAP environment to see if you actually can receive or send messages from or to SAP System.

In my opinion there are a lack of documentation, or good documentation and many of the existing one is obsolete, regarding some topics/feature of BizTalk Server – Microsoft BizTalk Accelerator for RosettaNet is one of them.

Today while I was trying to extend Microsoft BizTalk 2013 Accelerator for RosettaNet (BTARN) with a new receive Partner Interface Process (PIP) schema I was receiving the following error in the event log:

Source module:
RNDisAssembler

Correlation information:

Description:
Receive pipeline rejected incoming message due to the following RNIF exception:
UNP.SCON.VALERR: A failure occurred while validating the service content.

Details:
The document specification <MyNamespace.MyPIP> from assembly <MyAssemblyName, Version=1.0.0.0, Culture=neutral, PublicKeyToken=…> failed to load. Verify the schema for this document specification is deployed and is in the Global Assembly Cache.

Note: this same error can occur in previous or new versions of BizTalk Server.

Based on the received error and according to a possible cause described by Microsoft at Troubleshooting: Microsoft BizTalk Accelerator for RosettaNet Issues and Resolutions either the document namespace or the root node property the deployed schema for the instance that you are trying to extend is incorrect.

In fact this could be the issue but I found really odd because in the error details my DLL, containing the schema that I was trying to extend, was being properly referenced, nevertheless I confirmed that it was properly deployed… But as I suspected this wasn’t my problem.

Note: this behavior does not happen with PIP that include out-of-the-box with BTARN (in the Microsoft.Solutions.BTARN.Schemas.RNPIPs dll) only with new ones and only in the incoming process.

Microsoft-Solutions-BTARN-Schemas-RNPIPs

Fortunately for us we have an extended BizTalk Server: List of Errors and Warnings, Causes, and Solutions at TechNet Wiki, I usually update with some frequency and that I used as a reference, which gave me some clues to the final solution. Nick Heppleston mention in his blog that:

“RN Disassembler does in-fact attempt to validate the message contained within the Service Content against a deployed schema just like the standard XmlDisassembler. The message that our trading partner was sending did not validate and hence the RosettaNet Accelerator threw this error message; once we had corrected the schema and redeployed, the error went away.

This is certainly one to be aware of if you are developing custom PIP’s to use with the RosettaNet Accelerator: ensure that the message in the Service Content validates against your custom PIP schema”

Again this isn’t my problem, because the Schema was properly deployed in the system, I only mention this because is important for understanding how the BTARN engine/processes works.

CAUSE

By default BTARN App Pool Settings are configured to use .NET framework 2.0, doesn’t enable 32-bit applications and the application pool is in Classic mode

BTARN-App-Pool-default-Settings

Important for you to know is that the accelerator requires both an in-process and out of process host, both of which must be marked as "Authentication Trusted" and "32-bit only".

To run the BTARN End to End scenario you need two important things:

  • Enable the BTARN Application Pools for 32 bit.
  • Add a HTTP Handler for *.dll refering the IsapiModule Filters.

But what I found today is that when you want to extend BTARN with new incoming PIP’s, the artifacts generated are going to be .NET 4.0/4.5 unless Visual Studio is configured to do otherwise, which we really cannot control using BizTalk projects:

re-targeting-Visual-Studio-BizTalk-Projects-error

This means along with setting the application pools for the 32-bit the .NET setting must be set to match.

SOLUTION

To solve this issue you must:

  • Type “Internet Information Services (IIS) Manager” or “IIS” in the Windows Start screen and click in “Internet Information Services (IIS) Manager” option on Apps menu.
  • Expand the server and click on “Application Pools” to display available application pools in center panel.
  • Right-click on “BTARNAppPool” and select “Advanced Settings”.
  • On the “Advanced Settings” window:
    • Change the value of “.NET Framework version” from “v2.0” to “v4.0”.
  • Change the value of “Enable 32-bit Applications” from “False” to “True”.
  • And then click “OK”.

BTARN-App-Pool-correct-Settings

Advice: you should do the same for the “BTARNHttpReceivePool” application pool

After I fix these configurations I was able to successfully receive the PIP message from my external Partner.

Credits to LemMotlo – BTARN App Pool Settings for BizTalk 2010