BizTalk BAM Authentication Error

I ran into this issue today on my VPC running BTS2009 on WIN2K8/IIS7:

HTTP Error 401: The requested resource requires user authentication

I found the following blog posting which helped me resolve the issue:

http://sharepointservices.wordpress.com/2009/04/28/http-error-401-the-requested-resource-requires-user-authentication-in-iis7/

Indeed turning off kernel-mode authentication fixed it but I too am concerned with the validity of this fix.  For now I am not too concerned because it is a local dev VPC so I will see if the new production servers will have the same issue.

Dynamics AX AIF Adapter Progress

Well after some initial struggles with locked channels (never resolved this, just built a new VPC!!) I have both the AIF tutorial and some PoCs working. 

Just to clarify that in a previous post I mentioned the latest issue of BizTalk HotRod which has an article on the adapter.  While most of the configuration is the same, it’s important to note that the article uses Dynamics AX 4.0, not 2009.

A particular challenge I recently had to overcome was regarding security on the send port.  I had setup and tested the AIF tutorials using a Proxy User (providing an AX user account/password right in the send port).  This worked great.  However, once I wanted to enable another document service (namely LedgerPurchaseInvoiceService) and follow an identical approach (correctly assigning a Data Policy, verifying my endpoints) I continued to get errors in the event log indicating permission was denied.  I was using the administrator account as both the BC service account and the gateway user.  I looked at a few other reports of similar issues but couldn’t get it working.

Some colleagues had mentioned that security configuration with AIF can be a challenge, and particularly that using anything other than the Host User configuration can sometimes just not work.  I remain convinced that it should work, I’m just not doing something right.  However I didn’t have a lot of time to debug it and/or open a PSS ticket, so I proceeded to change the send port to use the identity of the Host User (ensuring the service account was a user in DAX with the right permissions) and it worked. 

I am still concerned about the cause of issue with the Proxy User configuration, but the reality is that it likely makes more sense to use the host instance account for authentication regardless as it simplifies deployment (no password to maintain) and you can keep whatever degree of account isolation you need (one account for all Host Instances, or one per endpoint/service).

OutOfMemoryException on MemoryStream in Pipeline Component

On one of my projects, I was using a custom pipeline component to decompress a file in a send pipeline (*see side note on design decision at end of post).  In the production environment, we began to see pipeline failures caused by mscorlib OutOfMemoryExceptions in the pipeline component.  It was happening sporadically, so at first was not clear what was happening, though we knew the issue was occurring with growing frequency, roughly in correlation to the increase in the size of the file (the compressed file was a statement of account balances which grew as the number of accounts grew).

As many BizTalkers do, I use the SharpZipLib library for Zip compression (see Pro BizTalk 2006 by Dunphy and Metwally for a great example of this) and was taking the approach of loading the zipstream into a MemoryStream object.  The exception was being thrown in my loop which copied 4K segments into the stream.

After consulting my trusted advisor I saw a few discussions related to the need to provide a contiguous segment of memory for a MemoryStream to work.

I quickly inferred the likely culprit: I was trying to load data into a MemoryStream without the OS knowing how much memory to allocate it, so in many cases it was allocating a contiguous block that would not be enough for the uncompressed file.

Short term fix:

For the time being, I have simply updated my pipeline component to declare the size of the MemoryStream up front based on the zipstream’s Size property which is the number of bytes of the fiie uncompressed.

DANGER: the Size property is a Long, whereas you can only instantiate a MemoryStream with an Int32 (of course from the 2GB memory limit for 32-bit processes).  Knowing the file size will not grow beyond 1GB uncompressed, I have squished the long into an int, which of course is terrible.  Hence a long-term fix:

Long term fix:

Though I have yet to implement this, the sensible approach is instead to use something like the VirtualStream which will offload data to the filesystem if a stream exceeds a configured size, saving your poor BTSNTSvc.exe

Indeed hardware should never be used to mask up bad code, but it’s interesting to consider that this issue would likely not have arisen if it had been deployed on a 64-bit OS, which we can hopefully encourage all clients to do in the future.

 

* as a side note, the unzip was done in a send pipeline, in opposition to the usual approach of decompressing a file in a receive pipeline because the contents of the file were not needed.  All we wanted to do was route based on the file name, so by delaying the decompression we only needed to load a 16MB compressed file to the messagebox instead of a 700MB uncompressed file.

Relative .snk file paths in Visual Studio 2008

I’m sure someone has already blogged on this, but as a reminder to myself and others, in Visual Studio 2008 it is not as straightforward to provide a relative path for a shared .snk file (for instance when signing multiple BizTalk projects with the same key file).

In the past, you could simply type in the path to the snk file in the project properties. Now this field is a dropdown and selecting browse will only let you find and copy the file to the root of the project.

To get around this, you can edit the project file (in the case of BizTalk, the .btproj). The following is an example where the .snk is both included as a shortcut in the project (not a copy) and uses the .snk for signing:

(Note: this project is the updated project schema for BizTalk 2009, so don't try to use this approach for BTS2006+VS2005)

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="3.5" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
<ProductVersion>9.0.30729</ProductVersion>
<SchemaVersion>2.0</SchemaVersion>
<ProjectGuid>{6092B853-D9B4-4C5E-B480-746703C74E80}</ProjectGuid>
<ProjectTypeGuids>{EF7E3281-CD33-11D4-8326-00C04FA0CE8D};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
<OutputType>library</OutputType>
<GenericProcessing>true</GenericProcessing>
<RootNamespace>Test.Orchestrations</RootNamespace>
<AssemblyName>Test.Orchestrations</AssemblyName>
<TargetFrameworkVersion>v3.5</TargetFrameworkVersion>
<BpelCompliance>True</BpelCompliance>
<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>..\Test.snk</AssemblyOriginatorKeyFile>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
<DebugSymbols>true</DebugSymbols>
<DebugType>full</DebugType>
<Optimize>false</Optimize>
<OutputPath>bin\Debug\</OutputPath>
<DefineConstants>DEBUG;TRACE</DefineConstants>
<ErrorReport>prompt</ErrorReport>
<WarningLevel>4</WarningLevel>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
<DebugType>pdbonly</DebugType>
<Optimize>true</Optimize>
<OutputPath>bin\Release\</OutputPath>
<DefineConstants>TRACE</DefineConstants>
<ErrorReport>prompt</ErrorReport>
<WarningLevel>4</WarningLevel>
</PropertyGroup>
<ItemGroup>
<Reference Include="System">
<Name>System</Name>
</Reference>
<Reference Include="System.Xml">
<Name>System.XML</Name>
</Reference>
<Reference Include="System.Configuration">
<Name>System.Configuration</Name>
</Reference>
<Reference Include="Microsoft.BizTalk.Pipeline">
<SpecificVersion>False</SpecificVersion>
</Reference>
<Reference Include="Microsoft.BizTalk.DefaultPipelines">
<Name>Microsoft.BizTalk.DefaultPipelines</Name>
</Reference>
<Reference Include="Microsoft.BizTalk.GlobalPropertySchemas">
<Name>Microsoft.BizTalk.GlobalPropertySchemas</Name>
</Reference>
<Reference Include="Microsoft.BizTalk.TestTools">
<Name>Microsoft.BizTalk.TestTools</Name>
</Reference>
<Reference Include="Microsoft.XLANGs.BaseTypes">
<Name>Microsoft.XLANGs.BaseTypes</Name>
</Reference>
</ItemGroup>
<ItemGroup>
<Compile Include="Properties\AssemblyInfo.cs" />
</ItemGroup>
<ItemGroup>
<None Include="..\Test.snk" />
</ItemGroup>
<Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
<Import Project="$(MSBuildExtensionsPath)\Microsoft\BizTalk\BizTalkC.targets" />
</Project>

Just save the file and VS will reload the project showing “..\Test.snk” as the location of the strong name key file.

SysPreping a BizTalk development machine

Great info from MSDN on how to set up a development VM that can be shared with other developers in team scenarios. The scripts are included in the BizTalk SDK.

http://msdn.microsoft.com/en-us/library/dd792685%28BTS.10%29.aspx

Note that it mentions that WSS installations will likely need to be reconfigured to get working again following the sysprep. If WSS is required, considering either leaving WSS unconfigured or not installing until after sysprepping.

An error occurred when validating an AS2 message

In an AS2 BizTalk solution for a client, we began to see a number of errors in the event log and the behaviour was that no messages or MDNs would arrive from a particular third party:

We checked the certificate store and determined the certificate had not been revoked or expired. Searching the certificate store for the certificate to see where it was installed revealed that the certificate was no longer in the Other People store which is used by BizTalk for verifying signed messages. We found the resolution on our own, but after the fact noticed the following MSDN article which seems to be pretty thorough about resolving this issue. Note that our resolution is the last item on the "User Action" list:

http://msdn.microsoft.com/en-us/library/bb898960%28BTS.10%29.aspx

BizTalk and Dynamics AX 2009 AIF Integration

I'm taking the first steps down the road of integrating BizTalk and DAX. By many accounts, it's not the straightest road, but I'm up for the challenge :) I'll include any learnings on this as I go. In the mean time, here is where I have started on this road:

Microsoft Dynamics AX 2009 AIF BizTalk Adapter Configuration White Paper
http://www.microsoft.com/downloads/details.aspx?familyid=EDC62433-5B21-4F74-B065-B075BA6DC86D&displaylang=en

Latest BizTalk HotRod issue
http://biztalkhotrod.com/Documents/Issue7_Q3_2009.pdf

BizTalk BAM Data Archiving

Being a bit late to the finer points of managing a BAM installation, I'm just discovering some useful and important things to know about BAM, particularly managing its archive and understanding its data age policy.

http://seroter.wordpress.com/2007/06/22/biztalk-bam-data-archiving-explained/

As always, thanks Richard for making such in-depth-yet-easy-to-understand posts.  Makes my job a lot easier :)

BAM Activity and Real-time Aggregations Windows

You can modify the duration data is kept in the BAMPrimaryImport database using the bam_Metadata_Activities table:

http://technet.microsoft.com/en-us/library/ms962341.aspx

In addition, if you are using Real-time Aggregations for BAM data, you can set the window of data kept using the bm utility and the set-rtawindow command:

http://msdn.microsoft.com/en-us/library/aa559458.aspx