Scribe Software Blog

Using Scribe DTS Parameters to Change Connections on the Fly

Don Panek, a Scribe MVP (Most Valuable Professional), recently completed a global deployment of Microsoft Dynamics CRM and Dynamics AX.   The integration platform used Scribe Insight. In this article Don details how they configured the integration to manage the complex requirements.  

 “This client has multiple AX servers throughout the world which means our Scribe target connection AX endpoint will need to be changed dynamically. Likewise, we may also need to change the target AX Company as well for a given endpoint…”

A challenge with a complex CRM to ERP integration is that the ERP system is often configured for multiple companies, and the CRM system does not have a way to manage that type of configuration.  Don explains how he configured Scribe to handle this challenge using a combination of CRM customizations and Scribe configuration.  Specifically he uses a nifty feature called DTS Parameters
which allows the Scribe tools to pull in variables at runtime. This feature enabled him to change the connection to the correct Destination Company in AX on the fly.

 

 

“..we will see that based on the values we have stored in CRM we can easily and reliably drive our integration to multiple AX instances and companies using this method and the powerful DTS Parameters.”

The details of this method can be found on his new blog: http://crmdirectory.com/crmblogs/using-scribe-dts-parameters-to-drive-your-dynamics-crm-to-dynamics-ax-integration/

Don Panek is a Scribe MVP, he has been involved in the implementation of Scribe integrations and migrations for Hitachi clients as well as clients with previous employers over the last 10 years. He works at Hitachi Solutions http://www.hitachi-solutions.com/ 

Scribe MVP Blog: Increase Dynamics CRM Online Scribe Data Migration Performance

Scribe MVP (Most Valuabe Professional) Jayson Goldinger, of Slalom Consulting, posted this useful blog today on Dynamics CRM Online integration with Scribe and we thought our readers would find it useful. Thanks Jayson!

"For those of you who have done data migration or integration with Dynamics CRM Online, you know that the performance may be a challenge. I recently faced that challenge during a recent project.
 
I had approximately 52,000 account records with associated contacts to import. I also needed to do a check (Seek Step) against CRM to see if the record was already in CRM. I was using Scribe Insight with my source being Excel and my target was Dynamics CRM Online using the Dynamics CRM Adapter. On my first attempt my job ran for 9 hours and only imported approx. 15,000 records. At that rate it would have taken over 30 hours to complete....."

Read Jayson's entire blog here: http://blog.slalom.com/2012/08/20/increase-dynamics-crm-online-scribe-data-migration-performance/ 

 

Integrating Dynamics CRM and Microsoft SharePoint

As a Sales Engineer at Scribe, I get many questions about Scribe capability in specific integration scenarios. One such question is how we can integrate Dynamics CRM and Microsoft SharePoint. In this post, I provide an overview of how I managed to integrate these two systems, some of the tools I had to use, and some of the pitfalls I ran into. I used the Scribe Adapter for Web Services and the Scribe Adapter for Dynamics CRM 2011 when designing this integration. I hope this helps shed some light on how this integration can work between these two systems.

Adapter for Web Services

This adapter is a generic connection to SOAP based web services provided by Scribe. It allows Insight to utilize many SOAP based web services that are publicly available, and also allows you to create your own in order to access legacy systems with poor or no database structure. You can utilize this to either pull or push data to the back end systems, as it exposes all the methods and parameters for the particular WSDL you are using. This adapter also exposes some additional configuration that can be used to streamline your integration design, like providing default values to parameters that can be overridden, logon and logoff methods, as well as error capturing using Regular Expressions.

SharePoint Web Services

Microsoft SharePoint has a very well documented web service back end to assist you in integrating with it. It is not just one web service, but instead a collection of roughly 21 separate WSDL endpoints that allow you to manage and manipulate data. When starting with these SharePoint web services I highly recommend utilizing MSDN for help on how to work with the web services as it is very well documented, gives you code samples that you can utilize to figure out what type of data needs to be passed in, and teaches you what each method does. SharePoint also has a unique way of working with separate sites and its web services. Instead of giving you just those 21 web service endpoints, if you have 3 separate sites built within SharePoint, you get 63 separate end points, 21 for each site. This makes it very easy to distinguish what the particular web service connection is being used for, removing all ambiguity from the integration design. In my case, I did two separate integrations, one for Help Desk tickets and one for Marking Campaigns, which had two separate sites in SharePoint. When I created my connections to the web services, I could see right inside the URL what site I was connecting to.

Integration Overview

The integration that I will overview here is a Marketing Campaign integration, bringing Campaign data down to SharePoint lists. This includes not only the core data about the Campaign, but any related marketing material as attachments. The use case behind this scenario is that a company has users who need access to marketing material such as PowerPoint templates, email drafts, and white papers, but who do not have access to Dynamics CRM. This integration allows those users to utilize the corporate SharePoint server to gain access to materials that were previously only available inside Dynamics CRM.

When approaching this integration, several decisions had to be made up front about how this should run. I decided to make this a real time one way integration using Queue based integrations so that the data could be presented as soon as changes are made, and it would make use of multi-threading. In order to do this, I had to work outside of the box a bit. Because I need to bring all attachment data to SharePoint, I had to have a direct connection to my CRM system. This means that I could not just use the XML message as the source of my DTS file. So, to overcome this I used a Prompted Variable to capture the GUID of the campaign I was working with, and fed that into my source query using XPATH. That allows me to extract that CampaignID value from the XML that triggered the integration, and feed it into the source query at run time, giving me not only the campaign I care about, but also the attachments that come with it.

So, this setup will allow me to use a real time, multi-threaded integration, but I do not have to use XML as the source of the DTS file in order to make this happen.

Now, within the setup of the job itself there are some complex formulas that must be used in order to work with the SharePoint web services. There is the concept of an xmlAny attribute, which is how data must be sent to SharePoint. This attribute allows me to send in a piece of XML with field information and values to match what fields exist in SharePoint. It must be done this way because SharePoint is so easily customizable there is no way for the web service to constantly be re-configuring itself to display all the fields you are adding/changing within SharePoint. By using Scribe formulas to create XML header and node information, I am able to pass in whatever fields are available at that time, and if I need to make a change to add/remove a field, I simply alter the XML I am passing in. As you can see in the screen shot, I was able to create calculated variables which contained XML node information and data, and concatenate them together with XML header information to present a large block of XML with a minimal amount of data in my formula.

Another aspect in dealing with the SharePoint web service is working with return values. Much as is with passing data into SharePoint as XML, the values you receive from SharePoint are also formatted as XML. So, instead of creating formulas to stitch together XML, we must now find a way to parse the XML and locate the values we are looking for. I did this using text functions like FIND() and MID(), in order to locate the nodes I cared about, and get just that value from the XML. I could then use this in subsequent steps.

There are many more design details to this integration that can be found in our latest video within the Inside Track, if you would like to view this new video but are not yet a subscriber of the Inside Track, you can register for a free trial here using the registration code "InsideTrackTrial". For those of you who have already subscribed, there should be a new book on your shelf titled Integrating Dynamics CRM and SharePoint.

What Role Does Integration Play in Microsoft’s Cloud Strategy?

MSDynamicsWorld.com published Betsy Bilhorn's article on Microsoft's Cloud strategy, the temptations for short-cuts, and long term considerations for your business.  Here's a brief excerpt:

Call it whatever you like - infrastructure, IT environment, or ecosystem - Microsoft's technology stack still dominates the IT landscape and continues to be as attractive to customers as it is challenging to competitors. An integrated stack is a great selling point: Windows, SQL, SharePoint, Dynamics, and Office are all Microsoft, all compatible. Great! But do they really integrate together right "out of the box?"

Read the entire article on MSDynamicsWorld.com and see how well your integration strategy "plays" with Microsoft's.

Microsoft mainstream support ending for CRM 3.x

With their upcoming release of CRM 2011, Microsoft is preparing to sunset support for v3.x.  You can find their timeline here: http://support.microsoft.com/lifecycle/?LN=en-us&p1=9776&x=8&y=11, but to save you the round trip: official mainstream support will end in April '11 and extended support ending in April '16.

As Scribe moves forward with CRM 2011, we will be following Microsoft's example & will be discontinuing support for Microsoft CRM 3.x as of the upcoming CRM Adapter release in March '11. While we will not be making any technical decisions that would prohibit Scribe Solutions from working with MS CRM 3.x, we will not be testing our solutions on MS CRM v3.x & Scribe Support will not be fielding questions about how to resolve issues on MS CRM 3.x.

If you have any questions or concerns, please let us know via a reply to this post or by emailing us at ProductManagment at Scribesoft.com (<-I've obfuscated the email address here to fool them web bots).

Revelations on Efficiency with Scribe 7 and Multi Target

One of my clients here at Infinity Info Systems has been involved in an elaborate integration between Microsoft Dynamics GP and Microsoft Dynamics CRM that I built in Scribe 6.4 and have been tweaking ever since. The client is a medical component parts supplier with over 9000 component parts in stock, and thousands more that they source on a yearly basis.

The original GP to CRM integration I designed used an hourly batch process to migrate their data from GP (SQL direct) to CRM. This, while acceptable, was not the preferred solution due to lag-time between data entry in GP and usage in CRM, as well as a lack of true net-change management.

With the additional stability offered in Scribe 6.5.2 and the GP adapter enhancements in version 3.9, the client finally agreed to implement the adapter in their production environment. Earlier this year, I moved the client to a transactional integration using the GP publisher and MSMQ for Accounts, Contacts, Products, Units, Unit Groups and Price Lists. The effectiveness of the GP Publisher instilled confidence in the client and allowed us to move forward, tackling some of the more complex problems that we were facing.

One of the more challenging tasks I was assigned was the concept of implementing a “New Product Release” process. This process would allow the client to replace an aging database used to source new products that they do not currently carry in stock. Since this customization set was already in place in CRM, the client wanted to leverage it to store these “non-inventory” items and then convert them to GP inventory when a certain threshold was met.  This would reduce repetitive data entry in multiple systems, and allow end users to work in CRM where they felt comfortable, rather than the legacy database.

The two major challenges with this process were that the client wanted the process to be nearly instantaneous using MSMQ, and the client wanted to empower the end user to troubleshoot conversion issues themselves to reduce IT Department burden.

Unfortunately, if there was a problem on any step in the process, I was limited to one set of data from the Message published from the MSMQ to write error specific information to the end user. This frustrated me to no end, because in order to be successful with this process, the client wanted to be able to write back detailed failure information to CRM if there was a problem, such as:

• Error Reason
• (CRM) Item ID
• (CRM) Item Description
• (CRM) Item Created On
• (GP) Item Number
• (GP) Item Description
• (GP) Item Created On
• (CRM) Conversion Attempted On
• (CRM) Conversion Attempted By

This information was difficult to write back, because of the limited source update capabilities using MSMQ. In addition to the limitations, the process felt extremely clunky and prone to error due to the back and forth of the message handoffs between DTS files on success of each job. The only way for me to do this was to create multiple jobs to handle failures, creating a temporary table in Scribe Internal to gather information about the failure, get information from GP and CRM, and then push that information to CRM after the processing was complete.

With the release of Scribe 7, I attended one of the multi-target webinars, and one night while in the shower (where my brain tends to do most of its intense thinking), had one of those “Holy Cow!” moments where I realized the power of multi-target and the huge reduction in overhead it could provide. I decided to install and configure Scribe 7 in our test environment to see if I could reduce the complexity of this process by leveraging multi-target.

After redesigning the process, I was able to take 8 individual DTS files performing multiple tasks, and merge them down into one DTS that has multiple steps going against GP and CRM. The new DTS gets its source from the same message as the old process; however the resulting complexity has been reduced by a significant factor, cutting out temp tables and multiple message handoffs.

Multi-target has also provided us the ability to write back to CRM with failure info from GP (Steps 7 and 8 in Exhibit 1), should we find duplicates in GP or if the insert fails. Using temporary variables (Exhibit 2) I can capture CRM and GP data points for each record, writing back robust error information to CRM if the job encounters issues.

Additionally, upgrading to 7 from 6.5.2 for the existing processes was seamless. I took the 20+ DTS files and two publishers related to the other pieces of the GP-CRM integration and updated them all in a manner of an hour. The only issue I ran into was that I had to update my DBLOOKUP statements, because with the addition of multi-target, you now do DBLOOKUP against the connection name, not “S, T or I”.

Once I realized the power of multi-target, I haven’t looked back. I am pushing all my clients to Scribe 7.0.1 if they need any kind of new DTS development, because it makes their systems more efficient and makes my life significantly easier.

Exhibit 1

Exhibit 2

Scribe Software Named to 2010 President’s Club for Microsoft Dynamics

Scribe is pleased to announce we've been named to the 2010 President's Club for Microsoft Dynamics, and recognized by Microsoft for our committment to customers.

President's Club for Microsoft Dynamics honors high-performing Microsoft Dynamics partners whose commitment to customers is reflected in their business success and growth. This prestigious group represents the top 5 percent of Microsoft Dynamics partners worldwide. Membership is granted based on their continual, committed efforts aimed at offering solutions that meet the needs of their customers.

"Microsoft is proud to recognize Scribe Software for its outstanding sales achievement," said Doug Kennedy, vice president of Microsoft Dynamics Partners. "This honor is a direct reflection of Scribe's outstanding group of employees and management team, all of whom should be very proud of this accomplishment. Scribe plays a very important role in the growth and success of Microsoft Dynamics."

This acknowledgment came during the Microsoft Worldwide Partner Conference (WPC), the company’s annual premier partner event, which is taking place this week in Washington, D.C. WPC provides Microsoft Corp.’s partner community with the largest opportunity to learn about the company’s road map for the upcoming year, establish connections, share best practices, experience the latest product innovations and learn new skills.

Microsoft Releases new Statement of Direction for Dynamics CRM

Hot off the presses here is the latest Statement of Direction for Microsoft Dynamics CRM Download CRM_StatementOfDirection_April2010

Refer to the Sonoma Partners blog post for an in depth analysis of the Statement of Direction

Integrating Dynamics CRM to Dynamics GP in 72 Hours

Check out this article from MSDynamicsWorld.com where Mariano Gomez, the CEO of Maximum Global Business, LLC provides some really good advice on how to approach integrating Dynamics CRM to Dynamics GP. One of his key nuggets of wisdom is to choose the best tool for the job; in the case of the customer he references, that happened to be Scribe Insight for its flexibility and scalability. It was also noteworthy that this customer already owns eConnect, Integration Manager, and Web Services.

Aside from the selection of the tool, the suggestions about finding a good partner, knowing your business process, and using subject matter experts are spot on.

The Microsoft Adapter or Scribe Insight: It’s as Plain as Trains and Automobiles

Microsoft recently released their Microsoft Dynamics CRM 4.0 Adapter for Microsoft Dynamics GP 10.0 that allows Dynamics CRM and Dynamics GP to share data across a common set of integration points including customers, products, orders, and invoices. Since a large number of our customers use Scribe Insight for the same purpose, we are being asked to compare and contrast the two solutions. At the risk of oversimplification and with deference to the two late, great Johns, Candy and Hughes, this blog attempts to use a comparison of trains and automobiles to explain the difference.

(You can find a more detailed technical comparison here.)

Think of the Microsoft Adapter like taking the express train. If your destination (i.e. integration need) is close to one of the pre-defined stops along the route, then the Adapter is right for you. Because the outcome has been engineered into the solution, taking the train is easy, inexpensive, and requires no special training. Getting there has already been figured out for you. The Microsoft Adapter is provided free to customers with current support plans and is a purpose-built solution that provides "out of the box" integration capabilities between the latest versions of Dynamics CRM and Dynamics GP. There is a graphical utility that enables some customization to the pre-set mapping and transformation of fields, but for the most part, your end solution is relatively fixed.

Taking the train analogy one step further, you can reach destinations that are close to the stops on your route, by taking a bus or a taxi. These options involve greater cost and complexity and become quite impractical the further you deviate from the route. Microsoft plans to offer an SDK in the coming months where the solution could be expanded to incorporate additional objects and process logic through custom code. Like the taxi or bus, the SDK will allow for simple extensions, but significantly expanding the scope of your integration will be highly impractical.

Scribe Insight on the other hand is like driving a car. Once you made the initial investment and learn how to drive it, it enables to reach any destination you wish. You can even change your destination as often as you wish in the middle of the journey. Scribe Insight is an open integration platform that enables technical folks to quickly create integration processes across two or more applications without the need to write custom code. The depth of the product allows companies to configure these integration processes precisely to their business needs all within the Insight user interface. Since these integration processes are metadata driven, they can be modified and changed quickly and easily as business needs change. Insight also allows you to easily incorporate any of your other business applications or data stores into your integration processes.

In addition to Insight, Scribe also offers Adapters for common applications including Dynamics CRM and Dynamics GP, along with Scribe's template for integrating Dynamics CRM and Dynamics GP. This template is used by many companies as a jumping off point for a solution that works exactly the way that they do. At the risk of taking the car analogy too far (I fear it may already be too late), you can think of these Adapters and Templates like a navigation system. These Adapters make it a lot easier to navigate around these business applications, taking a lot of the guess work out of the process. And the templates provide you with a suggested route that you can choose to use, or not.

The important message here is that like trains and automobiles, both the Microsoft Adapter and Scribe Insight are excellent solutions depending on your specific needs. The key is choosing the right one. So if the Adapter gives you what you want, then hop on the Microsoft express and enjoy the ride. If you need to plot your own route, then fire up Scribe Insight, it will get you where you want to go.