If you're about to embark on a Data Migration or Data Integration project, and you're used to working with a traditional relational database that sits in your own data center, you may be in for a few surprises.
I have worked with Salesforce and a number of other SaaS providers for a number of years, working on both Data Migration and Data Integration projects. I thought I'd share some of the lessons that I have learned.
When you're working with SaaS, you'll usually be talking through an API that is provided by the vendor. This API is typically using REST or SOAP.
Talend provides excellent support for Salesforce and many other SaaS vendors with out-of-the-box components. If Talend does not directly support your vendor, then you can do it yourself using components such as tRest and tSOAP. If you're feeling really adventurous, you could write your own custom components and then make them available on the Talend Exchange for the benefit of others.
The Fragility of the Internet
Whilst the technology underlying the Internet may be robust, the reality is that your connection may sometimes be less reliable than you would hope for and bandwidth is always under pressure. You may feel less confident pushing 100M rows to your SaaS provider, than you would to an Oracle database that is hosted in your organization’s own data center. With this in mind, you will want to give careful consideration to the techniques that you use and how you would recover should you encounter a mid-point failure.
If you’re using Salesforce, for example, they provide both Standard and Bulk APIs, so try to make sure that you use the correct API, depending on your data volume and your requirement. Get a good understanding of the APIs that your own vendor provides before investing in the wrong approach.
Try to ensure that unnecessary obstacles do not get in your way. If your organization uses Proxy Servers to access the Internet, try to avoid these as they can often interfere with your activity. There should, hopefully, not be a strong case for your interfaces to be forced to use these servers as they’re really intended for managing employee interactive Internet access rather than the mission-critical interface between your CRM and Finance systems.
You will, of course, always want to make sure that you are sending your data across the Internet in a secure manner. If FTP plays a part in your interface, make sure that you are using Secure FTP and not sending your highly valued data in plain text for all to see.
Sometimes, working with SaaS may seem like a step back in time.
As well as yourself, your SaaS provider will have other organizations to whom they will need to provide a good service. They will want to make sure that they can always provide the service that they have promised.