Data onboarding generally means bringing in data from some outside source into some specific system.
When we talk about customer data onboarding, we’re talking specifically about taking data from your customers and loading it into your system or platform where you can work with it and return something of value to those customers.
The most common scenario we see is a SaaS platform needing to onboard data to get a new customer up and running – and there’s usually also a need for some ongoing data ingestion too, so the customer is seeing their up to date data in the platform.
"The onboarding process is our first chance at making a good impression with our customers after the dollars are already spent"
- CloverDX customer Bryan Kahlig, Senior Director Product Development, Zywave
How you handle data onboarding gives a critical first impression of you as a vendor. And this can happen even before you’ve signed your customer. As part of your pitch you want to be able to give reassuring and positive answers when prospects are asking questions about your onboarding process such as:
And once you’ve signed the deal, you want to make sure that good impression carries on.
Of course the onboarding process may be iterative, requiring adjustments or re-mapping, but the crucial thing is that this should be as transparent to the client as possible. A lack of transparency, unexpected delays or lots of errors can leave your new customers asking themselves ‘did we make the right decision?’
The speed and efficiency of the onboarding process is also valuable to you as the ‘onboard-er’. If your engineering team is having to do a lot of time-consuming, manual work to onboard each new client, it not only costs you money, but also means you may not be able to keep up with the number of customers you need to onboard.
These bottlenecks not only hinder your scalability but can also leave your new customers frustrated with the time it takes to get them live on your platform.
The #1 challenge with onboarding data from multiple clients is that you’ve got no control over the format or quality you receive the data in. In an ideal world, you want to be able to take whatever data they have and easily (and automatically) get it into your system.
"When I go in and I speak to a prospective customer, I don't ever worry about data. I did before. The question was always ‘Where's your data? What does it look like? What format is it? How much are we talking?’
So what we were doing before was kind of going in with handcuffs. And what CloverDX really allowed us to do is go in and say ‘It doesn't matter how you're giving us this stuff, we're just going to stitch it all together.’"
Other data onboarding challenges include:
Data coming from different systems, in different formats, all needs to be integrated and consolidated into the format your platform needs. Every customer is going to have their own rules they need implementing, whether it’s making sure First Name and Last Name are split into two fields, or converting multiple currencies to one.
Especially if companies are moving to your platform from an aging legacy system, there may be an expectation that ‘new system = new, improved data’, and that this migration will solve all their data quality issues. It’s on you to identify those data quality issues and clean up the data as it’s onboarded, not just propagate the same issues into a new platform.
It’s common to make customers themselves do the work to get their data into the format you need. This can often cause delays and bottlenecks (especially if the customer isn’t very tech-savvy or doesn’t have their own engineering team to help).
It’s also not a great first impression when your asking your customer to do a lot of work with their data before delivering it you, especially when they've just spent money with you. Wouldn’t it be better to make their life easier and take the work off their hands?
You also want to make the data onboarding process as easy as possible for your engineering teams and not overburden them with repetitive manual work.
The goal here is to avoid using expensive engineering resources to custom build a data onboarding solution for each new client, and instead create a process that can be almost entirely automated, and where any work can be managed by non-developer or less-technical staff.
The solution to all these problems is to automate the process. Building an automated customer data onboarding pipeline enables you to:
Your automated pipeline not only needs to handle the entire end-to-end process, but it also needs to account for the variations in format, quality and frequency of data delivery that come from different customers.
How Zywave freed up engineer time by automating data onboardingIf we break down the individual steps we want a data onboarding pipeline to handle, they’re generally common to every job:
These stages apply to any type of data integration job – data migrations, system integrations, data warehousing, as well as data onboarding.
Specifically for customer data onboarding, those broad stages can be broken down into specific steps that we want our automated framework to handle automatically:
The steps of a data onboarding pipeline
The second part of this post looks at how you can automate this customer data onboarding process in CloverDX.
The CloverDX Data Management Platform is designed to build and operate data pipelines. It enables you to design pipelines in a visual editor, but also to code whenever you need, so you’re not constrained and can build a completely custom onboarding framework for your specific needs.
CloverDX can connect to any type of data, ingest it, shape it, cleanse it and write it to any target. And automation means onboarding jobs can run automatically and unattended, with monitoring and error alerting to notify you of any issues.
The video on the next post shows you the step by step process of how this works, and walks through how you can build a single pipeline to work with many different clients, by using configuration files to drive the whole process. (What does that mean? Mainly it means you don’t have to build a new pipeline for every new client, or for every change you need, and it means that this configuration – which can just be a human-readable Excel file – can be managed by less technical people).