The biggest challenge for any CRM data is having accurate and complete data. Better data quality benefits an organisation with better interactions, higher trust in the data, reduced cost (eg. less communications to incorrect users, reduced data storage etc.) So how can we:

  • Improve data quality
  • Check entered data is correct
  • Alert users to when there may have entered incorrect data
  • Easily Manage and reports on the quality of the data

I have worked in the world of retail customer data and in working with organisations and the quality and validity of customer data is essential to deliver better customer experience, interactions and value behind contacts. CRM data quality refers to how valuable the information you track in CRM actually is. If you’re going to the trouble of entering it in the first place, your data needs to be high-value/quality. Quality of data is made up by two factors:

1: Data Accuracy

Is your information both precise and correct? It needs to be, or you team has no reason to trust it. Once trust in the data is gone, all your information in CRM is suspect, and you may not be able to convince your team to use the system even if you make improvements to resolve later.

2: Data Completeness

Do you have all the information you need for each of your records? Incomplete data isn’t uncommon, especially for capturing customer data, but chronically incomplete data can be a big problem.

To handle these it is critical that you identify what information is required and then processes for entering, updating, validating and verifying the data. This can help you avoid the nightmare scenario of a database full of inaccuracies.

What it does

Best practice for CRM validation is to validate when data is entered. To help with this Data Validator App provides out of the box validation and verification automations so that data is validated inline easily and quickly as a user enters it. It also provides a set of dashboard components to allow you to visualise the quality of data easily.

Validation Automations

Email Validator and Verifier - this checks to see if the entered email field is correctly formatted as an email eg. The verify service takes this further by checking aspects like DNS server and SMTP to check if the email actually exists. For instance you can see in the demo video the email even though it is a valid email address comes back as undeliverable. This adds another level of value to the email captured.

Phone Validator - this checks to see if the phone number is in a valid format for any global location eg. +44 7525 362525

Link Validator and Verifier - checks to see if link is provided in correct format eg. verify service checks validates if the url exists by pinging the server looking for a successful response making sure you know if the link is correct and exists.

Address Validator - checks fields like Postcode/Zipcode format validation for which you can specify the global location you would like to use to validated eg. Global, USA, UK, etc. so that entered addresses are correct and valid.

Customer Field Validator - checks fields like Name, Social Security Number, Passport Number, National Insurance number etc. This can help if you are running CRM for a hotel with guest information, events where you are tracking admission, HR records where you need to store key information etc.

Vehicle Validator - check license plates, VINS number etc. to capture accurate vehicle information. This is ideal if you are managing a fleet of vehicles, car dealership portal CRM (track customer car license plates to greet them at the door), building guest management etc.

Number Validator - check fields are whole numbers, decimals etc. When you have restrictions around numbers requiring a certain format this is idea.

String Validator - check that fields are alphanumeric or alphanumeric with spaces etc. This means you can have fields that have correct content restrictions

However if these are not enough out of the box you can always use the regex validator which allows you to define your own custom rule. Regex is an easy way of defining custom string matches so if you have a customer product id format or a data field value set that you want to validated Data Validator has you covered.

Dashboard Widgets

(Completeness means if there is data entered, Quality is data is validated with one of the validators)

Data Quality/Completeness Guage - In a single widget see the overall quality of your data inside for the validators you have installed. This high level metric provides you an easy way of viewing the quality and setting KPIs or targets to maintain your CRM data to the highest standard.

Data Quality/Completeness Widget - view image This widget breaks down your data quality or completeness by column, which means you can quickly and easily see just where you are capturing better data and where you have worse data. For instance in a CRM you may see that you are capturing better phone number data compared to email so the team can set a target to capture more emails data.

Data Quality/Completeness by Person Widget - for a team where they capture customer information this provides a list of users where they can track their captured data quality and see how it compares to others.

Low Quality Data List - this component provides you with a list of the worst data quality elements allowing you to take actions to improve it such as creating tasks to get in touch with the customer easily and quickly.

How I built it

The validations are built using AWS Lambda and the Monday OS framework. I used the _ When column changes _ trigger and then custom actions that pass over to AWS Lambda which then performs the regex checks for validation steps and for verification it performs service calls. I chose AWS Lambda as it provides massive scalable performant execution globally so if Data Validation has 1 use or 1,000,000 uses the performance and service should be the same.

The quality widgets for dashboards are written using reactjs with my own queries on top of the Monday APIs.

Challenges I ran into

I had challenges in porting the example integration over to AWS Lambda as this required setup for AWS Gateway and other parameters to make the end to end process work.

Accomplishments that I'm proud of

I am impressed how these validations can be used to make really rich and engaging boards with good quality data at it's core. The fact that you can keep adding the validations to have complete coverage of your data is something I am really proud of. Overall this could make a huge difference to CRM data captured on Monday boards.

What I learned

I have learned how easy it is to use APIs and how to leverage the components to make an engaging UI. I have also learned more about customer data validation and the more I understand the more I can see it as critical to any CRM or customer data solution.

What's next for Data Validation for

  • Adding more out of the box validations and regex services
  • Adding some more verification services such as for Addresses, Street Codes etc.
  • Supporting dark mode in the widgets
  • Setting notifications if data quality thresholds drop

Built With

Share this project: