Tuesday, 01 July 2014 23:21

Why ETL Validator when there is Python?

Written by
Rate this item
(0 votes)

I am not sure if you ever wrote Python but it is one of the coolest language out there. Easy to learn, easy to write and do yeah, easy to read! Writing code in Python is something I enjoy when I want to get a break from my routine job of putting together slides or preparing for upcoming demos.

 Over the last few years, the usage of Python has gone up drastically and one such area is testing automation. With very few lines of code, you can achieve remarkable things. As an example, sometime back I had to compare the data in two CSV files (tens of thousands or rows)  and then spit out the differences. The code looked somewhat like this:



 #Python 2.7.5

orderFile = open('Source.csv','r')

customOrderFile = open('Target.csv','r')

orderLines = orderFile.readlines()

customOrderLines = customOrderFile.readlines()



orderSet = set(orderLines);

customOrderSet = set(customOrderLines);

print orderSet - customOrderSet;

#Response is below. Exactly, what I wanted to see.

set(['492023-13942389\n', '492023-139584\n', '492023-139325\n', '492023-13939\n', '492023-13945\n', '492023-1394545\n']) 


Very simple. right? For a one-off, writing a script in Perl, Python or whatever your favorite language is fine. However, it is unfair to expect QA Analysts, Engineers to write scripts for their regular job. Few problems that often surface are:

1. There is already enough work to do. Not many want to learn something new.

2. There may be no common framework: The version of the scripting language may be different on each user's machine or worse yet, each user may prefer to write scripts in languages of their choice.

3. Let's say we want to do regression testing on a bi-weekly basis (at the end of every sprint), how can we schedule them? Write another script? :)

4. When we want to connect different databases (Netezza etc), we need specific drivers for each type. Painful!

5. How do you get reporting on top of the results?

6. How can you notify all the stakeholders? 

Answers are not easy! So, to the extent possible, it is always better to have a consistent framework across all the business units within an organization to solve data testing challenges. ETL Validator provides you exactly what you need without writing code. Check it out! You will love it.

Read 2002 times Last modified on Tuesday, 26 January 2016 23:44