Andrew McDonagh
6/5/2006 6:50:00 PM
sunkids@gmail.com wrote:
> Hi All:
> Thanks everyone so activating in ruby community. I am newbi to
> ruby. This is all because WATIR, a web testing tool.
> I know some people post before saying there is no need for ruby
> have a runtime debugging tool, because ruby is simple enough or so easy
>
> to find where the error is, which I totally agree with. However, my
> company deside to try WATIR/Ruby in our product functional testing(Web
> application testing), and ask me to give a full demo to the automation
> board.
> The challenges of swithing from our existing automation tool to
> Ruby/WATIR tool are fast development and less maintenance effort.
> First, I am strongly believe fast development can be achieved by strait
>
> forward coding and pure OO language advantage. Second, less maintenance
>
> effort has to be achieved with propret tool. However, because of our
> large size of web application, our testing framework is also very large
>
> and complex, and it is still grow fast. In order to maintain the
> testing framework release to release we are at boundary equal effort
> between manual testing and recalibrate existing automation. That's also
>
> why we can looking for better tools.
> The current WATIR/Ruby debugging flow is all based on exception,
> fix and rerun the program. However, with frequent source code
> change(our product), we have to face the ugly truth, our automation
> scripts have to be constantly change. Even worse, our automation
> scripts can't be just simply rerun, due to all the testing senario data
>
> is half way send to server. We have to restore the database at server
> and rerun the script, which is VERY TIME CONSUMING, maybe more than our
>
> code creation.
> Which makes me thinking about a tool, that will allow me running
> Ruby/WATIR code under it, show me the exception, pause the code and
> resumed the code after I saved my fix to the code. If this is doable
> (which I believe, because ruby is interpreted language), that will be
> ideal solution for our maintaining such large testing framework.
> Thanks in advance for all your commens.
>
> Regards,
> Jonathan Ni
>
ok, so there's a few different things here that under lay your desire
for a debugger - notably that Watir tests are difficult to write/test
because they are working the system end-to-end - i.e. Web ui to DB.
Automated tests are an investment - they take time to make and need
maintaining just like the application code they exercise. However, this
investment is normally a factor lower than the time and cost of
repeatedly running manual tests.
Once a test is automated it can and should, be run as manay times as
possible - by everyone on the team including the build system if you
have one.
However, as you are seeing, your tests need to Arrange some form of
initial state, do the desired Action to test and then Assert the results.
I.e. the 3 A's
Arrange
Action
Assert
However, like I said, your tests are currently end-to-end. Which is
often not needed nor desired.
A more reliable, flexible, less maintenance approach is to partition the
tests so that very few actually go end-to-end.
For example, say we have a login screen, which accepts the users name
and password, which the web application then uses to authenticate
against an LDAP service.
Now, we could setup the test environment and the test itself so that we
use a real users name and password and the system really does connect to
the LDAP.
However, then the user name and password needs updating when ever they
change (because company policy says all passwords expire every month, or
the user leaves the company, etc).
A better, flexible, less maintenance approach would be to substitute the
LDAP look up with a fake LDAP implementation.
This way, we can create acceptance tests that test all types of
conditions without the application ever going to a real LDAP server and
the test never need updating because of a data value change, because the
values never need changing - they are fake values.