Starting my Journey into Automation with Selenium

Leave a comment

Why learn how to automate? There’s plenty of answers to that question around the internet, but personally I feel it’s become an indispensable part of a tester’s toolset. If you want to be a better tester, then at least a basic understanding of automation, needs to form part of your skillset. This was brought home to me especially at Testbash Manchester, October 2017, and the workshops that accompanied it.

So why is it now indispensable? With the increasing complexity of applications and processes that we’re testing, combined with the ever-present time-pressures of getting software to market (whether that be internal or external) as quickly as possible, leveraging the power of automation is a key time-saving tool to have available in the toolbelt.

While it’s important to remember that automation doesn’t replace testing (a point made especially well in Michael Bolton’s presentation at the same Testbash), it’s potential to reduce the monkey grind; for example, repetitive activities around data generation and regression checks, can’t be ignored. So, as well as knowing how to automate, it’s important to consider what, and when, to automate. Automate to assist in testing, not in an attempt to let it do the testing for you.

Consider a process where you’re looking at a change to an invoice generated for the customer. Automation can focus on the repetitive grind of generating orders, while you spend your testing time analysing the invoice produced at the very end of that process. As an example, I find that a 5 minute manual process to generate said invoice, can be as little as 30 seconds once automated.


I’m focussing here on my own journey – which as of writing is less than 3 months in. The start of the year gave me a critical opportunity to spend some time learning, in that a major project had finished, and the next large project was still getting up to speed. I grasped this opportunity to teach myself both Java, and Selenium. Since I’m very much still learning, I expect much of what I write might not be the most efficient way of doing things. Some other things might be just flat out wrong! Please, add comments to point these out, we learn more from our failures than our successes.

Being a beginner, I started trying to look at what is considered the beginner’s entry point, the Selenium record and play extension available within Firefox. Unfortunately, as of last summer, Firefox has broken this extension and there’s no timeline on when the extension might be upgraded to support current versions of Firefox.

I could have, of course, used an old version of Firefox, however the issue with that is that if something doesn’t work, I now must consider that aside from the other potential issues, the problem could simply be version related. That, plus the apps I’m looking at are mainly IE and Chrome focussed, meant that I quickly realised I needed to go for the ‘full-fat’ Selenium.


To start this, I used the excellent, free online courses offered by @TheFriendlyTester. They used to be on his personal website, but are now located at – if you’re starting your own journey, I heartily recommend starting here.

I supplemented that with a course available within Pluralsight, that I’m lucky enough to have access to through work. As I’ve gone on, Google, and especially StackOverflow have become increasingly useful resources to get answers to questions.


So the next few blog posts I’ve got lined up focus on how I’m doing, and what I’m learning, from automation.

As I go on, I plan to start by looking at:

  • Tools
  • Classes and Methods including Visio diagrams
  • Getting Data In and Out using CSV files
  • CSV vs Excel
  • Taking screenshots
  • Using Select Dropdowns
  • Formatting Excel data as a table

Is E-mail a defect?

Leave a comment

Does anyone – other than Microsoft – still use ‘e-mail’, with the hyphen in? I know that historically email started out as a contraction of electronic mail, complete with a hyphen, but I think we’re well past the point of email becoming mainstream.

Here are some examples:

Going to shows email as the clear winner

Googling for ‘Apple email’, ‘Google email’ or ‘Yahoo email’ (or ‘e-mail’), the top results for each all show email without a hyphen.

Googling for ‘Microsoft email’ is the only one where the results include ‘e-mail’ – although the results themselves are inconsistent, with some instances of ‘email’ also present.

From this, it seems that the only thing keeping ‘e-mail’ as high in the rankings as it is, is that Outlook uses it, and Outlook is the most popular enterprise solution for e-mail/email. Given Microsoft are themselves inconsistent in their use, then the validity of Outlook’s usage of the word is called into question. Given all the other major tech players seem to use ‘email’, then surely this must now be the commonly accepted term?

What are your thoughts – has email supplanted e-mail yet? If not, is it on course to do so, and by when? What is the tipping point?

The reasoning behind this is to ask a simple question – if an app uses the term ‘e-mail’, is that a defect?

The Feeling of Rejection


In scope or out of scope? This has got to be one of the most common issues that we face when raising defects. I raise an issue that then comes back as rejected because the documentation isn’t specific enough around that area and development is often focussed more on time and effort involved than usability.

There are four possible outcomes:

  1. The defect remains rejected and is ultimately closed
  2. The rejection is overturned, and the defect is accepted as valid, and fixed
  3. The rejection is valid, but leads to a change in the documentation, and in turn, a fix to the defect
  4. The rejection remains, but the issue is progressed to an ‘Enhancement’ or other suitable status, to be addressed as part of some future release.

Which outcome should you push for?

As with everything in our craft, it’s all down to the context.

Ask yourself five questions

  1. How likely is the defect to occur?
  2. How severe is it?
  3. How easy is it to fix?
  4. Is there a workaround?
  5. Is this internal, or external facing?

The combination of the first two questions gives you an ‘impact assessment’. You can go as far as scoring each answer and multiplying the two to give you a figure (management loves to report on figures rather than words).

The third question should then be considered in light of this assessment. If it’s easy to fix and has a high impact, then push for an immediate fix. If it’s very difficult to fix but has a low impact, you might want to let this one slide.

Question four adds a bit more qualification – if there’s no workaround available then you are in a better position to push for the fix now. If there is one available then maybe it can wait a while.

Question five is probably the most important one. If this is external, then it can be a real game-changer, as suddenly even the smallest of defects can cause real damage to a brand if exposed to the public. If it’s internal, then the opposite is true, large defects can then be handled by ‘business process’, ‘workflow’ or my favourite, as a ‘training issue’!

Consider a spelling mistake on a page. Normally it’s not a huge issue and a few of them often get through. Consider though that this is a website for a company selling English language educational assets. Having a spelling mistake on your front page is likely to cause people to think twice before purchasing from you.

Ultimately though, any software is a solution to a problem. If the problem isn’t solved, then the solution is broken. If the solution is broken, then a defect needs fixing.

Applied Epistemology

Leave a comment

Dipping back into ‘Lessons Learned in Software Testing’ after bit of a break from it, I rediscovered the topics on Applied Epistemology, namely that this is what testing is. It jumped out at me, because I’ve just encountered an example of it in my own testing.
As the book states, one topic within epistemology is ‘meaning and ambiguity in natural language’. Think about that for a minute or two, and I’m sure you’ll easily come up with examples that have affected your testing. My example is below:

The product I was testing holds address data, and as with any product that holds this kind of information, the data can easily become invalid or out of date, and need cleansing or maintenance. Part of this will include marking addresses as ‘Goneaways’. There are a number of ways of doing this, you could either compare your data to a screening file (in the UK we have a GAS, or Goneaway suppression file, for example) or you could rely on the fact that any post to a goneaway address should be returned. Since there’s no point sending out communications to these addresses, but given that legally we need to retain all information that we have for a set period, then it follows that we need a way to mark these addresses without deleting them. They’re not valid addresses for us to send to, and you might even go so far as to label them as ‘Bad’.

Unfortunately, that’s exactly what happened. Somewhere between gathering requirements, and producing code, ‘Goneaway’ got converted to ‘Bad’. Now, a label is just a marker, and if everyone had the same understanding, then this wouldn’t have been a huge problem. It became a problem, because ‘Bad’ was then interpreted to mean structurally incorrect and in need of correction. Of course, a Goneaway’s address is usually structurally sound, and is a valid address in the sense that the postal service can happily locate it, so to mark it as ‘Bad’ – as in incorrect – was not the correct solution to our problem.

‘So what?’ you might be asking at this point – well, the key is in the possible scenarios that would make use of this marker. Imagine that someone else moves into this ‘Goneaway’ address, but that this someone else is also on our database. If we associate this Bad Address to them, then we won’t send out any communications to them. If we unmark the address as ‘Bad’, then we’ll end up sending communications to both the new resident and the previous resident, at the same address.

Marking an address as a Goneaway subtly alters the requirement, as there is nothing wrong with the address itself – we actually want to flag the link between the address and the resident. That way, if the address is reused, then a new link can be created to a new resident, and this new link can remain unmarked. The use of the word ‘Bad’ led to ambiguity, and this ambiguity was successfully picked up by walking through test scenarios based on the original requirements.

Job ads

Leave a comment

A few days ago, I got the following email from a recruitment agent.


I’m sorry if the email does not apply to you.

I’m currently working on behalf of a client who are looking for a Contract Systems Tester for an intial 3 month period to start immediately. The role is to help implement best testing practice across the company,please find a brief outline below

They need to be able to work on their own and not need handholding. Be able to supervise others in the team that are less experience. In brief their tasks will include:

– Writing of System Test Plans in compliance with the project Gantt and plan

– Writing of System Test Scripts, Conditions and Results

– Completion of test runs in accordance with Functional requirements and solution design

– Reporting of bugs and follow through to resolution

– Managing the customer during UAT reporting of bugs to technical staff and resolution thereafter

– Reporting of progress to the Project Manager

– Be deadline driven

Most of that is standard cookie cutter stuff, there’s certainly nothing that stands out and grabs my attention, nothing that would make me think, ‘Wow, I want that role’.

There were two lines though that I was struck by, but for the wrong reasons.

To help implement best testing practice

I’m more taken with the principles of the Context-Driven school of testing than ISEB/ISQTB, and they quite clearly point out that there is no such thing as best practices.

Maybe if you’re repeating the same task day after day, then there are some ways of doing things that will become ‘best’, but testing is about finding new things every day; looking at changes to existing applications as well as entirely new ones, always trying to come up with new ways of finding defects.

Be deadline driven

‘Deadline Aware’ maybe at a stretch, but deadlines aren’t what drive me, quality is. By that, I mean that I want to make sure that as many of the bugs in there have been found, and that the development team has had a chance to address them. I’d prefer to ship a higher quality application late, than a broken application on time.

(Note: There have been times when an application I’ve been testing has had to go live with serious known defects. This has usually been due to external influences, such as tax and month end issues, but this has so far proven the exception rather than the rule.)

I’m also compelled to point out the typo on ‘experience‘ – we testers do tend to have an eye for detail! (And yes, I know that’s not the only issue with the text.)

It’s a tough economic climate out there, with hundreds of applicants chasing every application. Does that really justify ads like the above?

Mr. Monty Hall


OK, so in my last blog I posed a question, and the answers that came back got it right – which I was hoping for, but wasn’t expecting. This leads me to believe that those brave enough to comment (as WordPress tells me that there were a lot more views than comments!) are more logically-thinking than average – or have possibly just heard of the problem before!

In doing a bit of research before writing this half of the post, I realised that this ‘hypothetical’ scenario I’d been taught was actually a real one, from a real quiz show aired in America. The host of that quiz show was named Monty Hall, and in his case, the prizes were goats and a car, rather than money and beans. The underlying puzzle goes even further back – full details can be found on Wikipedia.

The basic answer of course, is that you should take the opportunity to move, every time. You started out with a one in three chance, and Monty gives you the option to switch to a two in three chance.

According to Wikipedia, when this was first discussed thousands of people, many with PHDs, disagreed and insisted that it didn’t really matter whether you switched as the chances were now 50:50. I found the same when discussing this with colleagues. Some others insist that you’re better off staying where you are, although I’ve never really got a rational explanation as to why they thought this.

The reason why it isn’t 50:50 is because the door Monty chose wasn’t chosen randomly. He couldn’t choose the door you were already on, and he couldn’t choose the one with the prize behind it – as otherwise he can’t ask you if you want to switch. Because of that, Monty is limited to choosing the losing door, which fundamentally alters the statistics of the choice you’re then given.

The fact that many people think the choice boils down to 50:50 is another example of cognitive bias – it’s the opposite of the gambler’s fallacy where someone predicting the result of an 11th coin toss where the first ten have all been heads, will generally vote tails as they feel it’s overdue. In reality, the coin is either biased or the chances remain 50:50 – unlike our example above, the previous result has no bearing on future results.

For testers, awareness of these cognitive biases is important. As far as possible, we need to appreciate the actual expected results of our tests, without bias.

One place to go to get this awareness is John Stevenson’s blog, which often looks at psychology and its impact on testing.

Congratulations, you’ve won!


Congratulations, you’ve won!

Consider this, you’ve just won a quiz show and the chance of a million pounds (or the equivalent in your local currency). There’s just one catch, you have to guess which of three doors the cash is hiding behind.

The quiz show host takes you over to the doors and makes a big song and dance about how there’s a million pounds behind one of the doors, but tins of beans behind the two losing ones. Unless you’re a big fan of beans, I’m guessing you’re still trying for the million pounds.

It’s now crunch time and you’ve got to make a decision. This bit is easy as you’ve effectively got a one-in-three chance, unless you believe that fate, luck or something else is guiding your decision. Let’s assume that you pick door number one.

Here comes the twist. The quiz show host now explains that at least one of the remaining two doors must conceal a tin of beans, and he’s going to open that door. With great fanfare, he opens door number two, to reveal a fairly large tin of beans. He then gives you the chance to change your mind. Do you want to stick with your first choice, your gut instinct – or do you want to switch to door number three?

The quiz show host knows which door the million pounds is behind, but his face isn’t giving anything away. Your choice is to stick, or move. What choice do you make?

(I’ll reveal the answer and the reasons in the next blog post, unless anyone fancies trying to beat me to it in the comments)

Older Entries