Mac OSX error: Sauce Labs – Error fetching PAC

Posted on in Environment

I'm using Sencha Test together with the Sauce Labs browser farm. In order to see all the various browsers in my Sencha Test Studio, I have to setup a Sauce Labs Connect tunnel.

  1. You can download it from here:
    https://wiki.saucelabs.com/display/DOCS/Setting+Up+Sauce+Connect

  2. Extract the zip, and place the contents of the folder under this location:
    /Users/username/bin/SauceLabs/sauceconnect/

  3. Create a new shell script, under /Users/username/bin/SauceLabs/ called “tunnel”:
    touch tunnel

  4. Open up this file:
    open tunnel

  5. Populate with the following contents, substituting the login name and access key with your own. You can find this in the Sauce Labs portal (https://saucelabs.com/beta/user-settings)

ulimit -n 8192
sauceconnect/bin/sc -u saucelabsusername -k mykey
  1. Save the script, then make it executable using this command:
    chmod +x tunnel

  2. Now run the script: ./tunnel

The result should be:

MacBook-Pro-3:Saucelabs leeboonstra$ ./tunnel
11 Mar 17:40:11 - Sauce Connect 4.3.13, build 1879 4494856
11 Mar 17:40:11 - Starting up; pid 19205
11 Mar 17:40:11 - Command line arguments: sauceconnect/bin/sc -u leeboonstra -k ****
11 Mar 17:40:11 - Using no proxy for connecting to Sauce Labs REST API.
11 Mar 17:40:11 - Resolving saucelabs.com to 162.222.75.243 took 16 ms.
11 Mar 17:40:12 - Started scproxy on port 59721.
11 Mar 17:40:12 - Please wait for 'you may start your tests' to start your tests.
11 Mar 17:40:12 - Starting secure remote tunnel VM...
11 Mar 17:40:17 - Secure remote tunnel VM provisioned.
11 Mar 17:40:17 - Tunnel ID: e506b17963574528b900eba652ec6958
11 Mar 17:40:18 - Secure remote tunnel VM is now: booting
11 Mar 17:40:20 - Secure remote tunnel VM is now: running
11 Mar 17:40:20 - Using no proxy for connecting to tunnel VM.
11 Mar 17:40:20 - Resolving tunnel hostname to 162.222.75.24 took 44ms.
11 Mar 17:40:20 - Starting Selenium listener...
11 Mar 17:40:20 - Establishing secure TLS connection to tunnel...
11 Mar 17:40:20 - Selenium listener started on port 4445.
11 Mar 17:40:33 - Sauce Connect is up, you may start your tests.

However, in my case, I received an error:

11 Mar 17:30:54 - Sauce Connect 4.3.13, build 1879 4494856
11 Mar 17:30:54 - Detected PAC URL http://wpad/wpad.dat.
11 Mar 17:30:54 - Starting up; pid 18123
11 Mar 17:30:54 - Command line arguments: sauceconnect/bin/sc -u leeboonstra -k ****
11 Mar 17:30:54 - Error fetching PAC http://wpad/wpad.dat: Couldn't resolve host name.
11 Mar 17:30:54 - Sauce Connect could not establish a connection.
11 Mar 17:30:54 - Please check your firewall and proxy settings.
11 Mar 17:30:54 - You can also use sc --doctor to launch Sauce Connect in diagnostic mode.
11 Mar 17:30:54 - Goodbye.

This is how I solved it.
First I need to make sure outgoing port 443, is open:

nc -v portquiz.net 443

This seems to be ok.
The next thing that I need to check is my network proxy settings.

Open your network settings.
Click Advanced, Proxies tab. Make sure Auto Proxy Discovery is UNCHECKED.

Now try to run the tunnel again. ./tunnel
For me it worked!

How does ST.screenshot work?

Posted on in Questions Sencha Test

One of the powerful capabilities of Sencha Test is the ability to capture screenshots during test runs. By default, on the first test run, captured screenshots will be used as the baseline. Any future screenshots will be compared with that baseline and any differences in the captured UI, such as a shift in pixels, or missing components, will be flagged as a failure.

This functionality doesn't work in the first version of Sencha Test studio. It does work via the command-line interface: stc.
You will need to have a TestArchive setup.

http://docs.sencha.com/sencha_test/ST.html#method-screenshot

 describe("initial state", function(){
    it("should match the expected screenshot", function(done) {
                ST.screenshot('email', done);
 }, 30000);

The resolution of the image, is the resolution of your running browser. For example one of my screenshots has 1040x642px and is 35kb, which will be stored the test archive server. (you will need to setup the Test Archiver). (I run it locally on my machine with "stc server").

The extension is PNG. (i am not 100% sure, but it could be that under the hood this script has been used: https://huddle.github.io/Resemble.js/).

The first image is stored in folder baseline and then you must go deeper. Then you can change baseline in ST or replace this image in folder - it is up to you. (There is a -b command to change the baseline.)

--

How often is the screen of driven browser captured? Is it based on event (test passes/test fails) or is it time based (i.e. each 5 seconds)?

The screenshot API functionality is mainly for visual screen comparison capability. The screen is captured wherever the screenshot api is called from the test case.

What is the format, resolution and size of the outputted captured images?

Right now there is no settings for format or resolution that the user can make. The images are .PNG files. The size of the image is around 50KB - 500 KB based on the content on the screen(approximately). The resolution is depended on the browser and machine you are running them.

**How about the image comparison performance and disk requirements? Is it done pixel by pixel? Is there any recommendation on disk size we should have? **

I ran 250 tests in 6 minutes that includes creating all the baseline images as well. It took the same time when I re-ran the tests on Sauce labs and it takes the same amount of time as well. The file size of the results zip that includes 250 tests are around 4 MB that includes over 30 screenshot tests. This depends on the size and the content of the application. Screenshot tests are allowed only through command line. This means the tests are stored in archive server which is again maintained in a shared repository with more memory than normal PC. So I would say even with 100's of runs we should be fine.

How does the Sencha Test Event Recorder work?

Posted on in Questions Sencha Test

About the event recorder:

"The event recorder allows you to quickly grab events, as you go about performing various actions on the application under test (AUT). The event recorder leverages many methods to arrive at the right property to create stable tests. The default methodology adopted by the event recorder is to use a combination of Component Query and DOM selector to find the stable property for a particular component such as a “check box” or “combo box”. It also allows you to choose from XPATH or ID based on your preference.

Tests can be executed manually on a developer’s machine at the stage of application development as well as integrated with a CI process for regression tests. The CI system invokes the build agent and runs the tests against the AUT and posts results back to the Sencha Test Archiver. Unlike the results in a CI environment that are in plain HTML or text file format, the test results in Sencha Test are provided in a rich, matrix format. You can view the results in Sencha Test and can drill down into failed tests for additional information."

How it works

  1. Open a test scenario script.

  2. First look into your log tab, to figure out where your Test server is parked. (For example localhost:8800). Then visit it in the browser.

  3. Write an it block and put the cursor in it.

   it("Should enter username before running all test.", function() {
        //cursor here
    });
  1. Click the Event Recorder button, and select the browser that you are using for the recording.

  2. Now go back to the browser, click on wherever you want to click, and make your UI action, which needs to be recorded.

  3. When done, click on the Insert Event Recording button, in the Test Studio. You will see that the Event Recorder wrote specific play() syntax, something like this:

   it("Should enter username before running all test.", function() {
        ST.play([
            { type: "tap", target: "@ext-element-9", x: 124, y: 207 },
            { type: "tap", target: "@button-1005-btnInnerEl", x: 13, y: 10446 },
            { type: "tap", target: "@textfield-1014-inputEl", x: 108, y: 14 },
            { type: "type", target: "@textfield-1014-inputEl", text: "savelee" },
            { type: "tap", target: "@combobox-1015-trigger-picker", x: 14, y: 17 },
            { type: "mousedown", target: "@combobox-1015-picker-listEl/li[2]", x: 108, y: 339, detail: 1 },
            { type: "mouseup", target: "@ext-element-12", x: 108, y: 339, detail: 1 },
            { type: "click", target: "@ext-element-12", x: 108, y: 339, detail: 1 },
            { type: "tap", target: "@button-1017-btnInnerEl", x: 6, y: 13 }
        ]);
    });

This is a piece of script that you play.

Which version of Jasmine is integrated in Sencha Test?

Posted on in Questions Sencha Test TDD

Jasmine 2.4.1 is included in Sencha Test. Tests written to Jasmine 2.x will most likely work but Jasmine 1.x tests may have some issues due to Jasmine changes between 1.x and 2.x.

Sencha Test will automatically load all *.js files in a specified directory (what we call the Scenario) so the only issue I can foresee there is file order potentially. This would be a poorly written test perhaps but it could happen if they controlled the order manually. Other then that I don't see any reason the tests themselves won't work.