Getting Started Guide

Once you have installed pywinauto - how do you get going? The very first necessary thing is to determine which accessibility technology (pywinauto’s backend) could be used for your application.

The list of supported accessibility technologies on Windows:

  • Win32 API (backend="win32") - a default backend for now
    • MFC, VB6, VCL, simple WinForms controls and most of the old legacy apps
  • MS UI Automation (backend="uia")
    • WinForms, WPF, Store apps, Qt5, browsers

    Notes: Chrome requires --force-renderer-accessibility cmd flag before starting. Custom properties and controls are not supported because of comtypes Python library restrictions.

Not supported: Java AWT/Swing, GTK+, Tkinter.

AT SPI on Linux and Apple Accessibility API are in the long term plans so far.

GUI Objects Inspection / Spy Tools

If you’re still not sure which backend is most appropriate for you then try using object inspection / spy tools that are available for free: download them from GitHub repo gui-inspect-tool.

  • Spy++ is included into MS Visual Studio distribution (even Express or Community) and is accessible through Start menu. It uses Win32 API. It means if Spy++ can show all the controls the "win32" backend is what you need. AutoIt Window Info tool is a kind of Spy++ clone.

  • Inspect.exe is another great tool created by Microsoft. It’s included into Windows SDK so that it can be found in the following location on x64 Windows:

    C:\Program Files (x86)\Windows Kits\<winver>\bin\x64
    

    Switch Inspect.exe into UIA mode (using MS UI Automation). If it can show more controls and their properties than Spy++, probably the "uia" backend is your choice.

If some or all controls are not visible to all the inspection tools it’s still possible to control the application by generating mouse and keyboard events using basic modules mouse and keyboard.

Entry Points for Automation

So you have an application, you know it supports one of the mentioned accessibility technologies. What’s the next?

First you should start your application or connect to an existing app instance. It can be done with an Application object. This is not just a clone of subprocess.Popen, but an entry point for further automation limiting all the scope by process boundaries. It’s useful to control potentially few instances of an application (you work with one instance not bothering another ones).

from pywinauto.application import Application
app = Application(backend="uia").start('notepad.exe')

# describe the window inside Notepad.exe process
dlg_spec = app.UntitledNotepad
# wait till the window is really open
actionable_dlg = dlg_spec.wait('visible')

If you want to navigate across process boundaries (say Win10 Calculator surprisingly draws its widgets in more than one process) your entry point is a Desktop object.

from subprocess import Popen
from pywinauto import Desktop

Popen('calc.exe', shell=True)
dlg = Desktop(backend="uia").Calculator
dlg.wait('visible')

Application and Desktop objects are both backend-specific. No need to use backend name in further actions explicitly.

Window Specification

It’s a core concept for the high level pywinauto API. You are able to describe any window or control approximately or in more details even if it doesn’t exist yet or already closed. Window specification also keeps information about matching/search algorithm that will be used to get a real window or control.

Let’s create a detailed window specification:

>>> dlg_spec = app.window(title='Untitled - Notepad')

>>> dlg_spec
<pywinauto.application.WindowSpecification object at 0x0568B790>

>>> dlg_spec.wrapper_object()
<pywinauto.controls.win32_controls.DialogWrapper object at 0x05639B70>

Actual window lookup is performed by wrapper_object() method. It returns some wrapper for the real existing window/control or raises ElementNotFoundError. This wrapper can deal with the window/control by sending actions or retrieving data.

But Python can hide this wrapper_object() call so that you have more compact code in production. The following statements do absolutely the same:

dlg_spec.wrapper_object().minimize() # while debugging
dlg_spec.minimize() # in production

There are many possible criteria for creating window specifications. These are just a few examples.

# can be multi-level
app.window(title_re='.* - Notepad$').window(class_name='Edit')

# can combine criteria
dlg = Desktop(backend="uia").Calculator
dlg.window(auto_id='num8Button', control_type='Button')

The list of possible criteria can be found in the pywinauto.findwindows.find_elements() function.

Attribute Resolution Magic

Python simplifies creating window specification by resolving object attributes dynamically. But an attibute name has the same limitations as any variable name: no spaces, commas and other special symbols. But fortunately pywinauto uses “best match” algorithm to make a lookup resistant to typos and small variations.

app.UntitledNotepad
# is equivalent to
app.window(best_match='UntitledNotepad')

Unicode characters and special symbols usage is possible through an item access in a dictionary like manner.

app['Untitled - Notepad']
# is the same as
app.window(best_match='Untitled - Notepad')

How to know magic attribute names

There are several principles how “best match” gold names are attached to the controls. So if a window specification is close to one of these names you will have a successful name matching.

  1. By title (window text, name): app.Properties.OK.click()
  2. By title and control type: app.Properties.OKButton.click()
  3. By control type and number: app.Properties.Button3.click() (Note: Button0 and Button1 match the same button, Button2 is the next etc.)
  4. By top-left label and control type: app.OpenDialog.FileNameEdit.set_text("")
  5. By control type and item text: app.Properties.TabControlSharing.select("General")

Often not all of these matching names are available simultaneously. To check these names for specified dialog you can use print_control_identifiers() method. Possible “best_match” names are displayed as a Python list for every control in a tree. More detailed window specification can also be just copied from the method output. Say app.Properties.child_window(title="Contains:", auto_id="13087", control_type="Edit").

>>> app.Properties.print_control_identifiers()

Control Identifiers:

Dialog - 'Windows NT Properties'    (L688, T518, R1065, B1006)
[u'Windows NT PropertiesDialog', u'Dialog', u'Windows NT Properties']
child_window(title="Windows NT Properties", control_type="Window")
   |
   | Image - ''    (L717, T589, R749, B622)
   | [u'', u'0', u'Image1', u'Image0', 'Image', u'1']
   | child_window(auto_id="13057", control_type="Image")
   |
   | Image - ''    (L717, T630, R1035, B632)
   | ['Image2', u'2']
   | child_window(auto_id="13095", control_type="Image")
   |
   | Edit - 'Folder name:'    (L790, T596, R1036, B619)
   | [u'3', 'Edit', u'Edit1', u'Edit0']
   | child_window(title="Folder name:", auto_id="13156", control_type="Edit")
   |
   | Static - 'Type:'    (L717, T643, R780, B658)
   | [u'Type:Static', u'Static', u'Static1', u'Static0', u'Type:']
   | child_window(title="Type:", auto_id="13080", control_type="Text")
   |
   | Edit - 'Type:'    (L790, T643, R1036, B666)
   | [u'4', 'Edit2', u'Type:Edit']
   | child_window(title="Type:", auto_id="13059", control_type="Edit")
   |
   | Static - 'Location:'    (L717, T669, R780, B684)
   | [u'Location:Static', u'Location:', u'Static2']
   | child_window(title="Location:", auto_id="13089", control_type="Text")
   |
   | Edit - 'Location:'    (L790, T669, R1036, B692)
   | ['Edit3', u'Location:Edit', u'5']
   | child_window(title="Location:", auto_id="13065", control_type="Edit")
   |
   | Static - 'Size:'    (L717, T695, R780, B710)
   | [u'Size:Static', u'Size:', u'Static3']
   | child_window(title="Size:", auto_id="13081", control_type="Text")
   |
   | Edit - 'Size:'    (L790, T695, R1036, B718)
   | ['Edit4', u'6', u'Size:Edit']
   | child_window(title="Size:", auto_id="13064", control_type="Edit")
   |
   | Static - 'Size on disk:'    (L717, T721, R780, B736)
   | [u'Size on disk:', u'Size on disk:Static', u'Static4']
   | child_window(title="Size on disk:", auto_id="13107", control_type="Text")
   |
   | Edit - 'Size on disk:'    (L790, T721, R1036, B744)
   | ['Edit5', u'7', u'Size on disk:Edit']
   | child_window(title="Size on disk:", auto_id="13106", control_type="Edit")
   |
   | Static - 'Contains:'    (L717, T747, R780, B762)
   | [u'Contains:1', u'Contains:0', u'Contains:Static', u'Static5', u'Contains:']
   | child_window(title="Contains:", auto_id="13088", control_type="Text")
   |
   | Edit - 'Contains:'    (L790, T747, R1036, B770)
   | [u'8', 'Edit6', u'Contains:Edit']
   | child_window(title="Contains:", auto_id="13087", control_type="Edit")
   |
   | Image - 'Contains:'    (L717, T773, R1035, B775)
   | [u'Contains:Image', 'Image3', u'Contains:2']
   | child_window(title="Contains:", auto_id="13096", control_type="Image")
   |
   | Static - 'Created:'    (L717, T786, R780, B801)
   | [u'Created:', u'Created:Static', u'Static6', u'Created:1', u'Created:0']
   | child_window(title="Created:", auto_id="13092", control_type="Text")
   |
   | Edit - 'Created:'    (L790, T786, R1036, B809)
   | [u'Created:Edit', 'Edit7', u'9']
   | child_window(title="Created:", auto_id="13072", control_type="Edit")
   |
   | Image - 'Created:'    (L717, T812, R1035, B814)
   | [u'Created:Image', 'Image4', u'Created:2']
   | child_window(title="Created:", auto_id="13097", control_type="Image")
   |
   | Static - 'Attributes:'    (L717, T825, R780, B840)
   | [u'Attributes:Static', u'Static7', u'Attributes:']
   | child_window(title="Attributes:", auto_id="13091", control_type="Text")
   |
   | CheckBox - 'Read-only (Only applies to files in folder)'    (L790, T825, R1035, B841)
   | [u'CheckBox0', u'CheckBox1', 'CheckBox', u'Read-only (Only applies to files in folder)CheckBox', u'Read-only (Only applies to files in folder)']
   | child_window(title="Read-only (Only applies to files in folder)", auto_id="13075", control_type="CheckBox")
   |
   | CheckBox - 'Hidden'    (L790, T848, R865, B864)
   | ['CheckBox2', u'HiddenCheckBox', u'Hidden']
   | child_window(title="Hidden", auto_id="13076", control_type="CheckBox")
   |
   | Button - 'Advanced...'    (L930, T845, R1035, B868)
   | [u'Advanced...', u'Advanced...Button', 'Button', u'Button1', u'Button0']
   | child_window(title="Advanced...", auto_id="13154", control_type="Button")
   |
   | Button - 'OK'    (L814, T968, R889, B991)
   | ['Button2', u'OK', u'OKButton']
   | child_window(title="OK", auto_id="1", control_type="Button")
   |
   | Button - 'Cancel'    (L895, T968, R970, B991)
   | ['Button3', u'CancelButton', u'Cancel']
   | child_window(title="Cancel", auto_id="2", control_type="Button")
   |
   | Button - 'Apply'    (L976, T968, R1051, B991)
   | ['Button4', u'ApplyButton', u'Apply']
   | child_window(title="Apply", auto_id="12321", control_type="Button")
   |
   | TabControl - ''    (L702, T556, R1051, B962)
   | [u'10', u'TabControlSharing', u'TabControlPrevious Versions', u'TabControlSecurity', u'TabControl', u'TabControlCustomize']
   | child_window(auto_id="12320", control_type="Tab")
   |    |
   |    | TabItem - 'General'    (L704, T558, R753, B576)
   |    | [u'GeneralTabItem', 'TabItem', u'General', u'TabItem0', u'TabItem1']
   |    | child_window(title="General", control_type="TabItem")
   |    |
   |    | TabItem - 'Sharing'    (L753, T558, R801, B576)
   |    | [u'Sharing', u'SharingTabItem', 'TabItem2']
   |    | child_window(title="Sharing", control_type="TabItem")
   |    |
   |    | TabItem - 'Security'    (L801, T558, R851, B576)
   |    | [u'Security', 'TabItem3', u'SecurityTabItem']
   |    | child_window(title="Security", control_type="TabItem")
   |    |
   |    | TabItem - 'Previous Versions'    (L851, T558, R947, B576)
   |    | [u'Previous VersionsTabItem', u'Previous Versions', 'TabItem4']
   |    | child_window(title="Previous Versions", control_type="TabItem")
   |    |
   |    | TabItem - 'Customize'    (L947, T558, R1007, B576)
   |    | [u'CustomizeTabItem', 'TabItem5', u'Customize']
   |    | child_window(title="Customize", control_type="TabItem")
   |
   | TitleBar - 'None'    (L712, T521, R1057, B549)
   | ['TitleBar', u'11']
   |    |
   |    | Menu - 'System'    (L696, T526, R718, B548)
   |    | [u'System0', u'System', u'System1', u'Menu', u'SystemMenu']
   |    | child_window(title="System", auto_id="MenuBar", control_type="MenuBar")
   |    |    |
   |    |    | MenuItem - 'System'    (L696, T526, R718, B548)
   |    |    | [u'System2', u'MenuItem', u'SystemMenuItem']
   |    |    | child_window(title="System", control_type="MenuItem")
   |    |
   |    | Button - 'Close'    (L1024, T519, R1058, B549)
   |    | [u'CloseButton', u'Close', 'Button5']
   |    | child_window(title="Close", control_type="Button")

Look at the examples

The following examples are included: Note: Examples are language dependent - they will only work on the language of product that they were programmed for. All examples have been programmed for English Software except where highlighted.

  • mspaint.py Control MSPaint
  • notepad_fast.py Use fast timing settings to control Notepad
  • notepad_slow.py Use slow timing settings to control Notepad
  • notepad_item.py Use item rather then attribute access to control Notepad.
  • misc_examples.py Show some exceptions and how to get control identifiers.
  • save_from_internet_explorer.py Save a Web Page from Internet Explorer.
  • save_from_firefox.py Save a Web Page from Firefox.
  • get_winrar_info.py Example of how to do multilingual automation. This is not an ideal example (works on French, Czech and German WinRar)
  • forte_agent_sample.py Example of dealing with a complex application that is quite dynamic and gives different dialogs often when starting.
  • windowmediaplayer.py Just another example - deals with check boxes in a ListView.
  • test_sakura.py, test_sakura2.py Two examples of automating a Japanase product.

Automate notepad at the command line

Please find below a sample run

      C:\>python
      Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] on win32
      Type "help", "copyright", "credits" or "license" for more information.
(1)   >>> from pywinauto import application
(2)   >>> app = application.Application()
(3)   >>> app.start("Notepad.exe")
      <pywinauto.application.Application object at 0x00AE0990>
(4)   >>> app.UntitledNotepad.draw_outline()
(5)   >>> app.UntitledNotepad.menu_select("Edit -> Replace")
(6)   >>> app.Replace.print_control_identifiers()
        Control Identifiers:

        Dialog - 'Replace'    (L179, T174, R657, B409)
        ['ReplaceDialog', 'Dialog', 'Replace']
        child_window(title="Replace", class_name="#32770")
           |
           | Static - 'Fi&nd what:'    (L196, T230, R292, B246)
           | ['Fi&nd what:Static', 'Fi&nd what:', 'Static', 'Static0', 'Static1']
           | child_window(title="Fi&nd what:", class_name="Static")
           |
           | Edit - ''    (L296, T226, R524, B250)
           | ['Fi&nd what:Edit', 'Edit', 'Edit0', 'Edit1']
           | child_window(class_name="Edit")
           |
           | Static - 'Re&place with:'    (L196, T264, R292, B280)
           | ['Re&place with:', 'Re&place with:Static', 'Static2']
           | child_window(title="Re&place with:", class_name="Static")
           |
           | Edit - ''    (L296, T260, R524, B284)
           | ['Edit2', 'Re&place with:Edit']
           | child_window(class_name="Edit")
           |
           | CheckBox - 'Match &whole word only'    (L198, T304, R406, B328)
           | ['CheckBox', 'Match &whole word onlyCheckBox', 'Match &whole word only', 'CheckBox0', 'CheckBox1']
           | child_window(title="Match &whole word only", class_name="Button")
           |
           | CheckBox - 'Match &case'    (L198, T336, R316, B360)
           | ['CheckBox2', 'Match &case', 'Match &caseCheckBox']
           | child_window(title="Match &case", class_name="Button")
           |
           | Button - '&Find Next'    (L536, T220, R636, B248)
           | ['&Find Next', '&Find NextButton', 'Button', 'Button0', 'Button1']
           | child_window(title="&Find Next", class_name="Button")
           |
           | Button - '&Replace'    (L536, T254, R636, B282)
           | ['&ReplaceButton', '&Replace', 'Button2']
           | child_window(title="&Replace", class_name="Button")
           |
           | Button - 'Replace &All'    (L536, T288, R636, B316)
           | ['Replace &AllButton', 'Replace &All', 'Button3']
           | child_window(title="Replace &All", class_name="Button")
           |
           | Button - 'Cancel'    (L536, T322, R636, B350)
           | ['CancelButton', 'Cancel', 'Button4']
           | child_window(title="Cancel", class_name="Button")
           |
           | Button - '&Help'    (L536, T362, R636, B390)
           | ['&Help', '&HelpButton', 'Button5']
           | child_window(title="&Help", class_name="Button")
           |
           | Static - ''    (L196, T364, R198, B366)
           | ['ReplaceStatic', 'Static3']
           | child_window(class_name="Static")
(7)   >>> app.Replace.Cancel.click()
(8)   >>> app.UntitledNotepad.Edit.type_keys("Hi from Python interactive prompt %s" % str(dir()), with_spaces = True)
      <pywinauto.controls.win32_controls.EditWrapper object at 0x00DDC2D0>
(9)   >>> app.UntitledNotepad.menu_select("File -> Exit")
(10)  >>> app.Notepad.DontSave.click()
      >>>
  1. Import the pywinauto.application module (usually the only module you need to import directly)

  2. Create an Application instance. All access to the application is done through this object.

  3. We have created an Application instance in step 2 but we did not supply any information on the Windows application it referred to. By using the start() method we execute that application and connect it to the Application instance app.

  4. Draw a green rectangle around the Notepad dialog - so that we know we have the correct window.

  5. Select the Replace item from the Edit Menu on the Notepad Dialog of the application that app is connected to. This action will make the Replace dialog appear.

  6. Print the identifiers for the controls on the Replace dialog, for example the 1st edit control on the Replace dialog can be referred to by any of the following identifiers:

    app.Replace.Edit
    app.Replace.Edit0
    app.Replace.Edit1
    app.FindwhatEdit
    

    The last is the one that gives the user reading the script aftewards the best idea of what the script does.

  7. Close the Replace dialog. (In a script file it is safer to use close_click() rather than click() because close_click() waits a little longer to give windows time to close the dialog.)

  8. Let’s type some text into the Notepad text area. Without the with_spaces argument spaces would not be typed. Please see documentation for SendKeys for this method as it is a thin wrapper around SendKeys.

  9. Ask to exit Notepad

  10. We will be asked if we want to save - click on the “No” button.