fortrace.utility.applications package

Subpackages

• fortrace.utility.applications.console package
  • Submodules
  • fortrace.utility.applications.console.linux_terminal module
    • GenericLinuxTerminal
      • GenericLinuxTerminal.change_shell()
      • GenericLinuxTerminal.send_command()
      • GenericLinuxTerminal.toggle_command_history()
  • fortrace.utility.applications.console.powershell module
    • PowerShell
      • PowerShell.close()
      • PowerShell.send_command()
  • fortrace.utility.applications.console.terminal module
    • GenericTerminal
      • GenericTerminal.send_command()
      • GenericTerminal.toggle_command_history()
  • fortrace.utility.applications.console.terminal_factory module
    • get_terminal()
  • Module contents
• fortrace.utility.applications.file_manager package
  • Submodules
  • fortrace.utility.applications.file_manager.file_manager module
    • GenericFileManager
      • GenericFileManager.bookmark_current_location()
      • GenericFileManager.browse_to_directory()
      • GenericFileManager.copy_file()
      • GenericFileManager.create_folder()
      • GenericFileManager.delete_file()
      • GenericFileManager.empty_trash()
      • GenericFileManager.focus_on_item()
      • GenericFileManager.move_file()
      • GenericFileManager.open_file()
      • GenericFileManager.rename_file()
      • GenericFileManager.search()
      • GenericFileManager.show_hidden
      • GenericFileManager.toggle_show_hidden()
  • fortrace.utility.applications.file_manager.file_manager_factory module
    • get_file_browser()
  • fortrace.utility.applications.file_manager.files module
    • Files
      • Files.bookmark_current_location()
      • Files.create_folder()
      • Files.empty_trash()
      • Files.focus_on_item()
      • Files.search()
      • Files.toggle_show_hidden()
  • fortrace.utility.applications.file_manager.windows_explorer module
    • Explorer
      • Explorer.bookmark_current_location()
      • Explorer.browse_to_directory()
      • Explorer.empty_trash()
      • Explorer.focus_on_item()
      • Explorer.move_file()
      • Explorer.open_file()
      • Explorer.search()
      • Explorer.toggle_show_hidden()
  • Module contents
• fortrace.utility.applications.mail_client package
  • Submodules
  • fortrace.utility.applications.mail_client.email_message module
  • fortrace.utility.applications.mail_client.mail_client module
  • fortrace.utility.applications.mail_client.thunderbird module
  • Module contents
• fortrace.utility.applications.text_editor package
  • Submodules
  • fortrace.utility.applications.text_editor.gnome_text_editor module
    • GEdit
      • GEdit.close_file()
      • GEdit.open_file()
  • fortrace.utility.applications.text_editor.notepad module
    • Notepad
      • Notepad.open_file()
      • Notepad.save_as()
  • fortrace.utility.applications.text_editor.text_editor module
    • GenericTextEditor
      • GenericTextEditor.open_file()
      • GenericTextEditor.save()
      • GenericTextEditor.save_as()
      • GenericTextEditor.send_file()
  • fortrace.utility.applications.text_editor.text_editor_factory module
    • get_text_editor()
  • Module contents
• fortrace.utility.applications.web_browsers package
  • Submodules
  • fortrace.utility.applications.web_browsers.edge module
    • Edge
      • Edge.copilot()
      • Edge.save()
  • fortrace.utility.applications.web_browsers.firefox module
    • Firefox
  • fortrace.utility.applications.web_browsers.web_browser module
    • GenericWebBrowser
      • GenericWebBrowser.active_tab
      • GenericWebBrowser.browse_to_url()
      • GenericWebBrowser.close_tab()
      • GenericWebBrowser.focus_tab()
      • GenericWebBrowser.go_back()
      • GenericWebBrowser.go_forward()
      • GenericWebBrowser.remove_from_favorite()
      • GenericWebBrowser.save()
      • GenericWebBrowser.save_to_favorites()
      • GenericWebBrowser.start_web_search()
  • fortrace.utility.applications.web_browsers.web_browser_factory module
    • get_web_browser()
  • Module contents

Submodules

fortrace.utility.applications.application module

class fortrace.utility.applications.application.ApplicationEvent(*values)

Bases: Enum

Application events that are processed by the desktop environment.

APPLICATION_POPUP_CLOSED = 7

APPLICATION_POPUP_OPENED = 6

APPLICATION_RESIZED = 5

CLOSED = 1

FOCUS_APPLICATION = 4

FOCUS_SHIFTED = 3

NEW_APPLICATION_OPENED = 2

class fortrace.utility.applications.application.ApplicationType(*values)

Bases: Enum

Supported application types, used to select the correct factory method

FILE_MANAGER = 1

MAIL_CLIENT = 5

OTHER = 6

TERMINAL = 3

TEXT_EDITOR = 4

WEB_BROWSER = 2

class fortrace.utility.applications.application.FileDialogue(parent_application: GenericApplication)

Bases: GenericPopup

save_to_directory(destination: Path | None = None, name: str | None = None)

class fortrace.utility.applications.application.GenericApplication(name: str, application_type: ApplicationType, qemu_monitor_session: QEMUMonitorSession, parent_notifier: ParentNotifier)

Bases: object

Representation of a generic graphic application window.

This class resembles the general interface of all graphical applications with which ForTrace might interact. It can be used to control applications with keyboard shortcuts and simple commands, without writing any application specific Python classes.

close()

Close an application with ‘alt-f4’ key combination.

coordinates: tuple[int, int, int, int] | None

extract_text() → tuple[list[tuple[int, int, int, int]], list[str]]

Extract text from application window.

Extracts the text visible in the application window. Uses OCR, thus the result is not 100% accurate, thus applying string similarity search is highly recommended.

Returns:

tuple of list with bounding boxes and the detected strings within

focus_element(text: str)

Focus the element with the specified text.

This method acts as a stub for the integration of a screen parsing tool into ForTrace++. It ought to be overwritten on instance level, like it is done in the OmniParser example.

Parameters:

text – text to be searched for

property focused

Signals whether this application receives mouse and keyboard inputs.

property name: str

Name of this application (not unique if multiple instances are opened).

open_popup(screenshot_before_popup: bytes | PathLike | ndarray, popup_type: PopupType = PopupType.GENERIC, popup_min_size: int = 50000) → GenericPopup

React on the opening of a new popup window within the application window.

Parameters:

• screenshot_before_popup – a screenshot taken of the application before the popup is opened

• popup_type – specify the kind of popup to return

• popup_min_size – the size of the popup in px

Returns:

Instance of opened popup window to interact with

perform_complex_action(func: Callable[[GenericApplication], Any]) → Any

Performs a complex action in an application.

Parameters:

func – Callable representing the action to perform

Returns:

returns whatever func returns

send_key_combination(key_combination: str, times: int = 1)

Send a direct key combination to the application.

Parameters:

• key_combination – key combination to be sent (using qemu key_codes)

• times – number of times to send key combination

send_text(text: str, end_ret: bool = False)

Send text to the application.

Parameters:

• text – text to be sent

• end_ret – should return be pressed after the text is entered?

Note

The text is allowed to contain ‘n’ and ‘r’ characters, which are respected

property size

Compute the size of the application window.

Returns:

The size of the application window in pixels. 0, if coordinates are unknown.

take_screenshot() → ndarray

Take screenshot of the application window.

Returns:

OpenCV image of the application window with all color channels

property uuid: UUID

UUID of the application (unique, even for multiple instances).

class fortrace.utility.applications.application.GenericPopup(parent_application: GenericApplication)

Bases: GenericApplication

Representation of a popup window opened by another application.

Should not be used on its own, but created through GenericApplication’s open_popup method.

close()

Notifies the desktop env about the closing of a popup window.

Note

Popups are closed indirectly by pressing a certain key and NOT alt-f4

class fortrace.utility.applications.application.ParentNotifier(*args, **kwargs)

Bases: Protocol

Used to provide type hints and enable the IDE for static type checking.

It refers to DesktopEnvironment._on_change method.

class fortrace.utility.applications.application.PopupType(*values)

Bases: Enum

FILE_DIALOGUE = 2

GENERIC = 1

WARNING = 3

fortrace.utility.applications.application_factory module

fortrace.utility.applications.application_factory.get_application(application_type: ApplicationType, name: str, qs: QEMUMonitorSession, parent_notifier: ParentNotifier, **kwargs) → GenericApplication

Factory to create an application object by calling the correct sub-factory.

Parameters:

• application_type – ApplicationType to filter for correct sub-factory

• name – name of the application (if not included ValueError is thrown)

• qs – handle to QEMUMonitor

• parent_notifier – attribute of desktop environment to handle changes in windows

• **kwargs – relevant to certain applications that might be started with extra arguments, e.g., Terminals

Returns:

Handle to opened application

Module contents