An overview of how UI logging works in LibreOffice

Logging in LibreOffice is mainly handled by a class called UITestLogger, defined here. The logger class logs the user actions only if the member flag mbValid is set to true. The flag can be turned on by setting the environment variable LO_COLLECT_UIINFO to a file name where the logs should be collected (see the constructor of UITestLogger, defined here).

We maintain a pointer to an instance of UITestLogger class here. To use the logger object, the static function getInstance can be used to get access to the pointer.
The function logAction, defined in the same class, is used to log events from the classes which extend the class vcl::Control. The log statements corresponding to a particular class can be found in the function get_action of the UI wrapper classes. Most of the wrapper classes can be found with this OpenGrok search. The log statements get generated when VCL events get broadcasted. A list of the VCL events can be found in the enum class VclEventId here.
For other classes, w…

GSoC final report - UI logger

Overview The idea of this GSoC project was to implement a feature which logs the interaction of users with the UI elements in an interpretable Domain Specific Language (DSL), and an interpreter which converts the log into a file into something which will help in replaying the user actions. The project involved working with the UI elements and the UI testing framework of LibreOffice.
AchievementsThe logger -

The logging for the following is currently supported -The UI elements of Visual Component Library like Buttons, Checkboxes, Radio-buttons, Listboxes, Comboboxes, Spinbuttons, etc.Keys pressed in different UI elementsUNO commands sent with parametersSelection of ObjectsOpening of Sidebar/Deck
Some application-specific events can also be logged -Writer - Goto Page, Set Zoom, Text SelectionCalc - Autofilter Launch, Selection of cells/range of cells, Switching of SheetMath - Selection of elements from the element selector panelImpress - Set Zoom
Additionally, the opening and closing of som…

GSoC work report - Week 11

This week, I first completed the patch -
This adds a new function to call UNO commands with parameters in the UI tests.

Then, I completed the patch -
This involved adding logging for the opening of sidebar/deck in the LO apps, changing the log for object selection and UNO commands to make them interpretable, and removing the function logObjectSelection to merge it in the logEvent function.

Now, I am working on improving the log interpreter -
This fixes a crash on certain inputs and adds support for interpreting UNO command parameters, along with some other minor changes.

GSoC work report - Week 10

I first worked on adding logs for more events. The selection of symbols from "element_selector" in the math app now gets logged. Also, the selection of objects now gets logged in the writer, calc and impress. The patch is here -
I also worked on changing from multiple functions for logging to one generic function logEvent in the same patch.

Additionally, I also did some work for adding support of sending parameters in executeCommand function of the UI testing framework. The progress can be seen here -

GSoC work report - Week 9

At first, I worked on the keylogging issue. Key presses are now logged according to DSL inside the dialogs too (in ComboBoxes and SpinFields). The patch is here -

Then I worked on logging more events ( These are the events which this patch logs -
WriterGoto PageSet ZoomCalcLaunch AutofilterSelect Cell/Range/TableImpressSet Zoom In addition to this, I realized that we might also need to log the arguments sent with a UNO command, in addition to the UNO command itself. The patch has also done some work in that direction.

GSoC work report - Week 8

This week, I first worked on improving the log interpreter. It can now open existing files (in addition to starting with blank documents).

I then moved the logAction function from ImplCallEventListenersAndHandler to CallEventListeners. Events from down in the class hierarchy which directly called CallEventListeners from vcl::Window (or ImplCallEventListenersAndHandler from Control) will now get interrupted by the new function, register a log statement, and then go to the class Window. This also allowed to log SpinFieldUp and SpinFieldDown.

According to the feedback from Raal, I worked on resolving the ID conflicts of UI elements in the same dialog. The patch can be seen here.

For week 9, I will keep working on improving UI testing, including the key input inside dialogs as suggested by Markus.

GSoC work report - Week 7

During the 7th week, I sent 3 patches.

1) Gerrit change #56776

This change adds more features to the log interpreter. UNO commands, closing dialogs through buttons, sending UNO commands to open dialogs via keycode is now supported.

2) Gerrit change #56777

This change makes command-line argument parsing of interpreter more standard.

3) Gerrit change #56789

This change adds support for starting any of the 6 apps of LibreOffice through the create_doc_in_start_center() function.

Let's also have a look at a log file and the generated test file. 

Log File -

ButtonUIObject Action:CLICK Id:writer_all Parent: CommandSent ModalDialogExecuted Id:CharacterPropertiesDialog RadioButtonUIObject Action:CLICK Id:90deg Parent:CharacterPropertiesDialog ListBoxUIObject Action:SELECT Id:linestylelb Parent:CharacterPropertiesDialog {"POS": "6"} ButtonUIObject Action:CLICK Id:ok Parent:CharacterPropertiesDialog DialogClosed

Test File -

1 2 3 4 5 6 7 8 9 10 11 12 1…