Automatic Bug Reporting Tool

ABRT is a set of small utilities developed with three main intentions:

1. simplify the reporting of software problems for the end users
2. provide a comprehensive information about a problem in the reports for the developers
3. provide valuable crash statistics for prioritization and self-support

If you are interested in helping developers to prioritize their work and you want provide anonymous report please turn on the auto reporting of micro-reports using

abrt-auto-reporting enabled

Read the micro-report online documentation and abrt-auto-reporting(1) for details.

1. ABRT workflow

1. hooks own crash handlers in the most used programming environments
   • registers a core dump program for crashes of C/C++ applications
   • registers a handler for uncaught Python exceptions
   • registers a handler for uncaught Ruby exceptions
   • provides a JNI agent detecting uncaught Java exceptions

2. searches for "suspicious" strings in the system logs

   • detects Kernel oopses
   • detects Machine-check exceptions
   • detects Xorg crashes in Xorg's log
3. checks existence of files describing problems
   • detects and examines Kernel dump files made by kdump
4. provides CLI and GUI tooling (abrt-cli and gnome-abrt) to send and view already sent reports
5. gathers data on ABRT server crash statistics server. The main purpose of the ABRT server is to help the maintainers to prioritize their work and provide users guidance for known or already fixed issues.

2. ABRT architecture

Upon a detection of a problem, ABRT creates a directory in a well known path and puts all the gathered data into that directory in form of regular files.

The well known path where the problem directories are created is:

• /var/spool/abrt (CentOS 6)

• /var/tmp/abrt (CentOS 7)

And the most interesting files within a problem directory follows:

In case you want to examine the problem directory manually or you just want to know what the problem is about, ABRT provides you with several tiny utilities doing useful stuff. All the utilities have a name prefixed with "abrt-action", thus you can change the current working directory to a problem directory, type abrt-action and hit Tab key.

3. An example of getting backtrace from 'coredump' file:

ABRT does not extract backtrace from a core dump file automatically, because this task requires the debug info data to be available.

You can install the debug info data manually via the debuginfo-install command suggested by gdb:

$ gdb $(cat executable) -c coredump 

or you can let ABRT to do all the necessary work instead of you:

$ abrt-action-analyze-ccpp-local

abrt-action-analyze-ccpp-local enables all *debug* yum repositories, downloads the missing debug info packages, unpacks them in /var/cache/abrt-di/, runs gdb and stores its output in backtrace file.

---

• However, ABRT allows users to work with the problems in a more comfortable way through its command line (abrt-cli) and graphical (gnome-abrt) interfaces.

4. Notifications

ABRT provides several configurable ways of notifying users about a crash on the OS. The default notification is desktop notification, email sent to root user (using default MTA, settings can be modified in /etc/libreport/events.d/mailx_event.conf) and console notification such as

ABRT has detected 1 problem(s). For more info run: abrt-cli list --since 1414407227

5. Installation

Run the following command on headless systems:

$ yum install abrt-cli

or the following for desktops

$ yum install abrt-desktop

abrt-cli and abrt-desktop are meta-packages pulling all the most useful packages into your system.

6. Configuration

The configuration files are stored in:

• /etc/abrt

• /etc/libreport

• $HOME/.config/abrt

The configuration option values that you provide to the ABRT GUI are stored in GNOME Keyring.

7. Reporting to CentOS Bug Tracker

Starting with CentOS 7.1.1503, ABRT can report the crashes to bugs.centos.org. To be able to file a bug report, you have to create your own account at bugs.centos.org and provide your account details to ABRT when it prompts you for them before submitting the bug report.

This functionality also requires backtrace generated from the coredump. ABRT is capable of generating the backtrace either locally, which involves downloading debuginfo packages to /var/cache/abrt-di, or remotely to https://fedorahosted.org/abrt/wiki/AbrtRetraceServer, which involves uploading the coredump file to the Internet.

ABRT provides you with these two options to generate the backtrace because the coredump file might contain security sensitive information and the local backtrace generation might require downloading of GiBs of debug info packages. However, the retrace server has been developed with focus on security and its benefit for the users is highly appreciated.

Because the bug reports filed by ABRT are public and the crash data often contains security sensitive information such as passwords, the reporting process includes two steps of reviewing the published data. In the first step ABRT highlights "suspicious" strings and you can also search for your custom strings. In the second step you can exclude files from the published data set.

8. Automatic crash statistics submission

Besides the detection of problems and the creation of comprehensive bug reports, ABRT provides a functionality to automatically submit a short, anonymous description of a crash which is called micro-Report (uReport) at the time of crash detection.

Micro-reports are anonymous, they don’t carry any information about IP address, host-id, hostname, or even memory and filesystem snapshots. The content of the micro-report is stripped to bare minimum to allow finding duplicates on the server side.

The ABRT server collects micro-Reports to provide developers with the crash statistics useful for prioritization of their work.

This functionality of ABRT can be enabled by issuing the following command:

$ abrt-auto-reporting enabled 

In the default configuration, ABRT sends the crash statistics data to the ABRT server

9. References

For more details about how ABRT works follow this link: http://abrt.readthedocs.org/en/latest/

If something does not work for you, you may find an answer at: http://abrt.readthedocs.org/en/latest/faq.html

If you are interested in more advanced ABRT capabilities, checkout the examples page at: http://abrt.readthedocs.org/en/latest/examples.html

The ABRT project lives on github.com: https://github.com/abrt